2024-09-19T07:54:34.551 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T07:54:34.579 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T07:54:34.682 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911842 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911842' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLLtp+vkN5rRCfavooeJSJA7P41xf9R2J+EfL0bxUrpexCML3srWcljTLpeoq2pvGCoQhsePi2I1Lw/HzCsvAT0= smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM02q1LAQY//eQst5gTeVekpOkKyasrKJCj+b2ZC8xTqNKfmC3+2vxn5OBJcJ9iqzQgZnqOAbNhReGj4Jik3Cg4= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMRl+EVAxlpnfL3jZ3vWDwF2PEE2nmhU35gwu/VMmZb2BtUbOhbojy3bkt06Xe0LC2mz3WF/+dMccIUMziRL9WU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T07:54:34.683 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T07:54:34.684 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T07:54:34.684 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T07:54:34.685 INFO:teuthology.task.internal:Checking packages... 2024-09-19T07:54:34.712 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T07:54:34.712 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T07:54:34.712 INFO:teuthology.packaging:ref: None 2024-09-19T07:54:34.712 INFO:teuthology.packaging:tag: None 2024-09-19T07:54:34.712 INFO:teuthology.packaging:branch: squid 2024-09-19T07:54:34.712 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:54:34.712 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T07:54:34.957 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T07:54:34.959 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T07:54:34.964 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T07:54:34.964 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T07:54:34.996 INFO:teuthology.task.internal:Saving configuration 2024-09-19T07:54:35.024 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T07:54:35.060 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T07:54:35.085 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911842', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 07:48:57.142435', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLLtp+vkN5rRCfavooeJSJA7P41xf9R2J+EfL0bxUrpexCML3srWcljTLpeoq2pvGCoQhsePi2I1Lw/HzCsvAT0='} 2024-09-19T07:54:35.105 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911842', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 07:48:57.144513', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM02q1LAQY//eQst5gTeVekpOkKyasrKJCj+b2ZC8xTqNKfmC3+2vxn5OBJcJ9iqzQgZnqOAbNhReGj4Jik3Cg4='} 2024-09-19T07:54:35.128 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911842', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 07:48:57.145702', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMRl+EVAxlpnfL3jZ3vWDwF2PEE2nmhU35gwu/VMmZb2BtUbOhbojy3bkt06Xe0LC2mz3WF/+dMccIUMziRL9WU='} 2024-09-19T07:54:35.129 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T07:54:35.135 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T07:54:35.135 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T07:54:35.135 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T07:54:35.135 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T07:54:35.255 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3746457be0>, signals=[15]) 2024-09-19T07:54:35.255 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T07:54:35.260 INFO:teuthology.task.internal:Opening connections... 2024-09-19T07:54:35.260 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2024-09-19T07:54:35.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:54:35.357 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-09-19T07:54:35.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:54:35.435 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-09-19T07:54:35.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:54:35.517 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T07:54:35.524 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2024-09-19T07:54:35.543 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2024-09-19T07:54:35.543 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2024-09-19T07:54:35.600 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2024-09-19T07:54:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9" 2024-09-19T07:54:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2024-09-19T07:54:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2024-09-19T07:54:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9" 2024-09-19T07:54:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9" 2024-09-19T07:54:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T07:54:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2024-09-19T07:54:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon" 2024-09-19T07:54:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T07:54:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2024-09-19T07:54:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T07:54:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T07:54:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T07:54:35.602 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2024-09-19T07:54:35.640 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-09-19T07:54:35.658 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-09-19T07:54:35.659 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-09-19T07:54:35.716 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-09-19T07:54:35.716 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9" 2024-09-19T07:54:35.716 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-09-19T07:54:35.716 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-09-19T07:54:35.716 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9" 2024-09-19T07:54:35.716 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9" 2024-09-19T07:54:35.717 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T07:54:35.717 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-09-19T07:54:35.717 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon" 2024-09-19T07:54:35.717 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T07:54:35.717 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-09-19T07:54:35.717 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T07:54:35.717 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T07:54:35.717 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T07:54:35.717 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-09-19T07:54:35.749 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-09-19T07:54:35.769 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-09-19T07:54:35.769 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-09-19T07:54:35.826 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2024-09-19T07:54:35.826 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2024-09-19T07:54:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2024-09-19T07:54:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2024-09-19T07:54:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2024-09-19T07:54:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2024-09-19T07:54:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T07:54:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-09-19T07:54:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2024-09-19T07:54:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T07:54:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2024-09-19T07:54:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T07:54:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T07:54:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T07:54:35.828 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-09-19T07:54:35.864 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T07:54:35.934 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T07:54:35.948 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T07:54:35.948 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2024-09-19T07:54:35.951 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-09-19T07:54:35.953 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-09-19T07:54:35.970 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T07:54:35.976 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T07:54:35.976 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2024-09-19T07:54:36.007 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-09-19T07:54:36.010 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-09-19T07:54:36.089 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T07:54:36.152 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T07:54:36.167 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T07:54:36.167 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-19T07:54:36.167 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-19T07:54:36.168 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T07:54:36.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:54:36.185 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-09-19T07:54:36.244 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-503.el9.x86_64 2024-09-19T07:54:36.245 INFO:teuthology.task.kernel:Running kernel on smithi022: 5.14.0-503.el9.x86_64 2024-09-19T07:54:36.245 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum install -y kernel 2024-09-19T07:54:38.826 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 9.0 MB/s | 8.3 MB 00:00 2024-09-19T07:54:40.818 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-19T07:54:46.011 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.5 MB 00:01 2024-09-19T07:54:47.561 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-19T07:54:48.784 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-19T07:54:54.363 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-19T07:54:56.303 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T07:54:56.303 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T07:54:56.407 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T07:54:56.419 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T07:54:56.419 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-09-19T07:54:56.419 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T07:54:56.419 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T07:54:56.419 INFO:teuthology.orchestra.run.smithi022.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T07:54:56.419 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-19T07:54:56.419 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T07:54:56.420 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T07:54:56.420 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T07:54:56.420 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T07:54:56.420 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T07:54:56.420 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T07:54:56.420 INFO:teuthology.orchestra.run.smithi022.stdout:Install 4 Packages 2024-09-19T07:54:56.421 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T07:54:56.421 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 79 M 2024-09-19T07:54:56.421 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 126 M 2024-09-19T07:54:56.421 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T07:54:56.556 INFO:teuthology.orchestra.run.smithi022.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 408 kB/s | 36 kB 00:00 2024-09-19T07:54:57.048 INFO:teuthology.orchestra.run.smithi022.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-19T07:54:57.425 INFO:teuthology.orchestra.run.smithi022.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 36 MB/s | 35 MB 00:00 2024-09-19T07:54:57.765 INFO:teuthology.orchestra.run.smithi022.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-19T07:54:57.766 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:54:57.766 INFO:teuthology.orchestra.run.smithi022.stdout:Total 59 MB/s | 79 MB 00:01 2024-09-19T07:54:58.479 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T07:54:58.858 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T07:54:58.858 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T07:54:59.856 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T07:54:59.857 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T07:55:02.406 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T07:55:03.069 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T07:55:03.157 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T07:55:05.917 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T07:55:06.062 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T07:55:11.185 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T07:55:11.249 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:55:16.385 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:55:55.131 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:55:55.172 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:55:55.530 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:55:55.530 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T07:55:55.530 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T07:55:55.530 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T07:55:56.785 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T07:55:56.874 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T07:55:56.874 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T07:55:56.875 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T07:55:56.875 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T07:55:56.875 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T07:55:56.875 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-19T07:55:56.875 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T07:55:56.875 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T07:55:56.934 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T07:55:56.934 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T07:55:56.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo rpm -qi grub2-tools 2024-09-19T07:55:56.971 INFO:teuthology.orchestra.run.smithi022.stdout:Name : grub2-tools 2024-09-19T07:55:56.971 INFO:teuthology.orchestra.run.smithi022.stdout:Epoch : 1 2024-09-19T07:55:56.972 INFO:teuthology.orchestra.run.smithi022.stdout:Version : 2.06 2024-09-19T07:55:56.972 INFO:teuthology.orchestra.run.smithi022.stdout:Release : 82.el9 2024-09-19T07:55:56.972 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: x86_64 2024-09-19T07:55:56.972 INFO:teuthology.orchestra.run.smithi022.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T07:55:56.972 INFO:teuthology.orchestra.run.smithi022.stdout:Group : Unspecified 2024-09-19T07:55:56.972 INFO:teuthology.orchestra.run.smithi022.stdout:Size : 8274670 2024-09-19T07:55:56.972 INFO:teuthology.orchestra.run.smithi022.stdout:License : GPLv3+ 2024-09-19T07:55:56.972 INFO:teuthology.orchestra.run.smithi022.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T07:55:56.972 INFO:teuthology.orchestra.run.smithi022.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T07:55:56.972 INFO:teuthology.orchestra.run.smithi022.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T07:55:56.972 INFO:teuthology.orchestra.run.smithi022.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T07:55:56.972 INFO:teuthology.orchestra.run.smithi022.stdout:Packager : builder@centos.org 2024-09-19T07:55:56.973 INFO:teuthology.orchestra.run.smithi022.stdout:Vendor : CentOS 2024-09-19T07:55:56.973 INFO:teuthology.orchestra.run.smithi022.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T07:55:56.973 INFO:teuthology.orchestra.run.smithi022.stdout:Summary : Support tools for GRUB. 2024-09-19T07:55:56.973 INFO:teuthology.orchestra.run.smithi022.stdout:Description : 2024-09-19T07:55:56.973 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T07:55:56.973 INFO:teuthology.orchestra.run.smithi022.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T07:55:56.973 INFO:teuthology.orchestra.run.smithi022.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T07:55:56.973 INFO:teuthology.orchestra.run.smithi022.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T07:55:56.973 INFO:teuthology.orchestra.run.smithi022.stdout:hardware devices. 2024-09-19T07:55:56.973 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T07:55:56.973 INFO:teuthology.orchestra.run.smithi022.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T07:55:56.974 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T07:55:56.974 INFO:teuthology.task.kernel:Updating grub on smithi022 to boot 5.14.0-508.el9.x86_64 2024-09-19T07:55:56.974 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T07:55:57.640 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2024-09-19T07:55:58.922 INFO:teuthology.orchestra.run.smithi022.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T07:55:58.962 INFO:teuthology.orchestra.run.smithi022.stderr:done 2024-09-19T07:55:58.964 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-09-19T07:55:58.978 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.lnEtf3SrcW 2024-09-19T07:55:58.979 DEBUG:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lnEtf3SrcW 2024-09-19T07:55:59.059 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.lnEtf3SrcW 2024-09-19T07:55:59.191 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.lnEtf3SrcW is 6KB 2024-09-19T07:55:59.241 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.lnEtf3SrcW 2024-09-19T07:55:59.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T07:55:59.324 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T07:55:59.324 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T07:55:59.324 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T07:55:59.324 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T07:55:59.325 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T07:55:59.469 DEBUG:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2024-09-19T07:56:29.472 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T07:56:29.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-19T07:56:29.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:56:47.809 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.22 2024-09-19T07:56:56.810 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-19T07:56:56.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:57:56.814 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T07:58:08.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-19T07:58:08.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:58:09.203 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-09-19T07:58:09.567 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-19T07:58:09.567 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-19T07:58:09.568 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-09-19T07:58:09.585 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-508.el9.x86_64 2024-09-19T07:58:09.585 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T07:58:09.585 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T07:58:09.585 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T07:58:10.587 DEBUG:teuthology.parallel:result is None 2024-09-19T07:58:10.587 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T07:58:10.594 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T07:58:10.595 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T07:58:10.597 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T07:58:10.600 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T07:58:10.618 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T07:58:10.625 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T07:58:10.665 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T07:58:10.666 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T07:58:10.669 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T07:58:10.672 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T07:58:10.777 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T07:58:10.783 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T07:58:10.784 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T07:58:10.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:58:10.801 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T07:58:10.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:58:10.820 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T07:58:10.822 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T07:58:10.844 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T07:58:10.879 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:58:10.902 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:58:10.915 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:58:10.917 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:58:10.937 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:58:10.938 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:58:10.939 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T07:58:10.946 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T07:58:10.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T07:58:10.948 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T07:58:10.950 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T07:58:10.978 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T07:58:10.987 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T07:58:10.988 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T07:58:11.021 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T07:58:11.023 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T07:58:11.040 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:58:11.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:58:11.173 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:58:11.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:58:11.299 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T07:58:11.299 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T07:58:11.367 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:58:11.412 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:58:11.462 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:58:11.544 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:58:11.592 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T07:58:11.592 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T07:58:11.660 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:58:11.707 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:58:11.763 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:58:11.847 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:58:11.891 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T07:58:11.891 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T07:58:11.958 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2024-09-19T07:58:11.961 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-09-19T07:58:11.963 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-09-19T07:58:12.005 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:58:12.011 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:58:12.039 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:58:12.543 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T07:58:12.551 INFO:teuthology.task.internal:Starting timer... 2024-09-19T07:58:12.551 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T07:58:12.593 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T07:58:12.623 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T07:58:12.623 DEBUG:teuthology.orchestra.run.smithi022:> sudo service auditd rotate 2024-09-19T07:58:12.681 INFO:teuthology.orchestra.run.smithi022.stdout:Rotating logs: 2024-09-19T07:58:12.683 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-09-19T07:58:12.724 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-09-19T07:58:12.726 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-09-19T07:58:12.780 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-09-19T07:58:12.783 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T07:58:12.783 DEBUG:teuthology.orchestra.run.smithi022:> /usr/sbin/getenforce 2024-09-19T07:58:12.820 INFO:teuthology.orchestra.run.smithi022.stdout:Permissive 2024-09-19T07:58:12.820 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-09-19T07:58:12.844 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-09-19T07:58:12.845 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-09-19T07:58:12.870 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-09-19T07:58:12.871 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi022.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-09-19T07:58:12.871 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:58:12.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:58:12.901 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:58:12.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:58:12.930 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:58:12.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:58:12.959 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T07:58:12.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo /usr/sbin/setenforce permissive 2024-09-19T07:58:12.991 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-09-19T07:58:13.026 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-09-19T07:58:13.067 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T07:58:13.081 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T07:58:13.094 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T07:58:13.095 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi022.front.sepia.ceph.com,smithi064.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T08:02:29.205 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-09-19T08:02:29.206 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-19T08:02:29.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:02:29.286 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-09-19T08:02:29.369 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-19T08:02:29.369 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-19T08:02:29.370 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:02:29.443 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-09-19T08:02:29.521 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-19T08:02:29.521 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-19T08:02:29.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:02:29.595 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-09-19T08:02:29.682 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-19T08:02:29.683 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T08:02:29.693 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T08:02:29.693 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T08:02:29.694 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:02:29.696 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T08:02:29.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:02:29.699 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T08:02:29.699 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:02:29.727 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T08:02:29.728 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T08:02:29.743 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T08:02:29.743 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T08:02:29.756 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T08:02:29.768 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-09-19T08:02:29.770 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: ntpd: command not found 2024-09-19T08:02:29.772 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T08:02:29.780 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-09-19T08:02:29.782 INFO:teuthology.orchestra.run.smithi022.stdout:506 Cannot talk to daemon 2024-09-19T08:02:29.795 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T08:02:29.797 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-09-19T08:02:29.798 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T08:02:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T08:02:29.809 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-09-19T08:02:29.813 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T08:02:29.823 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T08:02:29.838 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T08:02:29.863 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:02:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:02:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-09-19T08:02:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:02:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:02:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:02:29.867 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:02:29.868 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:02:29.870 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:02:29.871 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-09-19T08:02:29.871 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:02:29.871 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:02:29.871 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:02:29.871 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:02:29.895 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:02:29.899 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:02:29.899 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-09-19T08:02:29.899 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:02:29.899 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:02:29.899 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:02:29.899 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:02:29.899 INFO:teuthology.run_tasks:Running task install... 2024-09-19T08:02:29.907 DEBUG:teuthology.task.install:project ceph 2024-09-19T08:02:29.907 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T08:02:29.907 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T08:02:29.907 INFO:teuthology.task.install:Using flavor: default 2024-09-19T08:02:29.913 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T08:02:29.913 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T08:02:29.914 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T08:02:29.914 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:02:29.916 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T08:02:29.917 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:02:29.917 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T08:02:29.918 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:02:30.072 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T08:02:30.072 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T08:02:30.077 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T08:02:30.077 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T08:02:30.082 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T08:02:30.083 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T08:02:30.182 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T08:02:30.182 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T08:02:30.182 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T08:02:30.189 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T08:02:30.189 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T08:02:30.189 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T08:02:30.197 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T08:02:30.198 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:02:30.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T08:02:30.209 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T08:02:30.210 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T08:02:30.215 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T08:02:30.215 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T08:02:30.225 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T08:02:30.226 DEBUG:teuthology.orchestra.run.smithi022:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T08:02:30.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T08:02:30.281 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T08:02:30.294 DEBUG:teuthology.orchestra.run.smithi022:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T08:02:30.349 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T08:02:30.355 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T08:02:30.371 DEBUG:teuthology.orchestra.run.smithi022:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T08:02:30.413 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-09-19T08:02:30.415 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-09-19T08:02:30.420 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-09-19T08:02:30.422 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-09-19T08:02:30.436 INFO:teuthology.orchestra.run.smithi022.stdout:check_obsoletes = 1 2024-09-19T08:02:30.438 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum clean all 2024-09-19T08:02:30.646 INFO:teuthology.orchestra.run.smithi064.stdout:45 files removed 2024-09-19T08:02:30.647 INFO:teuthology.orchestra.run.smithi157.stdout:45 files removed 2024-09-19T08:02:30.674 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T08:02:30.675 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T08:02:30.683 INFO:teuthology.orchestra.run.smithi022.stdout:45 files removed 2024-09-19T08:02:30.711 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T08:02:31.223 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 317 kB/s | 84 kB 00:00 2024-09-19T08:02:31.236 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 293 kB/s | 84 kB 00:00 2024-09-19T08:02:31.276 INFO:teuthology.orchestra.run.smithi022.stdout:ceph packages for x86_64 315 kB/s | 84 kB 00:00 2024-09-19T08:02:31.507 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T08:02:31.521 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-19T08:02:31.559 INFO:teuthology.orchestra.run.smithi022.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T08:02:31.782 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-19T08:02:31.804 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-19T08:02:31.835 INFO:teuthology.orchestra.run.smithi022.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-19T08:02:32.327 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.3 MB 00:00 2024-09-19T08:02:32.783 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 8.3 MB 00:00 2024-09-19T08:02:32.821 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 8.4 MB/s | 8.3 MB 00:00 2024-09-19T08:02:33.988 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 37 MB/s | 20 MB 00:00 2024-09-19T08:02:34.676 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-19T08:02:35.331 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 14 MB/s | 20 MB 00:01 2024-09-19T08:02:39.277 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.5 MB 00:00 2024-09-19T08:02:39.748 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-19T08:02:40.272 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.5 MB 00:02 2024-09-19T08:02:40.752 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 85 kB/s | 19 kB 00:00 2024-09-19T08:02:41.182 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 19 kB 00:00 2024-09-19T08:02:41.823 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 19 kB 00:00 2024-09-19T08:02:42.165 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-19T08:02:42.236 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-19T08:02:42.752 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-19T08:02:47.692 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-09-19T08:02:47.704 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-19T08:02:48.433 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-19T08:02:49.854 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:02:49.855 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:02:49.868 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T08:02:49.869 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T08:02:49.871 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T08:02:49.871 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T08:02:49.873 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T08:02:49.876 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T08:02:49.877 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T08:02:49.941 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-19T08:02:49.955 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-09-19T08:02:49.955 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-09-19T08:02:49.956 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-09-19T08:02:49.956 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-19T08:02:49.956 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T08:02:49.956 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T08:02:49.956 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T08:02:49.956 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T08:02:49.956 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T08:02:49.956 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T08:02:49.956 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T08:02:49.956 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T08:02:49.956 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T08:02:49.957 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T08:02:49.957 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T08:02:49.957 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T08:02:49.957 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T08:02:49.957 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T08:02:49.957 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T08:02:49.957 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T08:02:49.958 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T08:02:49.958 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T08:02:49.958 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T08:02:49.958 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T08:02:49.958 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T08:02:49.958 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T08:02:49.958 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T08:02:49.958 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T08:02:49.958 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T08:02:49.958 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T08:02:49.958 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T08:02:49.959 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T08:02:49.959 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T08:02:49.959 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T08:02:49.959 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T08:02:49.959 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T08:02:49.959 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T08:02:49.959 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T08:02:49.959 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T08:02:49.959 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T08:02:49.959 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T08:02:49.959 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-09-19T08:02:49.959 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T08:02:49.960 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T08:02:49.960 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T08:02:49.960 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T08:02:49.960 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T08:02:49.960 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T08:02:49.960 INFO:teuthology.orchestra.run.smithi157.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T08:02:49.960 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T08:02:49.960 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-19T08:02:49.960 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T08:02:49.960 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T08:02:49.960 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T08:02:49.960 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T08:02:49.960 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T08:02:49.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T08:02:49.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T08:02:49.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T08:02:49.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T08:02:49.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T08:02:49.961 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T08:02:49.961 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T08:02:49.961 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T08:02:49.961 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T08:02:49.961 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T08:02:49.961 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T08:02:49.961 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T08:02:49.962 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T08:02:49.962 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T08:02:49.962 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T08:02:49.962 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T08:02:49.962 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T08:02:49.962 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T08:02:49.962 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T08:02:49.962 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T08:02:49.962 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T08:02:49.962 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T08:02:49.962 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T08:02:49.962 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T08:02:49.962 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T08:02:49.963 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T08:02:49.963 INFO:teuthology.orchestra.run.smithi157.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T08:02:49.963 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T08:02:49.963 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T08:02:49.963 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T08:02:49.963 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T08:02:49.963 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T08:02:49.963 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T08:02:49.963 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T08:02:49.963 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T08:02:49.963 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T08:02:49.963 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T08:02:49.963 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T08:02:49.964 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T08:02:49.964 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T08:02:49.964 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T08:02:49.964 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T08:02:49.964 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T08:02:49.964 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T08:02:49.964 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T08:02:49.964 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T08:02:49.964 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T08:02:49.964 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T08:02:49.964 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T08:02:49.964 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T08:02:49.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T08:02:49.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T08:02:49.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T08:02:49.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T08:02:49.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T08:02:49.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T08:02:49.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T08:02:49.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T08:02:49.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T08:02:49.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T08:02:49.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T08:02:49.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T08:02:49.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T08:02:49.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T08:02:49.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T08:02:49.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T08:02:49.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T08:02:49.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T08:02:49.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T08:02:49.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T08:02:49.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T08:02:49.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T08:02:49.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T08:02:49.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T08:02:49.966 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T08:02:49.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T08:02:49.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T08:02:49.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T08:02:49.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T08:02:49.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T08:02:49.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T08:02:49.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T08:02:49.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T08:02:49.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T08:02:49.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T08:02:49.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T08:02:49.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T08:02:49.967 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T08:02:49.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T08:02:49.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T08:02:49.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T08:02:49.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T08:02:49.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T08:02:49.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T08:02:49.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T08:02:49.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T08:02:49.968 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T08:02:49.968 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T08:02:49.968 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T08:02:49.968 INFO:teuthology.orchestra.run.smithi157.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T08:02:49.968 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T08:02:49.968 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T08:02:49.969 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T08:02:49.969 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T08:02:49.969 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T08:02:49.969 INFO:teuthology.orchestra.run.smithi157.stdout:Installing weak dependencies: 2024-09-19T08:02:49.969 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T08:02:49.969 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:02:49.969 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-19T08:02:49.969 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-09-19T08:02:49.969 INFO:teuthology.orchestra.run.smithi157.stdout:Install 148 Packages 2024-09-19T08:02:49.969 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 8 Packages 2024-09-19T08:02:49.969 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:02:49.970 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 226 M 2024-09-19T08:02:49.970 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-19T08:02:49.990 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:02:49.991 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:02:50.003 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T08:02:50.004 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T08:02:50.005 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T08:02:50.006 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T08:02:50.008 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T08:02:50.010 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T08:02:50.011 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T08:02:50.081 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-19T08:02:50.096 INFO:teuthology.orchestra.run.smithi064.stdout:====================================================================================== 2024-09-19T08:02:50.096 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-09-19T08:02:50.096 INFO:teuthology.orchestra.run.smithi064.stdout:====================================================================================== 2024-09-19T08:02:50.096 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-19T08:02:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T08:02:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T08:02:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T08:02:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T08:02:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T08:02:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T08:02:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T08:02:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T08:02:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T08:02:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T08:02:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T08:02:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T08:02:50.097 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T08:02:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T08:02:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T08:02:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T08:02:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T08:02:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T08:02:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T08:02:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T08:02:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T08:02:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T08:02:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T08:02:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T08:02:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T08:02:50.098 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T08:02:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T08:02:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T08:02:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T08:02:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T08:02:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T08:02:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T08:02:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T08:02:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T08:02:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T08:02:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T08:02:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T08:02:50.099 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-09-19T08:02:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T08:02:50.099 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T08:02:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T08:02:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T08:02:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T08:02:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T08:02:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T08:02:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T08:02:50.100 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-19T08:02:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T08:02:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T08:02:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T08:02:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T08:02:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T08:02:50.100 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T08:02:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T08:02:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T08:02:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T08:02:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T08:02:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T08:02:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T08:02:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T08:02:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T08:02:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T08:02:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T08:02:50.101 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T08:02:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T08:02:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T08:02:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T08:02:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T08:02:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T08:02:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T08:02:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T08:02:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T08:02:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T08:02:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T08:02:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T08:02:50.102 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T08:02:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T08:02:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T08:02:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T08:02:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T08:02:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T08:02:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T08:02:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T08:02:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T08:02:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T08:02:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T08:02:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T08:02:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T08:02:50.103 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T08:02:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T08:02:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T08:02:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T08:02:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T08:02:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T08:02:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T08:02:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T08:02:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T08:02:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T08:02:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T08:02:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T08:02:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T08:02:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T08:02:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T08:02:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T08:02:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T08:02:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T08:02:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T08:02:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T08:02:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T08:02:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T08:02:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T08:02:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T08:02:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T08:02:50.105 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T08:02:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T08:02:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T08:02:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T08:02:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T08:02:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T08:02:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T08:02:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T08:02:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T08:02:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T08:02:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T08:02:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T08:02:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T08:02:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T08:02:50.106 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T08:02:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T08:02:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T08:02:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T08:02:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T08:02:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T08:02:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T08:02:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T08:02:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T08:02:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T08:02:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T08:02:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T08:02:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T08:02:50.107 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T08:02:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T08:02:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T08:02:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T08:02:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T08:02:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T08:02:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T08:02:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T08:02:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T08:02:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T08:02:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T08:02:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T08:02:50.108 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T08:02:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T08:02:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T08:02:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T08:02:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T08:02:50.109 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-09-19T08:02:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T08:02:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:02:50.109 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-19T08:02:50.109 INFO:teuthology.orchestra.run.smithi064.stdout:====================================================================================== 2024-09-19T08:02:50.109 INFO:teuthology.orchestra.run.smithi064.stdout:Install 148 Packages 2024-09-19T08:02:50.109 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 8 Packages 2024-09-19T08:02:50.109 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:02:50.110 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 226 M 2024-09-19T08:02:50.110 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-19T08:02:50.676 INFO:teuthology.orchestra.run.smithi157.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 53 kB/s | 6.6 kB 00:00 2024-09-19T08:02:50.727 INFO:teuthology.orchestra.run.smithi022.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:02:50.728 INFO:teuthology.orchestra.run.smithi022.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:02:50.740 INFO:teuthology.orchestra.run.smithi022.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T08:02:50.741 INFO:teuthology.orchestra.run.smithi022.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T08:02:50.742 INFO:teuthology.orchestra.run.smithi022.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T08:02:50.743 INFO:teuthology.orchestra.run.smithi022.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T08:02:50.745 INFO:teuthology.orchestra.run.smithi022.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T08:02:50.747 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T08:02:50.748 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T08:02:50.797 INFO:teuthology.orchestra.run.smithi064.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 49 kB/s | 6.6 kB 00:00 2024-09-19T08:02:50.815 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T08:02:50.835 INFO:teuthology.orchestra.run.smithi022.stdout:====================================================================================== 2024-09-19T08:02:50.835 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-19T08:02:50.835 INFO:teuthology.orchestra.run.smithi022.stdout:====================================================================================== 2024-09-19T08:02:50.835 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T08:02:50.835 INFO:teuthology.orchestra.run.smithi022.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T08:02:50.835 INFO:teuthology.orchestra.run.smithi022.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T08:02:50.836 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T08:02:50.836 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T08:02:50.836 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T08:02:50.836 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T08:02:50.836 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T08:02:50.836 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T08:02:50.836 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T08:02:50.836 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T08:02:50.836 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T08:02:50.836 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T08:02:50.836 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T08:02:50.836 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T08:02:50.836 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T08:02:50.837 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T08:02:50.837 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T08:02:50.837 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T08:02:50.837 INFO:teuthology.orchestra.run.smithi022.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T08:02:50.837 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T08:02:50.837 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T08:02:50.837 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T08:02:50.837 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T08:02:50.837 INFO:teuthology.orchestra.run.smithi022.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T08:02:50.837 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T08:02:50.837 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T08:02:50.837 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T08:02:50.837 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T08:02:50.838 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T08:02:50.838 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T08:02:50.838 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T08:02:50.838 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T08:02:50.838 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T08:02:50.838 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T08:02:50.838 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T08:02:50.838 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T08:02:50.838 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T08:02:50.838 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrading: 2024-09-19T08:02:50.838 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T08:02:50.838 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T08:02:50.838 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T08:02:50.839 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T08:02:50.839 INFO:teuthology.orchestra.run.smithi022.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T08:02:50.839 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T08:02:50.839 INFO:teuthology.orchestra.run.smithi022.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T08:02:50.839 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T08:02:50.839 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-19T08:02:50.839 INFO:teuthology.orchestra.run.smithi022.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T08:02:50.839 INFO:teuthology.orchestra.run.smithi022.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T08:02:50.839 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T08:02:50.839 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T08:02:50.839 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T08:02:50.839 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T08:02:50.839 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T08:02:50.839 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T08:02:50.840 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T08:02:50.840 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T08:02:50.840 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T08:02:50.840 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T08:02:50.840 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T08:02:50.840 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T08:02:50.840 INFO:teuthology.orchestra.run.smithi022.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T08:02:50.840 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T08:02:50.840 INFO:teuthology.orchestra.run.smithi022.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T08:02:50.840 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T08:02:50.841 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T08:02:50.841 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T08:02:50.841 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T08:02:50.841 INFO:teuthology.orchestra.run.smithi022.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T08:02:50.841 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T08:02:50.841 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T08:02:50.841 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T08:02:50.841 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T08:02:50.841 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T08:02:50.841 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T08:02:50.841 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T08:02:50.841 INFO:teuthology.orchestra.run.smithi022.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T08:02:50.842 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T08:02:50.842 INFO:teuthology.orchestra.run.smithi022.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T08:02:50.842 INFO:teuthology.orchestra.run.smithi022.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T08:02:50.842 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T08:02:50.842 INFO:teuthology.orchestra.run.smithi022.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T08:02:50.842 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T08:02:50.842 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T08:02:50.842 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T08:02:50.842 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T08:02:50.842 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T08:02:50.842 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T08:02:50.843 INFO:teuthology.orchestra.run.smithi022.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T08:02:50.843 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T08:02:50.843 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T08:02:50.843 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T08:02:50.843 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T08:02:50.843 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T08:02:50.843 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T08:02:50.843 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T08:02:50.843 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T08:02:50.843 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T08:02:50.843 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T08:02:50.843 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T08:02:50.843 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T08:02:50.844 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T08:02:50.844 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T08:02:50.844 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T08:02:50.844 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T08:02:50.844 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T08:02:50.844 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T08:02:50.844 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T08:02:50.844 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T08:02:50.844 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T08:02:50.844 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T08:02:50.844 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T08:02:50.844 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T08:02:50.845 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T08:02:50.845 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T08:02:50.845 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T08:02:50.845 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T08:02:50.845 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T08:02:50.845 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T08:02:50.845 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T08:02:50.845 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T08:02:50.845 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T08:02:50.845 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T08:02:50.845 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T08:02:50.845 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T08:02:50.845 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T08:02:50.846 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T08:02:50.846 INFO:teuthology.orchestra.run.smithi022.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T08:02:50.846 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T08:02:50.846 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T08:02:50.846 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T08:02:50.846 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T08:02:50.846 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T08:02:50.846 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T08:02:50.846 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T08:02:50.846 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T08:02:50.846 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T08:02:50.846 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T08:02:50.846 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T08:02:50.847 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T08:02:50.847 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T08:02:50.847 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T08:02:50.847 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T08:02:50.847 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T08:02:50.847 INFO:teuthology.orchestra.run.smithi022.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T08:02:50.847 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T08:02:50.847 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T08:02:50.847 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T08:02:50.847 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T08:02:50.847 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T08:02:50.847 INFO:teuthology.orchestra.run.smithi022.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T08:02:50.847 INFO:teuthology.orchestra.run.smithi022.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T08:02:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T08:02:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T08:02:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T08:02:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T08:02:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T08:02:50.848 INFO:teuthology.orchestra.run.smithi022.stdout:Installing weak dependencies: 2024-09-19T08:02:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T08:02:50.848 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:02:50.848 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T08:02:50.848 INFO:teuthology.orchestra.run.smithi022.stdout:====================================================================================== 2024-09-19T08:02:50.848 INFO:teuthology.orchestra.run.smithi022.stdout:Install 148 Packages 2024-09-19T08:02:50.848 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrade 8 Packages 2024-09-19T08:02:50.849 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:02:50.849 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 226 M 2024-09-19T08:02:50.849 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T08:02:50.935 INFO:teuthology.orchestra.run.smithi157.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.4 MB/s | 896 kB 00:00 2024-09-19T08:02:51.002 INFO:teuthology.orchestra.run.smithi157.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.1 MB/s | 145 kB 00:00 2024-09-19T08:02:51.030 INFO:teuthology.orchestra.run.smithi064.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.8 MB/s | 896 kB 00:00 2024-09-19T08:02:51.098 INFO:teuthology.orchestra.run.smithi064.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.1 MB/s | 145 kB 00:00 2024-09-19T08:02:51.327 INFO:teuthology.orchestra.run.smithi157.stdout:(4/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 7.3 MB/s | 2.4 MB 00:00 2024-09-19T08:02:51.342 INFO:teuthology.orchestra.run.smithi022.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 47 kB/s | 6.6 kB 00:00 2024-09-19T08:02:51.414 INFO:teuthology.orchestra.run.smithi064.stdout:(4/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 7.5 MB/s | 2.4 MB 00:00 2024-09-19T08:02:51.419 INFO:teuthology.orchestra.run.smithi157.stdout:(5/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 6.2 MB/s | 5.4 MB 00:00 2024-09-19T08:02:51.573 INFO:teuthology.orchestra.run.smithi064.stdout:(5/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 5.9 MB/s | 5.4 MB 00:00 2024-09-19T08:02:51.595 INFO:teuthology.orchestra.run.smithi157.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 5.7 MB/s | 1.5 MB 00:00 2024-09-19T08:02:51.690 INFO:teuthology.orchestra.run.smithi064.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 5.5 MB/s | 1.5 MB 00:00 2024-09-19T08:02:51.701 INFO:teuthology.orchestra.run.smithi022.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.4 MB/s | 896 kB 00:00 2024-09-19T08:02:51.843 INFO:teuthology.orchestra.run.smithi022.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.0 MB/s | 145 kB 00:00 2024-09-19T08:02:51.986 INFO:teuthology.orchestra.run.smithi157.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 8.3 MB/s | 4.7 MB 00:00 2024-09-19T08:02:52.240 INFO:teuthology.orchestra.run.smithi064.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 7.1 MB/s | 4.7 MB 00:00 2024-09-19T08:02:52.794 INFO:teuthology.orchestra.run.smithi022.stdout:(4/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 2.5 MB/s | 2.4 MB 00:00 2024-09-19T08:02:52.960 INFO:teuthology.orchestra.run.smithi022.stdout:(5/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 3.1 MB/s | 5.4 MB 00:01 2024-09-19T08:02:52.995 INFO:teuthology.orchestra.run.smithi157.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 8.7 MB/s | 21 MB 00:02 2024-09-19T08:02:53.087 INFO:teuthology.orchestra.run.smithi157.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 275 kB/s | 25 kB 00:00 2024-09-19T08:02:53.279 INFO:teuthology.orchestra.run.smithi157.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 8.2 MB/s | 11 MB 00:01 2024-09-19T08:02:53.294 INFO:teuthology.orchestra.run.smithi022.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 3.0 MB/s | 1.5 MB 00:00 2024-09-19T08:02:53.324 INFO:teuthology.orchestra.run.smithi064.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 8.0 MB/s | 21 MB 00:02 2024-09-19T08:02:53.345 INFO:teuthology.orchestra.run.smithi157.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.4 MB/s | 228 kB 00:00 2024-09-19T08:02:53.399 INFO:teuthology.orchestra.run.smithi064.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 336 kB/s | 25 kB 00:00 2024-09-19T08:02:53.421 INFO:teuthology.orchestra.run.smithi157.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 361 kB/s | 27 kB 00:00 2024-09-19T08:02:53.496 INFO:teuthology.orchestra.run.smithi157.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 423 kB/s | 32 kB 00:00 2024-09-19T08:02:53.558 INFO:teuthology.orchestra.run.smithi064.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 8.0 MB/s | 11 MB 00:01 2024-09-19T08:02:53.630 INFO:teuthology.orchestra.run.smithi157.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 5.5 MB/s | 751 kB 00:00 2024-09-19T08:02:53.641 INFO:teuthology.orchestra.run.smithi064.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 2.7 MB/s | 228 kB 00:00 2024-09-19T08:02:53.708 INFO:teuthology.orchestra.run.smithi064.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 406 kB/s | 27 kB 00:00 2024-09-19T08:02:53.767 INFO:teuthology.orchestra.run.smithi064.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 544 kB/s | 32 kB 00:00 2024-09-19T08:02:53.805 INFO:teuthology.orchestra.run.smithi157.stdout:(15/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 7.6 MB/s | 17 MB 00:02 2024-09-19T08:02:53.830 INFO:teuthology.orchestra.run.smithi157.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 816 kB/s | 163 kB 00:00 2024-09-19T08:02:53.880 INFO:teuthology.orchestra.run.smithi157.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 1.7 MB/s | 127 kB 00:00 2024-09-19T08:02:53.922 INFO:teuthology.orchestra.run.smithi157.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 5.4 MB/s | 506 kB 00:00 2024-09-19T08:02:53.928 INFO:teuthology.orchestra.run.smithi022.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 4.9 MB/s | 4.7 MB 00:00 2024-09-19T08:02:53.973 INFO:teuthology.orchestra.run.smithi157.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 906 kB/s | 45 kB 00:00 2024-09-19T08:02:53.983 INFO:teuthology.orchestra.run.smithi064.stdout:(14/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 7.4 MB/s | 17 MB 00:02 2024-09-19T08:02:54.009 INFO:teuthology.orchestra.run.smithi064.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 3.0 MB/s | 751 kB 00:00 2024-09-19T08:02:54.031 INFO:teuthology.orchestra.run.smithi157.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.3 MB/s | 136 kB 00:00 2024-09-19T08:02:54.059 INFO:teuthology.orchestra.run.smithi064.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.1 MB/s | 163 kB 00:00 2024-09-19T08:02:54.084 INFO:teuthology.orchestra.run.smithi064.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 1.6 MB/s | 127 kB 00:00 2024-09-19T08:02:54.106 INFO:teuthology.orchestra.run.smithi157.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.1 MB/s | 161 kB 00:00 2024-09-19T08:02:54.176 INFO:teuthology.orchestra.run.smithi064.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 4.2 MB/s | 506 kB 00:00 2024-09-19T08:02:54.199 INFO:teuthology.orchestra.run.smithi157.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 3.4 MB/s | 320 kB 00:00 2024-09-19T08:02:54.251 INFO:teuthology.orchestra.run.smithi064.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 601 kB/s | 45 kB 00:00 2024-09-19T08:02:54.282 INFO:teuthology.orchestra.run.smithi157.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 3.5 MB/s | 300 kB 00:00 2024-09-19T08:02:54.335 INFO:teuthology.orchestra.run.smithi064.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.6 MB/s | 136 kB 00:00 2024-09-19T08:02:54.366 INFO:teuthology.orchestra.run.smithi157.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.2 MB/s | 99 kB 00:00 2024-09-19T08:02:54.419 INFO:teuthology.orchestra.run.smithi064.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.9 MB/s | 161 kB 00:00 2024-09-19T08:02:54.449 INFO:teuthology.orchestra.run.smithi157.stdout:(25/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-19T08:02:54.502 INFO:teuthology.orchestra.run.smithi064.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 3.8 MB/s | 320 kB 00:00 2024-09-19T08:02:54.525 INFO:teuthology.orchestra.run.smithi157.stdout:(26/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 8.3 MB/s | 5.3 MB 00:00 2024-09-19T08:02:54.586 INFO:teuthology.orchestra.run.smithi064.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 3.5 MB/s | 300 kB 00:00 2024-09-19T08:02:54.617 INFO:teuthology.orchestra.run.smithi157.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 1.8 MB/s | 171 kB 00:00 2024-09-19T08:02:54.669 INFO:teuthology.orchestra.run.smithi064.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.2 MB/s | 99 kB 00:00 2024-09-19T08:02:54.700 INFO:teuthology.orchestra.run.smithi157.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 382 kB/s | 32 kB 00:00 2024-09-19T08:02:54.770 INFO:teuthology.orchestra.run.smithi064.stdout:(25/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 7.8 MB/s | 5.3 MB 00:00 2024-09-19T08:02:54.784 INFO:teuthology.orchestra.run.smithi157.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.7 MB/s | 146 kB 00:00 2024-09-19T08:02:54.795 INFO:teuthology.orchestra.run.smithi064.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 680 kB/s | 85 kB 00:00 2024-09-19T08:02:54.835 INFO:teuthology.orchestra.run.smithi157.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 8.1 MB/s | 3.1 MB 00:00 2024-09-19T08:02:54.879 INFO:teuthology.orchestra.run.smithi064.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.0 MB/s | 171 kB 00:00 2024-09-19T08:02:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 382 kB/s | 32 kB 00:00 2024-09-19T08:02:55.063 INFO:teuthology.orchestra.run.smithi064.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.4 MB/s | 146 kB 00:00 2024-09-19T08:02:55.221 INFO:teuthology.orchestra.run.smithi064.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 6.9 MB/s | 3.1 MB 00:00 2024-09-19T08:02:55.293 INFO:teuthology.orchestra.run.smithi157.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 7.0 MB/s | 3.6 MB 00:00 2024-09-19T08:02:55.402 INFO:teuthology.orchestra.run.smithi157.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.3 MB/s | 249 kB 00:00 2024-09-19T08:02:55.494 INFO:teuthology.orchestra.run.smithi157.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 539 kB/s | 49 kB 00:00 2024-09-19T08:02:55.586 INFO:teuthology.orchestra.run.smithi157.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 180 kB/s | 16 kB 00:00 2024-09-19T08:02:55.647 INFO:teuthology.orchestra.run.smithi064.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 6.1 MB/s | 3.6 MB 00:00 2024-09-19T08:02:55.678 INFO:teuthology.orchestra.run.smithi157.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 3.0 MB/s | 285 kB 00:00 2024-09-19T08:02:55.747 INFO:teuthology.orchestra.run.smithi064.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.4 MB/s | 249 kB 00:00 2024-09-19T08:02:55.762 INFO:teuthology.orchestra.run.smithi157.stdout:(36/156): ceph-mgr-diskprediction-local-19.1.1- 8.0 MB/s | 7.4 MB 00:00 2024-09-19T08:02:55.805 INFO:teuthology.orchestra.run.smithi064.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 848 kB/s | 49 kB 00:00 2024-09-19T08:02:55.845 INFO:teuthology.orchestra.run.smithi157.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 4.5 MB/s | 765 kB 00:00 2024-09-19T08:02:55.870 INFO:teuthology.orchestra.run.smithi157.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 168 kB/s | 18 kB 00:00 2024-09-19T08:02:55.881 INFO:teuthology.orchestra.run.smithi064.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 219 kB/s | 16 kB 00:00 2024-09-19T08:02:55.964 INFO:teuthology.orchestra.run.smithi064.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 3.3 MB/s | 285 kB 00:00 2024-09-19T08:02:55.979 INFO:teuthology.orchestra.run.smithi157.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 598 kB/s | 80 kB 00:00 2024-09-19T08:02:56.046 INFO:teuthology.orchestra.run.smithi157.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-19T08:02:56.071 INFO:teuthology.orchestra.run.smithi157.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-19T08:02:56.097 INFO:teuthology.orchestra.run.smithi157.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-09-19T08:02:56.115 INFO:teuthology.orchestra.run.smithi064.stdout:(36/156): ceph-mgr-diskprediction-local-19.1.1- 8.3 MB/s | 7.4 MB 00:00 2024-09-19T08:02:56.140 INFO:teuthology.orchestra.run.smithi064.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 4.3 MB/s | 765 kB 00:00 2024-09-19T08:02:56.147 INFO:teuthology.orchestra.run.smithi157.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-09-19T08:02:56.172 INFO:teuthology.orchestra.run.smithi157.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-19T08:02:56.174 INFO:teuthology.orchestra.run.smithi064.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 311 kB/s | 18 kB 00:00 2024-09-19T08:02:56.197 INFO:teuthology.orchestra.run.smithi157.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-19T08:02:56.231 INFO:teuthology.orchestra.run.smithi157.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-19T08:02:56.253 INFO:teuthology.orchestra.run.smithi022.stdout:(8/156): ceph-radosgw-19.1.1-361.g2f599a46.el9. 4.6 MB/s | 11 MB 00:02 2024-09-19T08:02:56.256 INFO:teuthology.orchestra.run.smithi157.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-19T08:02:56.282 INFO:teuthology.orchestra.run.smithi064.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 562 kB/s | 80 kB 00:00 2024-09-19T08:02:56.294 INFO:teuthology.orchestra.run.smithi022.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 609 kB/s | 25 kB 00:00 2024-09-19T08:02:56.366 INFO:teuthology.orchestra.run.smithi064.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-19T08:02:56.449 INFO:teuthology.orchestra.run.smithi064.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-19T08:02:56.483 INFO:teuthology.orchestra.run.smithi064.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-19T08:02:56.533 INFO:teuthology.orchestra.run.smithi064.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-19T08:02:56.584 INFO:teuthology.orchestra.run.smithi064.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-19T08:02:56.642 INFO:teuthology.orchestra.run.smithi064.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-19T08:02:56.701 INFO:teuthology.orchestra.run.smithi064.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.2 MB 00:00 2024-09-19T08:02:56.726 INFO:teuthology.orchestra.run.smithi064.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-19T08:02:56.818 INFO:teuthology.orchestra.run.smithi064.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 5.7 MB/s | 689 kB 00:00 2024-09-19T08:02:56.849 INFO:teuthology.orchestra.run.smithi157.stdout:(48/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 13 MB/s | 49 MB 00:03 2024-09-19T08:02:56.869 INFO:teuthology.orchestra.run.smithi064.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-19T08:02:56.899 INFO:teuthology.orchestra.run.smithi157.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-09-19T08:02:56.941 INFO:teuthology.orchestra.run.smithi157.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 943 kB 00:00 2024-09-19T08:02:56.966 INFO:teuthology.orchestra.run.smithi157.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 324 kB/s | 22 kB 00:00 2024-09-19T08:02:57.011 INFO:teuthology.orchestra.run.smithi022.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 4.5 MB/s | 17 MB 00:03 2024-09-19T08:02:57.016 INFO:teuthology.orchestra.run.smithi157.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-09-19T08:02:57.042 INFO:teuthology.orchestra.run.smithi157.stdout:(53/156): boost-program-options-1.75.0-8.el9.x8 555 kB/s | 107 kB 00:00 2024-09-19T08:02:57.067 INFO:teuthology.orchestra.run.smithi157.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-19T08:02:57.092 INFO:teuthology.orchestra.run.smithi157.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-19T08:02:57.117 INFO:teuthology.orchestra.run.smithi157.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-19T08:02:57.168 INFO:teuthology.orchestra.run.smithi157.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-09-19T08:02:57.193 INFO:teuthology.orchestra.run.smithi157.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-19T08:02:57.237 INFO:teuthology.orchestra.run.smithi022.stdout:(11/156): ceph-common-19.1.1-361.g2f599a46.el9. 3.5 MB/s | 21 MB 00:06 2024-09-19T08:02:57.262 INFO:teuthology.orchestra.run.smithi022.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 912 kB/s | 228 kB 00:00 2024-09-19T08:02:57.287 INFO:teuthology.orchestra.run.smithi022.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 542 kB/s | 27 kB 00:00 2024-09-19T08:02:57.294 INFO:teuthology.orchestra.run.smithi064.stdout:(50/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 12 MB/s | 49 MB 00:03 2024-09-19T08:02:57.311 INFO:teuthology.orchestra.run.smithi064.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 49 kB/s | 22 kB 00:00 2024-09-19T08:02:57.312 INFO:teuthology.orchestra.run.smithi022.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 633 kB/s | 32 kB 00:00 2024-09-19T08:02:57.353 INFO:teuthology.orchestra.run.smithi064.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 1.5 MB/s | 943 kB 00:00 2024-09-19T08:02:57.363 INFO:teuthology.orchestra.run.smithi022.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 3.2 MB/s | 163 kB 00:00 2024-09-19T08:02:57.368 INFO:teuthology.orchestra.run.smithi157.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 3.2 MB/s | 581 kB 00:00 2024-09-19T08:02:57.369 INFO:teuthology.orchestra.run.smithi064.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-19T08:02:57.402 INFO:teuthology.orchestra.run.smithi157.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-09-19T08:02:57.411 INFO:teuthology.orchestra.run.smithi064.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-09-19T08:02:57.413 INFO:teuthology.orchestra.run.smithi022.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T08:02:57.427 INFO:teuthology.orchestra.run.smithi157.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-19T08:02:57.438 INFO:teuthology.orchestra.run.smithi022.stdout:(17/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 4.9 MB/s | 751 kB 00:00 2024-09-19T08:02:57.452 INFO:teuthology.orchestra.run.smithi157.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-19T08:02:57.453 INFO:teuthology.orchestra.run.smithi064.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-09-19T08:02:57.514 INFO:teuthology.orchestra.run.smithi022.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 4.9 MB/s | 506 kB 00:00 2024-09-19T08:02:57.520 INFO:teuthology.orchestra.run.smithi064.stdout:(56/156): flex-2.6.4-9.el9.x86_64.rpm 1.4 MB/s | 313 kB 00:00 2024-09-19T08:02:57.546 INFO:teuthology.orchestra.run.smithi064.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-19T08:02:57.561 INFO:teuthology.orchestra.run.smithi157.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-19T08:02:57.564 INFO:teuthology.orchestra.run.smithi022.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 908 kB/s | 45 kB 00:00 2024-09-19T08:02:57.581 INFO:teuthology.orchestra.run.smithi064.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 755 kB/s | 45 kB 00:00 2024-09-19T08:02:57.586 INFO:teuthology.orchestra.run.smithi157.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2024-09-19T08:02:57.612 INFO:teuthology.orchestra.run.smithi157.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 765 kB/s | 38 kB 00:00 2024-09-19T08:02:57.614 INFO:teuthology.orchestra.run.smithi022.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.7 MB/s | 136 kB 00:00 2024-09-19T08:02:57.664 INFO:teuthology.orchestra.run.smithi022.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 3.2 MB/s | 161 kB 00:00 2024-09-19T08:02:57.689 INFO:teuthology.orchestra.run.smithi064.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-19T08:02:57.731 INFO:teuthology.orchestra.run.smithi022.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 4.7 MB/s | 320 kB 00:00 2024-09-19T08:02:57.748 INFO:teuthology.orchestra.run.smithi064.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 3.4 MB/s | 581 kB 00:00 2024-09-19T08:02:57.773 INFO:teuthology.orchestra.run.smithi064.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 293 kB/s | 24 kB 00:00 2024-09-19T08:02:57.798 INFO:teuthology.orchestra.run.smithi022.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 4.4 MB/s | 300 kB 00:00 2024-09-19T08:02:57.806 INFO:teuthology.orchestra.run.smithi064.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-09-19T08:02:57.849 INFO:teuthology.orchestra.run.smithi022.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T08:02:57.865 INFO:teuthology.orchestra.run.smithi064.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-19T08:02:57.891 INFO:teuthology.orchestra.run.smithi064.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-19T08:02:57.899 INFO:teuthology.orchestra.run.smithi022.stdout:(25/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T08:02:57.937 INFO:teuthology.orchestra.run.smithi157.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-09-19T08:02:57.941 INFO:teuthology.orchestra.run.smithi064.stdout:(65/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.1 MB/s | 3.0 MB 00:00 2024-09-19T08:02:57.966 INFO:teuthology.orchestra.run.smithi064.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 383 kB/s | 38 kB 00:00 2024-09-19T08:02:58.099 INFO:teuthology.orchestra.run.smithi022.stdout:(26/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 8.1 MB/s | 5.3 MB 00:00 2024-09-19T08:02:58.149 INFO:teuthology.orchestra.run.smithi022.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-19T08:02:58.200 INFO:teuthology.orchestra.run.smithi064.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-09-19T08:02:58.200 INFO:teuthology.orchestra.run.smithi022.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 639 kB/s | 32 kB 00:00 2024-09-19T08:02:58.258 INFO:teuthology.orchestra.run.smithi022.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.5 MB/s | 146 kB 00:00 2024-09-19T08:02:58.329 INFO:teuthology.orchestra.run.smithi157.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.1 MB/s | 1.6 MB 00:00 2024-09-19T08:02:58.342 INFO:teuthology.orchestra.run.smithi022.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 7.0 MB/s | 3.1 MB 00:00 2024-09-19T08:02:58.358 INFO:teuthology.orchestra.run.smithi064.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-19T08:02:58.392 INFO:teuthology.orchestra.run.smithi064.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 4.6 MB/s | 910 kB 00:00 2024-09-19T08:02:58.417 INFO:teuthology.orchestra.run.smithi064.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-19T08:02:58.509 INFO:teuthology.orchestra.run.smithi064.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-09-19T08:02:58.563 INFO:teuthology.orchestra.run.smithi157.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 3.8 MB/s | 910 kB 00:00 2024-09-19T08:02:58.576 INFO:teuthology.orchestra.run.smithi064.stdout:(72/156): openblas-openmp-0.3.26-2.el9.x86_64.r 7.2 MB/s | 4.9 MB 00:00 2024-09-19T08:02:58.601 INFO:teuthology.orchestra.run.smithi064.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 952 kB/s | 87 kB 00:00 2024-09-19T08:02:58.613 INFO:teuthology.orchestra.run.smithi157.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.7 MB/s | 137 kB 00:00 2024-09-19T08:02:58.643 INFO:teuthology.orchestra.run.smithi064.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-19T08:02:58.702 INFO:teuthology.orchestra.run.smithi064.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-19T08:02:58.743 INFO:teuthology.orchestra.run.smithi022.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 7.4 MB/s | 3.6 MB 00:00 2024-09-19T08:02:58.760 INFO:teuthology.orchestra.run.smithi064.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-19T08:02:58.809 INFO:teuthology.orchestra.run.smithi022.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.7 MB/s | 249 kB 00:00 2024-09-19T08:02:58.810 INFO:teuthology.orchestra.run.smithi064.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-19T08:02:58.821 INFO:teuthology.orchestra.run.smithi157.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.0 MB/s | 4.9 MB 00:01 2024-09-19T08:02:58.851 INFO:teuthology.orchestra.run.smithi022.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 1.2 MB/s | 49 kB 00:00 2024-09-19T08:02:58.861 INFO:teuthology.orchestra.run.smithi064.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-19T08:02:58.888 INFO:teuthology.orchestra.run.smithi157.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-09-19T08:02:58.893 INFO:teuthology.orchestra.run.smithi022.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 398 kB/s | 16 kB 00:00 2024-09-19T08:02:58.911 INFO:teuthology.orchestra.run.smithi064.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-19T08:02:58.938 INFO:teuthology.orchestra.run.smithi157.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-19T08:02:58.969 INFO:teuthology.orchestra.run.smithi022.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 3.7 MB/s | 285 kB 00:00 2024-09-19T08:02:58.986 INFO:teuthology.orchestra.run.smithi064.stdout:(80/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-19T08:02:59.003 INFO:teuthology.orchestra.run.smithi064.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 378 kB/s | 35 kB 00:00 2024-09-19T08:02:59.078 INFO:teuthology.orchestra.run.smithi064.stdout:(82/156): postgresql-server-13.16-1.el9.x86_64. 8.4 MB/s | 5.8 MB 00:00 2024-09-19T08:02:59.085 INFO:teuthology.orchestra.run.smithi022.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 6.4 MB/s | 765 kB 00:00 2024-09-19T08:02:59.089 INFO:teuthology.orchestra.run.smithi157.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.8 MB/s | 1.0 MB 00:00 2024-09-19T08:02:59.104 INFO:teuthology.orchestra.run.smithi064.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-19T08:02:59.128 INFO:teuthology.orchestra.run.smithi022.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 439 kB/s | 18 kB 00:00 2024-09-19T08:02:59.137 INFO:teuthology.orchestra.run.smithi064.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-19T08:02:59.139 INFO:teuthology.orchestra.run.smithi157.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-19T08:02:59.162 INFO:teuthology.orchestra.run.smithi064.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-19T08:02:59.180 INFO:teuthology.orchestra.run.smithi064.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-19T08:02:59.222 INFO:teuthology.orchestra.run.smithi064.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-19T08:02:59.228 INFO:teuthology.orchestra.run.smithi022.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 8.4 MB/s | 7.4 MB 00:00 2024-09-19T08:02:59.231 INFO:teuthology.orchestra.run.smithi157.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-19T08:02:59.247 INFO:teuthology.orchestra.run.smithi064.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-19T08:02:59.314 INFO:teuthology.orchestra.run.smithi064.stdout:(89/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-19T08:02:59.315 INFO:teuthology.orchestra.run.smithi157.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-19T08:02:59.339 INFO:teuthology.orchestra.run.smithi064.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 553 kB/s | 51 kB 00:00 2024-09-19T08:02:59.364 INFO:teuthology.orchestra.run.smithi064.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-09-19T08:02:59.365 INFO:teuthology.orchestra.run.smithi157.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-19T08:02:59.390 INFO:teuthology.orchestra.run.smithi064.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-09-19T08:02:59.415 INFO:teuthology.orchestra.run.smithi064.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-09-19T08:02:59.415 INFO:teuthology.orchestra.run.smithi157.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-19T08:02:59.436 INFO:teuthology.orchestra.run.smithi022.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 259 kB/s | 80 kB 00:00 2024-09-19T08:02:59.440 INFO:teuthology.orchestra.run.smithi064.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 879 kB/s | 44 kB 00:00 2024-09-19T08:02:59.465 INFO:teuthology.orchestra.run.smithi064.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-19T08:02:59.491 INFO:teuthology.orchestra.run.smithi064.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-19T08:02:59.499 INFO:teuthology.orchestra.run.smithi157.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-19T08:02:59.516 INFO:teuthology.orchestra.run.smithi064.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-09-19T08:02:59.549 INFO:teuthology.orchestra.run.smithi157.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 698 kB/s | 35 kB 00:00 2024-09-19T08:02:59.549 INFO:teuthology.orchestra.run.smithi064.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-19T08:02:59.575 INFO:teuthology.orchestra.run.smithi064.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 990 kB/s | 58 kB 00:00 2024-09-19T08:02:59.608 INFO:teuthology.orchestra.run.smithi064.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-19T08:02:59.634 INFO:teuthology.orchestra.run.smithi064.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-19T08:02:59.653 INFO:teuthology.orchestra.run.smithi022.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 882 kB/s | 191 kB 00:00 2024-09-19T08:02:59.667 INFO:teuthology.orchestra.run.smithi064.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-09-19T08:02:59.751 INFO:teuthology.orchestra.run.smithi064.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-19T08:02:59.782 INFO:teuthology.orchestra.run.smithi157.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 4.9 MB/s | 5.8 MB 00:01 2024-09-19T08:02:59.820 INFO:teuthology.orchestra.run.smithi022.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-09-19T08:02:59.826 INFO:teuthology.orchestra.run.smithi064.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 7.1 MB/s | 548 kB 00:00 2024-09-19T08:02:59.866 INFO:teuthology.orchestra.run.smithi157.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-19T08:02:59.868 INFO:teuthology.orchestra.run.smithi064.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-19T08:02:59.893 INFO:teuthology.orchestra.run.smithi064.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-09-19T08:02:59.946 INFO:teuthology.orchestra.run.smithi022.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-19T08:02:59.966 INFO:teuthology.orchestra.run.smithi157.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-19T08:02:59.985 INFO:teuthology.orchestra.run.smithi064.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-09-19T08:03:00.017 INFO:teuthology.orchestra.run.smithi157.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T08:03:00.035 INFO:teuthology.orchestra.run.smithi064.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 522 kB/s | 26 kB 00:00 2024-09-19T08:03:00.067 INFO:teuthology.orchestra.run.smithi157.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-09-19T08:03:00.069 INFO:teuthology.orchestra.run.smithi064.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-19T08:03:00.071 INFO:teuthology.orchestra.run.smithi022.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 852 kB/s | 106 kB 00:00 2024-09-19T08:03:00.159 INFO:teuthology.orchestra.run.smithi157.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-09-19T08:03:00.196 INFO:teuthology.orchestra.run.smithi022.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-19T08:03:00.217 INFO:teuthology.orchestra.run.smithi157.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-19T08:03:00.228 INFO:teuthology.orchestra.run.smithi064.stdout:(110/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-09-19T08:03:00.253 INFO:teuthology.orchestra.run.smithi064.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2024-09-19T08:03:00.263 INFO:teuthology.orchestra.run.smithi022.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-19T08:03:00.301 INFO:teuthology.orchestra.run.smithi157.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-09-19T08:03:00.345 INFO:teuthology.orchestra.run.smithi064.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-09-19T08:03:00.347 INFO:teuthology.orchestra.run.smithi022.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 2.3 MB/s | 1.2 MB 00:00 2024-09-19T08:03:00.351 INFO:teuthology.orchestra.run.smithi157.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T08:03:00.387 INFO:teuthology.orchestra.run.smithi064.stdout:(113/156): luarocks-3.9.2-5.el9.noarch.rpm 956 kB/s | 151 kB 00:00 2024-09-19T08:03:00.401 INFO:teuthology.orchestra.run.smithi157.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T08:03:00.412 INFO:teuthology.orchestra.run.smithi064.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 8.3 MB/s | 548 kB 00:00 2024-09-19T08:03:00.414 INFO:teuthology.orchestra.run.smithi022.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.4 MB/s | 215 kB 00:00 2024-09-19T08:03:00.437 INFO:teuthology.orchestra.run.smithi064.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 865 kB/s | 43 kB 00:00 2024-09-19T08:03:00.463 INFO:teuthology.orchestra.run.smithi064.stdout:(116/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 839 kB 00:00 2024-09-19T08:03:00.488 INFO:teuthology.orchestra.run.smithi064.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-19T08:03:00.513 INFO:teuthology.orchestra.run.smithi064.stdout:(118/156): python3-cachetools-4.2.4-1.el9.noarc 319 kB/s | 32 kB 00:00 2024-09-19T08:03:00.539 INFO:teuthology.orchestra.run.smithi064.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-09-19T08:03:00.560 INFO:teuthology.orchestra.run.smithi157.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.0 MB/s | 5.1 MB 00:01 2024-09-19T08:03:00.564 INFO:teuthology.orchestra.run.smithi064.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-19T08:03:00.590 INFO:teuthology.orchestra.run.smithi064.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-19T08:03:00.610 INFO:teuthology.orchestra.run.smithi157.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 837 kB/s | 42 kB 00:00 2024-09-19T08:03:00.623 INFO:teuthology.orchestra.run.smithi064.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-19T08:03:00.648 INFO:teuthology.orchestra.run.smithi064.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-19T08:03:00.661 INFO:teuthology.orchestra.run.smithi157.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 878 kB/s | 44 kB 00:00 2024-09-19T08:03:00.674 INFO:teuthology.orchestra.run.smithi064.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-09-19T08:03:00.699 INFO:teuthology.orchestra.run.smithi064.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-19T08:03:00.724 INFO:teuthology.orchestra.run.smithi064.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-19T08:03:00.744 INFO:teuthology.orchestra.run.smithi157.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-19T08:03:00.749 INFO:teuthology.orchestra.run.smithi064.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-19T08:03:00.774 INFO:teuthology.orchestra.run.smithi064.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-19T08:03:00.795 INFO:teuthology.orchestra.run.smithi157.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-09-19T08:03:00.800 INFO:teuthology.orchestra.run.smithi064.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T08:03:00.825 INFO:teuthology.orchestra.run.smithi064.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T08:03:00.851 INFO:teuthology.orchestra.run.smithi064.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T08:03:00.870 INFO:teuthology.orchestra.run.smithi157.stdout:(96/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.1 MB/s | 6.0 MB 00:01 2024-09-19T08:03:00.892 INFO:teuthology.orchestra.run.smithi064.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-19T08:03:00.923 INFO:teuthology.orchestra.run.smithi157.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 369 kB/s | 47 kB 00:00 2024-09-19T08:03:00.926 INFO:teuthology.orchestra.run.smithi064.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-19T08:03:00.939 INFO:teuthology.orchestra.run.smithi022.stdout:(48/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 10 MB/s | 49 MB 00:04 2024-09-19T08:03:00.951 INFO:teuthology.orchestra.run.smithi064.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-09-19T08:03:00.977 INFO:teuthology.orchestra.run.smithi064.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-19T08:03:00.981 INFO:teuthology.orchestra.run.smithi022.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 689 kB 00:00 2024-09-19T08:03:00.987 INFO:teuthology.orchestra.run.smithi157.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 910 kB/s | 58 kB 00:00 2024-09-19T08:03:01.003 INFO:teuthology.orchestra.run.smithi064.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-19T08:03:01.023 INFO:teuthology.orchestra.run.smithi022.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 521 kB/s | 22 kB 00:00 2024-09-19T08:03:01.027 INFO:teuthology.orchestra.run.smithi064.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-19T08:03:01.037 INFO:teuthology.orchestra.run.smithi157.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.8 MB/s | 304 kB 00:00 2024-09-19T08:03:01.052 INFO:teuthology.orchestra.run.smithi064.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-19T08:03:01.063 INFO:teuthology.orchestra.run.smithi157.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-19T08:03:01.065 INFO:teuthology.orchestra.run.smithi022.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 1.4 MB/s | 943 kB 00:00 2024-09-19T08:03:01.078 INFO:teuthology.orchestra.run.smithi064.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-19T08:03:01.090 INFO:teuthology.orchestra.run.smithi022.stdout:(52/156): boost-program-options-1.75.0-8.el9.x8 711 kB/s | 107 kB 00:00 2024-09-19T08:03:01.103 INFO:teuthology.orchestra.run.smithi064.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-19T08:03:01.113 INFO:teuthology.orchestra.run.smithi157.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 896 kB/s | 45 kB 00:00 2024-09-19T08:03:01.115 INFO:teuthology.orchestra.run.smithi022.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-09-19T08:03:01.128 INFO:teuthology.orchestra.run.smithi064.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-19T08:03:01.138 INFO:teuthology.orchestra.run.smithi157.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 633 kB/s | 64 kB 00:00 2024-09-19T08:03:01.141 INFO:teuthology.orchestra.run.smithi022.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-19T08:03:01.166 INFO:teuthology.orchestra.run.smithi022.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-19T08:03:01.179 INFO:teuthology.orchestra.run.smithi064.stdout:(142/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-09-19T08:03:01.191 INFO:teuthology.orchestra.run.smithi022.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-19T08:03:01.204 INFO:teuthology.orchestra.run.smithi064.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-19T08:03:01.225 INFO:teuthology.orchestra.run.smithi022.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-19T08:03:01.229 INFO:teuthology.orchestra.run.smithi064.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 893 kB/s | 90 kB 00:00 2024-09-19T08:03:01.242 INFO:teuthology.orchestra.run.smithi022.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 897 kB/s | 45 kB 00:00 2024-09-19T08:03:01.246 INFO:teuthology.orchestra.run.smithi064.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-09-19T08:03:01.247 INFO:teuthology.orchestra.run.smithi157.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-09-19T08:03:01.271 INFO:teuthology.orchestra.run.smithi064.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-19T08:03:01.292 INFO:teuthology.orchestra.run.smithi022.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 11 MB/s | 581 kB 00:00 2024-09-19T08:03:01.297 INFO:teuthology.orchestra.run.smithi064.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-19T08:03:01.306 INFO:teuthology.orchestra.run.smithi157.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T08:03:01.326 INFO:teuthology.orchestra.run.smithi022.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-09-19T08:03:01.343 INFO:teuthology.orchestra.run.smithi022.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 488 kB/s | 24 kB 00:00 2024-09-19T08:03:01.348 INFO:teuthology.orchestra.run.smithi157.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-19T08:03:01.360 INFO:teuthology.orchestra.run.smithi022.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-19T08:03:01.364 INFO:teuthology.orchestra.run.smithi064.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-19T08:03:01.373 INFO:teuthology.orchestra.run.smithi157.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 782 kB/s | 19 kB 00:00 2024-09-19T08:03:01.419 INFO:teuthology.orchestra.run.smithi022.stdout:(63/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.1 MB/s | 3.0 MB 00:00 2024-09-19T08:03:01.452 INFO:teuthology.orchestra.run.smithi022.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-19T08:03:01.469 INFO:teuthology.orchestra.run.smithi022.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-19T08:03:01.473 INFO:teuthology.orchestra.run.smithi157.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-09-19T08:03:01.494 INFO:teuthology.orchestra.run.smithi022.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-19T08:03:01.557 INFO:teuthology.orchestra.run.smithi157.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-09-19T08:03:01.582 INFO:teuthology.orchestra.run.smithi157.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-19T08:03:01.608 INFO:teuthology.orchestra.run.smithi157.stdout:(110/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 194 kB/s | 26 kB 00:00 2024-09-19T08:03:01.628 INFO:teuthology.orchestra.run.smithi022.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-09-19T08:03:01.632 INFO:teuthology.orchestra.run.smithi157.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-19T08:03:01.666 INFO:teuthology.orchestra.run.smithi157.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-19T08:03:01.700 INFO:teuthology.orchestra.run.smithi157.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-19T08:03:01.703 INFO:teuthology.orchestra.run.smithi022.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 20 MB/s | 4.9 MB 00:00 2024-09-19T08:03:01.717 INFO:teuthology.orchestra.run.smithi157.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-19T08:03:01.731 INFO:teuthology.orchestra.run.smithi064.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 7.3 MB/s | 3.2 MB 00:00 2024-09-19T08:03:01.734 INFO:teuthology.orchestra.run.smithi157.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-19T08:03:01.751 INFO:teuthology.orchestra.run.smithi157.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 857 kB/s | 14 kB 00:00 2024-09-19T08:03:01.759 INFO:teuthology.orchestra.run.smithi022.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 6.8 MB/s | 910 kB 00:00 2024-09-19T08:03:01.781 INFO:teuthology.orchestra.run.smithi064.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 5.6 MB/s | 2.3 MB 00:00 2024-09-19T08:03:01.784 INFO:teuthology.orchestra.run.smithi157.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 4.5 MB/s | 548 kB 00:00 2024-09-19T08:03:01.787 INFO:teuthology.orchestra.run.smithi022.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-19T08:03:01.810 INFO:teuthology.orchestra.run.smithi157.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-19T08:03:01.829 INFO:teuthology.orchestra.run.smithi022.stdout:(71/156): postgresql-13.16-1.el9.x86_64.rpm 4.7 MB/s | 1.6 MB 00:00 2024-09-19T08:03:01.843 INFO:teuthology.orchestra.run.smithi157.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-19T08:03:01.863 INFO:teuthology.orchestra.run.smithi022.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-19T08:03:01.865 INFO:teuthology.orchestra.run.smithi064.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 492 kB/s | 41 kB 00:00 2024-09-19T08:03:01.869 INFO:teuthology.orchestra.run.smithi157.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-19T08:03:01.888 INFO:teuthology.orchestra.run.smithi022.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T08:03:01.894 INFO:teuthology.orchestra.run.smithi157.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T08:03:01.915 INFO:teuthology.orchestra.run.smithi064.stdout:(152/156): librados2-19.1.1-361.g2f599a46.el9.x 5.3 MB/s | 3.4 MB 00:00 2024-09-19T08:03:01.928 INFO:teuthology.orchestra.run.smithi157.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-19T08:03:01.940 INFO:teuthology.orchestra.run.smithi064.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 3.2 MB/s | 249 kB 00:00 2024-09-19T08:03:01.953 INFO:teuthology.orchestra.run.smithi157.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-19T08:03:01.955 INFO:teuthology.orchestra.run.smithi022.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 919 kB/s | 61 kB 00:00 2024-09-19T08:03:01.978 INFO:teuthology.orchestra.run.smithi157.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 712 kB/s | 18 kB 00:00 2024-09-19T08:03:02.003 INFO:teuthology.orchestra.run.smithi157.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 932 kB/s | 23 kB 00:00 2024-09-19T08:03:02.029 INFO:teuthology.orchestra.run.smithi157.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 778 kB/s | 19 kB 00:00 2024-09-19T08:03:02.030 INFO:teuthology.orchestra.run.smithi022.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-09-19T08:03:02.054 INFO:teuthology.orchestra.run.smithi157.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 790 kB/s | 20 kB 00:00 2024-09-19T08:03:02.055 INFO:teuthology.orchestra.run.smithi022.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-19T08:03:02.089 INFO:teuthology.orchestra.run.smithi022.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-19T08:03:02.104 INFO:teuthology.orchestra.run.smithi157.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-19T08:03:02.114 INFO:teuthology.orchestra.run.smithi022.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 436 kB/s | 26 kB 00:00 2024-09-19T08:03:02.130 INFO:teuthology.orchestra.run.smithi157.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T08:03:02.140 INFO:teuthology.orchestra.run.smithi022.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-19T08:03:02.155 INFO:teuthology.orchestra.run.smithi157.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T08:03:02.173 INFO:teuthology.orchestra.run.smithi022.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-19T08:03:02.180 INFO:teuthology.orchestra.run.smithi157.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-19T08:03:02.190 INFO:teuthology.orchestra.run.smithi064.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 2.4 MB 00:00 2024-09-19T08:03:02.198 INFO:teuthology.orchestra.run.smithi022.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 591 kB/s | 35 kB 00:00 2024-09-19T08:03:02.282 INFO:teuthology.orchestra.run.smithi022.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-19T08:03:02.283 INFO:teuthology.orchestra.run.smithi064.stdout:(155/156): NetworkManager-libnm-1.51.0-1.el9.x8 3.3 MB/s | 1.8 MB 00:00 2024-09-19T08:03:02.316 INFO:teuthology.orchestra.run.smithi022.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-19T08:03:02.339 INFO:teuthology.orchestra.run.smithi157.stdout:(132/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.4 MB/s | 16 MB 00:01 2024-09-19T08:03:02.349 INFO:teuthology.orchestra.run.smithi064.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 3.1 MB/s | 1.4 MB 00:00 2024-09-19T08:03:02.349 INFO:teuthology.orchestra.run.smithi022.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-09-19T08:03:02.355 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-19T08:03:02.355 INFO:teuthology.orchestra.run.smithi064.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-19T08:03:02.373 INFO:teuthology.orchestra.run.smithi157.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-09-19T08:03:02.374 INFO:teuthology.orchestra.run.smithi022.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-19T08:03:02.400 INFO:teuthology.orchestra.run.smithi022.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-19T08:03:02.406 INFO:teuthology.orchestra.run.smithi157.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 2.9 MB/s | 1.0 MB 00:00 2024-09-19T08:03:02.425 INFO:teuthology.orchestra.run.smithi022.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-19T08:03:02.432 INFO:teuthology.orchestra.run.smithi157.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-09-19T08:03:02.451 INFO:teuthology.orchestra.run.smithi022.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-19T08:03:02.457 INFO:teuthology.orchestra.run.smithi157.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-09-19T08:03:02.476 INFO:teuthology.orchestra.run.smithi022.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-19T08:03:02.482 INFO:teuthology.orchestra.run.smithi157.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-19T08:03:02.501 INFO:teuthology.orchestra.run.smithi022.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T08:03:02.507 INFO:teuthology.orchestra.run.smithi157.stdout:(138/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 680 kB/s | 90 kB 00:00 2024-09-19T08:03:02.527 INFO:teuthology.orchestra.run.smithi022.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T08:03:02.533 INFO:teuthology.orchestra.run.smithi157.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-19T08:03:02.558 INFO:teuthology.orchestra.run.smithi157.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-19T08:03:02.583 INFO:teuthology.orchestra.run.smithi157.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-19T08:03:02.610 INFO:teuthology.orchestra.run.smithi022.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-19T08:03:02.617 INFO:teuthology.orchestra.run.smithi157.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-19T08:03:02.636 INFO:teuthology.orchestra.run.smithi022.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-19T08:03:02.642 INFO:teuthology.orchestra.run.smithi157.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-19T08:03:02.661 INFO:teuthology.orchestra.run.smithi022.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-09-19T08:03:02.667 INFO:teuthology.orchestra.run.smithi157.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-19T08:03:02.686 INFO:teuthology.orchestra.run.smithi022.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-19T08:03:02.692 INFO:teuthology.orchestra.run.smithi157.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-19T08:03:02.711 INFO:teuthology.orchestra.run.smithi022.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-09-19T08:03:02.726 INFO:teuthology.orchestra.run.smithi157.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-19T08:03:02.737 INFO:teuthology.orchestra.run.smithi022.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-19T08:03:02.751 INFO:teuthology.orchestra.run.smithi157.stdout:(147/156): re2-20211101-20.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-19T08:03:02.762 INFO:teuthology.orchestra.run.smithi022.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-19T08:03:02.787 INFO:teuthology.orchestra.run.smithi022.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T08:03:02.802 INFO:teuthology.orchestra.run.smithi157.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T08:03:02.812 INFO:teuthology.orchestra.run.smithi022.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T08:03:02.829 INFO:teuthology.orchestra.run.smithi022.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-19T08:03:02.846 INFO:teuthology.orchestra.run.smithi022.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-19T08:03:02.930 INFO:teuthology.orchestra.run.smithi022.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-09-19T08:03:02.989 INFO:teuthology.orchestra.run.smithi022.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T08:03:02.993 INFO:teuthology.orchestra.run.smithi157.stdout:(149/156): NetworkManager-1.51.0-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-09-19T08:03:03.060 INFO:teuthology.orchestra.run.smithi157.stdout:(150/156): NetworkManager-libnm-1.51.0-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-09-19T08:03:03.094 INFO:teuthology.orchestra.run.smithi157.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-09-19T08:03:03.136 INFO:teuthology.orchestra.run.smithi157.stdout:(152/156): NetworkManager-tui-1.51.0-1.el9.x86_ 5.8 MB/s | 249 kB 00:00 2024-09-19T08:03:03.147 INFO:teuthology.orchestra.run.smithi022.stdout:(105/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-19T08:03:03.195 INFO:teuthology.orchestra.run.smithi157.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 23 MB/s | 1.4 MB 00:00 2024-09-19T08:03:03.197 INFO:teuthology.orchestra.run.smithi022.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-09-19T08:03:03.231 INFO:teuthology.orchestra.run.smithi022.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 1.2 MB/s | 308 kB 00:00 2024-09-19T08:03:03.256 INFO:teuthology.orchestra.run.smithi022.stdout:(108/156): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-09-19T08:03:03.279 INFO:teuthology.orchestra.run.smithi157.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-09-19T08:03:03.282 INFO:teuthology.orchestra.run.smithi022.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 537 kB/s | 26 kB 00:00 2024-09-19T08:03:03.307 INFO:teuthology.orchestra.run.smithi022.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 967 kB/s | 49 kB 00:00 2024-09-19T08:03:03.332 INFO:teuthology.orchestra.run.smithi022.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-19T08:03:03.358 INFO:teuthology.orchestra.run.smithi022.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-19T08:03:03.408 INFO:teuthology.orchestra.run.smithi022.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-19T08:03:03.433 INFO:teuthology.orchestra.run.smithi022.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-09-19T08:03:03.458 INFO:teuthology.orchestra.run.smithi022.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 866 kB/s | 43 kB 00:00 2024-09-19T08:03:03.479 INFO:teuthology.orchestra.run.smithi157.stdout:(155/156): librbd1-19.1.1-361.g2f599a46.el9.x86 4.3 MB/s | 3.2 MB 00:00 2024-09-19T08:03:03.483 INFO:teuthology.orchestra.run.smithi022.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-09-19T08:03:03.517 INFO:teuthology.orchestra.run.smithi022.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 241 kB/s | 14 kB 00:00 2024-09-19T08:03:03.529 INFO:teuthology.orchestra.run.smithi157.stdout:(156/156): librados2-19.1.1-361.g2f599a46.el9.x 4.4 MB/s | 3.4 MB 00:00 2024-09-19T08:03:03.535 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-19T08:03:03.536 INFO:teuthology.orchestra.run.smithi157.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-19T08:03:03.551 INFO:teuthology.orchestra.run.smithi022.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-19T08:03:03.585 INFO:teuthology.orchestra.run.smithi022.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-19T08:03:03.600 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-19T08:03:03.627 INFO:teuthology.orchestra.run.smithi022.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-19T08:03:03.652 INFO:teuthology.orchestra.run.smithi022.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-19T08:03:03.686 INFO:teuthology.orchestra.run.smithi022.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-19T08:03:03.711 INFO:teuthology.orchestra.run.smithi022.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-19T08:03:03.736 INFO:teuthology.orchestra.run.smithi022.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-19T08:03:03.761 INFO:teuthology.orchestra.run.smithi022.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-19T08:03:03.766 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-19T08:03:03.766 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-19T08:03:03.885 INFO:teuthology.orchestra.run.smithi022.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 157 kB/s | 19 kB 00:00 2024-09-19T08:03:03.912 INFO:teuthology.orchestra.run.smithi022.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 750 kB/s | 20 kB 00:00 2024-09-19T08:03:03.963 INFO:teuthology.orchestra.run.smithi022.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.1 MB/s | 2.0 MB 00:00 2024-09-19T08:03:04.029 INFO:teuthology.orchestra.run.smithi022.stdout:(129/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-09-19T08:03:04.055 INFO:teuthology.orchestra.run.smithi022.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 504 kB/s | 46 kB 00:00 2024-09-19T08:03:04.080 INFO:teuthology.orchestra.run.smithi022.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-19T08:03:04.105 INFO:teuthology.orchestra.run.smithi022.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-19T08:03:04.130 INFO:teuthology.orchestra.run.smithi022.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 665 kB/s | 16 kB 00:00 2024-09-19T08:03:04.156 INFO:teuthology.orchestra.run.smithi022.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-19T08:03:04.181 INFO:teuthology.orchestra.run.smithi022.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-19T08:03:04.206 INFO:teuthology.orchestra.run.smithi022.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T08:03:04.240 INFO:teuthology.orchestra.run.smithi022.stdout:(137/156): python3-kubernetes-26.1.0-2.el9.noar 3.1 MB/s | 1.0 MB 00:00 2024-09-19T08:03:04.265 INFO:teuthology.orchestra.run.smithi022.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 369 kB/s | 31 kB 00:00 2024-09-19T08:03:04.290 INFO:teuthology.orchestra.run.smithi022.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-19T08:03:04.315 INFO:teuthology.orchestra.run.smithi022.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-19T08:03:04.341 INFO:teuthology.orchestra.run.smithi022.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-19T08:03:04.366 INFO:teuthology.orchestra.run.smithi022.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-19T08:03:04.391 INFO:teuthology.orchestra.run.smithi022.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 3.6 MB/s | 90 kB 00:00 2024-09-19T08:03:04.417 INFO:teuthology.orchestra.run.smithi022.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-19T08:03:04.433 INFO:teuthology.orchestra.run.smithi022.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 476 kB/s | 20 kB 00:00 2024-09-19T08:03:04.459 INFO:teuthology.orchestra.run.smithi022.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-19T08:03:04.484 INFO:teuthology.orchestra.run.smithi022.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-19T08:03:04.560 INFO:teuthology.orchestra.run.smithi022.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-19T08:03:04.739 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-19T08:03:04.802 INFO:teuthology.orchestra.run.smithi022.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-19T08:03:04.891 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-19T08:03:04.891 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-19T08:03:04.918 INFO:teuthology.orchestra.run.smithi022.stdout:(150/156): librados2-19.1.1-361.g2f599a46.el9.x 7.4 MB/s | 3.4 MB 00:00 2024-09-19T08:03:05.119 INFO:teuthology.orchestra.run.smithi022.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 205 kB/s | 41 kB 00:00 2024-09-19T08:03:05.369 INFO:teuthology.orchestra.run.smithi022.stdout:(152/156): NetworkManager-tui-1.51.0-1.el9.x86_ 996 kB/s | 249 kB 00:00 2024-09-19T08:03:05.702 INFO:teuthology.orchestra.run.smithi022.stdout:(153/156): NetworkManager-1.51.0-1.el9.x86_64.r 2.0 MB/s | 2.3 MB 00:01 2024-09-19T08:03:05.800 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-19T08:03:05.801 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-19T08:03:06.094 INFO:teuthology.orchestra.run.smithi022.stdout:(154/156): NetworkManager-libnm-1.51.0-1.el9.x8 1.4 MB/s | 1.8 MB 00:01 2024-09-19T08:03:06.488 INFO:teuthology.orchestra.run.smithi022.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.0 MB/s | 2.4 MB 00:00 2024-09-19T08:03:06.521 INFO:teuthology.orchestra.run.smithi022.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 1.2 MB/s | 1.4 MB 00:01 2024-09-19T08:03:06.528 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T08:03:06.528 INFO:teuthology.orchestra.run.smithi022.stdout:Total 14 MB/s | 226 MB 00:15 2024-09-19T08:03:06.696 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-19T08:03:06.697 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-19T08:03:07.770 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T08:03:07.939 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T08:03:07.939 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T08:03:10.240 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T08:03:10.241 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T08:03:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-19T08:03:12.628 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T08:03:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T08:03:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T08:03:12.754 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T08:03:12.779 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T08:03:12.823 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T08:03:12.925 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-19T08:03:13.032 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T08:03:13.071 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T08:03:13.132 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T08:03:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T08:03:13.175 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T08:03:13.200 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T08:03:13.242 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T08:03:13.280 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:03:13.396 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:03:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:03:13.515 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:03:13.544 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:03:13.564 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T08:03:13.705 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T08:03:13.767 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T08:03:13.785 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:03:13.794 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:03:13.891 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:03:13.920 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:03:13.966 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:03:13.988 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:03:14.010 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:03:14.191 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:03:14.207 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:03:14.219 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:03:14.310 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:03:14.335 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:03:14.349 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:03:14.469 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T08:03:14.506 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T08:03:14.562 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T08:03:14.569 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:03:14.590 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:03:14.637 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:03:14.685 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:03:14.707 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:03:14.770 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:03:14.792 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:03:14.812 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:03:14.849 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:03:14.952 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:03:14.993 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T08:03:15.086 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T08:03:15.113 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:03:15.126 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T08:03:15.171 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T08:03:15.249 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T08:03:15.262 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T08:03:15.313 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T08:03:15.349 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:03:15.352 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T08:03:15.396 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T08:03:15.429 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T08:03:15.433 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:03:15.456 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T08:03:15.529 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T08:03:15.580 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T08:03:15.609 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T08:03:15.653 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T08:03:15.660 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:03:15.769 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T08:03:15.779 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T08:03:15.805 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T08:03:15.849 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T08:03:15.873 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T08:03:15.877 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T08:03:15.909 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T08:03:15.944 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T08:03:15.957 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T08:03:16.004 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T08:03:16.017 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T08:03:16.078 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T08:03:16.084 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T08:03:16.118 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T08:03:16.162 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T08:03:16.193 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T08:03:16.222 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T08:03:16.233 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T08:03:16.292 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T08:03:16.292 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T08:03:16.344 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T08:03:16.376 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T08:03:16.410 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T08:03:16.497 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T08:03:16.529 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T08:03:16.572 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T08:03:16.601 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T08:03:16.671 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T08:03:16.716 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T08:03:16.778 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T08:03:16.790 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T08:03:16.875 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T08:03:16.875 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T08:03:16.933 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T08:03:16.945 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T08:03:16.956 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T08:03:16.989 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T08:03:16.993 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T08:03:17.032 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T08:03:17.070 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T08:03:17.113 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T08:03:17.212 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T08:03:17.252 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T08:03:17.274 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T08:03:17.276 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T08:03:17.339 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T08:03:17.340 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T08:03:17.397 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T08:03:17.481 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T08:03:17.517 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T08:03:17.524 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T08:03:17.560 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T08:03:17.597 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T08:03:17.609 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T08:03:17.635 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T08:03:17.643 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T08:03:17.669 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T08:03:17.685 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T08:03:17.687 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T08:03:17.716 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T08:03:17.729 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T08:03:17.757 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T08:03:17.786 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:03:17.876 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T08:03:17.910 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:03:17.914 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T08:03:17.921 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T08:03:17.937 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:03:17.948 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T08:03:17.977 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T08:03:18.029 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:03:18.032 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T08:03:18.059 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T08:03:18.066 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:03:18.097 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T08:03:18.104 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T08:03:18.137 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T08:03:18.145 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T08:03:18.162 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T08:03:18.187 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T08:03:18.189 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T08:03:18.237 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T08:03:18.238 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T08:03:18.266 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T08:03:18.272 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T08:03:18.297 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T08:03:18.315 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T08:03:18.324 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:03:18.357 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T08:03:18.430 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:03:18.458 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:03:18.495 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T08:03:18.522 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:03:18.542 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T08:03:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:03:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:03:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:03:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:03:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T08:03:19.110 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T08:03:19.110 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T08:03:19.110 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T08:03:19.110 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T08:03:19.110 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T08:03:19.110 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T08:03:19.110 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T08:03:19.110 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T08:03:19.111 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T08:03:19.111 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T08:03:19.111 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T08:03:19.111 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T08:03:19.111 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T08:03:19.111 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T08:03:19.111 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T08:03:19.111 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T08:03:19.112 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T08:03:19.112 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T08:03:19.112 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T08:03:19.112 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T08:03:19.112 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T08:03:19.112 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T08:03:19.112 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T08:03:19.112 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T08:03:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:03:19.145 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T08:03:19.185 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T08:03:19.216 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T08:03:19.218 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:03:19.225 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T08:03:19.263 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T08:03:19.284 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T08:03:19.297 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T08:03:19.330 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T08:03:19.372 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T08:03:19.440 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T08:03:19.444 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:03:19.552 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T08:03:19.649 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T08:03:19.681 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T08:03:19.698 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T08:03:19.717 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T08:03:19.792 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T08:03:19.852 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T08:03:19.891 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T08:03:19.935 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T08:03:19.968 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T08:03:19.995 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T08:03:20.068 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T08:03:20.080 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T08:03:20.088 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T08:03:20.119 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T08:03:20.128 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T08:03:20.148 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T08:03:20.162 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T08:03:20.168 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T08:03:20.221 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T08:03:20.269 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T08:03:20.272 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T08:03:20.296 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T08:03:20.326 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T08:03:20.335 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T08:03:20.336 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T08:03:20.371 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T08:03:20.380 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T08:03:20.408 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T08:03:20.473 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T08:03:20.483 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T08:03:20.523 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T08:03:20.531 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T08:03:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T08:03:20.713 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T08:03:20.745 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T08:03:20.758 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T08:03:20.819 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T08:03:20.824 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T08:03:21.058 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T08:03:21.108 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T08:03:21.141 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T08:03:21.239 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T08:03:21.313 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T08:03:21.357 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T08:03:21.391 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T08:03:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T08:03:21.474 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T08:03:21.481 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T08:03:21.508 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T08:03:21.528 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T08:03:21.552 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T08:03:21.590 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T08:03:21.633 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T08:03:21.642 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T08:03:21.669 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T08:03:21.700 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T08:03:21.777 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T08:03:21.805 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T08:03:21.868 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T08:03:21.925 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T08:03:21.999 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T08:03:22.036 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T08:03:22.072 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T08:03:22.120 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T08:03:22.154 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T08:03:22.198 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T08:03:22.341 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T08:03:22.416 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T08:03:22.491 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T08:03:22.533 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T08:03:22.539 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T08:03:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T08:03:22.682 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T08:03:22.725 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T08:03:22.766 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T08:03:22.799 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T08:03:22.823 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T08:03:22.916 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T08:03:22.977 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T08:03:23.051 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T08:03:23.057 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T08:03:23.084 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T08:03:23.085 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T08:03:23.118 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T08:03:23.121 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T08:03:23.155 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T08:03:23.194 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T08:03:23.243 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T08:03:23.289 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T08:03:23.323 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T08:03:23.356 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T08:03:23.399 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T08:03:23.433 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T08:03:23.444 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T08:03:23.497 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:03:23.523 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T08:03:23.572 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T08:03:23.589 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:03:23.603 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T08:03:23.619 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T08:03:23.656 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T08:03:23.711 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T08:03:23.772 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T08:03:23.782 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T08:03:23.862 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T08:03:23.890 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T08:03:23.923 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T08:03:24.164 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T08:03:24.192 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:03:24.233 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T08:03:24.264 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:03:24.286 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:03:24.294 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T08:03:24.367 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:03:24.399 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T08:03:24.436 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T08:03:24.555 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T08:03:24.601 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T08:03:24.635 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T08:03:24.742 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T08:03:24.821 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T08:03:24.871 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T08:03:24.915 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T08:03:24.939 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:03:24.971 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T08:03:25.006 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T08:03:25.010 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:03:25.049 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T08:03:25.196 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T08:03:25.228 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T08:03:25.308 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T08:03:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T08:03:26.123 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T08:03:26.704 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:03:26.740 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:03:27.273 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:03:27.311 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:03:27.314 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:03:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:03:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:03:27.447 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T08:03:27.448 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T08:03:27.448 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:03:27.480 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:03:27.481 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T08:03:27.557 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T08:03:27.586 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T08:03:27.618 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T08:03:27.856 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:03:27.899 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:03:27.953 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T08:03:27.990 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:03:27.990 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T08:03:27.990 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T08:03:27.990 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:03:28.006 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:03:28.025 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:03:28.097 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:03:28.136 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T08:03:28.173 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T08:03:28.681 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T08:03:28.709 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:03:28.781 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:03:28.810 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T08:03:31.221 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:03:31.257 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:03:31.844 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:03:31.885 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:03:31.979 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:03:31.980 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T08:03:31.980 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T08:03:31.980 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:03:32.020 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:03:39.668 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:03:39.668 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-09-19T08:03:39.668 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-09-19T08:03:39.668 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-09-19T08:03:39.668 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-09-19T08:03:39.669 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-09-19T08:03:39.669 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /root 2024-09-19T08:03:39.669 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-09-19T08:03:39.669 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:03:39.735 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:03:39.735 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-09-19T08:03:39.735 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-09-19T08:03:39.735 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-09-19T08:03:39.735 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-09-19T08:03:39.735 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-09-19T08:03:39.736 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /root 2024-09-19T08:03:39.736 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-09-19T08:03:39.736 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:03:39.852 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:03:39.871 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:03:39.871 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:39.871 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T08:03:39.871 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:03:39.917 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:03:39.933 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:03:39.933 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:39.933 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T08:03:39.933 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:03:41.363 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:03:41.391 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:03:41.391 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:41.391 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T08:03:41.391 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:03:41.391 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:03:41.391 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:03:41.462 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:03:41.488 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:03:41.488 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:41.488 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T08:03:41.488 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:03:41.488 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:03:41.489 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:03:41.622 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:03:41.646 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:03:41.646 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:41.646 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T08:03:41.646 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:03:41.647 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:03:41.647 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:03:41.720 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:03:41.744 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:03:41.745 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:41.745 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T08:03:41.745 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:03:41.745 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:03:41.745 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:03:42.081 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:03:42.105 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:03:42.105 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:42.105 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T08:03:42.105 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:03:42.105 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:03:42.105 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:03:42.205 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:03:42.211 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T08:03:42.227 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:03:42.227 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:42.228 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T08:03:42.228 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:03:42.228 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:03:42.228 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:03:42.268 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T08:03:42.332 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T08:03:42.344 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T08:03:42.402 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T08:03:42.467 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T08:03:42.518 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T08:03:42.573 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T08:03:42.652 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T08:03:42.674 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T08:03:42.723 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T08:03:42.749 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T08:03:42.805 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T08:03:42.897 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T08:03:42.903 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T08:03:42.949 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T08:03:42.981 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T08:03:43.008 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T08:03:43.048 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T08:03:43.098 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T08:03:43.129 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T08:03:43.157 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T08:03:43.496 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:03:43.519 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:03:43.523 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:03:43.547 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:03:43.555 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T08:03:43.571 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T08:03:43.617 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T08:03:43.628 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T08:03:43.655 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:03:43.661 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:03:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:03:44.217 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /sys 2024-09-19T08:03:44.217 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /proc 2024-09-19T08:03:44.217 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /mnt 2024-09-19T08:03:44.217 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /var/tmp 2024-09-19T08:03:44.217 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /home 2024-09-19T08:03:44.217 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /root 2024-09-19T08:03:44.217 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /tmp 2024-09-19T08:03:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:03:44.400 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:03:44.421 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:03:44.421 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:44.421 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T08:03:44.421 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:03:44.705 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:03:44.749 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:03:44.778 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:03:44.814 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:03:45.710 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:03:45.740 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:03:45.825 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:03:45.843 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:03:45.855 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:03:45.945 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:03:45.966 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:03:45.974 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:03:45.974 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:45.974 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T08:03:45.974 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:03:45.974 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:03:45.974 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:03:46.002 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T08:03:46.034 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:03:46.058 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:03:46.058 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:46.058 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T08:03:46.058 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:03:46.058 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:03:46.058 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:03:46.110 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:03:46.121 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:03:46.133 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T08:03:46.157 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:03:46.182 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:03:46.182 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:46.182 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T08:03:46.182 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:03:46.182 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:03:46.182 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:03:46.212 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:03:46.219 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T08:03:46.233 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:03:46.237 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:03:46.237 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:46.237 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T08:03:46.237 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:03:46.237 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:03:46.237 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:03:46.245 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:03:46.266 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T08:03:46.347 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T08:03:46.400 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T08:03:46.696 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:03:46.721 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:03:46.721 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:46.721 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T08:03:46.721 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:03:46.721 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:03:46.721 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:03:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T08:03:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T08:03:46.958 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T08:03:47.155 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T08:03:47.191 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T08:03:47.206 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T08:03:47.229 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:03:47.253 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:03:47.253 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:47.253 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T08:03:47.253 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:03:47.253 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:03:47.254 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:03:47.287 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T08:03:47.305 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:03:47.328 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:03:47.328 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:47.328 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T08:03:47.328 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:03:47.372 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:03:47.375 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T08:03:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T08:03:47.395 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:03:47.395 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:47.395 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T08:03:47.395 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:03:47.396 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:03:47.396 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:03:47.402 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:03:47.427 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:03:47.427 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:47.427 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T08:03:47.427 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:03:47.427 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:03:47.427 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:03:47.470 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:03:47.492 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:03:47.492 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:47.493 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T08:03:47.493 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:03:47.530 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T08:03:47.537 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:03:47.560 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:03:47.560 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:47.560 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T08:03:47.560 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:03:47.560 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:03:47.561 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:03:47.593 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T08:03:47.630 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T08:03:47.657 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T08:03:47.698 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:03:47.721 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:03:47.722 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:47.722 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T08:03:47.722 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:03:47.722 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:03:47.722 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:03:47.879 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:03:47.903 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:03:47.903 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:47.903 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T08:03:47.903 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:03:47.903 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:03:47.903 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:03:48.238 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:03:48.264 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:03:48.296 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T08:03:48.387 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T08:03:48.412 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:03:49.524 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:03:49.572 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:03:50.586 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:03:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:03:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:03:50.917 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T08:03:50.941 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:03:50.967 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:03:50.967 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:50.967 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T08:03:50.967 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:03:50.967 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:03:50.967 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:03:51.017 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:03:51.028 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:03:51.133 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T08:03:51.189 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T08:03:52.168 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T08:03:52.194 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:03:52.219 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:03:52.219 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:52.219 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T08:03:52.220 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:03:52.220 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:03:52.220 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:03:52.260 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T08:03:52.261 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:03:52.286 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:03:52.286 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:52.286 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T08:03:52.286 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:03:52.301 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T08:03:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:03:52.342 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T08:03:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:03:52.357 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:52.357 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T08:03:52.357 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:03:52.357 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:03:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:03:52.374 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T08:03:52.401 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T08:03:52.585 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T08:03:52.631 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T08:03:52.669 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T08:03:52.679 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:03:52.699 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T08:03:52.704 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:03:52.704 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:03:52.704 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T08:03:52.705 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:03:52.705 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:03:52.705 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:03:52.709 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T08:03:52.737 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T08:03:52.750 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T08:03:52.792 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T08:03:52.823 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T08:03:52.858 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T08:03:53.063 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T08:03:53.113 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T08:03:53.152 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T08:03:53.182 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T08:03:53.219 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T08:03:53.343 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:03:53.385 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:03:53.406 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:03:53.854 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:03:53.892 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:03:53.913 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:03:54.265 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T08:03:54.356 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T08:03:54.398 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T08:03:54.478 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T08:03:54.509 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T08:03:54.535 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T08:03:54.560 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T08:03:54.561 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:03:54.645 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:03:54.645 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:03:54.746 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:03:54.770 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T08:03:54.776 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:03:54.776 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:03:54.837 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T08:03:54.922 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T08:03:54.955 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:03:54.955 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:03:54.964 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T08:03:55.040 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:03:55.049 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T08:03:55.083 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T08:03:55.104 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:03:55.117 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T08:03:55.143 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T08:03:55.143 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:03:55.211 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:03:55.211 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:03:55.295 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:03:55.319 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T08:03:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:03:55.325 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:03:55.505 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:03:55.505 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:03:55.589 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:03:55.664 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:03:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T08:03:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T08:03:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T08:03:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T08:03:57.691 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T08:03:57.913 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T08:03:57.963 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T08:03:57.993 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T08:03:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T08:03:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T08:03:58.636 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:03:58.674 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:03:58.697 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:03:59.598 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T08:03:59.688 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T08:03:59.745 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T08:03:59.849 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T08:03:59.877 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T08:03:59.910 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T08:03:59.949 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T08:03:59.949 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:04:00.027 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:04:00.027 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:04:00.114 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:04:00.135 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T08:04:00.141 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:04:00.141 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:04:00.323 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:04:00.323 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:04:00.405 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:04:00.481 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:04:00.684 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:04:00.684 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T08:04:00.684 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T08:04:00.684 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T08:04:00.684 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T08:04:00.684 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T08:04:00.685 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T08:04:00.685 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T08:04:00.685 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:04:00.685 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:04:00.685 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:04:00.685 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T08:04:00.685 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T08:04:00.685 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:04:00.685 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T08:04:00.685 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T08:04:00.686 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T08:04:00.686 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T08:04:00.686 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T08:04:00.686 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T08:04:00.686 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T08:04:00.686 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T08:04:00.687 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T08:04:00.687 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T08:04:00.687 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T08:04:00.687 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T08:04:00.687 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T08:04:00.687 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T08:04:00.687 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T08:04:00.687 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T08:04:00.687 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T08:04:00.687 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T08:04:00.688 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T08:04:00.688 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T08:04:00.688 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T08:04:00.688 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T08:04:00.688 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T08:04:00.689 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T08:04:00.689 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T08:04:00.689 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T08:04:00.689 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T08:04:00.689 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T08:04:00.689 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T08:04:00.689 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T08:04:00.689 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T08:04:00.690 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T08:04:00.690 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T08:04:00.690 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T08:04:00.690 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T08:04:00.690 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T08:04:00.690 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T08:04:00.690 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T08:04:00.690 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T08:04:00.690 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T08:04:00.690 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T08:04:00.691 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T08:04:00.691 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T08:04:00.691 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T08:04:00.691 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T08:04:00.691 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T08:04:00.691 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T08:04:00.691 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T08:04:00.691 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T08:04:00.691 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T08:04:00.692 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T08:04:00.692 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T08:04:00.692 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T08:04:00.692 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T08:04:00.692 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T08:04:00.692 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T08:04:00.692 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T08:04:00.692 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T08:04:00.692 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T08:04:00.693 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T08:04:00.693 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T08:04:00.693 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T08:04:00.693 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T08:04:00.693 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T08:04:00.693 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T08:04:00.693 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T08:04:00.693 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T08:04:00.694 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T08:04:00.694 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T08:04:00.694 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T08:04:00.694 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T08:04:00.694 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T08:04:00.694 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T08:04:00.694 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T08:04:00.695 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T08:04:00.695 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T08:04:00.695 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T08:04:00.695 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T08:04:00.695 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T08:04:00.695 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T08:04:00.695 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T08:04:00.696 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T08:04:00.696 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T08:04:00.696 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T08:04:00.696 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T08:04:00.696 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T08:04:00.696 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T08:04:00.697 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T08:04:00.697 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T08:04:00.697 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T08:04:00.697 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T08:04:00.697 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T08:04:00.697 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T08:04:00.697 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T08:04:00.697 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T08:04:00.697 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T08:04:00.698 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T08:04:00.698 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T08:04:00.698 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T08:04:00.698 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T08:04:00.698 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T08:04:00.698 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T08:04:00.698 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T08:04:00.698 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T08:04:00.699 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T08:04:00.699 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T08:04:00.699 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T08:04:00.699 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T08:04:00.699 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T08:04:00.699 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T08:04:00.699 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T08:04:00.700 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T08:04:00.700 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T08:04:00.700 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T08:04:00.700 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T08:04:00.700 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T08:04:00.700 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T08:04:00.700 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T08:04:00.700 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T08:04:00.700 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T08:04:00.700 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T08:04:00.700 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T08:04:00.700 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T08:04:00.700 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T08:04:00.701 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T08:04:00.701 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T08:04:00.701 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T08:04:00.701 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T08:04:00.701 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T08:04:00.701 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T08:04:00.701 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T08:04:00.701 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T08:04:00.701 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T08:04:00.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T08:04:00.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T08:04:00.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T08:04:00.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T08:04:00.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T08:04:00.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T08:04:00.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T08:04:00.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T08:04:00.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T08:04:00.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T08:04:00.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T08:04:00.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T08:04:00.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T08:04:00.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T08:04:00.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T08:04:00.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T08:04:00.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T08:04:01.214 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:04:01.215 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T08:04:01.215 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T08:04:01.215 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T08:04:01.215 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T08:04:01.215 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T08:04:01.215 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T08:04:01.215 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T08:04:01.215 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:04:01.215 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:04:01.215 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:04:01.216 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T08:04:01.216 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T08:04:01.216 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:04:01.216 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T08:04:01.216 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T08:04:01.216 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T08:04:01.216 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T08:04:01.216 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T08:04:01.216 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T08:04:01.216 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T08:04:01.216 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T08:04:01.216 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T08:04:01.217 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T08:04:01.217 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T08:04:01.217 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T08:04:01.217 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T08:04:01.217 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T08:04:01.217 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T08:04:01.217 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T08:04:01.217 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T08:04:01.217 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T08:04:01.218 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T08:04:01.218 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T08:04:01.218 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T08:04:01.218 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T08:04:01.218 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T08:04:01.218 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T08:04:01.218 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T08:04:01.218 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T08:04:01.218 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T08:04:01.218 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T08:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T08:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T08:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T08:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T08:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T08:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T08:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T08:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T08:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T08:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T08:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T08:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T08:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T08:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T08:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T08:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T08:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T08:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T08:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T08:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T08:04:01.221 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T08:04:01.221 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T08:04:01.221 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T08:04:01.221 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T08:04:01.221 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T08:04:01.221 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T08:04:01.221 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T08:04:01.221 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T08:04:01.221 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T08:04:01.221 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T08:04:01.221 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T08:04:01.222 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T08:04:01.222 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T08:04:01.222 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T08:04:01.222 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T08:04:01.222 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T08:04:01.222 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T08:04:01.222 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T08:04:01.222 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T08:04:01.223 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T08:04:01.223 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T08:04:01.223 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T08:04:01.223 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T08:04:01.223 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T08:04:01.223 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T08:04:01.223 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T08:04:01.223 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T08:04:01.223 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T08:04:01.223 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T08:04:01.223 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T08:04:01.224 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T08:04:01.224 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T08:04:01.224 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T08:04:01.224 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T08:04:01.224 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T08:04:01.224 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T08:04:01.224 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T08:04:01.224 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T08:04:01.224 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T08:04:01.225 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T08:04:01.225 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T08:04:01.225 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T08:04:01.225 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T08:04:01.225 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T08:04:01.225 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T08:04:01.225 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T08:04:01.225 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T08:04:01.226 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T08:04:01.226 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T08:04:01.226 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T08:04:01.226 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T08:04:01.226 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T08:04:01.226 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T08:04:01.226 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T08:04:01.226 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T08:04:01.227 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T08:04:01.227 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T08:04:01.227 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T08:04:01.227 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T08:04:01.227 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T08:04:01.227 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T08:04:01.227 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T08:04:01.228 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T08:04:01.228 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T08:04:01.228 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T08:04:01.228 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T08:04:01.228 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T08:04:01.228 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T08:04:01.228 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T08:04:01.228 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T08:04:01.228 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T08:04:01.229 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T08:04:01.229 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T08:04:01.229 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T08:04:01.229 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T08:04:01.229 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T08:04:01.229 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T08:04:01.229 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T08:04:01.229 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T08:04:01.229 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T08:04:01.229 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T08:04:01.229 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T08:04:01.229 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T08:04:01.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T08:04:01.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T08:04:01.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T08:04:01.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T08:04:01.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T08:04:01.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T08:04:01.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T08:04:01.230 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T08:04:01.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T08:04:01.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T08:04:01.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T08:04:01.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T08:04:01.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T08:04:01.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T08:04:01.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T08:04:01.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T08:04:01.231 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T08:04:01.232 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T08:04:01.232 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T08:04:03.328 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:04:03.328 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:04:03.328 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-09-19T08:04:03.329 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T08:04:03.329 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T08:04:03.329 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T08:04:03.329 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T08:04:03.329 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.329 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.329 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T08:04:03.329 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T08:04:03.329 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-19T08:04:03.329 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T08:04:03.329 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T08:04:03.329 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T08:04:03.329 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T08:04:03.330 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.330 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.330 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.330 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.330 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:03.330 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.330 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.330 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.330 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:03.330 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:03.330 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:03.330 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:03.331 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:03.331 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.331 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.331 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:03.331 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.331 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.331 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.331 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:03.331 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:03.331 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.331 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.331 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:03.332 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T08:04:03.332 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T08:04:03.332 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T08:04:03.332 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T08:04:03.332 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T08:04:03.332 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T08:04:03.332 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T08:04:03.332 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T08:04:03.333 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T08:04:03.333 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T08:04:03.333 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T08:04:03.333 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.333 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.333 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.333 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T08:04:03.333 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T08:04:03.333 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T08:04:03.334 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T08:04:03.334 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T08:04:03.334 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.334 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.334 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T08:04:03.334 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.334 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T08:04:03.334 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T08:04:03.334 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T08:04:03.334 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T08:04:03.334 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T08:04:03.334 INFO:teuthology.orchestra.run.smithi157.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T08:04:03.334 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T08:04:03.335 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T08:04:03.335 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T08:04:03.335 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T08:04:03.335 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T08:04:03.335 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T08:04:03.335 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T08:04:03.335 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T08:04:03.335 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T08:04:03.335 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T08:04:03.335 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T08:04:03.335 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T08:04:03.336 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T08:04:03.336 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T08:04:03.336 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T08:04:03.336 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T08:04:03.336 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T08:04:03.336 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.336 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.336 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.336 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T08:04:03.336 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T08:04:03.336 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T08:04:03.337 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T08:04:03.337 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T08:04:03.337 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T08:04:03.337 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T08:04:03.337 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T08:04:03.337 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T08:04:03.337 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T08:04:03.337 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T08:04:03.337 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T08:04:03.337 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T08:04:03.337 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T08:04:03.337 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T08:04:03.338 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T08:04:03.338 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T08:04:03.338 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T08:04:03.338 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T08:04:03.338 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T08:04:03.338 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T08:04:03.338 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T08:04:03.338 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T08:04:03.338 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T08:04:03.338 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T08:04:03.339 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T08:04:03.339 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T08:04:03.339 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T08:04:03.339 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T08:04:03.339 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T08:04:03.339 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T08:04:03.339 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T08:04:03.339 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T08:04:03.339 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T08:04:03.339 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T08:04:03.339 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T08:04:03.339 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T08:04:03.339 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T08:04:03.340 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T08:04:03.340 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T08:04:03.340 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.340 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.340 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T08:04:03.340 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T08:04:03.340 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T08:04:03.340 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.340 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T08:04:03.340 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T08:04:03.340 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T08:04:03.340 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T08:04:03.341 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T08:04:03.341 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T08:04:03.341 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T08:04:03.341 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T08:04:03.341 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T08:04:03.341 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T08:04:03.341 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T08:04:03.341 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T08:04:03.341 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T08:04:03.341 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T08:04:03.341 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T08:04:03.341 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.342 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.342 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:03.342 INFO:teuthology.orchestra.run.smithi157.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T08:04:03.342 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T08:04:03.342 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T08:04:03.342 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T08:04:03.342 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T08:04:03.342 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T08:04:03.343 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T08:04:03.343 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:04:03.343 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-19T08:04:03.790 DEBUG:teuthology.parallel:result is None 2024-09-19T08:04:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:04:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:04:04.074 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-09-19T08:04:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T08:04:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T08:04:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T08:04:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T08:04:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T08:04:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T08:04:04.075 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-19T08:04:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T08:04:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T08:04:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T08:04:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T08:04:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T08:04:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T08:04:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T08:04:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T08:04:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T08:04:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T08:04:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T08:04:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T08:04:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T08:04:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T08:04:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T08:04:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T08:04:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T08:04:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T08:04:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T08:04:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T08:04:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T08:04:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T08:04:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T08:04:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T08:04:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T08:04:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T08:04:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T08:04:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T08:04:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T08:04:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T08:04:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T08:04:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T08:04:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T08:04:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T08:04:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T08:04:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T08:04:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T08:04:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T08:04:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T08:04:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T08:04:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T08:04:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T08:04:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T08:04:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T08:04:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T08:04:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T08:04:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T08:04:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T08:04:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T08:04:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T08:04:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T08:04:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T08:04:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T08:04:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T08:04:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T08:04:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T08:04:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T08:04:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T08:04:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T08:04:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T08:04:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T08:04:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T08:04:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T08:04:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T08:04:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T08:04:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T08:04:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T08:04:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T08:04:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T08:04:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T08:04:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T08:04:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T08:04:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T08:04:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T08:04:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T08:04:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T08:04:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T08:04:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T08:04:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T08:04:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T08:04:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T08:04:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T08:04:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T08:04:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T08:04:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T08:04:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T08:04:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T08:04:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T08:04:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T08:04:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T08:04:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T08:04:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T08:04:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T08:04:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T08:04:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T08:04:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T08:04:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T08:04:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T08:04:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T08:04:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T08:04:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T08:04:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T08:04:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T08:04:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T08:04:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T08:04:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T08:04:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T08:04:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T08:04:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T08:04:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:04:04.088 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-19T08:04:04.577 DEBUG:teuthology.parallel:result is None 2024-09-19T08:04:05.912 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:04:05.912 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T08:04:05.912 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T08:04:05.912 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T08:04:05.913 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T08:04:05.913 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T08:04:05.913 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T08:04:05.913 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T08:04:05.913 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:04:05.913 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:04:05.913 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:04:05.913 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T08:04:05.913 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T08:04:05.913 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:04:05.913 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T08:04:05.913 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T08:04:05.913 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T08:04:05.914 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T08:04:05.914 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T08:04:05.914 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T08:04:05.914 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T08:04:05.914 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T08:04:05.914 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T08:04:05.914 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T08:04:05.914 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T08:04:05.914 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T08:04:05.914 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T08:04:05.914 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T08:04:05.914 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T08:04:05.914 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T08:04:05.927 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T08:04:05.927 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T08:04:05.927 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T08:04:05.927 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T08:04:05.927 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T08:04:05.927 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T08:04:05.927 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T08:04:05.927 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T08:04:05.927 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T08:04:05.927 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T08:04:05.927 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T08:04:05.928 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T08:04:05.928 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T08:04:05.928 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T08:04:05.928 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T08:04:05.928 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T08:04:05.928 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T08:04:05.928 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T08:04:05.928 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T08:04:05.928 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T08:04:05.928 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T08:04:05.928 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T08:04:05.929 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T08:04:05.929 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T08:04:05.929 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T08:04:05.929 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T08:04:05.929 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T08:04:05.929 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T08:04:05.929 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T08:04:05.929 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T08:04:05.929 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T08:04:05.929 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T08:04:05.929 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T08:04:05.929 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T08:04:05.930 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T08:04:05.930 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T08:04:05.930 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T08:04:05.930 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T08:04:05.930 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T08:04:05.930 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T08:04:05.930 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T08:04:05.930 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T08:04:05.930 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T08:04:05.930 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T08:04:05.930 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T08:04:05.930 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T08:04:05.930 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T08:04:05.931 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T08:04:05.931 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T08:04:05.931 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T08:04:05.931 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T08:04:05.931 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T08:04:05.931 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T08:04:05.931 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T08:04:05.931 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T08:04:05.931 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T08:04:05.931 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T08:04:05.931 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T08:04:05.931 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T08:04:05.932 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T08:04:05.932 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T08:04:05.932 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T08:04:05.932 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T08:04:05.932 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T08:04:05.932 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T08:04:05.932 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T08:04:05.932 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T08:04:05.932 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T08:04:05.932 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T08:04:05.932 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T08:04:05.933 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T08:04:05.933 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T08:04:05.933 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T08:04:05.933 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T08:04:05.933 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T08:04:05.933 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T08:04:05.933 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T08:04:05.933 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T08:04:05.933 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T08:04:05.933 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T08:04:05.933 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T08:04:05.933 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T08:04:05.934 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T08:04:05.934 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T08:04:05.934 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T08:04:05.934 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T08:04:05.934 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T08:04:05.934 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T08:04:05.934 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T08:04:05.934 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T08:04:05.934 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T08:04:05.934 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T08:04:05.934 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T08:04:05.934 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T08:04:05.935 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T08:04:05.935 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T08:04:05.935 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T08:04:05.935 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T08:04:05.935 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T08:04:05.935 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T08:04:05.935 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T08:04:05.935 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T08:04:05.935 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T08:04:05.935 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T08:04:05.935 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T08:04:05.935 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T08:04:05.935 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T08:04:05.936 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T08:04:05.936 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T08:04:05.936 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T08:04:05.936 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T08:04:05.936 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T08:04:05.936 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T08:04:05.936 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T08:04:05.936 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T08:04:05.936 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T08:04:05.936 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T08:04:05.936 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T08:04:05.936 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T08:04:05.937 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T08:04:05.937 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T08:04:05.937 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T08:04:05.937 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T08:04:05.937 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T08:04:05.937 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T08:04:05.937 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T08:04:05.937 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T08:04:05.937 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T08:04:05.937 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T08:04:05.937 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T08:04:05.938 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T08:04:05.938 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T08:04:05.938 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T08:04:05.938 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T08:04:08.749 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:04:08.749 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:04:08.749 INFO:teuthology.orchestra.run.smithi022.stdout:Upgraded: 2024-09-19T08:04:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T08:04:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T08:04:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T08:04:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T08:04:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T08:04:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T08:04:08.750 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T08:04:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T08:04:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T08:04:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T08:04:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T08:04:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:04:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T08:04:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T08:04:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T08:04:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T08:04:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T08:04:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T08:04:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T08:04:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T08:04:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T08:04:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T08:04:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T08:04:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T08:04:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T08:04:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T08:04:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T08:04:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T08:04:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T08:04:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T08:04:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T08:04:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T08:04:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T08:04:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T08:04:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T08:04:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T08:04:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T08:04:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T08:04:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T08:04:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T08:04:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T08:04:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T08:04:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T08:04:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T08:04:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T08:04:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T08:04:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T08:04:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T08:04:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T08:04:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T08:04:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T08:04:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T08:04:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T08:04:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T08:04:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T08:04:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T08:04:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T08:04:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T08:04:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T08:04:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T08:04:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T08:04:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T08:04:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T08:04:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T08:04:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T08:04:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T08:04:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T08:04:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T08:04:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T08:04:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T08:04:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T08:04:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T08:04:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T08:04:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T08:04:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T08:04:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T08:04:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T08:04:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T08:04:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T08:04:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T08:04:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T08:04:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T08:04:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T08:04:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T08:04:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T08:04:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T08:04:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T08:04:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T08:04:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T08:04:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T08:04:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T08:04:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T08:04:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T08:04:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T08:04:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T08:04:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T08:04:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T08:04:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T08:04:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T08:04:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T08:04:08.762 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T08:04:08.762 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T08:04:08.762 INFO:teuthology.orchestra.run.smithi022.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T08:04:08.762 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T08:04:08.762 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T08:04:08.762 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T08:04:08.762 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T08:04:08.762 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T08:04:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T08:04:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T08:04:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:04:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T08:04:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T08:04:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T08:04:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T08:04:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T08:04:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T08:04:08.764 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T08:04:08.764 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:04:08.764 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T08:04:09.281 DEBUG:teuthology.parallel:result is None 2024-09-19T08:04:09.281 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:04:09.421 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T08:04:09.445 INFO:teuthology.orchestra.run.smithi022.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T08:04:09.445 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T08:04:09.445 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T08:04:09.447 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:04:09.582 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T08:04:09.608 INFO:teuthology.orchestra.run.smithi064.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T08:04:09.608 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T08:04:09.608 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T08:04:09.610 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:04:09.742 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T08:04:09.769 INFO:teuthology.orchestra.run.smithi157.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T08:04:09.769 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T08:04:09.769 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T08:04:09.771 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T08:04:09.771 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:04:09.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T08:04:09.803 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T08:04:09.803 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T08:04:09.835 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T08:04:09.835 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T08:04:09.867 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T08:04:09.868 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:04:09.868 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T08:04:09.897 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T08:04:09.965 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T08:04:09.966 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T08:04:09.995 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T08:04:10.064 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T08:04:10.064 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T08:04:10.093 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T08:04:10.162 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T08:04:10.162 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:04:10.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T08:04:10.193 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T08:04:10.263 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T08:04:10.263 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T08:04:10.293 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T08:04:10.361 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T08:04:10.361 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T08:04:10.389 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T08:04:10.455 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T08:04:10.455 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:04:10.456 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T08:04:10.486 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T08:04:10.554 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T08:04:10.555 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T08:04:10.582 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T08:04:10.649 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T08:04:10.649 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T08:04:10.676 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T08:04:10.743 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T08:04:10.839 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2}} 2024-09-19T08:04:10.839 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:04:10.840 INFO:tasks.cephadm:Cluster fsid is c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:04:10.840 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T08:04:10.840 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.22', 'mon.b': '172.21.15.64', 'mon.c': '172.21.15.157'} 2024-09-19T08:04:10.840 INFO:tasks.cephadm:First mon is mon.a on smithi022 2024-09-19T08:04:10.840 INFO:tasks.cephadm:First mgr is x 2024-09-19T08:04:10.840 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T08:04:10.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname $(hostname -s) 2024-09-19T08:04:10.872 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2024-09-19T08:04:10.901 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname $(hostname -s) 2024-09-19T08:04:10.928 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T08:04:10.929 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:04:11.070 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T08:04:11.205 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:04:11.207 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T08:04:11.208 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T08:04:11.208 DEBUG:teuthology.orchestra.run.smithi022:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T08:04:11.672 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 08:04 /home/ubuntu/cephtest/cephadm 2024-09-19T08:04:11.673 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T08:04:12.050 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 08:04 /home/ubuntu/cephtest/cephadm 2024-09-19T08:04:12.050 DEBUG:teuthology.orchestra.run.smithi157:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T08:04:12.432 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 08:04 /home/ubuntu/cephtest/cephadm 2024-09-19T08:04:12.433 DEBUG:teuthology.orchestra.run.smithi022:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T08:04:12.454 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T08:04:12.474 DEBUG:teuthology.orchestra.run.smithi157:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T08:04:12.510 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T08:04:12.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T08:04:12.513 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T08:04:12.517 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T08:04:14.011 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:04:14.136 INFO:teuthology.orchestra.run.smithi022.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:04:14.194 INFO:teuthology.orchestra.run.smithi157.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:04:57.756 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:04:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:04:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T08:04:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "repo_digests": [ 2024-09-19T08:04:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T08:04:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-19T08:04:57.757 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:04:59.732 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T08:04:59.732 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:04:59.732 INFO:teuthology.orchestra.run.smithi064.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T08:04:59.732 INFO:teuthology.orchestra.run.smithi064.stdout: "repo_digests": [ 2024-09-19T08:04:59.733 INFO:teuthology.orchestra.run.smithi064.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T08:04:59.733 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-19T08:04:59.733 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T08:04:59.932 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-09-19T08:04:59.933 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:04:59.933 INFO:teuthology.orchestra.run.smithi157.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T08:04:59.933 INFO:teuthology.orchestra.run.smithi157.stdout: "repo_digests": [ 2024-09-19T08:04:59.933 INFO:teuthology.orchestra.run.smithi157.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T08:04:59.933 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2024-09-19T08:04:59.933 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-09-19T08:04:59.957 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph 2024-09-19T08:04:59.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2024-09-19T08:05:00.022 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph 2024-09-19T08:05:00.049 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /etc/ceph 2024-09-19T08:05:00.079 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2024-09-19T08:05:00.105 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /etc/ceph 2024-09-19T08:05:00.130 INFO:tasks.cephadm:Writing seed config... 2024-09-19T08:05:00.131 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T08:05:00.131 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T08:05:00.131 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T08:05:00.132 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T08:05:00.132 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T08:05:00.132 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-19T08:05:00.132 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T08:05:00.132 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T08:05:00.132 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T08:05:00.132 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T08:05:00.132 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T08:05:00.132 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T08:05:00.132 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T08:05:00.133 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T08:05:00.133 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T08:05:00.133 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T08:05:00.133 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T08:05:00.133 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-19T08:05:00.133 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T08:05:00.133 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T08:05:00.133 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T08:05:00.133 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T08:05:00.133 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-19T08:05:00.133 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T08:05:00.133 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T08:05:00.133 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T08:05:00.133 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T08:05:00.134 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T08:05:00.134 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T08:05:00.134 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T08:05:00.134 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T08:05:00.134 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T08:05:00.134 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T08:05:00.134 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-19T08:05:00.134 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T08:05:00.134 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-19T08:05:00.134 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-19T08:05:00.134 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T08:05:00.134 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T08:05:00.134 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-19T08:05:00.134 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T08:05:00.135 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T08:05:00.135 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T08:05:00.135 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T08:05:00.135 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T08:05:00.135 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T08:05:00.135 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T08:05:00.135 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-19T08:05:00.136 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:05:00.136 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T08:05:00.154 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c0ad60c6-765d-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T08:05:00.155 DEBUG:teuthology.orchestra.run.smithi022:mon.a> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T08:05:00.197 DEBUG:teuthology.orchestra.run.smithi022:mgr.x> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T08:05:00.241 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T08:05:00.241 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.22 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:05:00.456 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T08:05:00.456 INFO:teuthology.orchestra.run.smithi022.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', 'c0ad60c6-765d-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.22', '--skip-admin-label'] 2024-09-19T08:05:00.457 INFO:teuthology.orchestra.run.smithi022.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T08:05:00.457 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying podman|docker is present... 2024-09-19T08:05:00.500 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 5.2.2 2024-09-19T08:05:00.500 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying lvm2 is present... 2024-09-19T08:05:00.500 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying time synchronization is in place... 2024-09-19T08:05:00.508 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T08:05:00.508 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T08:05:00.514 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T08:05:00.514 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-09-19T08:05:00.521 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2024-09-19T08:05:00.527 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2024-09-19T08:05:00.528 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2024-09-19T08:05:00.528 INFO:teuthology.orchestra.run.smithi022.stdout:Repeating the final host check... 2024-09-19T08:05:00.551 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 5.2.2 2024-09-19T08:05:00.551 INFO:teuthology.orchestra.run.smithi022.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T08:05:00.551 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl is present 2024-09-19T08:05:00.551 INFO:teuthology.orchestra.run.smithi022.stdout:lvcreate is present 2024-09-19T08:05:00.558 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T08:05:00.558 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T08:05:00.564 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T08:05:00.564 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-09-19T08:05:00.571 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2024-09-19T08:05:00.577 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2024-09-19T08:05:00.578 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2024-09-19T08:05:00.578 INFO:teuthology.orchestra.run.smithi022.stdout:Host looks OK 2024-09-19T08:05:00.578 INFO:teuthology.orchestra.run.smithi022.stdout:Cluster fsid: c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:05:00.578 INFO:teuthology.orchestra.run.smithi022.stdout:Acquiring lock 139764757091328 on /run/cephadm/c0ad60c6-765d-11ef-bceb-c7b262605968.lock 2024-09-19T08:05:00.578 INFO:teuthology.orchestra.run.smithi022.stdout:Lock 139764757091328 acquired on /run/cephadm/c0ad60c6-765d-11ef-bceb-c7b262605968.lock 2024-09-19T08:05:00.578 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 3300 ... 2024-09-19T08:05:00.579 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 6789 ... 2024-09-19T08:05:00.579 INFO:teuthology.orchestra.run.smithi022.stdout:Base mon IP(s) is [172.21.15.22:3300, 172.21.15.22:6789], mon addrv is [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2024-09-19T08:05:00.582 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.22 metric 100 2024-09-19T08:05:00.582 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.22 metric 100 2024-09-19T08:05:00.585 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T08:05:00.585 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-19T08:05:00.585 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1765sec hoplimit 64 pref medium 2024-09-19T08:05:00.587 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T08:05:00.587 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T08:05:00.587 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T08:05:00.588 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-19T08:05:00.588 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-19T08:05:00.588 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T08:05:00.588 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2024-09-19T08:05:00.589 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2024-09-19T08:05:00.589 INFO:teuthology.orchestra.run.smithi022.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T08:05:00.590 INFO:teuthology.orchestra.run.smithi022.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T08:05:00.590 INFO:teuthology.orchestra.run.smithi022.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:05:01.075 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T08:05:01.075 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:05:01.075 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T08:05:01.075 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T08:05:01.075 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T08:05:01.075 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T08:05:01.075 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T08:05:02.013 INFO:teuthology.orchestra.run.smithi022.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T08:05:02.013 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T08:05:02.013 INFO:teuthology.orchestra.run.smithi022.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T08:05:02.927 INFO:teuthology.orchestra.run.smithi022.stdout:stat: stdout 167 167 2024-09-19T08:05:02.927 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial keys... 2024-09-19T08:05:03.999 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQAv2+tmR/2WFxAAtux3D9vXCFe+TXr9ZTFCag== 2024-09-19T08:05:04.871 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQAw2+tmqCXbEhAA9OAYtSGSeXGuRtPWG9n6PA== 2024-09-19T08:05:12.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQAx2+tmjJSjLBAAtq/qxbjbXLBkdiPLhd9gYQ== 2024-09-19T08:05:12.260 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial monmap... 2024-09-19T08:05:16.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T08:05:16.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T08:05:16.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:05:16.820 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T08:05:16.821 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool for a [v2:172.21.15.22:3300,v1:172.21.15.22:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T08:05:16.821 INFO:teuthology.orchestra.run.smithi022.stdout:setting min_mon_release = quincy 2024-09-19T08:05:16.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: set fsid to c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:05:16.821 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T08:05:16.821 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:05:16.821 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mon... 2024-09-19T08:05:17.983 INFO:teuthology.orchestra.run.smithi022.stdout:create mon.a on 2024-09-19T08:05:18.206 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T08:05:18.366 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T08:05:18.622 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c0ad60c6-765d-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-c0ad60c6-765d-11ef-bceb-c7b262605968.target. 2024-09-19T08:05:18.623 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c0ad60c6-765d-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-c0ad60c6-765d-11ef-bceb-c7b262605968.target. 2024-09-19T08:05:18.873 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.a 2024-09-19T08:05:18.873 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T08:05:19.035 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c0ad60c6-765d-11ef-bceb-c7b262605968.target.wants/ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@.service. 2024-09-19T08:05:19.517 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T08:05:19.517 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-09-19T08:05:19.524 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T08:05:19.524 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-09-19T08:05:19.524 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-09-19T08:05:19.524 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T08:05:19.525 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon to start... 2024-09-19T08:05:19.525 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon... 2024-09-19T08:05:19.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:19 smithi022 systemd[1]: Started Ceph mon.a for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:05:20.544 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T08:05:20.544 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout id: c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:05:20.544 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T08:05:20.544 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-19T08:05:20.544 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout services: 2024-09-19T08:05:20.544 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.530693s) 2024-09-19T08:05:20.544 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T08:05:20.544 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T08:05:20.545 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-19T08:05:20.545 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout data: 2024-09-19T08:05:20.545 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T08:05:20.545 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T08:05:20.545 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T08:05:20.545 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T08:05:20.545 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-19T08:05:20.545 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.934+0000 7f8d11da1640 1 Processor -- start 2024-09-19T08:05:20.545 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.934+0000 7f8d11da1640 1 -- start start 2024-09-19T08:05:20.545 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.935+0000 7f8d11da1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c1063f0 0x7f8d0c1067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:20.546 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.935+0000 7f8d11da1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c106dc0 con 0x7f8d0c1063f0 2024-09-19T08:05:20.546 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.935+0000 7f8d10d9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c1063f0 0x7f8d0c1067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:20.546 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.935+0000 7f8d10d9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c1063f0 0x7f8d0c1067f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48880/0 (socket says 172.21.15.22:48880) 2024-09-19T08:05:20.546 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.935+0000 7f8d10d9f640 1 -- 172.21.15.22:0/422109253 learned_addr learned my addr 172.21.15.22:0/422109253 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:20.546 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.935+0000 7f8d10d9f640 1 -- 172.21.15.22:0/422109253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c1075f0 con 0x7f8d0c1063f0 2024-09-19T08:05:20.546 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.937+0000 7f8d10d9f640 1 --2- 172.21.15.22:0/422109253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c1063f0 0x7f8d0c1067f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009820 tx=0x7f8d0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269308c863ff8cad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:20.546 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.937+0000 7f8d0b7fe640 1 -- 172.21.15.22:0/422109253 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d04037070 con 0x7f8d0c1063f0 2024-09-19T08:05:20.546 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.937+0000 7f8d0b7fe640 1 -- 172.21.15.22:0/422109253 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d04032440 con 0x7f8d0c1063f0 2024-09-19T08:05:20.547 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.937+0000 7f8d0b7fe640 1 -- 172.21.15.22:0/422109253 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d04032740 con 0x7f8d0c1063f0 2024-09-19T08:05:20.547 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.938+0000 7f8d11da1640 1 -- 172.21.15.22:0/422109253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c1063f0 msgr2=0x7f8d0c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:20.547 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.938+0000 7f8d11da1640 1 --2- 172.21.15.22:0/422109253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c1063f0 0x7f8d0c1067f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009820 tx=0x7f8d0402a270 comp rx=0 tx=0).stop 2024-09-19T08:05:20.547 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.938+0000 7f8d11da1640 1 -- 172.21.15.22:0/422109253 shutdown_connections 2024-09-19T08:05:20.547 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.938+0000 7f8d11da1640 1 --2- 172.21.15.22:0/422109253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c1063f0 0x7f8d0c1067f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:20.547 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.938+0000 7f8d11da1640 1 -- 172.21.15.22:0/422109253 >> 172.21.15.22:0/422109253 conn(0x7f8d0c101c40 msgr2=0x7f8d0c104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:20.547 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.938+0000 7f8d11da1640 1 -- 172.21.15.22:0/422109253 shutdown_connections 2024-09-19T08:05:20.547 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.938+0000 7f8d11da1640 1 -- 172.21.15.22:0/422109253 wait complete. 2024-09-19T08:05:20.547 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.939+0000 7f8d11da1640 1 Processor -- start 2024-09-19T08:05:20.547 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.939+0000 7f8d11da1640 1 -- start start 2024-09-19T08:05:20.547 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.939+0000 7f8d11da1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c1063f0 0x7f8d0c19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:20.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.939+0000 7f8d11da1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c107b90 con 0x7f8d0c1063f0 2024-09-19T08:05:20.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.939+0000 7f8d10d9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c1063f0 0x7f8d0c19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:20.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.939+0000 7f8d10d9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c1063f0 0x7f8d0c19acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48882/0 (socket says 172.21.15.22:48882) 2024-09-19T08:05:20.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.939+0000 7f8d10d9f640 1 -- 172.21.15.22:0/2773905946 learned_addr learned my addr 172.21.15.22:0/2773905946 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:20.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.939+0000 7f8d10d9f640 1 -- 172.21.15.22:0/2773905946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c19b200 con 0x7f8d0c1063f0 2024-09-19T08:05:20.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.940+0000 7f8d10d9f640 1 --2- 172.21.15.22:0/2773905946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c1063f0 0x7f8d0c19acc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8d040090b0 tx=0x7f8d040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:20.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.940+0000 7f8d09ffb640 1 -- 172.21.15.22:0/2773905946 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d04041070 con 0x7f8d0c1063f0 2024-09-19T08:05:20.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.940+0000 7f8d09ffb640 1 -- 172.21.15.22:0/2773905946 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d0403c7a0 con 0x7f8d0c1063f0 2024-09-19T08:05:20.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.940+0000 7f8d09ffb640 1 -- 172.21.15.22:0/2773905946 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d04037040 con 0x7f8d0c1063f0 2024-09-19T08:05:20.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.940+0000 7f8d11da1640 1 -- 172.21.15.22:0/2773905946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0c19b490 con 0x7f8d0c1063f0 2024-09-19T08:05:20.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.940+0000 7f8d11da1640 1 -- 172.21.15.22:0/2773905946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0c19e180 con 0x7f8d0c1063f0 2024-09-19T08:05:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.940+0000 7f8d09ffb640 1 -- 172.21.15.22:0/2773905946 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8d04044070 con 0x7f8d0c1063f0 2024-09-19T08:05:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.940+0000 7f8d09ffb640 1 -- 172.21.15.22:0/2773905946 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8d0403d3f0 con 0x7f8d0c1063f0 2024-09-19T08:05:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.941+0000 7f8d11da1640 1 -- 172.21.15.22:0/2773905946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0c1081e0 con 0x7f8d0c1063f0 2024-09-19T08:05:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:19.944+0000 7f8d09ffb640 1 -- 172.21.15.22:0/2773905946 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8d04046e10 con 0x7f8d0c1063f0 2024-09-19T08:05:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.005+0000 7f8d11da1640 1 -- 172.21.15.22:0/2773905946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f8d0c19e740 con 0x7f8d0c1063f0 2024-09-19T08:05:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.006+0000 7f8d09ffb640 1 -- 172.21.15.22:0/2773905946 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f8d04046450 con 0x7f8d0c1063f0 2024-09-19T08:05:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.006+0000 7f8d11da1640 1 -- 172.21.15.22:0/2773905946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c1063f0 msgr2=0x7f8d0c19acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.006+0000 7f8d11da1640 1 --2- 172.21.15.22:0/2773905946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c1063f0 0x7f8d0c19acc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8d040090b0 tx=0x7f8d040046c0 comp rx=0 tx=0).stop 2024-09-19T08:05:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.006+0000 7f8d11da1640 1 -- 172.21.15.22:0/2773905946 shutdown_connections 2024-09-19T08:05:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.006+0000 7f8d11da1640 1 --2- 172.21.15.22:0/2773905946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c1063f0 0x7f8d0c19acc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.006+0000 7f8d11da1640 1 -- 172.21.15.22:0/2773905946 >> 172.21.15.22:0/2773905946 conn(0x7f8d0c101c40 msgr2=0x7f8d0c07b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.007+0000 7f8d11da1640 1 -- 172.21.15.22:0/2773905946 shutdown_connections 2024-09-19T08:05:20.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.007+0000 7f8d11da1640 1 -- 172.21.15.22:0/2773905946 wait complete. 2024-09-19T08:05:20.551 INFO:teuthology.orchestra.run.smithi022.stdout:mon is available 2024-09-19T08:05:20.551 INFO:teuthology.orchestra.run.smithi022.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T08:05:20.659 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:20 smithi022 ceph-mon[30594]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T08:05:20.659 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:20 smithi022 ceph-mon[30594]: monmap epoch 1 2024-09-19T08:05:20.659 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:20 smithi022 ceph-mon[30594]: fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:05:20.659 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:20 smithi022 ceph-mon[30594]: last_changed 2024-09-19T08:05:15.003832+0000 2024-09-19T08:05:20.659 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:20 smithi022 ceph-mon[30594]: created 2024-09-19T08:05:15.003832+0000 2024-09-19T08:05:20.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:20 smithi022 ceph-mon[30594]: min_mon_release 19 (squid) 2024-09-19T08:05:20.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:20 smithi022 ceph-mon[30594]: election_strategy: 1 2024-09-19T08:05:20.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:20 smithi022 ceph-mon[30594]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-09-19T08:05:20.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:20 smithi022 ceph-mon[30594]: fsmap 2024-09-19T08:05:20.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:20 smithi022 ceph-mon[30594]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T08:05:20.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:20 smithi022 ceph-mon[30594]: mgrmap e1: no daemons active 2024-09-19T08:05:20.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:20 smithi022 ceph-mon[30594]: from='client.? 172.21.15.22:0/2773905946' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T08:05:21.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:21 smithi022 ceph-mon[30594]: from='client.? 172.21.15.22:0/225978095' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T08:05:21.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:21 smithi022 ceph-mon[30594]: from='client.? 172.21.15.22:0/225978095' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T08:05:22.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-19T08:05:22.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T08:05:22.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:05:22.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2024-09-19T08:05:22.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T08:05:22.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T08:05:22.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T08:05:22.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T08:05:22.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-19T08:05:22.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T08:05:22.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T08:05:22.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-19T08:05:22.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T08:05:22.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T08:05:22.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T08:05:22.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T08:05:22.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T08:05:22.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T08:05:22.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.941+0000 7f02958f7640 1 Processor -- start 2024-09-19T08:05:22.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.941+0000 7f02958f7640 1 -- start start 2024-09-19T08:05:22.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.941+0000 7f02958f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02901081a0 0x7f029010a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:22.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.941+0000 7f02958f7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f029007be70 con 0x7f02901081a0 2024-09-19T08:05:22.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.942+0000 7f028effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02901081a0 0x7f029010a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:22.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.942+0000 7f028effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02901081a0 0x7f029010a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48896/0 (socket says 172.21.15.22:48896) 2024-09-19T08:05:22.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.942+0000 7f028effd640 1 -- 172.21.15.22:0/1355653685 learned_addr learned my addr 172.21.15.22:0/1355653685 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:22.299 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.942+0000 7f028effd640 1 -- 172.21.15.22:0/1355653685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029010aad0 con 0x7f02901081a0 2024-09-19T08:05:22.299 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.943+0000 7f028effd640 1 --2- 172.21.15.22:0/1355653685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02901081a0 0x7f029010a590 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0288009e80 tx=0x7f028802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf3c77e87912c48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:22.299 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.943+0000 7f028dffb640 1 -- 172.21.15.22:0/1355653685 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0288036070 con 0x7f02901081a0 2024-09-19T08:05:22.299 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.943+0000 7f028dffb640 1 -- 172.21.15.22:0/1355653685 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0288031400 con 0x7f02901081a0 2024-09-19T08:05:22.299 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.943+0000 7f028dffb640 1 -- 172.21.15.22:0/1355653685 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0288031700 con 0x7f02901081a0 2024-09-19T08:05:22.299 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.944+0000 7f02958f7640 1 -- 172.21.15.22:0/1355653685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02901081a0 msgr2=0x7f029010a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:22.299 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.944+0000 7f02958f7640 1 --2- 172.21.15.22:0/1355653685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02901081a0 0x7f029010a590 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0288009e80 tx=0x7f028802a7a0 comp rx=0 tx=0).stop 2024-09-19T08:05:22.299 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.944+0000 7f02958f7640 1 -- 172.21.15.22:0/1355653685 shutdown_connections 2024-09-19T08:05:22.299 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.944+0000 7f02958f7640 1 --2- 172.21.15.22:0/1355653685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02901081a0 0x7f029010a590 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:22.299 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.944+0000 7f02958f7640 1 -- 172.21.15.22:0/1355653685 >> 172.21.15.22:0/1355653685 conn(0x7f0290101e20 msgr2=0x7f0290104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:22.299 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.944+0000 7f02958f7640 1 -- 172.21.15.22:0/1355653685 shutdown_connections 2024-09-19T08:05:22.299 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.944+0000 7f02958f7640 1 -- 172.21.15.22:0/1355653685 wait complete. 2024-09-19T08:05:22.299 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.944+0000 7f02958f7640 1 Processor -- start 2024-09-19T08:05:22.300 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.945+0000 7f02958f7640 1 -- start start 2024-09-19T08:05:22.300 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.945+0000 7f02958f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02901081a0 0x7f029019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:22.300 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.945+0000 7f02958f7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f029010afb0 con 0x7f02901081a0 2024-09-19T08:05:22.300 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.945+0000 7f028effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02901081a0 0x7f029019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:22.300 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.945+0000 7f028effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02901081a0 0x7f029019ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48902/0 (socket says 172.21.15.22:48902) 2024-09-19T08:05:22.300 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.945+0000 7f028effd640 1 -- 172.21.15.22:0/225978095 learned_addr learned my addr 172.21.15.22:0/225978095 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:22.300 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.945+0000 7f028effd640 1 -- 172.21.15.22:0/225978095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029019b3b0 con 0x7f02901081a0 2024-09-19T08:05:22.300 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.945+0000 7f028effd640 1 --2- 172.21.15.22:0/225978095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02901081a0 0x7f029019ae70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0288034040 tx=0x7f0288004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:22.300 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.945+0000 7f02948f5640 1 -- 172.21.15.22:0/225978095 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0288041070 con 0x7f02901081a0 2024-09-19T08:05:22.300 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.945+0000 7f02948f5640 1 -- 172.21.15.22:0/225978095 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028803c650 con 0x7f02901081a0 2024-09-19T08:05:22.300 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.945+0000 7f02958f7640 1 -- 172.21.15.22:0/225978095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029019b640 con 0x7f02901081a0 2024-09-19T08:05:22.300 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.946+0000 7f02948f5640 1 -- 172.21.15.22:0/225978095 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0288036040 con 0x7f02901081a0 2024-09-19T08:05:22.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.946+0000 7f02958f7640 1 -- 172.21.15.22:0/225978095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f029019e330 con 0x7f02901081a0 2024-09-19T08:05:22.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.946+0000 7f02948f5640 1 -- 172.21.15.22:0/225978095 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0288044070 con 0x7f02901081a0 2024-09-19T08:05:22.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.946+0000 7f02948f5640 1 -- 172.21.15.22:0/225978095 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0288046c10 con 0x7f02901081a0 2024-09-19T08:05:22.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.947+0000 7f02958f7640 1 -- 172.21.15.22:0/225978095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0290108600 con 0x7f02901081a0 2024-09-19T08:05:22.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:20.949+0000 7f02948f5640 1 -- 172.21.15.22:0/225978095 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f028803c950 con 0x7f02901081a0 2024-09-19T08:05:22.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:21.008+0000 7f02958f7640 1 -- 172.21.15.22:0/225978095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f029002d910 con 0x7f02901081a0 2024-09-19T08:05:22.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:21.012+0000 7f02948f5640 1 -- 172.21.15.22:0/225978095 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f028803cb30 con 0x7f02901081a0 2024-09-19T08:05:22.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:21.012+0000 7f02948f5640 1 -- 172.21.15.22:0/225978095 <== mon.0 v2:172.21.15.22:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0288056070 con 0x7f02901081a0 2024-09-19T08:05:22.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:21.013+0000 7f02958f7640 1 -- 172.21.15.22:0/225978095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02901081a0 msgr2=0x7f029019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:22.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:21.013+0000 7f02958f7640 1 --2- 172.21.15.22:0/225978095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02901081a0 0x7f029019ae70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0288034040 tx=0x7f0288004a40 comp rx=0 tx=0).stop 2024-09-19T08:05:22.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:21.013+0000 7f02958f7640 1 -- 172.21.15.22:0/225978095 shutdown_connections 2024-09-19T08:05:22.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:21.013+0000 7f02958f7640 1 --2- 172.21.15.22:0/225978095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02901081a0 0x7f029019ae70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:22.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:21.013+0000 7f02958f7640 1 -- 172.21.15.22:0/225978095 >> 172.21.15.22:0/225978095 conn(0x7f0290101e20 msgr2=0x7f0290102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:22.302 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:21.013+0000 7f02958f7640 1 -- 172.21.15.22:0/225978095 shutdown_connections 2024-09-19T08:05:22.302 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:21.013+0000 7f02958f7640 1 -- 172.21.15.22:0/225978095 wait complete. 2024-09-19T08:05:22.302 INFO:teuthology.orchestra.run.smithi022.stdout:Generating new minimal ceph.conf... 2024-09-19T08:05:23.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:23 smithi022 ceph-mon[30594]: from='client.? 172.21.15.22:0/1002707991' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:05:23.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.944+0000 7fc85ff35640 1 Processor -- start 2024-09-19T08:05:23.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.944+0000 7fc85ff35640 1 -- start start 2024-09-19T08:05:23.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.945+0000 7fc85ff35640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8581081e0 0x7fc85810a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:23.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.945+0000 7fc85ff35640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc85807be30 con 0x7fc8581081e0 2024-09-19T08:05:23.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.945+0000 7fc85dcd1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8581081e0 0x7fc85810a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:23.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.945+0000 7fc85dcd1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8581081e0 0x7fc85810a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48912/0 (socket says 172.21.15.22:48912) 2024-09-19T08:05:23.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.945+0000 7fc85dcd1640 1 -- 172.21.15.22:0/3657530030 learned_addr learned my addr 172.21.15.22:0/3657530030 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:23.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.945+0000 7fc85dcd1640 1 -- 172.21.15.22:0/3657530030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85810ab10 con 0x7fc8581081e0 2024-09-19T08:05:23.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.946+0000 7fc85dcd1640 1 --2- 172.21.15.22:0/3657530030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8581081e0 0x7fc85810a5d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc850009820 tx=0x7fc85002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e69ae069e9bacf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:23.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.946+0000 7fc85cccf640 1 -- 172.21.15.22:0/3657530030 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc850037070 con 0x7fc8581081e0 2024-09-19T08:05:23.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.946+0000 7fc85cccf640 1 -- 172.21.15.22:0/3657530030 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc850032900 con 0x7fc8581081e0 2024-09-19T08:05:23.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.946+0000 7fc85cccf640 1 -- 172.21.15.22:0/3657530030 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc850030850 con 0x7fc8581081e0 2024-09-19T08:05:23.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.947+0000 7fc85ff35640 1 -- 172.21.15.22:0/3657530030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8581081e0 msgr2=0x7fc85810a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:23.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.947+0000 7fc85ff35640 1 --2- 172.21.15.22:0/3657530030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8581081e0 0x7fc85810a5d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc850009820 tx=0x7fc85002a270 comp rx=0 tx=0).stop 2024-09-19T08:05:23.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.947+0000 7fc85ff35640 1 -- 172.21.15.22:0/3657530030 shutdown_connections 2024-09-19T08:05:23.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.947+0000 7fc85ff35640 1 --2- 172.21.15.22:0/3657530030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8581081e0 0x7fc85810a5d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:23.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.947+0000 7fc85ff35640 1 -- 172.21.15.22:0/3657530030 >> 172.21.15.22:0/3657530030 conn(0x7fc858101e20 msgr2=0x7fc858104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:23.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.947+0000 7fc85ff35640 1 -- 172.21.15.22:0/3657530030 shutdown_connections 2024-09-19T08:05:23.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.947+0000 7fc85ff35640 1 -- 172.21.15.22:0/3657530030 wait complete. 2024-09-19T08:05:23.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.947+0000 7fc85ff35640 1 Processor -- start 2024-09-19T08:05:23.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.948+0000 7fc85ff35640 1 -- start start 2024-09-19T08:05:23.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.948+0000 7fc85ff35640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8581081e0 0x7fc85819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:23.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.948+0000 7fc85ff35640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc85810baf0 con 0x7fc8581081e0 2024-09-19T08:05:23.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.948+0000 7fc85dcd1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8581081e0 0x7fc85819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:23.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.948+0000 7fc85dcd1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8581081e0 0x7fc85819b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48922/0 (socket says 172.21.15.22:48922) 2024-09-19T08:05:23.564 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.948+0000 7fc85dcd1640 1 -- 172.21.15.22:0/1002707991 learned_addr learned my addr 172.21.15.22:0/1002707991 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:23.564 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.948+0000 7fc85dcd1640 1 -- 172.21.15.22:0/1002707991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85819bc30 con 0x7fc8581081e0 2024-09-19T08:05:23.564 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.948+0000 7fc85dcd1640 1 --2- 172.21.15.22:0/1002707991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8581081e0 0x7fc85819b6f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc850031000 tx=0x7fc8500326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:23.564 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.948+0000 7fc856ffd640 1 -- 172.21.15.22:0/1002707991 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc850042070 con 0x7fc8581081e0 2024-09-19T08:05:23.564 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.949+0000 7fc856ffd640 1 -- 172.21.15.22:0/1002707991 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc850032be0 con 0x7fc8581081e0 2024-09-19T08:05:23.564 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.949+0000 7fc85ff35640 1 -- 172.21.15.22:0/1002707991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85819bec0 con 0x7fc8581081e0 2024-09-19T08:05:23.564 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.949+0000 7fc856ffd640 1 -- 172.21.15.22:0/1002707991 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc850037040 con 0x7fc8581081e0 2024-09-19T08:05:23.564 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.949+0000 7fc85ff35640 1 -- 172.21.15.22:0/1002707991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc85819ebb0 con 0x7fc8581081e0 2024-09-19T08:05:23.564 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.949+0000 7fc856ffd640 1 -- 172.21.15.22:0/1002707991 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc850045070 con 0x7fc8581081e0 2024-09-19T08:05:23.564 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.949+0000 7fc856ffd640 1 -- 172.21.15.22:0/1002707991 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc850047b10 con 0x7fc8581081e0 2024-09-19T08:05:23.564 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.950+0000 7fc85ff35640 1 -- 172.21.15.22:0/1002707991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc858108640 con 0x7fc8581081e0 2024-09-19T08:05:23.564 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:22.952+0000 7fc856ffd640 1 -- 172.21.15.22:0/1002707991 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc85004c030 con 0x7fc8581081e0 2024-09-19T08:05:23.565 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:23.013+0000 7fc85ff35640 1 -- 172.21.15.22:0/1002707991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc85819ed80 con 0x7fc8581081e0 2024-09-19T08:05:23.565 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:23.013+0000 7fc856ffd640 1 -- 172.21.15.22:0/1002707991 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fc8500328c0 con 0x7fc8581081e0 2024-09-19T08:05:23.565 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:23.014+0000 7fc85ff35640 1 -- 172.21.15.22:0/1002707991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8581081e0 msgr2=0x7fc85819b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:23.565 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:23.014+0000 7fc85ff35640 1 --2- 172.21.15.22:0/1002707991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8581081e0 0x7fc85819b6f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc850031000 tx=0x7fc8500326e0 comp rx=0 tx=0).stop 2024-09-19T08:05:23.565 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:23.014+0000 7fc85ff35640 1 -- 172.21.15.22:0/1002707991 shutdown_connections 2024-09-19T08:05:23.565 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:23.014+0000 7fc85ff35640 1 --2- 172.21.15.22:0/1002707991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8581081e0 0x7fc85819b6f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:23.565 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:23.014+0000 7fc85ff35640 1 -- 172.21.15.22:0/1002707991 >> 172.21.15.22:0/1002707991 conn(0x7fc858101e20 msgr2=0x7fc858102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:23.565 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:23.014+0000 7fc85ff35640 1 -- 172.21.15.22:0/1002707991 shutdown_connections 2024-09-19T08:05:23.565 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:23.014+0000 7fc85ff35640 1 -- 172.21.15.22:0/1002707991 wait complete. 2024-09-19T08:05:23.565 INFO:teuthology.orchestra.run.smithi022.stdout:Restarting the monitor... 2024-09-19T08:05:23.832 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:23 smithi022 systemd[1]: Stopping Ceph mon.a for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:05:24.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:23 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a[30590]: 2024-09-19T08:05:23.831+0000 7fcd28a7c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:05:24.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:23 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a[30590]: 2024-09-19T08:05:23.831+0000 7fcd28a7c640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T08:05:24.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:23 smithi022 podman[30837]: 2024-09-19 08:05:23.944291164 +0000 UTC m=+0.214755438 container died 8f495c72d3633d1d9f637c0b9c48ec0adcd8dfea076df22194d4ebd9bb73e2af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T08:05:24.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:24 smithi022 podman[30837]: 2024-09-19 08:05:24.289357295 +0000 UTC m=+0.559821614 container cleanup 8f495c72d3633d1d9f637c0b9c48ec0adcd8dfea076df22194d4ebd9bb73e2af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T08:05:24.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:24 smithi022 bash[30837]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a 2024-09-19T08:05:24.924 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:24 smithi022 podman[30849]: 2024-09-19 08:05:24.556950332 +0000 UTC m=+0.610874984 container remove 8f495c72d3633d1d9f637c0b9c48ec0adcd8dfea076df22194d4ebd9bb73e2af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:05:24.924 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:24 smithi022 systemd[1]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T08:05:24.924 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:24 smithi022 systemd[1]: Stopped Ceph mon.a for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:05:24.924 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:24 smithi022 systemd[1]: Starting Ceph mon.a for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:05:25.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:24 smithi022 podman[30944]: 2024-09-19 08:05:24.827507369 +0000 UTC m=+0.016588758 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:05:25.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:24 smithi022 podman[30944]: 2024-09-19 08:05:24.924507055 +0000 UTC m=+0.113588438 container create 3ded52544efcb5ce71aa7934b2bcd324cdb40cddc4de64baffba0593d6b24d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T08:05:25.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 podman[30944]: 2024-09-19 08:05:25.135002571 +0000 UTC m=+0.324083951 container init 3ded52544efcb5ce71aa7934b2bcd324cdb40cddc4de64baffba0593d6b24d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:05:25.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 podman[30944]: 2024-09-19 08:05:25.138260003 +0000 UTC m=+0.327341381 container start 3ded52544efcb5ce71aa7934b2bcd324cdb40cddc4de64baffba0593d6b24d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T08:05:25.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T08:05:25.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T08:05:25.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: pidfile_write: ignore empty --pid-file 2024-09-19T08:05:25.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: load: jerasure load: lrc 2024-09-19T08:05:25.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: RocksDB version: 7.9.2 2024-09-19T08:05:25.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Git sha 0 2024-09-19T08:05:25.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T08:05:25.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: DB SUMMARY 2024-09-19T08:05:25.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: DB Session ID: X9ZR5ZTG7CNOG3ESCLQT 2024-09-19T08:05:25.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: CURRENT file: CURRENT 2024-09-19T08:05:25.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T08:05:25.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T08:05:25.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T08:05:25.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105854 ; 2024-09-19T08:05:25.220 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.error_if_exists: 0 2024-09-19T08:05:25.220 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.create_if_missing: 0 2024-09-19T08:05:25.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.paranoid_checks: 1 2024-09-19T08:05:25.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T08:05:25.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T08:05:25.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T08:05:25.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.env: 0x55a8b551b940 2024-09-19T08:05:25.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T08:05:25.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.info_log: 0x55a8b607d6e0 2024-09-19T08:05:25.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T08:05:25.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.statistics: (nil) 2024-09-19T08:05:25.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.use_fsync: 0 2024-09-19T08:05:25.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_log_file_size: 0 2024-09-19T08:05:25.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T08:05:25.222 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T08:05:25.222 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T08:05:25.222 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T08:05:25.222 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.allow_fallocate: 1 2024-09-19T08:05:25.222 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T08:05:25.222 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T08:05:25.222 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.use_direct_reads: 0 2024-09-19T08:05:25.222 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T08:05:25.222 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T08:05:25.222 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.db_log_dir: 2024-09-19T08:05:25.222 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.wal_dir: 2024-09-19T08:05:25.223 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T08:05:25.223 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T08:05:25.223 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T08:05:25.223 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T08:05:25.223 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T08:05:25.223 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T08:05:25.223 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T08:05:25.223 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T08:05:25.223 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.write_buffer_manager: 0x55a8b608cdc0 2024-09-19T08:05:25.223 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T08:05:25.223 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T08:05:25.223 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T08:05:25.224 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T08:05:25.224 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T08:05:25.224 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T08:05:25.224 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T08:05:25.224 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T08:05:25.224 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.unordered_write: 0 2024-09-19T08:05:25.224 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T08:05:25.224 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T08:05:25.224 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T08:05:25.224 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T08:05:25.224 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.row_cache: None 2024-09-19T08:05:25.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.wal_filter: None 2024-09-19T08:05:25.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T08:05:25.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T08:05:25.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.two_write_queues: 0 2024-09-19T08:05:25.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T08:05:25.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.wal_compression: 0 2024-09-19T08:05:25.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.atomic_flush: 0 2024-09-19T08:05:25.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T08:05:25.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T08:05:25.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T08:05:25.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.log_readahead_size: 0 2024-09-19T08:05:25.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T08:05:25.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T08:05:25.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T08:05:25.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T08:05:25.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T08:05:25.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T08:05:25.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T08:05:25.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_background_jobs: 2 2024-09-19T08:05:25.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_background_compactions: -1 2024-09-19T08:05:25.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_subcompactions: 1 2024-09-19T08:05:25.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T08:05:25.227 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T08:05:25.227 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T08:05:25.227 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T08:05:25.227 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T08:05:25.227 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T08:05:25.227 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T08:05:25.227 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T08:05:25.227 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_open_files: -1 2024-09-19T08:05:25.227 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T08:05:25.227 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T08:05:25.227 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T08:05:25.227 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T08:05:25.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_background_flushes: -1 2024-09-19T08:05:25.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Compression algorithms supported: 2024-09-19T08:05:25.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: kZSTD supported: 0 2024-09-19T08:05:25.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: kXpressCompression supported: 0 2024-09-19T08:05:25.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: kBZip2Compression supported: 0 2024-09-19T08:05:25.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T08:05:25.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: kLZ4Compression supported: 1 2024-09-19T08:05:25.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: kZlibCompression supported: 1 2024-09-19T08:05:25.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T08:05:25.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: kSnappyCompression supported: 1 2024-09-19T08:05:25.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T08:05:25.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T08:05:25.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T08:05:25.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T08:05:25.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T08:05:25.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.merge_operator: 2024-09-19T08:05:25.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compaction_filter: None 2024-09-19T08:05:25.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T08:05:25.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T08:05:25.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T08:05:25.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T08:05:25.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a8b607d320) 2024-09-19T08:05:25.230 INFO:journalctl@ceph.mon.a.smithi022.stdout: cache_index_and_filter_blocks: 1 2024-09-19T08:05:25.230 INFO:journalctl@ceph.mon.a.smithi022.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T08:05:25.230 INFO:journalctl@ceph.mon.a.smithi022.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T08:05:25.230 INFO:journalctl@ceph.mon.a.smithi022.stdout: pin_top_level_index_and_filter: 1 2024-09-19T08:05:25.230 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_type: 0 2024-09-19T08:05:25.230 INFO:journalctl@ceph.mon.a.smithi022.stdout: data_block_index_type: 0 2024-09-19T08:05:25.230 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_shortening: 1 2024-09-19T08:05:25.230 INFO:journalctl@ceph.mon.a.smithi022.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T08:05:25.230 INFO:journalctl@ceph.mon.a.smithi022.stdout: checksum: 4 2024-09-19T08:05:25.230 INFO:journalctl@ceph.mon.a.smithi022.stdout: no_block_cache: 0 2024-09-19T08:05:25.230 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache: 0x55a8b60711f0 2024-09-19T08:05:25.230 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_name: BinnedLRUCache 2024-09-19T08:05:25.230 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_options: 2024-09-19T08:05:25.231 INFO:journalctl@ceph.mon.a.smithi022.stdout: capacity : 536870912 2024-09-19T08:05:25.231 INFO:journalctl@ceph.mon.a.smithi022.stdout: num_shard_bits : 4 2024-09-19T08:05:25.231 INFO:journalctl@ceph.mon.a.smithi022.stdout: strict_capacity_limit : 0 2024-09-19T08:05:25.231 INFO:journalctl@ceph.mon.a.smithi022.stdout: high_pri_pool_ratio: 0.000 2024-09-19T08:05:25.231 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_compressed: (nil) 2024-09-19T08:05:25.231 INFO:journalctl@ceph.mon.a.smithi022.stdout: persistent_cache: (nil) 2024-09-19T08:05:25.231 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_size: 4096 2024-09-19T08:05:25.231 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_size_deviation: 10 2024-09-19T08:05:25.231 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_restart_interval: 16 2024-09-19T08:05:25.231 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_block_restart_interval: 1 2024-09-19T08:05:25.232 INFO:journalctl@ceph.mon.a.smithi022.stdout: metadata_block_size: 4096 2024-09-19T08:05:25.232 INFO:journalctl@ceph.mon.a.smithi022.stdout: partition_filters: 0 2024-09-19T08:05:25.232 INFO:journalctl@ceph.mon.a.smithi022.stdout: use_delta_encoding: 1 2024-09-19T08:05:25.233 INFO:journalctl@ceph.mon.a.smithi022.stdout: filter_policy: bloomfilter 2024-09-19T08:05:25.233 INFO:journalctl@ceph.mon.a.smithi022.stdout: whole_key_filtering: 1 2024-09-19T08:05:25.233 INFO:journalctl@ceph.mon.a.smithi022.stdout: verify_compression: 0 2024-09-19T08:05:25.233 INFO:journalctl@ceph.mon.a.smithi022.stdout: read_amp_bytes_per_bit: 0 2024-09-19T08:05:25.233 INFO:journalctl@ceph.mon.a.smithi022.stdout: format_version: 5 2024-09-19T08:05:25.233 INFO:journalctl@ceph.mon.a.smithi022.stdout: enable_index_compression: 1 2024-09-19T08:05:25.233 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_align: 0 2024-09-19T08:05:25.233 INFO:journalctl@ceph.mon.a.smithi022.stdout: max_auto_readahead_size: 262144 2024-09-19T08:05:25.233 INFO:journalctl@ceph.mon.a.smithi022.stdout: prepopulate_block_cache: 0 2024-09-19T08:05:25.233 INFO:journalctl@ceph.mon.a.smithi022.stdout: initial_auto_readahead_size: 8192 2024-09-19T08:05:25.233 INFO:journalctl@ceph.mon.a.smithi022.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T08:05:25.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T08:05:25.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T08:05:25.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compression: NoCompression 2024-09-19T08:05:25.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T08:05:25.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T08:05:25.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T08:05:25.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.num_levels: 7 2024-09-19T08:05:25.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T08:05:25.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T08:05:25.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T08:05:25.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T08:05:25.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T08:05:25.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T08:05:25.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T08:05:25.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T08:05:25.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T08:05:25.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T08:05:25.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T08:05:25.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T08:05:25.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T08:05:25.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T08:05:25.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T08:05:25.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T08:05:25.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T08:05:25.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T08:05:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T08:05:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T08:05:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T08:05:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T08:05:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T08:05:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T08:05:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T08:05:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T08:05:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T08:05:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T08:05:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T08:05:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T08:05:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T08:05:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T08:05:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T08:05:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T08:05:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T08:05:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T08:05:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T08:05:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T08:05:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T08:05:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T08:05:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T08:05:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T08:05:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T08:05:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T08:05:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T08:05:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T08:05:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T08:05:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T08:05:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T08:05:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T08:05:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T08:05:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T08:05:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T08:05:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T08:05:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.inplace_update_support: 0 2024-09-19T08:05:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T08:05:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T08:05:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T08:05:25.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T08:05:25.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.bloom_locality: 0 2024-09-19T08:05:25.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.max_successive_merges: 0 2024-09-19T08:05:25.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T08:05:25.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T08:05:25.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T08:05:25.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T08:05:25.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.ttl: 2592000 2024-09-19T08:05:25.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T08:05:25.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T08:05:25.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T08:05:25.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.enable_blob_files: false 2024-09-19T08:05:25.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.min_blob_size: 0 2024-09-19T08:05:25.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T08:05:25.240 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T08:05:25.240 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T08:05:25.240 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T08:05:25.240 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T08:05:25.240 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T08:05:25.240 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T08:05:25.240 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T08:05:25.240 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T08:05:25.240 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T08:05:25.240 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ce79433a-bc78-4241-b3f5-bbf9e5f1026d 2024-09-19T08:05:25.240 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726733125165402, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T08:05:25.240 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T08:05:25.240 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726733125166504, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100863, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98748, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14431, "raw_average_key_size": 49, "raw_value_size": 90948, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726733125, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ce79433a-bc78-4241-b3f5-bbf9e5f1026d", "db_session_id": "X9ZR5ZTG7CNOG3ESCLQT", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T08:05:25.241 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726733125166601, "job": 1, "event": "recovery_finished"} 2024-09-19T08:05:25.241 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T08:05:25.241 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T08:05:25.241 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a8b609ee00 2024-09-19T08:05:25.241 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: DB pointer 0x55a8b619a000 2024-09-19T08:05:25.241 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T08:05:25.241 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T08:05:25.241 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** DB Stats ** 2024-09-19T08:05:25.241 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T08:05:25.241 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T08:05:25.241 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T08:05:25.241 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T08:05:25.242 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T08:05:25.242 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T08:05:25.242 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T08:05:25.242 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-09-19T08:05:25.242 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2024-09-19T08:05:25.242 INFO:journalctl@ceph.mon.a.smithi022.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T08:05:25.242 INFO:journalctl@ceph.mon.a.smithi022.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T08:05:25.242 INFO:journalctl@ceph.mon.a.smithi022.stdout: L0 2/0 100.36 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T08:05:25.242 INFO:journalctl@ceph.mon.a.smithi022.stdout: Sum 2/0 100.36 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T08:05:25.242 INFO:journalctl@ceph.mon.a.smithi022.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T08:05:25.242 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-09-19T08:05:25.242 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2024-09-19T08:05:25.242 INFO:journalctl@ceph.mon.a.smithi022.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T08:05:25.243 INFO:journalctl@ceph.mon.a.smithi022.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T08:05:25.243 INFO:teuthology.orchestra.run.smithi022.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T08:05:25.243 INFO:journalctl@ceph.mon.a.smithi022.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 140.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T08:05:25.243 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-09-19T08:05:25.243 INFO:journalctl@ceph.mon.a.smithi022.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T08:05:25.243 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-09-19T08:05:25.244 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T08:05:25.244 INFO:journalctl@ceph.mon.a.smithi022.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T08:05:25.244 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T08:05:25.244 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T08:05:25.244 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T08:05:25.244 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T08:05:25.244 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative compaction: 0.00 GB write, 29.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T08:05:25.244 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval compaction: 0.00 GB write, 29.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T08:05:25.244 INFO:journalctl@ceph.mon.a.smithi022.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T08:05:25.244 INFO:journalctl@ceph.mon.a.smithi022.stdout: Block cache BinnedLRUCache@0x55a8b60711f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T08:05:25.244 INFO:journalctl@ceph.mon.a.smithi022.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T08:05:25.244 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-09-19T08:05:25.244 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T08:05:25.245 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: starting mon.a rank 0 at public addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] at bind addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:05:25.245 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: mon.a@-1(???) e1 preinit fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:05:25.245 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: mon.a@-1(???).mds e1 new map 2024-09-19T08:05:25.245 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: mon.a@-1(???).mds e1 print_map 2024-09-19T08:05:25.245 INFO:journalctl@ceph.mon.a.smithi022.stdout: e1 2024-09-19T08:05:25.245 INFO:journalctl@ceph.mon.a.smithi022.stdout: btime 2024-09-19T08:05:19:476769+0000 2024-09-19T08:05:25.245 INFO:journalctl@ceph.mon.a.smithi022.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T08:05:25.245 INFO:journalctl@ceph.mon.a.smithi022.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T08:05:25.245 INFO:journalctl@ceph.mon.a.smithi022.stdout: legacy client fscid: -1 2024-09-19T08:05:25.245 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-09-19T08:05:25.246 INFO:journalctl@ceph.mon.a.smithi022.stdout: No filesystems configured 2024-09-19T08:05:25.246 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T08:05:25.246 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:05:25.246 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:05:25.246 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:05:25.246 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T08:05:25.246 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 bash[30944]: 3ded52544efcb5ce71aa7934b2bcd324cdb40cddc4de64baffba0593d6b24d3b 2024-09-19T08:05:25.246 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 systemd[1]: Started Ceph mon.a for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:05:25.470 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T08:05:25.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: monmap epoch 1 2024-09-19T08:05:25.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:05:25.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: last_changed 2024-09-19T08:05:15.003832+0000 2024-09-19T08:05:25.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: created 2024-09-19T08:05:15.003832+0000 2024-09-19T08:05:25.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: min_mon_release 19 (squid) 2024-09-19T08:05:25.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: election_strategy: 1 2024-09-19T08:05:25.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-09-19T08:05:25.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: fsmap 2024-09-19T08:05:25.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T08:05:25.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:25 smithi022 ceph-mon[30963]: mgrmap e1: no daemons active 2024-09-19T08:05:26.323 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.631+0000 7f4fdb313640 1 Processor -- start 2024-09-19T08:05:26.323 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.632+0000 7f4fdb313640 1 -- start start 2024-09-19T08:05:26.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.632+0000 7f4fdb313640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41065f0 0x7f4fd41069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:26.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.632+0000 7f4fdb313640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4106fc0 con 0x7f4fd41065f0 2024-09-19T08:05:26.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.632+0000 7f4fd90af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41065f0 0x7f4fd41069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:26.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.632+0000 7f4fd90af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41065f0 0x7f4fd41069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48924/0 (socket says 172.21.15.22:48924) 2024-09-19T08:05:26.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.632+0000 7f4fd90af640 1 -- 172.21.15.22:0/4170542096 learned_addr learned my addr 172.21.15.22:0/4170542096 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:26.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.633+0000 7f4fd90af640 1 -- 172.21.15.22:0/4170542096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd41077f0 con 0x7f4fd41065f0 2024-09-19T08:05:26.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.634+0000 7f4fd90af640 1 --2- 172.21.15.22:0/4170542096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41065f0 0x7f4fd41069f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009820 tx=0x7f4fd002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42095ed1e23be764 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:26.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.634+0000 7f4fcbfff640 1 -- 172.21.15.22:0/4170542096 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd0037070 con 0x7f4fd41065f0 2024-09-19T08:05:26.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.634+0000 7f4fcbfff640 1 -- 172.21.15.22:0/4170542096 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4fd0032900 con 0x7f4fd41065f0 2024-09-19T08:05:26.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.634+0000 7f4fcbfff640 1 -- 172.21.15.22:0/4170542096 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd0030850 con 0x7f4fd41065f0 2024-09-19T08:05:26.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.635+0000 7f4fdb313640 1 -- 172.21.15.22:0/4170542096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41065f0 msgr2=0x7f4fd41069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:26.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.635+0000 7f4fdb313640 1 --2- 172.21.15.22:0/4170542096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41065f0 0x7f4fd41069f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009820 tx=0x7f4fd002a270 comp rx=0 tx=0).stop 2024-09-19T08:05:26.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.635+0000 7f4fdb313640 1 -- 172.21.15.22:0/4170542096 shutdown_connections 2024-09-19T08:05:26.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.635+0000 7f4fdb313640 1 --2- 172.21.15.22:0/4170542096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41065f0 0x7f4fd41069f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:26.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.635+0000 7f4fdb313640 1 -- 172.21.15.22:0/4170542096 >> 172.21.15.22:0/4170542096 conn(0x7f4fd4101e20 msgr2=0x7f4fd4104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:26.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.635+0000 7f4fdb313640 1 -- 172.21.15.22:0/4170542096 shutdown_connections 2024-09-19T08:05:26.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.635+0000 7f4fdb313640 1 -- 172.21.15.22:0/4170542096 wait complete. 2024-09-19T08:05:26.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.636+0000 7f4fdb313640 1 Processor -- start 2024-09-19T08:05:26.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.636+0000 7f4fdb313640 1 -- start start 2024-09-19T08:05:26.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.636+0000 7f4fdb313640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41065f0 0x7f4fd4191820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:26.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.636+0000 7f4fdb313640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd410aba0 con 0x7f4fd41065f0 2024-09-19T08:05:26.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.636+0000 7f4fd90af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41065f0 0x7f4fd4191820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:26.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.636+0000 7f4fd90af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41065f0 0x7f4fd4191820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48932/0 (socket says 172.21.15.22:48932) 2024-09-19T08:05:26.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.636+0000 7f4fd90af640 1 -- 172.21.15.22:0/2381055428 learned_addr learned my addr 172.21.15.22:0/2381055428 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:26.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.637+0000 7f4fd90af640 1 -- 172.21.15.22:0/2381055428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd4191d60 con 0x7f4fd41065f0 2024-09-19T08:05:26.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.637+0000 7f4fd90af640 1 --2- 172.21.15.22:0/2381055428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41065f0 0x7f4fd4191820 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009940 tx=0x7f4fd00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:26.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.637+0000 7f4fca7fc640 1 -- 172.21.15.22:0/2381055428 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd0042070 con 0x7f4fd41065f0 2024-09-19T08:05:26.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.637+0000 7f4fca7fc640 1 -- 172.21.15.22:0/2381055428 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4fd0032be0 con 0x7f4fd41065f0 2024-09-19T08:05:26.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.637+0000 7f4fca7fc640 1 -- 172.21.15.22:0/2381055428 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd0037040 con 0x7f4fd41065f0 2024-09-19T08:05:26.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.637+0000 7f4fdb313640 1 -- 172.21.15.22:0/2381055428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd418ff50 con 0x7f4fd41065f0 2024-09-19T08:05:26.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.637+0000 7f4fdb313640 1 -- 172.21.15.22:0/2381055428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd4190490 con 0x7f4fd41065f0 2024-09-19T08:05:26.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.637+0000 7f4fca7fc640 1 -- 172.21.15.22:0/2381055428 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4fd0045070 con 0x7f4fd41065f0 2024-09-19T08:05:26.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.637+0000 7f4fca7fc640 1 -- 172.21.15.22:0/2381055428 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4fd0047df0 con 0x7f4fd41065f0 2024-09-19T08:05:26.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.638+0000 7f4fdb313640 1 -- 172.21.15.22:0/2381055428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd410ad20 con 0x7f4fd41065f0 2024-09-19T08:05:26.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.640+0000 7f4fca7fc640 1 -- 172.21.15.22:0/2381055428 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4fd0032d90 con 0x7f4fd41065f0 2024-09-19T08:05:26.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.700+0000 7f4fdb313640 1 -- 172.21.15.22:0/2381055428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f4fd4195130 con 0x7f4fd41065f0 2024-09-19T08:05:26.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.701+0000 7f4fca7fc640 1 -- 172.21.15.22:0/2381055428 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f4fd0047440 con 0x7f4fd41065f0 2024-09-19T08:05:26.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.702+0000 7f4fdb313640 1 -- 172.21.15.22:0/2381055428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41065f0 msgr2=0x7f4fd4191820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:26.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.702+0000 7f4fdb313640 1 --2- 172.21.15.22:0/2381055428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41065f0 0x7f4fd4191820 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009940 tx=0x7f4fd00326e0 comp rx=0 tx=0).stop 2024-09-19T08:05:26.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.702+0000 7f4fdb313640 1 -- 172.21.15.22:0/2381055428 shutdown_connections 2024-09-19T08:05:26.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.702+0000 7f4fdb313640 1 --2- 172.21.15.22:0/2381055428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fd41065f0 0x7f4fd4191820 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:26.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.702+0000 7f4fdb313640 1 -- 172.21.15.22:0/2381055428 >> 172.21.15.22:0/2381055428 conn(0x7f4fd4101e20 msgr2=0x7f4fd4102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:26.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.702+0000 7f4fdb313640 1 -- 172.21.15.22:0/2381055428 shutdown_connections 2024-09-19T08:05:26.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:25.702+0000 7f4fdb313640 1 -- 172.21.15.22:0/2381055428 wait complete. 2024-09-19T08:05:26.327 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T08:05:26.340 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:05:26.340 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mgr... 2024-09-19T08:05:26.341 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T08:05:26.341 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T08:05:26.606 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mgr.x 2024-09-19T08:05:26.606 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T08:05:26.761 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c0ad60c6-765d-11ef-bceb-c7b262605968.target.wants/ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@.service. 2024-09-19T08:05:26.993 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:26 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2381055428' entity='client.admin' 2024-09-19T08:05:26.993 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:26 smithi022 systemd[1]: Starting Ceph mgr.x for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:05:27.245 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:26 smithi022 podman[31174]: 2024-09-19 08:05:26.993453612 +0000 UTC m=+0.107560213 container create e278914c599895cc093a3e397c64fe28ec77a9c06767e2c794178116786fa934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T08:05:27.246 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:26 smithi022 podman[31174]: 2024-09-19 08:05:26.902101039 +0000 UTC m=+0.016207615 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:05:27.246 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:27 smithi022 podman[31174]: 2024-09-19 08:05:27.170996919 +0000 UTC m=+0.285103484 container init e278914c599895cc093a3e397c64fe28ec77a9c06767e2c794178116786fa934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:05:27.246 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:27 smithi022 podman[31174]: 2024-09-19 08:05:27.173927403 +0000 UTC m=+0.288033959 container start e278914c599895cc093a3e397c64fe28ec77a9c06767e2c794178116786fa934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T08:05:27.246 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:27 smithi022 bash[31174]: e278914c599895cc093a3e397c64fe28ec77a9c06767e2c794178116786fa934 2024-09-19T08:05:27.262 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T08:05:27.262 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-09-19T08:05:27.270 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T08:05:27.270 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-09-19T08:05:27.270 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-09-19T08:05:27.270 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T08:05:27.280 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T08:05:27.280 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-09-19T08:05:27.288 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T08:05:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-09-19T08:05:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-09-19T08:05:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T08:05:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr to start... 2024-09-19T08:05:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr... 2024-09-19T08:05:27.507 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:27 smithi022 systemd[1]: Started Ceph mgr.x for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:05:27.507 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:27 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:27.307+0000 7f1f78ab4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:05:27.507 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:27 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:27.368+0000 7f1f78ab4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:05:27.792 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:27 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:27.525+0000 7f1f78ab4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:05:27.793 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:27 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:27.599+0000 7f1f78ab4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:05:27.793 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:27 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:27.760+0000 7f1f78ab4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:05:28.090 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:27 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1012093299' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T08:05:28.091 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:27 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:27.910+0000 7f1f78ab4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:05:28.352 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:28 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:28.245+0000 7f1f78ab4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:05:28.352 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:28 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:28.310+0000 7f1f78ab4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:05:28.627 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:28 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:28.505+0000 7f1f78ab4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:05:28.642 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-19T08:05:28.642 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:05:28.642 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "c0ad60c6-765d-11ef-bceb-c7b262605968", 2024-09-19T08:05:28.642 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T08:05:28.642 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T08:05:28.642 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T08:05:28.642 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T08:05:28.642 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:28.642 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T08:05:28.643 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T08:05:28.643 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-09-19T08:05:28.643 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:05:28.643 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T08:05:28.643 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T08:05:28.643 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:05:28.643 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-19T08:05:28.643 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T08:05:28.643 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:05:28.643 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T08:05:28.643 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T08:05:28.643 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:28.643 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T08:05:28.644 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:05:28.644 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T08:05:28.644 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T08:05:28.644 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T08:05:28.644 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T08:05:28.644 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T08:05:28.644 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T08:05:28.644 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:28.644 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T08:05:28.644 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T08:05:28.644 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T08:05:28.644 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T08:05:28.644 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T08:05:28.644 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T08:05:28.645 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T08:05:28.645 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T08:05:28.645 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T08:05:28.645 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:28.645 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T08:05:28.645 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:05:28.645 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T08:05:19:476769+0000", 2024-09-19T08:05:28.645 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T08:05:28.645 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T08:05:28.645 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:28.645 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T08:05:28.645 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T08:05:28.645 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T08:05:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T08:05:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T08:05:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T08:05:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T08:05:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:05:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:05:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T08:05:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:05:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T08:05:19.477539+0000", 2024-09-19T08:05:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:05:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T08:05:28.646 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:05:28.647 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.720+0000 7f4eb2308640 1 Processor -- start 2024-09-19T08:05:28.647 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.721+0000 7f4eb2308640 1 -- start start 2024-09-19T08:05:28.647 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.721+0000 7f4eb2308640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4eac108210 0x7f4eac10a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:28.647 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.721+0000 7f4eb2308640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac07bdf0 con 0x7f4eac108210 2024-09-19T08:05:28.647 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.721+0000 7f4eabfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4eac108210 0x7f4eac10a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:28.647 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.721+0000 7f4eabfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4eac108210 0x7f4eac10a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38654/0 (socket says 172.21.15.22:38654) 2024-09-19T08:05:28.647 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.721+0000 7f4eabfff640 1 -- 172.21.15.22:0/2146183456 learned_addr learned my addr 172.21.15.22:0/2146183456 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:28.647 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.722+0000 7f4eabfff640 1 -- 172.21.15.22:0/2146183456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac10ab40 con 0x7f4eac108210 2024-09-19T08:05:28.647 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.722+0000 7f4eabfff640 1 --2- 172.21.15.22:0/2146183456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4eac108210 0x7f4eac10a600 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009820 tx=0x7f4e9c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968a3ac7400b7a21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:28.647 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.723+0000 7f4eaaffd640 1 -- 172.21.15.22:0/2146183456 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e9c037070 con 0x7f4eac108210 2024-09-19T08:05:28.647 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.723+0000 7f4eaaffd640 1 -- 172.21.15.22:0/2146183456 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e9c032900 con 0x7f4eac108210 2024-09-19T08:05:28.647 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.723+0000 7f4eaaffd640 1 -- 172.21.15.22:0/2146183456 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e9c030850 con 0x7f4eac108210 2024-09-19T08:05:28.648 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.723+0000 7f4eb2308640 1 -- 172.21.15.22:0/2146183456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4eac108210 msgr2=0x7f4eac10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:28.648 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.723+0000 7f4eb2308640 1 --2- 172.21.15.22:0/2146183456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4eac108210 0x7f4eac10a600 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009820 tx=0x7f4e9c02a270 comp rx=0 tx=0).stop 2024-09-19T08:05:28.648 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.723+0000 7f4eb2308640 1 -- 172.21.15.22:0/2146183456 shutdown_connections 2024-09-19T08:05:28.648 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.723+0000 7f4eb2308640 1 --2- 172.21.15.22:0/2146183456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4eac108210 0x7f4eac10a600 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:28.648 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.723+0000 7f4eb2308640 1 -- 172.21.15.22:0/2146183456 >> 172.21.15.22:0/2146183456 conn(0x7f4eac101d90 msgr2=0x7f4eac1041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:28.648 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.723+0000 7f4eb2308640 1 -- 172.21.15.22:0/2146183456 shutdown_connections 2024-09-19T08:05:28.648 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.723+0000 7f4eb2308640 1 -- 172.21.15.22:0/2146183456 wait complete. 2024-09-19T08:05:28.648 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.724+0000 7f4eb2308640 1 Processor -- start 2024-09-19T08:05:28.648 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.724+0000 7f4eb2308640 1 -- start start 2024-09-19T08:05:28.648 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.724+0000 7f4eb2308640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4eac108210 0x7f4eac19b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:28.648 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.724+0000 7f4eb2308640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac10bb20 con 0x7f4eac108210 2024-09-19T08:05:28.649 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.725+0000 7f4eabfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4eac108210 0x7f4eac19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:28.649 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.725+0000 7f4eabfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4eac108210 0x7f4eac19b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38660/0 (socket says 172.21.15.22:38660) 2024-09-19T08:05:28.649 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.725+0000 7f4eabfff640 1 -- 172.21.15.22:0/1012093299 learned_addr learned my addr 172.21.15.22:0/1012093299 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:28.649 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.725+0000 7f4eabfff640 1 -- 172.21.15.22:0/1012093299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac19bc60 con 0x7f4eac108210 2024-09-19T08:05:28.649 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.725+0000 7f4eabfff640 1 --2- 172.21.15.22:0/1012093299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4eac108210 0x7f4eac19b720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0097f0 tx=0x7f4e9c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:28.649 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.725+0000 7f4ea97fa640 1 -- 172.21.15.22:0/1012093299 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e9c042070 con 0x7f4eac108210 2024-09-19T08:05:28.649 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.725+0000 7f4ea97fa640 1 -- 172.21.15.22:0/1012093299 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e9c032ca0 con 0x7f4eac108210 2024-09-19T08:05:28.649 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.725+0000 7f4eb2308640 1 -- 172.21.15.22:0/1012093299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eac19bef0 con 0x7f4eac108210 2024-09-19T08:05:28.649 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.725+0000 7f4ea97fa640 1 -- 172.21.15.22:0/1012093299 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e9c037040 con 0x7f4eac108210 2024-09-19T08:05:28.649 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.725+0000 7f4eb2308640 1 -- 172.21.15.22:0/1012093299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eac19ebe0 con 0x7f4eac108210 2024-09-19T08:05:28.650 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.725+0000 7f4ea97fa640 1 -- 172.21.15.22:0/1012093299 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4e9c045070 con 0x7f4eac108210 2024-09-19T08:05:28.650 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.725+0000 7f4ea97fa640 1 -- 172.21.15.22:0/1012093299 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4e9c047af0 con 0x7f4eac108210 2024-09-19T08:05:28.650 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.726+0000 7f4eb2308640 1 -- 172.21.15.22:0/1012093299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eac108670 con 0x7f4eac108210 2024-09-19T08:05:28.650 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.730+0000 7f4ea97fa640 1 -- 172.21.15.22:0/1012093299 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4e9c03db10 con 0x7f4eac108210 2024-09-19T08:05:28.650 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.791+0000 7f4eb2308640 1 -- 172.21.15.22:0/1012093299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4eac19f150 con 0x7f4eac108210 2024-09-19T08:05:28.650 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.792+0000 7f4ea97fa640 1 -- 172.21.15.22:0/1012093299 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f4e9c03d430 con 0x7f4eac108210 2024-09-19T08:05:28.650 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.792+0000 7f4eb2308640 1 -- 172.21.15.22:0/1012093299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4eac108210 msgr2=0x7f4eac19b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:28.650 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.793+0000 7f4eb2308640 1 --2- 172.21.15.22:0/1012093299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4eac108210 0x7f4eac19b720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0097f0 tx=0x7f4e9c030ec0 comp rx=0 tx=0).stop 2024-09-19T08:05:28.650 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.793+0000 7f4eb2308640 1 -- 172.21.15.22:0/1012093299 shutdown_connections 2024-09-19T08:05:28.650 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.793+0000 7f4eb2308640 1 --2- 172.21.15.22:0/1012093299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4eac108210 0x7f4eac19b720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:28.650 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.793+0000 7f4eb2308640 1 -- 172.21.15.22:0/1012093299 >> 172.21.15.22:0/1012093299 conn(0x7f4eac101d90 msgr2=0x7f4eac102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:28.650 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.793+0000 7f4eb2308640 1 -- 172.21.15.22:0/1012093299 shutdown_connections 2024-09-19T08:05:28.650 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:27.793+0000 7f4eb2308640 1 -- 172.21.15.22:0/1012093299 wait complete. 2024-09-19T08:05:28.650 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (1/15)... 2024-09-19T08:05:29.756 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:29 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:29.434+0000 7f1f78ab4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:05:29.757 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:29 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:29.504+0000 7f1f78ab4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:05:30.538 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:30 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:30.267+0000 7f1f78ab4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:05:30.538 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:30 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:30.388+0000 7f1f78ab4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:05:30.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:30 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:30.537+0000 7f1f78ab4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:05:30.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:30 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:30.600+0000 7f1f78ab4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:05:30.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:30 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:30.760+0000 7f1f78ab4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:05:30.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:30 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:30.833+0000 7f1f78ab4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:05:31.132 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:30 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:30.898+0000 7f1f78ab4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:05:31.132 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:31 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:31.070+0000 7f1f78ab4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:05:31.430 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:31 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3491607394' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T08:05:31.431 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:31 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:31.140+0000 7f1f78ab4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:05:31.431 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:31 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:31.217+0000 7f1f78ab4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:05:31.431 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:31 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:31.305+0000 7f1f78ab4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:05:31.431 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:31 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:31.367+0000 7f1f78ab4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:05:31.431 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:31 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:31.429+0000 7f1f78ab4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:05:31.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-19T08:05:31.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:05:31.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "c0ad60c6-765d-11ef-bceb-c7b262605968", 2024-09-19T08:05:31.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T08:05:31.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T08:05:31.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T08:05:31.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T08:05:31.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:31.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T08:05:31.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T08:05:31.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-09-19T08:05:31.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:05:31.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T08:05:31.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T08:05:31.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:05:31.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-19T08:05:31.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T08:05:31.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:05:31.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T08:05:31.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T08:05:31.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:31.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T08:05:31.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:05:31.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T08:05:31.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T08:05:31.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T08:05:31.722 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T08:05:31.722 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T08:05:31.722 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T08:05:31.722 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:31.722 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T08:05:31.723 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T08:05:31.723 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T08:05:31.723 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T08:05:31.723 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T08:05:31.723 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T08:05:31.723 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T08:05:31.723 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T08:05:31.723 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T08:05:31.723 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:31.723 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T08:05:31.723 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:05:31.723 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T08:05:19:476769+0000", 2024-09-19T08:05:31.723 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T08:05:31.723 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T08:05:31.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:31.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T08:05:31.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T08:05:31.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T08:05:31.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T08:05:31.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T08:05:31.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T08:05:31.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T08:05:31.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:05:31.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:05:31.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:31.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T08:05:31.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:05:31.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T08:05:19.477539+0000", 2024-09-19T08:05:31.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:05:31.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:31.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T08:05:31.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:05:31.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.058+0000 7f2621e6c640 1 Processor -- start 2024-09-19T08:05:31.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.058+0000 7f2621e6c640 1 -- start start 2024-09-19T08:05:31.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.058+0000 7f2621e6c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f261c1081a0 0x7f261c10a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:31.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.058+0000 7f2621e6c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f261c07be70 con 0x7f261c1081a0 2024-09-19T08:05:31.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.059+0000 7f261b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f261c1081a0 0x7f261c10a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:31.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.059+0000 7f261b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f261c1081a0 0x7f261c10a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38670/0 (socket says 172.21.15.22:38670) 2024-09-19T08:05:31.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.059+0000 7f261b7fe640 1 -- 172.21.15.22:0/1974518686 learned_addr learned my addr 172.21.15.22:0/1974518686 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:31.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.059+0000 7f261b7fe640 1 -- 172.21.15.22:0/1974518686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261c10aad0 con 0x7f261c1081a0 2024-09-19T08:05:31.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.060+0000 7f261b7fe640 1 --2- 172.21.15.22:0/1974518686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f261c1081a0 0x7f261c10a590 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f260c009820 tx=0x7f260c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78da23ce8f8d5b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:31.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.060+0000 7f261affd640 1 -- 172.21.15.22:0/1974518686 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f260c037070 con 0x7f261c1081a0 2024-09-19T08:05:31.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.060+0000 7f261affd640 1 -- 172.21.15.22:0/1974518686 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f260c032900 con 0x7f261c1081a0 2024-09-19T08:05:31.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.060+0000 7f261affd640 1 -- 172.21.15.22:0/1974518686 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f260c030850 con 0x7f261c1081a0 2024-09-19T08:05:31.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.061+0000 7f2621e6c640 1 -- 172.21.15.22:0/1974518686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f261c1081a0 msgr2=0x7f261c10a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:31.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.061+0000 7f2621e6c640 1 --2- 172.21.15.22:0/1974518686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f261c1081a0 0x7f261c10a590 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f260c009820 tx=0x7f260c02a270 comp rx=0 tx=0).stop 2024-09-19T08:05:31.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.061+0000 7f2621e6c640 1 -- 172.21.15.22:0/1974518686 shutdown_connections 2024-09-19T08:05:31.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.061+0000 7f2621e6c640 1 --2- 172.21.15.22:0/1974518686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f261c1081a0 0x7f261c10a590 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:31.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.061+0000 7f2621e6c640 1 -- 172.21.15.22:0/1974518686 >> 172.21.15.22:0/1974518686 conn(0x7f261c101e20 msgr2=0x7f261c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:31.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.061+0000 7f2621e6c640 1 -- 172.21.15.22:0/1974518686 shutdown_connections 2024-09-19T08:05:31.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.061+0000 7f2621e6c640 1 -- 172.21.15.22:0/1974518686 wait complete. 2024-09-19T08:05:31.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.061+0000 7f2621e6c640 1 Processor -- start 2024-09-19T08:05:31.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.061+0000 7f2621e6c640 1 -- start start 2024-09-19T08:05:31.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.062+0000 7f2621e6c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f261c1081a0 0x7f261c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:31.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.062+0000 7f2621e6c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f261c10bb10 con 0x7f261c1081a0 2024-09-19T08:05:31.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.062+0000 7f261b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f261c1081a0 0x7f261c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:31.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.062+0000 7f261b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f261c1081a0 0x7f261c19b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38680/0 (socket says 172.21.15.22:38680) 2024-09-19T08:05:31.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.062+0000 7f261b7fe640 1 -- 172.21.15.22:0/3491607394 learned_addr learned my addr 172.21.15.22:0/3491607394 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:31.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.062+0000 7f261b7fe640 1 -- 172.21.15.22:0/3491607394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261c19bc30 con 0x7f261c1081a0 2024-09-19T08:05:31.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.062+0000 7f261b7fe640 1 --2- 172.21.15.22:0/3491607394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f261c1081a0 0x7f261c19b6f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f260c0097f0 tx=0x7f260c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:31.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.063+0000 7f26197fa640 1 -- 172.21.15.22:0/3491607394 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f260c042070 con 0x7f261c1081a0 2024-09-19T08:05:31.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.063+0000 7f2621e6c640 1 -- 172.21.15.22:0/3491607394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261c19bec0 con 0x7f261c1081a0 2024-09-19T08:05:31.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.063+0000 7f26197fa640 1 -- 172.21.15.22:0/3491607394 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f260c032ca0 con 0x7f261c1081a0 2024-09-19T08:05:31.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.063+0000 7f26197fa640 1 -- 172.21.15.22:0/3491607394 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f260c037040 con 0x7f261c1081a0 2024-09-19T08:05:31.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.063+0000 7f2621e6c640 1 -- 172.21.15.22:0/3491607394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f261c19ebb0 con 0x7f261c1081a0 2024-09-19T08:05:31.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.063+0000 7f26197fa640 1 -- 172.21.15.22:0/3491607394 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f260c045070 con 0x7f261c1081a0 2024-09-19T08:05:31.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.064+0000 7f26197fa640 1 -- 172.21.15.22:0/3491607394 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f260c047800 con 0x7f261c1081a0 2024-09-19T08:05:31.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.064+0000 7f2621e6c640 1 -- 172.21.15.22:0/3491607394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f261c108600 con 0x7f261c1081a0 2024-09-19T08:05:31.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.068+0000 7f26197fa640 1 -- 172.21.15.22:0/3491607394 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f260c03db10 con 0x7f261c1081a0 2024-09-19T08:05:31.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.131+0000 7f2621e6c640 1 -- 172.21.15.22:0/3491607394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f261c19ed80 con 0x7f261c1081a0 2024-09-19T08:05:31.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.131+0000 7f26197fa640 1 -- 172.21.15.22:0/3491607394 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f260c03d770 con 0x7f261c1081a0 2024-09-19T08:05:31.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.132+0000 7f2621e6c640 1 -- 172.21.15.22:0/3491607394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f261c1081a0 msgr2=0x7f261c19b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:31.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.132+0000 7f2621e6c640 1 --2- 172.21.15.22:0/3491607394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f261c1081a0 0x7f261c19b6f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f260c0097f0 tx=0x7f260c030ec0 comp rx=0 tx=0).stop 2024-09-19T08:05:31.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.132+0000 7f2621e6c640 1 -- 172.21.15.22:0/3491607394 shutdown_connections 2024-09-19T08:05:31.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.132+0000 7f2621e6c640 1 --2- 172.21.15.22:0/3491607394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f261c1081a0 0x7f261c19b6f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:31.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.132+0000 7f2621e6c640 1 -- 172.21.15.22:0/3491607394 >> 172.21.15.22:0/3491607394 conn(0x7f261c101e20 msgr2=0x7f261c102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:31.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.132+0000 7f2621e6c640 1 -- 172.21.15.22:0/3491607394 shutdown_connections 2024-09-19T08:05:31.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:31.132+0000 7f2621e6c640 1 -- 172.21.15.22:0/3491607394 wait complete. 2024-09-19T08:05:31.728 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (2/15)... 2024-09-19T08:05:32.757 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:32 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:32.265+0000 7f1f78ab4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:05:32.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:32 smithi022 ceph-mon[30963]: Activating manager daemon x 2024-09-19T08:05:32.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:32 smithi022 ceph-mon[30963]: mgrmap e2: x(active, starting, since 0.00309112s) 2024-09-19T08:05:32.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:32 smithi022 ceph-mon[30963]: from='mgr.14100 172.21.15.22:0/1151737757' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:05:32.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:32 smithi022 ceph-mon[30963]: from='mgr.14100 172.21.15.22:0/1151737757' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:05:32.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:32 smithi022 ceph-mon[30963]: from='mgr.14100 172.21.15.22:0/1151737757' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:05:32.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:32 smithi022 ceph-mon[30963]: from='mgr.14100 172.21.15.22:0/1151737757' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:05:32.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:32 smithi022 ceph-mon[30963]: from='mgr.14100 172.21.15.22:0/1151737757' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:05:32.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:32 smithi022 ceph-mon[30963]: Manager daemon x is now available 2024-09-19T08:05:32.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:32 smithi022 ceph-mon[30963]: from='mgr.14100 172.21.15.22:0/1151737757' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:05:32.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:32 smithi022 ceph-mon[30963]: from='mgr.14100 172.21.15.22:0/1151737757' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:05:32.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:32 smithi022 ceph-mon[30963]: from='mgr.14100 172.21.15.22:0/1151737757' entity='mgr.x' 2024-09-19T08:05:32.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:32 smithi022 ceph-mon[30963]: from='mgr.14100 172.21.15.22:0/1151737757' entity='mgr.x' 2024-09-19T08:05:32.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:32 smithi022 ceph-mon[30963]: from='mgr.14100 172.21.15.22:0/1151737757' entity='mgr.x' 2024-09-19T08:05:34.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:34 smithi022 ceph-mon[30963]: mgrmap e3: x(active, since 1.00776s) 2024-09-19T08:05:34.944 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-19T08:05:34.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:05:34.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "c0ad60c6-765d-11ef-bceb-c7b262605968", 2024-09-19T08:05:34.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T08:05:34.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T08:05:34.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T08:05:34.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T08:05:34.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:34.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T08:05:34.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T08:05:34.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-09-19T08:05:34.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:05:34.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T08:05:34.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T08:05:34.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:05:34.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-19T08:05:34.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T08:05:34.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:05:34.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T08:05:34.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T08:05:34.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:34.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T08:05:34.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:05:34.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T08:05:34.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T08:05:34.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T08:05:34.955 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T08:05:34.955 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T08:05:34.955 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T08:05:34.955 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:34.955 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T08:05:34.955 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T08:05:34.955 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T08:05:34.955 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T08:05:34.956 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T08:05:34.956 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T08:05:34.956 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T08:05:34.956 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T08:05:34.956 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T08:05:34.956 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:34.956 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T08:05:34.956 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:05:34.956 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T08:05:19:476769+0000", 2024-09-19T08:05:34.956 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T08:05:34.956 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T08:05:34.956 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:34.956 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T08:05:34.957 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T08:05:34.957 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T08:05:34.957 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T08:05:34.957 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T08:05:34.957 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T08:05:34.957 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T08:05:34.957 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:05:34.957 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:05:34.957 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:34.957 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T08:05:34.957 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:05:34.957 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T08:05:19.477539+0000", 2024-09-19T08:05:34.957 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:05:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:05:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T08:05:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:05:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.141+0000 7fba79447640 1 Processor -- start 2024-09-19T08:05:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.141+0000 7fba79447640 1 -- start start 2024-09-19T08:05:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.142+0000 7fba79447640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba741059f0 0x7fba74105df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.142+0000 7fba79447640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba741063c0 con 0x7fba741059f0 2024-09-19T08:05:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.142+0000 7fba72ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba741059f0 0x7fba74105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.142+0000 7fba72ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba741059f0 0x7fba74105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38760/0 (socket says 172.21.15.22:38760) 2024-09-19T08:05:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.142+0000 7fba72ffd640 1 -- 172.21.15.22:0/1803315478 learned_addr learned my addr 172.21.15.22:0/1803315478 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.142+0000 7fba72ffd640 1 -- 172.21.15.22:0/1803315478 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba74106540 con 0x7fba741059f0 2024-09-19T08:05:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.143+0000 7fba72ffd640 1 --2- 172.21.15.22:0/1803315478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba741059f0 0x7fba74105df0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fba64009e80 tx=0x7fba6402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d15d719f5a11c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.143+0000 7fba71ffb640 1 -- 172.21.15.22:0/1803315478 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba64036070 con 0x7fba741059f0 2024-09-19T08:05:34.958 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.143+0000 7fba71ffb640 1 -- 172.21.15.22:0/1803315478 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fba640318c0 con 0x7fba741059f0 2024-09-19T08:05:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.143+0000 7fba71ffb640 1 -- 172.21.15.22:0/1803315478 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba64007ab0 con 0x7fba741059f0 2024-09-19T08:05:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.144+0000 7fba79447640 1 -- 172.21.15.22:0/1803315478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba741059f0 msgr2=0x7fba74105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.144+0000 7fba79447640 1 --2- 172.21.15.22:0/1803315478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba741059f0 0x7fba74105df0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fba64009e80 tx=0x7fba6402a7a0 comp rx=0 tx=0).stop 2024-09-19T08:05:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.144+0000 7fba79447640 1 -- 172.21.15.22:0/1803315478 shutdown_connections 2024-09-19T08:05:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.144+0000 7fba79447640 1 --2- 172.21.15.22:0/1803315478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba741059f0 0x7fba74105df0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.144+0000 7fba79447640 1 -- 172.21.15.22:0/1803315478 >> 172.21.15.22:0/1803315478 conn(0x7fba74101560 msgr2=0x7fba74103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.144+0000 7fba79447640 1 -- 172.21.15.22:0/1803315478 shutdown_connections 2024-09-19T08:05:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.144+0000 7fba79447640 1 -- 172.21.15.22:0/1803315478 wait complete. 2024-09-19T08:05:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.145+0000 7fba79447640 1 Processor -- start 2024-09-19T08:05:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.145+0000 7fba79447640 1 -- start start 2024-09-19T08:05:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.145+0000 7fba79447640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba741059f0 0x7fba74199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:34.959 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.145+0000 7fba79447640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba74106860 con 0x7fba741059f0 2024-09-19T08:05:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.145+0000 7fba72ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba741059f0 0x7fba74199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.145+0000 7fba72ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba741059f0 0x7fba74199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38772/0 (socket says 172.21.15.22:38772) 2024-09-19T08:05:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.145+0000 7fba72ffd640 1 -- 172.21.15.22:0/3861956664 learned_addr learned my addr 172.21.15.22:0/3861956664 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.145+0000 7fba72ffd640 1 -- 172.21.15.22:0/3861956664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba74199a80 con 0x7fba741059f0 2024-09-19T08:05:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.146+0000 7fba72ffd640 1 --2- 172.21.15.22:0/3861956664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba741059f0 0x7fba74199540 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fba64007ce0 tx=0x7fba64031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.146+0000 7fba5bfff640 1 -- 172.21.15.22:0/3861956664 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba64036050 con 0x7fba741059f0 2024-09-19T08:05:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.146+0000 7fba5bfff640 1 -- 172.21.15.22:0/3861956664 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fba64042070 con 0x7fba741059f0 2024-09-19T08:05:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.146+0000 7fba79447640 1 -- 172.21.15.22:0/3861956664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba74199d10 con 0x7fba741059f0 2024-09-19T08:05:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.146+0000 7fba5bfff640 1 -- 172.21.15.22:0/3861956664 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba6403d8a0 con 0x7fba741059f0 2024-09-19T08:05:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.146+0000 7fba79447640 1 -- 172.21.15.22:0/3861956664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7419ca00 con 0x7fba741059f0 2024-09-19T08:05:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.147+0000 7fba5bfff640 1 -- 172.21.15.22:0/3861956664 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7fba6403db50 con 0x7fba741059f0 2024-09-19T08:05:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.147+0000 7fba5bfff640 1 --2- 172.21.15.22:0/3861956664 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7fba5c03d4f0 0x7fba5c03f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:34.960 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.147+0000 7fba79447640 1 -- 172.21.15.22:0/3861956664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba74105df0 con 0x7fba741059f0 2024-09-19T08:05:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.147+0000 7fba5bfff640 1 -- 172.21.15.22:0/3861956664 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fba64007e00 con 0x7fba741059f0 2024-09-19T08:05:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.147+0000 7fba727fc640 1 --2- 172.21.15.22:0/3861956664 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7fba5c03d4f0 0x7fba5c03f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.148+0000 7fba727fc640 1 --2- 172.21.15.22:0/3861956664 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7fba5c03d4f0 0x7fba5c03f9b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fba680098c0 tx=0x7fba68006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.152+0000 7fba5bfff640 1 -- 172.21.15.22:0/3861956664 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fba64039de0 con 0x7fba741059f0 2024-09-19T08:05:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.274+0000 7fba5bfff640 1 -- 172.21.15.22:0/3861956664 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fba64047980 con 0x7fba741059f0 2024-09-19T08:05:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.397+0000 7fba79447640 1 -- 172.21.15.22:0/3861956664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fba7402d9a0 con 0x7fba741059f0 2024-09-19T08:05:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.397+0000 7fba5bfff640 1 -- 172.21.15.22:0/3861956664 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fba6406a980 con 0x7fba741059f0 2024-09-19T08:05:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.401+0000 7fba79447640 1 -- 172.21.15.22:0/3861956664 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7fba5c03d4f0 msgr2=0x7fba5c03f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.401+0000 7fba79447640 1 --2- 172.21.15.22:0/3861956664 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7fba5c03d4f0 0x7fba5c03f9b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fba680098c0 tx=0x7fba68006db0 comp rx=0 tx=0).stop 2024-09-19T08:05:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.401+0000 7fba79447640 1 -- 172.21.15.22:0/3861956664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba741059f0 msgr2=0x7fba74199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.401+0000 7fba79447640 1 --2- 172.21.15.22:0/3861956664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba741059f0 0x7fba74199540 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fba64007ce0 tx=0x7fba64031ca0 comp rx=0 tx=0).stop 2024-09-19T08:05:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.401+0000 7fba79447640 1 -- 172.21.15.22:0/3861956664 shutdown_connections 2024-09-19T08:05:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.401+0000 7fba79447640 1 --2- 172.21.15.22:0/3861956664 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7fba5c03d4f0 0x7fba5c03f9b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:34.961 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.401+0000 7fba79447640 1 --2- 172.21.15.22:0/3861956664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba741059f0 0x7fba74199540 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:34.962 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.401+0000 7fba79447640 1 -- 172.21.15.22:0/3861956664 >> 172.21.15.22:0/3861956664 conn(0x7fba74101560 msgr2=0x7fba741025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:34.962 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.401+0000 7fba79447640 1 -- 172.21.15.22:0/3861956664 shutdown_connections 2024-09-19T08:05:34.962 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:34.401+0000 7fba79447640 1 -- 172.21.15.22:0/3861956664 wait complete. 2024-09-19T08:05:34.962 INFO:teuthology.orchestra.run.smithi022.stdout:mgr is available 2024-09-19T08:05:35.358 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:35 smithi022 ceph-mon[30963]: mgrmap e4: x(active, since 2s) 2024-09-19T08:05:35.359 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:35 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3861956664' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T08:05:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-19T08:05:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T08:05:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:05:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2024-09-19T08:05:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T08:05:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T08:05:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T08:05:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T08:05:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-19T08:05:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T08:05:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T08:05:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-19T08:05:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T08:05:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T08:05:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T08:05:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T08:05:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T08:05:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T08:05:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.364+0000 7f157c10e640 1 Processor -- start 2024-09-19T08:05:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.364+0000 7f157c10e640 1 -- start start 2024-09-19T08:05:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.364+0000 7f157c10e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157407de70 0x7f157407c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.364+0000 7f157c10e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f157407c8f0 con 0x7f157407de70 2024-09-19T08:05:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.365+0000 7f1579eaa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157407de70 0x7f157407c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.365+0000 7f1579eaa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157407de70 0x7f157407c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38776/0 (socket says 172.21.15.22:38776) 2024-09-19T08:05:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.365+0000 7f1579eaa640 1 -- 172.21.15.22:0/3407537681 learned_addr learned my addr 172.21.15.22:0/3407537681 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.365+0000 7f1579eaa640 1 -- 172.21.15.22:0/3407537681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157407ca70 con 0x7f157407de70 2024-09-19T08:05:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.366+0000 7f1579eaa640 1 --2- 172.21.15.22:0/3407537681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157407de70 0x7f157407c320 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1570009820 tx=0x7f157002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514175f17c480641 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.366+0000 7f1578ea8640 1 -- 172.21.15.22:0/3407537681 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1570037070 con 0x7f157407de70 2024-09-19T08:05:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.366+0000 7f1578ea8640 1 -- 172.21.15.22:0/3407537681 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1570032900 con 0x7f157407de70 2024-09-19T08:05:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.366+0000 7f1578ea8640 1 -- 172.21.15.22:0/3407537681 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1570030850 con 0x7f157407de70 2024-09-19T08:05:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.366+0000 7f157c10e640 1 -- 172.21.15.22:0/3407537681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157407de70 msgr2=0x7f157407c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.366+0000 7f157c10e640 1 --2- 172.21.15.22:0/3407537681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157407de70 0x7f157407c320 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1570009820 tx=0x7f157002a270 comp rx=0 tx=0).stop 2024-09-19T08:05:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.366+0000 7f157c10e640 1 -- 172.21.15.22:0/3407537681 shutdown_connections 2024-09-19T08:05:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.366+0000 7f157c10e640 1 --2- 172.21.15.22:0/3407537681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157407de70 0x7f157407c320 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.366+0000 7f157c10e640 1 -- 172.21.15.22:0/3407537681 >> 172.21.15.22:0/3407537681 conn(0x7f1574102ef0 msgr2=0x7f1574105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.367+0000 7f157c10e640 1 -- 172.21.15.22:0/3407537681 shutdown_connections 2024-09-19T08:05:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.367+0000 7f157c10e640 1 -- 172.21.15.22:0/3407537681 wait complete. 2024-09-19T08:05:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.367+0000 7f157c10e640 1 Processor -- start 2024-09-19T08:05:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.367+0000 7f157c10e640 1 -- start start 2024-09-19T08:05:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.367+0000 7f157c10e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157407de70 0x7f157419d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.367+0000 7f157c10e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f157410dd10 con 0x7f157407de70 2024-09-19T08:05:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.367+0000 7f1579eaa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157407de70 0x7f157419d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.367+0000 7f1579eaa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157407de70 0x7f157419d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38788/0 (socket says 172.21.15.22:38788) 2024-09-19T08:05:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.367+0000 7f1579eaa640 1 -- 172.21.15.22:0/20705020 learned_addr learned my addr 172.21.15.22:0/20705020 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.368+0000 7f1579eaa640 1 -- 172.21.15.22:0/20705020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157419dd90 con 0x7f157407de70 2024-09-19T08:05:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.368+0000 7f1579eaa640 1 --2- 172.21.15.22:0/20705020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157407de70 0x7f157419d850 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1570032ce0 tx=0x7f1570032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.368+0000 7f156affd640 1 -- 172.21.15.22:0/20705020 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1570042070 con 0x7f157407de70 2024-09-19T08:05:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.368+0000 7f156affd640 1 -- 172.21.15.22:0/20705020 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f15700308c0 con 0x7f157407de70 2024-09-19T08:05:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.368+0000 7f157c10e640 1 -- 172.21.15.22:0/20705020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157419e020 con 0x7f157407de70 2024-09-19T08:05:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.368+0000 7f156affd640 1 -- 172.21.15.22:0/20705020 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1570037040 con 0x7f157407de70 2024-09-19T08:05:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.368+0000 7f157c10e640 1 -- 172.21.15.22:0/20705020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15741a0d10 con 0x7f157407de70 2024-09-19T08:05:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.369+0000 7f156affd640 1 -- 172.21.15.22:0/20705020 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f1570045070 con 0x7f157407de70 2024-09-19T08:05:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.369+0000 7f156affd640 1 --2- 172.21.15.22:0/20705020 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f15600391b0 0x7f156003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.369+0000 7f157c10e640 1 -- 172.21.15.22:0/20705020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f157407c3a0 con 0x7f157407de70 2024-09-19T08:05:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.370+0000 7f156affd640 1 -- 172.21.15.22:0/20705020 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f157006c1e0 con 0x7f157407de70 2024-09-19T08:05:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.370+0000 7f15796a9640 1 --2- 172.21.15.22:0/20705020 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f15600391b0 0x7f156003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.370+0000 7f15796a9640 1 --2- 172.21.15.22:0/20705020 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f15600391b0 0x7f156003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f15640098c0 tx=0x7f1564006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.374+0000 7f156affd640 1 -- 172.21.15.22:0/20705020 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f157004b170 con 0x7f157407de70 2024-09-19T08:05:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.549+0000 7f157c10e640 1 -- 172.21.15.22:0/20705020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f15741a2510 con 0x7f157407de70 2024-09-19T08:05:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.550+0000 7f156affd640 1 -- 172.21.15.22:0/20705020 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f157004b3d0 con 0x7f157407de70 2024-09-19T08:05:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.552+0000 7f157c10e640 1 -- 172.21.15.22:0/20705020 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f15600391b0 msgr2=0x7f156003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.552+0000 7f157c10e640 1 --2- 172.21.15.22:0/20705020 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f15600391b0 0x7f156003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f15640098c0 tx=0x7f1564006db0 comp rx=0 tx=0).stop 2024-09-19T08:05:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.552+0000 7f157c10e640 1 -- 172.21.15.22:0/20705020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157407de70 msgr2=0x7f157419d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.552+0000 7f157c10e640 1 --2- 172.21.15.22:0/20705020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157407de70 0x7f157419d850 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1570032ce0 tx=0x7f1570032d10 comp rx=0 tx=0).stop 2024-09-19T08:05:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.552+0000 7f157c10e640 1 -- 172.21.15.22:0/20705020 shutdown_connections 2024-09-19T08:05:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.552+0000 7f157c10e640 1 --2- 172.21.15.22:0/20705020 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f15600391b0 0x7f156003b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.552+0000 7f157c10e640 1 --2- 172.21.15.22:0/20705020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157407de70 0x7f157419d850 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.552+0000 7f157c10e640 1 -- 172.21.15.22:0/20705020 >> 172.21.15.22:0/20705020 conn(0x7f1574102ef0 msgr2=0x7f1574103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.552+0000 7f157c10e640 1 -- 172.21.15.22:0/20705020 shutdown_connections 2024-09-19T08:05:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:35.552+0000 7f157c10e640 1 -- 172.21.15.22:0/20705020 wait complete. 2024-09-19T08:05:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling cephadm module... 2024-09-19T08:05:36.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:36 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/20705020' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T08:05:37.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:37 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3595886659' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T08:05:37.420 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: ignoring --setuser ceph since I am not root 2024-09-19T08:05:37.420 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: ignoring --setgroup ceph since I am not root 2024-09-19T08:05:37.420 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:37.418+0000 7f64b6c11040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:05:37.705 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:37.478+0000 7f64b6c11040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:05:37.705 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:37.634+0000 7f64b6c11040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:05:37.705 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:37.704+0000 7f64b6c11040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:05:37.843 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.538+0000 7f2c3ab98640 1 Processor -- start 2024-09-19T08:05:37.843 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.538+0000 7f2c3ab98640 1 -- start start 2024-09-19T08:05:37.844 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.539+0000 7f2c3ab98640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c341081e0 0x7f2c3410a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:37.844 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.539+0000 7f2c3ab98640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3407be50 con 0x7f2c341081e0 2024-09-19T08:05:37.844 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.539+0000 7f2c38934640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c341081e0 0x7f2c3410a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:37.844 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.539+0000 7f2c38934640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c341081e0 0x7f2c3410a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38800/0 (socket says 172.21.15.22:38800) 2024-09-19T08:05:37.844 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.539+0000 7f2c38934640 1 -- 172.21.15.22:0/178820339 learned_addr learned my addr 172.21.15.22:0/178820339 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:37.844 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.539+0000 7f2c38934640 1 -- 172.21.15.22:0/178820339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3410ab10 con 0x7f2c341081e0 2024-09-19T08:05:37.844 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.540+0000 7f2c38934640 1 --2- 172.21.15.22:0/178820339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c341081e0 0x7f2c3410a5d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009e80 tx=0x7f2c2c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fef5475339e92a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:37.844 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.540+0000 7f2c337fe640 1 -- 172.21.15.22:0/178820339 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c2c036070 con 0x7f2c341081e0 2024-09-19T08:05:37.844 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.540+0000 7f2c337fe640 1 -- 172.21.15.22:0/178820339 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c2c0318c0 con 0x7f2c341081e0 2024-09-19T08:05:37.844 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.540+0000 7f2c337fe640 1 -- 172.21.15.22:0/178820339 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c2c007ab0 con 0x7f2c341081e0 2024-09-19T08:05:37.845 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.541+0000 7f2c3ab98640 1 -- 172.21.15.22:0/178820339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c341081e0 msgr2=0x7f2c3410a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:37.845 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.541+0000 7f2c3ab98640 1 --2- 172.21.15.22:0/178820339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c341081e0 0x7f2c3410a5d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009e80 tx=0x7f2c2c02a7a0 comp rx=0 tx=0).stop 2024-09-19T08:05:37.845 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.541+0000 7f2c3ab98640 1 -- 172.21.15.22:0/178820339 shutdown_connections 2024-09-19T08:05:37.845 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.541+0000 7f2c3ab98640 1 --2- 172.21.15.22:0/178820339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c341081e0 0x7f2c3410a5d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:37.845 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.541+0000 7f2c3ab98640 1 -- 172.21.15.22:0/178820339 >> 172.21.15.22:0/178820339 conn(0x7f2c34101da0 msgr2=0x7f2c341041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:37.845 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.541+0000 7f2c3ab98640 1 -- 172.21.15.22:0/178820339 shutdown_connections 2024-09-19T08:05:37.845 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.541+0000 7f2c3ab98640 1 -- 172.21.15.22:0/178820339 wait complete. 2024-09-19T08:05:37.845 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.541+0000 7f2c3ab98640 1 Processor -- start 2024-09-19T08:05:37.845 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.542+0000 7f2c3ab98640 1 -- start start 2024-09-19T08:05:37.845 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.542+0000 7f2c3ab98640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c341081e0 0x7f2c3419b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:37.846 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.542+0000 7f2c3ab98640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3410bb50 con 0x7f2c341081e0 2024-09-19T08:05:37.846 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.542+0000 7f2c38934640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c341081e0 0x7f2c3419b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:37.846 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.542+0000 7f2c38934640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c341081e0 0x7f2c3419b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38806/0 (socket says 172.21.15.22:38806) 2024-09-19T08:05:37.846 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.542+0000 7f2c38934640 1 -- 172.21.15.22:0/3595886659 learned_addr learned my addr 172.21.15.22:0/3595886659 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:37.846 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.542+0000 7f2c38934640 1 -- 172.21.15.22:0/3595886659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3419bc50 con 0x7f2c341081e0 2024-09-19T08:05:37.846 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.542+0000 7f2c38934640 1 --2- 172.21.15.22:0/3595886659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c341081e0 0x7f2c3419b710 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c02acc0 tx=0x7f2c2c031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:37.846 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.542+0000 7f2c31ffb640 1 -- 172.21.15.22:0/3595886659 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c2c036050 con 0x7f2c341081e0 2024-09-19T08:05:37.846 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.543+0000 7f2c3ab98640 1 -- 172.21.15.22:0/3595886659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3419bee0 con 0x7f2c341081e0 2024-09-19T08:05:37.846 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.543+0000 7f2c31ffb640 1 -- 172.21.15.22:0/3595886659 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c2c037040 con 0x7f2c341081e0 2024-09-19T08:05:37.846 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.543+0000 7f2c31ffb640 1 -- 172.21.15.22:0/3595886659 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c2c03d920 con 0x7f2c341081e0 2024-09-19T08:05:37.846 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.543+0000 7f2c3ab98640 1 -- 172.21.15.22:0/3595886659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3419ebd0 con 0x7f2c341081e0 2024-09-19T08:05:37.846 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.544+0000 7f2c31ffb640 1 -- 172.21.15.22:0/3595886659 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f2c2c044020 con 0x7f2c341081e0 2024-09-19T08:05:37.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.544+0000 7f2c31ffb640 1 --2- 172.21.15.22:0/3595886659 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f2c20039160 0x7f2c2003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:37.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.544+0000 7f2c3ab98640 1 -- 172.21.15.22:0/3595886659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c34108640 con 0x7f2c341081e0 2024-09-19T08:05:37.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.544+0000 7f2c31ffb640 1 -- 172.21.15.22:0/3595886659 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2c2c042020 con 0x7f2c341081e0 2024-09-19T08:05:37.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.544+0000 7f2c33fff640 1 --2- 172.21.15.22:0/3595886659 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f2c20039160 0x7f2c2003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:37.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.544+0000 7f2c33fff640 1 --2- 172.21.15.22:0/3595886659 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f2c20039160 0x7f2c2003b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0098c0 tx=0x7f2c1c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:37.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.549+0000 7f2c31ffb640 1 -- 172.21.15.22:0/3595886659 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2c2c04b020 con 0x7f2c341081e0 2024-09-19T08:05:37.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:36.775+0000 7f2c3ab98640 1 -- 172.21.15.22:0/3595886659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f2c3419f250 con 0x7f2c341081e0 2024-09-19T08:05:37.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:37.277+0000 7f2c31ffb640 1 -- 172.21.15.22:0/3595886659 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f2c2c0371f0 con 0x7f2c341081e0 2024-09-19T08:05:37.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:37.278+0000 7f2c31ffb640 1 -- 172.21.15.22:0/3595886659 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f2c2c046790 con 0x7f2c341081e0 2024-09-19T08:05:37.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:37.279+0000 7f2c3ab98640 1 -- 172.21.15.22:0/3595886659 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f2c20039160 msgr2=0x7f2c2003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:37.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:37.279+0000 7f2c3ab98640 1 --2- 172.21.15.22:0/3595886659 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f2c20039160 0x7f2c2003b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0098c0 tx=0x7f2c1c006db0 comp rx=0 tx=0).stop 2024-09-19T08:05:37.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:37.279+0000 7f2c3ab98640 1 -- 172.21.15.22:0/3595886659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c341081e0 msgr2=0x7f2c3419b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:37.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:37.279+0000 7f2c3ab98640 1 --2- 172.21.15.22:0/3595886659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c341081e0 0x7f2c3419b710 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c02acc0 tx=0x7f2c2c031ca0 comp rx=0 tx=0).stop 2024-09-19T08:05:37.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:37.280+0000 7f2c3ab98640 1 -- 172.21.15.22:0/3595886659 shutdown_connections 2024-09-19T08:05:37.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:37.280+0000 7f2c3ab98640 1 --2- 172.21.15.22:0/3595886659 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f2c20039160 0x7f2c2003b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:37.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:37.280+0000 7f2c3ab98640 1 --2- 172.21.15.22:0/3595886659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c341081e0 0x7f2c3419b710 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:37.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:37.280+0000 7f2c3ab98640 1 -- 172.21.15.22:0/3595886659 >> 172.21.15.22:0/3595886659 conn(0x7f2c34101da0 msgr2=0x7f2c34102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:37.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:37.280+0000 7f2c3ab98640 1 -- 172.21.15.22:0/3595886659 shutdown_connections 2024-09-19T08:05:37.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:37.280+0000 7f2c3ab98640 1 -- 172.21.15.22:0/3595886659 wait complete. 2024-09-19T08:05:37.970 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:37.872+0000 7f64b6c11040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:05:38.238 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:38 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:38.021+0000 7f64b6c11040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:05:38.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:38 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3595886659' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T08:05:38.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:38 smithi022 ceph-mon[30963]: mgrmap e5: x(active, since 5s) 2024-09-19T08:05:38.492 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:38 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:38.357+0000 7f64b6c11040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:05:38.492 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:38 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:38.421+0000 7f64b6c11040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:05:38.757 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:38 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:38.629+0000 7f64b6c11040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:05:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:05:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T08:05:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T08:05:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T08:05:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T08:05:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:05:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.242+0000 7f9be6cb6640 1 Processor -- start 2024-09-19T08:05:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.242+0000 7f9be6cb6640 1 -- start start 2024-09-19T08:05:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.242+0000 7f9be6cb6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be0107740 0x7f9be0107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.242+0000 7f9be6cb6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9be0108110 con 0x7f9be0107740 2024-09-19T08:05:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.242+0000 7f9be4a52640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be0107740 0x7f9be0107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.242+0000 7f9be4a52640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be0107740 0x7f9be0107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57378/0 (socket says 172.21.15.22:57378) 2024-09-19T08:05:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.242+0000 7f9be4a52640 1 -- 172.21.15.22:0/1012368686 learned_addr learned my addr 172.21.15.22:0/1012368686 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.243+0000 7f9be4a52640 1 -- 172.21.15.22:0/1012368686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be01088f0 con 0x7f9be0107740 2024-09-19T08:05:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.243+0000 7f9be4a52640 1 --2- 172.21.15.22:0/1012368686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be0107740 0x7f9be0107b40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009820 tx=0x7f9bd402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9278156cc9af5a4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.244+0000 7f9bdf7fe640 1 -- 172.21.15.22:0/1012368686 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bd4037070 con 0x7f9be0107740 2024-09-19T08:05:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.244+0000 7f9bdf7fe640 1 -- 172.21.15.22:0/1012368686 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9bd4032900 con 0x7f9be0107740 2024-09-19T08:05:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.244+0000 7f9bdf7fe640 1 -- 172.21.15.22:0/1012368686 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bd4030850 con 0x7f9be0107740 2024-09-19T08:05:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.244+0000 7f9be6cb6640 1 -- 172.21.15.22:0/1012368686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be0107740 msgr2=0x7f9be0107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.244+0000 7f9be6cb6640 1 --2- 172.21.15.22:0/1012368686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be0107740 0x7f9be0107b40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009820 tx=0x7f9bd402a270 comp rx=0 tx=0).stop 2024-09-19T08:05:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.244+0000 7f9be6cb6640 1 -- 172.21.15.22:0/1012368686 shutdown_connections 2024-09-19T08:05:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.244+0000 7f9be6cb6640 1 --2- 172.21.15.22:0/1012368686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be0107740 0x7f9be0107b40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.244+0000 7f9be6cb6640 1 -- 172.21.15.22:0/1012368686 >> 172.21.15.22:0/1012368686 conn(0x7f9be0102ef0 msgr2=0x7f9be0105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.244+0000 7f9be6cb6640 1 -- 172.21.15.22:0/1012368686 shutdown_connections 2024-09-19T08:05:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.244+0000 7f9be6cb6640 1 -- 172.21.15.22:0/1012368686 wait complete. 2024-09-19T08:05:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.245+0000 7f9be6cb6640 1 Processor -- start 2024-09-19T08:05:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.245+0000 7f9be6cb6640 1 -- start start 2024-09-19T08:05:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.245+0000 7f9be6cb6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be0107740 0x7f9be007db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.245+0000 7f9be6cb6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9be010bca0 con 0x7f9be0107740 2024-09-19T08:05:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.245+0000 7f9be4a52640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be0107740 0x7f9be007db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.245+0000 7f9be4a52640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be0107740 0x7f9be007db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57382/0 (socket says 172.21.15.22:57382) 2024-09-19T08:05:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.245+0000 7f9be4a52640 1 -- 172.21.15.22:0/3455600126 learned_addr learned my addr 172.21.15.22:0/3455600126 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.246+0000 7f9be4a52640 1 -- 172.21.15.22:0/3455600126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be007e0b0 con 0x7f9be0107740 2024-09-19T08:05:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.246+0000 7f9be4a52640 1 --2- 172.21.15.22:0/3455600126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be0107740 0x7f9be007db70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40097f0 tx=0x7f9bd4030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.246+0000 7f9bddffb640 1 -- 172.21.15.22:0/3455600126 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bd4042070 con 0x7f9be0107740 2024-09-19T08:05:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.246+0000 7f9bddffb640 1 -- 172.21.15.22:0/3455600126 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9bd4032ca0 con 0x7f9be0107740 2024-09-19T08:05:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.246+0000 7f9be6cb6640 1 -- 172.21.15.22:0/3455600126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be007c2a0 con 0x7f9be0107740 2024-09-19T08:05:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.246+0000 7f9bddffb640 1 -- 172.21.15.22:0/3455600126 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bd4037040 con 0x7f9be0107740 2024-09-19T08:05:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.246+0000 7f9be6cb6640 1 -- 172.21.15.22:0/3455600126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be007c7e0 con 0x7f9be0107740 2024-09-19T08:05:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.247+0000 7f9bddffb640 1 -- 172.21.15.22:0/3455600126 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f9bd4045070 con 0x7f9be0107740 2024-09-19T08:05:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.247+0000 7f9bddffb640 1 --2- 172.21.15.22:0/3455600126 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f9bd00391b0 0x7f9bd003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:39.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.248+0000 7f9bddffb640 1 -- 172.21.15.22:0/3455600126 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9bd406c110 con 0x7f9be0107740 2024-09-19T08:05:39.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.248+0000 7f9bdffff640 1 -- 172.21.15.22:0/3455600126 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f9bd00391b0 msgr2=0x7f9bd003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2990808963 2024-09-19T08:05:39.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.248+0000 7f9bdffff640 1 --2- 172.21.15.22:0/3455600126 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f9bd00391b0 0x7f9bd003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:05:39.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.248+0000 7f9be6cb6640 1 -- 172.21.15.22:0/3455600126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be0107bc0 con 0x7f9be0107740 2024-09-19T08:05:39.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.252+0000 7f9bddffb640 1 -- 172.21.15.22:0/3455600126 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9bd4045520 con 0x7f9be0107740 2024-09-19T08:05:39.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.449+0000 7f9bdffff640 1 -- 172.21.15.22:0/3455600126 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f9bd00391b0 msgr2=0x7f9bd003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2990808963 2024-09-19T08:05:39.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.449+0000 7f9bdffff640 1 --2- 172.21.15.22:0/3455600126 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f9bd00391b0 0x7f9bd003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:05:39.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.478+0000 7f9be6cb6640 1 -- 172.21.15.22:0/3455600126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9be01a17b0 con 0x7f9be0107740 2024-09-19T08:05:39.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.478+0000 7f9bddffb640 1 -- 172.21.15.22:0/3455600126 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f9bd403ccd0 con 0x7f9be0107740 2024-09-19T08:05:39.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.481+0000 7f9be6cb6640 1 -- 172.21.15.22:0/3455600126 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f9bd00391b0 msgr2=0x7f9bd003b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:05:39.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.481+0000 7f9be6cb6640 1 --2- 172.21.15.22:0/3455600126 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f9bd00391b0 0x7f9bd003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:39.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.481+0000 7f9be6cb6640 1 -- 172.21.15.22:0/3455600126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be0107740 msgr2=0x7f9be007db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:39.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.481+0000 7f9be6cb6640 1 --2- 172.21.15.22:0/3455600126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be0107740 0x7f9be007db70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40097f0 tx=0x7f9bd4030ec0 comp rx=0 tx=0).stop 2024-09-19T08:05:39.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.481+0000 7f9be6cb6640 1 -- 172.21.15.22:0/3455600126 shutdown_connections 2024-09-19T08:05:39.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.481+0000 7f9be6cb6640 1 --2- 172.21.15.22:0/3455600126 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7f9bd00391b0 0x7f9bd003b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:39.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.481+0000 7f9be6cb6640 1 --2- 172.21.15.22:0/3455600126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be0107740 0x7f9be007db70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:39.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.481+0000 7f9be6cb6640 1 -- 172.21.15.22:0/3455600126 >> 172.21.15.22:0/3455600126 conn(0x7f9be0102ef0 msgr2=0x7f9be0103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:39.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.481+0000 7f9be6cb6640 1 -- 172.21.15.22:0/3455600126 shutdown_connections 2024-09-19T08:05:39.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:38.481+0000 7f9be6cb6640 1 -- 172.21.15.22:0/3455600126 wait complete. 2024-09-19T08:05:39.113 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2024-09-19T08:05:39.114 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 5... 2024-09-19T08:05:39.355 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:39 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3455600126' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T08:05:39.643 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:39 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:39.571+0000 7f64b6c11040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:05:39.643 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:39 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:39.642+0000 7f64b6c11040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:05:40.507 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:40 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:40.389+0000 7f64b6c11040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:05:40.879 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:40 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:40.509+0000 7f64b6c11040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:05:40.880 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:40 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:40.658+0000 7f64b6c11040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:05:40.880 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:40 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:40.722+0000 7f64b6c11040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:05:40.880 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:40 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:40.878+0000 7f64b6c11040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:05:41.178 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:40 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:40.949+0000 7f64b6c11040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:05:41.178 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:41.011+0000 7f64b6c11040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:05:41.178 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:41.177+0000 7f64b6c11040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:05:41.463 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:41.245+0000 7f64b6c11040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:05:41.464 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:41.314+0000 7f64b6c11040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:05:41.464 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:41.400+0000 7f64b6c11040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:05:41.464 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:41.462+0000 7f64b6c11040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:05:41.757 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:41.524+0000 7f64b6c11040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:05:42.757 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:42.351+0000 7f64b6c11040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:05:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: Active manager daemon x restarted 2024-09-19T08:05:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: Activating manager daemon x 2024-09-19T08:05:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T08:05:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: mgrmap e6: x(active, starting, since 0.00333913s) 2024-09-19T08:05:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:05:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:05:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:05:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:05:42.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:05:42.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: Manager daemon x is now available 2024-09-19T08:05:42.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' 2024-09-19T08:05:42.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' 2024-09-19T08:05:42.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:05:42.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:05:42.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:05:42.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:42 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:05:43.672 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:43 smithi022 ceph-mon[30963]: Found migration_current of "None". Setting to last migration. 2024-09-19T08:05:43.672 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:43 smithi022 ceph-mon[30963]: mgrmap e7: x(active, since 1.00542s) 2024-09-19T08:05:43.933 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:05:43.933 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T08:05:43.933 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T08:05:43.933 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:05:43.933 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.533+0000 7efec6a1b640 1 Processor -- start 2024-09-19T08:05:43.933 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.533+0000 7efec6a1b640 1 -- start start 2024-09-19T08:05:43.933 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.533+0000 7efec6a1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec0108230 0x7efec010a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:43.934 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.533+0000 7efec6a1b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efec007bdc0 con 0x7efec0108230 2024-09-19T08:05:43.934 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.533+0000 7efebffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec0108230 0x7efec010a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:43.934 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.533+0000 7efebffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec0108230 0x7efec010a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57386/0 (socket says 172.21.15.22:57386) 2024-09-19T08:05:43.934 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.533+0000 7efebffff640 1 -- 172.21.15.22:0/1978884927 learned_addr learned my addr 172.21.15.22:0/1978884927 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:43.934 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.534+0000 7efebffff640 1 -- 172.21.15.22:0/1978884927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec010ab60 con 0x7efec0108230 2024-09-19T08:05:43.934 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.535+0000 7efebffff640 1 --2- 172.21.15.22:0/1978884927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec0108230 0x7efec010a620 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009820 tx=0x7efeb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819178b7e4d51bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:43.934 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.535+0000 7efebeffd640 1 -- 172.21.15.22:0/1978884927 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeb0037070 con 0x7efec0108230 2024-09-19T08:05:43.934 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.535+0000 7efebeffd640 1 -- 172.21.15.22:0/1978884927 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efeb0032900 con 0x7efec0108230 2024-09-19T08:05:43.934 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.535+0000 7efebeffd640 1 -- 172.21.15.22:0/1978884927 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeb0030850 con 0x7efec0108230 2024-09-19T08:05:43.934 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.536+0000 7efec6a1b640 1 -- 172.21.15.22:0/1978884927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec0108230 msgr2=0x7efec010a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:43.934 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.536+0000 7efec6a1b640 1 --2- 172.21.15.22:0/1978884927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec0108230 0x7efec010a620 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009820 tx=0x7efeb002a270 comp rx=0 tx=0).stop 2024-09-19T08:05:43.934 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.536+0000 7efec6a1b640 1 -- 172.21.15.22:0/1978884927 shutdown_connections 2024-09-19T08:05:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.536+0000 7efec6a1b640 1 --2- 172.21.15.22:0/1978884927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec0108230 0x7efec010a620 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.536+0000 7efec6a1b640 1 -- 172.21.15.22:0/1978884927 >> 172.21.15.22:0/1978884927 conn(0x7efec0101db0 msgr2=0x7efec01041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.536+0000 7efec6a1b640 1 -- 172.21.15.22:0/1978884927 shutdown_connections 2024-09-19T08:05:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.536+0000 7efec6a1b640 1 -- 172.21.15.22:0/1978884927 wait complete. 2024-09-19T08:05:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.536+0000 7efec6a1b640 1 Processor -- start 2024-09-19T08:05:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.536+0000 7efec6a1b640 1 -- start start 2024-09-19T08:05:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.537+0000 7efec6a1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec0108230 0x7efec019b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.537+0000 7efec6a1b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efec010bae0 con 0x7efec0108230 2024-09-19T08:05:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.537+0000 7efebffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec0108230 0x7efec019b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.537+0000 7efebffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec0108230 0x7efec019b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57392/0 (socket says 172.21.15.22:57392) 2024-09-19T08:05:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.537+0000 7efebffff640 1 -- 172.21.15.22:0/637108122 learned_addr learned my addr 172.21.15.22:0/637108122 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.537+0000 7efebffff640 1 -- 172.21.15.22:0/637108122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec019bc30 con 0x7efec0108230 2024-09-19T08:05:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.537+0000 7efebffff640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec0108230 0x7efec019b6f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efeb00097f0 tx=0x7efeb0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:43.936 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.538+0000 7efebd7fa640 1 -- 172.21.15.22:0/637108122 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeb0042070 con 0x7efec0108230 2024-09-19T08:05:43.936 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.538+0000 7efec6a1b640 1 -- 172.21.15.22:0/637108122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec019bec0 con 0x7efec0108230 2024-09-19T08:05:43.936 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.538+0000 7efebd7fa640 1 -- 172.21.15.22:0/637108122 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efeb0032ca0 con 0x7efec0108230 2024-09-19T08:05:43.936 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.538+0000 7efec6a1b640 1 -- 172.21.15.22:0/637108122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec019ebb0 con 0x7efec0108230 2024-09-19T08:05:43.936 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.538+0000 7efebd7fa640 1 -- 172.21.15.22:0/637108122 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeb0037040 con 0x7efec0108230 2024-09-19T08:05:43.936 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.539+0000 7efebd7fa640 1 -- 172.21.15.22:0/637108122 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7efeb0045070 con 0x7efec0108230 2024-09-19T08:05:43.936 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.539+0000 7efebd7fa640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7efea80391b0 0x7efea803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:43.936 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.539+0000 7efebd7fa640 1 -- 172.21.15.22:0/637108122 --> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efea803bd80 con 0x7efea80391b0 2024-09-19T08:05:43.936 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.539+0000 7efebd7fa640 1 -- 172.21.15.22:0/637108122 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efeb006c1c0 con 0x7efec0108230 2024-09-19T08:05:43.936 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.539+0000 7efebf7fe640 1 -- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7efea80391b0 msgr2=0x7efea803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2990808963 2024-09-19T08:05:43.936 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.539+0000 7efebf7fe640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7efea80391b0 0x7efea803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:05:43.936 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.740+0000 7efebf7fe640 1 -- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7efea80391b0 msgr2=0x7efea803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2990808963 2024-09-19T08:05:43.936 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:39.740+0000 7efebf7fe640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7efea80391b0 0x7efea803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:05:43.937 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:40.141+0000 7efebf7fe640 1 -- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7efea80391b0 msgr2=0x7efea803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2990808963 2024-09-19T08:05:43.937 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:40.141+0000 7efebf7fe640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7efea80391b0 0x7efea803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T08:05:43.937 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:40.942+0000 7efebf7fe640 1 -- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7efea80391b0 msgr2=0x7efea803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2990808963 2024-09-19T08:05:43.937 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:40.942+0000 7efebf7fe640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7efea80391b0 0x7efea803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T08:05:43.937 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:42.355+0000 7efebd7fa640 1 -- 172.21.15.22:0/637108122 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7efeb006ad40 con 0x7efec0108230 2024-09-19T08:05:43.937 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:42.355+0000 7efebd7fa640 1 -- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7efea80391b0 msgr2=0x7efea803b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:05:43.937 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:42.355+0000 7efebd7fa640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7efea80391b0 0x7efea803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:43.937 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.356+0000 7efebd7fa640 1 -- 172.21.15.22:0/637108122 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7efeb003cd10 con 0x7efec0108230 2024-09-19T08:05:43.937 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.356+0000 7efebd7fa640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7efea803cae0 0x7efea803eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:43.937 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.356+0000 7efebd7fa640 1 -- 172.21.15.22:0/637108122 --> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efea803bd80 con 0x7efea803cae0 2024-09-19T08:05:43.937 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.356+0000 7efebf7fe640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7efea803cae0 0x7efea803eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:43.937 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.357+0000 7efebf7fe640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7efea803cae0 0x7efea803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efeb4003980 tx=0x7efeb4006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:43.937 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.358+0000 7efebd7fa640 1 -- 172.21.15.22:0/637108122 <== mgr.14118 v2:172.21.15.22:6800/4065414102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7efea803bd80 con 0x7efea803cae0 2024-09-19T08:05:43.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.364+0000 7efec6a1b640 1 -- 172.21.15.22:0/637108122 --> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7efec0108690 con 0x7efea803cae0 2024-09-19T08:05:43.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.365+0000 7efebd7fa640 1 -- 172.21.15.22:0/637108122 <== mgr.14118 v2:172.21.15.22:6800/4065414102 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7efec0108690 con 0x7efea803cae0 2024-09-19T08:05:43.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.365+0000 7efec6a1b640 1 -- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7efea803cae0 msgr2=0x7efea803eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:43.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.365+0000 7efec6a1b640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7efea803cae0 0x7efea803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efeb4003980 tx=0x7efeb4006e10 comp rx=0 tx=0).stop 2024-09-19T08:05:43.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.365+0000 7efec6a1b640 1 -- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec0108230 msgr2=0x7efec019b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:43.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.365+0000 7efec6a1b640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec0108230 0x7efec019b6f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efeb00097f0 tx=0x7efeb0030ec0 comp rx=0 tx=0).stop 2024-09-19T08:05:43.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.365+0000 7efec6a1b640 1 -- 172.21.15.22:0/637108122 shutdown_connections 2024-09-19T08:05:43.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.365+0000 7efec6a1b640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7efea803cae0 0x7efea803eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:43.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.365+0000 7efec6a1b640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:6800/2990808963,v1:172.21.15.22:6801/2990808963] conn(0x7efea80391b0 0x7efea803b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:43.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.365+0000 7efec6a1b640 1 --2- 172.21.15.22:0/637108122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec0108230 0x7efec019b6f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:43.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.365+0000 7efec6a1b640 1 -- 172.21.15.22:0/637108122 >> 172.21.15.22:0/637108122 conn(0x7efec0101db0 msgr2=0x7efec0102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:43.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.365+0000 7efec6a1b640 1 -- 172.21.15.22:0/637108122 shutdown_connections 2024-09-19T08:05:43.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:43.365+0000 7efec6a1b640 1 -- 172.21.15.22:0/637108122 wait complete. 2024-09-19T08:05:43.938 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 5 is available 2024-09-19T08:05:43.939 INFO:teuthology.orchestra.run.smithi022.stdout:Setting orchestrator backend to cephadm... 2024-09-19T08:05:44.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-mon[30963]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T08:05:44.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-mon[30963]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T08:05:44.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' 2024-09-19T08:05:44.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' 2024-09-19T08:05:44.845 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:44.645+0000 7f6423776640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:08:05:44] ENGINE Error in HTTPServer.serve 2024-09-19T08:05:44.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: Traceback (most recent call last): 2024-09-19T08:05:44.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:05:44.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: self._connections.run(self.expiration_interval) 2024-09-19T08:05:44.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:05:44.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: self._run(expiration_interval) 2024-09-19T08:05:44.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:05:44.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:05:44.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:05:44.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:05:44.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:05:44.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: s = self.context.wrap_socket( 2024-09-19T08:05:44.847 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:05:44.847 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: return self.sslsocket_class._create( 2024-09-19T08:05:44.847 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:05:44.847 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: self.do_handshake() 2024-09-19T08:05:44.847 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:05:44.847 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: self._sslobj.do_handshake() 2024-09-19T08:05:44.847 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:05:44.847 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:05:45.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.330+0000 7f3f87032640 1 Processor -- start 2024-09-19T08:05:45.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.330+0000 7f3f87032640 1 -- start start 2024-09-19T08:05:45.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.330+0000 7f3f87032640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f80108160 0x7f3f8010a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:45.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.330+0000 7f3f87032640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8007beb0 con 0x7f3f80108160 2024-09-19T08:05:45.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.330+0000 7f3f84dce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f80108160 0x7f3f8010a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:45.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.330+0000 7f3f84dce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f80108160 0x7f3f8010a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57458/0 (socket says 172.21.15.22:57458) 2024-09-19T08:05:45.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.330+0000 7f3f84dce640 1 -- 172.21.15.22:0/3010981595 learned_addr learned my addr 172.21.15.22:0/3010981595 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:45.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.330+0000 7f3f84dce640 1 -- 172.21.15.22:0/3010981595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8010aa90 con 0x7f3f80108160 2024-09-19T08:05:45.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.331+0000 7f3f84dce640 1 --2- 172.21.15.22:0/3010981595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f80108160 0x7f3f8010a550 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009820 tx=0x7f3f7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645dc8370091b418 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:45.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.331+0000 7f3f7f7fe640 1 -- 172.21.15.22:0/3010981595 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f74037070 con 0x7f3f80108160 2024-09-19T08:05:45.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.331+0000 7f3f7f7fe640 1 -- 172.21.15.22:0/3010981595 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3f74032900 con 0x7f3f80108160 2024-09-19T08:05:45.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.331+0000 7f3f7f7fe640 1 -- 172.21.15.22:0/3010981595 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f74030850 con 0x7f3f80108160 2024-09-19T08:05:45.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.332+0000 7f3f87032640 1 -- 172.21.15.22:0/3010981595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f80108160 msgr2=0x7f3f8010a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:45.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.332+0000 7f3f87032640 1 --2- 172.21.15.22:0/3010981595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f80108160 0x7f3f8010a550 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009820 tx=0x7f3f7402a270 comp rx=0 tx=0).stop 2024-09-19T08:05:45.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.332+0000 7f3f87032640 1 -- 172.21.15.22:0/3010981595 shutdown_connections 2024-09-19T08:05:45.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.332+0000 7f3f87032640 1 --2- 172.21.15.22:0/3010981595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f80108160 0x7f3f8010a550 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:45.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.332+0000 7f3f87032640 1 -- 172.21.15.22:0/3010981595 >> 172.21.15.22:0/3010981595 conn(0x7f3f80101e20 msgr2=0x7f3f80104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:45.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.332+0000 7f3f87032640 1 -- 172.21.15.22:0/3010981595 shutdown_connections 2024-09-19T08:05:45.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.332+0000 7f3f87032640 1 -- 172.21.15.22:0/3010981595 wait complete. 2024-09-19T08:05:45.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.333+0000 7f3f87032640 1 Processor -- start 2024-09-19T08:05:45.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.333+0000 7f3f87032640 1 -- start start 2024-09-19T08:05:45.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.333+0000 7f3f87032640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f80108160 0x7f3f8019b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:45.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.333+0000 7f3f87032640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8010bb30 con 0x7f3f80108160 2024-09-19T08:05:45.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.333+0000 7f3f84dce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f80108160 0x7f3f8019b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:45.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.333+0000 7f3f84dce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f80108160 0x7f3f8019b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57470/0 (socket says 172.21.15.22:57470) 2024-09-19T08:05:45.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.333+0000 7f3f84dce640 1 -- 172.21.15.22:0/1936238922 learned_addr learned my addr 172.21.15.22:0/1936238922 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:45.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.333+0000 7f3f84dce640 1 -- 172.21.15.22:0/1936238922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8019bc20 con 0x7f3f80108160 2024-09-19T08:05:45.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.333+0000 7f3f84dce640 1 --2- 172.21.15.22:0/1936238922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f80108160 0x7f3f8019b6e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3f74031000 tx=0x7f3f740326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:45.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.334+0000 7f3f7dffb640 1 -- 172.21.15.22:0/1936238922 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f74042070 con 0x7f3f80108160 2024-09-19T08:05:45.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.334+0000 7f3f7dffb640 1 -- 172.21.15.22:0/1936238922 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3f74032be0 con 0x7f3f80108160 2024-09-19T08:05:45.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.334+0000 7f3f7dffb640 1 -- 172.21.15.22:0/1936238922 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f74037040 con 0x7f3f80108160 2024-09-19T08:05:45.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.334+0000 7f3f87032640 1 -- 172.21.15.22:0/1936238922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8019beb0 con 0x7f3f80108160 2024-09-19T08:05:45.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.334+0000 7f3f87032640 1 -- 172.21.15.22:0/1936238922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8019eba0 con 0x7f3f80108160 2024-09-19T08:05:45.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.335+0000 7f3f7dffb640 1 -- 172.21.15.22:0/1936238922 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f3f74045070 con 0x7f3f80108160 2024-09-19T08:05:45.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.335+0000 7f3f87032640 1 -- 172.21.15.22:0/1936238922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f801085c0 con 0x7f3f80108160 2024-09-19T08:05:45.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.335+0000 7f3f7dffb640 1 --2- 172.21.15.22:0/1936238922 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f3f700390f0 0x7f3f7003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:45.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.335+0000 7f3f7dffb640 1 -- 172.21.15.22:0/1936238922 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3f7406c060 con 0x7f3f80108160 2024-09-19T08:05:45.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.335+0000 7f3f7ffff640 1 --2- 172.21.15.22:0/1936238922 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f3f700390f0 0x7f3f7003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.336+0000 7f3f7ffff640 1 --2- 172.21.15.22:0/1936238922 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f3f700390f0 0x7f3f7003b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0098c0 tx=0x7f3f6c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.340+0000 7f3f7dffb640 1 -- 172.21.15.22:0/1936238922 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3f74032d90 con 0x7f3f80108160 2024-09-19T08:05:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.466+0000 7f3f7dffb640 1 -- 172.21.15.22:0/1936238922 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f3f74039d70 con 0x7f3f80108160 2024-09-19T08:05:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.535+0000 7f3f87032640 1 -- 172.21.15.22:0/1936238922 --> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f3f801030f0 con 0x7f3f700390f0 2024-09-19T08:05:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.539+0000 7f3f7dffb640 1 -- 172.21.15.22:0/1936238922 <== mgr.14118 v2:172.21.15.22:6800/4065414102 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3f801030f0 con 0x7f3f700390f0 2024-09-19T08:05:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.542+0000 7f3f87032640 1 -- 172.21.15.22:0/1936238922 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f3f700390f0 msgr2=0x7f3f7003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.542+0000 7f3f87032640 1 --2- 172.21.15.22:0/1936238922 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f3f700390f0 0x7f3f7003b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0098c0 tx=0x7f3f6c006db0 comp rx=0 tx=0).stop 2024-09-19T08:05:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.542+0000 7f3f87032640 1 -- 172.21.15.22:0/1936238922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f80108160 msgr2=0x7f3f8019b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.542+0000 7f3f87032640 1 --2- 172.21.15.22:0/1936238922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f80108160 0x7f3f8019b6e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3f74031000 tx=0x7f3f740326e0 comp rx=0 tx=0).stop 2024-09-19T08:05:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.542+0000 7f3f87032640 1 -- 172.21.15.22:0/1936238922 shutdown_connections 2024-09-19T08:05:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.542+0000 7f3f87032640 1 --2- 172.21.15.22:0/1936238922 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f3f700390f0 0x7f3f7003b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:45.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.542+0000 7f3f87032640 1 --2- 172.21.15.22:0/1936238922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f80108160 0x7f3f8019b6e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:45.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.542+0000 7f3f87032640 1 -- 172.21.15.22:0/1936238922 >> 172.21.15.22:0/1936238922 conn(0x7f3f80101e20 msgr2=0x7f3f801029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:45.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.542+0000 7f3f87032640 1 -- 172.21.15.22:0/1936238922 shutdown_connections 2024-09-19T08:05:45.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:44.543+0000 7f3f87032640 1 -- 172.21.15.22:0/1936238922 wait complete. 2024-09-19T08:05:45.690 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:45 smithi022 ceph-mon[30963]: mgrmap e8: x(active, since 2s) 2024-09-19T08:05:45.690 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:45 smithi022 ceph-mon[30963]: [19/Sep/2024:08:05:44] ENGINE Bus STARTING 2024-09-19T08:05:45.690 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:45 smithi022 ceph-mon[30963]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:05:45.690 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:45 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' 2024-09-19T08:05:45.690 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:45 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:05:45.690 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:45 smithi022 ceph-mon[30963]: [19/Sep/2024:08:05:44] ENGINE Serving on https://172.21.15.22:7150 2024-09-19T08:05:45.690 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:45 smithi022 ceph-mon[30963]: [19/Sep/2024:08:05:44] ENGINE Error in HTTPServer.serve 2024-09-19T08:05:45.690 INFO:journalctl@ceph.mon.a.smithi022.stdout: Traceback (most recent call last): 2024-09-19T08:05:45.690 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:05:45.690 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._connections.run(self.expiration_interval) 2024-09-19T08:05:45.690 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:05:45.691 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._run(expiration_interval) 2024-09-19T08:05:45.691 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:05:45.691 INFO:journalctl@ceph.mon.a.smithi022.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:05:45.691 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:05:45.691 INFO:journalctl@ceph.mon.a.smithi022.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:05:45.691 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:05:45.691 INFO:journalctl@ceph.mon.a.smithi022.stdout: s = self.context.wrap_socket( 2024-09-19T08:05:45.691 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:05:45.691 INFO:journalctl@ceph.mon.a.smithi022.stdout: return self.sslsocket_class._create( 2024-09-19T08:05:45.691 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:05:45.691 INFO:journalctl@ceph.mon.a.smithi022.stdout: self.do_handshake() 2024-09-19T08:05:45.691 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:05:45.691 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._sslobj.do_handshake() 2024-09-19T08:05:45.691 INFO:journalctl@ceph.mon.a.smithi022.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:05:45.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:45 smithi022 ceph-mon[30963]: [19/Sep/2024:08:05:44] ENGINE Serving on http://172.21.15.22:8765 2024-09-19T08:05:45.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:45 smithi022 ceph-mon[30963]: [19/Sep/2024:08:05:44] ENGINE Bus STARTED 2024-09-19T08:05:45.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:45 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:05:46.276 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T08:05:46.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.500+0000 7f5a967da640 1 Processor -- start 2024-09-19T08:05:46.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.500+0000 7f5a967da640 1 -- start start 2024-09-19T08:05:46.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.500+0000 7f5a967da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a901081e0 0x7f5a9010a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:46.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.500+0000 7f5a967da640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9007be50 con 0x7f5a901081e0 2024-09-19T08:05:46.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.500+0000 7f5a8ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a901081e0 0x7f5a9010a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:46.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.500+0000 7f5a8ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a901081e0 0x7f5a9010a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57480/0 (socket says 172.21.15.22:57480) 2024-09-19T08:05:46.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.500+0000 7f5a8ffff640 1 -- 172.21.15.22:0/957862250 learned_addr learned my addr 172.21.15.22:0/957862250 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:46.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.500+0000 7f5a8ffff640 1 -- 172.21.15.22:0/957862250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9010ab10 con 0x7f5a901081e0 2024-09-19T08:05:46.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.501+0000 7f5a8ffff640 1 --2- 172.21.15.22:0/957862250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a901081e0 0x7f5a9010a5d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009820 tx=0x7f5a8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bcba2e2bd39a9f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:46.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.501+0000 7f5a8f7fe640 1 -- 172.21.15.22:0/957862250 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a80037070 con 0x7f5a901081e0 2024-09-19T08:05:46.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.501+0000 7f5a8f7fe640 1 -- 172.21.15.22:0/957862250 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5a80032900 con 0x7f5a901081e0 2024-09-19T08:05:46.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.501+0000 7f5a8f7fe640 1 -- 172.21.15.22:0/957862250 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a80030850 con 0x7f5a901081e0 2024-09-19T08:05:46.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.502+0000 7f5a967da640 1 -- 172.21.15.22:0/957862250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a901081e0 msgr2=0x7f5a9010a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:46.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.502+0000 7f5a967da640 1 --2- 172.21.15.22:0/957862250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a901081e0 0x7f5a9010a5d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009820 tx=0x7f5a8002a270 comp rx=0 tx=0).stop 2024-09-19T08:05:46.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.502+0000 7f5a967da640 1 -- 172.21.15.22:0/957862250 shutdown_connections 2024-09-19T08:05:46.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.502+0000 7f5a967da640 1 --2- 172.21.15.22:0/957862250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a901081e0 0x7f5a9010a5d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:46.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.502+0000 7f5a967da640 1 -- 172.21.15.22:0/957862250 >> 172.21.15.22:0/957862250 conn(0x7f5a90101e60 msgr2=0x7f5a90104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:46.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.502+0000 7f5a967da640 1 -- 172.21.15.22:0/957862250 shutdown_connections 2024-09-19T08:05:46.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.502+0000 7f5a967da640 1 -- 172.21.15.22:0/957862250 wait complete. 2024-09-19T08:05:46.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.503+0000 7f5a967da640 1 Processor -- start 2024-09-19T08:05:46.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.503+0000 7f5a967da640 1 -- start start 2024-09-19T08:05:46.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.503+0000 7f5a967da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a901081e0 0x7f5a90107a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:46.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.503+0000 7f5a967da640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9010bb50 con 0x7f5a901081e0 2024-09-19T08:05:46.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.503+0000 7f5a8ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a901081e0 0x7f5a90107a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:46.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.503+0000 7f5a8ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a901081e0 0x7f5a90107a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57494/0 (socket says 172.21.15.22:57494) 2024-09-19T08:05:46.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.503+0000 7f5a8ffff640 1 -- 172.21.15.22:0/3689486262 learned_addr learned my addr 172.21.15.22:0/3689486262 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:46.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.503+0000 7f5a8ffff640 1 -- 172.21.15.22:0/3689486262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a90107fa0 con 0x7f5a901081e0 2024-09-19T08:05:46.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.503+0000 7f5a8ffff640 1 --2- 172.21.15.22:0/3689486262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a901081e0 0x7f5a90107a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a80031000 tx=0x7f5a800326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:46.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.504+0000 7f5a8dffb640 1 -- 172.21.15.22:0/3689486262 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a80042070 con 0x7f5a901081e0 2024-09-19T08:05:46.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.504+0000 7f5a8dffb640 1 -- 172.21.15.22:0/3689486262 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5a80032be0 con 0x7f5a901081e0 2024-09-19T08:05:46.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.504+0000 7f5a967da640 1 -- 172.21.15.22:0/3689486262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a90106190 con 0x7f5a901081e0 2024-09-19T08:05:46.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.504+0000 7f5a8dffb640 1 -- 172.21.15.22:0/3689486262 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a80037040 con 0x7f5a901081e0 2024-09-19T08:05:46.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.504+0000 7f5a967da640 1 -- 172.21.15.22:0/3689486262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a901066d0 con 0x7f5a901081e0 2024-09-19T08:05:46.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.505+0000 7f5a8dffb640 1 -- 172.21.15.22:0/3689486262 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f5a80045070 con 0x7f5a901081e0 2024-09-19T08:05:46.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.505+0000 7f5a8dffb640 1 --2- 172.21.15.22:0/3689486262 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f5a7c039200 0x7f5a7c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:46.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.505+0000 7f5a967da640 1 -- 172.21.15.22:0/3689486262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a90108640 con 0x7f5a901081e0 2024-09-19T08:05:46.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.505+0000 7f5a8dffb640 1 -- 172.21.15.22:0/3689486262 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5a8006c1e0 con 0x7f5a901081e0 2024-09-19T08:05:46.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.505+0000 7f5a87fff640 1 --2- 172.21.15.22:0/3689486262 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f5a7c039200 0x7f5a7c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:46.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.506+0000 7f5a87fff640 1 --2- 172.21.15.22:0/3689486262 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f5a7c039200 0x7f5a7c03b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5a780098c0 tx=0x7f5a78006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:46.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.510+0000 7f5a8dffb640 1 -- 172.21.15.22:0/3689486262 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5a8004b170 con 0x7f5a901081e0 2024-09-19T08:05:46.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.689+0000 7f5a967da640 1 -- 172.21.15.22:0/3689486262 --> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f5a90106ff0 con 0x7f5a7c039200 2024-09-19T08:05:46.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.690+0000 7f5a8dffb640 1 -- 172.21.15.22:0/3689486262 <== mgr.14118 v2:172.21.15.22:6800/4065414102 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f5a90106ff0 con 0x7f5a7c039200 2024-09-19T08:05:46.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.692+0000 7f5a967da640 1 -- 172.21.15.22:0/3689486262 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f5a7c039200 msgr2=0x7f5a7c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:46.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.692+0000 7f5a967da640 1 --2- 172.21.15.22:0/3689486262 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f5a7c039200 0x7f5a7c03b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5a780098c0 tx=0x7f5a78006db0 comp rx=0 tx=0).stop 2024-09-19T08:05:46.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.692+0000 7f5a967da640 1 -- 172.21.15.22:0/3689486262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a901081e0 msgr2=0x7f5a90107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:46.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.692+0000 7f5a967da640 1 --2- 172.21.15.22:0/3689486262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a901081e0 0x7f5a90107a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a80031000 tx=0x7f5a800326e0 comp rx=0 tx=0).stop 2024-09-19T08:05:46.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.692+0000 7f5a967da640 1 -- 172.21.15.22:0/3689486262 shutdown_connections 2024-09-19T08:05:46.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.692+0000 7f5a967da640 1 --2- 172.21.15.22:0/3689486262 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f5a7c039200 0x7f5a7c03b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:46.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.692+0000 7f5a967da640 1 --2- 172.21.15.22:0/3689486262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a901081e0 0x7f5a90107a60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:46.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.692+0000 7f5a967da640 1 -- 172.21.15.22:0/3689486262 >> 172.21.15.22:0/3689486262 conn(0x7f5a90101e60 msgr2=0x7f5a90102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:46.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.692+0000 7f5a967da640 1 -- 172.21.15.22:0/3689486262 shutdown_connections 2024-09-19T08:05:46.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:45.692+0000 7f5a967da640 1 -- 172.21.15.22:0/3689486262 wait complete. 2024-09-19T08:05:46.281 INFO:teuthology.orchestra.run.smithi022.stdout:Generating ssh key... 2024-09-19T08:05:46.537 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:46 smithi022 ceph-mon[30963]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:05:47.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: Generating public/private rsa key pair. 2024-09-19T08:05:47.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: Your identification has been saved in /tmp/tmp0gziswpz/key 2024-09-19T08:05:47.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: Your public key has been saved in /tmp/tmp0gziswpz/key.pub 2024-09-19T08:05:47.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: The key fingerprint is: 2024-09-19T08:05:47.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: SHA256:nSZqjdd9JWWUyYHKmuAbMj1RE3FRSVxT0cuBQtPrW9U ceph-c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:05:47.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: The key's randomart image is: 2024-09-19T08:05:47.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: +---[RSA 3072]----+ 2024-09-19T08:05:47.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: | o+=*o*=X| 2024-09-19T08:05:47.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: | o...* *o| 2024-09-19T08:05:47.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: | . o o o *| 2024-09-19T08:05:47.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: | o . + . =E| 2024-09-19T08:05:47.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: | o S * . . o| 2024-09-19T08:05:47.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: | o O * . . + | 2024-09-19T08:05:47.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: | * * . . + | 2024-09-19T08:05:47.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: | . o o | 2024-09-19T08:05:47.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: | | 2024-09-19T08:05:47.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:05:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: +----[SHA256]-----+ 2024-09-19T08:05:47.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.690+0000 7f9511c5c640 1 Processor -- start 2024-09-19T08:05:47.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.690+0000 7f9511c5c640 1 -- start start 2024-09-19T08:05:47.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.690+0000 7f9511c5c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f950c1076d0 0x7f950c107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:47.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.690+0000 7f9511c5c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f950c1080a0 con 0x7f950c1076d0 2024-09-19T08:05:47.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.690+0000 7f950b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f950c1076d0 0x7f950c107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:47.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.690+0000 7f950b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f950c1076d0 0x7f950c107ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57500/0 (socket says 172.21.15.22:57500) 2024-09-19T08:05:47.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.690+0000 7f950b7fe640 1 -- 172.21.15.22:0/1115958713 learned_addr learned my addr 172.21.15.22:0/1115958713 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:47.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.691+0000 7f950b7fe640 1 -- 172.21.15.22:0/1115958713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f950c1088d0 con 0x7f950c1076d0 2024-09-19T08:05:47.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.691+0000 7f950b7fe640 1 --2- 172.21.15.22:0/1115958713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f950c1076d0 0x7f950c107ad0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009820 tx=0x7f94fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37048a16b9fa6b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:47.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.691+0000 7f950a7fc640 1 -- 172.21.15.22:0/1115958713 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94fc037070 con 0x7f950c1076d0 2024-09-19T08:05:47.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.692+0000 7f950a7fc640 1 -- 172.21.15.22:0/1115958713 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f94fc032900 con 0x7f950c1076d0 2024-09-19T08:05:47.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.692+0000 7f950a7fc640 1 -- 172.21.15.22:0/1115958713 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94fc030850 con 0x7f950c1076d0 2024-09-19T08:05:47.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.692+0000 7f9511c5c640 1 -- 172.21.15.22:0/1115958713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f950c1076d0 msgr2=0x7f950c107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:47.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.692+0000 7f9511c5c640 1 --2- 172.21.15.22:0/1115958713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f950c1076d0 0x7f950c107ad0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009820 tx=0x7f94fc02a270 comp rx=0 tx=0).stop 2024-09-19T08:05:47.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.692+0000 7f9511c5c640 1 -- 172.21.15.22:0/1115958713 shutdown_connections 2024-09-19T08:05:47.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.692+0000 7f9511c5c640 1 --2- 172.21.15.22:0/1115958713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f950c1076d0 0x7f950c107ad0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:47.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.692+0000 7f9511c5c640 1 -- 172.21.15.22:0/1115958713 >> 172.21.15.22:0/1115958713 conn(0x7f950c102e60 msgr2=0x7f950c1052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:47.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.692+0000 7f9511c5c640 1 -- 172.21.15.22:0/1115958713 shutdown_connections 2024-09-19T08:05:47.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.692+0000 7f9511c5c640 1 -- 172.21.15.22:0/1115958713 wait complete. 2024-09-19T08:05:47.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.693+0000 7f9511c5c640 1 Processor -- start 2024-09-19T08:05:47.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.693+0000 7f9511c5c640 1 -- start start 2024-09-19T08:05:47.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.693+0000 7f9511c5c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f950c1076d0 0x7f950c19b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:47.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.693+0000 7f9511c5c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f950c10bc80 con 0x7f950c1076d0 2024-09-19T08:05:47.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.693+0000 7f950b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f950c1076d0 0x7f950c19b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:47.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.694+0000 7f950b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f950c1076d0 0x7f950c19b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57516/0 (socket says 172.21.15.22:57516) 2024-09-19T08:05:47.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.694+0000 7f950b7fe640 1 -- 172.21.15.22:0/2345635653 learned_addr learned my addr 172.21.15.22:0/2345635653 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:47.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.694+0000 7f950b7fe640 1 -- 172.21.15.22:0/2345635653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f950c19bc00 con 0x7f950c1076d0 2024-09-19T08:05:47.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.694+0000 7f950b7fe640 1 --2- 172.21.15.22:0/2345635653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f950c1076d0 0x7f950c19b6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009940 tx=0x7f94fc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:47.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.694+0000 7f9508ff9640 1 -- 172.21.15.22:0/2345635653 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94fc042070 con 0x7f950c1076d0 2024-09-19T08:05:47.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.694+0000 7f9508ff9640 1 -- 172.21.15.22:0/2345635653 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f94fc0308c0 con 0x7f950c1076d0 2024-09-19T08:05:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.694+0000 7f9511c5c640 1 -- 172.21.15.22:0/2345635653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f950c19be90 con 0x7f950c1076d0 2024-09-19T08:05:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.694+0000 7f9508ff9640 1 -- 172.21.15.22:0/2345635653 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94fc037040 con 0x7f950c1076d0 2024-09-19T08:05:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.694+0000 7f9511c5c640 1 -- 172.21.15.22:0/2345635653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f950c19eb80 con 0x7f950c1076d0 2024-09-19T08:05:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.695+0000 7f9508ff9640 1 -- 172.21.15.22:0/2345635653 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f94fc045070 con 0x7f950c1076d0 2024-09-19T08:05:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.695+0000 7f9508ff9640 1 --2- 172.21.15.22:0/2345635653 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f94f4039200 0x7f94f403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.695+0000 7f9508ff9640 1 -- 172.21.15.22:0/2345635653 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f94fc06c330 con 0x7f950c1076d0 2024-09-19T08:05:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.695+0000 7f950affd640 1 --2- 172.21.15.22:0/2345635653 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f94f4039200 0x7f94f403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.695+0000 7f9511c5c640 1 -- 172.21.15.22:0/2345635653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f950c10be00 con 0x7f950c1076d0 2024-09-19T08:05:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.696+0000 7f950affd640 1 --2- 172.21.15.22:0/2345635653 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f94f4039200 0x7f94f403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95000098c0 tx=0x7f9500006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.700+0000 7f9508ff9640 1 -- 172.21.15.22:0/2345635653 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f94fc045360 con 0x7f950c1076d0 2024-09-19T08:05:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:46.878+0000 7f9511c5c640 1 -- 172.21.15.22:0/2345635653 --> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f950c1040b0 con 0x7f94f4039200 2024-09-19T08:05:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.068+0000 7f9508ff9640 1 -- 172.21.15.22:0/2345635653 <== mgr.14118 v2:172.21.15.22:6800/4065414102 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f950c1040b0 con 0x7f94f4039200 2024-09-19T08:05:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.070+0000 7f9511c5c640 1 -- 172.21.15.22:0/2345635653 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f94f4039200 msgr2=0x7f94f403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.070+0000 7f9511c5c640 1 --2- 172.21.15.22:0/2345635653 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f94f4039200 0x7f94f403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95000098c0 tx=0x7f9500006db0 comp rx=0 tx=0).stop 2024-09-19T08:05:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.070+0000 7f9511c5c640 1 -- 172.21.15.22:0/2345635653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f950c1076d0 msgr2=0x7f950c19b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.070+0000 7f9511c5c640 1 --2- 172.21.15.22:0/2345635653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f950c1076d0 0x7f950c19b6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009940 tx=0x7f94fc032ce0 comp rx=0 tx=0).stop 2024-09-19T08:05:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.070+0000 7f9511c5c640 1 -- 172.21.15.22:0/2345635653 shutdown_connections 2024-09-19T08:05:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.070+0000 7f9511c5c640 1 --2- 172.21.15.22:0/2345635653 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f94f4039200 0x7f94f403b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.070+0000 7f9511c5c640 1 --2- 172.21.15.22:0/2345635653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f950c1076d0 0x7f950c19b6c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.070+0000 7f9511c5c640 1 -- 172.21.15.22:0/2345635653 >> 172.21.15.22:0/2345635653 conn(0x7f950c102e60 msgr2=0x7f950c1039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.071+0000 7f9511c5c640 1 -- 172.21.15.22:0/2345635653 shutdown_connections 2024-09-19T08:05:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.071+0000 7f9511c5c640 1 -- 172.21.15.22:0/2345635653 wait complete. 2024-09-19T08:05:48.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:48 smithi022 ceph-mon[30963]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:05:48.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:48 smithi022 ceph-mon[30963]: Generating ssh key... 2024-09-19T08:05:48.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:48 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' 2024-09-19T08:05:48.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:48 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' 2024-09-19T08:05:48.789 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQx3In6CZL2XP/phaLxNEQaroG0CENFfGvetSTatbsT4ULm2suNDOdmV6gKMxLA/R/odNpqVi8sDtwiszluqCQQWwL0c6UV/kQcPOX76kzY6czILcCwRd7MrrqTXR5jgUMR+wwe/lFN/84WI0hYNAYl4fwBwk4xcJ6nptBrt08a0clMi4v1t3gnvMWUHrxDTSNMIBofIn50XI9jKRrY9H2gC1UfhK7x2SKtGttSU0j4bTm17TLhDx5O6mJiFo9vVmMngMoJQIUMsxDkRyTYeIqVuMi8GLrWM/AGU8N9ZrZBrDtRPzqpfUYMXrK1dWcTx2wiUJBN++aiyYCC6uTI/E0cEqErqrd4JMMA/1zZ7S19095TRCq9tyEzP+FWpCxgRInJOA1juFg1bYila1zI7pri1kkeUakCge7LjLQao4EEErxoMwakUCnbd5xuAVQ1xgvSvrVxvgx9cgOpLC9qdBW64EKBjkWi+vQfyyQ9Ah6jRpZpvycEZmnqKAnnmsMTRk= ceph-c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:05:48.789 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.991+0000 7fb506e27640 1 Processor -- start 2024-09-19T08:05:48.789 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.991+0000 7fb506e27640 1 -- start start 2024-09-19T08:05:48.789 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.991+0000 7fb506e27640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb500108200 0x7fb50010a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:48.789 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.991+0000 7fb506e27640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb50007be30 con 0x7fb500108200 2024-09-19T08:05:48.790 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.991+0000 7fb504bc3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb500108200 0x7fb50010a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:48.790 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.991+0000 7fb504bc3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb500108200 0x7fb50010a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51698/0 (socket says 172.21.15.22:51698) 2024-09-19T08:05:48.790 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.991+0000 7fb504bc3640 1 -- 172.21.15.22:0/2670629447 learned_addr learned my addr 172.21.15.22:0/2670629447 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:48.790 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.992+0000 7fb504bc3640 1 -- 172.21.15.22:0/2670629447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50010ab30 con 0x7fb500108200 2024-09-19T08:05:48.790 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.992+0000 7fb504bc3640 1 --2- 172.21.15.22:0/2670629447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb500108200 0x7fb50010a5f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009820 tx=0x7fb4f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1265c54565ccb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:48.790 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.993+0000 7fb4ff7fe640 1 -- 172.21.15.22:0/2670629447 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f8037070 con 0x7fb500108200 2024-09-19T08:05:48.790 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.993+0000 7fb4ff7fe640 1 -- 172.21.15.22:0/2670629447 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4f8032900 con 0x7fb500108200 2024-09-19T08:05:48.790 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.993+0000 7fb4ff7fe640 1 -- 172.21.15.22:0/2670629447 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f8030850 con 0x7fb500108200 2024-09-19T08:05:48.790 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.993+0000 7fb506e27640 1 -- 172.21.15.22:0/2670629447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb500108200 msgr2=0x7fb50010a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:48.791 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.993+0000 7fb506e27640 1 --2- 172.21.15.22:0/2670629447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb500108200 0x7fb50010a5f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009820 tx=0x7fb4f802a270 comp rx=0 tx=0).stop 2024-09-19T08:05:48.791 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.993+0000 7fb506e27640 1 -- 172.21.15.22:0/2670629447 shutdown_connections 2024-09-19T08:05:48.791 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.993+0000 7fb506e27640 1 --2- 172.21.15.22:0/2670629447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb500108200 0x7fb50010a5f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:48.791 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.993+0000 7fb506e27640 1 -- 172.21.15.22:0/2670629447 >> 172.21.15.22:0/2670629447 conn(0x7fb500101da0 msgr2=0x7fb5001041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:48.791 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.993+0000 7fb506e27640 1 -- 172.21.15.22:0/2670629447 shutdown_connections 2024-09-19T08:05:48.791 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.993+0000 7fb506e27640 1 -- 172.21.15.22:0/2670629447 wait complete. 2024-09-19T08:05:48.791 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.994+0000 7fb506e27640 1 Processor -- start 2024-09-19T08:05:48.791 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.994+0000 7fb506e27640 1 -- start start 2024-09-19T08:05:48.791 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.994+0000 7fb506e27640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb500108200 0x7fb50019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:48.791 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.994+0000 7fb506e27640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb50010bb10 con 0x7fb500108200 2024-09-19T08:05:48.791 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.994+0000 7fb504bc3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb500108200 0x7fb50019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:48.791 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.994+0000 7fb504bc3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb500108200 0x7fb50019b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51714/0 (socket says 172.21.15.22:51714) 2024-09-19T08:05:48.791 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.994+0000 7fb504bc3640 1 -- 172.21.15.22:0/2545850248 learned_addr learned my addr 172.21.15.22:0/2545850248 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:48.792 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.995+0000 7fb504bc3640 1 -- 172.21.15.22:0/2545850248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50019bc50 con 0x7fb500108200 2024-09-19T08:05:48.792 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.995+0000 7fb504bc3640 1 --2- 172.21.15.22:0/2545850248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb500108200 0x7fb50019b710 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8031000 tx=0x7fb4f80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:48.792 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.995+0000 7fb4fdffb640 1 -- 172.21.15.22:0/2545850248 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f8042070 con 0x7fb500108200 2024-09-19T08:05:48.792 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.995+0000 7fb4fdffb640 1 -- 172.21.15.22:0/2545850248 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4f8032be0 con 0x7fb500108200 2024-09-19T08:05:48.792 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.995+0000 7fb4fdffb640 1 -- 172.21.15.22:0/2545850248 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f8037040 con 0x7fb500108200 2024-09-19T08:05:48.792 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.995+0000 7fb506e27640 1 -- 172.21.15.22:0/2545850248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50019bee0 con 0x7fb500108200 2024-09-19T08:05:48.792 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.995+0000 7fb506e27640 1 -- 172.21.15.22:0/2545850248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb50019ebd0 con 0x7fb500108200 2024-09-19T08:05:48.792 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.996+0000 7fb4fdffb640 1 -- 172.21.15.22:0/2545850248 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fb4f8045070 con 0x7fb500108200 2024-09-19T08:05:48.792 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.996+0000 7fb4fdffb640 1 --2- 172.21.15.22:0/2545850248 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fb4ec039200 0x7fb4ec03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:48.793 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.996+0000 7fb506e27640 1 -- 172.21.15.22:0/2545850248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb500108660 con 0x7fb500108200 2024-09-19T08:05:48.793 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.996+0000 7fb4fdffb640 1 -- 172.21.15.22:0/2545850248 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb4f806c2d0 con 0x7fb500108200 2024-09-19T08:05:48.793 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.996+0000 7fb4fffff640 1 --2- 172.21.15.22:0/2545850248 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fb4ec039200 0x7fb4ec03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:48.793 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:47.997+0000 7fb4fffff640 1 --2- 172.21.15.22:0/2545850248 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fb4ec039200 0x7fb4ec03b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000ad30 tx=0x7fb4f00093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:48.793 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:48.001+0000 7fb4fdffb640 1 -- 172.21.15.22:0/2545850248 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb4f8007040 con 0x7fb500108200 2024-09-19T08:05:48.793 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:48.177+0000 7fb506e27640 1 -- 172.21.15.22:0/2545850248 --> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fb500103060 con 0x7fb4ec039200 2024-09-19T08:05:48.793 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:48.178+0000 7fb4fdffb640 1 -- 172.21.15.22:0/2545850248 <== mgr.14118 v2:172.21.15.22:6800/4065414102 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fb500103060 con 0x7fb4ec039200 2024-09-19T08:05:48.793 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:48.180+0000 7fb506e27640 1 -- 172.21.15.22:0/2545850248 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fb4ec039200 msgr2=0x7fb4ec03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:48.793 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:48.180+0000 7fb506e27640 1 --2- 172.21.15.22:0/2545850248 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fb4ec039200 0x7fb4ec03b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000ad30 tx=0x7fb4f00093f0 comp rx=0 tx=0).stop 2024-09-19T08:05:48.793 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:48.180+0000 7fb506e27640 1 -- 172.21.15.22:0/2545850248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb500108200 msgr2=0x7fb50019b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:48.793 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:48.180+0000 7fb506e27640 1 --2- 172.21.15.22:0/2545850248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb500108200 0x7fb50019b710 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8031000 tx=0x7fb4f80326e0 comp rx=0 tx=0).stop 2024-09-19T08:05:48.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:48.180+0000 7fb506e27640 1 -- 172.21.15.22:0/2545850248 shutdown_connections 2024-09-19T08:05:48.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:48.180+0000 7fb506e27640 1 --2- 172.21.15.22:0/2545850248 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fb4ec039200 0x7fb4ec03b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:48.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:48.180+0000 7fb506e27640 1 --2- 172.21.15.22:0/2545850248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb500108200 0x7fb50019b710 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:48.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:48.180+0000 7fb506e27640 1 -- 172.21.15.22:0/2545850248 >> 172.21.15.22:0/2545850248 conn(0x7fb500101da0 msgr2=0x7fb500102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:48.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:48.180+0000 7fb506e27640 1 -- 172.21.15.22:0/2545850248 shutdown_connections 2024-09-19T08:05:48.794 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:48.180+0000 7fb506e27640 1 -- 172.21.15.22:0/2545850248 wait complete. 2024-09-19T08:05:48.794 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T08:05:48.794 INFO:teuthology.orchestra.run.smithi022.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T08:05:48.795 INFO:teuthology.orchestra.run.smithi022.stdout:Adding host smithi022... 2024-09-19T08:05:49.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:49 smithi022 ceph-mon[30963]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:05:50.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:50 smithi022 ceph-mon[30963]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi022", "addr": "172.21.15.22", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:05:51.678 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:51 smithi022 ceph-mon[30963]: Deploying cephadm binary to smithi022 2024-09-19T08:05:53.926 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Added host 'smithi022' with addr '172.21.15.22' 2024-09-19T08:05:53.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.210+0000 7f2997605640 1 Processor -- start 2024-09-19T08:05:53.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.211+0000 7f2997605640 1 -- start start 2024-09-19T08:05:53.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.211+0000 7f2997605640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2990109960 0x7f2990109d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:53.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.211+0000 7f2997605640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f299010a330 con 0x7f2990109960 2024-09-19T08:05:53.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.211+0000 7f29953a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2990109960 0x7f2990109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:53.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.211+0000 7f29953a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2990109960 0x7f2990109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51722/0 (socket says 172.21.15.22:51722) 2024-09-19T08:05:53.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.211+0000 7f29953a1640 1 -- 172.21.15.22:0/4075520928 learned_addr learned my addr 172.21.15.22:0/4075520928 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:53.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.211+0000 7f29953a1640 1 -- 172.21.15.22:0/4075520928 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299010aad0 con 0x7f2990109960 2024-09-19T08:05:53.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.212+0000 7f29953a1640 1 --2- 172.21.15.22:0/4075520928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2990109960 0x7f2990109d60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f298c009820 tx=0x7f298c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fa89c636ecc32e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:53.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.212+0000 7f2987fff640 1 -- 172.21.15.22:0/4075520928 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f298c037070 con 0x7f2990109960 2024-09-19T08:05:53.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.212+0000 7f2987fff640 1 -- 172.21.15.22:0/4075520928 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f298c032900 con 0x7f2990109960 2024-09-19T08:05:53.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.212+0000 7f2987fff640 1 -- 172.21.15.22:0/4075520928 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f298c030850 con 0x7f2990109960 2024-09-19T08:05:53.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.213+0000 7f2997605640 1 -- 172.21.15.22:0/4075520928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2990109960 msgr2=0x7f2990109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:53.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.213+0000 7f2997605640 1 --2- 172.21.15.22:0/4075520928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2990109960 0x7f2990109d60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f298c009820 tx=0x7f298c02a270 comp rx=0 tx=0).stop 2024-09-19T08:05:53.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.213+0000 7f2997605640 1 -- 172.21.15.22:0/4075520928 shutdown_connections 2024-09-19T08:05:53.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.213+0000 7f2997605640 1 --2- 172.21.15.22:0/4075520928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2990109960 0x7f2990109d60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:53.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.213+0000 7f2997605640 1 -- 172.21.15.22:0/4075520928 >> 172.21.15.22:0/4075520928 conn(0x7f299007cf30 msgr2=0x7f299007d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:53.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.213+0000 7f2997605640 1 -- 172.21.15.22:0/4075520928 shutdown_connections 2024-09-19T08:05:53.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.213+0000 7f2997605640 1 -- 172.21.15.22:0/4075520928 wait complete. 2024-09-19T08:05:53.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.213+0000 7f2997605640 1 Processor -- start 2024-09-19T08:05:53.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.214+0000 7f2997605640 1 -- start start 2024-09-19T08:05:53.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.214+0000 7f2997605640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2990109960 0x7f299019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:53.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.214+0000 7f2997605640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f299010de80 con 0x7f2990109960 2024-09-19T08:05:53.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.214+0000 7f29953a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2990109960 0x7f299019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:53.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.214+0000 7f29953a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2990109960 0x7f299019d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51734/0 (socket says 172.21.15.22:51734) 2024-09-19T08:05:53.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.214+0000 7f29953a1640 1 -- 172.21.15.22:0/1389904792 learned_addr learned my addr 172.21.15.22:0/1389904792 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:53.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.214+0000 7f29953a1640 1 -- 172.21.15.22:0/1389904792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299019de40 con 0x7f2990109960 2024-09-19T08:05:53.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.214+0000 7f29953a1640 1 --2- 172.21.15.22:0/1389904792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2990109960 0x7f299019d900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f298c009940 tx=0x7f298c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:53.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.214+0000 7f29867fc640 1 -- 172.21.15.22:0/1389904792 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f298c042070 con 0x7f2990109960 2024-09-19T08:05:53.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.214+0000 7f29867fc640 1 -- 172.21.15.22:0/1389904792 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f298c032be0 con 0x7f2990109960 2024-09-19T08:05:53.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.214+0000 7f2997605640 1 -- 172.21.15.22:0/1389904792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299019e0d0 con 0x7f2990109960 2024-09-19T08:05:53.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.215+0000 7f29867fc640 1 -- 172.21.15.22:0/1389904792 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f298c037040 con 0x7f2990109960 2024-09-19T08:05:53.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.215+0000 7f2997605640 1 -- 172.21.15.22:0/1389904792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29901a0dc0 con 0x7f2990109960 2024-09-19T08:05:53.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.215+0000 7f29867fc640 1 -- 172.21.15.22:0/1389904792 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f298c045070 con 0x7f2990109960 2024-09-19T08:05:53.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.216+0000 7f29867fc640 1 --2- 172.21.15.22:0/1389904792 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f297c039200 0x7f297c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:53.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.216+0000 7f29867fc640 1 -- 172.21.15.22:0/1389904792 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f298c06c1b0 con 0x7f2990109960 2024-09-19T08:05:53.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.216+0000 7f2997605640 1 -- 172.21.15.22:0/1389904792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f299010e000 con 0x7f2990109960 2024-09-19T08:05:53.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.216+0000 7f2994ba0640 1 --2- 172.21.15.22:0/1389904792 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f297c039200 0x7f297c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:53.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.216+0000 7f2994ba0640 1 --2- 172.21.15.22:0/1389904792 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f297c039200 0x7f297c03b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29800098c0 tx=0x7f2980006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:53.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.220+0000 7f29867fc640 1 -- 172.21.15.22:0/1389904792 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f298c046e00 con 0x7f2990109960 2024-09-19T08:05:53.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:49.399+0000 7f2997605640 1 -- 172.21.15.22:0/1389904792 --> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi022", "addr": "172.21.15.22", "target": ["mon-mgr", ""]}) -- 0x7f2990107c20 con 0x7f297c039200 2024-09-19T08:05:53.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:53.317+0000 7f29867fc640 1 -- 172.21.15.22:0/1389904792 <== mgr.14118 v2:172.21.15.22:6800/4065414102 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f2990107c20 con 0x7f297c039200 2024-09-19T08:05:53.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:53.321+0000 7f2997605640 1 -- 172.21.15.22:0/1389904792 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f297c039200 msgr2=0x7f297c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:53.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:53.321+0000 7f2997605640 1 --2- 172.21.15.22:0/1389904792 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f297c039200 0x7f297c03b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29800098c0 tx=0x7f2980006db0 comp rx=0 tx=0).stop 2024-09-19T08:05:53.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:53.321+0000 7f2997605640 1 -- 172.21.15.22:0/1389904792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2990109960 msgr2=0x7f299019d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:53.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:53.321+0000 7f2997605640 1 --2- 172.21.15.22:0/1389904792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2990109960 0x7f299019d900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f298c009940 tx=0x7f298c0326e0 comp rx=0 tx=0).stop 2024-09-19T08:05:53.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:53.321+0000 7f2997605640 1 -- 172.21.15.22:0/1389904792 shutdown_connections 2024-09-19T08:05:53.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:53.321+0000 7f2997605640 1 --2- 172.21.15.22:0/1389904792 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f297c039200 0x7f297c03b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:53.931 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:53.321+0000 7f2997605640 1 --2- 172.21.15.22:0/1389904792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2990109960 0x7f299019d900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:53.931 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:53.321+0000 7f2997605640 1 -- 172.21.15.22:0/1389904792 >> 172.21.15.22:0/1389904792 conn(0x7f299007cf30 msgr2=0x7f2990107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:53.931 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:53.321+0000 7f2997605640 1 -- 172.21.15.22:0/1389904792 shutdown_connections 2024-09-19T08:05:53.931 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:53.321+0000 7f2997605640 1 -- 172.21.15.22:0/1389904792 wait complete. 2024-09-19T08:05:53.931 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying unmanaged mon service... 2024-09-19T08:05:54.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:54 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' 2024-09-19T08:05:54.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:54 smithi022 ceph-mon[30963]: Added host smithi022 2024-09-19T08:05:54.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:54 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:05:55.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T08:05:55.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.331+0000 7fe1915be640 1 Processor -- start 2024-09-19T08:05:55.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.331+0000 7fe1915be640 1 -- start start 2024-09-19T08:05:55.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.331+0000 7fe1915be640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe18c108200 0x7fe18c10a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:55.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.331+0000 7fe1915be640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c07be30 con 0x7fe18c108200 2024-09-19T08:05:55.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.331+0000 7fe18affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe18c108200 0x7fe18c10a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:55.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.331+0000 7fe18affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe18c108200 0x7fe18c10a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51742/0 (socket says 172.21.15.22:51742) 2024-09-19T08:05:55.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.331+0000 7fe18affd640 1 -- 172.21.15.22:0/4192347577 learned_addr learned my addr 172.21.15.22:0/4192347577 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:55.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.331+0000 7fe18affd640 1 -- 172.21.15.22:0/4192347577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c10ab30 con 0x7fe18c108200 2024-09-19T08:05:55.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.332+0000 7fe18affd640 1 --2- 172.21.15.22:0/4192347577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe18c108200 0x7fe18c10a5f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009820 tx=0x7fe17c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fd424ffc267436 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:55.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.332+0000 7fe18a7fc640 1 -- 172.21.15.22:0/4192347577 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe17c037070 con 0x7fe18c108200 2024-09-19T08:05:55.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.333+0000 7fe18a7fc640 1 -- 172.21.15.22:0/4192347577 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe17c032900 con 0x7fe18c108200 2024-09-19T08:05:55.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.333+0000 7fe18a7fc640 1 -- 172.21.15.22:0/4192347577 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe17c030850 con 0x7fe18c108200 2024-09-19T08:05:55.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.333+0000 7fe1915be640 1 -- 172.21.15.22:0/4192347577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe18c108200 msgr2=0x7fe18c10a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:55.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.333+0000 7fe1915be640 1 --2- 172.21.15.22:0/4192347577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe18c108200 0x7fe18c10a5f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009820 tx=0x7fe17c02a270 comp rx=0 tx=0).stop 2024-09-19T08:05:55.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.333+0000 7fe1915be640 1 -- 172.21.15.22:0/4192347577 shutdown_connections 2024-09-19T08:05:55.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.333+0000 7fe1915be640 1 --2- 172.21.15.22:0/4192347577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe18c108200 0x7fe18c10a5f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:55.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.333+0000 7fe1915be640 1 -- 172.21.15.22:0/4192347577 >> 172.21.15.22:0/4192347577 conn(0x7fe18c101da0 msgr2=0x7fe18c1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:55.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.333+0000 7fe1915be640 1 -- 172.21.15.22:0/4192347577 shutdown_connections 2024-09-19T08:05:55.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.333+0000 7fe1915be640 1 -- 172.21.15.22:0/4192347577 wait complete. 2024-09-19T08:05:55.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.334+0000 7fe1915be640 1 Processor -- start 2024-09-19T08:05:55.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.334+0000 7fe1915be640 1 -- start start 2024-09-19T08:05:55.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.334+0000 7fe1915be640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe18c108200 0x7fe18c19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:55.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.334+0000 7fe1915be640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c10bb10 con 0x7fe18c108200 2024-09-19T08:05:55.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.334+0000 7fe18affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe18c108200 0x7fe18c19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:55.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.334+0000 7fe18affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe18c108200 0x7fe18c19b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51744/0 (socket says 172.21.15.22:51744) 2024-09-19T08:05:55.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.334+0000 7fe18affd640 1 -- 172.21.15.22:0/583234388 learned_addr learned my addr 172.21.15.22:0/583234388 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:55.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.334+0000 7fe18affd640 1 -- 172.21.15.22:0/583234388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c19bc50 con 0x7fe18c108200 2024-09-19T08:05:55.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.334+0000 7fe18affd640 1 --2- 172.21.15.22:0/583234388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe18c108200 0x7fe18c19b710 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe17c031000 tx=0x7fe17c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:55.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.335+0000 7fe188ff9640 1 -- 172.21.15.22:0/583234388 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe17c042070 con 0x7fe18c108200 2024-09-19T08:05:55.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.335+0000 7fe1915be640 1 -- 172.21.15.22:0/583234388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18c19bee0 con 0x7fe18c108200 2024-09-19T08:05:55.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.335+0000 7fe188ff9640 1 -- 172.21.15.22:0/583234388 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe17c032be0 con 0x7fe18c108200 2024-09-19T08:05:55.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.335+0000 7fe188ff9640 1 -- 172.21.15.22:0/583234388 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe17c037040 con 0x7fe18c108200 2024-09-19T08:05:55.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.335+0000 7fe1915be640 1 -- 172.21.15.22:0/583234388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18c19ebd0 con 0x7fe18c108200 2024-09-19T08:05:55.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.335+0000 7fe188ff9640 1 -- 172.21.15.22:0/583234388 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fe17c045070 con 0x7fe18c108200 2024-09-19T08:05:55.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.336+0000 7fe1915be640 1 -- 172.21.15.22:0/583234388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe18c108660 con 0x7fe18c108200 2024-09-19T08:05:55.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.336+0000 7fe188ff9640 1 --2- 172.21.15.22:0/583234388 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fe1780391b0 0x7fe17803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:55.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.336+0000 7fe188ff9640 1 -- 172.21.15.22:0/583234388 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe17c06c290 con 0x7fe18c108200 2024-09-19T08:05:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.336+0000 7fe183fff640 1 --2- 172.21.15.22:0/583234388 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fe1780391b0 0x7fe17803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.336+0000 7fe183fff640 1 --2- 172.21.15.22:0/583234388 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fe1780391b0 0x7fe17803b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe1740098c0 tx=0x7fe174006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.341+0000 7fe188ff9640 1 -- 172.21.15.22:0/583234388 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe17c02f2a0 con 0x7fe18c108200 2024-09-19T08:05:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.537+0000 7fe1915be640 1 -- 172.21.15.22:0/583234388 --> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe18c103060 con 0x7fe1780391b0 2024-09-19T08:05:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.542+0000 7fe188ff9640 1 -- 172.21.15.22:0/583234388 <== mgr.14118 v2:172.21.15.22:6800/4065414102 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe18c103060 con 0x7fe1780391b0 2024-09-19T08:05:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.544+0000 7fe1915be640 1 -- 172.21.15.22:0/583234388 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fe1780391b0 msgr2=0x7fe17803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.544+0000 7fe1915be640 1 --2- 172.21.15.22:0/583234388 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fe1780391b0 0x7fe17803b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe1740098c0 tx=0x7fe174006db0 comp rx=0 tx=0).stop 2024-09-19T08:05:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.544+0000 7fe1915be640 1 -- 172.21.15.22:0/583234388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe18c108200 msgr2=0x7fe18c19b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.544+0000 7fe1915be640 1 --2- 172.21.15.22:0/583234388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe18c108200 0x7fe18c19b710 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe17c031000 tx=0x7fe17c0326e0 comp rx=0 tx=0).stop 2024-09-19T08:05:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.544+0000 7fe1915be640 1 -- 172.21.15.22:0/583234388 shutdown_connections 2024-09-19T08:05:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.544+0000 7fe1915be640 1 --2- 172.21.15.22:0/583234388 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fe1780391b0 0x7fe17803b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.544+0000 7fe1915be640 1 --2- 172.21.15.22:0/583234388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe18c108200 0x7fe18c19b710 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.544+0000 7fe1915be640 1 -- 172.21.15.22:0/583234388 >> 172.21.15.22:0/583234388 conn(0x7fe18c101da0 msgr2=0x7fe18c102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.544+0000 7fe1915be640 1 -- 172.21.15.22:0/583234388 shutdown_connections 2024-09-19T08:05:55.262 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:54.544+0000 7fe1915be640 1 -- 172.21.15.22:0/583234388 wait complete. 2024-09-19T08:05:55.262 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying unmanaged mgr service... 2024-09-19T08:05:55.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:55 smithi022 ceph-mon[30963]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:05:55.846 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:55 smithi022 ceph-mon[30963]: Saving service mon spec with placement count:5 2024-09-19T08:05:55.846 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:55 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' 2024-09-19T08:05:57.180 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:56 smithi022 ceph-mon[30963]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:05:57.180 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:56 smithi022 ceph-mon[30963]: Saving service mgr spec with placement count:2 2024-09-19T08:05:57.180 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:56 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' 2024-09-19T08:05:57.188 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T08:05:57.188 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.641+0000 7f2682251640 1 Processor -- start 2024-09-19T08:05:57.188 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.641+0000 7f2682251640 1 -- start start 2024-09-19T08:05:57.188 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.641+0000 7f2682251640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267c107720 0x7f267c107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:57.189 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.641+0000 7f2682251640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1080f0 con 0x7f267c107720 2024-09-19T08:05:57.189 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.642+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267c107720 0x7f267c107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:57.189 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.642+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267c107720 0x7f267c107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51750/0 (socket says 172.21.15.22:51750) 2024-09-19T08:05:57.189 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.642+0000 7f267b7fe640 1 -- 172.21.15.22:0/893029256 learned_addr learned my addr 172.21.15.22:0/893029256 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:57.189 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.642+0000 7f267b7fe640 1 -- 172.21.15.22:0/893029256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c1088d0 con 0x7f267c107720 2024-09-19T08:05:57.189 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.643+0000 7f267b7fe640 1 --2- 172.21.15.22:0/893029256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267c107720 0x7f267c107b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f266c009820 tx=0x7f266c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3a17b1ceed5932 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:57.189 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.643+0000 7f267affd640 1 -- 172.21.15.22:0/893029256 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f266c037070 con 0x7f267c107720 2024-09-19T08:05:57.189 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.643+0000 7f267affd640 1 -- 172.21.15.22:0/893029256 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f266c032900 con 0x7f267c107720 2024-09-19T08:05:57.189 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.643+0000 7f267affd640 1 -- 172.21.15.22:0/893029256 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f266c030850 con 0x7f267c107720 2024-09-19T08:05:57.189 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.644+0000 7f2682251640 1 -- 172.21.15.22:0/893029256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267c107720 msgr2=0x7f267c107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:57.189 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.644+0000 7f2682251640 1 --2- 172.21.15.22:0/893029256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267c107720 0x7f267c107b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f266c009820 tx=0x7f266c02a270 comp rx=0 tx=0).stop 2024-09-19T08:05:57.190 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.644+0000 7f2682251640 1 -- 172.21.15.22:0/893029256 shutdown_connections 2024-09-19T08:05:57.190 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.644+0000 7f2682251640 1 --2- 172.21.15.22:0/893029256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267c107720 0x7f267c107b20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:57.190 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.644+0000 7f2682251640 1 -- 172.21.15.22:0/893029256 >> 172.21.15.22:0/893029256 conn(0x7f267c102ef0 msgr2=0x7f267c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:57.190 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.644+0000 7f2682251640 1 -- 172.21.15.22:0/893029256 shutdown_connections 2024-09-19T08:05:57.190 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.644+0000 7f2682251640 1 -- 172.21.15.22:0/893029256 wait complete. 2024-09-19T08:05:57.190 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.644+0000 7f2682251640 1 Processor -- start 2024-09-19T08:05:57.190 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.644+0000 7f2682251640 1 -- start start 2024-09-19T08:05:57.190 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.645+0000 7f2682251640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267c107720 0x7f267c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:57.190 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.645+0000 7f2682251640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f267c10bc80 con 0x7f267c107720 2024-09-19T08:05:57.190 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.645+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267c107720 0x7f267c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:57.190 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.645+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267c107720 0x7f267c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51760/0 (socket says 172.21.15.22:51760) 2024-09-19T08:05:57.190 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.645+0000 7f267b7fe640 1 -- 172.21.15.22:0/165121210 learned_addr learned my addr 172.21.15.22:0/165121210 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:57.190 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.645+0000 7f267b7fe640 1 -- 172.21.15.22:0/165121210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c07e0b0 con 0x7f267c107720 2024-09-19T08:05:57.191 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.645+0000 7f267b7fe640 1 --2- 172.21.15.22:0/165121210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267c107720 0x7f267c07db70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f266c009940 tx=0x7f266c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:57.191 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.645+0000 7f26797fa640 1 -- 172.21.15.22:0/165121210 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f266c042070 con 0x7f267c107720 2024-09-19T08:05:57.191 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.645+0000 7f26797fa640 1 -- 172.21.15.22:0/165121210 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f266c0308c0 con 0x7f267c107720 2024-09-19T08:05:57.191 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.645+0000 7f2682251640 1 -- 172.21.15.22:0/165121210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c07c2a0 con 0x7f267c107720 2024-09-19T08:05:57.191 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.645+0000 7f26797fa640 1 -- 172.21.15.22:0/165121210 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f266c037040 con 0x7f267c107720 2024-09-19T08:05:57.191 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.646+0000 7f2682251640 1 -- 172.21.15.22:0/165121210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c07c7e0 con 0x7f267c107720 2024-09-19T08:05:57.191 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.646+0000 7f26797fa640 1 -- 172.21.15.22:0/165121210 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f266c045070 con 0x7f267c107720 2024-09-19T08:05:57.191 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.647+0000 7f2682251640 1 -- 172.21.15.22:0/165121210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267c107ba0 con 0x7f267c107720 2024-09-19T08:05:57.191 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.647+0000 7f26797fa640 1 --2- 172.21.15.22:0/165121210 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f26680391b0 0x7f266803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:57.191 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.647+0000 7f26797fa640 1 -- 172.21.15.22:0/165121210 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f266c06c360 con 0x7f267c107720 2024-09-19T08:05:57.191 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.647+0000 7f2672ffd640 1 --2- 172.21.15.22:0/165121210 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f26680391b0 0x7f266803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:57.191 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.647+0000 7f2672ffd640 1 --2- 172.21.15.22:0/165121210 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f26680391b0 0x7f266803b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f26640098c0 tx=0x7f2664006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:57.191 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.652+0000 7f26797fa640 1 -- 172.21.15.22:0/165121210 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f266c03cd90 con 0x7f267c107720 2024-09-19T08:05:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.845+0000 7f2682251640 1 -- 172.21.15.22:0/165121210 --> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f267c104130 con 0x7f26680391b0 2024-09-19T08:05:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.850+0000 7f26797fa640 1 -- 172.21.15.22:0/165121210 <== mgr.14118 v2:172.21.15.22:6800/4065414102 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f267c104130 con 0x7f26680391b0 2024-09-19T08:05:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.852+0000 7f2682251640 1 -- 172.21.15.22:0/165121210 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f26680391b0 msgr2=0x7f266803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.852+0000 7f2682251640 1 --2- 172.21.15.22:0/165121210 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f26680391b0 0x7f266803b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f26640098c0 tx=0x7f2664006db0 comp rx=0 tx=0).stop 2024-09-19T08:05:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.852+0000 7f2682251640 1 -- 172.21.15.22:0/165121210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267c107720 msgr2=0x7f267c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.852+0000 7f2682251640 1 --2- 172.21.15.22:0/165121210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267c107720 0x7f267c07db70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f266c009940 tx=0x7f266c032ce0 comp rx=0 tx=0).stop 2024-09-19T08:05:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.852+0000 7f2682251640 1 -- 172.21.15.22:0/165121210 shutdown_connections 2024-09-19T08:05:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.852+0000 7f2682251640 1 --2- 172.21.15.22:0/165121210 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f26680391b0 0x7f266803b670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.852+0000 7f2682251640 1 --2- 172.21.15.22:0/165121210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267c107720 0x7f267c07db70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.852+0000 7f2682251640 1 -- 172.21.15.22:0/165121210 >> 172.21.15.22:0/165121210 conn(0x7f267c102ef0 msgr2=0x7f267c103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.852+0000 7f2682251640 1 -- 172.21.15.22:0/165121210 shutdown_connections 2024-09-19T08:05:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:55.852+0000 7f2682251640 1 -- 172.21.15.22:0/165121210 wait complete. 2024-09-19T08:05:58.089 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:57 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' 2024-09-19T08:05:58.089 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:57 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' 2024-09-19T08:05:59.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:58 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2081150542' entity='client.admin' 2024-09-19T08:05:59.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:05:58 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' 2024-09-19T08:05:59.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.687+0000 7f7c96b58640 1 Processor -- start 2024-09-19T08:05:59.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.687+0000 7f7c96b58640 1 -- start start 2024-09-19T08:05:59.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.687+0000 7f7c96b58640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c901081a0 0x7f7c9010a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:59.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.687+0000 7f7c96b58640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9007be70 con 0x7f7c901081a0 2024-09-19T08:05:59.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.687+0000 7f7c948f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c901081a0 0x7f7c9010a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:59.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.687+0000 7f7c948f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c901081a0 0x7f7c9010a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59958/0 (socket says 172.21.15.22:59958) 2024-09-19T08:05:59.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.687+0000 7f7c948f4640 1 -- 172.21.15.22:0/1922464821 learned_addr learned my addr 172.21.15.22:0/1922464821 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:59.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.688+0000 7f7c948f4640 1 -- 172.21.15.22:0/1922464821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9010aad0 con 0x7f7c901081a0 2024-09-19T08:05:59.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.688+0000 7f7c948f4640 1 --2- 172.21.15.22:0/1922464821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c901081a0 0x7f7c9010a590 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009820 tx=0x7f7c8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e841fe4668355441 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:59.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.688+0000 7f7c8f7fe640 1 -- 172.21.15.22:0/1922464821 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c84037070 con 0x7f7c901081a0 2024-09-19T08:05:59.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.689+0000 7f7c8f7fe640 1 -- 172.21.15.22:0/1922464821 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c84032920 con 0x7f7c901081a0 2024-09-19T08:05:59.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.689+0000 7f7c8f7fe640 1 -- 172.21.15.22:0/1922464821 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c84030810 con 0x7f7c901081a0 2024-09-19T08:05:59.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.689+0000 7f7c96b58640 1 -- 172.21.15.22:0/1922464821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c901081a0 msgr2=0x7f7c9010a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:59.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.689+0000 7f7c96b58640 1 --2- 172.21.15.22:0/1922464821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c901081a0 0x7f7c9010a590 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009820 tx=0x7f7c8402a270 comp rx=0 tx=0).stop 2024-09-19T08:05:59.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.689+0000 7f7c96b58640 1 -- 172.21.15.22:0/1922464821 shutdown_connections 2024-09-19T08:05:59.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.689+0000 7f7c96b58640 1 --2- 172.21.15.22:0/1922464821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c901081a0 0x7f7c9010a590 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:59.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.689+0000 7f7c96b58640 1 -- 172.21.15.22:0/1922464821 >> 172.21.15.22:0/1922464821 conn(0x7f7c90101e20 msgr2=0x7f7c90104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:59.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.689+0000 7f7c96b58640 1 -- 172.21.15.22:0/1922464821 shutdown_connections 2024-09-19T08:05:59.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.689+0000 7f7c96b58640 1 -- 172.21.15.22:0/1922464821 wait complete. 2024-09-19T08:05:59.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.690+0000 7f7c96b58640 1 Processor -- start 2024-09-19T08:05:59.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.690+0000 7f7c96b58640 1 -- start start 2024-09-19T08:05:59.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.690+0000 7f7c96b58640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c901081a0 0x7f7c9019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:59.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.690+0000 7f7c96b58640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9010bb50 con 0x7f7c901081a0 2024-09-19T08:05:59.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.690+0000 7f7c948f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c901081a0 0x7f7c9019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:59.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.690+0000 7f7c948f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c901081a0 0x7f7c9019b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59974/0 (socket says 172.21.15.22:59974) 2024-09-19T08:05:59.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.690+0000 7f7c948f4640 1 -- 172.21.15.22:0/2081150542 learned_addr learned my addr 172.21.15.22:0/2081150542 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:05:59.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.690+0000 7f7c948f4640 1 -- 172.21.15.22:0/2081150542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9019bc50 con 0x7f7c901081a0 2024-09-19T08:05:59.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.691+0000 7f7c948f4640 1 --2- 172.21.15.22:0/2081150542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c901081a0 0x7f7c9019b710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7c840097f0 tx=0x7f7c840327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:59.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.691+0000 7f7c8dffb640 1 -- 172.21.15.22:0/2081150542 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c84037040 con 0x7f7c901081a0 2024-09-19T08:05:59.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.691+0000 7f7c8dffb640 1 -- 172.21.15.22:0/2081150542 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c84030c40 con 0x7f7c901081a0 2024-09-19T08:05:59.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.691+0000 7f7c96b58640 1 -- 172.21.15.22:0/2081150542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9019bee0 con 0x7f7c901081a0 2024-09-19T08:05:59.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.691+0000 7f7c8dffb640 1 -- 172.21.15.22:0/2081150542 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c8403a5d0 con 0x7f7c901081a0 2024-09-19T08:05:59.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.691+0000 7f7c96b58640 1 -- 172.21.15.22:0/2081150542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9019ebd0 con 0x7f7c901081a0 2024-09-19T08:05:59.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.692+0000 7f7c8dffb640 1 -- 172.21.15.22:0/2081150542 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f7c8403a7c0 con 0x7f7c901081a0 2024-09-19T08:05:59.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.692+0000 7f7c96b58640 1 -- 172.21.15.22:0/2081150542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c90108600 con 0x7f7c901081a0 2024-09-19T08:05:59.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.692+0000 7f7c8dffb640 1 --2- 172.21.15.22:0/2081150542 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f7c80039200 0x7f7c8003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:05:59.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.692+0000 7f7c8dffb640 1 -- 172.21.15.22:0/2081150542 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7c8406c910 con 0x7f7c901081a0 2024-09-19T08:05:59.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.693+0000 7f7c8ffff640 1 --2- 172.21.15.22:0/2081150542 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f7c80039200 0x7f7c8003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:05:59.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.693+0000 7f7c8ffff640 1 --2- 172.21.15.22:0/2081150542 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f7c80039200 0x7f7c8003b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0098c0 tx=0x7f7c7c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:05:59.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.698+0000 7f7c8dffb640 1 -- 172.21.15.22:0/2081150542 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7c840371e0 con 0x7f7c901081a0 2024-09-19T08:05:59.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.909+0000 7f7c96b58640 1 -- 172.21.15.22:0/2081150542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f7c9019f1c0 con 0x7f7c901081a0 2024-09-19T08:05:59.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.912+0000 7f7c8dffb640 1 -- 172.21.15.22:0/2081150542 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f7c840319f0 con 0x7f7c901081a0 2024-09-19T08:05:59.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.914+0000 7f7c96b58640 1 -- 172.21.15.22:0/2081150542 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f7c80039200 msgr2=0x7f7c8003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:59.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.914+0000 7f7c96b58640 1 --2- 172.21.15.22:0/2081150542 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f7c80039200 0x7f7c8003b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0098c0 tx=0x7f7c7c006db0 comp rx=0 tx=0).stop 2024-09-19T08:05:59.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.914+0000 7f7c96b58640 1 -- 172.21.15.22:0/2081150542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c901081a0 msgr2=0x7f7c9019b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:05:59.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.914+0000 7f7c96b58640 1 --2- 172.21.15.22:0/2081150542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c901081a0 0x7f7c9019b710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7c840097f0 tx=0x7f7c840327b0 comp rx=0 tx=0).stop 2024-09-19T08:05:59.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.914+0000 7f7c96b58640 1 -- 172.21.15.22:0/2081150542 shutdown_connections 2024-09-19T08:05:59.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.914+0000 7f7c96b58640 1 --2- 172.21.15.22:0/2081150542 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f7c80039200 0x7f7c8003b6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:59.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.914+0000 7f7c96b58640 1 --2- 172.21.15.22:0/2081150542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c901081a0 0x7f7c9019b710 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:05:59.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.914+0000 7f7c96b58640 1 -- 172.21.15.22:0/2081150542 >> 172.21.15.22:0/2081150542 conn(0x7f7c90101e20 msgr2=0x7f7c90102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:05:59.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.914+0000 7f7c96b58640 1 -- 172.21.15.22:0/2081150542 shutdown_connections 2024-09-19T08:05:59.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:57.914+0000 7f7c96b58640 1 -- 172.21.15.22:0/2081150542 wait complete. 2024-09-19T08:06:00.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.566+0000 7f27f2924640 1 Processor -- start 2024-09-19T08:06:00.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.566+0000 7f27f2924640 1 -- start start 2024-09-19T08:06:00.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.566+0000 7f27f2924640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27ec108150 0x7f27ec10a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:00.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.566+0000 7f27f2924640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec07beb0 con 0x7f27ec108150 2024-09-19T08:06:00.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.566+0000 7f27ebfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27ec108150 0x7f27ec10a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:00.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.566+0000 7f27ebfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27ec108150 0x7f27ec10a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59976/0 (socket says 172.21.15.22:59976) 2024-09-19T08:06:00.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.567+0000 7f27ebfff640 1 -- 172.21.15.22:0/2585041170 learned_addr learned my addr 172.21.15.22:0/2585041170 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:00.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.567+0000 7f27ebfff640 1 -- 172.21.15.22:0/2585041170 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec10aa80 con 0x7f27ec108150 2024-09-19T08:06:00.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.567+0000 7f27ebfff640 1 --2- 172.21.15.22:0/2585041170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27ec108150 0x7f27ec10a540 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009820 tx=0x7f27dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134d258c0e83d3eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:00.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.568+0000 7f27eaffd640 1 -- 172.21.15.22:0/2585041170 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27dc037070 con 0x7f27ec108150 2024-09-19T08:06:00.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.568+0000 7f27eaffd640 1 -- 172.21.15.22:0/2585041170 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27dc032920 con 0x7f27ec108150 2024-09-19T08:06:00.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.568+0000 7f27eaffd640 1 -- 172.21.15.22:0/2585041170 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27dc030810 con 0x7f27ec108150 2024-09-19T08:06:00.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.568+0000 7f27f2924640 1 -- 172.21.15.22:0/2585041170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27ec108150 msgr2=0x7f27ec10a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:00.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.569+0000 7f27f2924640 1 --2- 172.21.15.22:0/2585041170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27ec108150 0x7f27ec10a540 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009820 tx=0x7f27dc02a270 comp rx=0 tx=0).stop 2024-09-19T08:06:00.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.569+0000 7f27f2924640 1 -- 172.21.15.22:0/2585041170 shutdown_connections 2024-09-19T08:06:00.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.569+0000 7f27f2924640 1 --2- 172.21.15.22:0/2585041170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27ec108150 0x7f27ec10a540 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:00.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.569+0000 7f27f2924640 1 -- 172.21.15.22:0/2585041170 >> 172.21.15.22:0/2585041170 conn(0x7f27ec101d90 msgr2=0x7f27ec1041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:00.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.569+0000 7f27f2924640 1 -- 172.21.15.22:0/2585041170 shutdown_connections 2024-09-19T08:06:00.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.569+0000 7f27f2924640 1 -- 172.21.15.22:0/2585041170 wait complete. 2024-09-19T08:06:00.357 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.569+0000 7f27f2924640 1 Processor -- start 2024-09-19T08:06:00.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.569+0000 7f27f2924640 1 -- start start 2024-09-19T08:06:00.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.570+0000 7f27f2924640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27ec108150 0x7f27ec19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:00.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.570+0000 7f27f2924640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec10bb60 con 0x7f27ec108150 2024-09-19T08:06:00.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.570+0000 7f27ebfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27ec108150 0x7f27ec19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:00.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.570+0000 7f27ebfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27ec108150 0x7f27ec19d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59988/0 (socket says 172.21.15.22:59988) 2024-09-19T08:06:00.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.570+0000 7f27ebfff640 1 -- 172.21.15.22:0/3985812138 learned_addr learned my addr 172.21.15.22:0/3985812138 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:00.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.570+0000 7f27ebfff640 1 -- 172.21.15.22:0/3985812138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec19de40 con 0x7f27ec108150 2024-09-19T08:06:00.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.570+0000 7f27ebfff640 1 --2- 172.21.15.22:0/3985812138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27ec108150 0x7f27ec19d900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f27dc030420 tx=0x7f27dc032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:00.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.571+0000 7f27e97fa640 1 -- 172.21.15.22:0/3985812138 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27dc037040 con 0x7f27ec108150 2024-09-19T08:06:00.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.571+0000 7f27f2924640 1 -- 172.21.15.22:0/3985812138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ec19e0d0 con 0x7f27ec108150 2024-09-19T08:06:00.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.571+0000 7f27e97fa640 1 -- 172.21.15.22:0/3985812138 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27dc040070 con 0x7f27ec108150 2024-09-19T08:06:00.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.571+0000 7f27e97fa640 1 -- 172.21.15.22:0/3985812138 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27dc03a6e0 con 0x7f27ec108150 2024-09-19T08:06:00.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.571+0000 7f27f2924640 1 -- 172.21.15.22:0/3985812138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ec1a0dc0 con 0x7f27ec108150 2024-09-19T08:06:00.358 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.572+0000 7f27e97fa640 1 -- 172.21.15.22:0/3985812138 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f27dc03a990 con 0x7f27ec108150 2024-09-19T08:06:00.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.572+0000 7f27f2924640 1 -- 172.21.15.22:0/3985812138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27ec1085b0 con 0x7f27ec108150 2024-09-19T08:06:00.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.572+0000 7f27e97fa640 1 --2- 172.21.15.22:0/3985812138 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f27d4039200 0x7f27d403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:00.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.572+0000 7f27e97fa640 1 -- 172.21.15.22:0/3985812138 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f27dc06c830 con 0x7f27ec108150 2024-09-19T08:06:00.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.572+0000 7f27eb7fe640 1 --2- 172.21.15.22:0/3985812138 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f27d4039200 0x7f27d403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:00.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.573+0000 7f27eb7fe640 1 --2- 172.21.15.22:0/3985812138 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f27d4039200 0x7f27d403b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f27e00098c0 tx=0x7f27e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:00.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.577+0000 7f27e97fa640 1 -- 172.21.15.22:0/3985812138 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f27dc042e10 con 0x7f27ec108150 2024-09-19T08:06:00.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.761+0000 7f27f2924640 1 -- 172.21.15.22:0/3985812138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f27ec1a1420 con 0x7f27ec108150 2024-09-19T08:06:00.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.764+0000 7f27e97fa640 1 -- 172.21.15.22:0/3985812138 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f27dc043b80 con 0x7f27ec108150 2024-09-19T08:06:00.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.766+0000 7f27f2924640 1 -- 172.21.15.22:0/3985812138 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f27d4039200 msgr2=0x7f27d403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:00.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.766+0000 7f27f2924640 1 --2- 172.21.15.22:0/3985812138 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f27d4039200 0x7f27d403b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f27e00098c0 tx=0x7f27e0006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:00.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.766+0000 7f27f2924640 1 -- 172.21.15.22:0/3985812138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27ec108150 msgr2=0x7f27ec19d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:00.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.766+0000 7f27f2924640 1 --2- 172.21.15.22:0/3985812138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27ec108150 0x7f27ec19d900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f27dc030420 tx=0x7f27dc032670 comp rx=0 tx=0).stop 2024-09-19T08:06:00.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.766+0000 7f27f2924640 1 -- 172.21.15.22:0/3985812138 shutdown_connections 2024-09-19T08:06:00.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.766+0000 7f27f2924640 1 --2- 172.21.15.22:0/3985812138 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f27d4039200 0x7f27d403b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:00.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.766+0000 7f27f2924640 1 --2- 172.21.15.22:0/3985812138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27ec108150 0x7f27ec19d900 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:00.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.766+0000 7f27f2924640 1 -- 172.21.15.22:0/3985812138 >> 172.21.15.22:0/3985812138 conn(0x7f27ec101d90 msgr2=0x7f27ec1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:00.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.767+0000 7f27f2924640 1 -- 172.21.15.22:0/3985812138 shutdown_connections 2024-09-19T08:06:00.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:05:59.767+0000 7f27f2924640 1 -- 172.21.15.22:0/3985812138 wait complete. 2024-09-19T08:06:00.360 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling the dashboard module... 2024-09-19T08:06:00.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:00 smithi022 ceph-mon[30963]: from='mgr.14118 172.21.15.22:0/318949662' entity='mgr.x' 2024-09-19T08:06:00.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:00 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3985812138' entity='client.admin' 2024-09-19T08:06:01.978 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:01 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3056342783' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T08:06:01.979 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: ignoring --setuser ceph since I am not root 2024-09-19T08:06:01.979 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: ignoring --setgroup ceph since I am not root 2024-09-19T08:06:01.979 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:01.763+0000 7efdd8f7c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:06:01.979 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:01.824+0000 7efdd8f7c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:06:01.979 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:01.978+0000 7efdd8f7c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:06:02.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:02.047+0000 7efdd8f7c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:06:02.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:02.204+0000 7efdd8f7c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:06:02.453 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.759+0000 7f9f8f60d640 1 Processor -- start 2024-09-19T08:06:02.454 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.759+0000 7f9f8f60d640 1 -- start start 2024-09-19T08:06:02.454 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.759+0000 7f9f8f60d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8807de70 0x7f9f8807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:02.454 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.759+0000 7f9f8f60d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8807c8f0 con 0x7f9f8807de70 2024-09-19T08:06:02.454 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.760+0000 7f9f8d3a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8807de70 0x7f9f8807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:02.454 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.760+0000 7f9f8d3a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8807de70 0x7f9f8807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59998/0 (socket says 172.21.15.22:59998) 2024-09-19T08:06:02.454 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.760+0000 7f9f8d3a9640 1 -- 172.21.15.22:0/3271637278 learned_addr learned my addr 172.21.15.22:0/3271637278 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:02.454 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.760+0000 7f9f8d3a9640 1 -- 172.21.15.22:0/3271637278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8807ca70 con 0x7f9f8807de70 2024-09-19T08:06:02.454 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.760+0000 7f9f8d3a9640 1 --2- 172.21.15.22:0/3271637278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8807de70 0x7f9f8807c320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009820 tx=0x7f9f8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcb8355610a7015 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:02.454 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.761+0000 7f9f7ffff640 1 -- 172.21.15.22:0/3271637278 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f84037070 con 0x7f9f8807de70 2024-09-19T08:06:02.454 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.761+0000 7f9f7ffff640 1 -- 172.21.15.22:0/3271637278 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f84032920 con 0x7f9f8807de70 2024-09-19T08:06:02.454 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.761+0000 7f9f7ffff640 1 -- 172.21.15.22:0/3271637278 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f84030810 con 0x7f9f8807de70 2024-09-19T08:06:02.455 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.761+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3271637278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8807de70 msgr2=0x7f9f8807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:02.455 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.761+0000 7f9f8f60d640 1 --2- 172.21.15.22:0/3271637278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8807de70 0x7f9f8807c320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009820 tx=0x7f9f8402a270 comp rx=0 tx=0).stop 2024-09-19T08:06:02.455 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.761+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3271637278 shutdown_connections 2024-09-19T08:06:02.455 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.761+0000 7f9f8f60d640 1 --2- 172.21.15.22:0/3271637278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8807de70 0x7f9f8807c320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:02.455 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.761+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3271637278 >> 172.21.15.22:0/3271637278 conn(0x7f9f88102ef0 msgr2=0x7f9f88105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:02.455 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.761+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3271637278 shutdown_connections 2024-09-19T08:06:02.455 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.761+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3271637278 wait complete. 2024-09-19T08:06:02.455 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.762+0000 7f9f8f60d640 1 Processor -- start 2024-09-19T08:06:02.455 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.762+0000 7f9f8f60d640 1 -- start start 2024-09-19T08:06:02.455 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.762+0000 7f9f8f60d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8807de70 0x7f9f8819d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:02.455 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.762+0000 7f9f8f60d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8810dd70 con 0x7f9f8807de70 2024-09-19T08:06:02.455 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.762+0000 7f9f8d3a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8807de70 0x7f9f8819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:02.455 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.762+0000 7f9f8d3a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8807de70 0x7f9f8819d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60014/0 (socket says 172.21.15.22:60014) 2024-09-19T08:06:02.455 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.762+0000 7f9f8d3a9640 1 -- 172.21.15.22:0/3056342783 learned_addr learned my addr 172.21.15.22:0/3056342783 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:02.456 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.763+0000 7f9f8d3a9640 1 -- 172.21.15.22:0/3056342783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8819dda0 con 0x7f9f8807de70 2024-09-19T08:06:02.456 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.763+0000 7f9f8d3a9640 1 --2- 172.21.15.22:0/3056342783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8807de70 0x7f9f8819d860 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f84030380 tx=0x7f9f84032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:02.456 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.763+0000 7f9f7e7fc640 1 -- 172.21.15.22:0/3056342783 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f84037040 con 0x7f9f8807de70 2024-09-19T08:06:02.456 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.763+0000 7f9f7e7fc640 1 -- 172.21.15.22:0/3056342783 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f84040070 con 0x7f9f8807de70 2024-09-19T08:06:02.456 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.763+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3056342783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8819e030 con 0x7f9f8807de70 2024-09-19T08:06:02.456 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.763+0000 7f9f7e7fc640 1 -- 172.21.15.22:0/3056342783 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f8403a7b0 con 0x7f9f8807de70 2024-09-19T08:06:02.456 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.763+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3056342783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f881a0d20 con 0x7f9f8807de70 2024-09-19T08:06:02.456 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.764+0000 7f9f7e7fc640 1 -- 172.21.15.22:0/3056342783 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f9f8403bc00 con 0x7f9f8807de70 2024-09-19T08:06:02.456 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.764+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3056342783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f8807c3a0 con 0x7f9f8807de70 2024-09-19T08:06:02.456 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.764+0000 7f9f7e7fc640 1 --2- 172.21.15.22:0/3056342783 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f9f74039200 0x7f9f7403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:02.456 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.764+0000 7f9f7e7fc640 1 -- 172.21.15.22:0/3056342783 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9f8406c860 con 0x7f9f8807de70 2024-09-19T08:06:02.456 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.764+0000 7f9f8cba8640 1 --2- 172.21.15.22:0/3056342783 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f9f74039200 0x7f9f7403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.765+0000 7f9f8cba8640 1 --2- 172.21.15.22:0/3056342783 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f9f74039200 0x7f9f7403b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9f780098c0 tx=0x7f9f78006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:00.769+0000 7f9f7e7fc640 1 -- 172.21.15.22:0/3056342783 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9f840313a0 con 0x7f9f8807de70 2024-09-19T08:06:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:01.004+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3056342783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f9f881a1310 con 0x7f9f8807de70 2024-09-19T08:06:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:01.605+0000 7f9f7e7fc640 1 -- 172.21.15.22:0/3056342783 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f9f8404a200 con 0x7f9f8807de70 2024-09-19T08:06:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:01.607+0000 7f9f7e7fc640 1 -- 172.21.15.22:0/3056342783 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f9f8406cbc0 con 0x7f9f8807de70 2024-09-19T08:06:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:01.609+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3056342783 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f9f74039200 msgr2=0x7f9f7403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:01.609+0000 7f9f8f60d640 1 --2- 172.21.15.22:0/3056342783 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f9f74039200 0x7f9f7403b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9f780098c0 tx=0x7f9f78006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:01.609+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3056342783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8807de70 msgr2=0x7f9f8819d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:01.609+0000 7f9f8f60d640 1 --2- 172.21.15.22:0/3056342783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8807de70 0x7f9f8819d860 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f84030380 tx=0x7f9f84032670 comp rx=0 tx=0).stop 2024-09-19T08:06:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:01.609+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3056342783 shutdown_connections 2024-09-19T08:06:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:01.609+0000 7f9f8f60d640 1 --2- 172.21.15.22:0/3056342783 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f9f74039200 0x7f9f7403b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:01.609+0000 7f9f8f60d640 1 --2- 172.21.15.22:0/3056342783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8807de70 0x7f9f8819d860 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:01.609+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3056342783 >> 172.21.15.22:0/3056342783 conn(0x7f9f88102ef0 msgr2=0x7f9f88103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:01.609+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3056342783 shutdown_connections 2024-09-19T08:06:02.458 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:01.609+0000 7f9f8f60d640 1 -- 172.21.15.22:0/3056342783 wait complete. 2024-09-19T08:06:02.606 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:02.351+0000 7efdd8f7c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:06:02.863 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:02 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3056342783' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T08:06:02.863 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:02 smithi022 ceph-mon[30963]: mgrmap e9: x(active, since 19s) 2024-09-19T08:06:02.863 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:02.687+0000 7efdd8f7c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:06:02.863 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:02.750+0000 7efdd8f7c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:06:03.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:02.949+0000 7efdd8f7c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:06:03.689 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:06:03.689 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T08:06:03.689 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T08:06:03.689 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T08:06:03.689 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T08:06:03.689 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:06:03.690 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.863+0000 7f86f822e640 1 Processor -- start 2024-09-19T08:06:03.690 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.863+0000 7f86f822e640 1 -- start start 2024-09-19T08:06:03.690 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.863+0000 7f86f822e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f0105f50 0x7f86f0108360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:03.690 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.863+0000 7f86f822e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f86f007beb0 con 0x7f86f0105f50 2024-09-19T08:06:03.690 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.863+0000 7f86f5fca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f0105f50 0x7f86f0108360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:03.690 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.863+0000 7f86f5fca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f0105f50 0x7f86f0108360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60044/0 (socket says 172.21.15.22:60044) 2024-09-19T08:06:03.690 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.863+0000 7f86f5fca640 1 -- 172.21.15.22:0/2741307155 learned_addr learned my addr 172.21.15.22:0/2741307155 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:03.690 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.864+0000 7f86f5fca640 1 -- 172.21.15.22:0/2741307155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f01088a0 con 0x7f86f0105f50 2024-09-19T08:06:03.690 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.865+0000 7f86f5fca640 1 --2- 172.21.15.22:0/2741307155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f0105f50 0x7f86f0108360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009820 tx=0x7f86ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecc14a61590f5f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:03.690 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.865+0000 7f86f4fc8640 1 -- 172.21.15.22:0/2741307155 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86ec037070 con 0x7f86f0105f50 2024-09-19T08:06:03.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.865+0000 7f86f4fc8640 1 -- 172.21.15.22:0/2741307155 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ec032920 con 0x7f86f0105f50 2024-09-19T08:06:03.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.865+0000 7f86f4fc8640 1 -- 172.21.15.22:0/2741307155 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86ec030810 con 0x7f86f0105f50 2024-09-19T08:06:03.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.865+0000 7f86f822e640 1 -- 172.21.15.22:0/2741307155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f0105f50 msgr2=0x7f86f0108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:03.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.865+0000 7f86f822e640 1 --2- 172.21.15.22:0/2741307155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f0105f50 0x7f86f0108360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009820 tx=0x7f86ec02a270 comp rx=0 tx=0).stop 2024-09-19T08:06:03.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.865+0000 7f86f822e640 1 -- 172.21.15.22:0/2741307155 shutdown_connections 2024-09-19T08:06:03.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.865+0000 7f86f822e640 1 --2- 172.21.15.22:0/2741307155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f0105f50 0x7f86f0108360 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:03.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.865+0000 7f86f822e640 1 -- 172.21.15.22:0/2741307155 >> 172.21.15.22:0/2741307155 conn(0x7f86f0101e20 msgr2=0x7f86f0104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:03.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.866+0000 7f86f822e640 1 -- 172.21.15.22:0/2741307155 shutdown_connections 2024-09-19T08:06:03.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.866+0000 7f86f822e640 1 -- 172.21.15.22:0/2741307155 wait complete. 2024-09-19T08:06:03.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.866+0000 7f86f822e640 1 Processor -- start 2024-09-19T08:06:03.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.866+0000 7f86f822e640 1 -- start start 2024-09-19T08:06:03.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.866+0000 7f86f822e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f0105f50 0x7f86f01a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:03.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.866+0000 7f86f822e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01099e0 con 0x7f86f0105f50 2024-09-19T08:06:03.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.867+0000 7f86f5fca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f0105f50 0x7f86f01a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:03.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.867+0000 7f86f5fca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f0105f50 0x7f86f01a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60056/0 (socket says 172.21.15.22:60056) 2024-09-19T08:06:03.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.867+0000 7f86f5fca640 1 -- 172.21.15.22:0/15297723 learned_addr learned my addr 172.21.15.22:0/15297723 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:03.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.867+0000 7f86f5fca640 1 -- 172.21.15.22:0/15297723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f01a2220 con 0x7f86f0105f50 2024-09-19T08:06:03.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.867+0000 7f86f5fca640 1 --2- 172.21.15.22:0/15297723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f0105f50 0x7f86f01a1ce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f86ec030420 tx=0x7f86ec032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:03.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.867+0000 7f86e6ffd640 1 -- 172.21.15.22:0/15297723 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86ec037040 con 0x7f86f0105f50 2024-09-19T08:06:03.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.867+0000 7f86e6ffd640 1 -- 172.21.15.22:0/15297723 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ec040070 con 0x7f86f0105f50 2024-09-19T08:06:03.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.867+0000 7f86f822e640 1 -- 172.21.15.22:0/15297723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f01a24b0 con 0x7f86f0105f50 2024-09-19T08:06:03.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.867+0000 7f86e6ffd640 1 -- 172.21.15.22:0/15297723 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86ec03a6e0 con 0x7f86f0105f50 2024-09-19T08:06:03.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.867+0000 7f86f822e640 1 -- 172.21.15.22:0/15297723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01a5190 con 0x7f86f0105f50 2024-09-19T08:06:03.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.868+0000 7f86f822e640 1 -- 172.21.15.22:0/15297723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f0106350 con 0x7f86f0105f50 2024-09-19T08:06:03.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.868+0000 7f86e6ffd640 1 -- 172.21.15.22:0/15297723 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f86ec03bb20 con 0x7f86f0105f50 2024-09-19T08:06:03.693 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.869+0000 7f86e6ffd640 1 --2- 172.21.15.22:0/15297723 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f86dc039260 0x7f86dc03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:03.693 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.869+0000 7f86e6ffd640 1 -- 172.21.15.22:0/15297723 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f86ec06c8b0 con 0x7f86f0105f50 2024-09-19T08:06:03.693 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.869+0000 7f86f57c9640 1 -- 172.21.15.22:0/15297723 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f86dc039260 msgr2=0x7f86dc03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4065414102 2024-09-19T08:06:03.693 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.869+0000 7f86f57c9640 1 --2- 172.21.15.22:0/15297723 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f86dc039260 0x7f86dc03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:06:03.693 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:02.873+0000 7f86e6ffd640 1 -- 172.21.15.22:0/15297723 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f86ec043360 con 0x7f86f0105f50 2024-09-19T08:06:03.693 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:03.070+0000 7f86f57c9640 1 -- 172.21.15.22:0/15297723 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f86dc039260 msgr2=0x7f86dc03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4065414102 2024-09-19T08:06:03.693 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:03.070+0000 7f86f57c9640 1 --2- 172.21.15.22:0/15297723 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f86dc039260 0x7f86dc03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:06:03.693 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:03.096+0000 7f86f822e640 1 -- 172.21.15.22:0/15297723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f86f002da10 con 0x7f86f0105f50 2024-09-19T08:06:03.693 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:03.097+0000 7f86e6ffd640 1 -- 172.21.15.22:0/15297723 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f86ec040220 con 0x7f86f0105f50 2024-09-19T08:06:03.693 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:03.099+0000 7f86f822e640 1 -- 172.21.15.22:0/15297723 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f86dc039260 msgr2=0x7f86dc03b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:06:03.694 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:03.099+0000 7f86f822e640 1 --2- 172.21.15.22:0/15297723 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f86dc039260 0x7f86dc03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:03.694 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:03.099+0000 7f86f822e640 1 -- 172.21.15.22:0/15297723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f0105f50 msgr2=0x7f86f01a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:03.694 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:03.099+0000 7f86f822e640 1 --2- 172.21.15.22:0/15297723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f0105f50 0x7f86f01a1ce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f86ec030420 tx=0x7f86ec032670 comp rx=0 tx=0).stop 2024-09-19T08:06:03.694 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:03.099+0000 7f86f822e640 1 -- 172.21.15.22:0/15297723 shutdown_connections 2024-09-19T08:06:03.694 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:03.099+0000 7f86f822e640 1 --2- 172.21.15.22:0/15297723 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7f86dc039260 0x7f86dc03b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:03.694 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:03.099+0000 7f86f822e640 1 --2- 172.21.15.22:0/15297723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f0105f50 0x7f86f01a1ce0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:03.694 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:03.099+0000 7f86f822e640 1 -- 172.21.15.22:0/15297723 >> 172.21.15.22:0/15297723 conn(0x7f86f0101e20 msgr2=0x7f86f0104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:03.694 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:03.099+0000 7f86f822e640 1 -- 172.21.15.22:0/15297723 shutdown_connections 2024-09-19T08:06:03.695 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:03.099+0000 7f86f822e640 1 -- 172.21.15.22:0/15297723 wait complete. 2024-09-19T08:06:03.695 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2024-09-19T08:06:03.695 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 9... 2024-09-19T08:06:03.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:03 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/15297723' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T08:06:04.073 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:03.888+0000 7efdd8f7c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:06:04.073 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:03.960+0000 7efdd8f7c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:06:04.980 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:04 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:04.711+0000 7efdd8f7c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:06:04.980 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:04 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:04.831+0000 7efdd8f7c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:06:05.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:04 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:04.980+0000 7efdd8f7c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:06:05.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:05.043+0000 7efdd8f7c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:06:05.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:05.199+0000 7efdd8f7c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:06:05.566 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:05.270+0000 7efdd8f7c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:06:05.566 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:05.333+0000 7efdd8f7c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:06:05.566 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:05.498+0000 7efdd8f7c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:06:05.567 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:05.566+0000 7efdd8f7c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:06:05.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:05.635+0000 7efdd8f7c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:06:05.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:05.722+0000 7efdd8f7c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:06:05.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:05.784+0000 7efdd8f7c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:06:05.846 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:05.845+0000 7efdd8f7c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:06:06.933 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:06 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:06.674+0000 7efdd8f7c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:06:06.933 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:06 smithi022 ceph-mon[30963]: Active manager daemon x restarted 2024-09-19T08:06:06.933 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:06 smithi022 ceph-mon[30963]: Activating manager daemon x 2024-09-19T08:06:06.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:06 smithi022 ceph-mon[30963]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T08:06:06.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:06 smithi022 ceph-mon[30963]: mgrmap e10: x(active, starting, since 0.00339323s) 2024-09-19T08:06:06.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:06:06.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:06:06.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:06:06.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:06:06.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:06:06.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:06 smithi022 ceph-mon[30963]: Manager daemon x is now available 2024-09-19T08:06:06.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:06:06.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:06:06.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:06:08.373 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:06:08.373 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T08:06:08.373 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T08:06:08.373 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:06:08.373 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.079+0000 7fbaefc58640 1 Processor -- start 2024-09-19T08:06:08.373 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.080+0000 7fbaefc58640 1 -- start start 2024-09-19T08:06:08.373 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.080+0000 7fbaefc58640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae81059b0 0x7fbae8105db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:08.373 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.080+0000 7fbaefc58640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbae8106380 con 0x7fbae81059b0 2024-09-19T08:06:08.373 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.080+0000 7fbaed9f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae81059b0 0x7fbae8105db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:08.374 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.080+0000 7fbaed9f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae81059b0 0x7fbae8105db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60072/0 (socket says 172.21.15.22:60072) 2024-09-19T08:06:08.374 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.080+0000 7fbaed9f4640 1 -- 172.21.15.22:0/46089408 learned_addr learned my addr 172.21.15.22:0/46089408 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:08.374 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.080+0000 7fbaed9f4640 1 -- 172.21.15.22:0/46089408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae8108990 con 0x7fbae81059b0 2024-09-19T08:06:08.374 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.081+0000 7fbaed9f4640 1 --2- 172.21.15.22:0/46089408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae81059b0 0x7fbae8105db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009820 tx=0x7fbae402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c58f5e4d7a3cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:08.374 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.081+0000 7fbaec9f2640 1 -- 172.21.15.22:0/46089408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae4037070 con 0x7fbae81059b0 2024-09-19T08:06:08.374 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.081+0000 7fbaec9f2640 1 -- 172.21.15.22:0/46089408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae4032920 con 0x7fbae81059b0 2024-09-19T08:06:08.374 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.081+0000 7fbaec9f2640 1 -- 172.21.15.22:0/46089408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae4030810 con 0x7fbae81059b0 2024-09-19T08:06:08.374 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.082+0000 7fbaefc58640 1 -- 172.21.15.22:0/46089408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae81059b0 msgr2=0x7fbae8105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:08.374 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.082+0000 7fbaefc58640 1 --2- 172.21.15.22:0/46089408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae81059b0 0x7fbae8105db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009820 tx=0x7fbae402a270 comp rx=0 tx=0).stop 2024-09-19T08:06:08.374 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.082+0000 7fbaefc58640 1 -- 172.21.15.22:0/46089408 shutdown_connections 2024-09-19T08:06:08.374 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.082+0000 7fbaefc58640 1 --2- 172.21.15.22:0/46089408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae81059b0 0x7fbae8105db0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:08.374 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.082+0000 7fbaefc58640 1 -- 172.21.15.22:0/46089408 >> 172.21.15.22:0/46089408 conn(0x7fbae81014e0 msgr2=0x7fbae8103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:08.374 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.082+0000 7fbaefc58640 1 -- 172.21.15.22:0/46089408 shutdown_connections 2024-09-19T08:06:08.375 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.082+0000 7fbaefc58640 1 -- 172.21.15.22:0/46089408 wait complete. 2024-09-19T08:06:08.375 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.083+0000 7fbaefc58640 1 Processor -- start 2024-09-19T08:06:08.375 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.083+0000 7fbaefc58640 1 -- start start 2024-09-19T08:06:08.375 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.083+0000 7fbaefc58640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae81059b0 0x7fbae8108210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:08.375 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.083+0000 7fbaefc58640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbae810bbc0 con 0x7fbae81059b0 2024-09-19T08:06:08.375 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.083+0000 7fbaed9f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae81059b0 0x7fbae8108210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:08.375 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.083+0000 7fbaed9f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae81059b0 0x7fbae8108210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60076/0 (socket says 172.21.15.22:60076) 2024-09-19T08:06:08.375 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.083+0000 7fbaed9f4640 1 -- 172.21.15.22:0/2488394860 learned_addr learned my addr 172.21.15.22:0/2488394860 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:08.375 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.083+0000 7fbaed9f4640 1 -- 172.21.15.22:0/2488394860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae8108750 con 0x7fbae81059b0 2024-09-19T08:06:08.375 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.084+0000 7fbaed9f4640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae81059b0 0x7fbae8108210 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbae4030560 tx=0x7fbae4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:08.375 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.084+0000 7fbadeffd640 1 -- 172.21.15.22:0/2488394860 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae4037040 con 0x7fbae81059b0 2024-09-19T08:06:08.375 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.084+0000 7fbadeffd640 1 -- 172.21.15.22:0/2488394860 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae4040070 con 0x7fbae81059b0 2024-09-19T08:06:08.376 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.084+0000 7fbadeffd640 1 -- 172.21.15.22:0/2488394860 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae403a6d0 con 0x7fbae81059b0 2024-09-19T08:06:08.376 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.084+0000 7fbaefc58640 1 -- 172.21.15.22:0/2488394860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae8106940 con 0x7fbae81059b0 2024-09-19T08:06:08.376 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.084+0000 7fbaefc58640 1 -- 172.21.15.22:0/2488394860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae8106e80 con 0x7fbae81059b0 2024-09-19T08:06:08.376 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.085+0000 7fbadeffd640 1 -- 172.21.15.22:0/2488394860 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fbae403a870 con 0x7fbae81059b0 2024-09-19T08:06:08.376 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.085+0000 7fbadeffd640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fbad4039260 0x7fbad403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:08.376 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.085+0000 7fbadeffd640 1 -- 172.21.15.22:0/2488394860 --> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbad403be30 con 0x7fbad4039260 2024-09-19T08:06:08.376 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.085+0000 7fbadeffd640 1 -- 172.21.15.22:0/2488394860 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbae406c940 con 0x7fbae81059b0 2024-09-19T08:06:08.376 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.085+0000 7fbaed1f3640 1 -- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fbad4039260 msgr2=0x7fbad403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4065414102 2024-09-19T08:06:08.376 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.085+0000 7fbaed1f3640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fbad4039260 0x7fbad403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:06:08.376 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.286+0000 7fbaed1f3640 1 -- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fbad4039260 msgr2=0x7fbad403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4065414102 2024-09-19T08:06:08.376 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.286+0000 7fbaed1f3640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fbad4039260 0x7fbad403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:06:08.376 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.687+0000 7fbaed1f3640 1 -- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fbad4039260 msgr2=0x7fbad403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4065414102 2024-09-19T08:06:08.376 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:04.687+0000 7fbaed1f3640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fbad4039260 0x7fbad403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T08:06:08.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:05.488+0000 7fbaed1f3640 1 -- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fbad4039260 msgr2=0x7fbad403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4065414102 2024-09-19T08:06:08.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:05.488+0000 7fbaed1f3640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fbad4039260 0x7fbad403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T08:06:08.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:06.677+0000 7fbadeffd640 1 -- 172.21.15.22:0/2488394860 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7fbae406b3f0 con 0x7fbae81059b0 2024-09-19T08:06:08.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:06.678+0000 7fbadeffd640 1 -- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fbad4039260 msgr2=0x7fbad403b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:06:08.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:06.678+0000 7fbadeffd640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fbad4039260 0x7fbad403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:08.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.679+0000 7fbadeffd640 1 -- 172.21.15.22:0/2488394860 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7fbae4045460 con 0x7fbae81059b0 2024-09-19T08:06:08.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.679+0000 7fbadeffd640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fbad403cd20 0x7fbad403f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:08.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.679+0000 7fbadeffd640 1 -- 172.21.15.22:0/2488394860 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbae406b1e0 con 0x7fbad403cd20 2024-09-19T08:06:08.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.679+0000 7fbaed1f3640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fbad403cd20 0x7fbad403f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:08.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.680+0000 7fbaed1f3640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fbad403cd20 0x7fbad403f110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbad8003980 tx=0x7fbad80092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:08.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.681+0000 7fbadeffd640 1 -- 172.21.15.22:0/2488394860 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fbae406b1e0 con 0x7fbad403cd20 2024-09-19T08:06:08.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.688+0000 7fbaefc58640 1 -- 172.21.15.22:0/2488394860 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fbae8105e30 con 0x7fbad403cd20 2024-09-19T08:06:08.377 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.688+0000 7fbadeffd640 1 -- 172.21.15.22:0/2488394860 <== mgr.14150 v2:172.21.15.22:6800/1866276413 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fbae8105e30 con 0x7fbad403cd20 2024-09-19T08:06:08.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.688+0000 7fbaefc58640 1 -- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fbad403cd20 msgr2=0x7fbad403f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:08.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.688+0000 7fbaefc58640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fbad403cd20 0x7fbad403f110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbad8003980 tx=0x7fbad80092b0 comp rx=0 tx=0).stop 2024-09-19T08:06:08.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.688+0000 7fbaefc58640 1 -- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae81059b0 msgr2=0x7fbae8108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:08.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.688+0000 7fbaefc58640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae81059b0 0x7fbae8108210 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbae4030560 tx=0x7fbae4032670 comp rx=0 tx=0).stop 2024-09-19T08:06:08.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.688+0000 7fbaefc58640 1 -- 172.21.15.22:0/2488394860 shutdown_connections 2024-09-19T08:06:08.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.688+0000 7fbaefc58640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fbad403cd20 0x7fbad403f110 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:08.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.689+0000 7fbaefc58640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:6800/4065414102,v1:172.21.15.22:6801/4065414102] conn(0x7fbad4039260 0x7fbad403b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:08.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.689+0000 7fbaefc58640 1 --2- 172.21.15.22:0/2488394860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae81059b0 0x7fbae8108210 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:08.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.689+0000 7fbaefc58640 1 -- 172.21.15.22:0/2488394860 >> 172.21.15.22:0/2488394860 conn(0x7fbae81014e0 msgr2=0x7fbae8102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:08.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.689+0000 7fbaefc58640 1 -- 172.21.15.22:0/2488394860 shutdown_connections 2024-09-19T08:06:08.378 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:07.689+0000 7fbaefc58640 1 -- 172.21.15.22:0/2488394860 wait complete. 2024-09-19T08:06:08.378 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 9 is available 2024-09-19T08:06:08.378 INFO:teuthology.orchestra.run.smithi022.stdout:Generating a dashboard self-signed certificate... 2024-09-19T08:06:09.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:08 smithi022 ceph-mon[30963]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T08:06:09.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:08 smithi022 ceph-mon[30963]: mgrmap e11: x(active, since 1.00595s) 2024-09-19T08:06:09.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:08 smithi022 ceph-mon[30963]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T08:06:09.766 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T08:06:09.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.796+0000 7f2ba5b88640 1 Processor -- start 2024-09-19T08:06:09.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.796+0000 7f2ba5b88640 1 -- start start 2024-09-19T08:06:09.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.797+0000 7f2ba5b88640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba0107740 0x7f2ba0107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:09.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.797+0000 7f2ba5b88640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba0108110 con 0x7f2ba0107740 2024-09-19T08:06:09.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.797+0000 7f2b9f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba0107740 0x7f2ba0107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:09.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.797+0000 7f2b9f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba0107740 0x7f2ba0107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57164/0 (socket says 172.21.15.22:57164) 2024-09-19T08:06:09.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.797+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/4155086084 learned_addr learned my addr 172.21.15.22:0/4155086084 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:09.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.797+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/4155086084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba01088f0 con 0x7f2ba0107740 2024-09-19T08:06:09.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.798+0000 7f2b9f7fe640 1 --2- 172.21.15.22:0/4155086084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba0107740 0x7f2ba0107b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009820 tx=0x7f2b9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ed206e2c276adb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:09.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.798+0000 7f2b9e7fc640 1 -- 172.21.15.22:0/4155086084 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b90037070 con 0x7f2ba0107740 2024-09-19T08:06:09.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.798+0000 7f2b9e7fc640 1 -- 172.21.15.22:0/4155086084 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b90032920 con 0x7f2ba0107740 2024-09-19T08:06:09.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.798+0000 7f2b9e7fc640 1 -- 172.21.15.22:0/4155086084 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b90030810 con 0x7f2ba0107740 2024-09-19T08:06:09.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.799+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4155086084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba0107740 msgr2=0x7f2ba0107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:09.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.799+0000 7f2ba5b88640 1 --2- 172.21.15.22:0/4155086084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba0107740 0x7f2ba0107b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009820 tx=0x7f2b9002a270 comp rx=0 tx=0).stop 2024-09-19T08:06:09.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.799+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4155086084 shutdown_connections 2024-09-19T08:06:09.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.799+0000 7f2ba5b88640 1 --2- 172.21.15.22:0/4155086084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba0107740 0x7f2ba0107b40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:09.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.799+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4155086084 >> 172.21.15.22:0/4155086084 conn(0x7f2ba0102ef0 msgr2=0x7f2ba0105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:09.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.799+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4155086084 shutdown_connections 2024-09-19T08:06:09.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.799+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4155086084 wait complete. 2024-09-19T08:06:09.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.799+0000 7f2ba5b88640 1 Processor -- start 2024-09-19T08:06:09.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.800+0000 7f2ba5b88640 1 -- start start 2024-09-19T08:06:09.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.800+0000 7f2ba5b88640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba0107740 0x7f2ba019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:09.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.800+0000 7f2ba5b88640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba010bca0 con 0x7f2ba0107740 2024-09-19T08:06:09.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.800+0000 7f2b9f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba0107740 0x7f2ba019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:09.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.800+0000 7f2b9f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba0107740 0x7f2ba019b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57166/0 (socket says 172.21.15.22:57166) 2024-09-19T08:06:09.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.800+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/4256533643 learned_addr learned my addr 172.21.15.22:0/4256533643 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:09.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.800+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/4256533643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba019bcc0 con 0x7f2ba0107740 2024-09-19T08:06:09.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.800+0000 7f2b9f7fe640 1 --2- 172.21.15.22:0/4256533643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba0107740 0x7f2ba019b780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b90030560 tx=0x7f2b90032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:09.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.801+0000 7f2b9cff9640 1 -- 172.21.15.22:0/4256533643 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b90037040 con 0x7f2ba0107740 2024-09-19T08:06:09.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.801+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4256533643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba019bf50 con 0x7f2ba0107740 2024-09-19T08:06:09.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.801+0000 7f2b9cff9640 1 -- 172.21.15.22:0/4256533643 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b90040070 con 0x7f2ba0107740 2024-09-19T08:06:09.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.801+0000 7f2b9cff9640 1 -- 172.21.15.22:0/4256533643 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b9003a6d0 con 0x7f2ba0107740 2024-09-19T08:06:09.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.801+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4256533643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba019ec40 con 0x7f2ba0107740 2024-09-19T08:06:09.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.802+0000 7f2b9cff9640 1 -- 172.21.15.22:0/4256533643 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f2b9003a870 con 0x7f2ba0107740 2024-09-19T08:06:09.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.802+0000 7f2b9cff9640 1 --2- 172.21.15.22:0/4256533643 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b88039260 0x7f2b8803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:09.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.802+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4256533643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba010be20 con 0x7f2ba0107740 2024-09-19T08:06:09.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.802+0000 7f2b9cff9640 1 -- 172.21.15.22:0/4256533643 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2b9006cb60 con 0x7f2ba0107740 2024-09-19T08:06:09.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.802+0000 7f2b9effd640 1 --2- 172.21.15.22:0/4256533643 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b88039260 0x7f2b8803b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:09.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.802+0000 7f2b9effd640 1 --2- 172.21.15.22:0/4256533643 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b88039260 0x7f2b8803b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2b940098c0 tx=0x7f2b94006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:09.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:08.811+0000 7f2b9cff9640 1 -- 172.21.15.22:0/4256533643 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2b90044c70 con 0x7f2ba0107740 2024-09-19T08:06:09.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:09.023+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4256533643 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f2ba01041b0 con 0x7f2b88039260 2024-09-19T08:06:09.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:09.095+0000 7f2b9cff9640 1 -- 172.21.15.22:0/4256533643 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f2ba01041b0 con 0x7f2b88039260 2024-09-19T08:06:09.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:09.098+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4256533643 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b88039260 msgr2=0x7f2b8803b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:09.775 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:09.098+0000 7f2ba5b88640 1 --2- 172.21.15.22:0/4256533643 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b88039260 0x7f2b8803b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2b940098c0 tx=0x7f2b94006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:09.775 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:09.098+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4256533643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba0107740 msgr2=0x7f2ba019b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:09.775 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:09.098+0000 7f2ba5b88640 1 --2- 172.21.15.22:0/4256533643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba0107740 0x7f2ba019b780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b90030560 tx=0x7f2b90032670 comp rx=0 tx=0).stop 2024-09-19T08:06:09.775 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:09.098+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4256533643 shutdown_connections 2024-09-19T08:06:09.775 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:09.098+0000 7f2ba5b88640 1 --2- 172.21.15.22:0/4256533643 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b88039260 0x7f2b8803b720 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:09.775 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:09.098+0000 7f2ba5b88640 1 --2- 172.21.15.22:0/4256533643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba0107740 0x7f2ba019b780 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:09.775 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:09.098+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4256533643 >> 172.21.15.22:0/4256533643 conn(0x7f2ba0102ef0 msgr2=0x7f2ba0103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:09.775 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:09.098+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4256533643 shutdown_connections 2024-09-19T08:06:09.775 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:09.098+0000 7f2ba5b88640 1 -- 172.21.15.22:0/4256533643 wait complete. 2024-09-19T08:06:09.775 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial admin user... 2024-09-19T08:06:09.978 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-mon[30963]: mgrmap e12: x(active, since 2s) 2024-09-19T08:06:09.978 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:09.978 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-mon[30963]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:06:09.978 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:09.978 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:10.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:09.977+0000 7efd3e885640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:08:06:09] ENGINE Error in HTTPServer.serve 2024-09-19T08:06:10.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: Traceback (most recent call last): 2024-09-19T08:06:10.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:06:10.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: self._connections.run(self.expiration_interval) 2024-09-19T08:06:10.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:06:10.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: self._run(expiration_interval) 2024-09-19T08:06:10.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:06:10.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:06:10.259 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:06:10.259 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:06:10.259 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:06:10.259 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: s = self.context.wrap_socket( 2024-09-19T08:06:10.259 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:06:10.259 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: return self.sslsocket_class._create( 2024-09-19T08:06:10.259 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:06:10.259 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: self.do_handshake() 2024-09-19T08:06:10.260 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:06:10.260 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: self._sslobj.do_handshake() 2024-09-19T08:06:10.260 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:06:10.260 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:06:09 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:06:11.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:10 smithi022 ceph-mon[30963]: [19/Sep/2024:08:06:09] ENGINE Bus STARTING 2024-09-19T08:06:11.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:11.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:10 smithi022 ceph-mon[30963]: [19/Sep/2024:08:06:09] ENGINE Serving on http://172.21.15.22:8765 2024-09-19T08:06:11.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:10 smithi022 ceph-mon[30963]: [19/Sep/2024:08:06:09] ENGINE Serving on https://172.21.15.22:7150 2024-09-19T08:06:11.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:10 smithi022 ceph-mon[30963]: [19/Sep/2024:08:06:09] ENGINE Bus STARTED 2024-09-19T08:06:11.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:10 smithi022 ceph-mon[30963]: [19/Sep/2024:08:06:09] ENGINE Error in HTTPServer.serve 2024-09-19T08:06:11.007 INFO:journalctl@ceph.mon.a.smithi022.stdout: Traceback (most recent call last): 2024-09-19T08:06:11.007 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:06:11.007 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._connections.run(self.expiration_interval) 2024-09-19T08:06:11.007 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:06:11.008 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._run(expiration_interval) 2024-09-19T08:06:11.008 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:06:11.008 INFO:journalctl@ceph.mon.a.smithi022.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:06:11.008 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:06:11.008 INFO:journalctl@ceph.mon.a.smithi022.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:06:11.008 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:06:11.008 INFO:journalctl@ceph.mon.a.smithi022.stdout: s = self.context.wrap_socket( 2024-09-19T08:06:11.008 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:06:11.008 INFO:journalctl@ceph.mon.a.smithi022.stdout: return self.sslsocket_class._create( 2024-09-19T08:06:11.008 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:06:11.008 INFO:journalctl@ceph.mon.a.smithi022.stdout: self.do_handshake() 2024-09-19T08:06:11.008 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:06:11.008 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._sslobj.do_handshake() 2024-09-19T08:06:11.008 INFO:journalctl@ceph.mon.a.smithi022.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:06:11.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:10 smithi022 ceph-mon[30963]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:06:11.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:11.458 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$8wI6qnV.GgfSQsFn8/T.u.38FJ3gAcnkeVgpHuFsFWZhuvR5KrzeO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726733170, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T08:06:11.458 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.194+0000 7fb8247c3640 1 Processor -- start 2024-09-19T08:06:11.458 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.194+0000 7fb8247c3640 1 -- start start 2024-09-19T08:06:11.458 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.194+0000 7fb8247c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb81c1081c0 0x7fb81c10a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:11.458 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.194+0000 7fb8247c3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c07be50 con 0x7fb81c1081c0 2024-09-19T08:06:11.458 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.195+0000 7fb82255f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb81c1081c0 0x7fb81c10a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:11.458 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.195+0000 7fb82255f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb81c1081c0 0x7fb81c10a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57174/0 (socket says 172.21.15.22:57174) 2024-09-19T08:06:11.458 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.195+0000 7fb82255f640 1 -- 172.21.15.22:0/3481232460 learned_addr learned my addr 172.21.15.22:0/3481232460 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:11.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.195+0000 7fb82255f640 1 -- 172.21.15.22:0/3481232460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c10aaf0 con 0x7fb81c1081c0 2024-09-19T08:06:11.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.196+0000 7fb82255f640 1 --2- 172.21.15.22:0/3481232460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb81c1081c0 0x7fb81c10a5b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb818009820 tx=0x7fb81802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677d43c783e66091 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:11.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.196+0000 7fb82155d640 1 -- 172.21.15.22:0/3481232460 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb818037070 con 0x7fb81c1081c0 2024-09-19T08:06:11.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.196+0000 7fb82155d640 1 -- 172.21.15.22:0/3481232460 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb818032920 con 0x7fb81c1081c0 2024-09-19T08:06:11.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.196+0000 7fb82155d640 1 -- 172.21.15.22:0/3481232460 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb818030810 con 0x7fb81c1081c0 2024-09-19T08:06:11.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.197+0000 7fb8247c3640 1 -- 172.21.15.22:0/3481232460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb81c1081c0 msgr2=0x7fb81c10a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:11.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.197+0000 7fb8247c3640 1 --2- 172.21.15.22:0/3481232460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb81c1081c0 0x7fb81c10a5b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb818009820 tx=0x7fb81802a270 comp rx=0 tx=0).stop 2024-09-19T08:06:11.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.197+0000 7fb8247c3640 1 -- 172.21.15.22:0/3481232460 shutdown_connections 2024-09-19T08:06:11.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.197+0000 7fb8247c3640 1 --2- 172.21.15.22:0/3481232460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb81c1081c0 0x7fb81c10a5b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:11.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.197+0000 7fb8247c3640 1 -- 172.21.15.22:0/3481232460 >> 172.21.15.22:0/3481232460 conn(0x7fb81c101e20 msgr2=0x7fb81c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:11.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.197+0000 7fb8247c3640 1 -- 172.21.15.22:0/3481232460 shutdown_connections 2024-09-19T08:06:11.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.197+0000 7fb8247c3640 1 -- 172.21.15.22:0/3481232460 wait complete. 2024-09-19T08:06:11.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.198+0000 7fb8247c3640 1 Processor -- start 2024-09-19T08:06:11.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.198+0000 7fb8247c3640 1 -- start start 2024-09-19T08:06:11.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.198+0000 7fb8247c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb81c1081c0 0x7fb81c19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:11.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.198+0000 7fb8247c3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c10bb70 con 0x7fb81c1081c0 2024-09-19T08:06:11.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.198+0000 7fb82255f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb81c1081c0 0x7fb81c19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:11.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.198+0000 7fb82255f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb81c1081c0 0x7fb81c19b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57178/0 (socket says 172.21.15.22:57178) 2024-09-19T08:06:11.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.198+0000 7fb82255f640 1 -- 172.21.15.22:0/2711089825 learned_addr learned my addr 172.21.15.22:0/2711089825 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:11.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.199+0000 7fb82255f640 1 -- 172.21.15.22:0/2711089825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c19bc50 con 0x7fb81c1081c0 2024-09-19T08:06:11.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.199+0000 7fb82255f640 1 --2- 172.21.15.22:0/2711089825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb81c1081c0 0x7fb81c19b710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb818030420 tx=0x7fb818032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:11.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.199+0000 7fb8137fe640 1 -- 172.21.15.22:0/2711089825 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb818037040 con 0x7fb81c1081c0 2024-09-19T08:06:11.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.199+0000 7fb8137fe640 1 -- 172.21.15.22:0/2711089825 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb818040070 con 0x7fb81c1081c0 2024-09-19T08:06:11.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.199+0000 7fb8137fe640 1 -- 172.21.15.22:0/2711089825 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb81803a6e0 con 0x7fb81c1081c0 2024-09-19T08:06:11.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.199+0000 7fb8247c3640 1 -- 172.21.15.22:0/2711089825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81c19bee0 con 0x7fb81c1081c0 2024-09-19T08:06:11.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.199+0000 7fb8247c3640 1 -- 172.21.15.22:0/2711089825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81c19ebd0 con 0x7fb81c1081c0 2024-09-19T08:06:11.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.200+0000 7fb8137fe640 1 -- 172.21.15.22:0/2711089825 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fb81803a990 con 0x7fb81c1081c0 2024-09-19T08:06:11.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.200+0000 7fb8137fe640 1 --2- 172.21.15.22:0/2711089825 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb808039260 0x7fb80803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:11.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.200+0000 7fb8137fe640 1 -- 172.21.15.22:0/2711089825 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb81806c9b0 con 0x7fb81c1081c0 2024-09-19T08:06:11.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.200+0000 7fb8247c3640 1 -- 172.21.15.22:0/2711089825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb81c108620 con 0x7fb81c1081c0 2024-09-19T08:06:11.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.201+0000 7fb821d5e640 1 --2- 172.21.15.22:0/2711089825 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb808039260 0x7fb80803b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:11.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.201+0000 7fb821d5e640 1 --2- 172.21.15.22:0/2711089825 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb808039260 0x7fb80803b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0098c0 tx=0x7fb80c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:11.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.210+0000 7fb8137fe640 1 -- 172.21.15.22:0/2711089825 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb818041270 con 0x7fb81c1081c0 2024-09-19T08:06:11.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.419+0000 7fb8247c3640 1 -- 172.21.15.22:0/2711089825 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fb81c103130 con 0x7fb808039260 2024-09-19T08:06:11.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.668+0000 7fb8137fe640 1 -- 172.21.15.22:0/2711089825 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fb81c103130 con 0x7fb808039260 2024-09-19T08:06:11.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.671+0000 7fb8247c3640 1 -- 172.21.15.22:0/2711089825 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb808039260 msgr2=0x7fb80803b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:11.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.671+0000 7fb8247c3640 1 --2- 172.21.15.22:0/2711089825 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb808039260 0x7fb80803b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0098c0 tx=0x7fb80c006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:11.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.671+0000 7fb8247c3640 1 -- 172.21.15.22:0/2711089825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb81c1081c0 msgr2=0x7fb81c19b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:11.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.671+0000 7fb8247c3640 1 --2- 172.21.15.22:0/2711089825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb81c1081c0 0x7fb81c19b710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb818030420 tx=0x7fb818032670 comp rx=0 tx=0).stop 2024-09-19T08:06:11.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.671+0000 7fb8247c3640 1 -- 172.21.15.22:0/2711089825 shutdown_connections 2024-09-19T08:06:11.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.671+0000 7fb8247c3640 1 --2- 172.21.15.22:0/2711089825 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb808039260 0x7fb80803b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:11.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.671+0000 7fb8247c3640 1 --2- 172.21.15.22:0/2711089825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb81c1081c0 0x7fb81c19b710 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:11.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.671+0000 7fb8247c3640 1 -- 172.21.15.22:0/2711089825 >> 172.21.15.22:0/2711089825 conn(0x7fb81c101e20 msgr2=0x7fb81c102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:11.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.671+0000 7fb8247c3640 1 -- 172.21.15.22:0/2711089825 shutdown_connections 2024-09-19T08:06:11.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:10.671+0000 7fb8247c3640 1 -- 172.21.15.22:0/2711089825 wait complete. 2024-09-19T08:06:11.463 INFO:teuthology.orchestra.run.smithi022.stdout:Fetching dashboard port number... 2024-09-19T08:06:12.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:12 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2286167017' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T08:06:12.969 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T08:06:12.969 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.867+0000 7fcac216b640 1 Processor -- start 2024-09-19T08:06:12.969 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.867+0000 7fcac216b640 1 -- start start 2024-09-19T08:06:12.969 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.867+0000 7fcac216b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcabc108200 0x7fcabc10a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:12.970 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.867+0000 7fcac216b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc07be10 con 0x7fcabc108200 2024-09-19T08:06:12.970 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.867+0000 7fcabb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcabc108200 0x7fcabc10a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:12.970 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.867+0000 7fcabb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcabc108200 0x7fcabc10a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57180/0 (socket says 172.21.15.22:57180) 2024-09-19T08:06:12.970 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.867+0000 7fcabb7fe640 1 -- 172.21.15.22:0/3727582265 learned_addr learned my addr 172.21.15.22:0/3727582265 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:12.970 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.868+0000 7fcabb7fe640 1 -- 172.21.15.22:0/3727582265 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc10ab30 con 0x7fcabc108200 2024-09-19T08:06:12.970 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.869+0000 7fcabb7fe640 1 --2- 172.21.15.22:0/3727582265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcabc108200 0x7fcabc10a5f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009820 tx=0x7fcaac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc978f6492690cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:12.970 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.869+0000 7fcaba7fc640 1 -- 172.21.15.22:0/3727582265 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaac037070 con 0x7fcabc108200 2024-09-19T08:06:12.970 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.869+0000 7fcaba7fc640 1 -- 172.21.15.22:0/3727582265 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaac032920 con 0x7fcabc108200 2024-09-19T08:06:12.970 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.869+0000 7fcaba7fc640 1 -- 172.21.15.22:0/3727582265 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaac030810 con 0x7fcabc108200 2024-09-19T08:06:12.970 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.870+0000 7fcac216b640 1 -- 172.21.15.22:0/3727582265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcabc108200 msgr2=0x7fcabc10a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:12.970 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.870+0000 7fcac216b640 1 --2- 172.21.15.22:0/3727582265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcabc108200 0x7fcabc10a5f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009820 tx=0x7fcaac02a270 comp rx=0 tx=0).stop 2024-09-19T08:06:12.970 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.870+0000 7fcac216b640 1 -- 172.21.15.22:0/3727582265 shutdown_connections 2024-09-19T08:06:12.970 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.870+0000 7fcac216b640 1 --2- 172.21.15.22:0/3727582265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcabc108200 0x7fcabc10a5f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:12.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.870+0000 7fcac216b640 1 -- 172.21.15.22:0/3727582265 >> 172.21.15.22:0/3727582265 conn(0x7fcabc101e20 msgr2=0x7fcabc104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:12.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.870+0000 7fcac216b640 1 -- 172.21.15.22:0/3727582265 shutdown_connections 2024-09-19T08:06:12.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.870+0000 7fcac216b640 1 -- 172.21.15.22:0/3727582265 wait complete. 2024-09-19T08:06:12.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.870+0000 7fcac216b640 1 Processor -- start 2024-09-19T08:06:12.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.870+0000 7fcac216b640 1 -- start start 2024-09-19T08:06:12.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.871+0000 7fcac216b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcabc108200 0x7fcabc107a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:12.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.871+0000 7fcac216b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc10bb50 con 0x7fcabc108200 2024-09-19T08:06:12.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.871+0000 7fcabb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcabc108200 0x7fcabc107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:12.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.871+0000 7fcabb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcabc108200 0x7fcabc107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57186/0 (socket says 172.21.15.22:57186) 2024-09-19T08:06:12.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.871+0000 7fcabb7fe640 1 -- 172.21.15.22:0/2286167017 learned_addr learned my addr 172.21.15.22:0/2286167017 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:12.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.871+0000 7fcabb7fe640 1 -- 172.21.15.22:0/2286167017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc107fc0 con 0x7fcabc108200 2024-09-19T08:06:12.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.871+0000 7fcabb7fe640 1 --2- 172.21.15.22:0/2286167017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcabc108200 0x7fcabc107a80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcaac030420 tx=0x7fcaac032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:12.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.871+0000 7fcab8ff9640 1 -- 172.21.15.22:0/2286167017 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaac037040 con 0x7fcabc108200 2024-09-19T08:06:12.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.871+0000 7fcab8ff9640 1 -- 172.21.15.22:0/2286167017 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaac040070 con 0x7fcabc108200 2024-09-19T08:06:12.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.871+0000 7fcac216b640 1 -- 172.21.15.22:0/2286167017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc1061b0 con 0x7fcabc108200 2024-09-19T08:06:12.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.871+0000 7fcab8ff9640 1 -- 172.21.15.22:0/2286167017 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaac03a6e0 con 0x7fcabc108200 2024-09-19T08:06:12.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.872+0000 7fcac216b640 1 -- 172.21.15.22:0/2286167017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc1066f0 con 0x7fcabc108200 2024-09-19T08:06:12.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.872+0000 7fcab8ff9640 1 -- 172.21.15.22:0/2286167017 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fcaac03a9e0 con 0x7fcabc108200 2024-09-19T08:06:12.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.873+0000 7fcab8ff9640 1 --2- 172.21.15.22:0/2286167017 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcaa4039200 0x7fcaa403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:12.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.873+0000 7fcab8ff9640 1 -- 172.21.15.22:0/2286167017 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcaac06ca00 con 0x7fcabc108200 2024-09-19T08:06:12.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.873+0000 7fcabaffd640 1 --2- 172.21.15.22:0/2286167017 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcaa4039200 0x7fcaa403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:12.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.873+0000 7fcac216b640 1 -- 172.21.15.22:0/2286167017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcabc108660 con 0x7fcabc108200 2024-09-19T08:06:12.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.873+0000 7fcabaffd640 1 --2- 172.21.15.22:0/2286167017 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcaa4039200 0x7fcaa403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcab00098c0 tx=0x7fcab0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:12.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:11.878+0000 7fcab8ff9640 1 -- 172.21.15.22:0/2286167017 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcaac042e10 con 0x7fcabc108200 2024-09-19T08:06:12.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:12.075+0000 7fcac216b640 1 -- 172.21.15.22:0/2286167017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fcabc106ca0 con 0x7fcabc108200 2024-09-19T08:06:12.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:12.076+0000 7fcab8ff9640 1 -- 172.21.15.22:0/2286167017 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fcaac043b80 con 0x7fcabc108200 2024-09-19T08:06:12.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:12.078+0000 7fcac216b640 1 -- 172.21.15.22:0/2286167017 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcaa4039200 msgr2=0x7fcaa403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:12.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:12.078+0000 7fcac216b640 1 --2- 172.21.15.22:0/2286167017 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcaa4039200 0x7fcaa403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcab00098c0 tx=0x7fcab0006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:12.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:12.078+0000 7fcac216b640 1 -- 172.21.15.22:0/2286167017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcabc108200 msgr2=0x7fcabc107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:12.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:12.078+0000 7fcac216b640 1 --2- 172.21.15.22:0/2286167017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcabc108200 0x7fcabc107a80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcaac030420 tx=0x7fcaac032670 comp rx=0 tx=0).stop 2024-09-19T08:06:12.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:12.078+0000 7fcac216b640 1 -- 172.21.15.22:0/2286167017 shutdown_connections 2024-09-19T08:06:12.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:12.078+0000 7fcac216b640 1 --2- 172.21.15.22:0/2286167017 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcaa4039200 0x7fcaa403b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:12.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:12.078+0000 7fcac216b640 1 --2- 172.21.15.22:0/2286167017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcabc108200 0x7fcabc107a80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:12.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:12.078+0000 7fcac216b640 1 -- 172.21.15.22:0/2286167017 >> 172.21.15.22:0/2286167017 conn(0x7fcabc101e20 msgr2=0x7fcabc102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:12.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:12.078+0000 7fcac216b640 1 -- 172.21.15.22:0/2286167017 shutdown_connections 2024-09-19T08:06:12.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:12.078+0000 7fcac216b640 1 -- 172.21.15.22:0/2286167017 wait complete. 2024-09-19T08:06:12.976 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T08:06:12.976 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-09-19T08:06:12.984 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T08:06:12.984 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-09-19T08:06:12.985 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-09-19T08:06:12.985 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T08:06:12.987 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph Dashboard is now available at: 2024-09-19T08:06:12.987 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:06:12.987 INFO:teuthology.orchestra.run.smithi022.stdout: URL: https://smithi022.front.sepia.ceph.com:8443/ 2024-09-19T08:06:12.987 INFO:teuthology.orchestra.run.smithi022.stdout: User: admin 2024-09-19T08:06:12.987 INFO:teuthology.orchestra.run.smithi022.stdout: Password: kap93c8u5d 2024-09-19T08:06:12.987 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:06:12.987 INFO:teuthology.orchestra.run.smithi022.stdout:Saving cluster configuration to /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config directory 2024-09-19T08:06:14.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.405+0000 7f27e2b44640 1 Processor -- start 2024-09-19T08:06:14.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.405+0000 7f27e2b44640 1 -- start start 2024-09-19T08:06:14.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.405+0000 7f27e2b44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc107c10 0x7f27dc108010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:14.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.405+0000 7f27e2b44640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc1085e0 con 0x7f27dc107c10 2024-09-19T08:06:14.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.406+0000 7f27e08e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc107c10 0x7f27dc108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:14.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.406+0000 7f27e08e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc107c10 0x7f27dc108010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57200/0 (socket says 172.21.15.22:57200) 2024-09-19T08:06:14.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.406+0000 7f27e08e0640 1 -- 172.21.15.22:0/1952647939 learned_addr learned my addr 172.21.15.22:0/1952647939 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:14.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.406+0000 7f27e08e0640 1 -- 172.21.15.22:0/1952647939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc1068c0 con 0x7f27dc107c10 2024-09-19T08:06:14.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.407+0000 7f27e08e0640 1 --2- 172.21.15.22:0/1952647939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc107c10 0x7f27dc108010 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009820 tx=0x7f27d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4660cfa7e243613 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:14.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.407+0000 7f27db7fe640 1 -- 172.21.15.22:0/1952647939 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d0037070 con 0x7f27dc107c10 2024-09-19T08:06:14.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.407+0000 7f27db7fe640 1 -- 172.21.15.22:0/1952647939 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d0032920 con 0x7f27dc107c10 2024-09-19T08:06:14.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.407+0000 7f27db7fe640 1 -- 172.21.15.22:0/1952647939 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d0030810 con 0x7f27dc107c10 2024-09-19T08:06:14.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.407+0000 7f27e2b44640 1 -- 172.21.15.22:0/1952647939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc107c10 msgr2=0x7f27dc108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:14.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.407+0000 7f27e2b44640 1 --2- 172.21.15.22:0/1952647939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc107c10 0x7f27dc108010 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009820 tx=0x7f27d002a270 comp rx=0 tx=0).stop 2024-09-19T08:06:14.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.407+0000 7f27e2b44640 1 -- 172.21.15.22:0/1952647939 shutdown_connections 2024-09-19T08:06:14.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.407+0000 7f27e2b44640 1 --2- 172.21.15.22:0/1952647939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc107c10 0x7f27dc108010 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:14.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.407+0000 7f27e2b44640 1 -- 172.21.15.22:0/1952647939 >> 172.21.15.22:0/1952647939 conn(0x7f27dc101560 msgr2=0x7f27dc103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:14.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.408+0000 7f27e2b44640 1 -- 172.21.15.22:0/1952647939 shutdown_connections 2024-09-19T08:06:14.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.408+0000 7f27e2b44640 1 -- 172.21.15.22:0/1952647939 wait complete. 2024-09-19T08:06:14.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.408+0000 7f27e2b44640 1 Processor -- start 2024-09-19T08:06:14.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.408+0000 7f27e2b44640 1 -- start start 2024-09-19T08:06:14.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.408+0000 7f27e2b44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc107c10 0x7f27dc19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:14.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.408+0000 7f27e2b44640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc10ab00 con 0x7f27dc107c10 2024-09-19T08:06:14.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.408+0000 7f27e08e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc107c10 0x7f27dc19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:14.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.409+0000 7f27e08e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc107c10 0x7f27dc19b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57206/0 (socket says 172.21.15.22:57206) 2024-09-19T08:06:14.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.409+0000 7f27e08e0640 1 -- 172.21.15.22:0/3227857066 learned_addr learned my addr 172.21.15.22:0/3227857066 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:14.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.409+0000 7f27e08e0640 1 -- 172.21.15.22:0/3227857066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc19bd00 con 0x7f27dc107c10 2024-09-19T08:06:14.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.409+0000 7f27e08e0640 1 --2- 172.21.15.22:0/3227857066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc107c10 0x7f27dc19b7c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f27d0030560 tx=0x7f27d0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:14.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.409+0000 7f27d9ffb640 1 -- 172.21.15.22:0/3227857066 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d0037040 con 0x7f27dc107c10 2024-09-19T08:06:14.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.409+0000 7f27d9ffb640 1 -- 172.21.15.22:0/3227857066 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d0040070 con 0x7f27dc107c10 2024-09-19T08:06:14.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.409+0000 7f27d9ffb640 1 -- 172.21.15.22:0/3227857066 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d003a6d0 con 0x7f27dc107c10 2024-09-19T08:06:14.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.409+0000 7f27e2b44640 1 -- 172.21.15.22:0/3227857066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27dc19bf90 con 0x7f27dc107c10 2024-09-19T08:06:14.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.409+0000 7f27e2b44640 1 -- 172.21.15.22:0/3227857066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27dc19ec80 con 0x7f27dc107c10 2024-09-19T08:06:14.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.410+0000 7f27d9ffb640 1 -- 172.21.15.22:0/3227857066 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f27d003bb20 con 0x7f27dc107c10 2024-09-19T08:06:14.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.410+0000 7f27e2b44640 1 -- 172.21.15.22:0/3227857066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27dc108090 con 0x7f27dc107c10 2024-09-19T08:06:14.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.411+0000 7f27d9ffb640 1 --2- 172.21.15.22:0/3227857066 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f27cc0392a0 0x7f27cc03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:14.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.411+0000 7f27d9ffb640 1 -- 172.21.15.22:0/3227857066 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f27d006ca00 con 0x7f27dc107c10 2024-09-19T08:06:14.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.411+0000 7f27dbfff640 1 --2- 172.21.15.22:0/3227857066 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f27cc0392a0 0x7f27cc03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:14.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.411+0000 7f27dbfff640 1 --2- 172.21.15.22:0/3227857066 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f27cc0392a0 0x7f27cc03b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f27c80098c0 tx=0x7f27c8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:14.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.415+0000 7f27d9ffb640 1 -- 172.21.15.22:0/3227857066 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f27d00328f0 con 0x7f27dc107c10 2024-09-19T08:06:14.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.682+0000 7f27e2b44640 1 -- 172.21.15.22:0/3227857066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f27dc02d9b0 con 0x7f27dc107c10 2024-09-19T08:06:14.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.685+0000 7f27d9ffb640 1 -- 172.21.15.22:0/3227857066 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f27d004a200 con 0x7f27dc107c10 2024-09-19T08:06:14.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T08:06:14.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.687+0000 7f27e2b44640 1 -- 172.21.15.22:0/3227857066 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f27cc0392a0 msgr2=0x7f27cc03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:14.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.687+0000 7f27e2b44640 1 --2- 172.21.15.22:0/3227857066 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f27cc0392a0 0x7f27cc03b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f27c80098c0 tx=0x7f27c8006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:14.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.687+0000 7f27e2b44640 1 -- 172.21.15.22:0/3227857066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc107c10 msgr2=0x7f27dc19b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:14.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.687+0000 7f27e2b44640 1 --2- 172.21.15.22:0/3227857066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc107c10 0x7f27dc19b7c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f27d0030560 tx=0x7f27d0032670 comp rx=0 tx=0).stop 2024-09-19T08:06:14.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.687+0000 7f27e2b44640 1 -- 172.21.15.22:0/3227857066 shutdown_connections 2024-09-19T08:06:14.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.687+0000 7f27e2b44640 1 --2- 172.21.15.22:0/3227857066 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f27cc0392a0 0x7f27cc03b760 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:14.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.687+0000 7f27e2b44640 1 --2- 172.21.15.22:0/3227857066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27dc107c10 0x7f27dc19b7c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:14.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.687+0000 7f27e2b44640 1 -- 172.21.15.22:0/3227857066 >> 172.21.15.22:0/3227857066 conn(0x7f27dc101560 msgr2=0x7f27dc1024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:14.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.687+0000 7f27e2b44640 1 -- 172.21.15.22:0/3227857066 shutdown_connections 2024-09-19T08:06:14.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-09-19T08:06:13.688+0000 7f27e2b44640 1 -- 172.21.15.22:0/3227857066 wait complete. 2024-09-19T08:06:14.236 INFO:teuthology.orchestra.run.smithi022.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T08:06:14.236 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:06:14.236 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:06:14.236 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:06:14.236 INFO:teuthology.orchestra.run.smithi022.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T08:06:14.236 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:06:14.236 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T08:06:14.236 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:06:14.236 INFO:teuthology.orchestra.run.smithi022.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T08:06:14.236 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:06:14.236 INFO:teuthology.orchestra.run.smithi022.stdout: ceph telemetry on 2024-09-19T08:06:14.237 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:06:14.237 INFO:teuthology.orchestra.run.smithi022.stdout:For more information see: 2024-09-19T08:06:14.237 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:06:14.237 INFO:teuthology.orchestra.run.smithi022.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T08:06:14.237 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:06:14.237 INFO:teuthology.orchestra.run.smithi022.stdout:Bootstrap complete. 2024-09-19T08:06:14.269 INFO:tasks.cephadm:Fetching config... 2024-09-19T08:06:14.270 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:06:14.270 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T08:06:14.286 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T08:06:14.286 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:06:14.286 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T08:06:14.342 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T08:06:14.342 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:06:14.342 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T08:06:14.404 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:14 smithi022 ceph-mon[30963]: mgrmap e13: x(active, since 6s) 2024-09-19T08:06:14.404 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:14 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3227857066' entity='client.admin' 2024-09-19T08:06:14.408 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T08:06:14.408 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:06:14.408 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T08:06:14.464 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T08:06:14.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQx3In6CZL2XP/phaLxNEQaroG0CENFfGvetSTatbsT4ULm2suNDOdmV6gKMxLA/R/odNpqVi8sDtwiszluqCQQWwL0c6UV/kQcPOX76kzY6czILcCwRd7MrrqTXR5jgUMR+wwe/lFN/84WI0hYNAYl4fwBwk4xcJ6nptBrt08a0clMi4v1t3gnvMWUHrxDTSNMIBofIn50XI9jKRrY9H2gC1UfhK7x2SKtGttSU0j4bTm17TLhDx5O6mJiFo9vVmMngMoJQIUMsxDkRyTYeIqVuMi8GLrWM/AGU8N9ZrZBrDtRPzqpfUYMXrK1dWcTx2wiUJBN++aiyYCC6uTI/E0cEqErqrd4JMMA/1zZ7S19095TRCq9tyEzP+FWpCxgRInJOA1juFg1bYila1zI7pri1kkeUakCge7LjLQao4EEErxoMwakUCnbd5xuAVQ1xgvSvrVxvgx9cgOpLC9qdBW64EKBjkWi+vQfyyQ9Ah6jRpZpvycEZmnqKAnnmsMTRk= ceph-c0ad60c6-765d-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T08:06:14.540 INFO:teuthology.orchestra.run.smithi022.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQx3In6CZL2XP/phaLxNEQaroG0CENFfGvetSTatbsT4ULm2suNDOdmV6gKMxLA/R/odNpqVi8sDtwiszluqCQQWwL0c6UV/kQcPOX76kzY6czILcCwRd7MrrqTXR5jgUMR+wwe/lFN/84WI0hYNAYl4fwBwk4xcJ6nptBrt08a0clMi4v1t3gnvMWUHrxDTSNMIBofIn50XI9jKRrY9H2gC1UfhK7x2SKtGttSU0j4bTm17TLhDx5O6mJiFo9vVmMngMoJQIUMsxDkRyTYeIqVuMi8GLrWM/AGU8N9ZrZBrDtRPzqpfUYMXrK1dWcTx2wiUJBN++aiyYCC6uTI/E0cEqErqrd4JMMA/1zZ7S19095TRCq9tyEzP+FWpCxgRInJOA1juFg1bYila1zI7pri1kkeUakCge7LjLQao4EEErxoMwakUCnbd5xuAVQ1xgvSvrVxvgx9cgOpLC9qdBW64EKBjkWi+vQfyyQ9Ah6jRpZpvycEZmnqKAnnmsMTRk= ceph-c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:14.551 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQx3In6CZL2XP/phaLxNEQaroG0CENFfGvetSTatbsT4ULm2suNDOdmV6gKMxLA/R/odNpqVi8sDtwiszluqCQQWwL0c6UV/kQcPOX76kzY6czILcCwRd7MrrqTXR5jgUMR+wwe/lFN/84WI0hYNAYl4fwBwk4xcJ6nptBrt08a0clMi4v1t3gnvMWUHrxDTSNMIBofIn50XI9jKRrY9H2gC1UfhK7x2SKtGttSU0j4bTm17TLhDx5O6mJiFo9vVmMngMoJQIUMsxDkRyTYeIqVuMi8GLrWM/AGU8N9ZrZBrDtRPzqpfUYMXrK1dWcTx2wiUJBN++aiyYCC6uTI/E0cEqErqrd4JMMA/1zZ7S19095TRCq9tyEzP+FWpCxgRInJOA1juFg1bYila1zI7pri1kkeUakCge7LjLQao4EEErxoMwakUCnbd5xuAVQ1xgvSvrVxvgx9cgOpLC9qdBW64EKBjkWi+vQfyyQ9Ah6jRpZpvycEZmnqKAnnmsMTRk= ceph-c0ad60c6-765d-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T08:06:14.592 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQx3In6CZL2XP/phaLxNEQaroG0CENFfGvetSTatbsT4ULm2suNDOdmV6gKMxLA/R/odNpqVi8sDtwiszluqCQQWwL0c6UV/kQcPOX76kzY6czILcCwRd7MrrqTXR5jgUMR+wwe/lFN/84WI0hYNAYl4fwBwk4xcJ6nptBrt08a0clMi4v1t3gnvMWUHrxDTSNMIBofIn50XI9jKRrY9H2gC1UfhK7x2SKtGttSU0j4bTm17TLhDx5O6mJiFo9vVmMngMoJQIUMsxDkRyTYeIqVuMi8GLrWM/AGU8N9ZrZBrDtRPzqpfUYMXrK1dWcTx2wiUJBN++aiyYCC6uTI/E0cEqErqrd4JMMA/1zZ7S19095TRCq9tyEzP+FWpCxgRInJOA1juFg1bYila1zI7pri1kkeUakCge7LjLQao4EEErxoMwakUCnbd5xuAVQ1xgvSvrVxvgx9cgOpLC9qdBW64EKBjkWi+vQfyyQ9Ah6jRpZpvycEZmnqKAnnmsMTRk= ceph-c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:14.605 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQx3In6CZL2XP/phaLxNEQaroG0CENFfGvetSTatbsT4ULm2suNDOdmV6gKMxLA/R/odNpqVi8sDtwiszluqCQQWwL0c6UV/kQcPOX76kzY6czILcCwRd7MrrqTXR5jgUMR+wwe/lFN/84WI0hYNAYl4fwBwk4xcJ6nptBrt08a0clMi4v1t3gnvMWUHrxDTSNMIBofIn50XI9jKRrY9H2gC1UfhK7x2SKtGttSU0j4bTm17TLhDx5O6mJiFo9vVmMngMoJQIUMsxDkRyTYeIqVuMi8GLrWM/AGU8N9ZrZBrDtRPzqpfUYMXrK1dWcTx2wiUJBN++aiyYCC6uTI/E0cEqErqrd4JMMA/1zZ7S19095TRCq9tyEzP+FWpCxgRInJOA1juFg1bYila1zI7pri1kkeUakCge7LjLQao4EEErxoMwakUCnbd5xuAVQ1xgvSvrVxvgx9cgOpLC9qdBW64EKBjkWi+vQfyyQ9Ah6jRpZpvycEZmnqKAnnmsMTRk= ceph-c0ad60c6-765d-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T08:06:14.646 INFO:teuthology.orchestra.run.smithi157.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQx3In6CZL2XP/phaLxNEQaroG0CENFfGvetSTatbsT4ULm2suNDOdmV6gKMxLA/R/odNpqVi8sDtwiszluqCQQWwL0c6UV/kQcPOX76kzY6czILcCwRd7MrrqTXR5jgUMR+wwe/lFN/84WI0hYNAYl4fwBwk4xcJ6nptBrt08a0clMi4v1t3gnvMWUHrxDTSNMIBofIn50XI9jKRrY9H2gC1UfhK7x2SKtGttSU0j4bTm17TLhDx5O6mJiFo9vVmMngMoJQIUMsxDkRyTYeIqVuMi8GLrWM/AGU8N9ZrZBrDtRPzqpfUYMXrK1dWcTx2wiUJBN++aiyYCC6uTI/E0cEqErqrd4JMMA/1zZ7S19095TRCq9tyEzP+FWpCxgRInJOA1juFg1bYila1zI7pri1kkeUakCge7LjLQao4EEErxoMwakUCnbd5xuAVQ1xgvSvrVxvgx9cgOpLC9qdBW64EKBjkWi+vQfyyQ9Ah6jRpZpvycEZmnqKAnnmsMTRk= ceph-c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:14.660 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T08:06:14.913 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:06:15.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.390+0000 7f2487a42640 1 -- 172.21.15.22:0/2423964422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2480102070 msgr2=0x7f2480104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:15.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.390+0000 7f2487a42640 1 --2- 172.21.15.22:0/2423964422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2480102070 0x7f2480104460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f247c009880 tx=0x7f247c02a5d0 comp rx=0 tx=0).stop 2024-09-19T08:06:15.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.390+0000 7f2487a42640 1 -- 172.21.15.22:0/2423964422 shutdown_connections 2024-09-19T08:06:15.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.390+0000 7f2487a42640 1 --2- 172.21.15.22:0/2423964422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2480102070 0x7f2480104460 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:15.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.390+0000 7f2487a42640 1 -- 172.21.15.22:0/2423964422 >> 172.21.15.22:0/2423964422 conn(0x7f24800fbc90 msgr2=0x7f24800fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:15.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.390+0000 7f2487a42640 1 -- 172.21.15.22:0/2423964422 shutdown_connections 2024-09-19T08:06:15.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.390+0000 7f2487a42640 1 -- 172.21.15.22:0/2423964422 wait complete. 2024-09-19T08:06:15.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.391+0000 7f2487a42640 1 Processor -- start 2024-09-19T08:06:15.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.391+0000 7f2487a42640 1 -- start start 2024-09-19T08:06:15.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.392+0000 7f2487a42640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2480102070 0x7f2480196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:15.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.392+0000 7f2487a42640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2480107680 con 0x7f2480102070 2024-09-19T08:06:15.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.392+0000 7f24857de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2480102070 0x7f2480196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:15.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.392+0000 7f24857de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2480102070 0x7f2480196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57226/0 (socket says 172.21.15.22:57226) 2024-09-19T08:06:15.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.392+0000 7f24857de640 1 -- 172.21.15.22:0/3109556683 learned_addr learned my addr 172.21.15.22:0/3109556683 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:15.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.392+0000 7f24857de640 1 -- 172.21.15.22:0/3109556683 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2480197210 con 0x7f2480102070 2024-09-19T08:06:15.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.393+0000 7f24857de640 1 --2- 172.21.15.22:0/3109556683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2480102070 0x7f2480196cd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f247c004f60 tx=0x7f247c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:15.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.393+0000 7f24767fc640 1 -- 172.21.15.22:0/3109556683 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f247c03f070 con 0x7f2480102070 2024-09-19T08:06:15.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.393+0000 7f24767fc640 1 -- 172.21.15.22:0/3109556683 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f247c003ad0 con 0x7f2480102070 2024-09-19T08:06:15.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.393+0000 7f24767fc640 1 -- 172.21.15.22:0/3109556683 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f247c035030 con 0x7f2480102070 2024-09-19T08:06:15.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.393+0000 7f2487a42640 1 -- 172.21.15.22:0/3109556683 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24801974a0 con 0x7f2480102070 2024-09-19T08:06:15.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.393+0000 7f2487a42640 1 -- 172.21.15.22:0/3109556683 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24800ffe60 con 0x7f2480102070 2024-09-19T08:06:15.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.394+0000 7f24767fc640 1 -- 172.21.15.22:0/3109556683 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f247c042070 con 0x7f2480102070 2024-09-19T08:06:15.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.395+0000 7f2487a42640 1 -- 172.21.15.22:0/3109556683 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2480107e00 con 0x7f2480102070 2024-09-19T08:06:15.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.395+0000 7f24767fc640 1 --2- 172.21.15.22:0/3109556683 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f246c03d6b0 0x7f246c03fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:15.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.395+0000 7f24767fc640 1 -- 172.21.15.22:0/3109556683 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f247c06d160 con 0x7f2480102070 2024-09-19T08:06:15.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.395+0000 7f2484fdd640 1 --2- 172.21.15.22:0/3109556683 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f246c03d6b0 0x7f246c03fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:15.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.395+0000 7f2484fdd640 1 --2- 172.21.15.22:0/3109556683 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f246c03d6b0 0x7f246c03fb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f24700098c0 tx=0x7f2470006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:15.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.400+0000 7f24767fc640 1 -- 172.21.15.22:0/3109556683 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f247c03a3c0 con 0x7f2480102070 2024-09-19T08:06:15.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.578+0000 7f2487a42640 1 -- 172.21.15.22:0/3109556683 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f248002d7c0 con 0x7f2480102070 2024-09-19T08:06:15.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.580+0000 7f24767fc640 1 -- 172.21.15.22:0/3109556683 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f247c0412c0 con 0x7f2480102070 2024-09-19T08:06:15.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.583+0000 7f2487a42640 1 -- 172.21.15.22:0/3109556683 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f246c03d6b0 msgr2=0x7f246c03fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:15.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.583+0000 7f2487a42640 1 --2- 172.21.15.22:0/3109556683 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f246c03d6b0 0x7f246c03fb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f24700098c0 tx=0x7f2470006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:15.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.583+0000 7f2487a42640 1 -- 172.21.15.22:0/3109556683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2480102070 msgr2=0x7f2480196cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:15.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.583+0000 7f2487a42640 1 --2- 172.21.15.22:0/3109556683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2480102070 0x7f2480196cd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f247c004f60 tx=0x7f247c0048f0 comp rx=0 tx=0).stop 2024-09-19T08:06:15.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.583+0000 7f2487a42640 1 -- 172.21.15.22:0/3109556683 shutdown_connections 2024-09-19T08:06:15.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.583+0000 7f2487a42640 1 --2- 172.21.15.22:0/3109556683 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f246c03d6b0 0x7f246c03fb70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:15.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.583+0000 7f2487a42640 1 --2- 172.21.15.22:0/3109556683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2480102070 0x7f2480196cd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:15.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.583+0000 7f2487a42640 1 -- 172.21.15.22:0/3109556683 >> 172.21.15.22:0/3109556683 conn(0x7f24800fbc90 msgr2=0x7f24800fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:15.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.583+0000 7f2487a42640 1 -- 172.21.15.22:0/3109556683 shutdown_connections 2024-09-19T08:06:15.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:15.583+0000 7f2487a42640 1 -- 172.21.15.22:0/3109556683 wait complete. 2024-09-19T08:06:16.215 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T08:06:16.215 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T08:06:16.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:16 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3109556683' entity='client.admin' 2024-09-19T08:06:16.885 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:06:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.501+0000 7f2faf1a7640 1 -- 172.21.15.22:0/1405484288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fa80fff20 msgr2=0x7f2fa8100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.501+0000 7f2faf1a7640 1 --2- 172.21.15.22:0/1405484288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fa80fff20 0x7f2fa8100320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009880 tx=0x7f2f9c02a5d0 comp rx=0 tx=0).stop 2024-09-19T08:06:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.501+0000 7f2faf1a7640 1 -- 172.21.15.22:0/1405484288 shutdown_connections 2024-09-19T08:06:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.501+0000 7f2faf1a7640 1 --2- 172.21.15.22:0/1405484288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fa80fff20 0x7f2fa8100320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.501+0000 7f2faf1a7640 1 -- 172.21.15.22:0/1405484288 >> 172.21.15.22:0/1405484288 conn(0x7f2fa80fbcd0 msgr2=0x7f2fa80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.501+0000 7f2faf1a7640 1 -- 172.21.15.22:0/1405484288 shutdown_connections 2024-09-19T08:06:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.501+0000 7f2faf1a7640 1 -- 172.21.15.22:0/1405484288 wait complete. 2024-09-19T08:06:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.502+0000 7f2faf1a7640 1 Processor -- start 2024-09-19T08:06:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.502+0000 7f2faf1a7640 1 -- start start 2024-09-19T08:06:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.502+0000 7f2faf1a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fa80fff20 0x7f2fa8198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.502+0000 7f2faf1a7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa81098d0 con 0x7f2fa80fff20 2024-09-19T08:06:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.502+0000 7f2facf43640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fa80fff20 0x7f2fa8198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.502+0000 7f2facf43640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fa80fff20 0x7f2fa8198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55490/0 (socket says 172.21.15.22:55490) 2024-09-19T08:06:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.502+0000 7f2facf43640 1 -- 172.21.15.22:0/3045122227 learned_addr learned my addr 172.21.15.22:0/3045122227 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.503+0000 7f2facf43640 1 -- 172.21.15.22:0/3045122227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa81994b0 con 0x7f2fa80fff20 2024-09-19T08:06:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.503+0000 7f2facf43640 1 --2- 172.21.15.22:0/3045122227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fa80fff20 0x7f2fa8198f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c004f60 tx=0x7f2f9c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.503+0000 7f2fa5ffb640 1 -- 172.21.15.22:0/3045122227 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f9c03f070 con 0x7f2fa80fff20 2024-09-19T08:06:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.503+0000 7f2fa5ffb640 1 -- 172.21.15.22:0/3045122227 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f9c003ad0 con 0x7f2fa80fff20 2024-09-19T08:06:17.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.503+0000 7f2fa5ffb640 1 -- 172.21.15.22:0/3045122227 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f9c035030 con 0x7f2fa80fff20 2024-09-19T08:06:17.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.503+0000 7f2faf1a7640 1 -- 172.21.15.22:0/3045122227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa8199740 con 0x7f2fa80fff20 2024-09-19T08:06:17.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.503+0000 7f2faf1a7640 1 -- 172.21.15.22:0/3045122227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa819c430 con 0x7f2fa80fff20 2024-09-19T08:06:17.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.504+0000 7f2fa5ffb640 1 -- 172.21.15.22:0/3045122227 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f2f9c042070 con 0x7f2fa80fff20 2024-09-19T08:06:17.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.504+0000 7f2fa5ffb640 1 --2- 172.21.15.22:0/3045122227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2f98039300 0x7f2f9803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:17.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.504+0000 7f2fa5ffb640 1 -- 172.21.15.22:0/3045122227 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2f9c06d130 con 0x7f2fa80fff20 2024-09-19T08:06:17.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.505+0000 7f2fa7fff640 1 --2- 172.21.15.22:0/3045122227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2f98039300 0x7f2f9803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:17.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.505+0000 7f2faf1a7640 1 -- 172.21.15.22:0/3045122227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa810a050 con 0x7f2fa80fff20 2024-09-19T08:06:17.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.505+0000 7f2fa7fff640 1 --2- 172.21.15.22:0/3045122227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2f98039300 0x7f2f9803b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f940098c0 tx=0x7f2f94006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:17.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.512+0000 7f2fa5ffb640 1 -- 172.21.15.22:0/3045122227 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2f9c03a3c0 con 0x7f2fa80fff20 2024-09-19T08:06:17.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.716+0000 7f2faf1a7640 1 -- 172.21.15.22:0/3045122227 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f2fa802d830 con 0x7f2f98039300 2024-09-19T08:06:17.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.719+0000 7f2fa5ffb640 1 -- 172.21.15.22:0/3045122227 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2fa802d830 con 0x7f2f98039300 2024-09-19T08:06:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.721+0000 7f2faf1a7640 1 -- 172.21.15.22:0/3045122227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2f98039300 msgr2=0x7f2f9803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:17.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.721+0000 7f2faf1a7640 1 --2- 172.21.15.22:0/3045122227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2f98039300 0x7f2f9803b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f940098c0 tx=0x7f2f94006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:17.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.721+0000 7f2faf1a7640 1 -- 172.21.15.22:0/3045122227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fa80fff20 msgr2=0x7f2fa8198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:17.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.721+0000 7f2faf1a7640 1 --2- 172.21.15.22:0/3045122227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fa80fff20 0x7f2fa8198f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c004f60 tx=0x7f2f9c0048f0 comp rx=0 tx=0).stop 2024-09-19T08:06:17.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.721+0000 7f2faf1a7640 1 -- 172.21.15.22:0/3045122227 shutdown_connections 2024-09-19T08:06:17.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.721+0000 7f2faf1a7640 1 --2- 172.21.15.22:0/3045122227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2f98039300 0x7f2f9803b7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:17.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.721+0000 7f2faf1a7640 1 --2- 172.21.15.22:0/3045122227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fa80fff20 0x7f2fa8198f70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:17.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.721+0000 7f2faf1a7640 1 -- 172.21.15.22:0/3045122227 >> 172.21.15.22:0/3045122227 conn(0x7f2fa80fbcd0 msgr2=0x7f2fa80fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:17.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.721+0000 7f2faf1a7640 1 -- 172.21.15.22:0/3045122227 shutdown_connections 2024-09-19T08:06:17.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:17.721+0000 7f2faf1a7640 1 -- 172.21.15.22:0/3045122227 wait complete. 2024-09-19T08:06:18.403 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi064 2024-09-19T08:06:18.403 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T08:06:18.404 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/etc/ceph/ceph.conf 2024-09-19T08:06:18.424 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T08:06:18.424 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:06:18.482 INFO:tasks.cephadm:Adding host smithi064 to orchestrator... 2024-09-19T08:06:18.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch host add smithi064 2024-09-19T08:06:18.852 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:06:18.974 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:18 smithi022 ceph-mon[30963]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:06:18.974 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:18 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:19.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.387+0000 7ff659289640 1 -- 172.21.15.22:0/787285820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff654101620 msgr2=0x7ff654101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:19.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.387+0000 7ff659289640 1 --2- 172.21.15.22:0/787285820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff654101620 0x7ff654101a20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff644009880 tx=0x7ff64402a5d0 comp rx=0 tx=0).stop 2024-09-19T08:06:19.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.387+0000 7ff659289640 1 -- 172.21.15.22:0/787285820 shutdown_connections 2024-09-19T08:06:19.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.387+0000 7ff659289640 1 --2- 172.21.15.22:0/787285820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff654101620 0x7ff654101a20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:19.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.387+0000 7ff659289640 1 -- 172.21.15.22:0/787285820 >> 172.21.15.22:0/787285820 conn(0x7ff6540fcdb0 msgr2=0x7ff6540ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:19.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.387+0000 7ff659289640 1 -- 172.21.15.22:0/787285820 shutdown_connections 2024-09-19T08:06:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.387+0000 7ff659289640 1 -- 172.21.15.22:0/787285820 wait complete. 2024-09-19T08:06:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.388+0000 7ff659289640 1 Processor -- start 2024-09-19T08:06:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.388+0000 7ff659289640 1 -- start start 2024-09-19T08:06:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.388+0000 7ff659289640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff654101620 0x7ff654198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.388+0000 7ff659289640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff654107730 con 0x7ff654101620 2024-09-19T08:06:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.388+0000 7ff652ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff654101620 0x7ff654198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.388+0000 7ff652ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff654101620 0x7ff654198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55496/0 (socket says 172.21.15.22:55496) 2024-09-19T08:06:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.388+0000 7ff652ffd640 1 -- 172.21.15.22:0/840618912 learned_addr learned my addr 172.21.15.22:0/840618912 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.389+0000 7ff652ffd640 1 -- 172.21.15.22:0/840618912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6541994c0 con 0x7ff654101620 2024-09-19T08:06:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.389+0000 7ff652ffd640 1 --2- 172.21.15.22:0/840618912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff654101620 0x7ff654198f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff644004f60 tx=0x7ff6440048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.389+0000 7ff63bfff640 1 -- 172.21.15.22:0/840618912 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff64403f070 con 0x7ff654101620 2024-09-19T08:06:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.389+0000 7ff63bfff640 1 -- 172.21.15.22:0/840618912 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff644004d40 con 0x7ff654101620 2024-09-19T08:06:19.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.389+0000 7ff659289640 1 -- 172.21.15.22:0/840618912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff654199750 con 0x7ff654101620 2024-09-19T08:06:19.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.389+0000 7ff659289640 1 -- 172.21.15.22:0/840618912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65419c440 con 0x7ff654101620 2024-09-19T08:06:19.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.389+0000 7ff63bfff640 1 -- 172.21.15.22:0/840618912 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff644035030 con 0x7ff654101620 2024-09-19T08:06:19.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.390+0000 7ff63bfff640 1 -- 172.21.15.22:0/840618912 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ff644042070 con 0x7ff654101620 2024-09-19T08:06:19.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.390+0000 7ff659289640 1 -- 172.21.15.22:0/840618912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6400052c0 con 0x7ff654101620 2024-09-19T08:06:19.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.390+0000 7ff63bfff640 1 --2- 172.21.15.22:0/840618912 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff63c039300 0x7ff63c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:19.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.390+0000 7ff63bfff640 1 -- 172.21.15.22:0/840618912 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff64406d090 con 0x7ff654101620 2024-09-19T08:06:19.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.390+0000 7ff6527fc640 1 --2- 172.21.15.22:0/840618912 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff63c039300 0x7ff63c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:19.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.391+0000 7ff6527fc640 1 --2- 172.21.15.22:0/840618912 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff63c039300 0x7ff63c03b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff6480098c0 tx=0x7ff648006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:19.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.395+0000 7ff63bfff640 1 -- 172.21.15.22:0/840618912 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff644042360 con 0x7ff654101620 2024-09-19T08:06:19.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:19.590+0000 7ff659289640 1 -- 172.21.15.22:0/840618912 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi064", "target": ["mon-mgr", ""]}) -- 0x7ff640000bf0 con 0x7ff63c039300 2024-09-19T08:06:20.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:20.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:20.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi022", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:06:20.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:20.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:06:20.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: Updating smithi022:/etc/ceph/ceph.conf 2024-09-19T08:06:20.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: Updating smithi022:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:06:20.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:06:20.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: Updating smithi022:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:06:20.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:06:20.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:20.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:20.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:20.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:06:20.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:20.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:06:20.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:22.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:21 smithi022 ceph-mon[30963]: Deploying cephadm binary to smithi064 2024-09-19T08:06:23.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:23.503+0000 7ff63bfff640 1 -- 172.21.15.22:0/840618912 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7ff640000bf0 con 0x7ff63c039300 2024-09-19T08:06:23.504 INFO:teuthology.orchestra.run.smithi022.stdout:Added host 'smithi064' with addr '172.21.15.64' 2024-09-19T08:06:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:23.506+0000 7ff659289640 1 -- 172.21.15.22:0/840618912 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff63c039300 msgr2=0x7ff63c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:23.506+0000 7ff659289640 1 --2- 172.21.15.22:0/840618912 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff63c039300 0x7ff63c03b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff6480098c0 tx=0x7ff648006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:23.506+0000 7ff659289640 1 -- 172.21.15.22:0/840618912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff654101620 msgr2=0x7ff654198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:23.506+0000 7ff659289640 1 --2- 172.21.15.22:0/840618912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff654101620 0x7ff654198f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff644004f60 tx=0x7ff6440048f0 comp rx=0 tx=0).stop 2024-09-19T08:06:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:23.506+0000 7ff659289640 1 -- 172.21.15.22:0/840618912 shutdown_connections 2024-09-19T08:06:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:23.506+0000 7ff659289640 1 --2- 172.21.15.22:0/840618912 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff63c039300 0x7ff63c03b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:23.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:23.506+0000 7ff659289640 1 --2- 172.21.15.22:0/840618912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff654101620 0x7ff654198f80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:23.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:23.506+0000 7ff659289640 1 -- 172.21.15.22:0/840618912 >> 172.21.15.22:0/840618912 conn(0x7ff6540fcdb0 msgr2=0x7ff6540fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:23.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:23.506+0000 7ff659289640 1 -- 172.21.15.22:0/840618912 shutdown_connections 2024-09-19T08:06:23.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:23.506+0000 7ff659289640 1 -- 172.21.15.22:0/840618912 wait complete. 2024-09-19T08:06:24.059 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T08:06:24.310 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:06:24.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:24 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:24.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:24 smithi022 ceph-mon[30963]: Added host smithi064 2024-09-19T08:06:24.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:24 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:06:24.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:24 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:24.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:24 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.766+0000 7fc9bea97640 1 -- 172.21.15.22:0/3310493071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9b8106540 msgr2=0x7fc9b8106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.766+0000 7fc9bea97640 1 --2- 172.21.15.22:0/3310493071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9b8106540 0x7fc9b8106920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009810 tx=0x7fc9b002a540 comp rx=0 tx=0).stop 2024-09-19T08:06:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.766+0000 7fc9bea97640 1 -- 172.21.15.22:0/3310493071 shutdown_connections 2024-09-19T08:06:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.766+0000 7fc9bea97640 1 --2- 172.21.15.22:0/3310493071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9b8106540 0x7fc9b8106920 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.766+0000 7fc9bea97640 1 -- 172.21.15.22:0/3310493071 >> 172.21.15.22:0/3310493071 conn(0x7fc9b8077310 msgr2=0x7fc9b80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.766+0000 7fc9bea97640 1 -- 172.21.15.22:0/3310493071 shutdown_connections 2024-09-19T08:06:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.766+0000 7fc9bea97640 1 -- 172.21.15.22:0/3310493071 wait complete. 2024-09-19T08:06:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.766+0000 7fc9bea97640 1 Processor -- start 2024-09-19T08:06:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.767+0000 7fc9bea97640 1 -- start start 2024-09-19T08:06:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.767+0000 7fc9bea97640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9b8106540 0x7fc9b8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.767+0000 7fc9bea97640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b8109a80 con 0x7fc9b8106540 2024-09-19T08:06:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.767+0000 7fc9bc833640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9b8106540 0x7fc9b8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.767+0000 7fc9bc833640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9b8106540 0x7fc9b8199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55510/0 (socket says 172.21.15.22:55510) 2024-09-19T08:06:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.767+0000 7fc9bc833640 1 -- 172.21.15.22:0/581031860 learned_addr learned my addr 172.21.15.22:0/581031860 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.767+0000 7fc9bc833640 1 -- 172.21.15.22:0/581031860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b8199870 con 0x7fc9b8106540 2024-09-19T08:06:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.767+0000 7fc9bc833640 1 --2- 172.21.15.22:0/581031860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9b8106540 0x7fc9b8199330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009930 tx=0x7fc9b0004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.767+0000 7fc9b5ffb640 1 -- 172.21.15.22:0/581031860 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b0031400 con 0x7fc9b8106540 2024-09-19T08:06:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.768+0000 7fc9bea97640 1 -- 172.21.15.22:0/581031860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b8199b00 con 0x7fc9b8106540 2024-09-19T08:06:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.768+0000 7fc9b5ffb640 1 -- 172.21.15.22:0/581031860 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9b00035f0 con 0x7fc9b8106540 2024-09-19T08:06:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.768+0000 7fc9b5ffb640 1 -- 172.21.15.22:0/581031860 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b003c440 con 0x7fc9b8106540 2024-09-19T08:06:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.768+0000 7fc9bea97640 1 -- 172.21.15.22:0/581031860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b819dc00 con 0x7fc9b8106540 2024-09-19T08:06:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.769+0000 7fc9b5ffb640 1 -- 172.21.15.22:0/581031860 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fc9b0039040 con 0x7fc9b8106540 2024-09-19T08:06:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.769+0000 7fc9bea97640 1 -- 172.21.15.22:0/581031860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b8101bb0 con 0x7fc9b8106540 2024-09-19T08:06:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.769+0000 7fc9b5ffb640 1 --2- 172.21.15.22:0/581031860 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc9a40392a0 0x7fc9a403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.769+0000 7fc9b5ffb640 1 -- 172.21.15.22:0/581031860 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc9b006d540 con 0x7fc9b8106540 2024-09-19T08:06:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.769+0000 7fc9b7fff640 1 --2- 172.21.15.22:0/581031860 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc9a40392a0 0x7fc9a403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.769+0000 7fc9b7fff640 1 --2- 172.21.15.22:0/581031860 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc9a40392a0 0x7fc9a403b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800ad30 tx=0x7fc9a80093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.774+0000 7fc9b5ffb640 1 -- 172.21.15.22:0/581031860 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc9b0040030 con 0x7fc9b8106540 2024-09-19T08:06:24.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.957+0000 7fc9bea97640 1 -- 172.21.15.22:0/581031860 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc9b819a690 con 0x7fc9a40392a0 2024-09-19T08:06:24.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.958+0000 7fc9b5ffb640 1 -- 172.21.15.22:0/581031860 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fc9b819a690 con 0x7fc9a40392a0 2024-09-19T08:06:24.958 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:06:24.958 INFO:teuthology.orchestra.run.smithi022.stdout:[{"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}] 2024-09-19T08:06:24.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.960+0000 7fc9bea97640 1 -- 172.21.15.22:0/581031860 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc9a40392a0 msgr2=0x7fc9a403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:24.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.960+0000 7fc9bea97640 1 --2- 172.21.15.22:0/581031860 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc9a40392a0 0x7fc9a403b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800ad30 tx=0x7fc9a80093f0 comp rx=0 tx=0).stop 2024-09-19T08:06:24.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.960+0000 7fc9bea97640 1 -- 172.21.15.22:0/581031860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9b8106540 msgr2=0x7fc9b8199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:24.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.960+0000 7fc9bea97640 1 --2- 172.21.15.22:0/581031860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9b8106540 0x7fc9b8199330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009930 tx=0x7fc9b0004570 comp rx=0 tx=0).stop 2024-09-19T08:06:24.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.960+0000 7fc9bea97640 1 -- 172.21.15.22:0/581031860 shutdown_connections 2024-09-19T08:06:24.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.960+0000 7fc9bea97640 1 --2- 172.21.15.22:0/581031860 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc9a40392a0 0x7fc9a403b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:24.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.960+0000 7fc9bea97640 1 --2- 172.21.15.22:0/581031860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9b8106540 0x7fc9b8199330 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:24.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.960+0000 7fc9bea97640 1 -- 172.21.15.22:0/581031860 >> 172.21.15.22:0/581031860 conn(0x7fc9b8077310 msgr2=0x7fc9b80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:24.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.961+0000 7fc9bea97640 1 -- 172.21.15.22:0/581031860 shutdown_connections 2024-09-19T08:06:24.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:24.961+0000 7fc9bea97640 1 -- 172.21.15.22:0/581031860 wait complete. 2024-09-19T08:06:25.603 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi157 2024-09-19T08:06:25.604 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T08:06:25.604 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.conf 2024-09-19T08:06:25.623 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T08:06:25.623 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:06:25.682 INFO:tasks.cephadm:Adding host smithi157 to orchestrator... 2024-09-19T08:06:25.682 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch host add smithi157 2024-09-19T08:06:25.936 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:06:26.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:26.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:25 smithi022 ceph-mon[30963]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:06:26.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.334+0000 7fcdddd32640 1 -- 172.21.15.22:0/2090912951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd8106540 msgr2=0x7fcdd8106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:26.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.334+0000 7fcdddd32640 1 --2- 172.21.15.22:0/2090912951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd8106540 0x7fcdd8106920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009810 tx=0x7fcdc802a540 comp rx=0 tx=0).stop 2024-09-19T08:06:26.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.334+0000 7fcdddd32640 1 -- 172.21.15.22:0/2090912951 shutdown_connections 2024-09-19T08:06:26.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.334+0000 7fcdddd32640 1 --2- 172.21.15.22:0/2090912951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd8106540 0x7fcdd8106920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:26.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.334+0000 7fcdddd32640 1 -- 172.21.15.22:0/2090912951 >> 172.21.15.22:0/2090912951 conn(0x7fcdd8077310 msgr2=0x7fcdd80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:26.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.334+0000 7fcdddd32640 1 -- 172.21.15.22:0/2090912951 shutdown_connections 2024-09-19T08:06:26.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.334+0000 7fcdddd32640 1 -- 172.21.15.22:0/2090912951 wait complete. 2024-09-19T08:06:26.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.335+0000 7fcdddd32640 1 Processor -- start 2024-09-19T08:06:26.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.335+0000 7fcdddd32640 1 -- start start 2024-09-19T08:06:26.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.335+0000 7fcdddd32640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd8106540 0x7fcdd8196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:26.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.335+0000 7fcdddd32640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd810ab70 con 0x7fcdd8106540 2024-09-19T08:06:26.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.335+0000 7fcdd77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd8106540 0x7fcdd8196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:26.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.335+0000 7fcdd77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd8106540 0x7fcdd8196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55526/0 (socket says 172.21.15.22:55526) 2024-09-19T08:06:26.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.335+0000 7fcdd77fe640 1 -- 172.21.15.22:0/3673193855 learned_addr learned my addr 172.21.15.22:0/3673193855 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:26.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.336+0000 7fcdd77fe640 1 -- 172.21.15.22:0/3673193855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd81965a0 con 0x7fcdd8106540 2024-09-19T08:06:26.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.336+0000 7fcdd77fe640 1 --2- 172.21.15.22:0/3673193855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd8106540 0x7fcdd8196060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80045e0 tx=0x7fcdc8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:26.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.336+0000 7fcdd4ff9640 1 -- 172.21.15.22:0/3673193855 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdc8030e70 con 0x7fcdd8106540 2024-09-19T08:06:26.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.336+0000 7fcdd4ff9640 1 -- 172.21.15.22:0/3673193855 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc80318e0 con 0x7fcdd8106540 2024-09-19T08:06:26.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.336+0000 7fcdddd32640 1 -- 172.21.15.22:0/3673193855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd8196830 con 0x7fcdd8106540 2024-09-19T08:06:26.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.336+0000 7fcdd4ff9640 1 -- 172.21.15.22:0/3673193855 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdc80049c0 con 0x7fcdd8106540 2024-09-19T08:06:26.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.336+0000 7fcdddd32640 1 -- 172.21.15.22:0/3673193855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd8196c90 con 0x7fcdd8106540 2024-09-19T08:06:26.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.337+0000 7fcdddd32640 1 -- 172.21.15.22:0/3673193855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd8101bb0 con 0x7fcdd8106540 2024-09-19T08:06:26.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.337+0000 7fcdd4ff9640 1 -- 172.21.15.22:0/3673193855 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fcdc8039040 con 0x7fcdd8106540 2024-09-19T08:06:26.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.338+0000 7fcdd4ff9640 1 --2- 172.21.15.22:0/3673193855 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcdc00392a0 0x7fcdc003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:26.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.338+0000 7fcdd4ff9640 1 -- 172.21.15.22:0/3673193855 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcdc806d680 con 0x7fcdd8106540 2024-09-19T08:06:26.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.338+0000 7fcdd6ffd640 1 --2- 172.21.15.22:0/3673193855 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcdc00392a0 0x7fcdc003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:26.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.338+0000 7fcdd6ffd640 1 --2- 172.21.15.22:0/3673193855 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcdc00392a0 0x7fcdc003b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0098c0 tx=0x7fcdcc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:26.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.347+0000 7fcdd4ff9640 1 -- 172.21.15.22:0/3673193855 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcdc8040030 con 0x7fcdd8106540 2024-09-19T08:06:26.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:26.528+0000 7fcdddd32640 1 -- 172.21.15.22:0/3673193855 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}) -- 0x7fcdd81973c0 con 0x7fcdc00392a0 2024-09-19T08:06:27.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:26 smithi022 ceph-mon[30963]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:06:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:27 smithi022 ceph-mon[30963]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:27 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:27 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:27 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:27 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:27 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:06:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:27 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:27 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:06:30.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:29 smithi022 ceph-mon[30963]: Deploying cephadm binary to smithi157 2024-09-19T08:06:30.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:29 smithi022 ceph-mon[30963]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-19T08:06:30.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:29 smithi022 ceph-mon[30963]: Updating smithi064:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:06:30.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:29 smithi022 ceph-mon[30963]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:06:30.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:29 smithi022 ceph-mon[30963]: Updating smithi064:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:06:30.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:30.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:30.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:30.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:30.445+0000 7fcdd4ff9640 1 -- 172.21.15.22:0/3673193855 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fcdd81973c0 con 0x7fcdc00392a0 2024-09-19T08:06:30.446 INFO:teuthology.orchestra.run.smithi022.stdout:Added host 'smithi157' with addr '172.21.15.157' 2024-09-19T08:06:30.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:30.448+0000 7fcdddd32640 1 -- 172.21.15.22:0/3673193855 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcdc00392a0 msgr2=0x7fcdc003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:30.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:30.448+0000 7fcdddd32640 1 --2- 172.21.15.22:0/3673193855 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcdc00392a0 0x7fcdc003b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0098c0 tx=0x7fcdcc006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:30.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:30.448+0000 7fcdddd32640 1 -- 172.21.15.22:0/3673193855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd8106540 msgr2=0x7fcdd8196060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:30.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:30.448+0000 7fcdddd32640 1 --2- 172.21.15.22:0/3673193855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd8106540 0x7fcdd8196060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80045e0 tx=0x7fcdc8004610 comp rx=0 tx=0).stop 2024-09-19T08:06:30.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:30.448+0000 7fcdddd32640 1 -- 172.21.15.22:0/3673193855 shutdown_connections 2024-09-19T08:06:30.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:30.448+0000 7fcdddd32640 1 --2- 172.21.15.22:0/3673193855 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcdc00392a0 0x7fcdc003b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:30.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:30.448+0000 7fcdddd32640 1 --2- 172.21.15.22:0/3673193855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd8106540 0x7fcdd8196060 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:30.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:30.448+0000 7fcdddd32640 1 -- 172.21.15.22:0/3673193855 >> 172.21.15.22:0/3673193855 conn(0x7fcdd8077310 msgr2=0x7fcdd80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:30.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:30.448+0000 7fcdddd32640 1 -- 172.21.15.22:0/3673193855 shutdown_connections 2024-09-19T08:06:30.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:30.448+0000 7fcdddd32640 1 -- 172.21.15.22:0/3673193855 wait complete. 2024-09-19T08:06:30.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:30 smithi022 ceph-mon[30963]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:30.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:30 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:30.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:30 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:06:31.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T08:06:31.300 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:06:31.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:31 smithi022 ceph-mon[30963]: Added host smithi157 2024-09-19T08:06:31.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:31 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:31.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:31 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:31.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.747+0000 7f85f061a640 1 -- 172.21.15.22:0/3343024699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e8107660 msgr2=0x7f85e8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:31.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.747+0000 7f85f061a640 1 --2- 172.21.15.22:0/3343024699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e8107660 0x7f85e8107a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009880 tx=0x7f85e402a5d0 comp rx=0 tx=0).stop 2024-09-19T08:06:31.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.747+0000 7f85f061a640 1 -- 172.21.15.22:0/3343024699 shutdown_connections 2024-09-19T08:06:31.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.747+0000 7f85f061a640 1 --2- 172.21.15.22:0/3343024699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e8107660 0x7f85e8107a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.747+0000 7f85f061a640 1 -- 172.21.15.22:0/3343024699 >> 172.21.15.22:0/3343024699 conn(0x7f85e80fd290 msgr2=0x7f85e80ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.747+0000 7f85f061a640 1 -- 172.21.15.22:0/3343024699 shutdown_connections 2024-09-19T08:06:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.747+0000 7f85f061a640 1 -- 172.21.15.22:0/3343024699 wait complete. 2024-09-19T08:06:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.748+0000 7f85f061a640 1 Processor -- start 2024-09-19T08:06:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.748+0000 7f85f061a640 1 -- start start 2024-09-19T08:06:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.748+0000 7f85f061a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e8107660 0x7f85e81970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.748+0000 7f85f061a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f85e810bc90 con 0x7f85e8107660 2024-09-19T08:06:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.748+0000 7f85ee3b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e8107660 0x7f85e81970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.749+0000 7f85ee3b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e8107660 0x7f85e81970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55654/0 (socket says 172.21.15.22:55654) 2024-09-19T08:06:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.749+0000 7f85ee3b6640 1 -- 172.21.15.22:0/3400854055 learned_addr learned my addr 172.21.15.22:0/3400854055 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.749+0000 7f85ee3b6640 1 -- 172.21.15.22:0/3400854055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e8197630 con 0x7f85e8107660 2024-09-19T08:06:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.749+0000 7f85ee3b6640 1 --2- 172.21.15.22:0/3400854055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e8107660 0x7f85e81970f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009230 tx=0x7f85e4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.749+0000 7f85df7fe640 1 -- 172.21.15.22:0/3400854055 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85e403f070 con 0x7f85e8107660 2024-09-19T08:06:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.749+0000 7f85df7fe640 1 -- 172.21.15.22:0/3400854055 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e403bc80 con 0x7f85e8107660 2024-09-19T08:06:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.749+0000 7f85f061a640 1 -- 172.21.15.22:0/3400854055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e81978c0 con 0x7f85e8107660 2024-09-19T08:06:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.749+0000 7f85df7fe640 1 -- 172.21.15.22:0/3400854055 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85e4035030 con 0x7f85e8107660 2024-09-19T08:06:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.749+0000 7f85f061a640 1 -- 172.21.15.22:0/3400854055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e819b9c0 con 0x7f85e8107660 2024-09-19T08:06:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.750+0000 7f85df7fe640 1 -- 172.21.15.22:0/3400854055 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f85e4042070 con 0x7f85e8107660 2024-09-19T08:06:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.750+0000 7f85df7fe640 1 --2- 172.21.15.22:0/3400854055 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f85d40392a0 0x7f85d403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.750+0000 7f85df7fe640 1 -- 172.21.15.22:0/3400854055 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f85e406cf90 con 0x7f85e8107660 2024-09-19T08:06:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.750+0000 7f85edbb5640 1 --2- 172.21.15.22:0/3400854055 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f85d40392a0 0x7f85d403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.750+0000 7f85f061a640 1 -- 172.21.15.22:0/3400854055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e8102cd0 con 0x7f85e8107660 2024-09-19T08:06:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.751+0000 7f85edbb5640 1 --2- 172.21.15.22:0/3400854055 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f85d40392a0 0x7f85d403b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f85d80098c0 tx=0x7f85d8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.755+0000 7f85df7fe640 1 -- 172.21.15.22:0/3400854055 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f85e404c350 con 0x7f85e8107660 2024-09-19T08:06:31.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.940+0000 7f85f061a640 1 -- 172.21.15.22:0/3400854055 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85e8198450 con 0x7f85d40392a0 2024-09-19T08:06:31.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.942+0000 7f85df7fe640 1 -- 172.21.15.22:0/3400854055 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f85e8198450 con 0x7f85d40392a0 2024-09-19T08:06:31.942 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:06:31.942 INFO:teuthology.orchestra.run.smithi022.stdout:[{"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.157", "hostname": "smithi157", "labels": [], "status": ""}] 2024-09-19T08:06:31.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.944+0000 7f85f061a640 1 -- 172.21.15.22:0/3400854055 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f85d40392a0 msgr2=0x7f85d403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:31.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.944+0000 7f85f061a640 1 --2- 172.21.15.22:0/3400854055 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f85d40392a0 0x7f85d403b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f85d80098c0 tx=0x7f85d8006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:31.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.944+0000 7f85f061a640 1 -- 172.21.15.22:0/3400854055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e8107660 msgr2=0x7f85e81970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:31.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.944+0000 7f85f061a640 1 --2- 172.21.15.22:0/3400854055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e8107660 0x7f85e81970f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009230 tx=0x7f85e4003d40 comp rx=0 tx=0).stop 2024-09-19T08:06:31.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.944+0000 7f85f061a640 1 -- 172.21.15.22:0/3400854055 shutdown_connections 2024-09-19T08:06:31.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.944+0000 7f85f061a640 1 --2- 172.21.15.22:0/3400854055 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f85d40392a0 0x7f85d403b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:31.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.944+0000 7f85f061a640 1 --2- 172.21.15.22:0/3400854055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e8107660 0x7f85e81970f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:31.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.944+0000 7f85f061a640 1 -- 172.21.15.22:0/3400854055 >> 172.21.15.22:0/3400854055 conn(0x7f85e80fd290 msgr2=0x7f85e80febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:31.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.944+0000 7f85f061a640 1 -- 172.21.15.22:0/3400854055 shutdown_connections 2024-09-19T08:06:31.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:31.944+0000 7f85f061a640 1 -- 172.21.15.22:0/3400854055 wait complete. 2024-09-19T08:06:32.532 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T08:06:32.532 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T08:06:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:06:32.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:32 smithi022 ceph-mon[30963]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:32.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:32 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:33.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.222+0000 7fa06a314640 1 -- 172.21.15.22:0/1828891388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa064107660 msgr2=0x7fa064107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:33.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.222+0000 7fa06a314640 1 --2- 172.21.15.22:0/1828891388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa064107660 0x7fa064107a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa054009880 tx=0x7fa05402a5d0 comp rx=0 tx=0).stop 2024-09-19T08:06:33.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.222+0000 7fa06a314640 1 -- 172.21.15.22:0/1828891388 shutdown_connections 2024-09-19T08:06:33.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.222+0000 7fa06a314640 1 --2- 172.21.15.22:0/1828891388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa064107660 0x7fa064107a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:33.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.222+0000 7fa06a314640 1 -- 172.21.15.22:0/1828891388 >> 172.21.15.22:0/1828891388 conn(0x7fa0640fd290 msgr2=0x7fa0640ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:33.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.222+0000 7fa06a314640 1 -- 172.21.15.22:0/1828891388 shutdown_connections 2024-09-19T08:06:33.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.222+0000 7fa06a314640 1 -- 172.21.15.22:0/1828891388 wait complete. 2024-09-19T08:06:33.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.223+0000 7fa06a314640 1 Processor -- start 2024-09-19T08:06:33.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.223+0000 7fa06a314640 1 -- start start 2024-09-19T08:06:33.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.223+0000 7fa06a314640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa064107660 0x7fa0641992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:33.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.223+0000 7fa06a314640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa06410bc90 con 0x7fa064107660 2024-09-19T08:06:33.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.223+0000 7fa063fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa064107660 0x7fa0641992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:33.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.223+0000 7fa063fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa064107660 0x7fa0641992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55672/0 (socket says 172.21.15.22:55672) 2024-09-19T08:06:33.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.223+0000 7fa063fff640 1 -- 172.21.15.22:0/1956329726 learned_addr learned my addr 172.21.15.22:0/1956329726 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:33.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.223+0000 7fa063fff640 1 -- 172.21.15.22:0/1956329726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa064199800 con 0x7fa064107660 2024-09-19T08:06:33.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.224+0000 7fa063fff640 1 --2- 172.21.15.22:0/1956329726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa064107660 0x7fa0641992c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa054009230 tx=0x7fa054003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:33.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.224+0000 7fa0617fa640 1 -- 172.21.15.22:0/1956329726 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa05403f070 con 0x7fa064107660 2024-09-19T08:06:33.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.224+0000 7fa0617fa640 1 -- 172.21.15.22:0/1956329726 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05403bc80 con 0x7fa064107660 2024-09-19T08:06:33.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.224+0000 7fa06a314640 1 -- 172.21.15.22:0/1956329726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa064199a90 con 0x7fa064107660 2024-09-19T08:06:33.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.224+0000 7fa0617fa640 1 -- 172.21.15.22:0/1956329726 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa054035030 con 0x7fa064107660 2024-09-19T08:06:33.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.224+0000 7fa06a314640 1 -- 172.21.15.22:0/1956329726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06419db90 con 0x7fa064107660 2024-09-19T08:06:33.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.225+0000 7fa0617fa640 1 -- 172.21.15.22:0/1956329726 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fa054042070 con 0x7fa064107660 2024-09-19T08:06:33.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.225+0000 7fa06a314640 1 -- 172.21.15.22:0/1956329726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa064102cd0 con 0x7fa064107660 2024-09-19T08:06:33.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.225+0000 7fa0617fa640 1 --2- 172.21.15.22:0/1956329726 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa04c039300 0x7fa04c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:33.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.225+0000 7fa0617fa640 1 -- 172.21.15.22:0/1956329726 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa05406d060 con 0x7fa064107660 2024-09-19T08:06:33.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.225+0000 7fa0637fe640 1 --2- 172.21.15.22:0/1956329726 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa04c039300 0x7fa04c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:33.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.226+0000 7fa0637fe640 1 --2- 172.21.15.22:0/1956329726 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa04c039300 0x7fa04c03b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa0580098c0 tx=0x7fa058006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.230+0000 7fa0617fa640 1 -- 172.21.15.22:0/1956329726 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa054031dd0 con 0x7fa064107660 2024-09-19T08:06:33.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.410+0000 7fa06a314640 1 -- 172.21.15.22:0/1956329726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fa06419a620 con 0x7fa064107660 2024-09-19T08:06:33.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.620+0000 7fa0617fa640 1 -- 172.21.15.22:0/1956329726 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fa05403a3c0 con 0x7fa064107660 2024-09-19T08:06:33.620 INFO:teuthology.orchestra.run.smithi022.stderr:adjusted tunables profile to default 2024-09-19T08:06:33.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.622+0000 7fa06a314640 1 -- 172.21.15.22:0/1956329726 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa04c039300 msgr2=0x7fa04c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:33.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.622+0000 7fa06a314640 1 --2- 172.21.15.22:0/1956329726 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa04c039300 0x7fa04c03b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa0580098c0 tx=0x7fa058006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:33.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.622+0000 7fa06a314640 1 -- 172.21.15.22:0/1956329726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa064107660 msgr2=0x7fa0641992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:33.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.622+0000 7fa06a314640 1 --2- 172.21.15.22:0/1956329726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa064107660 0x7fa0641992c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa054009230 tx=0x7fa054003d40 comp rx=0 tx=0).stop 2024-09-19T08:06:33.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.622+0000 7fa06a314640 1 -- 172.21.15.22:0/1956329726 shutdown_connections 2024-09-19T08:06:33.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.622+0000 7fa06a314640 1 --2- 172.21.15.22:0/1956329726 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa04c039300 0x7fa04c03b7c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:33.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.622+0000 7fa06a314640 1 --2- 172.21.15.22:0/1956329726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa064107660 0x7fa0641992c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:33.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.622+0000 7fa06a314640 1 -- 172.21.15.22:0/1956329726 >> 172.21.15.22:0/1956329726 conn(0x7fa0640fd290 msgr2=0x7fa064075df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:33.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.622+0000 7fa06a314640 1 -- 172.21.15.22:0/1956329726 shutdown_connections 2024-09-19T08:06:33.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:33.622+0000 7fa06a314640 1 -- 172.21.15.22:0/1956329726 wait complete. 2024-09-19T08:06:33.975 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:33 smithi022 ceph-mon[30963]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:06:33.975 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:33 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1956329726' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T08:06:34.282 INFO:tasks.cephadm:Adding mon.a on smithi022 2024-09-19T08:06:34.282 INFO:tasks.cephadm:Adding mon.b on smithi064 2024-09-19T08:06:34.282 INFO:tasks.cephadm:Adding mon.c on smithi157 2024-09-19T08:06:34.282 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi022:172.21.15.22=a;smithi064:172.21.15.64=b;smithi157:172.21.15.157=c' 2024-09-19T08:06:34.527 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T08:06:34.573 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T08:06:35.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:34 smithi022 ceph-mon[30963]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:35.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:34 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1956329726' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T08:06:35.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:34 smithi022 ceph-mon[30963]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:06:35.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:35.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:35.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:35.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:35.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:06:35.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:35.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:06:35.155 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.127+0000 7fec24487640 1 -- 172.21.15.157:0/3395417331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec1c0ff900 msgr2=0x7fec1c0ffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:35.155 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.127+0000 7fec24487640 1 --2- 172.21.15.157:0/3395417331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec1c0ff900 0x7fec1c0ffd00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec18009880 tx=0x7fec1802a5d0 comp rx=0 tx=0).stop 2024-09-19T08:06:35.155 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.127+0000 7fec24487640 1 -- 172.21.15.157:0/3395417331 shutdown_connections 2024-09-19T08:06:35.155 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.127+0000 7fec24487640 1 --2- 172.21.15.157:0/3395417331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec1c0ff900 0x7fec1c0ffd00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:35.155 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.127+0000 7fec24487640 1 -- 172.21.15.157:0/3395417331 >> 172.21.15.157:0/3395417331 conn(0x7fec1c0fb450 msgr2=0x7fec1c0fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:35.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.127+0000 7fec24487640 1 -- 172.21.15.157:0/3395417331 shutdown_connections 2024-09-19T08:06:35.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.127+0000 7fec24487640 1 -- 172.21.15.157:0/3395417331 wait complete. 2024-09-19T08:06:35.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.128+0000 7fec24487640 1 Processor -- start 2024-09-19T08:06:35.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.128+0000 7fec24487640 1 -- start start 2024-09-19T08:06:35.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.128+0000 7fec24487640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec1c0ff900 0x7fec1c196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:35.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.128+0000 7fec24487640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c1076e0 con 0x7fec1c0ff900 2024-09-19T08:06:35.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.128+0000 7fec22223640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec1c0ff900 0x7fec1c196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:35.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.128+0000 7fec22223640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec1c0ff900 0x7fec1c196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.157:53130/0 (socket says 172.21.15.157:53130) 2024-09-19T08:06:35.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.128+0000 7fec22223640 1 -- 172.21.15.157:0/91858516 learned_addr learned my addr 172.21.15.157:0/91858516 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T08:06:35.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.129+0000 7fec22223640 1 -- 172.21.15.157:0/91858516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c1972f0 con 0x7fec1c0ff900 2024-09-19T08:06:35.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.129+0000 7fec22223640 1 --2- 172.21.15.157:0/91858516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec1c0ff900 0x7fec1c196db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec18004f60 tx=0x7fec180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:35.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.129+0000 7fec137fe640 1 -- 172.21.15.157:0/91858516 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec1803f070 con 0x7fec1c0ff900 2024-09-19T08:06:35.156 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.129+0000 7fec24487640 1 -- 172.21.15.157:0/91858516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1c197580 con 0x7fec1c0ff900 2024-09-19T08:06:35.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.129+0000 7fec24487640 1 -- 172.21.15.157:0/91858516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1c19a270 con 0x7fec1c0ff900 2024-09-19T08:06:35.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.129+0000 7fec137fe640 1 -- 172.21.15.157:0/91858516 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec18004d40 con 0x7fec1c0ff900 2024-09-19T08:06:35.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.129+0000 7fec137fe640 1 -- 172.21.15.157:0/91858516 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec18049060 con 0x7fec1c0ff900 2024-09-19T08:06:35.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.130+0000 7fec137fe640 1 -- 172.21.15.157:0/91858516 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fec18042070 con 0x7fec1c0ff900 2024-09-19T08:06:35.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.130+0000 7fec24487640 1 -- 172.21.15.157:0/91858516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1c107e60 con 0x7fec1c0ff900 2024-09-19T08:06:35.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.130+0000 7fec137fe640 1 --2- 172.21.15.157:0/91858516 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fec080392a0 0x7fec0803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:35.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.130+0000 7fec137fe640 1 -- 172.21.15.157:0/91858516 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fec1806d030 con 0x7fec1c0ff900 2024-09-19T08:06:35.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.131+0000 7fec21a22640 1 --2- 172.21.15.157:0/91858516 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fec080392a0 0x7fec0803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:35.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.131+0000 7fec21a22640 1 --2- 172.21.15.157:0/91858516 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fec080392a0 0x7fec0803b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0098c0 tx=0x7fec0c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:35.157 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.137+0000 7fec137fe640 1 -- 172.21.15.157:0/91858516 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fec18039b70 con 0x7fec1c0ff900 2024-09-19T08:06:35.341 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.334+0000 7fec24487640 1 -- 172.21.15.157:0/91858516 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi064:172.21.15.64=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}) -- 0x7fec1c062630 con 0x7fec080392a0 2024-09-19T08:06:35.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.340+0000 7fec137fe640 1 -- 172.21.15.157:0/91858516 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fec1c062630 con 0x7fec080392a0 2024-09-19T08:06:35.391 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mon update... 2024-09-19T08:06:35.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.342+0000 7fec24487640 1 -- 172.21.15.157:0/91858516 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fec080392a0 msgr2=0x7fec0803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:35.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.342+0000 7fec24487640 1 --2- 172.21.15.157:0/91858516 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fec080392a0 0x7fec0803b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0098c0 tx=0x7fec0c006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:35.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.342+0000 7fec24487640 1 -- 172.21.15.157:0/91858516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec1c0ff900 msgr2=0x7fec1c196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:35.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.342+0000 7fec24487640 1 --2- 172.21.15.157:0/91858516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec1c0ff900 0x7fec1c196db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec18004f60 tx=0x7fec180048f0 comp rx=0 tx=0).stop 2024-09-19T08:06:35.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.342+0000 7fec24487640 1 -- 172.21.15.157:0/91858516 shutdown_connections 2024-09-19T08:06:35.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.342+0000 7fec24487640 1 --2- 172.21.15.157:0/91858516 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fec080392a0 0x7fec0803b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:35.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.342+0000 7fec24487640 1 --2- 172.21.15.157:0/91858516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec1c0ff900 0x7fec1c196db0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:35.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.342+0000 7fec24487640 1 -- 172.21.15.157:0/91858516 >> 172.21.15.157:0/91858516 conn(0x7fec1c0fb450 msgr2=0x7fec1c0fbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:35.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.342+0000 7fec24487640 1 -- 172.21.15.157:0/91858516 shutdown_connections 2024-09-19T08:06:35.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:35.342+0000 7fec24487640 1 -- 172.21.15.157:0/91858516 wait complete. 2024-09-19T08:06:35.925 DEBUG:teuthology.orchestra.run.smithi064:mon.b> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T08:06:35.927 DEBUG:teuthology.orchestra.run.smithi157:mon.c> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T08:06:35.930 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T08:06:35.930 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T08:06:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:35 smithi022 ceph-mon[30963]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-19T08:06:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:35 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:35 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:35 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:35 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:35 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:06:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:35 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:36.578 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:06:36.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:36 smithi022 ceph-mon[30963]: Updating smithi157:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:06:36.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:36 smithi022 ceph-mon[30963]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:36.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:36 smithi022 ceph-mon[30963]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:06:36.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:36 smithi022 ceph-mon[30963]: Updating smithi157:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:06:36.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:36 smithi022 ceph-mon[30963]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi064:172.21.15.64=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:06:36.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:36 smithi022 ceph-mon[30963]: Saving service mon spec with placement smithi022:172.21.15.22=a;smithi064:172.21.15.64=b;smithi157:172.21.15.157=c;count:3 2024-09-19T08:06:36.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:36 smithi022 ceph-mon[30963]: Deploying daemon mon.c on smithi157 2024-09-19T08:06:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.975+0000 7ffad582b640 1 -- 172.21.15.157:0/3906804140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad0106540 msgr2=0x7ffad0106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.975+0000 7ffad582b640 1 --2- 172.21.15.157:0/3906804140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad0106540 0x7ffad0106920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009c10 tx=0x7ffac002aa10 comp rx=0 tx=0).stop 2024-09-19T08:06:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.975+0000 7ffad582b640 1 -- 172.21.15.157:0/3906804140 shutdown_connections 2024-09-19T08:06:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.975+0000 7ffad582b640 1 --2- 172.21.15.157:0/3906804140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad0106540 0x7ffad0106920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.975+0000 7ffad582b640 1 -- 172.21.15.157:0/3906804140 >> 172.21.15.157:0/3906804140 conn(0x7ffad0077310 msgr2=0x7ffad00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.975+0000 7ffad582b640 1 -- 172.21.15.157:0/3906804140 shutdown_connections 2024-09-19T08:06:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.975+0000 7ffad582b640 1 -- 172.21.15.157:0/3906804140 wait complete. 2024-09-19T08:06:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.975+0000 7ffad582b640 1 Processor -- start 2024-09-19T08:06:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.976+0000 7ffad582b640 1 -- start start 2024-09-19T08:06:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.976+0000 7ffad582b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad0106540 0x7ffad01093c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.976+0000 7ffad582b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffad010bc90 con 0x7ffad0106540 2024-09-19T08:06:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.976+0000 7ffaceffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad0106540 0x7ffad01093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.976+0000 7ffaceffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad0106540 0x7ffad01093c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.157:46036/0 (socket says 172.21.15.157:46036) 2024-09-19T08:06:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.976+0000 7ffaceffd640 1 -- 172.21.15.157:0/2008258880 learned_addr learned my addr 172.21.15.157:0/2008258880 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T08:06:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.976+0000 7ffaceffd640 1 -- 172.21.15.157:0/2008258880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad010b2b0 con 0x7ffad0106540 2024-09-19T08:06:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.977+0000 7ffaceffd640 1 --2- 172.21.15.157:0/2008258880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad0106540 0x7ffad01093c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffac0005dd0 tx=0x7ffac0030c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.977+0000 7ffad4829640 1 -- 172.21.15.157:0/2008258880 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffac0034070 con 0x7ffad0106540 2024-09-19T08:06:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.977+0000 7ffad582b640 1 -- 172.21.15.157:0/2008258880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad0109900 con 0x7ffad0106540 2024-09-19T08:06:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.977+0000 7ffad4829640 1 -- 172.21.15.157:0/2008258880 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac002f8c0 con 0x7ffad0106540 2024-09-19T08:06:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.977+0000 7ffad4829640 1 -- 172.21.15.157:0/2008258880 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffac0004520 con 0x7ffad0106540 2024-09-19T08:06:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.977+0000 7ffad582b640 1 -- 172.21.15.157:0/2008258880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad0109d00 con 0x7ffad0106540 2024-09-19T08:06:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.978+0000 7ffad4829640 1 -- 172.21.15.157:0/2008258880 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ffac0037020 con 0x7ffad0106540 2024-09-19T08:06:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.978+0000 7ffad582b640 1 -- 172.21.15.157:0/2008258880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad0101bb0 con 0x7ffad0106540 2024-09-19T08:06:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.978+0000 7ffad4829640 1 --2- 172.21.15.157:0/2008258880 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ffabc0392a0 0x7ffabc03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.978+0000 7ffad4829640 1 -- 172.21.15.157:0/2008258880 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffac006d2b0 con 0x7ffad0106540 2024-09-19T08:06:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.978+0000 7fface7fc640 1 --2- 172.21.15.157:0/2008258880 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ffabc0392a0 0x7ffabc03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.979+0000 7fface7fc640 1 --2- 172.21.15.157:0/2008258880 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ffabc0392a0 0x7ffabc03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffab80098c0 tx=0x7ffab8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:36.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:36.983+0000 7ffad4829640 1 -- 172.21.15.157:0/2008258880 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffac003f070 con 0x7ffad0106540 2024-09-19T08:06:37.217 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:37.217+0000 7ffad582b640 1 -- 172.21.15.157:0/2008258880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ffad010a8f0 con 0x7ffad0106540 2024-09-19T08:06:37.218 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:37.217+0000 7ffad4829640 1 -- 172.21.15.157:0/2008258880 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7ffac002c040 con 0x7ffad0106540 2024-09-19T08:06:37.218 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:06:37.218 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","modified":"2024-09-19T08:05:15.003832Z","created":"2024-09-19T08:05:15.003832Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T08:06:37.219 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-09-19T08:06:37.220 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:37.219+0000 7ffad582b640 1 -- 172.21.15.157:0/2008258880 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ffabc0392a0 msgr2=0x7ffabc03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:37.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:37.219+0000 7ffad582b640 1 --2- 172.21.15.157:0/2008258880 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ffabc0392a0 0x7ffabc03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffab80098c0 tx=0x7ffab8006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:37.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:37.219+0000 7ffad582b640 1 -- 172.21.15.157:0/2008258880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad0106540 msgr2=0x7ffad01093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:37.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:37.219+0000 7ffad582b640 1 --2- 172.21.15.157:0/2008258880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad0106540 0x7ffad01093c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffac0005dd0 tx=0x7ffac0030c30 comp rx=0 tx=0).stop 2024-09-19T08:06:37.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:37.219+0000 7ffad582b640 1 -- 172.21.15.157:0/2008258880 shutdown_connections 2024-09-19T08:06:37.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:37.219+0000 7ffad582b640 1 --2- 172.21.15.157:0/2008258880 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ffabc0392a0 0x7ffabc03b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:37.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:37.219+0000 7ffad582b640 1 --2- 172.21.15.157:0/2008258880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad0106540 0x7ffad01093c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:37.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:37.219+0000 7ffad582b640 1 -- 172.21.15.157:0/2008258880 >> 172.21.15.157:0/2008258880 conn(0x7ffad0077310 msgr2=0x7ffad00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:37.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:37.220+0000 7ffad582b640 1 -- 172.21.15.157:0/2008258880 shutdown_connections 2024-09-19T08:06:37.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:37.220+0000 7ffad582b640 1 -- 172.21.15.157:0/2008258880 wait complete. 2024-09-19T08:06:38.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:37 smithi022 ceph-mon[30963]: from='client.? 172.21.15.157:0/2008258880' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:06:39.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:38 smithi022 ceph-mon[30963]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:39.552 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T08:06:39.553 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T08:06:39.829 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:06:40.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.252+0000 7fdfa4678640 1 -- 172.21.15.157:0/2911816008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf9c106680 msgr2=0x7fdf9c106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:40.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.252+0000 7fdfa4678640 1 --2- 172.21.15.157:0/2911816008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf9c106680 0x7fdf9c106a60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009880 tx=0x7fdf9802a5d0 comp rx=0 tx=0).stop 2024-09-19T08:06:40.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.252+0000 7fdfa4678640 1 -- 172.21.15.157:0/2911816008 shutdown_connections 2024-09-19T08:06:40.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.252+0000 7fdfa4678640 1 --2- 172.21.15.157:0/2911816008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf9c106680 0x7fdf9c106a60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:40.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.252+0000 7fdfa4678640 1 -- 172.21.15.157:0/2911816008 >> 172.21.15.157:0/2911816008 conn(0x7fdf9c0fc290 msgr2=0x7fdf9c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:40.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.252+0000 7fdfa4678640 1 -- 172.21.15.157:0/2911816008 shutdown_connections 2024-09-19T08:06:40.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.252+0000 7fdfa4678640 1 -- 172.21.15.157:0/2911816008 wait complete. 2024-09-19T08:06:40.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.252+0000 7fdfa4678640 1 Processor -- start 2024-09-19T08:06:40.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.252+0000 7fdfa4678640 1 -- start start 2024-09-19T08:06:40.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.252+0000 7fdfa4678640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf9c106680 0x7fdf9c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:40.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.252+0000 7fdfa4678640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c10acb0 con 0x7fdf9c106680 2024-09-19T08:06:40.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.253+0000 7fdfa2414640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf9c106680 0x7fdf9c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:40.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.253+0000 7fdfa2414640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf9c106680 0x7fdf9c197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.157:46052/0 (socket says 172.21.15.157:46052) 2024-09-19T08:06:40.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.253+0000 7fdfa2414640 1 -- 172.21.15.157:0/1258075184 learned_addr learned my addr 172.21.15.157:0/1258075184 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T08:06:40.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.253+0000 7fdfa2414640 1 -- 172.21.15.157:0/1258075184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c1977c0 con 0x7fdf9c106680 2024-09-19T08:06:40.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.253+0000 7fdfa2414640 1 --2- 172.21.15.157:0/1258075184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf9c106680 0x7fdf9c197280 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009230 tx=0x7fdf98003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:40.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.253+0000 7fdf937fe640 1 -- 172.21.15.157:0/1258075184 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf9803f070 con 0x7fdf9c106680 2024-09-19T08:06:40.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.253+0000 7fdfa4678640 1 -- 172.21.15.157:0/1258075184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c197a50 con 0x7fdf9c106680 2024-09-19T08:06:40.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.254+0000 7fdf937fe640 1 -- 172.21.15.157:0/1258075184 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf9803bc80 con 0x7fdf9c106680 2024-09-19T08:06:40.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.254+0000 7fdf937fe640 1 -- 172.21.15.157:0/1258075184 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf98035030 con 0x7fdf9c106680 2024-09-19T08:06:40.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.254+0000 7fdfa4678640 1 -- 172.21.15.157:0/1258075184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c19bb50 con 0x7fdf9c106680 2024-09-19T08:06:40.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.254+0000 7fdfa4678640 1 -- 172.21.15.157:0/1258075184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9c101cf0 con 0x7fdf9c106680 2024-09-19T08:06:40.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.254+0000 7fdf937fe640 1 -- 172.21.15.157:0/1258075184 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fdf98042070 con 0x7fdf9c106680 2024-09-19T08:06:40.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.255+0000 7fdf937fe640 1 --2- 172.21.15.157:0/1258075184 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdf880392a0 0x7fdf8803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:40.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.255+0000 7fdf937fe640 1 -- 172.21.15.157:0/1258075184 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdf9806cff0 con 0x7fdf9c106680 2024-09-19T08:06:40.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.255+0000 7fdfa1c13640 1 --2- 172.21.15.157:0/1258075184 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdf880392a0 0x7fdf8803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:40.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.255+0000 7fdfa1c13640 1 --2- 172.21.15.157:0/1258075184 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdf880392a0 0x7fdf8803b760 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0098c0 tx=0x7fdf8c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:40.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.260+0000 7fdf937fe640 1 -- 172.21.15.157:0/1258075184 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdf980372d0 con 0x7fdf9c106680 2024-09-19T08:06:40.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.514+0000 7fdfa4678640 1 -- 172.21.15.157:0/1258075184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fdf9c106ae0 con 0x7fdf9c106680 2024-09-19T08:06:40.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.515+0000 7fdf937fe640 1 -- 172.21.15.157:0/1258075184 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fdf98031570 con 0x7fdf9c106680 2024-09-19T08:06:40.516 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:06:40.516 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","modified":"2024-09-19T08:05:15.003832Z","created":"2024-09-19T08:05:15.003832Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T08:06:40.516 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-09-19T08:06:40.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.517+0000 7fdfa4678640 1 -- 172.21.15.157:0/1258075184 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdf880392a0 msgr2=0x7fdf8803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:40.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.517+0000 7fdfa4678640 1 --2- 172.21.15.157:0/1258075184 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdf880392a0 0x7fdf8803b760 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0098c0 tx=0x7fdf8c006db0 comp rx=0 tx=0).stop 2024-09-19T08:06:40.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.517+0000 7fdfa4678640 1 -- 172.21.15.157:0/1258075184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf9c106680 msgr2=0x7fdf9c197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:40.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.517+0000 7fdfa4678640 1 --2- 172.21.15.157:0/1258075184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf9c106680 0x7fdf9c197280 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009230 tx=0x7fdf98003d40 comp rx=0 tx=0).stop 2024-09-19T08:06:40.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.517+0000 7fdfa4678640 1 -- 172.21.15.157:0/1258075184 shutdown_connections 2024-09-19T08:06:40.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.517+0000 7fdfa4678640 1 --2- 172.21.15.157:0/1258075184 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdf880392a0 0x7fdf8803b760 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:40.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.517+0000 7fdfa4678640 1 --2- 172.21.15.157:0/1258075184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf9c106680 0x7fdf9c197280 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:40.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.517+0000 7fdfa4678640 1 -- 172.21.15.157:0/1258075184 >> 172.21.15.157:0/1258075184 conn(0x7fdf9c0fc290 msgr2=0x7fdf9c06a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:40.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.517+0000 7fdfa4678640 1 -- 172.21.15.157:0/1258075184 shutdown_connections 2024-09-19T08:06:40.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:40.517+0000 7fdfa4678640 1 -- 172.21.15.157:0/1258075184 wait complete. 2024-09-19T08:06:41.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:40 smithi022 ceph-mon[30963]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:41.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:40 smithi022 ceph-mon[30963]: from='client.? 172.21.15.157:0/1258075184' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:06:41.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:41 smithi157 systemd[1]: Started Ceph mon.c for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:06:42.096 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T08:06:42.097 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T08:06:42.356 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:06:46.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.395+0000 7f8382df8640 1 -- 172.21.15.157:0/3225314878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f836c003530 msgr2=0x7f836c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:46.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.395+0000 7f8382df8640 1 --2- 172.21.15.157:0/3225314878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f836c003530 0x7f836c0059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f837c1016a0 tx=0x7f837002c3f0 comp rx=0 tx=0).stop 2024-09-19T08:06:46.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.395+0000 7f8382df8640 1 -- 172.21.15.157:0/3225314878 shutdown_connections 2024-09-19T08:06:46.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.395+0000 7f8382df8640 1 --2- 172.21.15.157:0/3225314878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f836c003530 0x7f836c0059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:46.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.395+0000 7f8382df8640 1 -- 172.21.15.157:0/3225314878 >> 172.21.15.157:0/3225314878 conn(0x7f837c0fc220 msgr2=0x7f837c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:46.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.395+0000 7f8382df8640 1 -- 172.21.15.157:0/3225314878 shutdown_connections 2024-09-19T08:06:46.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.395+0000 7f8382df8640 1 -- 172.21.15.157:0/3225314878 wait complete. 2024-09-19T08:06:46.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.396+0000 7f8382df8640 1 Processor -- start 2024-09-19T08:06:46.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.396+0000 7f8382df8640 1 -- start start 2024-09-19T08:06:46.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f8382df8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f836c003530 0x7f837c1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:46.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f8382df8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837c102730 0x7f837c102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:46.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f8382df8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f837c108670 con 0x7f836c003530 2024-09-19T08:06:46.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f8382df8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1084f0 con 0x7f837c102730 2024-09-19T08:06:46.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f837bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837c102730 0x7f837c102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:46.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f837bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837c102730 0x7f837c102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51820/0 (socket says 172.21.15.157:51820) 2024-09-19T08:06:46.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f837bfff640 1 -- 172.21.15.157:0/2229071613 learned_addr learned my addr 172.21.15.157:0/2229071613 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T08:06:46.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f8380b94640 1 --2- 172.21.15.157:0/2229071613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f836c003530 0x7f837c1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:46.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f837bfff640 1 -- 172.21.15.157:0/2229071613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837c102730 msgr2=0x7f837c102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T08:06:46.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f837bfff640 1 -- 172.21.15.157:0/2229071613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837c102730 msgr2=0x7f837c102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T08:06:46.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f837bfff640 1 --2- 172.21.15.157:0/2229071613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837c102730 0x7f837c102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:06:46.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f837bfff640 1 --2- 172.21.15.157:0/2229071613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837c102730 0x7f837c102b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:06:46.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f8380b94640 1 -- 172.21.15.157:0/2229071613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837c102730 msgr2=0x7f837c102b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:06:46.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f8380b94640 1 --2- 172.21.15.157:0/2229071613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837c102730 0x7f837c102b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:46.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f8380b94640 1 -- 172.21.15.157:0/2229071613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837c1040e0 con 0x7f836c003530 2024-09-19T08:06:46.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.397+0000 7f8380b94640 1 --2- 172.21.15.157:0/2229071613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f836c003530 0x7f837c1021f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8370036040 tx=0x7f8370034bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:46.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.398+0000 7f8379ffb640 1 -- 172.21.15.157:0/2229071613 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f83700051f0 con 0x7f836c003530 2024-09-19T08:06:46.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.398+0000 7f8379ffb640 1 -- 172.21.15.157:0/2229071613 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8370005390 con 0x7f836c003530 2024-09-19T08:06:46.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.398+0000 7f8382df8640 1 -- 172.21.15.157:0/2229071613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837c1a3440 con 0x7f836c003530 2024-09-19T08:06:46.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.398+0000 7f8379ffb640 1 -- 172.21.15.157:0/2229071613 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f83700128c0 con 0x7f836c003530 2024-09-19T08:06:46.400 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.398+0000 7f8382df8640 1 -- 172.21.15.157:0/2229071613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f837c1a39d0 con 0x7f836c003530 2024-09-19T08:06:46.401 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.399+0000 7f8379ffb640 1 -- 172.21.15.157:0/2229071613 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f8370012a60 con 0x7f836c003530 2024-09-19T08:06:46.401 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.399+0000 7f8382df8640 1 -- 172.21.15.157:0/2229071613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f837c101ca0 con 0x7f836c003530 2024-09-19T08:06:46.401 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.399+0000 7f8379ffb640 1 --2- 172.21.15.157:0/2229071613 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8360039090 0x7f836003b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:46.402 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.399+0000 7f8379ffb640 1 -- 172.21.15.157:0/2229071613 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f837006e550 con 0x7f836c003530 2024-09-19T08:06:46.402 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.400+0000 7f837bfff640 1 --2- 172.21.15.157:0/2229071613 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8360039090 0x7f836003b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:46.402 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.400+0000 7f837bfff640 1 --2- 172.21.15.157:0/2229071613 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8360039090 0x7f836003b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8368009100 tx=0x7f8368008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:46.404 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.404+0000 7f8379ffb640 1 -- 172.21.15.157:0/2229071613 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8370044b40 con 0x7f836c003530 2024-09-19T08:06:46.640 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.640+0000 7f8382df8640 1 -- 172.21.15.157:0/2229071613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f837c1a3cb0 con 0x7f836c003530 2024-09-19T08:06:46.640 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:46.640 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:06:46.640 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:46.641 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: mon.a calling monitor election 2024-09-19T08:06:46.641 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:46.641 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:46.641 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:46.641 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: mon.c calling monitor election 2024-09-19T08:06:46.641 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:46.641 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:46.641 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:46.641 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:46.641 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T08:06:46.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: monmap epoch 2 2024-09-19T08:06:46.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:46.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: last_changed 2024-09-19T08:06:41.294125+0000 2024-09-19T08:06:46.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: created 2024-09-19T08:05:15.003832+0000 2024-09-19T08:06:46.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: min_mon_release 19 (squid) 2024-09-19T08:06:46.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: election_strategy: 1 2024-09-19T08:06:46.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-09-19T08:06:46.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T08:06:46.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: fsmap 2024-09-19T08:06:46.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:06:46.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: mgrmap e13: x(active, since 39s) 2024-09-19T08:06:46.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: overall HEALTH_OK 2024-09-19T08:06:46.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:46.643 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:46.643 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:46.643 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:06:46.643 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:46 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:46.643 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.640+0000 7f8379ffb640 1 -- 172.21.15.157:0/2229071613 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f837003c3e0 con 0x7f836c003530 2024-09-19T08:06:46.643 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:06:46.643 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","modified":"2024-09-19T08:06:41.294125Z","created":"2024-09-19T08:05:15.003832Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T08:06:46.644 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 2 2024-09-19T08:06:46.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.643+0000 7f8382df8640 1 -- 172.21.15.157:0/2229071613 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8360039090 msgr2=0x7f836003b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:46.644 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.643+0000 7f8382df8640 1 --2- 172.21.15.157:0/2229071613 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8360039090 0x7f836003b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8368009100 tx=0x7f8368008040 comp rx=0 tx=0).stop 2024-09-19T08:06:46.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.643+0000 7f8382df8640 1 -- 172.21.15.157:0/2229071613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f836c003530 msgr2=0x7f837c1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:46.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.643+0000 7f8382df8640 1 --2- 172.21.15.157:0/2229071613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f836c003530 0x7f837c1021f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8370036040 tx=0x7f8370034bb0 comp rx=0 tx=0).stop 2024-09-19T08:06:46.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.643+0000 7f8382df8640 1 -- 172.21.15.157:0/2229071613 shutdown_connections 2024-09-19T08:06:46.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.643+0000 7f8382df8640 1 --2- 172.21.15.157:0/2229071613 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8360039090 0x7f836003b550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:46.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.643+0000 7f8382df8640 1 --2- 172.21.15.157:0/2229071613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837c102730 0x7f837c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:46.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.643+0000 7f8382df8640 1 --2- 172.21.15.157:0/2229071613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f836c003530 0x7f837c1021f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:46.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.643+0000 7f8382df8640 1 -- 172.21.15.157:0/2229071613 >> 172.21.15.157:0/2229071613 conn(0x7f837c0fc220 msgr2=0x7f837c06a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:46.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.643+0000 7f8382df8640 1 -- 172.21.15.157:0/2229071613 shutdown_connections 2024-09-19T08:06:46.645 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:46.643+0000 7f8382df8640 1 -- 172.21.15.157:0/2229071613 wait complete. 2024-09-19T08:06:46.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:46.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:06:46.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:46.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: mon.a calling monitor election 2024-09-19T08:06:46.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:46.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:46.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:46.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: mon.c calling monitor election 2024-09-19T08:06:46.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:46.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:46.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:46.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:46.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T08:06:46.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: monmap epoch 2 2024-09-19T08:06:46.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:46.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: last_changed 2024-09-19T08:06:41.294125+0000 2024-09-19T08:06:46.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: created 2024-09-19T08:05:15.003832+0000 2024-09-19T08:06:46.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: min_mon_release 19 (squid) 2024-09-19T08:06:46.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: election_strategy: 1 2024-09-19T08:06:46.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-09-19T08:06:46.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T08:06:46.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: fsmap 2024-09-19T08:06:46.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:06:46.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: mgrmap e13: x(active, since 39s) 2024-09-19T08:06:46.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: overall HEALTH_OK 2024-09-19T08:06:46.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:46.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:46.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:46.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:06:46.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:46 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:47.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:47 smithi157 ceph-mon[35143]: Deploying daemon mon.b on smithi064 2024-09-19T08:06:47.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:47 smithi157 ceph-mon[35143]: from='client.? 172.21.15.157:0/2229071613' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:06:47.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:47 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:47.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:47 smithi022 ceph-mon[30963]: Deploying daemon mon.b on smithi064 2024-09-19T08:06:47.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:47 smithi022 ceph-mon[30963]: from='client.? 172.21.15.157:0/2229071613' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:06:47.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:47 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:48.261 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T08:06:48.262 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T08:06:48.518 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:06:48.624 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:48 smithi157 ceph-mon[35143]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:48.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:48 smithi022 ceph-mon[30963]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:48.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.982+0000 7f69208ad640 1 -- 172.21.15.157:0/3681663222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c106630 msgr2=0x7f691c106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:48.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.982+0000 7f69208ad640 1 --2- 172.21.15.157:0/3681663222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c106630 0x7f691c106a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f690c009880 tx=0x7f690c02a5d0 comp rx=0 tx=0).stop 2024-09-19T08:06:48.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.982+0000 7f69208ad640 1 -- 172.21.15.157:0/3681663222 shutdown_connections 2024-09-19T08:06:48.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.982+0000 7f69208ad640 1 --2- 172.21.15.157:0/3681663222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c106630 0x7f691c106a10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:48.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.982+0000 7f69208ad640 1 -- 172.21.15.157:0/3681663222 >> 172.21.15.157:0/3681663222 conn(0x7f691c0fc220 msgr2=0x7f691c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:48.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.982+0000 7f69208ad640 1 -- 172.21.15.157:0/3681663222 shutdown_connections 2024-09-19T08:06:48.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.982+0000 7f69208ad640 1 -- 172.21.15.157:0/3681663222 wait complete. 2024-09-19T08:06:48.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.983+0000 7f69208ad640 1 Processor -- start 2024-09-19T08:06:48.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.983+0000 7f69208ad640 1 -- start start 2024-09-19T08:06:48.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.983+0000 7f69208ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c106630 0x7f691c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:48.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.983+0000 7f69208ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c199ae0 0x7f691c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:48.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.983+0000 7f69208ad640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f691c109b70 con 0x7f691c106630 2024-09-19T08:06:48.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.983+0000 7f69208ad640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1099f0 con 0x7f691c199ae0 2024-09-19T08:06:48.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.983+0000 7f6913fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c199ae0 0x7f691c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:48.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.983+0000 7f6913fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c199ae0 0x7f691c19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51834/0 (socket says 172.21.15.157:51834) 2024-09-19T08:06:48.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.983+0000 7f6913fff640 1 -- 172.21.15.157:0/2442881227 learned_addr learned my addr 172.21.15.157:0/2442881227 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T08:06:48.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.983+0000 7f6913fff640 1 -- 172.21.15.157:0/2442881227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c199ae0 msgr2=0x7f691c19de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T08:06:48.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.983+0000 7f6913fff640 1 -- 172.21.15.157:0/2442881227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c199ae0 msgr2=0x7f691c19de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T08:06:48.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.983+0000 7f6913fff640 1 --2- 172.21.15.157:0/2442881227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c199ae0 0x7f691c19de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:06:48.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.983+0000 7f6913fff640 1 --2- 172.21.15.157:0/2442881227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c199ae0 0x7f691c19de70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:06:48.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.983+0000 7f691a59c640 1 --2- 172.21.15.157:0/2442881227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c106630 0x7f691c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:48.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.984+0000 7f691a59c640 1 -- 172.21.15.157:0/2442881227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c199ae0 msgr2=0x7f691c19de70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:06:48.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.984+0000 7f691a59c640 1 --2- 172.21.15.157:0/2442881227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c199ae0 0x7f691c19de70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:48.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.984+0000 7f691a59c640 1 -- 172.21.15.157:0/2442881227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c19a100 con 0x7f691c106630 2024-09-19T08:06:48.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.984+0000 7f691a59c640 1 --2- 172.21.15.157:0/2442881227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c106630 0x7f691c1995a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f690c005ab0 tx=0x7f690c032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:48.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.984+0000 7f69137fe640 1 -- 172.21.15.157:0/2442881227 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f690c035070 con 0x7f691c106630 2024-09-19T08:06:48.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.984+0000 7f69137fe640 1 -- 172.21.15.157:0/2442881227 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f690c032e40 con 0x7f691c106630 2024-09-19T08:06:48.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.984+0000 7f69208ad640 1 -- 172.21.15.157:0/2442881227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c19e3b0 con 0x7f691c106630 2024-09-19T08:06:48.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.984+0000 7f69137fe640 1 -- 172.21.15.157:0/2442881227 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f690c03bc10 con 0x7f691c106630 2024-09-19T08:06:48.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.984+0000 7f69208ad640 1 -- 172.21.15.157:0/2442881227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c19e910 con 0x7f691c106630 2024-09-19T08:06:48.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.985+0000 7f69137fe640 1 -- 172.21.15.157:0/2442881227 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f690c042030 con 0x7f691c106630 2024-09-19T08:06:48.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.985+0000 7f69208ad640 1 -- 172.21.15.157:0/2442881227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691c101ca0 con 0x7f691c106630 2024-09-19T08:06:48.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.985+0000 7f69137fe640 1 --2- 172.21.15.157:0/2442881227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6908039040 0x7f690803b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:48.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.985+0000 7f69137fe640 1 -- 172.21.15.157:0/2442881227 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f690c040070 con 0x7f691c106630 2024-09-19T08:06:48.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.986+0000 7f6913fff640 1 --2- 172.21.15.157:0/2442881227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6908039040 0x7f690803b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:48.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.986+0000 7f6913fff640 1 --2- 172.21.15.157:0/2442881227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6908039040 0x7f690803b500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6904009100 tx=0x7f6904008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:48.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:48.991+0000 7f69137fe640 1 -- 172.21.15.157:0/2442881227 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f690c03b510 con 0x7f691c106630 2024-09-19T08:06:49.219 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:49.219+0000 7f69208ad640 1 -- 172.21.15.157:0/2442881227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f691c0008d0 con 0x7f691c106630 2024-09-19T08:06:49.219 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:49.219+0000 7f69137fe640 1 -- 172.21.15.157:0/2442881227 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f690c03bdb0 con 0x7f691c106630 2024-09-19T08:06:49.220 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:06:49.220 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","modified":"2024-09-19T08:06:41.294125Z","created":"2024-09-19T08:05:15.003832Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T08:06:49.220 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 2 2024-09-19T08:06:49.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:49.221+0000 7f69208ad640 1 -- 172.21.15.157:0/2442881227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6908039040 msgr2=0x7f690803b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:49.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:49.221+0000 7f69208ad640 1 --2- 172.21.15.157:0/2442881227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6908039040 0x7f690803b500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6904009100 tx=0x7f6904008040 comp rx=0 tx=0).stop 2024-09-19T08:06:49.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:49.221+0000 7f69208ad640 1 -- 172.21.15.157:0/2442881227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c106630 msgr2=0x7f691c1995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:49.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:49.221+0000 7f69208ad640 1 --2- 172.21.15.157:0/2442881227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c106630 0x7f691c1995a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f690c005ab0 tx=0x7f690c032c50 comp rx=0 tx=0).stop 2024-09-19T08:06:49.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:49.222+0000 7f69208ad640 1 -- 172.21.15.157:0/2442881227 shutdown_connections 2024-09-19T08:06:49.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:49.222+0000 7f69208ad640 1 --2- 172.21.15.157:0/2442881227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6908039040 0x7f690803b500 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:49.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:49.222+0000 7f69208ad640 1 --2- 172.21.15.157:0/2442881227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c199ae0 0x7f691c19de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:49.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:49.222+0000 7f69208ad640 1 --2- 172.21.15.157:0/2442881227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f691c106630 0x7f691c1995a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:49.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:49.222+0000 7f69208ad640 1 -- 172.21.15.157:0/2442881227 >> 172.21.15.157:0/2442881227 conn(0x7f691c0fc220 msgr2=0x7f691c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:49.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:49.222+0000 7f69208ad640 1 -- 172.21.15.157:0/2442881227 shutdown_connections 2024-09-19T08:06:49.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:49.222+0000 7f69208ad640 1 -- 172.21.15.157:0/2442881227 wait complete. 2024-09-19T08:06:49.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:49 smithi157 ceph-mon[35143]: from='client.? 172.21.15.157:0/2442881227' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:06:49.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:49 smithi022 ceph-mon[30963]: from='client.? 172.21.15.157:0/2442881227' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:06:49.944 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:49 smithi064 podman[34881]: 2024-09-19 08:06:49.738224652 +0000 UTC m=+0.015772912 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:06:49.944 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:49 smithi064 podman[34881]: 2024-09-19 08:06:49.944479695 +0000 UTC m=+0.222027946 container create a7fa335b0bdd1fffcf20f9bdbd3902b2481410b5b1d060d2b7a4172705d129ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T08:06:50.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 podman[34881]: 2024-09-19 08:06:50.130440744 +0000 UTC m=+0.407988997 container init a7fa335b0bdd1fffcf20f9bdbd3902b2481410b5b1d060d2b7a4172705d129ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T08:06:50.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 podman[34881]: 2024-09-19 08:06:50.133723998 +0000 UTC m=+0.411272254 container start a7fa335b0bdd1fffcf20f9bdbd3902b2481410b5b1d060d2b7a4172705d129ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-b, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T08:06:50.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T08:06:50.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T08:06:50.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: pidfile_write: ignore empty --pid-file 2024-09-19T08:06:50.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: load: jerasure load: lrc 2024-09-19T08:06:50.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: RocksDB version: 7.9.2 2024-09-19T08:06:50.205 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Git sha 0 2024-09-19T08:06:50.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T08:06:50.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: DB SUMMARY 2024-09-19T08:06:50.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: DB Session ID: HOWNGP4J0EPE6DRKUAU4 2024-09-19T08:06:50.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: CURRENT file: CURRENT 2024-09-19T08:06:50.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T08:06:50.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T08:06:50.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-19T08:06:50.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-19T08:06:50.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.error_if_exists: 0 2024-09-19T08:06:50.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.create_if_missing: 0 2024-09-19T08:06:50.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.paranoid_checks: 1 2024-09-19T08:06:50.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T08:06:50.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T08:06:50.206 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T08:06:50.207 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.env: 0x55785b6e4940 2024-09-19T08:06:50.207 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T08:06:50.207 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.info_log: 0x55785c0256e0 2024-09-19T08:06:50.207 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T08:06:50.208 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.statistics: (nil) 2024-09-19T08:06:50.208 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.use_fsync: 0 2024-09-19T08:06:50.208 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_log_file_size: 0 2024-09-19T08:06:50.208 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T08:06:50.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T08:06:50.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T08:06:50.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T08:06:50.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.allow_fallocate: 1 2024-09-19T08:06:50.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T08:06:50.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T08:06:50.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.use_direct_reads: 0 2024-09-19T08:06:50.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T08:06:50.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T08:06:50.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.db_log_dir: 2024-09-19T08:06:50.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.wal_dir: 2024-09-19T08:06:50.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T08:06:50.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T08:06:50.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T08:06:50.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T08:06:50.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T08:06:50.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T08:06:50.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T08:06:50.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T08:06:50.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.write_buffer_manager: 0x55785c034dc0 2024-09-19T08:06:50.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T08:06:50.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T08:06:50.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T08:06:50.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T08:06:50.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T08:06:50.210 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T08:06:50.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T08:06:50.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T08:06:50.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.unordered_write: 0 2024-09-19T08:06:50.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T08:06:50.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T08:06:50.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T08:06:50.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T08:06:50.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.row_cache: None 2024-09-19T08:06:50.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.wal_filter: None 2024-09-19T08:06:50.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T08:06:50.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T08:06:50.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.two_write_queues: 0 2024-09-19T08:06:50.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T08:06:50.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.wal_compression: 0 2024-09-19T08:06:50.212 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.atomic_flush: 0 2024-09-19T08:06:50.212 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T08:06:50.212 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T08:06:50.212 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T08:06:50.212 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.log_readahead_size: 0 2024-09-19T08:06:50.212 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T08:06:50.212 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T08:06:50.212 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T08:06:50.212 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T08:06:50.212 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T08:06:50.212 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T08:06:50.212 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T08:06:50.212 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_background_jobs: 2 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_background_compactions: -1 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_subcompactions: 1 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_open_files: -1 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T08:06:50.213 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T08:06:50.214 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_background_flushes: -1 2024-09-19T08:06:50.214 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Compression algorithms supported: 2024-09-19T08:06:50.214 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: kZSTD supported: 0 2024-09-19T08:06:50.214 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: kXpressCompression supported: 0 2024-09-19T08:06:50.214 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: kBZip2Compression supported: 0 2024-09-19T08:06:50.214 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T08:06:50.214 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: kLZ4Compression supported: 1 2024-09-19T08:06:50.214 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: kZlibCompression supported: 1 2024-09-19T08:06:50.214 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T08:06:50.214 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: kSnappyCompression supported: 1 2024-09-19T08:06:50.214 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T08:06:50.214 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T08:06:50.214 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-19T08:06:50.214 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T08:06:50.215 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T08:06:50.215 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.merge_operator: 2024-09-19T08:06:50.215 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compaction_filter: None 2024-09-19T08:06:50.215 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T08:06:50.215 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T08:06:50.215 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T08:06:50.215 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T08:06:50.215 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55785c025320) 2024-09-19T08:06:50.215 INFO:journalctl@ceph.mon.b.smithi064.stdout: cache_index_and_filter_blocks: 1 2024-09-19T08:06:50.215 INFO:journalctl@ceph.mon.b.smithi064.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T08:06:50.215 INFO:journalctl@ceph.mon.b.smithi064.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T08:06:50.215 INFO:journalctl@ceph.mon.b.smithi064.stdout: pin_top_level_index_and_filter: 1 2024-09-19T08:06:50.215 INFO:journalctl@ceph.mon.b.smithi064.stdout: index_type: 0 2024-09-19T08:06:50.215 INFO:journalctl@ceph.mon.b.smithi064.stdout: data_block_index_type: 0 2024-09-19T08:06:50.216 INFO:journalctl@ceph.mon.b.smithi064.stdout: index_shortening: 1 2024-09-19T08:06:50.216 INFO:journalctl@ceph.mon.b.smithi064.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T08:06:50.216 INFO:journalctl@ceph.mon.b.smithi064.stdout: checksum: 4 2024-09-19T08:06:50.216 INFO:journalctl@ceph.mon.b.smithi064.stdout: no_block_cache: 0 2024-09-19T08:06:50.216 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_cache: 0x55785c0191f0 2024-09-19T08:06:50.216 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_cache_name: BinnedLRUCache 2024-09-19T08:06:50.216 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_cache_options: 2024-09-19T08:06:50.216 INFO:journalctl@ceph.mon.b.smithi064.stdout: capacity : 536870912 2024-09-19T08:06:50.216 INFO:journalctl@ceph.mon.b.smithi064.stdout: num_shard_bits : 4 2024-09-19T08:06:50.216 INFO:journalctl@ceph.mon.b.smithi064.stdout: strict_capacity_limit : 0 2024-09-19T08:06:50.216 INFO:journalctl@ceph.mon.b.smithi064.stdout: high_pri_pool_ratio: 0.000 2024-09-19T08:06:50.216 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_cache_compressed: (nil) 2024-09-19T08:06:50.216 INFO:journalctl@ceph.mon.b.smithi064.stdout: persistent_cache: (nil) 2024-09-19T08:06:50.217 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_size: 4096 2024-09-19T08:06:50.217 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_size_deviation: 10 2024-09-19T08:06:50.217 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_restart_interval: 16 2024-09-19T08:06:50.217 INFO:journalctl@ceph.mon.b.smithi064.stdout: index_block_restart_interval: 1 2024-09-19T08:06:50.217 INFO:journalctl@ceph.mon.b.smithi064.stdout: metadata_block_size: 4096 2024-09-19T08:06:50.217 INFO:journalctl@ceph.mon.b.smithi064.stdout: partition_filters: 0 2024-09-19T08:06:50.217 INFO:journalctl@ceph.mon.b.smithi064.stdout: use_delta_encoding: 1 2024-09-19T08:06:50.217 INFO:journalctl@ceph.mon.b.smithi064.stdout: filter_policy: bloomfilter 2024-09-19T08:06:50.217 INFO:journalctl@ceph.mon.b.smithi064.stdout: whole_key_filtering: 1 2024-09-19T08:06:50.217 INFO:journalctl@ceph.mon.b.smithi064.stdout: verify_compression: 0 2024-09-19T08:06:50.217 INFO:journalctl@ceph.mon.b.smithi064.stdout: read_amp_bytes_per_bit: 0 2024-09-19T08:06:50.217 INFO:journalctl@ceph.mon.b.smithi064.stdout: format_version: 5 2024-09-19T08:06:50.217 INFO:journalctl@ceph.mon.b.smithi064.stdout: enable_index_compression: 1 2024-09-19T08:06:50.217 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_align: 0 2024-09-19T08:06:50.218 INFO:journalctl@ceph.mon.b.smithi064.stdout: max_auto_readahead_size: 262144 2024-09-19T08:06:50.218 INFO:journalctl@ceph.mon.b.smithi064.stdout: prepopulate_block_cache: 0 2024-09-19T08:06:50.218 INFO:journalctl@ceph.mon.b.smithi064.stdout: initial_auto_readahead_size: 8192 2024-09-19T08:06:50.218 INFO:journalctl@ceph.mon.b.smithi064.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T08:06:50.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T08:06:50.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T08:06:50.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compression: NoCompression 2024-09-19T08:06:50.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T08:06:50.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T08:06:50.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T08:06:50.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.num_levels: 7 2024-09-19T08:06:50.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T08:06:50.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T08:06:50.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T08:06:50.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T08:06:50.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T08:06:50.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T08:06:50.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T08:06:50.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T08:06:50.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T08:06:50.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T08:06:50.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T08:06:50.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T08:06:50.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T08:06:50.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T08:06:50.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T08:06:50.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T08:06:50.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T08:06:50.222 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T08:06:50.222 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T08:06:50.222 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T08:06:50.222 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T08:06:50.222 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T08:06:50.222 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T08:06:50.222 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T08:06:50.222 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T08:06:50.222 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T08:06:50.222 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T08:06:50.222 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T08:06:50.222 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.inplace_update_support: 0 2024-09-19T08:06:50.222 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T08:06:50.222 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T08:06:50.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T08:06:50.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T08:06:50.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.bloom_locality: 0 2024-09-19T08:06:50.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.max_successive_merges: 0 2024-09-19T08:06:50.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T08:06:50.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T08:06:50.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T08:06:50.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T08:06:50.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.ttl: 2592000 2024-09-19T08:06:50.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T08:06:50.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T08:06:50.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T08:06:50.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.enable_blob_files: false 2024-09-19T08:06:50.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.min_blob_size: 0 2024-09-19T08:06:50.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T08:06:50.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T08:06:50.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T08:06:50.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T08:06:50.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T08:06:50.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T08:06:50.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T08:06:50.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T08:06:50.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T08:06:50.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T08:06:50.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9b14d37f-789b-47b1-92bd-f9042e0f42d9 2024-09-19T08:06:50.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726733210161222, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T08:06:50.225 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T08:06:50.225 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726733210161621, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726733210, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9b14d37f-789b-47b1-92bd-f9042e0f42d9", "db_session_id": "HOWNGP4J0EPE6DRKUAU4", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T08:06:50.225 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726733210161727, "job": 1, "event": "recovery_finished"} 2024-09-19T08:06:50.225 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T08:06:50.225 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T08:06:50.225 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55785c046e00 2024-09-19T08:06:50.225 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: DB pointer 0x55785c142000 2024-09-19T08:06:50.225 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T08:06:50.225 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T08:06:50.225 INFO:journalctl@ceph.mon.b.smithi064.stdout: ** DB Stats ** 2024-09-19T08:06:50.225 INFO:journalctl@ceph.mon.b.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T08:06:50.225 INFO:journalctl@ceph.mon.b.smithi064.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T08:06:50.225 INFO:journalctl@ceph.mon.b.smithi064.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T08:06:50.225 INFO:journalctl@ceph.mon.b.smithi064.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T08:06:50.226 INFO:journalctl@ceph.mon.b.smithi064.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T08:06:50.226 INFO:journalctl@ceph.mon.b.smithi064.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T08:06:50.226 INFO:journalctl@ceph.mon.b.smithi064.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T08:06:50.226 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2024-09-19T08:06:50.226 INFO:journalctl@ceph.mon.b.smithi064.stdout: ** Compaction Stats [default] ** 2024-09-19T08:06:50.226 INFO:journalctl@ceph.mon.b.smithi064.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T08:06:50.226 INFO:journalctl@ceph.mon.b.smithi064.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T08:06:50.226 INFO:journalctl@ceph.mon.b.smithi064.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T08:06:50.226 INFO:journalctl@ceph.mon.b.smithi064.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T08:06:50.226 INFO:journalctl@ceph.mon.b.smithi064.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T08:06:50.226 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2024-09-19T08:06:50.226 INFO:journalctl@ceph.mon.b.smithi064.stdout: ** Compaction Stats [default] ** 2024-09-19T08:06:50.226 INFO:journalctl@ceph.mon.b.smithi064.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T08:06:50.226 INFO:journalctl@ceph.mon.b.smithi064.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T08:06:50.227 INFO:journalctl@ceph.mon.b.smithi064.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T08:06:50.227 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2024-09-19T08:06:50.227 INFO:journalctl@ceph.mon.b.smithi064.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T08:06:50.227 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2024-09-19T08:06:50.227 INFO:journalctl@ceph.mon.b.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T08:06:50.227 INFO:journalctl@ceph.mon.b.smithi064.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T08:06:50.227 INFO:journalctl@ceph.mon.b.smithi064.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T08:06:50.227 INFO:journalctl@ceph.mon.b.smithi064.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T08:06:50.227 INFO:journalctl@ceph.mon.b.smithi064.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T08:06:50.227 INFO:journalctl@ceph.mon.b.smithi064.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T08:06:50.227 INFO:journalctl@ceph.mon.b.smithi064.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T08:06:50.227 INFO:journalctl@ceph.mon.b.smithi064.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T08:06:50.227 INFO:journalctl@ceph.mon.b.smithi064.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T08:06:50.227 INFO:journalctl@ceph.mon.b.smithi064.stdout: Block cache BinnedLRUCache@0x55785c0191f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T08:06:50.228 INFO:journalctl@ceph.mon.b.smithi064.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T08:06:50.228 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2024-09-19T08:06:50.228 INFO:journalctl@ceph.mon.b.smithi064.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T08:06:50.228 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-19T08:06:50.228 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: using public_addr v2:172.21.15.64:0/0 -> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] 2024-09-19T08:06:50.228 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: starting mon.b rank -1 at public addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] at bind addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:50.228 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b@-1(???) e0 preinit fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:50.228 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b@-1(synchronizing).mds e1 new map 2024-09-19T08:06:50.228 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-19T08:06:50.228 INFO:journalctl@ceph.mon.b.smithi064.stdout: e1 2024-09-19T08:06:50.228 INFO:journalctl@ceph.mon.b.smithi064.stdout: btime 2024-09-19T08:05:19:476769+0000 2024-09-19T08:06:50.228 INFO:journalctl@ceph.mon.b.smithi064.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T08:06:50.228 INFO:journalctl@ceph.mon.b.smithi064.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T08:06:50.229 INFO:journalctl@ceph.mon.b.smithi064.stdout: legacy client fscid: -1 2024-09-19T08:06:50.229 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2024-09-19T08:06:50.229 INFO:journalctl@ceph.mon.b.smithi064.stdout: No filesystems configured 2024-09-19T08:06:50.229 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T08:06:50.229 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T08:06:50.229 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T08:06:50.229 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T08:06:50.229 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T08:06:50.229 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T08:06:50.229 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T08:06:50.229 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:06:50.229 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:06:50.229 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:06:50.229 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:50.230 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:50.230 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:06:50.230 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1956329726' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T08:06:50.230 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:50.230 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1956329726' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T08:06:50.230 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:06:50.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:50.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:50.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:50.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:50.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:06:50.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:50.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:06:50.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-19T08:06:50.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:50.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:50.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:50.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:50.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:06:50.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:50.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: Updating smithi157:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:06:50.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:50.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:06:50.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: Updating smithi157:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:06:50.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi064:172.21.15.64=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:06:50.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: Saving service mon spec with placement smithi022:172.21.15.22=a;smithi064:172.21.15.64=b;smithi157:172.21.15.157=c;count:3 2024-09-19T08:06:50.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: Deploying daemon mon.c on smithi157 2024-09-19T08:06:50.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='client.? 172.21.15.157:0/2008258880' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:06:50.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:50.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:50.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='client.? 172.21.15.157:0/1258075184' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:06:50.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:50.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:06:50.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:50.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.a calling monitor election 2024-09-19T08:06:50.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:50.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:50.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:50.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.c calling monitor election 2024-09-19T08:06:50.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:50.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:50.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:50.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:50.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T08:06:50.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: monmap epoch 2 2024-09-19T08:06:50.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:50.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: last_changed 2024-09-19T08:06:41.294125+0000 2024-09-19T08:06:50.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: created 2024-09-19T08:05:15.003832+0000 2024-09-19T08:06:50.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: min_mon_release 19 (squid) 2024-09-19T08:06:50.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: election_strategy: 1 2024-09-19T08:06:50.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-09-19T08:06:50.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T08:06:50.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: fsmap 2024-09-19T08:06:50.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:06:50.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mgrmap e13: x(active, since 39s) 2024-09-19T08:06:50.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: overall HEALTH_OK 2024-09-19T08:06:50.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:50.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:50.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:50.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:06:50.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:50.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: Deploying daemon mon.b on smithi064 2024-09-19T08:06:50.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='client.? 172.21.15.157:0/2229071613' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:06:50.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:50.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:50.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: from='client.? 172.21.15.157:0/2442881227' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:06:50.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 ceph-mon[34900]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T08:06:50.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 bash[34881]: a7fa335b0bdd1fffcf20f9bdbd3902b2481410b5b1d060d2b7a4172705d129ae 2024-09-19T08:06:50.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:50 smithi064 systemd[1]: Started Ceph mon.b for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:06:50.749 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T08:06:50.750 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T08:06:50.998 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:06:55.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:06:55.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: mon.a calling monitor election 2024-09-19T08:06:55.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: mon.c calling monitor election 2024-09-19T08:06:55.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:55.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:55.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:55.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:55.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:55.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:55.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:55.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:55.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:55.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T08:06:55.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: monmap epoch 3 2024-09-19T08:06:55.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:55.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: last_changed 2024-09-19T08:06:50.186796+0000 2024-09-19T08:06:55.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: created 2024-09-19T08:05:15.003832+0000 2024-09-19T08:06:55.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: min_mon_release 19 (squid) 2024-09-19T08:06:55.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: election_strategy: 1 2024-09-19T08:06:55.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-09-19T08:06:55.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T08:06:55.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: 2: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.b 2024-09-19T08:06:55.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: fsmap 2024-09-19T08:06:55.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:06:55.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: mgrmap e13: x(active, since 48s) 2024-09-19T08:06:55.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: overall HEALTH_OK 2024-09-19T08:06:55.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:55.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:55.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:55.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:55 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:55.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:06:55.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: mon.a calling monitor election 2024-09-19T08:06:55.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: mon.c calling monitor election 2024-09-19T08:06:55.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:55.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:55.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:55.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:55.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:55.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:55.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:55.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:55.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:55.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T08:06:55.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: monmap epoch 3 2024-09-19T08:06:55.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:55.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: last_changed 2024-09-19T08:06:50.186796+0000 2024-09-19T08:06:55.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: created 2024-09-19T08:05:15.003832+0000 2024-09-19T08:06:55.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: min_mon_release 19 (squid) 2024-09-19T08:06:55.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: election_strategy: 1 2024-09-19T08:06:55.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-09-19T08:06:55.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T08:06:55.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: 2: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.b 2024-09-19T08:06:55.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: fsmap 2024-09-19T08:06:55.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:06:55.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: mgrmap e13: x(active, since 48s) 2024-09-19T08:06:55.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: overall HEALTH_OK 2024-09-19T08:06:55.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:55.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:55.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:55.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:55 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:55.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.871+0000 7f439c67d640 1 -- 172.21.15.157:0/3193438789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4380003530 msgr2=0x7f43800059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:55.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.871+0000 7f439c67d640 1 --2- 172.21.15.157:0/3193438789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4380003530 0x7f43800059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4394102af0 tx=0x7f438802c490 comp rx=0 tx=0).stop 2024-09-19T08:06:55.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.871+0000 7f439c67d640 1 -- 172.21.15.157:0/3193438789 shutdown_connections 2024-09-19T08:06:55.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.871+0000 7f439c67d640 1 --2- 172.21.15.157:0/3193438789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4380003530 0x7f43800059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:55.873 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.871+0000 7f439c67d640 1 -- 172.21.15.157:0/3193438789 >> 172.21.15.157:0/3193438789 conn(0x7f43940fd320 msgr2=0x7f43940ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:55.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.871+0000 7f439c67d640 1 -- 172.21.15.157:0/3193438789 shutdown_connections 2024-09-19T08:06:55.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.872+0000 7f439c67d640 1 -- 172.21.15.157:0/3193438789 wait complete. 2024-09-19T08:06:55.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.872+0000 7f439c67d640 1 Processor -- start 2024-09-19T08:06:55.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.872+0000 7f439c67d640 1 -- start start 2024-09-19T08:06:55.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.872+0000 7f439c67d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380003530 0x7f4394197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:55.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.872+0000 7f439c67d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43941978b0 0x7f439419bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:55.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.872+0000 7f439c67d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439419c180 0x7f439419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:55.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.872+0000 7f439c67d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4394109720 con 0x7f43941978b0 2024-09-19T08:06:55.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.872+0000 7f439c67d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43941095a0 con 0x7f439419c180 2024-09-19T08:06:55.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.872+0000 7f439c67d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f43941098a0 con 0x7f4380003530 2024-09-19T08:06:55.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f439a419640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380003530 0x7f4394197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:55.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f439a419640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380003530 0x7f4394197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51850/0 (socket says 172.21.15.157:51850) 2024-09-19T08:06:55.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f439a419640 1 -- 172.21.15.157:0/2254944681 learned_addr learned my addr 172.21.15.157:0/2254944681 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T08:06:55.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f439a419640 1 -- 172.21.15.157:0/2254944681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380003530 msgr2=0x7f4394197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T08:06:55.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f439a419640 1 -- 172.21.15.157:0/2254944681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380003530 msgr2=0x7f4394197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T08:06:55.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f439a419640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380003530 0x7f4394197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:06:55.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f439a419640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380003530 0x7f4394197370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:06:55.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f4399c18640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43941978b0 0x7f439419bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:55.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f439ac1a640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439419c180 0x7f439419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:55.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f439ac1a640 1 -- 172.21.15.157:0/2254944681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439419c180 msgr2=0x7f439419c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-19T08:06:55.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f439ac1a640 1 -- 172.21.15.157:0/2254944681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439419c180 msgr2=0x7f439419c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T08:06:55.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f439ac1a640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439419c180 0x7f439419c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:06:55.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f439ac1a640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439419c180 0x7f439419c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:06:55.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f4399c18640 1 -- 172.21.15.157:0/2254944681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380003530 msgr2=0x7f4394197370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:06:55.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f4399c18640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380003530 0x7f4394197370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:55.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f4399c18640 1 -- 172.21.15.157:0/2254944681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439419c180 msgr2=0x7f439419c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:06:55.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f4399c18640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439419c180 0x7f439419c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:55.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f4399c18640 1 -- 172.21.15.157:0/2254944681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43941a3ef0 con 0x7f43941978b0 2024-09-19T08:06:55.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.873+0000 7f4399c18640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43941978b0 0x7f439419bc40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f439000b450 tx=0x7f439000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:55.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.874+0000 7f43877fe640 1 -- 172.21.15.157:0/2254944681 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390004230 con 0x7f43941978b0 2024-09-19T08:06:55.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.874+0000 7f439c67d640 1 -- 172.21.15.157:0/2254944681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43941a4120 con 0x7f43941978b0 2024-09-19T08:06:55.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.874+0000 7f43877fe640 1 -- 172.21.15.157:0/2254944681 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43900043d0 con 0x7f43941978b0 2024-09-19T08:06:55.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.874+0000 7f43877fe640 1 -- 172.21.15.157:0/2254944681 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390003cb0 con 0x7f43941978b0 2024-09-19T08:06:55.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.874+0000 7f439c67d640 1 -- 172.21.15.157:0/2254944681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43941a4630 con 0x7f43941978b0 2024-09-19T08:06:55.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.875+0000 7f439c67d640 1 -- 172.21.15.157:0/2254944681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4394102da0 con 0x7f43941978b0 2024-09-19T08:06:55.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.875+0000 7f43877fe640 1 -- 172.21.15.157:0/2254944681 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f439000f050 con 0x7f43941978b0 2024-09-19T08:06:55.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.875+0000 7f43877fe640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4378039730 0x7f437803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:55.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.875+0000 7f43877fe640 1 -- 172.21.15.157:0/2254944681 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f439004c100 con 0x7f43941978b0 2024-09-19T08:06:55.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.875+0000 7f439a419640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4378039730 0x7f437803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:55.878 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.876+0000 7f439a419640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4378039730 0x7f437803bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f438802ca00 tx=0x7f438802c990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:55.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:55.880+0000 7f43877fe640 1 -- 172.21.15.157:0/2254944681 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f439000d040 con 0x7f43941978b0 2024-09-19T08:06:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:56.116+0000 7f439c67d640 1 -- 172.21.15.157:0/2254944681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f43941a49c0 con 0x7f43941978b0 2024-09-19T08:06:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:56.117+0000 7f43877fe640 1 -- 172.21.15.157:0/2254944681 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f439000f5a0 con 0x7f43941978b0 2024-09-19T08:06:56.118 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:06:56.118 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":3,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","modified":"2024-09-19T08:06:50.186796Z","created":"2024-09-19T08:05:15.003832Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T08:06:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 3 2024-09-19T08:06:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:56.119+0000 7f439c67d640 1 -- 172.21.15.157:0/2254944681 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4378039730 msgr2=0x7f437803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:56.119+0000 7f439c67d640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4378039730 0x7f437803bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f438802ca00 tx=0x7f438802c990 comp rx=0 tx=0).stop 2024-09-19T08:06:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:56.119+0000 7f439c67d640 1 -- 172.21.15.157:0/2254944681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43941978b0 msgr2=0x7f439419bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:56.119+0000 7f439c67d640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43941978b0 0x7f439419bc40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f439000b450 tx=0x7f439000b910 comp rx=0 tx=0).stop 2024-09-19T08:06:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:56.120+0000 7f439c67d640 1 -- 172.21.15.157:0/2254944681 shutdown_connections 2024-09-19T08:06:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:56.120+0000 7f439c67d640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4378039730 0x7f437803bbf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:56.120+0000 7f439c67d640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439419c180 0x7f439419c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:56.120+0000 7f439c67d640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43941978b0 0x7f439419bc40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:56.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:56.120+0000 7f439c67d640 1 --2- 172.21.15.157:0/2254944681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380003530 0x7f4394197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:56.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:56.120+0000 7f439c67d640 1 -- 172.21.15.157:0/2254944681 >> 172.21.15.157:0/2254944681 conn(0x7f43940fd320 msgr2=0x7f43940ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:56.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:56.120+0000 7f439c67d640 1 -- 172.21.15.157:0/2254944681 shutdown_connections 2024-09-19T08:06:56.122 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:06:56.120+0000 7f439c67d640 1 -- 172.21.15.157:0/2254944681 wait complete. 2024-09-19T08:06:56.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:56 smithi157 ceph-mon[35143]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:56.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:56 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:06:56.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:56 smithi157 ceph-mon[35143]: from='client.? 172.21.15.157:0/2254944681' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:06:56.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:56 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:56.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:56 smithi022 ceph-mon[30963]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:56.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:56 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:06:56.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:56 smithi022 ceph-mon[30963]: from='client.? 172.21.15.157:0/2254944681' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:06:56.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:56 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:57.042 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T08:06:57.043 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T08:06:57.312 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:06:57.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:06:57.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: mon.a calling monitor election 2024-09-19T08:06:57.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: mon.c calling monitor election 2024-09-19T08:06:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:06:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T08:06:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: monmap epoch 3 2024-09-19T08:06:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:57.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: last_changed 2024-09-19T08:06:50.186796+0000 2024-09-19T08:06:57.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: created 2024-09-19T08:05:15.003832+0000 2024-09-19T08:06:57.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: min_mon_release 19 (squid) 2024-09-19T08:06:57.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: election_strategy: 1 2024-09-19T08:06:57.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-09-19T08:06:57.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T08:06:57.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: 2: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.b 2024-09-19T08:06:57.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: fsmap 2024-09-19T08:06:57.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:06:57.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: mgrmap e13: x(active, since 48s) 2024-09-19T08:06:57.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: overall HEALTH_OK 2024-09-19T08:06:57.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:57.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:57.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:57.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:57.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:57.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:06:57.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='client.? 172.21.15.157:0/2254944681' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:06:57.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: mon.b calling monitor election 2024-09-19T08:06:58.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:58.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T08:06:58.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: Reconfiguring daemon mon.a on smithi022 2024-09-19T08:06:58.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: mon.b calling monitor election 2024-09-19T08:06:58.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: mon.c calling monitor election 2024-09-19T08:06:58.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: mon.a calling monitor election 2024-09-19T08:06:58.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T08:06:58.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: monmap epoch 3 2024-09-19T08:06:58.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:58.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: last_changed 2024-09-19T08:06:50.186796+0000 2024-09-19T08:06:58.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: created 2024-09-19T08:05:15.003832+0000 2024-09-19T08:06:58.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: min_mon_release 19 (squid) 2024-09-19T08:06:58.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: election_strategy: 1 2024-09-19T08:06:58.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-09-19T08:06:58.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T08:06:58.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: 2: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.b 2024-09-19T08:06:58.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: fsmap 2024-09-19T08:06:58.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:06:58.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: mgrmap e13: x(active, since 50s) 2024-09-19T08:06:58.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:58 smithi022 ceph-mon[30963]: overall HEALTH_OK 2024-09-19T08:06:58.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: mon.b calling monitor election 2024-09-19T08:06:58.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:58.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T08:06:58.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: Reconfiguring daemon mon.a on smithi022 2024-09-19T08:06:58.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: mon.b calling monitor election 2024-09-19T08:06:58.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: mon.c calling monitor election 2024-09-19T08:06:58.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: mon.a calling monitor election 2024-09-19T08:06:58.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T08:06:58.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: monmap epoch 3 2024-09-19T08:06:58.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:58.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: last_changed 2024-09-19T08:06:50.186796+0000 2024-09-19T08:06:58.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: created 2024-09-19T08:05:15.003832+0000 2024-09-19T08:06:58.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: min_mon_release 19 (squid) 2024-09-19T08:06:58.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: election_strategy: 1 2024-09-19T08:06:58.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-09-19T08:06:58.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T08:06:58.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: 2: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.b 2024-09-19T08:06:58.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: fsmap 2024-09-19T08:06:58.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:06:58.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: mgrmap e13: x(active, since 50s) 2024-09-19T08:06:58.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:58 smithi064 ceph-mon[34900]: overall HEALTH_OK 2024-09-19T08:06:58.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.365+0000 7fb36aacb640 1 -- 172.21.15.22:0/77813389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb364106560 msgr2=0x7fb364106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:58.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.365+0000 7fb36aacb640 1 --2- 172.21.15.22:0/77813389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb364106560 0x7fb364106940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb360009880 tx=0x7fb36002a5d0 comp rx=0 tx=0).stop 2024-09-19T08:06:58.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.365+0000 7fb36aacb640 1 -- 172.21.15.22:0/77813389 shutdown_connections 2024-09-19T08:06:58.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.365+0000 7fb36aacb640 1 --2- 172.21.15.22:0/77813389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb364106560 0x7fb364106940 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:58.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.365+0000 7fb36aacb640 1 -- 172.21.15.22:0/77813389 >> 172.21.15.22:0/77813389 conn(0x7fb3640fc1b0 msgr2=0x7fb3640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.365+0000 7fb36aacb640 1 -- 172.21.15.22:0/77813389 shutdown_connections 2024-09-19T08:06:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.365+0000 7fb36aacb640 1 -- 172.21.15.22:0/77813389 wait complete. 2024-09-19T08:06:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.366+0000 7fb36aacb640 1 Processor -- start 2024-09-19T08:06:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.366+0000 7fb36aacb640 1 -- start start 2024-09-19T08:06:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.366+0000 7fb36aacb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb364106560 0x7fb364197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.366+0000 7fb36aacb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb364197840 0x7fb36419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.366+0000 7fb36aacb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb36419c110 0x7fb36419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.366+0000 7fb36aacb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb36410ab40 con 0x7fb364106560 2024-09-19T08:06:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.366+0000 7fb36aacb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb36410a9c0 con 0x7fb36419c110 2024-09-19T08:06:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.366+0000 7fb36aacb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb36410acc0 con 0x7fb364197840 2024-09-19T08:06:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.366+0000 7fb368867640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb364106560 0x7fb364197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.366+0000 7fb368867640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb364106560 0x7fb364197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36486/0 (socket says 172.21.15.22:36486) 2024-09-19T08:06:58.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.366+0000 7fb368867640 1 -- 172.21.15.22:0/2862167327 learned_addr learned my addr 172.21.15.22:0/2862167327 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:06:58.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.367+0000 7fb35bfff640 1 --2- 172.21.15.22:0/2862167327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb364197840 0x7fb36419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:58.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.367+0000 7fb368867640 1 -- 172.21.15.22:0/2862167327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb364197840 msgr2=0x7fb36419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:58.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.367+0000 7fb368867640 1 --2- 172.21.15.22:0/2862167327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb364197840 0x7fb36419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:58.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.367+0000 7fb368867640 1 -- 172.21.15.22:0/2862167327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb36419c110 msgr2=0x7fb36419c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:58.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.367+0000 7fb368867640 1 --2- 172.21.15.22:0/2862167327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb36419c110 0x7fb36419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:58.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.367+0000 7fb368867640 1 -- 172.21.15.22:0/2862167327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3641a3e80 con 0x7fb364106560 2024-09-19T08:06:58.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.367+0000 7fb368867640 1 --2- 172.21.15.22:0/2862167327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb364106560 0x7fb364197300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb360031550 tx=0x7fb3600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:58.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.367+0000 7fb359ffb640 1 -- 172.21.15.22:0/2862167327 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36003f030 con 0x7fb364106560 2024-09-19T08:06:58.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.367+0000 7fb359ffb640 1 -- 172.21.15.22:0/2862167327 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb360003ce0 con 0x7fb364106560 2024-09-19T08:06:58.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.367+0000 7fb36aacb640 1 -- 172.21.15.22:0/2862167327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3641a4050 con 0x7fb364106560 2024-09-19T08:06:58.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.367+0000 7fb359ffb640 1 -- 172.21.15.22:0/2862167327 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360035070 con 0x7fb364106560 2024-09-19T08:06:58.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.367+0000 7fb36aacb640 1 -- 172.21.15.22:0/2862167327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3641a45d0 con 0x7fb364106560 2024-09-19T08:06:58.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.368+0000 7fb359ffb640 1 -- 172.21.15.22:0/2862167327 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fb360042030 con 0x7fb364106560 2024-09-19T08:06:58.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.368+0000 7fb359ffb640 1 --2- 172.21.15.22:0/2862167327 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb34c039730 0x7fb34c03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:06:58.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.368+0000 7fb359ffb640 1 -- 172.21.15.22:0/2862167327 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb36006d040 con 0x7fb364106560 2024-09-19T08:06:58.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.368+0000 7fb35bfff640 1 --2- 172.21.15.22:0/2862167327 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb34c039730 0x7fb34c03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:06:58.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.368+0000 7fb36aacb640 1 -- 172.21.15.22:0/2862167327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3641069c0 con 0x7fb364106560 2024-09-19T08:06:58.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.369+0000 7fb35bfff640 1 --2- 172.21.15.22:0/2862167327 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb34c039730 0x7fb34c03bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb354006fd0 tx=0x7fb354008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:06:58.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.373+0000 7fb359ffb640 1 -- 172.21.15.22:0/2862167327 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb36004dc30 con 0x7fb364106560 2024-09-19T08:06:58.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: mon.b calling monitor election 2024-09-19T08:06:58.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:06:58.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T08:06:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: Reconfiguring daemon mon.a on smithi022 2024-09-19T08:06:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: mon.b calling monitor election 2024-09-19T08:06:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: mon.c calling monitor election 2024-09-19T08:06:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: mon.a calling monitor election 2024-09-19T08:06:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T08:06:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: monmap epoch 3 2024-09-19T08:06:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: last_changed 2024-09-19T08:06:50.186796+0000 2024-09-19T08:06:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: created 2024-09-19T08:05:15.003832+0000 2024-09-19T08:06:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: min_mon_release 19 (squid) 2024-09-19T08:06:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: election_strategy: 1 2024-09-19T08:06:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-09-19T08:06:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-09-19T08:06:58.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: 2: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.b 2024-09-19T08:06:58.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: fsmap 2024-09-19T08:06:58.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:06:58.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: mgrmap e13: x(active, since 50s) 2024-09-19T08:06:58.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:58 smithi157 ceph-mon[35143]: overall HEALTH_OK 2024-09-19T08:06:58.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.551+0000 7fb36aacb640 1 -- 172.21.15.22:0/2862167327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb3641a4800 con 0x7fb364106560 2024-09-19T08:06:58.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.551+0000 7fb359ffb640 1 -- 172.21.15.22:0/2862167327 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fb36003bce0 con 0x7fb364106560 2024-09-19T08:06:58.552 INFO:teuthology.orchestra.run.smithi022.stdout:# minimal ceph.conf for c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:58.552 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-09-19T08:06:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:06:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: mon_host = [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-09-19T08:06:58.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.554+0000 7fb36aacb640 1 -- 172.21.15.22:0/2862167327 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb34c039730 msgr2=0x7fb34c03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:58.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.554+0000 7fb36aacb640 1 --2- 172.21.15.22:0/2862167327 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb34c039730 0x7fb34c03bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb354006fd0 tx=0x7fb354008040 comp rx=0 tx=0).stop 2024-09-19T08:06:58.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.554+0000 7fb36aacb640 1 -- 172.21.15.22:0/2862167327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb364106560 msgr2=0x7fb364197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:06:58.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.554+0000 7fb36aacb640 1 --2- 172.21.15.22:0/2862167327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb364106560 0x7fb364197300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb360031550 tx=0x7fb3600042f0 comp rx=0 tx=0).stop 2024-09-19T08:06:58.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.554+0000 7fb36aacb640 1 -- 172.21.15.22:0/2862167327 shutdown_connections 2024-09-19T08:06:58.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.554+0000 7fb36aacb640 1 --2- 172.21.15.22:0/2862167327 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb34c039730 0x7fb34c03bbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:58.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.554+0000 7fb36aacb640 1 --2- 172.21.15.22:0/2862167327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb36419c110 0x7fb36419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:58.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.554+0000 7fb36aacb640 1 --2- 172.21.15.22:0/2862167327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb364197840 0x7fb36419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:58.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.554+0000 7fb36aacb640 1 --2- 172.21.15.22:0/2862167327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb364106560 0x7fb364197300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:06:58.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.554+0000 7fb36aacb640 1 -- 172.21.15.22:0/2862167327 >> 172.21.15.22:0/2862167327 conn(0x7fb3640fc1b0 msgr2=0x7fb364104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:06:58.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.554+0000 7fb36aacb640 1 -- 172.21.15.22:0/2862167327 shutdown_connections 2024-09-19T08:06:58.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:06:58.554+0000 7fb36aacb640 1 -- 172.21.15.22:0/2862167327 wait complete. 2024-09-19T08:06:59.171 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:59 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:59.171 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:59 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2862167327' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:59.171 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:59 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:59.171 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:59 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:59.171 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:59 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:06:59.171 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:59 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:06:59.171 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:06:59 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:59.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:59 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:59.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:59 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2862167327' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:59.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:59 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:59.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:59 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:59.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:59 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:06:59.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:59 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:06:59.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:06:59 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:59.454 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T08:06:59.454 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:06:59.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T08:06:59.485 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:06:59.485 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:06:59.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:59 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:06:59.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:59 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2862167327' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:59.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:59 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:59.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:59 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:06:59.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:59 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:06:59.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:59 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:06:59.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:06:59 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:06:59.552 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T08:06:59.552 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T08:06:59.579 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T08:06:59.580 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:06:59.644 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T08:06:59.644 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T08:06:59.677 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T08:06:59.677 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:06:59.747 INFO:tasks.cephadm:Adding mgr.x on smithi022 2024-09-19T08:06:59.747 INFO:tasks.cephadm:Adding mgr.y on smithi064 2024-09-19T08:06:59.747 INFO:tasks.cephadm:Adding mgr.z on smithi157 2024-09-19T08:06:59.747 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi022=x;smithi064=y;smithi157=z' 2024-09-19T08:07:00.031 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:07:00.271 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:00 smithi157 ceph-mon[35143]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:00.271 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:00 smithi157 ceph-mon[35143]: Reconfiguring mon.b (monmap changed)... 2024-09-19T08:07:00.271 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:00 smithi157 ceph-mon[35143]: Reconfiguring daemon mon.b on smithi064 2024-09-19T08:07:00.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:00 smithi064 ceph-mon[34900]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:00.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:00 smithi064 ceph-mon[34900]: Reconfiguring mon.b (monmap changed)... 2024-09-19T08:07:00.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:00 smithi064 ceph-mon[34900]: Reconfiguring daemon mon.b on smithi064 2024-09-19T08:07:00.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.478+0000 7ff51b5e4640 1 -- 172.21.15.157:0/667412716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff514102180 msgr2=0x7ff514104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:00.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.478+0000 7ff51b5e4640 1 --2- 172.21.15.157:0/667412716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff514102180 0x7ff514104570 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009880 tx=0x7ff50c02a5d0 comp rx=0 tx=0).stop 2024-09-19T08:07:00.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.478+0000 7ff51b5e4640 1 -- 172.21.15.157:0/667412716 shutdown_connections 2024-09-19T08:07:00.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.478+0000 7ff51b5e4640 1 --2- 172.21.15.157:0/667412716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff514102180 0x7ff514104570 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:00.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.478+0000 7ff51b5e4640 1 -- 172.21.15.157:0/667412716 >> 172.21.15.157:0/667412716 conn(0x7ff5140fbda0 msgr2=0x7ff5140fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:07:00.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.479+0000 7ff51b5e4640 1 -- 172.21.15.157:0/667412716 shutdown_connections 2024-09-19T08:07:00.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.479+0000 7ff51b5e4640 1 -- 172.21.15.157:0/667412716 wait complete. 2024-09-19T08:07:00.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.479+0000 7ff51b5e4640 1 Processor -- start 2024-09-19T08:07:00.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.479+0000 7ff51b5e4640 1 -- start start 2024-09-19T08:07:00.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.479+0000 7ff51b5e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff514102180 0x7ff514199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:00.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.479+0000 7ff51b5e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff514199680 0x7ff51419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:00.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.479+0000 7ff51b5e4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff514199b00 0x7ff5141a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:00.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.479+0000 7ff51b5e4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff514107740 con 0x7ff514199b00 2024-09-19T08:07:00.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.479+0000 7ff51b5e4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141075c0 con 0x7ff514199680 2024-09-19T08:07:00.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.479+0000 7ff51b5e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141078c0 con 0x7ff514102180 2024-09-19T08:07:00.503 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.479+0000 7ff519380640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff514102180 0x7ff514199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:07:00.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.479+0000 7ff519380640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff514102180 0x7ff514199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45104/0 (socket says 172.21.15.157:45104) 2024-09-19T08:07:00.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.479+0000 7ff519380640 1 -- 172.21.15.157:0/4291925747 learned_addr learned my addr 172.21.15.157:0/4291925747 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T08:07:00.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.480+0000 7ff519b81640 1 --2- 172.21.15.157:0/4291925747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff514199b00 0x7ff5141a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:07:00.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.480+0000 7ff518b7f640 1 --2- 172.21.15.157:0/4291925747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff514199680 0x7ff51419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:07:00.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.480+0000 7ff519380640 1 -- 172.21.15.157:0/4291925747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff514199680 msgr2=0x7ff51419e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:00.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.480+0000 7ff519380640 1 --2- 172.21.15.157:0/4291925747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff514199680 0x7ff51419e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:00.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.480+0000 7ff519380640 1 -- 172.21.15.157:0/4291925747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff514199b00 msgr2=0x7ff5141a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:00.504 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.480+0000 7ff519380640 1 --2- 172.21.15.157:0/4291925747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff514199b00 0x7ff5141a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:00.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.480+0000 7ff519380640 1 -- 172.21.15.157:0/4291925747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5141a1540 con 0x7ff514102180 2024-09-19T08:07:00.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.480+0000 7ff519380640 1 --2- 172.21.15.157:0/4291925747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff514102180 0x7ff514199140 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff50c000c00 tx=0x7ff50c004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:07:00.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.480+0000 7ff5127fc640 1 -- 172.21.15.157:0/4291925747 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c03f030 con 0x7ff514102180 2024-09-19T08:07:00.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.480+0000 7ff5127fc640 1 -- 172.21.15.157:0/4291925747 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff50c003ce0 con 0x7ff514102180 2024-09-19T08:07:00.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.480+0000 7ff51b5e4640 1 -- 172.21.15.157:0/4291925747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5141a17d0 con 0x7ff514102180 2024-09-19T08:07:00.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.480+0000 7ff5127fc640 1 -- 172.21.15.157:0/4291925747 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c035070 con 0x7ff514102180 2024-09-19T08:07:00.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.480+0000 7ff51b5e4640 1 -- 172.21.15.157:0/4291925747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5141a1d90 con 0x7ff514102180 2024-09-19T08:07:00.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.481+0000 7ff5127fc640 1 -- 172.21.15.157:0/4291925747 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ff50c042030 con 0x7ff514102180 2024-09-19T08:07:00.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.481+0000 7ff51b5e4640 1 -- 172.21.15.157:0/4291925747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff514192900 con 0x7ff514102180 2024-09-19T08:07:00.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.481+0000 7ff5127fc640 1 --2- 172.21.15.157:0/4291925747 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff4fc039730 0x7ff4fc03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:00.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.481+0000 7ff5127fc640 1 -- 172.21.15.157:0/4291925747 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff50c06cf40 con 0x7ff514102180 2024-09-19T08:07:00.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.482+0000 7ff518b7f640 1 --2- 172.21.15.157:0/4291925747 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff4fc039730 0x7ff4fc03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:07:00.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.482+0000 7ff518b7f640 1 --2- 172.21.15.157:0/4291925747 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff4fc039730 0x7ff4fc03bbf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff51419a660 tx=0x7ff50800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:07:00.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.486+0000 7ff5127fc640 1 -- 172.21.15.157:0/4291925747 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff50c04c350 con 0x7ff514102180 2024-09-19T08:07:00.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:00 smithi022 ceph-mon[30963]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:00.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:00 smithi022 ceph-mon[30963]: Reconfiguring mon.b (monmap changed)... 2024-09-19T08:07:00.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:00 smithi022 ceph-mon[30963]: Reconfiguring daemon mon.b on smithi064 2024-09-19T08:07:00.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.673+0000 7ff51b5e4640 1 -- 172.21.15.157:0/4291925747 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi064=y;smithi157=z", "target": ["mon-mgr", ""]}) -- 0x7ff5140626c0 con 0x7ff4fc039730 2024-09-19T08:07:00.679 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.679+0000 7ff5127fc640 1 -- 172.21.15.157:0/4291925747 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff5140626c0 con 0x7ff4fc039730 2024-09-19T08:07:00.680 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mgr update... 2024-09-19T08:07:00.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.682+0000 7ff51b5e4640 1 -- 172.21.15.157:0/4291925747 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff4fc039730 msgr2=0x7ff4fc03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:00.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.682+0000 7ff51b5e4640 1 --2- 172.21.15.157:0/4291925747 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff4fc039730 0x7ff4fc03bbf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff51419a660 tx=0x7ff50800b040 comp rx=0 tx=0).stop 2024-09-19T08:07:00.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.682+0000 7ff51b5e4640 1 -- 172.21.15.157:0/4291925747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff514102180 msgr2=0x7ff514199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:00.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.682+0000 7ff51b5e4640 1 --2- 172.21.15.157:0/4291925747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff514102180 0x7ff514199140 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff50c000c00 tx=0x7ff50c004430 comp rx=0 tx=0).stop 2024-09-19T08:07:00.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.682+0000 7ff51b5e4640 1 -- 172.21.15.157:0/4291925747 shutdown_connections 2024-09-19T08:07:00.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.682+0000 7ff51b5e4640 1 --2- 172.21.15.157:0/4291925747 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff4fc039730 0x7ff4fc03bbf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:00.682 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.682+0000 7ff51b5e4640 1 --2- 172.21.15.157:0/4291925747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff514199b00 0x7ff5141a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:00.683 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.682+0000 7ff51b5e4640 1 --2- 172.21.15.157:0/4291925747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff514199680 0x7ff51419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:00.683 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.682+0000 7ff51b5e4640 1 --2- 172.21.15.157:0/4291925747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff514102180 0x7ff514199140 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:00.683 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.682+0000 7ff51b5e4640 1 -- 172.21.15.157:0/4291925747 >> 172.21.15.157:0/4291925747 conn(0x7ff5140fbda0 msgr2=0x7ff5140fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:07:00.683 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.682+0000 7ff51b5e4640 1 -- 172.21.15.157:0/4291925747 shutdown_connections 2024-09-19T08:07:00.683 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:00.682+0000 7ff51b5e4640 1 -- 172.21.15.157:0/4291925747 wait complete. 2024-09-19T08:07:01.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:01 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:01.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:01 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:01.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:01 smithi157 ceph-mon[35143]: Reconfiguring mon.c (monmap changed)... 2024-09-19T08:07:01.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:01 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:07:01.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:01 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:07:01.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:01 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:01.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:01 smithi157 ceph-mon[35143]: Reconfiguring daemon mon.c on smithi157 2024-09-19T08:07:01.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:01 smithi157 ceph-mon[35143]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi064=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:07:01.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:01 smithi157 ceph-mon[35143]: Saving service mgr spec with placement smithi022=x;smithi064=y;smithi157=z;count:3 2024-09-19T08:07:01.460 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:01 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:01.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:01 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:01.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:01 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:01.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:01 smithi022 ceph-mon[30963]: Reconfiguring mon.c (monmap changed)... 2024-09-19T08:07:01.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:01 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:07:01.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:01 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:07:01.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:01 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:01.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:01 smithi022 ceph-mon[30963]: Reconfiguring daemon mon.c on smithi157 2024-09-19T08:07:01.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:01 smithi022 ceph-mon[30963]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi064=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:07:01.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:01 smithi022 ceph-mon[30963]: Saving service mgr spec with placement smithi022=x;smithi064=y;smithi157=z;count:3 2024-09-19T08:07:01.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:01 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:01.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:01 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:01.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:01 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:01.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:01 smithi064 ceph-mon[34900]: Reconfiguring mon.c (monmap changed)... 2024-09-19T08:07:01.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:01 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:07:01.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:01 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:07:01.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:01 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:01.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:01 smithi064 ceph-mon[34900]: Reconfiguring daemon mon.c on smithi157 2024-09-19T08:07:01.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:01 smithi064 ceph-mon[34900]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi064=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:07:01.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:01 smithi064 ceph-mon[34900]: Saving service mgr spec with placement smithi022=x;smithi064=y;smithi157=z;count:3 2024-09-19T08:07:01.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:01 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:01.754 DEBUG:teuthology.orchestra.run.smithi064:mgr.y> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T08:07:01.757 DEBUG:teuthology.orchestra.run.smithi157:mgr.z> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T08:07:01.759 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T08:07:01.760 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:07:01.760 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T08:07:01.780 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T08:07:01.781 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2024-09-19T08:07:01.839 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T08:07:01.840 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:07:01.840 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-19T08:07:01.840 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:07:01.840 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:07:01.840 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-19 08:06:17.429300576 +0000 2024-09-19T08:07:01.840 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-19 08:01:43.515042359 +0000 2024-09-19T08:07:01.840 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-19 08:01:43.515042359 +0000 2024-09-19T08:07:01.840 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-19 08:01:43.515042359 +0000 2024-09-19T08:07:01.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T08:07:01.908 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-19T08:07:01.908 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-19T08:07:01.908 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000131249 s, 3.9 MB/s 2024-09-19T08:07:01.909 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T08:07:01.969 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2024-09-19T08:07:02.027 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T08:07:02.028 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:07:02.028 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-19T08:07:02.028 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:07:02.028 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:07:02.028 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-19 08:06:17.455300209 +0000 2024-09-19T08:07:02.028 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-19 08:01:43.837038043 +0000 2024-09-19T08:07:02.028 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-19 08:01:43.837038043 +0000 2024-09-19T08:07:02.028 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-19 08:01:43.837038043 +0000 2024-09-19T08:07:02.028 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T08:07:02.094 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-19T08:07:02.095 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-19T08:07:02.095 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000110444 s, 4.6 MB/s 2024-09-19T08:07:02.096 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T08:07:02.155 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2024-09-19T08:07:02.213 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T08:07:02.213 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:07:02.213 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-19T08:07:02.213 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:07:02.213 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:07:02.213 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-19 08:06:17.482299829 +0000 2024-09-19T08:07:02.214 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-19 08:01:44.150033849 +0000 2024-09-19T08:07:02.214 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-19 08:01:44.150033849 +0000 2024-09-19T08:07:02.214 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-19 08:01:44.150033849 +0000 2024-09-19T08:07:02.214 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T08:07:02.279 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-19T08:07:02.279 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-19T08:07:02.280 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000114046 s, 4.5 MB/s 2024-09-19T08:07:02.281 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T08:07:02.338 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2024-09-19T08:07:02.395 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T08:07:02.395 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:07:02.395 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-19T08:07:02.395 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:07:02.395 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:07:02.395 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-19 08:06:17.513299392 +0000 2024-09-19T08:07:02.395 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-19 08:01:44.487029332 +0000 2024-09-19T08:07:02.396 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-19 08:01:44.487029332 +0000 2024-09-19T08:07:02.396 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-19 08:01:44.487029332 +0000 2024-09-19T08:07:02.396 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T08:07:02.458 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:02 smithi022 ceph-mon[30963]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:02.458 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:02 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:02.460 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-19T08:07:02.460 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-19T08:07:02.460 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000106581 s, 4.8 MB/s 2024-09-19T08:07:02.462 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T08:07:02.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:02 smithi157 ceph-mon[35143]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:02.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:02 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:02.519 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T08:07:02.519 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T08:07:02.538 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T08:07:02.538 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-09-19T08:07:02.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:02 smithi064 ceph-mon[34900]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:02.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:02 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:02.597 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T08:07:02.597 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:07:02.597 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-19T08:07:02.597 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:07:02.597 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:07:02.597 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-19 08:06:49.550803538 +0000 2024-09-19T08:07:02.597 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-19 08:02:10.606638702 +0000 2024-09-19T08:07:02.598 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-19 08:02:10.606638702 +0000 2024-09-19T08:07:02.598 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-19 08:02:10.606638702 +0000 2024-09-19T08:07:02.598 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T08:07:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-19T08:07:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-19T08:07:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000268794 s, 1.9 MB/s 2024-09-19T08:07:02.666 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T08:07:02.725 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-09-19T08:07:02.781 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T08:07:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:07:02.782 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-19T08:07:02.782 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:07:02.782 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:07:02.782 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-19 08:06:49.551803518 +0000 2024-09-19T08:07:02.782 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-19 08:02:10.945631984 +0000 2024-09-19T08:07:02.782 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-19 08:02:10.945631984 +0000 2024-09-19T08:07:02.782 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-19 08:02:10.945631984 +0000 2024-09-19T08:07:02.782 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T08:07:02.847 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-19T08:07:02.847 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-19T08:07:02.847 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000218685 s, 2.3 MB/s 2024-09-19T08:07:02.848 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T08:07:02.905 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-09-19T08:07:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T08:07:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:07:02.962 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-19T08:07:02.962 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:07:02.962 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:07:02.962 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-19 08:06:49.551803518 +0000 2024-09-19T08:07:02.962 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-19 08:02:11.258625782 +0000 2024-09-19T08:07:02.962 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-19 08:02:11.258625782 +0000 2024-09-19T08:07:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-19 08:02:11.258625782 +0000 2024-09-19T08:07:02.962 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T08:07:03.026 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-19T08:07:03.026 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-19T08:07:03.026 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000233014 s, 2.2 MB/s 2024-09-19T08:07:03.027 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T08:07:03.085 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-09-19T08:07:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T08:07:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:07:03.140 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-19T08:07:03.141 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:07:03.141 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:07:03.141 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-19 08:06:49.551803518 +0000 2024-09-19T08:07:03.141 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-19 08:02:11.586619282 +0000 2024-09-19T08:07:03.141 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-19 08:02:11.586619282 +0000 2024-09-19T08:07:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-19 08:02:11.586619282 +0000 2024-09-19T08:07:03.141 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T08:07:03.205 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-19T08:07:03.205 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-19T08:07:03.205 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000237028 s, 2.2 MB/s 2024-09-19T08:07:03.206 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T08:07:03.263 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-19T08:07:03.263 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T08:07:03.279 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T08:07:03.279 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-09-19T08:07:03.335 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T08:07:03.335 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:07:03.335 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T08:07:03.335 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:07:03.335 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:07:03.335 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-19 08:06:39.986644805 +0000 2024-09-19T08:07:03.335 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-19 08:01:57.572894762 +0000 2024-09-19T08:07:03.335 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-19 08:01:57.572894762 +0000 2024-09-19T08:07:03.336 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-19 08:01:57.572894762 +0000 2024-09-19T08:07:03.336 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T08:07:03.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:03 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:03.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:03 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:03.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:03 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:03.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:03 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:03.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:03 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:03.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:03 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:07:03.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:03 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:07:03.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:03 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:07:03.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:03 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:03.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:03 smithi064 ceph-mon[34900]: Deploying daemon mgr.z on smithi157 2024-09-19T08:07:03.399 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-19T08:07:03.399 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-19T08:07:03.400 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000111293 s, 4.6 MB/s 2024-09-19T08:07:03.401 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T08:07:03.457 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-09-19T08:07:03.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:03 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:03.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:03 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:03.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:03 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:03.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:03 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:03.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:03 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:03.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:03 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:07:03.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:03 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:07:03.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:03 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:07:03.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:03 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:03.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:03 smithi157 ceph-mon[35143]: Deploying daemon mgr.z on smithi157 2024-09-19T08:07:03.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:03 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:03.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:03 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:03.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:03 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:03.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:03 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:03.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:03 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:03.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:03 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:07:03.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:03 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:07:03.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:03 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:07:03.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:03 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:03.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:03 smithi022 ceph-mon[30963]: Deploying daemon mgr.z on smithi157 2024-09-19T08:07:03.513 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T08:07:03.513 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:07:03.513 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-19T08:07:03.513 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:07:03.514 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:07:03.514 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-19 08:06:39.986644805 +0000 2024-09-19T08:07:03.514 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-19 08:01:57.895889001 +0000 2024-09-19T08:07:03.514 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-19 08:01:57.895889001 +0000 2024-09-19T08:07:03.514 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-19 08:01:57.895889001 +0000 2024-09-19T08:07:03.514 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T08:07:03.577 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-19T08:07:03.577 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-19T08:07:03.577 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 9.1572e-05 s, 5.6 MB/s 2024-09-19T08:07:03.578 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T08:07:03.635 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-09-19T08:07:03.691 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T08:07:03.691 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:07:03.691 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-19T08:07:03.691 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:07:03.692 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:07:03.692 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-19 08:06:39.987644787 +0000 2024-09-19T08:07:03.692 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-19 08:01:58.208883419 +0000 2024-09-19T08:07:03.692 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-19 08:01:58.208883419 +0000 2024-09-19T08:07:03.692 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-19 08:01:58.208883419 +0000 2024-09-19T08:07:03.692 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T08:07:03.860 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-19T08:07:03.860 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-19T08:07:03.860 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000108445 s, 4.7 MB/s 2024-09-19T08:07:03.861 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T08:07:03.877 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-09-19T08:07:03.939 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T08:07:03.939 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:07:03.939 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-19T08:07:03.939 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:07:03.939 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:07:03.939 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-19 08:07:03.816219821 +0000 2024-09-19T08:07:03.939 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-19 08:01:58.516877925 +0000 2024-09-19T08:07:03.939 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-19 08:01:58.516877925 +0000 2024-09-19T08:07:03.939 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-19 08:01:58.516877925 +0000 2024-09-19T08:07:03.940 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T08:07:04.044 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-19T08:07:04.045 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-19T08:07:04.045 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 9.5339e-05 s, 5.4 MB/s 2024-09-19T08:07:04.046 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T08:07:04.062 INFO:tasks.cephadm:Deploying osd.0 on smithi022 with /dev/vg_nvme/lv_4... 2024-09-19T08:07:04.062 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T08:07:04.210 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:04 smithi157 systemd[1]: Starting Ceph mgr.z for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:07:04.336 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:07:04.457 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:04 smithi022 ceph-mon[30963]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:04.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:04 smithi157 ceph-mon[35143]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:04.504 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:04 smithi157 podman[36925]: 2024-09-19 08:07:04.180167329 +0000 UTC m=+0.015888017 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:07:04.504 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:04 smithi157 podman[36925]: 2024-09-19 08:07:04.304508101 +0000 UTC m=+0.140228786 container create f6f15a448e8ba70a72f5f96b4f3beea7e7f18eb78c8e8db726aa811f3893735d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:07:04.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:04 smithi064 ceph-mon[34900]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:04.858 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:04 smithi157 podman[36925]: 2024-09-19 08:07:04.515098191 +0000 UTC m=+0.350818875 container init f6f15a448e8ba70a72f5f96b4f3beea7e7f18eb78c8e8db726aa811f3893735d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T08:07:04.858 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:04 smithi157 podman[36925]: 2024-09-19 08:07:04.517839113 +0000 UTC m=+0.353559790 container start f6f15a448e8ba70a72f5f96b4f3beea7e7f18eb78c8e8db726aa811f3893735d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T08:07:04.858 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:04 smithi157 ceph-mgr[36944]: -- 172.21.15.157:0/4006872046 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560487190e0 con 0x5560486f6800 2024-09-19T08:07:04.859 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:04 smithi157 bash[36925]: f6f15a448e8ba70a72f5f96b4f3beea7e7f18eb78c8e8db726aa811f3893735d 2024-09-19T08:07:04.859 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:04 smithi157 systemd[1]: Started Ceph mgr.z for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:07:04.859 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:04 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:04.650+0000 7f79b34cf040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:07:04.859 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:04 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:04.712+0000 7f79b34cf040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:07:04.859 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:04 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:04.858+0000 7f79b34cf040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:07:05.207 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:04 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:04.923+0000 7f79b34cf040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:07:05.207 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:05 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:05.069+0000 7f79b34cf040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:07:05.207 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:05 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:05.207+0000 7f79b34cf040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:07:05.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:05 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:05.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:05 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:05.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:05 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:05.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:05 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:07:05.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:05 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:07:05.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:05 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:07:05.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:05 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:05.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:05 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:05.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:05 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:05.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:05 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:05.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:05 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:07:05.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:05 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:07:05.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:05 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:07:05.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:05 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:06.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:05 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:06.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:05 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:06.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:05 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:06.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:05 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:07:06.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:05 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:07:06.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:05 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:07:06.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:05 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:06.004 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:05 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:05.519+0000 7f79b34cf040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:07:06.004 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:05 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:05.578+0000 7f79b34cf040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:07:06.004 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:05 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:05.764+0000 7f79b34cf040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:07:06.587 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:06 smithi064 systemd[1]: Starting Ceph mgr.y for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:07:06.682 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:07:06.713 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_4 2024-09-19T08:07:06.958 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:06 smithi064 ceph-mon[34900]: Deploying daemon mgr.y on smithi064 2024-09-19T08:07:06.958 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:06 smithi064 ceph-mon[34900]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:06.959 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:06 smithi064 podman[36250]: 2024-09-19 08:07:06.628629339 +0000 UTC m=+0.016305710 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:07:06.959 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:06 smithi064 podman[36250]: 2024-09-19 08:07:06.772565074 +0000 UTC m=+0.160241448 container create faa40f033561bc887f55731d1bffdeb39648aa7731317b172482669823256c85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T08:07:06.959 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:06 smithi064 podman[36250]: 2024-09-19 08:07:06.95845731 +0000 UTC m=+0.346133684 container init faa40f033561bc887f55731d1bffdeb39648aa7731317b172482669823256c85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T08:07:06.967 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:07:07.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:06 smithi157 ceph-mon[35143]: Deploying daemon mgr.y on smithi064 2024-09-19T08:07:07.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:06 smithi157 ceph-mon[35143]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:07.003 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:06 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:06.637+0000 7f79b34cf040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:07:07.003 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:06 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:06.705+0000 7f79b34cf040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:07:07.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:06 smithi022 ceph-mon[30963]: Deploying daemon mgr.y on smithi064 2024-09-19T08:07:07.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:06 smithi022 ceph-mon[30963]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:07.308 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:06 smithi064 podman[36250]: 2024-09-19 08:07:06.962428914 +0000 UTC m=+0.350105286 container start faa40f033561bc887f55731d1bffdeb39648aa7731317b172482669823256c85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T08:07:07.308 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:06 smithi064 ceph-mgr[36269]: -- 172.21.15.64:0/1237968850 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557fd6c7d0e0 con 0x557fd6c5b400 2024-09-19T08:07:07.308 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:07 smithi064 bash[36250]: faa40f033561bc887f55731d1bffdeb39648aa7731317b172482669823256c85 2024-09-19T08:07:07.308 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:07 smithi064 systemd[1]: Started Ceph mgr.y for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:07:07.309 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:07 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:07.089+0000 7fc2acb04040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:07:07.309 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:07 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:07.150+0000 7fc2acb04040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:07:07.309 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:07 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:07.308+0000 7fc2acb04040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:07:07.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.451+0000 7f1e93f28640 1 -- 172.21.15.22:0/3614932112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e8c107900 msgr2=0x7f1e8c109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:07.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.451+0000 7f1e93f28640 1 --2- 172.21.15.22:0/3614932112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e8c107900 0x7f1e8c109d50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009ca0 tx=0x7f1e8402a8c0 comp rx=0 tx=0).stop 2024-09-19T08:07:07.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.451+0000 7f1e93f28640 1 -- 172.21.15.22:0/3614932112 shutdown_connections 2024-09-19T08:07:07.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.451+0000 7f1e93f28640 1 --2- 172.21.15.22:0/3614932112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e8c107900 0x7f1e8c109d50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:07.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.451+0000 7f1e93f28640 1 --2- 172.21.15.22:0/3614932112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c104fd0 0x7f1e8c1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:07.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.451+0000 7f1e93f28640 1 --2- 172.21.15.22:0/3614932112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8c1026a0 0x7f1e8c104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:07.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.451+0000 7f1e93f28640 1 -- 172.21.15.22:0/3614932112 >> 172.21.15.22:0/3614932112 conn(0x7f1e8c0fc2a0 msgr2=0x7f1e8c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:07:07.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.451+0000 7f1e93f28640 1 -- 172.21.15.22:0/3614932112 shutdown_connections 2024-09-19T08:07:07.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.451+0000 7f1e93f28640 1 -- 172.21.15.22:0/3614932112 wait complete. 2024-09-19T08:07:07.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e93f28640 1 Processor -- start 2024-09-19T08:07:07.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e93f28640 1 -- start start 2024-09-19T08:07:07.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e93f28640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8c1026a0 0x7f1e8c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:07.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e93f28640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e8c104fd0 0x7f1e8c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:07.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e93f28640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c107900 0x7f1e8c19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:07.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e93f28640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c10d910 con 0x7f1e8c104fd0 2024-09-19T08:07:07.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e93f28640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c10d790 con 0x7f1e8c1026a0 2024-09-19T08:07:07.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e93f28640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c10da90 con 0x7f1e8c107900 2024-09-19T08:07:07.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e914c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e8c104fd0 0x7f1e8c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:07:07.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e914c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e8c104fd0 0x7f1e8c198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36986/0 (socket says 172.21.15.22:36986) 2024-09-19T08:07:07.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e914c3640 1 -- 172.21.15.22:0/2005998784 learned_addr learned my addr 172.21.15.22:0/2005998784 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:07:07.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e91cc4640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8c1026a0 0x7f1e8c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:07:07.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e924c5640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c107900 0x7f1e8c19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:07:07.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e914c3640 1 -- 172.21.15.22:0/2005998784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c107900 msgr2=0x7f1e8c19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:07.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e914c3640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c107900 0x7f1e8c19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:07.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e914c3640 1 -- 172.21.15.22:0/2005998784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8c1026a0 msgr2=0x7f1e8c197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:07.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e914c3640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8c1026a0 0x7f1e8c197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:07.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e914c3640 1 -- 172.21.15.22:0/2005998784 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c19fb10 con 0x7f1e8c104fd0 2024-09-19T08:07:07.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.452+0000 7f1e924c5640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c107900 0x7f1e8c19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:07:07.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.453+0000 7f1e914c3640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e8c104fd0 0x7f1e8c198420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00d740 tx=0x7f1e7c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:07:07.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.453+0000 7f1e82ffd640 1 -- 172.21.15.22:0/2005998784 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c017070 con 0x7f1e8c104fd0 2024-09-19T08:07:07.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.453+0000 7f1e82ffd640 1 -- 172.21.15.22:0/2005998784 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e7c00de50 con 0x7f1e8c104fd0 2024-09-19T08:07:07.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.453+0000 7f1e93f28640 1 -- 172.21.15.22:0/2005998784 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8c19fe00 con 0x7f1e8c104fd0 2024-09-19T08:07:07.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.453+0000 7f1e82ffd640 1 -- 172.21.15.22:0/2005998784 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c004e00 con 0x7f1e8c104fd0 2024-09-19T08:07:07.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.453+0000 7f1e93f28640 1 -- 172.21.15.22:0/2005998784 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8c1a02b0 con 0x7f1e8c104fd0 2024-09-19T08:07:07.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.454+0000 7f1e82ffd640 1 -- 172.21.15.22:0/2005998784 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f1e7c004fa0 con 0x7f1e8c104fd0 2024-09-19T08:07:07.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.454+0000 7f1e93f28640 1 -- 172.21.15.22:0/2005998784 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e8c0fdd90 con 0x7f1e8c104fd0 2024-09-19T08:07:07.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.454+0000 7f1e82ffd640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f1e78039500 0x7f1e7803b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:07.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.454+0000 7f1e82ffd640 1 -- 172.21.15.22:0/2005998784 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1e7c04bff0 con 0x7f1e8c104fd0 2024-09-19T08:07:07.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.454+0000 7f1e91cc4640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f1e78039500 0x7f1e7803b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:07:07.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.455+0000 7f1e91cc4640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f1e78039500 0x7f1e7803b9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800bbe0 tx=0x7f1e8800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:07:07.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.459+0000 7f1e82ffd640 1 -- 172.21.15.22:0/2005998784 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1e7c019310 con 0x7f1e8c104fd0 2024-09-19T08:07:07.587 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:07 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:07.378+0000 7fc2acb04040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:07:07.587 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:07 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:07.550+0000 7fc2acb04040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:07:07.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:07.643+0000 7f1e93f28640 1 -- 172.21.15.22:0/2005998784 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1e8c062630 con 0x7f1e78039500 2024-09-19T08:07:07.726 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:07 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:07.446+0000 7f79b34cf040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:07:07.727 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:07 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:07.578+0000 7f79b34cf040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:07:07.727 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:07 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:07.726+0000 7f79b34cf040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:07:08.002 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:07 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:07.707+0000 7fc2acb04040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:07:08.003 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:07 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:07.788+0000 7f79b34cf040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:07:08.003 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:07 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:07.938+0000 7f79b34cf040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:07:08.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-mon[35143]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:08.301 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-mon[35143]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:07:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:07:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:07:08.302 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:08.302 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:08.013+0000 7f79b34cf040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:07:08.302 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:08.074+0000 7f79b34cf040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:07:08.302 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:08.236+0000 7f79b34cf040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:07:08.303 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:08.301+0000 7f79b34cf040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:07:08.329 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:08 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:08.044+0000 7fc2acb04040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:07:08.329 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:08 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:08.107+0000 7fc2acb04040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:07:08.330 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:08 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:08.329+0000 7fc2acb04040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:07:08.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:08 smithi064 ceph-mon[34900]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:08.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:08 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:08.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:08 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:08.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:08 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:08.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:08 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:08.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:08 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:08.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:08 smithi064 ceph-mon[34900]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:07:08.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:08 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:07:08.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:08 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:07:08.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:08 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:08.343 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:08 smithi022 ceph-mon[30963]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:08.343 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:08 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:08.344 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:08 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:08.344 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:08 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:08.344 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:08 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:08.344 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:08 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:08.344 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:08 smithi022 ceph-mon[30963]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:07:08.344 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:08 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:07:08.344 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:08 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:07:08.344 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:08 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:08.581 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:08.369+0000 7f79b34cf040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:07:08.582 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:08.460+0000 7f79b34cf040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:07:08.582 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:08.520+0000 7f79b34cf040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:07:08.582 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:08.581+0000 7f79b34cf040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:07:09.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:09.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:09.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:09 smithi022 ceph-mon[30963]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T08:07:09.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:07:09.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:07:09.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:09.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:09 smithi022 ceph-mon[30963]: Reconfiguring daemon mgr.x on smithi022 2024-09-19T08:07:09.587 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:09 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:09.255+0000 7fc2acb04040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:07:09.587 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:09 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:09.327+0000 7fc2acb04040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:07:09.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:09.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:09.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:09 smithi064 ceph-mon[34900]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T08:07:09.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:07:09.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:07:09.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:09.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:09 smithi064 ceph-mon[34900]: Reconfiguring daemon mgr.x on smithi022 2024-09-19T08:07:09.753 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:07:09 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:07:09.364+0000 7f79b34cf040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:07:09.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:09.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:09.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:09.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:09 smithi157 ceph-mon[35143]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T08:07:09.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:07:09.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:07:09.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:09.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:09 smithi157 ceph-mon[35143]: Reconfiguring daemon mgr.x on smithi022 2024-09-19T08:07:10.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:10.305+0000 7f1e82ffd640 1 -- 172.21.15.22:0/2005998784 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7f1e7c010460 con 0x7f1e8c104fd0 2024-09-19T08:07:10.337 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:10.083+0000 7fc2acb04040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:07:10.337 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:10.203+0000 7fc2acb04040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:07:10.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-mon[34900]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:10.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-mon[34900]: Standby manager daemon z started 2024-09-19T08:07:10.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.157:0/1612817389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:07:10.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.157:0/1612817389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:07:10.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.157:0/1612817389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:07:10.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.157:0/1612817389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:07:10.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2994672117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ed67f2d-9b28-427a-b69f-676e7a2abcae"}]: dispatch 2024-09-19T08:07:10.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2994672117' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ed67f2d-9b28-427a-b69f-676e7a2abcae"}]': finished 2024-09-19T08:07:10.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-mon[34900]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T08:07:10.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:10 smithi022 ceph-mon[30963]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:10 smithi022 ceph-mon[30963]: Standby manager daemon z started 2024-09-19T08:07:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:10 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.157:0/1612817389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:07:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:10 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.157:0/1612817389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:07:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:10 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.157:0/1612817389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:07:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:10 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.157:0/1612817389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:07:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:10 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2994672117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ed67f2d-9b28-427a-b69f-676e7a2abcae"}]: dispatch 2024-09-19T08:07:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:10 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2994672117' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ed67f2d-9b28-427a-b69f-676e7a2abcae"}]': finished 2024-09-19T08:07:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:10 smithi022 ceph-mon[30963]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T08:07:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:10.646 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:10.357+0000 7fc2acb04040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:07:10.646 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:10.420+0000 7fc2acb04040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:07:10.647 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:10.575+0000 7fc2acb04040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:07:10.647 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:10.646+0000 7fc2acb04040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:07:10.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:10 smithi157 ceph-mon[35143]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:10.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:10 smithi157 ceph-mon[35143]: Standby manager daemon z started 2024-09-19T08:07:10.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:10 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.157:0/1612817389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:07:10.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:10 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.157:0/1612817389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:07:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:10 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.157:0/1612817389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:07:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:10 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.157:0/1612817389' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:07:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:10 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2994672117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ed67f2d-9b28-427a-b69f-676e7a2abcae"}]: dispatch 2024-09-19T08:07:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:10 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2994672117' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ed67f2d-9b28-427a-b69f-676e7a2abcae"}]': finished 2024-09-19T08:07:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:10 smithi157 ceph-mon[35143]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T08:07:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:10.941 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:10.708+0000 7fc2acb04040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:07:10.942 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:10.873+0000 7fc2acb04040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:07:10.942 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:10.941+0000 7fc2acb04040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:07:11.220 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:11 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:11.010+0000 7fc2acb04040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:07:11.220 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:11 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:11.096+0000 7fc2acb04040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:07:11.220 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:11 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:11.158+0000 7fc2acb04040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:07:11.220 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:11 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:11.219+0000 7fc2acb04040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:07:11.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:11 smithi064 ceph-mon[34900]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-19T08:07:11.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:11 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:07:11.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:11 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/440804177' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:07:11.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:11 smithi157 ceph-mon[35143]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-19T08:07:11.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:11 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:07:11.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:11 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/440804177' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:07:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:11 smithi022 ceph-mon[30963]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-19T08:07:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:11 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:07:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:11 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/440804177' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:07:12.310 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:07:12 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:07:12.048+0000 7fc2acb04040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:07:12.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:12 smithi064 ceph-mon[34900]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:12.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:12 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:12.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:12 smithi064 ceph-mon[34900]: Standby manager daemon y started 2024-09-19T08:07:12.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:12 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.64:0/2562356559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:07:12.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:12 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.64:0/2562356559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:07:12.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:12 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.64:0/2562356559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:07:12.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:12 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.64:0/2562356559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:07:12.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:12.728+0000 7f1e82ffd640 1 -- 172.21.15.22:0/2005998784 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f1e7c020ca0 con 0x7f1e8c104fd0 2024-09-19T08:07:12.728 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:12 smithi022 ceph-mon[30963]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:12.728 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:12 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:12.728 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:12 smithi022 ceph-mon[30963]: Standby manager daemon y started 2024-09-19T08:07:12.728 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:12 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.64:0/2562356559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:07:12.728 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:12 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.64:0/2562356559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:07:12.729 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:12 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.64:0/2562356559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:07:12.729 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:12 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.64:0/2562356559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:07:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:12 smithi157 ceph-mon[35143]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:12 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:12 smithi157 ceph-mon[35143]: Standby manager daemon y started 2024-09-19T08:07:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:12 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.64:0/2562356559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:07:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:12 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.64:0/2562356559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:07:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:12 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.64:0/2562356559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:07:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:12 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.64:0/2562356559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:07:14.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:13 smithi157 ceph-mon[35143]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:14.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:13 smithi157 ceph-mon[35143]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-19T08:07:14.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:13 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:07:14.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:13 smithi022 ceph-mon[30963]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:14.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:13 smithi022 ceph-mon[30963]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-19T08:07:14.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:13 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:07:14.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:13 smithi064 ceph-mon[34900]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:14.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:13 smithi064 ceph-mon[34900]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-19T08:07:14.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:13 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:07:16.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:15 smithi157 ceph-mon[35143]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:16.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:15 smithi022 ceph-mon[30963]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:16.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:15 smithi064 ceph-mon[34900]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:17.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:17 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:17.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:17 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:17.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:17 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:17.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:17 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:17.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:17 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:17.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:17 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:17.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:17 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:17.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:17 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:17.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:17 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:17.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:17 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:17.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:17 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:17.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:17 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:17.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:17 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:17.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:17 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:17.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:17 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:17.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:17 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:17.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:17 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:17.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:17 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:18.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:18 smithi022 ceph-mon[30963]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:18.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:18 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T08:07:18.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:18 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:18.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:18 smithi064 ceph-mon[34900]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:18.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:18 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T08:07:18.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:18 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:19.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:18 smithi157 ceph-mon[35143]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:19.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:18 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T08:07:19.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:18 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:19.805 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:19 smithi022 ceph-mon[30963]: Deploying daemon osd.0 on smithi022 2024-09-19T08:07:19.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:19 smithi064 ceph-mon[34900]: Deploying daemon osd.0 on smithi022 2024-09-19T08:07:20.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:19 smithi157 ceph-mon[35143]: Deploying daemon osd.0 on smithi022 2024-09-19T08:07:20.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:20 smithi022 ceph-mon[30963]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:20.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:20 smithi064 ceph-mon[34900]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:21.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:20 smithi157 ceph-mon[35143]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:22.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:22 smithi064 ceph-mon[34900]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:23.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:22 smithi157 ceph-mon[35143]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:23.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:22 smithi022 ceph-mon[30963]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:24.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:24 smithi064 ceph-mon[34900]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:25.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:24 smithi157 ceph-mon[35143]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:25.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:24 smithi022 ceph-mon[30963]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:25.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:25.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:25.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:26.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:26.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:26.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:26.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:26.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:26.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:26.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:26 smithi064 ceph-mon[34900]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:26.897 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:26 smithi022 ceph-mon[30963]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:27.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:26 smithi157 ceph-mon[35143]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:28.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:28 smithi157 ceph-mon[35143]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:28.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:28.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:28.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:28.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:28.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:28.728 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:28 smithi022 ceph-mon[30963]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:28.728 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:28.729 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:28.729 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:28.729 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:28.729 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:28.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:28 smithi064 ceph-mon[34900]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:28.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:28.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:28.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:28.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:28.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:29.073 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 0 on host 'smithi022' 2024-09-19T08:07:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:29.072+0000 7f1e82ffd640 1 -- 172.21.15.22:0/2005998784 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1e8c062630 con 0x7f1e78039500 2024-09-19T08:07:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:29.075+0000 7f1e93f28640 1 -- 172.21.15.22:0/2005998784 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f1e78039500 msgr2=0x7f1e7803b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:29.075+0000 7f1e93f28640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f1e78039500 0x7f1e7803b9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800bbe0 tx=0x7f1e8800b390 comp rx=0 tx=0).stop 2024-09-19T08:07:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:29.075+0000 7f1e93f28640 1 -- 172.21.15.22:0/2005998784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e8c104fd0 msgr2=0x7f1e8c198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:29.075+0000 7f1e93f28640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e8c104fd0 0x7f1e8c198420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00d740 tx=0x7f1e7c00dc00 comp rx=0 tx=0).stop 2024-09-19T08:07:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:29.075+0000 7f1e93f28640 1 -- 172.21.15.22:0/2005998784 shutdown_connections 2024-09-19T08:07:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:29.075+0000 7f1e93f28640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f1e78039500 0x7f1e7803b9c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:29.075+0000 7f1e93f28640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e8c107900 0x7f1e8c19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:29.075+0000 7f1e93f28640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e8c104fd0 0x7f1e8c198420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:29.075+0000 7f1e93f28640 1 --2- 172.21.15.22:0/2005998784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8c1026a0 0x7f1e8c197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:29.075+0000 7f1e93f28640 1 -- 172.21.15.22:0/2005998784 >> 172.21.15.22:0/2005998784 conn(0x7f1e8c0fc2a0 msgr2=0x7f1e8c102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:07:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:29.075+0000 7f1e93f28640 1 -- 172.21.15.22:0/2005998784 shutdown_connections 2024-09-19T08:07:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:07:29.075+0000 7f1e93f28640 1 -- 172.21.15.22:0/2005998784 wait complete. 2024-09-19T08:07:29.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:29.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:29.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:29.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:29.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:29.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:29.795 DEBUG:teuthology.orchestra.run.smithi022:osd.0> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T08:07:29.797 INFO:tasks.cephadm:Deploying osd.1 on smithi064 with /dev/vg_nvme/lv_4... 2024-09-19T08:07:29.797 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T08:07:29.825 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:29.825 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:29.825 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:30.057 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:07:30.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:30 smithi157 ceph-mon[35143]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:30.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:30 smithi157 ceph-mon[35143]: from='osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T08:07:30.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:30 smithi022 ceph-mon[30963]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:30.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:30 smithi022 ceph-mon[30963]: from='osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T08:07:30.757 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:07:30 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0[44787]: 2024-09-19T08:07:30.391+0000 7f543deb0640 -1 osd.0 0 log_to_monitors true 2024-09-19T08:07:30.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:30 smithi064 ceph-mon[34900]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:30.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:30 smithi064 ceph-mon[34900]: from='osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T08:07:31.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:31 smithi064 ceph-mon[34900]: from='osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T08:07:31.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:31 smithi064 ceph-mon[34900]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T08:07:31.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:31 smithi064 ceph-mon[34900]: from='osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-19T08:07:31.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:31 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:31.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:31 smithi064 ceph-mon[34900]: from='osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-19T08:07:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:31 smithi157 ceph-mon[35143]: from='osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T08:07:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:31 smithi157 ceph-mon[35143]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T08:07:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:31 smithi157 ceph-mon[35143]: from='osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-19T08:07:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:31 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:31 smithi157 ceph-mon[35143]: from='osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-19T08:07:31.757 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:07:31 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0[44787]: 2024-09-19T08:07:31.458+0000 7f5439e33640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T08:07:31.757 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:07:31 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0[44787]: 2024-09-19T08:07:31.465+0000 7f5435c5b640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:07:31.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:31 smithi022 ceph-mon[30963]: from='osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T08:07:31.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:31 smithi022 ceph-mon[30963]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T08:07:31.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:31 smithi022 ceph-mon[30963]: from='osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-19T08:07:31.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:31 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:31.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:31 smithi022 ceph-mon[30963]: from='osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-19T08:07:32.284 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:07:32.312 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_4 2024-09-19T08:07:32.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:32 smithi064 ceph-mon[34900]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:32.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:32 smithi064 ceph-mon[34900]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T08:07:32.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:32 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:32.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:32 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:32.603 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:07:32.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:32 smithi022 ceph-mon[30963]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:32.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:32 smithi022 ceph-mon[30963]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T08:07:32.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:32 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:32.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:32 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:32.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:32 smithi157 ceph-mon[35143]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:32.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:32 smithi157 ceph-mon[35143]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T08:07:32.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:32 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:32.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:32 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:33.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.036+0000 7febfc776640 1 -- 172.21.15.64:0/1228586910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febf4101c00 msgr2=0x7febf4102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:33.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.036+0000 7febfc776640 1 --2- 172.21.15.64:0/1228586910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febf4101c00 0x7febf4102000 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7febf0009900 tx=0x7febf002a410 comp rx=0 tx=0).stop 2024-09-19T08:07:33.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.036+0000 7febfc776640 1 -- 172.21.15.64:0/1228586910 shutdown_connections 2024-09-19T08:07:33.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.036+0000 7febfc776640 1 --2- 172.21.15.64:0/1228586910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febf41037c0 0x7febf410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:33.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.036+0000 7febfc776640 1 --2- 172.21.15.64:0/1228586910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf4102e00 0x7febf4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:33.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.036+0000 7febfc776640 1 --2- 172.21.15.64:0/1228586910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febf4101c00 0x7febf4102000 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:33.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.036+0000 7febfc776640 1 -- 172.21.15.64:0/1228586910 >> 172.21.15.64:0/1228586910 conn(0x7febf40fd3b0 msgr2=0x7febf40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:07:33.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.036+0000 7febfc776640 1 -- 172.21.15.64:0/1228586910 shutdown_connections 2024-09-19T08:07:33.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.036+0000 7febfc776640 1 -- 172.21.15.64:0/1228586910 wait complete. 2024-09-19T08:07:33.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.036+0000 7febfc776640 1 Processor -- start 2024-09-19T08:07:33.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfc776640 1 -- start start 2024-09-19T08:07:33.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfc776640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febf4101c00 0x7febf4075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:33.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfc776640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febf4102e00 0x7febf40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:33.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfc776640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf41037c0 0x7febf4074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:33.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfc776640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7febf410d860 con 0x7febf4102e00 2024-09-19T08:07:33.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfc776640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7febf410d6e0 con 0x7febf4101c00 2024-09-19T08:07:33.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfc776640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7febf410d9e0 con 0x7febf41037c0 2024-09-19T08:07:33.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfa512640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febf4101c00 0x7febf4075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:07:33.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfa512640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febf4101c00 0x7febf4075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55960/0 (socket says 172.21.15.64:55960) 2024-09-19T08:07:33.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfa512640 1 -- 172.21.15.64:0/1472987036 learned_addr learned my addr 172.21.15.64:0/1472987036 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T08:07:33.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febf9d11640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febf4102e00 0x7febf40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:07:33.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfa512640 1 -- 172.21.15.64:0/1472987036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf41037c0 msgr2=0x7febf4074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfad13640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf41037c0 0x7febf4074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:07:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfa512640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf41037c0 0x7febf4074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfa512640 1 -- 172.21.15.64:0/1472987036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febf4102e00 msgr2=0x7febf40745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfa512640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febf4102e00 0x7febf40745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfa512640 1 -- 172.21.15.64:0/1472987036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf40753e0 con 0x7febf4101c00 2024-09-19T08:07:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febf9d11640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febf4102e00 0x7febf40745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:07:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.037+0000 7febfa512640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febf4101c00 0x7febf4075f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7febf0031610 tx=0x7febf0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:07:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.038+0000 7febeb7fe640 1 -- 172.21.15.64:0/1472987036 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf003f030 con 0x7febf4101c00 2024-09-19T08:07:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.038+0000 7febeb7fe640 1 -- 172.21.15.64:0/1472987036 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf0003ad0 con 0x7febf4101c00 2024-09-19T08:07:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.038+0000 7febfc776640 1 -- 172.21.15.64:0/1472987036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf41a2020 con 0x7febf4101c00 2024-09-19T08:07:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.038+0000 7febeb7fe640 1 -- 172.21.15.64:0/1472987036 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0035070 con 0x7febf4101c00 2024-09-19T08:07:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.038+0000 7febfc776640 1 -- 172.21.15.64:0/1472987036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf41a2520 con 0x7febf4101c00 2024-09-19T08:07:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.039+0000 7febfc776640 1 -- 172.21.15.64:0/1472987036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf4109050 con 0x7febf4101c00 2024-09-19T08:07:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.040+0000 7febeb7fe640 1 -- 172.21.15.64:0/1472987036 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7febf0042030 con 0x7febf4101c00 2024-09-19T08:07:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.040+0000 7febeb7fe640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7febe00a3230 0x7febe00a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.041+0000 7febeb7fe640 1 -- 172.21.15.64:0/1472987036 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7febf00ee2e0 con 0x7febf4101c00 2024-09-19T08:07:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.041+0000 7febf9d11640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7febe00a3230 0x7febe00a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:07:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.041+0000 7febf9d11640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7febe00a3230 0x7febe00a56f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7febf4075d10 tx=0x7febe400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:07:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.045+0000 7febeb7fe640 1 -- 172.21.15.64:0/1472987036 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7febf00bca70 con 0x7febf4101c00 2024-09-19T08:07:33.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:33.231+0000 7febfc776640 1 -- 172.21.15.64:0/1472987036 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7febf4062630 con 0x7febe00a3230 2024-09-19T08:07:33.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: purged_snaps scrub starts 2024-09-19T08:07:33.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: purged_snaps scrub ok 2024-09-19T08:07:33.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:33.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307] boot 2024-09-19T08:07:33.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T08:07:33.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:33.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: Detected new or changed devices on smithi022 2024-09-19T08:07:33.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:33.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:33.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:07:33.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi022 to 17092M 2024-09-19T08:07:33.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:33.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:33.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:33.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:33.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:07:33.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:07:33.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:33 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:33.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: purged_snaps scrub starts 2024-09-19T08:07:33.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: purged_snaps scrub ok 2024-09-19T08:07:33.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:33.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307] boot 2024-09-19T08:07:33.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T08:07:33.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:33.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: Detected new or changed devices on smithi022 2024-09-19T08:07:33.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:33.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:33.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:07:33.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi022 to 17092M 2024-09-19T08:07:33.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:33.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:33.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:33.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:33.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:07:33.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:07:33.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:33 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:33.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: purged_snaps scrub starts 2024-09-19T08:07:33.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: purged_snaps scrub ok 2024-09-19T08:07:33.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:33.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: osd.0 [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307] boot 2024-09-19T08:07:33.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T08:07:33.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:07:33.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: Detected new or changed devices on smithi022 2024-09-19T08:07:33.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:33.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:33.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:07:33.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi022 to 17092M 2024-09-19T08:07:33.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:33.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:33.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:33.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:33.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:07:33.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:07:33.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:33 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:35.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:34 smithi157 ceph-mon[35143]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:35.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:34 smithi157 ceph-mon[35143]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:07:35.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:34 smithi157 ceph-mon[35143]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T08:07:35.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:34 smithi022 ceph-mon[30963]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:35.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:34 smithi022 ceph-mon[30963]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:07:35.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:34 smithi022 ceph-mon[30963]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T08:07:35.063 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:34 smithi064 ceph-mon[34900]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:07:35.063 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:34 smithi064 ceph-mon[34900]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:07:35.063 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:34 smithi064 ceph-mon[34900]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T08:07:35.934 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:35 smithi064 ceph-mon[34900]: from='client.? 172.21.15.64:0/4085440296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bff3e283-4ab1-49d7-86d8-45d361995122"}]: dispatch 2024-09-19T08:07:35.934 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:35 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bff3e283-4ab1-49d7-86d8-45d361995122"}]: dispatch 2024-09-19T08:07:35.934 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:35 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bff3e283-4ab1-49d7-86d8-45d361995122"}]': finished 2024-09-19T08:07:35.934 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:35 smithi064 ceph-mon[34900]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T08:07:35.934 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:35 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:07:36.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:35 smithi157 ceph-mon[35143]: from='client.? 172.21.15.64:0/4085440296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bff3e283-4ab1-49d7-86d8-45d361995122"}]: dispatch 2024-09-19T08:07:36.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:35 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bff3e283-4ab1-49d7-86d8-45d361995122"}]: dispatch 2024-09-19T08:07:36.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:35 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bff3e283-4ab1-49d7-86d8-45d361995122"}]': finished 2024-09-19T08:07:36.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:35 smithi157 ceph-mon[35143]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T08:07:36.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:35 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:07:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:35 smithi022 ceph-mon[30963]: from='client.? 172.21.15.64:0/4085440296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bff3e283-4ab1-49d7-86d8-45d361995122"}]: dispatch 2024-09-19T08:07:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:35 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bff3e283-4ab1-49d7-86d8-45d361995122"}]: dispatch 2024-09-19T08:07:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:35 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bff3e283-4ab1-49d7-86d8-45d361995122"}]': finished 2024-09-19T08:07:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:35 smithi022 ceph-mon[30963]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T08:07:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:35 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:07:37.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:36 smithi157 ceph-mon[35143]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:37.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:36 smithi157 ceph-mon[35143]: from='client.? 172.21.15.64:0/1637470462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:07:37.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:36 smithi022 ceph-mon[30963]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:37.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:36 smithi022 ceph-mon[30963]: from='client.? 172.21.15.64:0/1637470462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:07:37.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:36 smithi064 ceph-mon[34900]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:37.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:36 smithi064 ceph-mon[34900]: from='client.? 172.21.15.64:0/1637470462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:07:39.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:38 smithi157 ceph-mon[35143]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:39.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:38 smithi022 ceph-mon[30963]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:39.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:38 smithi064 ceph-mon[34900]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:40.823 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:40 smithi064 ceph-mon[34900]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:41.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:40 smithi157 ceph-mon[35143]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:41.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:40 smithi022 ceph-mon[30963]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:42.806 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:42 smithi064 ceph-mon[34900]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:43.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:42 smithi157 ceph-mon[35143]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:43.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:42 smithi022 ceph-mon[30963]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:43.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:43 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T08:07:43.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:43 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:44.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:43 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T08:07:44.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:43 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:44.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:43 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T08:07:44.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:43 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:44.998 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:44 smithi064 ceph-mon[34900]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:44.998 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:44 smithi064 ceph-mon[34900]: Deploying daemon osd.1 on smithi064 2024-09-19T08:07:45.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:44 smithi157 ceph-mon[35143]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:45.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:44 smithi157 ceph-mon[35143]: Deploying daemon osd.1 on smithi064 2024-09-19T08:07:45.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:44 smithi022 ceph-mon[30963]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:45.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:44 smithi022 ceph-mon[30963]: Deploying daemon osd.1 on smithi064 2024-09-19T08:07:46.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:45 smithi157 ceph-mon[35143]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:46.005 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:45 smithi064 ceph-mon[34900]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:46.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:45 smithi022 ceph-mon[30963]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:48.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:47 smithi157 ceph-mon[35143]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:48.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:47 smithi022 ceph-mon[30963]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:48.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:47 smithi064 ceph-mon[34900]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:50.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:49 smithi157 ceph-mon[35143]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:50.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:50.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:50.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:50.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:49 smithi022 ceph-mon[30963]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:50.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:50.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:50.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:50.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:49 smithi064 ceph-mon[34900]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:50.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:50.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:50.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:52.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:51 smithi157 ceph-mon[35143]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:52.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:51 smithi022 ceph-mon[30963]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:52.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:51 smithi064 ceph-mon[34900]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:53.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:53.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:53.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:53.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:53.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:53.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:53.331+0000 7febeb7fe640 1 -- 172.21.15.64:0/1472987036 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7febf4062630 con 0x7febe00a3230 2024-09-19T08:07:53.331 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 1 on host 'smithi064' 2024-09-19T08:07:53.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:53.333+0000 7febfc776640 1 -- 172.21.15.64:0/1472987036 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7febe00a3230 msgr2=0x7febe00a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:53.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:53.333+0000 7febfc776640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7febe00a3230 0x7febe00a56f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7febf4075d10 tx=0x7febe400b390 comp rx=0 tx=0).stop 2024-09-19T08:07:53.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:53.333+0000 7febfc776640 1 -- 172.21.15.64:0/1472987036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febf4101c00 msgr2=0x7febf4075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:53.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:53.333+0000 7febfc776640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febf4101c00 0x7febf4075f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7febf0031610 tx=0x7febf0004270 comp rx=0 tx=0).stop 2024-09-19T08:07:53.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:53.333+0000 7febfc776640 1 -- 172.21.15.64:0/1472987036 shutdown_connections 2024-09-19T08:07:53.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:53.333+0000 7febfc776640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7febe00a3230 0x7febe00a56f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:53.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:53.333+0000 7febfc776640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf41037c0 0x7febf4074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:53.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:53.333+0000 7febfc776640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febf4102e00 0x7febf40745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:53.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:53.333+0000 7febfc776640 1 --2- 172.21.15.64:0/1472987036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febf4101c00 0x7febf4075f90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:53.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:53.333+0000 7febfc776640 1 -- 172.21.15.64:0/1472987036 >> 172.21.15.64:0/1472987036 conn(0x7febf40fd3b0 msgr2=0x7febf4106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:07:53.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:53.333+0000 7febfc776640 1 -- 172.21.15.64:0/1472987036 shutdown_connections 2024-09-19T08:07:53.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:07:53.333+0000 7febfc776640 1 -- 172.21.15.64:0/1472987036 wait complete. 2024-09-19T08:07:53.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:53.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:53.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:53.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:53.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:53.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:53.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:53.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:53.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:53.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:53.917 DEBUG:teuthology.orchestra.run.smithi064:osd.1> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T08:07:53.920 INFO:tasks.cephadm:Deploying osd.2 on smithi157 with /dev/vg_nvme/lv_4... 2024-09-19T08:07:53.920 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T08:07:54.175 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:07:54.182 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:54 smithi064 ceph-mon[34900]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:54.182 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:54 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:54.182 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:54 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:54.182 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:54 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:54.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:54 smithi157 ceph-mon[35143]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:54.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:54 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:54.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:54 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:54.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:54 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:54.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:54 smithi022 ceph-mon[30963]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:54.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:54 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:07:54.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:54 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:54.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:54 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:56.088 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:07:55 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1[39381]: 2024-09-19T08:07:55.760+0000 7f24c8516640 -1 osd.1 0 log_to_monitors true 2024-09-19T08:07:56.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:56 smithi157 ceph-mon[35143]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:56.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:56 smithi157 ceph-mon[35143]: from='osd.1 [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:07:56.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:56 smithi157 ceph-mon[35143]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:07:56.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:56 smithi022 ceph-mon[30963]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:56.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:56 smithi022 ceph-mon[30963]: from='osd.1 [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:07:56.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:56 smithi022 ceph-mon[30963]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:07:56.540 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:07:56.564 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_4 2024-09-19T08:07:56.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:56 smithi064 ceph-mon[34900]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:56.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:56 smithi064 ceph-mon[34900]: from='osd.1 [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:07:56.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:56 smithi064 ceph-mon[34900]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:07:56.818 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:07:57.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.239+0000 7fea46ff7640 1 -- 172.21.15.157:0/2705510130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea40101c00 msgr2=0x7fea40102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:57.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.239+0000 7fea46ff7640 1 --2- 172.21.15.157:0/2705510130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea40101c00 0x7fea40102000 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fea34009900 tx=0x7fea3402a3f0 comp rx=0 tx=0).stop 2024-09-19T08:07:57.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.239+0000 7fea46ff7640 1 -- 172.21.15.157:0/2705510130 shutdown_connections 2024-09-19T08:07:57.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.239+0000 7fea46ff7640 1 --2- 172.21.15.157:0/2705510130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea401037c0 0x7fea4010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:57.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.239+0000 7fea46ff7640 1 --2- 172.21.15.157:0/2705510130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea40102e00 0x7fea40103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:57.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.239+0000 7fea46ff7640 1 --2- 172.21.15.157:0/2705510130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea40101c00 0x7fea40102000 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:57.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.239+0000 7fea46ff7640 1 -- 172.21.15.157:0/2705510130 >> 172.21.15.157:0/2705510130 conn(0x7fea400fd390 msgr2=0x7fea400ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:07:57.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.239+0000 7fea46ff7640 1 -- 172.21.15.157:0/2705510130 shutdown_connections 2024-09-19T08:07:57.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.239+0000 7fea46ff7640 1 -- 172.21.15.157:0/2705510130 wait complete. 2024-09-19T08:07:57.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.240+0000 7fea46ff7640 1 Processor -- start 2024-09-19T08:07:57.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.240+0000 7fea46ff7640 1 -- start start 2024-09-19T08:07:57.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.240+0000 7fea46ff7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea40101c00 0x7fea40197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:57.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.240+0000 7fea46ff7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea40102e00 0x7fea40198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:57.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.240+0000 7fea46ff7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea401037c0 0x7fea4019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:57.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.240+0000 7fea46ff7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fea4010d8b0 con 0x7fea401037c0 2024-09-19T08:07:57.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.240+0000 7fea46ff7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea4010d730 con 0x7fea40101c00 2024-09-19T08:07:57.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.240+0000 7fea46ff7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea4010da30 con 0x7fea40102e00 2024-09-19T08:07:57.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.240+0000 7fea3ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea40102e00 0x7fea40198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:07:57.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.240+0000 7fea3ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea40102e00 0x7fea40198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51054/0 (socket says 172.21.15.157:51054) 2024-09-19T08:07:57.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.240+0000 7fea3ffff640 1 -- 172.21.15.157:0/4229180409 learned_addr learned my addr 172.21.15.157:0/4229180409 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T08:07:57.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.241+0000 7fea3ffff640 1 -- 172.21.15.157:0/4229180409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea40101c00 msgr2=0x7fea40197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:57.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.241+0000 7fea3ffff640 1 --2- 172.21.15.157:0/4229180409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea40101c00 0x7fea40197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:57.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.241+0000 7fea3ffff640 1 -- 172.21.15.157:0/4229180409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea401037c0 msgr2=0x7fea4019f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:07:57.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.241+0000 7fea3ffff640 1 --2- 172.21.15.157:0/4229180409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea401037c0 0x7fea4019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:07:57.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.241+0000 7fea3ffff640 1 -- 172.21.15.157:0/4229180409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4019fa10 con 0x7fea40102e00 2024-09-19T08:07:57.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.241+0000 7fea3ffff640 1 --2- 172.21.15.157:0/4229180409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea40102e00 0x7fea40198320 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00c920 tx=0x7fea2c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:07:57.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.241+0000 7fea3dffb640 1 -- 172.21.15.157:0/4229180409 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c007c70 con 0x7fea40102e00 2024-09-19T08:07:57.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.241+0000 7fea3dffb640 1 -- 172.21.15.157:0/4229180409 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea2c007e10 con 0x7fea40102e00 2024-09-19T08:07:57.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.241+0000 7fea46ff7640 1 -- 172.21.15.157:0/4229180409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4019fd00 con 0x7fea40102e00 2024-09-19T08:07:57.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.241+0000 7fea3dffb640 1 -- 172.21.15.157:0/4229180409 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c018890 con 0x7fea40102e00 2024-09-19T08:07:57.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.241+0000 7fea46ff7640 1 -- 172.21.15.157:0/4229180409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea401a0230 con 0x7fea40102e00 2024-09-19T08:07:57.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.243+0000 7fea46ff7640 1 -- 172.21.15.157:0/4229180409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea40108f30 con 0x7fea40102e00 2024-09-19T08:07:57.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.243+0000 7fea3dffb640 1 -- 172.21.15.157:0/4229180409 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7fea2c018a30 con 0x7fea40102e00 2024-09-19T08:07:57.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.244+0000 7fea3dffb640 1 --2- 172.21.15.157:0/4229180409 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fea300a3260 0x7fea300a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:07:57.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.244+0000 7fea3dffb640 1 -- 172.21.15.157:0/4229180409 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fea2c0ce0a0 con 0x7fea40102e00 2024-09-19T08:07:57.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.244+0000 7fea44d93640 1 --2- 172.21.15.157:0/4229180409 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fea300a3260 0x7fea300a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:07:57.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.245+0000 7fea44d93640 1 --2- 172.21.15.157:0/4229180409 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fea300a3260 0x7fea300a5720 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea340057c0 tx=0x7fea34031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:07:57.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.249+0000 7fea3dffb640 1 -- 172.21.15.157:0/4229180409 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fea2c014030 con 0x7fea40102e00 2024-09-19T08:07:57.444 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:07:57.444+0000 7fea46ff7640 1 -- 172.21.15.157:0/4229180409 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fea40062630 con 0x7fea300a3260 2024-09-19T08:07:57.445 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:57 smithi157 ceph-mon[35143]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T08:07:57.445 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:57 smithi157 ceph-mon[35143]: from='osd.1 [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:07:57.445 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:57 smithi157 ceph-mon[35143]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T08:07:57.445 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:57 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:07:57.445 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:57 smithi157 ceph-mon[35143]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:07:57.445 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:57 smithi157 ceph-mon[35143]: Detected new or changed devices on smithi064 2024-09-19T08:07:57.445 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:57 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:57.445 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:57 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:57.445 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:57 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:07:57.445 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:57 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:57.445 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:57 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:57.446 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:57 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:57.446 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:57 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:57.446 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:57 smithi157 ceph-mon[35143]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T08:07:57.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:57 smithi022 ceph-mon[30963]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T08:07:57.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:57 smithi022 ceph-mon[30963]: from='osd.1 [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:07:57.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:57 smithi022 ceph-mon[30963]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T08:07:57.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:57 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:07:57.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:57 smithi022 ceph-mon[30963]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:07:57.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:57 smithi022 ceph-mon[30963]: Detected new or changed devices on smithi064 2024-09-19T08:07:57.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:57 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:57.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:57 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:57.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:57 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:07:57.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:57 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:57.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:57 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:57.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:57 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:57.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:57 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:57.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:57 smithi022 ceph-mon[30963]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T08:07:57.587 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1[39381]: 2024-09-19T08:07:57.141+0000 7f24c4499640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T08:07:57.587 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1[39381]: 2024-09-19T08:07:57.144+0000 7f24c02c1640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:07:57.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-mon[34900]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T08:07:57.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-mon[34900]: from='osd.1 [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:07:57.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-mon[34900]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T08:07:57.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:07:57.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-mon[34900]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:07:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-mon[34900]: Detected new or changed devices on smithi064 2024-09-19T08:07:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:07:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:07:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:07:57.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:57 smithi064 ceph-mon[34900]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T08:07:58.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:58 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi064 to 17088M 2024-09-19T08:07:58.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:58 smithi157 ceph-mon[35143]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:58 smithi157 ceph-mon[35143]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T08:07:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:58 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:07:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:58 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:07:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:58 smithi157 ceph-mon[35143]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:07:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:58 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:07:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:58 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:07:58.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:58 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:58.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:58 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi064 to 17088M 2024-09-19T08:07:58.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:58 smithi022 ceph-mon[30963]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:58.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:58 smithi022 ceph-mon[30963]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T08:07:58.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:58 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:07:58.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:58 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:07:58.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:58 smithi022 ceph-mon[30963]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:07:58.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:58 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:07:58.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:58 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:07:58.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:58 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:58.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:58 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi064 to 17088M 2024-09-19T08:07:58.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:58 smithi064 ceph-mon[34900]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:07:58.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:58 smithi064 ceph-mon[34900]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T08:07:58.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:58 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:07:58.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:58 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:07:58.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:58 smithi064 ceph-mon[34900]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:07:58.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:58 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:07:58.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:58 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:07:58.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:58 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:07:59.344 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:59 smithi157 ceph-mon[35143]: purged_snaps scrub starts 2024-09-19T08:07:59.344 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:59 smithi157 ceph-mon[35143]: purged_snaps scrub ok 2024-09-19T08:07:59.344 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:59 smithi157 ceph-mon[35143]: osd.1 [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228] boot 2024-09-19T08:07:59.344 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:59 smithi157 ceph-mon[35143]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T08:07:59.344 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:07:59 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:07:59.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:59 smithi022 ceph-mon[30963]: purged_snaps scrub starts 2024-09-19T08:07:59.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:59 smithi022 ceph-mon[30963]: purged_snaps scrub ok 2024-09-19T08:07:59.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:59 smithi022 ceph-mon[30963]: osd.1 [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228] boot 2024-09-19T08:07:59.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:59 smithi022 ceph-mon[30963]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T08:07:59.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:07:59 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:07:59.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:59 smithi064 ceph-mon[34900]: purged_snaps scrub starts 2024-09-19T08:07:59.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:59 smithi064 ceph-mon[34900]: purged_snaps scrub ok 2024-09-19T08:07:59.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:59 smithi064 ceph-mon[34900]: osd.1 [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228] boot 2024-09-19T08:07:59.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:59 smithi064 ceph-mon[34900]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T08:07:59.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:07:59 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:08:00.228 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:00 smithi157 ceph-mon[35143]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:08:00.228 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:00 smithi157 ceph-mon[35143]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T08:08:00.228 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:00 smithi157 ceph-mon[35143]: from='client.? 172.21.15.157:0/2320809653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81421d08-7dc6-419b-a566-dee1e1e13b59"}]: dispatch 2024-09-19T08:08:00.228 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:00 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81421d08-7dc6-419b-a566-dee1e1e13b59"}]: dispatch 2024-09-19T08:08:00.228 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:00 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81421d08-7dc6-419b-a566-dee1e1e13b59"}]': finished 2024-09-19T08:08:00.228 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:00 smithi157 ceph-mon[35143]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T08:08:00.228 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:00 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:00.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:00 smithi022 ceph-mon[30963]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:08:00.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:00 smithi022 ceph-mon[30963]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T08:08:00.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:00 smithi022 ceph-mon[30963]: from='client.? 172.21.15.157:0/2320809653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81421d08-7dc6-419b-a566-dee1e1e13b59"}]: dispatch 2024-09-19T08:08:00.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:00 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81421d08-7dc6-419b-a566-dee1e1e13b59"}]: dispatch 2024-09-19T08:08:00.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:00 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81421d08-7dc6-419b-a566-dee1e1e13b59"}]': finished 2024-09-19T08:08:00.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:00 smithi022 ceph-mon[30963]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T08:08:00.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:00 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:00.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:00 smithi064 ceph-mon[34900]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:08:00.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:00 smithi064 ceph-mon[34900]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T08:08:00.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:00 smithi064 ceph-mon[34900]: from='client.? 172.21.15.157:0/2320809653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81421d08-7dc6-419b-a566-dee1e1e13b59"}]: dispatch 2024-09-19T08:08:00.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:00 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81421d08-7dc6-419b-a566-dee1e1e13b59"}]: dispatch 2024-09-19T08:08:00.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:00 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81421d08-7dc6-419b-a566-dee1e1e13b59"}]': finished 2024-09-19T08:08:00.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:00 smithi064 ceph-mon[34900]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T08:08:00.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:00 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:01.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:01 smithi157 ceph-mon[35143]: from='client.? 172.21.15.157:0/3619001201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:08:01.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:01 smithi022 ceph-mon[30963]: from='client.? 172.21.15.157:0/3619001201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:08:01.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:01 smithi064 ceph-mon[34900]: from='client.? 172.21.15.157:0/3619001201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:08:02.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:02 smithi157 ceph-mon[35143]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:02.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:02 smithi022 ceph-mon[30963]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:02.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:02 smithi064 ceph-mon[34900]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:04.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:04 smithi157 ceph-mon[35143]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:04.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:04 smithi022 ceph-mon[30963]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:04.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:04 smithi064 ceph-mon[34900]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:06.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:06 smithi157 ceph-mon[35143]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:06.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:06 smithi022 ceph-mon[30963]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:06.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:06 smithi064 ceph-mon[34900]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:08.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:08 smithi157 ceph-mon[35143]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:08.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:08 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T08:08:08.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:08 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:08.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:08 smithi022 ceph-mon[30963]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:08.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:08 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T08:08:08.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:08 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:08.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:08 smithi064 ceph-mon[34900]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:08.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:08 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T08:08:08.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:08 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:09.382 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:09 smithi157 ceph-mon[35143]: Deploying daemon osd.2 on smithi157 2024-09-19T08:08:09.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:09 smithi022 ceph-mon[30963]: Deploying daemon osd.2 on smithi157 2024-09-19T08:08:09.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:09 smithi064 ceph-mon[34900]: Deploying daemon osd.2 on smithi157 2024-09-19T08:08:10.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:10 smithi157 ceph-mon[35143]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:10.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:10 smithi022 ceph-mon[30963]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:10.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:10 smithi064 ceph-mon[34900]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:12.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:12 smithi157 ceph-mon[35143]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:12.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:12 smithi022 ceph-mon[30963]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:12.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:12 smithi064 ceph-mon[34900]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:14.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:14 smithi157 ceph-mon[35143]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:14.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:14 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:08:14.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:14 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:14.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:14 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:14.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:14 smithi022 ceph-mon[30963]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:14 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:08:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:14 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:14 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:14.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:14 smithi064 ceph-mon[34900]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:14.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:14 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:08:14.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:14 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:14.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:14 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:16 smithi157 ceph-mon[35143]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:16.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:16 smithi022 ceph-mon[30963]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:16.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:16 smithi064 ceph-mon[34900]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:17.573 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:17 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:17.573 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:17 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:17.573 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:17 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:17.573 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:17 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:08:17.573 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:17 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:17.614 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:08:17.613+0000 7fea3dffb640 1 -- 172.21.15.157:0/4229180409 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fea40062630 con 0x7fea300a3260 2024-09-19T08:08:17.614 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 2 on host 'smithi157' 2024-09-19T08:08:17.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:08:17.616+0000 7fea46ff7640 1 -- 172.21.15.157:0/4229180409 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fea300a3260 msgr2=0x7fea300a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:08:17.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:08:17.616+0000 7fea46ff7640 1 --2- 172.21.15.157:0/4229180409 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fea300a3260 0x7fea300a5720 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea340057c0 tx=0x7fea34031400 comp rx=0 tx=0).stop 2024-09-19T08:08:17.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:08:17.616+0000 7fea46ff7640 1 -- 172.21.15.157:0/4229180409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea40102e00 msgr2=0x7fea40198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:08:17.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:08:17.616+0000 7fea46ff7640 1 --2- 172.21.15.157:0/4229180409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea40102e00 0x7fea40198320 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00c920 tx=0x7fea2c00cde0 comp rx=0 tx=0).stop 2024-09-19T08:08:17.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:08:17.616+0000 7fea46ff7640 1 -- 172.21.15.157:0/4229180409 shutdown_connections 2024-09-19T08:08:17.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:08:17.616+0000 7fea46ff7640 1 --2- 172.21.15.157:0/4229180409 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fea300a3260 0x7fea300a5720 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:17.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:08:17.616+0000 7fea46ff7640 1 --2- 172.21.15.157:0/4229180409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea401037c0 0x7fea4019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:17.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:08:17.616+0000 7fea46ff7640 1 --2- 172.21.15.157:0/4229180409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea40102e00 0x7fea40198320 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:17.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:08:17.616+0000 7fea46ff7640 1 --2- 172.21.15.157:0/4229180409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea40101c00 0x7fea40197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:17.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:08:17.616+0000 7fea46ff7640 1 -- 172.21.15.157:0/4229180409 >> 172.21.15.157:0/4229180409 conn(0x7fea400fd390 msgr2=0x7fea40106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:08:17.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:08:17.616+0000 7fea46ff7640 1 -- 172.21.15.157:0/4229180409 shutdown_connections 2024-09-19T08:08:17.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:08:17.616+0000 7fea46ff7640 1 -- 172.21.15.157:0/4229180409 wait complete. 2024-09-19T08:08:17.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:17 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:17.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:17 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:17.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:17 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:17.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:17 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:08:17.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:17 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:17.770 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:17 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:17.770 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:17 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:17.771 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:17 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:17.771 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:17 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:08:17.771 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:17 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:18.279 DEBUG:teuthology.orchestra.run.smithi157:osd.2> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T08:08:18.281 INFO:tasks.cephadm:Deploying osd.3 on smithi022 with /dev/vg_nvme/lv_3... 2024-09-19T08:08:18.281 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T08:08:18.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:18 smithi157 ceph-mon[35143]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:18.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:18 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:08:18.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:18 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:18.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:18 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:18.536 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:08:18.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:18 smithi022 ceph-mon[30963]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:18.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:18 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:08:18.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:18 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:18.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:18 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:18.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:18 smithi064 ceph-mon[34900]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:18.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:18 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:08:18.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:18 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:18.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:18 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:20.004 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:08:19 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2[40099]: 2024-09-19T08:08:19.801+0000 7f44725d0640 -1 osd.2 0 log_to_monitors true 2024-09-19T08:08:20.559 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:20 smithi022 ceph-mon[30963]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:20.559 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:20 smithi022 ceph-mon[30963]: from='osd.2 [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:08:20.559 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:20 smithi022 ceph-mon[30963]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:08:20.681 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:20 smithi157 ceph-mon[35143]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:20.681 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:20 smithi157 ceph-mon[35143]: from='osd.2 [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:08:20.681 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:20 smithi157 ceph-mon[35143]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:08:20.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:20 smithi064 ceph-mon[34900]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:20.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:20 smithi064 ceph-mon[34900]: from='osd.2 [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:08:20.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:20 smithi064 ceph-mon[34900]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:08:21.339 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:08:21.365 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_3 2024-09-19T08:08:21.594 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:21 smithi022 ceph-mon[30963]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T08:08:21.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:21 smithi022 ceph-mon[30963]: from='osd.2 [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:08:21.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:21 smithi022 ceph-mon[30963]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T08:08:21.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:21 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:21.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:21 smithi022 ceph-mon[30963]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:08:21.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:21 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:21.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:21 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:21.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:21 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:08:21.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:21 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:21.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:21 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:21.596 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:21 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:08:21.596 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:21 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:21.628 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:08:21.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:21 smithi157 ceph-mon[35143]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T08:08:21.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:21 smithi157 ceph-mon[35143]: from='osd.2 [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:08:21.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:21 smithi157 ceph-mon[35143]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T08:08:21.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:21 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:21.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:21 smithi157 ceph-mon[35143]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:08:21.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:21 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:21.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:21 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:21.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:21 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:08:21.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:21 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:21.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:21 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:21.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:21 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:08:21.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:21 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:21.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:21 smithi064 ceph-mon[34900]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T08:08:21.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:21 smithi064 ceph-mon[34900]: from='osd.2 [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:08:21.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:21 smithi064 ceph-mon[34900]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T08:08:21.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:21 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:21.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:21 smithi064 ceph-mon[34900]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:08:21.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:21 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:21.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:21 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:21.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:21 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:08:21.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:21 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:21.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:21 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:21.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:21 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:08:21.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:21 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:22.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.210+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/546391207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f681037d0 msgr2=0x7f4f6810a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:08:22.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.210+0000 7f4f6dc5d640 1 --2- 172.21.15.22:0/546391207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f681037d0 0x7f4f6810a060 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009ca0 tx=0x7f4f6002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:08:22.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.210+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/546391207 shutdown_connections 2024-09-19T08:08:22.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.210+0000 7f4f6dc5d640 1 --2- 172.21.15.22:0/546391207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f681037d0 0x7f4f6810a060 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:22.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.210+0000 7f4f6dc5d640 1 --2- 172.21.15.22:0/546391207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f68102e10 0x7f4f68103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:22.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.210+0000 7f4f6dc5d640 1 --2- 172.21.15.22:0/546391207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f68101c10 0x7f4f68102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.210+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/546391207 >> 172.21.15.22:0/546391207 conn(0x7f4f680fd3c0 msgr2=0x7f4f680ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:08:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.210+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/546391207 shutdown_connections 2024-09-19T08:08:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.210+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/546391207 wait complete. 2024-09-19T08:08:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.210+0000 7f4f6dc5d640 1 Processor -- start 2024-09-19T08:08:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.210+0000 7f4f6dc5d640 1 -- start start 2024-09-19T08:08:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f6dc5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f68101c10 0x7f4f68075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:08:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f6dc5d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f68102e10 0x7f4f680745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:08:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f6dc5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f681037d0 0x7f4f68074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:08:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f6dc5d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6810d890 con 0x7f4f681037d0 2024-09-19T08:08:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f6dc5d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6810d710 con 0x7f4f68101c10 2024-09-19T08:08:22.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f6dc5d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6810da10 con 0x7f4f68102e10 2024-09-19T08:08:22.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f67fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f681037d0 0x7f4f68074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:08:22.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f67fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f681037d0 0x7f4f68074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54336/0 (socket says 172.21.15.22:54336) 2024-09-19T08:08:22.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f67fff640 1 -- 172.21.15.22:0/2548845994 learned_addr learned my addr 172.21.15.22:0/2548845994 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:08:22.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f677fe640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f68101c10 0x7f4f68075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:08:22.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f66ffd640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f68102e10 0x7f4f680745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:08:22.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f67fff640 1 -- 172.21.15.22:0/2548845994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f68102e10 msgr2=0x7f4f680745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:08:22.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f67fff640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f68102e10 0x7f4f680745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:22.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f67fff640 1 -- 172.21.15.22:0/2548845994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f68101c10 msgr2=0x7f4f68075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:08:22.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f67fff640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f68101c10 0x7f4f68075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:22.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f67fff640 1 -- 172.21.15.22:0/2548845994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f680753e0 con 0x7f4f681037d0 2024-09-19T08:08:22.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f66ffd640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f68102e10 0x7f4f680745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:08:22.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f67fff640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f681037d0 0x7f4f68074b20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4f60005e30 tx=0x7f4f60031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:08:22.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.211+0000 7f4f64ff9640 1 -- 172.21.15.22:0/2548845994 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60003c30 con 0x7f4f681037d0 2024-09-19T08:08:22.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.212+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/2548845994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f681a1f30 con 0x7f4f681037d0 2024-09-19T08:08:22.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.212+0000 7f4f64ff9640 1 -- 172.21.15.22:0/2548845994 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f60003dd0 con 0x7f4f681037d0 2024-09-19T08:08:22.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.212+0000 7f4f64ff9640 1 -- 172.21.15.22:0/2548845994 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6002fde0 con 0x7f4f681037d0 2024-09-19T08:08:22.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.212+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/2548845994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f681a2430 con 0x7f4f681037d0 2024-09-19T08:08:22.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.213+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/2548845994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f68108f40 con 0x7f4f681037d0 2024-09-19T08:08:22.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.214+0000 7f4f64ff9640 1 -- 172.21.15.22:0/2548845994 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f4f60004050 con 0x7f4f681037d0 2024-09-19T08:08:22.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.214+0000 7f4f64ff9640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4f500a31b0 0x7f4f500a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:08:22.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.214+0000 7f4f677fe640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4f500a31b0 0x7f4f500a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:08:22.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.214+0000 7f4f64ff9640 1 -- 172.21.15.22:0/2548845994 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f4f600ef530 con 0x7f4f681037d0 2024-09-19T08:08:22.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.215+0000 7f4f677fe640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4f500a31b0 0x7f4f500a5670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f68102c70 tx=0x7f4f58005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:08:22.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.219+0000 7f4f64ff9640 1 -- 172.21.15.22:0/2548845994 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4f600bd9a0 con 0x7f4f681037d0 2024-09-19T08:08:22.253 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:08:21 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2[40099]: 2024-09-19T08:08:21.790+0000 7f446e553640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T08:08:22.253 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:08:21 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2[40099]: 2024-09-19T08:08:21.794+0000 7f4466373640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:08:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:22.405+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/2548845994 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4f68062630 con 0x7f4f500a31b0 2024-09-19T08:08:22.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:22 smithi022 ceph-mon[30963]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:22.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:22 smithi022 ceph-mon[30963]: Detected new or changed devices on smithi157 2024-09-19T08:08:22.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:22 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi157 to 17088M 2024-09-19T08:08:22.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:22 smithi022 ceph-mon[30963]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T08:08:22.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:22 smithi022 ceph-mon[30963]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T08:08:22.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:22 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:22.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:22 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:22.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:22 smithi022 ceph-mon[30963]: from='osd.2 ' entity='osd.2' 2024-09-19T08:08:22.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:22 smithi157 ceph-mon[35143]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:22.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:22 smithi157 ceph-mon[35143]: Detected new or changed devices on smithi157 2024-09-19T08:08:22.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:22 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi157 to 17088M 2024-09-19T08:08:22.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:22 smithi157 ceph-mon[35143]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T08:08:22.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:22 smithi157 ceph-mon[35143]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T08:08:22.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:22 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:22.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:22 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:22.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:22 smithi157 ceph-mon[35143]: from='osd.2 ' entity='osd.2' 2024-09-19T08:08:22.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:22 smithi064 ceph-mon[34900]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:22.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:22 smithi064 ceph-mon[34900]: Detected new or changed devices on smithi157 2024-09-19T08:08:22.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:22 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi157 to 17088M 2024-09-19T08:08:22.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:22 smithi064 ceph-mon[34900]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T08:08:22.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:22 smithi064 ceph-mon[34900]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T08:08:22.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:22 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:22.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:22 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:22.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:22 smithi064 ceph-mon[34900]: from='osd.2 ' entity='osd.2' 2024-09-19T08:08:23.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:23 smithi157 ceph-mon[35143]: purged_snaps scrub starts 2024-09-19T08:08:23.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:23 smithi157 ceph-mon[35143]: purged_snaps scrub ok 2024-09-19T08:08:23.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:23 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:23.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:23 smithi157 ceph-mon[35143]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:08:23.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:23 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:08:23.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:23 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:08:23.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:23 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:23.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:23 smithi157 ceph-mon[35143]: osd.2 [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867] boot 2024-09-19T08:08:23.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:23 smithi157 ceph-mon[35143]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T08:08:23.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:23 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:23.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:23 smithi022 ceph-mon[30963]: purged_snaps scrub starts 2024-09-19T08:08:23.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:23 smithi022 ceph-mon[30963]: purged_snaps scrub ok 2024-09-19T08:08:23.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:23 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:23.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:23 smithi022 ceph-mon[30963]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:08:23.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:23 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:08:23.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:23 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:08:23.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:23 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:23.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:23 smithi022 ceph-mon[30963]: osd.2 [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867] boot 2024-09-19T08:08:23.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:23 smithi022 ceph-mon[30963]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T08:08:23.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:23 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:23.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:23 smithi064 ceph-mon[34900]: purged_snaps scrub starts 2024-09-19T08:08:23.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:23 smithi064 ceph-mon[34900]: purged_snaps scrub ok 2024-09-19T08:08:23.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:23 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:23.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:23 smithi064 ceph-mon[34900]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:08:23.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:23 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:08:23.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:23 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:08:23.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:23 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:23.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:23 smithi064 ceph-mon[34900]: osd.2 [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867] boot 2024-09-19T08:08:23.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:23 smithi064 ceph-mon[34900]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T08:08:23.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:23 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:08:24.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:24 smithi022 ceph-mon[30963]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:24.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:24 smithi022 ceph-mon[30963]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T08:08:24.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:24 smithi157 ceph-mon[35143]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:24.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:24 smithi157 ceph-mon[35143]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T08:08:24.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:24 smithi064 ceph-mon[34900]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:08:24.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:24 smithi064 ceph-mon[34900]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T08:08:25.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:08:25.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:25 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2928078902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34debfd0-3226-4d78-b9e6-e03dbc5190f0"}]: dispatch 2024-09-19T08:08:25.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T08:08:25.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:25 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2928078902' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34debfd0-3226-4d78-b9e6-e03dbc5190f0"}]': finished 2024-09-19T08:08:25.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:25 smithi022 ceph-mon[30963]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T08:08:25.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:25.693 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:08:25.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:08:25.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:25 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2928078902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34debfd0-3226-4d78-b9e6-e03dbc5190f0"}]: dispatch 2024-09-19T08:08:25.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T08:08:25.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:25 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2928078902' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34debfd0-3226-4d78-b9e6-e03dbc5190f0"}]': finished 2024-09-19T08:08:25.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:25 smithi157 ceph-mon[35143]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T08:08:25.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:25.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:08:25.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:08:25.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:25 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2928078902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34debfd0-3226-4d78-b9e6-e03dbc5190f0"}]: dispatch 2024-09-19T08:08:25.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T08:08:25.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:25 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2928078902' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34debfd0-3226-4d78-b9e6-e03dbc5190f0"}]': finished 2024-09-19T08:08:25.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:25 smithi064 ceph-mon[34900]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T08:08:25.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:25.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:08:26.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:26 smithi157 ceph-mon[35143]: pgmap v78: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:26.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:26 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1197400770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:08:26.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:26 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:08:26.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:26 smithi157 ceph-mon[35143]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T08:08:26.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:26 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:26.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:26 smithi022 ceph-mon[30963]: pgmap v78: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:26.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:26 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1197400770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:08:26.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:26 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:08:26.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:26 smithi022 ceph-mon[30963]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T08:08:26.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:26 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:26.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:26 smithi064 ceph-mon[34900]: pgmap v78: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:26.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:26 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1197400770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:08:26.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:26 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:08:26.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:26 smithi064 ceph-mon[34900]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T08:08:26.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:26 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:27.257 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:08:26 smithi022 sudo[49440]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:08:27.257 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:08:26 smithi022 sudo[49440]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:08:27.257 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:08:26 smithi022 sudo[49440]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:08:27.257 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:08:26 smithi022 sudo[49440]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:27.412+0000 7f4f64ff9640 1 -- 172.21.15.22:0/2548845994 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4f600b9910 con 0x7f4f681037d0 2024-09-19T08:08:27.757 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:08:27 smithi022 sudo[49809]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:08:27.757 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:08:27 smithi022 sudo[49809]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:08:27.757 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:08:27 smithi022 sudo[49809]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:08:27.757 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:08:27 smithi022 sudo[49809]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:08:27.942 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:08:27 smithi064 sudo[43073]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:08:27.942 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:08:27 smithi064 sudo[43073]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:08:27.942 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:08:27 smithi064 sudo[43073]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:08:27.942 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:08:27 smithi064 sudo[43073]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:08:28.244 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:08:27 smithi064 sudo[43076]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:08:28.244 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:08:27 smithi064 sudo[43076]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:08:28.244 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:08:27 smithi064 sudo[43076]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:08:28.244 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:08:27 smithi064 sudo[43076]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:08:28.466 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:08:28 smithi157 sudo[44171]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:08:28.466 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:08:28 smithi157 sudo[44171]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:08:28.467 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:08:28 smithi157 sudo[44171]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:08:28.467 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:08:28 smithi157 sudo[44171]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:08:28.467 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:28 smithi157 ceph-mon[35143]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:28.467 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:28 smithi157 ceph-mon[35143]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T08:08:28.467 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:28.467 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:28 smithi157 ceph-mon[35143]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T08:08:28.467 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:28.467 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:28.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:28 smithi064 ceph-mon[34900]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:28.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:28 smithi064 ceph-mon[34900]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T08:08:28.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:28.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:28 smithi064 ceph-mon[34900]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T08:08:28.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:28.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:28.753 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:08:28 smithi157 sudo[44174]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:08:28.753 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:08:28 smithi157 sudo[44174]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:08:28.753 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:08:28 smithi157 sudo[44174]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:08:28.753 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:08:28 smithi157 sudo[44174]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:08:28.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:28 smithi022 ceph-mon[30963]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:28.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:28 smithi022 ceph-mon[30963]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T08:08:28.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:28.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:28 smithi022 ceph-mon[30963]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T08:08:28.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:28.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:28 smithi022 sudo[50188]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:08:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:28 smithi022 sudo[50188]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:08:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:28 smithi022 sudo[50188]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:08:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:28 smithi022 sudo[50188]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:08:29.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:29 smithi022 sudo[50374]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:08:29.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:29 smithi022 sudo[50374]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:08:29.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:29 smithi022 sudo[50374]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:08:29.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:29 smithi022 sudo[50374]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:08:29.896 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:29 smithi157 sudo[44177]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:08:29.896 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:29 smithi157 sudo[44177]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:08:29.896 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:29 smithi157 sudo[44177]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:08:29.896 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:29 smithi157 sudo[44177]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:08:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:29 smithi157 sudo[44180]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:08:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:29 smithi157 sudo[44180]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:08:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:29 smithi157 sudo[44180]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:08:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:29 smithi157 sudo[44180]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:08:30.426 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 sudo[43079]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:08:30.426 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 sudo[43079]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:08:30.426 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 sudo[43079]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:08:30.426 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 sudo[43079]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:08:30.640 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:30 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:30.640 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:30 smithi022 ceph-mon[30963]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:08:30.640 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:30 smithi022 ceph-mon[30963]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:08:30.641 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:30 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:08:30.641 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:30 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:08:30.641 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:30 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:08:30.641 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:30 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:08:30.641 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:30 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:08:30.641 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:30 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:08:30.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:30 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:30.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:30 smithi157 ceph-mon[35143]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:08:30.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:30 smithi157 ceph-mon[35143]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:08:30.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:30 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:08:30.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:30 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:08:30.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:30 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:08:30.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:30 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:08:30.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:30 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:08:30.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:30 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:08:30.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 sudo[43082]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:08:30.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 sudo[43082]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:08:30.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 sudo[43082]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:08:30.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 sudo[43082]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:08:30.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:30.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 ceph-mon[34900]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:08:30.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 ceph-mon[34900]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:08:30.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:08:30.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:08:30.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:08:30.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:08:30.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:08:30.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:30 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:08:31.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:31 smithi022 ceph-mon[30963]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:31.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:31 smithi022 ceph-mon[30963]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:08:31.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:31 smithi022 ceph-mon[30963]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:08:31.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:31 smithi022 ceph-mon[30963]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:08:31.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:31 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:08:31.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:31 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:08:31.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:31 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:08:31.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:31 smithi022 ceph-mon[30963]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:08:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:31 smithi157 ceph-mon[35143]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:31 smithi157 ceph-mon[35143]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:08:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:31 smithi157 ceph-mon[35143]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:08:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:31 smithi157 ceph-mon[35143]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:08:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:31 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:08:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:31 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:08:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:31 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:08:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:31 smithi157 ceph-mon[35143]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:08:31.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:31 smithi064 ceph-mon[34900]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:31.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:31 smithi064 ceph-mon[34900]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:08:31.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:31 smithi064 ceph-mon[34900]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:08:31.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:31 smithi064 ceph-mon[34900]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:08:31.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:31 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:08:31.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:31 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:08:31.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:31 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:08:31.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:31 smithi064 ceph-mon[34900]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:08:32.685 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:32 smithi022 ceph-mon[30963]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:32.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:32 smithi157 ceph-mon[35143]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:32.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:32 smithi064 ceph-mon[34900]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:34.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:34 smithi157 ceph-mon[35143]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:34.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:34 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T08:08:34.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:34 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:34.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:34 smithi022 ceph-mon[30963]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:34.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T08:08:34.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:34.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:34 smithi064 ceph-mon[34900]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:34.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:34 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T08:08:34.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:34 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:35.716 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:35 smithi022 ceph-mon[30963]: Deploying daemon osd.3 on smithi022 2024-09-19T08:08:35.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:35 smithi157 ceph-mon[35143]: Deploying daemon osd.3 on smithi022 2024-09-19T08:08:35.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:35 smithi064 ceph-mon[34900]: Deploying daemon osd.3 on smithi022 2024-09-19T08:08:36.682 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:36 smithi022 ceph-mon[30963]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T08:08:36.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:36 smithi157 ceph-mon[35143]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T08:08:36.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:36 smithi064 ceph-mon[34900]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T08:08:38.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:38 smithi157 ceph-mon[35143]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T08:08:38.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:38 smithi022 ceph-mon[30963]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T08:08:38.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:38 smithi064 ceph-mon[34900]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T08:08:40.739 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:40 smithi022 ceph-mon[30963]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T08:08:40.739 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:40 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:08:40.739 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:40 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:40.739 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:40 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:40.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:40 smithi157 ceph-mon[35143]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T08:08:40.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:40 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:08:40.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:40 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:40.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:40 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:40 smithi064 ceph-mon[34900]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T08:08:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:40 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:08:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:40 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:40 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:42.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:42 smithi157 ceph-mon[35143]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T08:08:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:42 smithi022 ceph-mon[30963]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T08:08:42.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:42 smithi064 ceph-mon[34900]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T08:08:44.085 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:43 smithi022 ceph-mon[30963]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T08:08:44.085 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:43 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:44.085 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:43 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:44.085 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:43 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:44.085 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:43 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:08:44.085 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:43 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:44.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:43 smithi064 ceph-mon[34900]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T08:08:44.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:43 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:44.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:43 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:44.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:43 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:44.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:43 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:08:44.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:43 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:44.124+0000 7f4f64ff9640 1 -- 172.21.15.22:0/2548845994 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4f68062630 con 0x7f4f500a31b0 2024-09-19T08:08:44.125 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 3 on host 'smithi022' 2024-09-19T08:08:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:44.127+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/2548845994 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4f500a31b0 msgr2=0x7f4f500a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:08:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:44.127+0000 7f4f6dc5d640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4f500a31b0 0x7f4f500a5670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f68102c70 tx=0x7f4f58005f90 comp rx=0 tx=0).stop 2024-09-19T08:08:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:44.127+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/2548845994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f681037d0 msgr2=0x7f4f68074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:08:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:44.127+0000 7f4f6dc5d640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f681037d0 0x7f4f68074b20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4f60005e30 tx=0x7f4f60031ee0 comp rx=0 tx=0).stop 2024-09-19T08:08:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:44.127+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/2548845994 shutdown_connections 2024-09-19T08:08:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:44.127+0000 7f4f6dc5d640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4f500a31b0 0x7f4f500a5670 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:44.127+0000 7f4f6dc5d640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f681037d0 0x7f4f68074b20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:44.127+0000 7f4f6dc5d640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f68102e10 0x7f4f680745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:44.127+0000 7f4f6dc5d640 1 --2- 172.21.15.22:0/2548845994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f68101c10 0x7f4f68075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:44.127+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/2548845994 >> 172.21.15.22:0/2548845994 conn(0x7f4f680fd3c0 msgr2=0x7f4f68106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:08:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:44.127+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/2548845994 shutdown_connections 2024-09-19T08:08:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:08:44.127+0000 7f4f6dc5d640 1 -- 172.21.15.22:0/2548845994 wait complete. 2024-09-19T08:08:44.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:43 smithi157 ceph-mon[35143]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T08:08:44.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:43 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:44.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:43 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:44.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:43 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:44.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:43 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:08:44.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:43 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:44.767 DEBUG:teuthology.orchestra.run.smithi022:osd.3> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T08:08:44.770 INFO:tasks.cephadm:Deploying osd.4 on smithi064 with /dev/vg_nvme/lv_3... 2024-09-19T08:08:44.770 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T08:08:44.973 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:44 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:08:44.973 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:44 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:44.973 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:44 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:45.029 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:08:45.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:44 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:08:45.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:44 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:45.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:44 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:45.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:44 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:08:45.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:44 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:45.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:44 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:46.352 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:46 smithi022 ceph-mon[30963]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T08:08:46.353 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:46 smithi022 ceph-mon[30963]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T08:08:46.353 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:08:46 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3[51394]: 2024-09-19T08:08:46.062+0000 7fa168377640 -1 osd.3 0 log_to_monitors true 2024-09-19T08:08:46.480 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:46 smithi064 ceph-mon[34900]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T08:08:46.480 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:46 smithi064 ceph-mon[34900]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T08:08:46.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:46 smithi157 ceph-mon[35143]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T08:08:46.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:46 smithi157 ceph-mon[35143]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T08:08:47.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:47 smithi022 ceph-mon[30963]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T08:08:47.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:47 smithi022 ceph-mon[30963]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T08:08:47.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:47 smithi022 ceph-mon[30963]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-19T08:08:47.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:47 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:47.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:47 smithi022 ceph-mon[30963]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-19T08:08:47.402 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:47 smithi064 ceph-mon[34900]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T08:08:47.402 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:47 smithi064 ceph-mon[34900]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T08:08:47.402 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:47 smithi064 ceph-mon[34900]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-19T08:08:47.402 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:47 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:47.403 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:47 smithi064 ceph-mon[34900]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-19T08:08:47.404 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:47 smithi157 ceph-mon[35143]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T08:08:47.404 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:47 smithi157 ceph-mon[35143]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T08:08:47.404 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:47 smithi157 ceph-mon[35143]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-19T08:08:47.404 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:47 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:47.404 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:47 smithi157 ceph-mon[35143]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-19T08:08:47.571 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:08:47.601 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_3 2024-09-19T08:08:47.689 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:08:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3[51394]: 2024-09-19T08:08:47.542+0000 7fa1642fa640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T08:08:47.689 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:08:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3[51394]: 2024-09-19T08:08:47.547+0000 7fa15c11a640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:08:47.853 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:08:48.248 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:48 smithi064 ceph-mon[34900]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:48.248 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:48 smithi064 ceph-mon[34900]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T08:08:48.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:48.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:48.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:48 smithi064 ceph-mon[34900]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' 2024-09-19T08:08:48.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:48.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:48.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:08:48.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:08:48.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:48.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:48.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:08:48.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:48.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.334+0000 7fd44d8d1640 1 -- 172.21.15.64:0/594432710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4481026c0 msgr2=0x7fd448108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:08:48.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.334+0000 7fd44d8d1640 1 --2- 172.21.15.64:0/594432710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4481026c0 0x7fd448108f50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd440009ca0 tx=0x7fd44002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:08:48.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.334+0000 7fd44d8d1640 1 -- 172.21.15.64:0/594432710 shutdown_connections 2024-09-19T08:08:48.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.334+0000 7fd44d8d1640 1 --2- 172.21.15.64:0/594432710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4481026c0 0x7fd448108f50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:48.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.334+0000 7fd44d8d1640 1 --2- 172.21.15.64:0/594432710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd448101d00 0x7fd448102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:48.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.334+0000 7fd44d8d1640 1 --2- 172.21.15.64:0/594432710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd448100b00 0x7fd448100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:48.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.334+0000 7fd44d8d1640 1 -- 172.21.15.64:0/594432710 >> 172.21.15.64:0/594432710 conn(0x7fd4480fc290 msgr2=0x7fd4480fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:08:48.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.334+0000 7fd44d8d1640 1 -- 172.21.15.64:0/594432710 shutdown_connections 2024-09-19T08:08:48.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.334+0000 7fd44d8d1640 1 -- 172.21.15.64:0/594432710 wait complete. 2024-09-19T08:08:48.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.335+0000 7fd44d8d1640 1 Processor -- start 2024-09-19T08:08:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.335+0000 7fd44d8d1640 1 -- start start 2024-09-19T08:08:48.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.335+0000 7fd44d8d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd448100b00 0x7fd448104a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:08:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.335+0000 7fd44d8d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd448101d00 0x7fd4481030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:08:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd44d8d1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4481026c0 0x7fd448103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:08:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd44d8d1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd44810c780 con 0x7fd4481026c0 2024-09-19T08:08:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd44d8d1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd44810c600 con 0x7fd448101d00 2024-09-19T08:08:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd44d8d1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd44810c900 con 0x7fd448100b00 2024-09-19T08:08:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd4467fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd448101d00 0x7fd4481030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:08:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd446ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd448100b00 0x7fd448104a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:08:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd446ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd448100b00 0x7fd448104a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.64:55694/0 (socket says 172.21.15.64:55694) 2024-09-19T08:08:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd4467fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd448101d00 0x7fd4481030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40150/0 (socket says 172.21.15.64:40150) 2024-09-19T08:08:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd446ffd640 1 -- 172.21.15.64:0/1780660845 learned_addr learned my addr 172.21.15.64:0/1780660845 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T08:08:48.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd4477fe640 1 --2- 172.21.15.64:0/1780660845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4481026c0 0x7fd448103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:08:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd446ffd640 1 -- 172.21.15.64:0/1780660845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd448101d00 msgr2=0x7fd4481030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:08:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd446ffd640 1 --2- 172.21.15.64:0/1780660845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd448101d00 0x7fd4481030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd446ffd640 1 -- 172.21.15.64:0/1780660845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4481026c0 msgr2=0x7fd448103620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:08:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd446ffd640 1 --2- 172.21.15.64:0/1780660845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4481026c0 0x7fd448103620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:08:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd446ffd640 1 -- 172.21.15.64:0/1780660845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd448103ee0 con 0x7fd448100b00 2024-09-19T08:08:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd4467fc640 1 --2- 172.21.15.64:0/1780660845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd448101d00 0x7fd4481030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:08:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd446ffd640 1 --2- 172.21.15.64:0/1780660845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd448100b00 0x7fd448104a90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd43800d770 tx=0x7fd43800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:08:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd44c8cf640 1 -- 172.21.15.64:0/1780660845 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438017070 con 0x7fd448100b00 2024-09-19T08:08:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd44c8cf640 1 -- 172.21.15.64:0/1780660845 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd438009c30 con 0x7fd448100b00 2024-09-19T08:08:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd44d8d1640 1 -- 172.21.15.64:0/1780660845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4481a2060 con 0x7fd448100b00 2024-09-19T08:08:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.336+0000 7fd44c8cf640 1 -- 172.21.15.64:0/1780660845 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438003dd0 con 0x7fd448100b00 2024-09-19T08:08:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.337+0000 7fd44d8d1640 1 -- 172.21.15.64:0/1780660845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4481a2570 con 0x7fd448100b00 2024-09-19T08:08:48.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.338+0000 7fd44d8d1640 1 -- 172.21.15.64:0/1780660845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4481a27a0 con 0x7fd448100b00 2024-09-19T08:08:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.339+0000 7fd44c8cf640 1 -- 172.21.15.64:0/1780660845 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd438003fb0 con 0x7fd448100b00 2024-09-19T08:08:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.340+0000 7fd44c8cf640 1 --2- 172.21.15.64:0/1780660845 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd4300a3240 0x7fd4300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:08:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.340+0000 7fd44c8cf640 1 -- 172.21.15.64:0/1780660845 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7fd4380ce040 con 0x7fd448100b00 2024-09-19T08:08:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.340+0000 7fd4467fc640 1 --2- 172.21.15.64:0/1780660845 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd4300a3240 0x7fd4300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:08:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.341+0000 7fd4467fc640 1 --2- 172.21.15.64:0/1780660845 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd4300a3240 0x7fd4300a5700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd448104810 tx=0x7fd43c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:08:48.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.343+0000 7fd44c8cf640 1 -- 172.21.15.64:0/1780660845 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd438009de0 con 0x7fd448100b00 2024-09-19T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:48 smithi157 ceph-mon[35143]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:48 smithi157 ceph-mon[35143]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:48.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:48 smithi157 ceph-mon[35143]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' 2024-09-19T08:08:48.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:48.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:48.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:08:48.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:08:48.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:48.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:48.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:08:48.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:48 smithi022 ceph-mon[30963]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:48 smithi022 ceph-mon[30963]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T08:08:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:48 smithi022 ceph-mon[30963]: from='osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456]' entity='osd.3' 2024-09-19T08:08:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:08:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:08:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:48.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:08:48.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:08:48.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:08:48.530+0000 7fd44d8d1640 1 -- 172.21.15.64:0/1780660845 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd44806aef0 con 0x7fd4300a3240 2024-09-19T08:08:49.250 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:49 smithi064 ceph-mon[34900]: purged_snaps scrub starts 2024-09-19T08:08:49.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:49 smithi064 ceph-mon[34900]: purged_snaps scrub ok 2024-09-19T08:08:49.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:49 smithi064 ceph-mon[34900]: Detected new or changed devices on smithi022 2024-09-19T08:08:49.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:49 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi022 to 8546M 2024-09-19T08:08:49.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:49.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:49 smithi064 ceph-mon[34900]: osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456] boot 2024-09-19T08:08:49.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:49 smithi064 ceph-mon[34900]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T08:08:49.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:49.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:49 smithi064 ceph-mon[34900]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:08:49.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:08:49.263 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:08:49.263 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:49.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:49 smithi157 ceph-mon[35143]: purged_snaps scrub starts 2024-09-19T08:08:49.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:49 smithi157 ceph-mon[35143]: purged_snaps scrub ok 2024-09-19T08:08:49.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:49 smithi157 ceph-mon[35143]: Detected new or changed devices on smithi022 2024-09-19T08:08:49.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:49 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi022 to 8546M 2024-09-19T08:08:49.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:49.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:49 smithi157 ceph-mon[35143]: osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456] boot 2024-09-19T08:08:49.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:49 smithi157 ceph-mon[35143]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T08:08:49.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:49.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:49 smithi157 ceph-mon[35143]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:08:49.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:08:49.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:08:49.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:49.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:49 smithi022 ceph-mon[30963]: purged_snaps scrub starts 2024-09-19T08:08:49.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:49 smithi022 ceph-mon[30963]: purged_snaps scrub ok 2024-09-19T08:08:49.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:49 smithi022 ceph-mon[30963]: Detected new or changed devices on smithi022 2024-09-19T08:08:49.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:49 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi022 to 8546M 2024-09-19T08:08:49.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:49.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:49 smithi022 ceph-mon[30963]: osd.3 [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456] boot 2024-09-19T08:08:49.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:49 smithi022 ceph-mon[30963]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T08:08:49.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:08:49.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:49 smithi022 ceph-mon[30963]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:08:49.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:08:49.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:08:49.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:08:50.432 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:50 smithi064 ceph-mon[34900]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:50.432 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:50 smithi064 ceph-mon[34900]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T08:08:50.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:50 smithi157 ceph-mon[35143]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:50.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:50 smithi157 ceph-mon[35143]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T08:08:50.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:50 smithi022 ceph-mon[30963]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:08:50.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:50 smithi022 ceph-mon[30963]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T08:08:51.492 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:51 smithi064 ceph-mon[34900]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T08:08:51.492 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:51 smithi064 ceph-mon[34900]: from='client.? 172.21.15.64:0/48416633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1293332c-d772-4ca9-bf76-79e0b29451ee"}]: dispatch 2024-09-19T08:08:51.492 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:51 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1293332c-d772-4ca9-bf76-79e0b29451ee"}]: dispatch 2024-09-19T08:08:51.492 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:51 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1293332c-d772-4ca9-bf76-79e0b29451ee"}]': finished 2024-09-19T08:08:51.492 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:51 smithi064 ceph-mon[34900]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T08:08:51.492 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:51 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:08:51.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:51 smithi157 ceph-mon[35143]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T08:08:51.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:51 smithi157 ceph-mon[35143]: from='client.? 172.21.15.64:0/48416633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1293332c-d772-4ca9-bf76-79e0b29451ee"}]: dispatch 2024-09-19T08:08:51.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:51 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1293332c-d772-4ca9-bf76-79e0b29451ee"}]: dispatch 2024-09-19T08:08:51.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:51 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1293332c-d772-4ca9-bf76-79e0b29451ee"}]': finished 2024-09-19T08:08:51.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:51 smithi157 ceph-mon[35143]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T08:08:51.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:51 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:08:51.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:51 smithi022 ceph-mon[30963]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T08:08:51.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:51 smithi022 ceph-mon[30963]: from='client.? 172.21.15.64:0/48416633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1293332c-d772-4ca9-bf76-79e0b29451ee"}]: dispatch 2024-09-19T08:08:51.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:51 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1293332c-d772-4ca9-bf76-79e0b29451ee"}]: dispatch 2024-09-19T08:08:51.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:51 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1293332c-d772-4ca9-bf76-79e0b29451ee"}]': finished 2024-09-19T08:08:51.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:51 smithi022 ceph-mon[30963]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T08:08:51.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:51 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:08:52.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:52 smithi157 ceph-mon[35143]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:08:52.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:52 smithi157 ceph-mon[35143]: from='client.? 172.21.15.64:0/3607840118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:08:52.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:52 smithi157 ceph-mon[35143]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T08:08:52.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:52 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:08:52.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:52 smithi022 ceph-mon[30963]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:08:52.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:52 smithi022 ceph-mon[30963]: from='client.? 172.21.15.64:0/3607840118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:08:52.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:52 smithi022 ceph-mon[30963]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T08:08:52.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:52 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:08:52.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:52 smithi064 ceph-mon[34900]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:08:52.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:52 smithi064 ceph-mon[34900]: from='client.? 172.21.15.64:0/3607840118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:08:52.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:52 smithi064 ceph-mon[34900]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T08:08:52.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:52 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:08:54.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:53 smithi157 ceph-mon[35143]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:08:54.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:53 smithi157 ceph-mon[35143]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T08:08:54.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:08:54.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:53 smithi022 ceph-mon[30963]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:08:54.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:53 smithi022 ceph-mon[30963]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T08:08:54.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:08:54.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:53 smithi064 ceph-mon[34900]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:08:54.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:53 smithi064 ceph-mon[34900]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T08:08:54.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:08:56.069 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:55 smithi064 ceph-mon[34900]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-19T08:08:56.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:55 smithi157 ceph-mon[35143]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-19T08:08:56.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:55 smithi022 ceph-mon[30963]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-19T08:08:58.068 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:57 smithi064 ceph-mon[34900]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:08:58.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:57 smithi157 ceph-mon[35143]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:08:58.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:57 smithi022 ceph-mon[30963]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:09:00.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:59 smithi064 ceph-mon[34900]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T08:09:00.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:59 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T08:09:00.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:59 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:00.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:08:59 smithi064 ceph-mon[34900]: Deploying daemon osd.4 on smithi064 2024-09-19T08:09:00.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:59 smithi157 ceph-mon[35143]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T08:09:00.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:59 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T08:09:00.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:59 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:00.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:08:59 smithi157 ceph-mon[35143]: Deploying daemon osd.4 on smithi064 2024-09-19T08:09:00.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:59 smithi022 ceph-mon[30963]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T08:09:00.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:59 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T08:09:00.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:59 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:00.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:08:59 smithi022 ceph-mon[30963]: Deploying daemon osd.4 on smithi064 2024-09-19T08:09:02.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:01 smithi064 ceph-mon[34900]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-19T08:09:02.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:01 smithi157 ceph-mon[35143]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-19T08:09:02.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:01 smithi022 ceph-mon[30963]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-19T08:09:04.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:03 smithi064 ceph-mon[34900]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:09:04.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:03 smithi157 ceph-mon[35143]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:09:04.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:03 smithi022 ceph-mon[30963]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:09:06.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:05 smithi157 ceph-mon[35143]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T08:09:06.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:05 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:06.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:05 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:06.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:05 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:06.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:05 smithi022 ceph-mon[30963]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T08:09:06.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:05 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:06.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:05 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:06.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:05 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:06.268 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:05 smithi064 ceph-mon[34900]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T08:09:06.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:05 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:06.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:05 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:06.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:05 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:08.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:07 smithi064 ceph-mon[34900]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:09:08.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:07 smithi157 ceph-mon[35143]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:09:08.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:07 smithi022 ceph-mon[30963]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:09:09.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:09.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:09.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:09.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:09:09.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:09.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:09:09.313+0000 7fd44c8cf640 1 -- 172.21.15.64:0/1780660845 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd44806aef0 con 0x7fd4300a3240 2024-09-19T08:09:09.313 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 4 on host 'smithi064' 2024-09-19T08:09:09.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:09:09.315+0000 7fd44d8d1640 1 -- 172.21.15.64:0/1780660845 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd4300a3240 msgr2=0x7fd4300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:09:09.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:09:09.315+0000 7fd44d8d1640 1 --2- 172.21.15.64:0/1780660845 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd4300a3240 0x7fd4300a5700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd448104810 tx=0x7fd43c005f90 comp rx=0 tx=0).stop 2024-09-19T08:09:09.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:09:09.315+0000 7fd44d8d1640 1 -- 172.21.15.64:0/1780660845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd448100b00 msgr2=0x7fd448104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:09:09.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:09:09.315+0000 7fd44d8d1640 1 --2- 172.21.15.64:0/1780660845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd448100b00 0x7fd448104a90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd43800d770 tx=0x7fd43800dc30 comp rx=0 tx=0).stop 2024-09-19T08:09:09.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:09:09.315+0000 7fd44d8d1640 1 -- 172.21.15.64:0/1780660845 shutdown_connections 2024-09-19T08:09:09.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:09:09.315+0000 7fd44d8d1640 1 --2- 172.21.15.64:0/1780660845 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd4300a3240 0x7fd4300a5700 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:09:09.315+0000 7fd44d8d1640 1 --2- 172.21.15.64:0/1780660845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4481026c0 0x7fd448103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:09:09.315+0000 7fd44d8d1640 1 --2- 172.21.15.64:0/1780660845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd448101d00 0x7fd4481030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:09:09.315+0000 7fd44d8d1640 1 --2- 172.21.15.64:0/1780660845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd448100b00 0x7fd448104a90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:09:09.315+0000 7fd44d8d1640 1 -- 172.21.15.64:0/1780660845 >> 172.21.15.64:0/1780660845 conn(0x7fd4480fc290 msgr2=0x7fd44806a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:09:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:09:09.315+0000 7fd44d8d1640 1 -- 172.21.15.64:0/1780660845 shutdown_connections 2024-09-19T08:09:09.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:09:09.315+0000 7fd44d8d1640 1 -- 172.21.15.64:0/1780660845 wait complete. 2024-09-19T08:09:09.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:09.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:09.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:09.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:09:09.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:09.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:09.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:09.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:09.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:09:09.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:09.901 DEBUG:teuthology.orchestra.run.smithi064:osd.4> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T08:09:09.904 INFO:tasks.cephadm:Deploying osd.5 on smithi157 with /dev/vg_nvme/lv_3... 2024-09-19T08:09:09.904 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T08:09:10.161 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:09:10.180 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:10 smithi064 ceph-mon[34900]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:09:10.180 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:10.180 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:10.180 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:10.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:10 smithi157 ceph-mon[35143]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:09:10.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:10.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:10.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:10 smithi022 ceph-mon[30963]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:09:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:10.837 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:09:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4[45813]: 2024-09-19T08:09:10.526+0000 7f5f7bc4e640 -1 osd.4 0 log_to_monitors true 2024-09-19T08:09:11.305 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:11 smithi064 ceph-mon[34900]: from='osd.4 [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:09:11.305 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:11 smithi064 ceph-mon[34900]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:09:11.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:11 smithi157 ceph-mon[35143]: from='osd.4 [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:09:11.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:11 smithi157 ceph-mon[35143]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:09:11.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:11 smithi022 ceph-mon[30963]: from='osd.4 [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:09:11.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:11 smithi022 ceph-mon[30963]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:09:12.314 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:12 smithi157 ceph-mon[35143]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:09:12.314 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:12 smithi157 ceph-mon[35143]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T08:09:12.315 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:12 smithi157 ceph-mon[35143]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T08:09:12.315 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:12 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:09:12.315 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:12 smithi157 ceph-mon[35143]: from='osd.4 [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:09:12.315 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:12 smithi157 ceph-mon[35143]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:09:12.324 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:09:12 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4[45813]: 2024-09-19T08:09:12.108+0000 7f5f77bd1640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T08:09:12.325 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:09:12 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4[45813]: 2024-09-19T08:09:12.116+0000 7f5f739f9640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:09:12.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:12 smithi064 ceph-mon[34900]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:09:12.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:12 smithi064 ceph-mon[34900]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T08:09:12.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:12 smithi064 ceph-mon[34900]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T08:09:12.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:12 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:09:12.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:12 smithi064 ceph-mon[34900]: from='osd.4 [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:09:12.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:12 smithi064 ceph-mon[34900]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:09:12.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:12 smithi022 ceph-mon[30963]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:09:12.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:12 smithi022 ceph-mon[30963]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T08:09:12.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:12 smithi022 ceph-mon[30963]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T08:09:12.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:12 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:09:12.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:12 smithi022 ceph-mon[30963]: from='osd.4 [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:09:12.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:12 smithi022 ceph-mon[30963]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:09:12.613 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:09:12.639 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_3 2024-09-19T08:09:12.891 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:09:13.296 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:13 smithi157 ceph-mon[35143]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T08:09:13.296 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:13 smithi157 ceph-mon[35143]: osdmap e32: 5 total, 4 up, 5 in 2024-09-19T08:09:13.296 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:13 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:09:13.296 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:13 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:09:13.296 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:13 smithi157 ceph-mon[35143]: Detected new or changed devices on smithi064 2024-09-19T08:09:13.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:13 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:13.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:13 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:13.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:13 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:09:13.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:13 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:09:13.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:13 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:13.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:13 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:13.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:13 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:09:13.297 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:13 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:13.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.380+0000 7fd0acbed640 1 -- 172.21.15.157:0/4290730612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a81026d0 msgr2=0x7fd0a8108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:09:13.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.380+0000 7fd0acbed640 1 --2- 172.21.15.157:0/4290730612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a81026d0 0x7fd0a8108f60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd098009ca0 tx=0x7fd09802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:09:13.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.380+0000 7fd0acbed640 1 -- 172.21.15.157:0/4290730612 shutdown_connections 2024-09-19T08:09:13.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.380+0000 7fd0acbed640 1 --2- 172.21.15.157:0/4290730612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a81026d0 0x7fd0a8108f60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:13.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.380+0000 7fd0acbed640 1 --2- 172.21.15.157:0/4290730612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0a8101d10 0x7fd0a8102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:13.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.380+0000 7fd0acbed640 1 --2- 172.21.15.157:0/4290730612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a8100b10 0x7fd0a8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.380+0000 7fd0acbed640 1 -- 172.21.15.157:0/4290730612 >> 172.21.15.157:0/4290730612 conn(0x7fd0a80fc2a0 msgr2=0x7fd0a80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:09:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.380+0000 7fd0acbed640 1 -- 172.21.15.157:0/4290730612 shutdown_connections 2024-09-19T08:09:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.380+0000 7fd0acbed640 1 -- 172.21.15.157:0/4290730612 wait complete. 2024-09-19T08:09:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0acbed640 1 Processor -- start 2024-09-19T08:09:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0acbed640 1 -- start start 2024-09-19T08:09:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0acbed640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a8100b10 0x7fd0a819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:09:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0acbed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a8101d10 0x7fd0a819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:09:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0acbed640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0a81026d0 0x7fd0a81a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:09:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0acbed640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a810c770 con 0x7fd0a8101d10 2024-09-19T08:09:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0acbed640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a810c5f0 con 0x7fd0a81026d0 2024-09-19T08:09:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0acbed640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a810c8f0 con 0x7fd0a8100b10 2024-09-19T08:09:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0a659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a8100b10 0x7fd0a819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:09:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0a659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a8100b10 0x7fd0a819a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56798/0 (socket says 172.21.15.157:56798) 2024-09-19T08:09:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0a659c640 1 -- 172.21.15.157:0/3098103692 learned_addr learned my addr 172.21.15.157:0/3098103692 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T08:09:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0a6d9d640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0a81026d0 0x7fd0a81a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:09:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0a5d9b640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a8101d10 0x7fd0a819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:09:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0a659c640 1 -- 172.21.15.157:0/3098103692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0a81026d0 msgr2=0x7fd0a81a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:09:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0a659c640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0a81026d0 0x7fd0a81a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0a659c640 1 -- 172.21.15.157:0/3098103692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a8101d10 msgr2=0x7fd0a819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:09:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0a659c640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a8101d10 0x7fd0a819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0a659c640 1 -- 172.21.15.157:0/3098103692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a81a1c50 con 0x7fd0a8100b10 2024-09-19T08:09:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0a6d9d640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0a81026d0 0x7fd0a81a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:09:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.381+0000 7fd0a5d9b640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a8101d10 0x7fd0a819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:09:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.382+0000 7fd0a659c640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a8100b10 0x7fd0a819a020 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000d770 tx=0x7fd0a000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:09:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.382+0000 7fd09f7fe640 1 -- 172.21.15.157:0/3098103692 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0017070 con 0x7fd0a8100b10 2024-09-19T08:09:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.382+0000 7fd09f7fe640 1 -- 172.21.15.157:0/3098103692 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0a0009c30 con 0x7fd0a8100b10 2024-09-19T08:09:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.382+0000 7fd0acbed640 1 -- 172.21.15.157:0/3098103692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a81a1f40 con 0x7fd0a8100b10 2024-09-19T08:09:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.382+0000 7fd09f7fe640 1 -- 172.21.15.157:0/3098103692 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0003dd0 con 0x7fd0a8100b10 2024-09-19T08:09:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.382+0000 7fd0acbed640 1 -- 172.21.15.157:0/3098103692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a81a2390 con 0x7fd0a8100b10 2024-09-19T08:09:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.383+0000 7fd0acbed640 1 -- 172.21.15.157:0/3098103692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a81a25f0 con 0x7fd0a8100b10 2024-09-19T08:09:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.384+0000 7fd09f7fe640 1 -- 172.21.15.157:0/3098103692 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd0a0003f70 con 0x7fd0a8100b10 2024-09-19T08:09:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.385+0000 7fd09f7fe640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd0940a3270 0x7fd0940a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:09:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.385+0000 7fd09f7fe640 1 -- 172.21.15.157:0/3098103692 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7fd0a00ce2b0 con 0x7fd0a8100b10 2024-09-19T08:09:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.385+0000 7fd0a5d9b640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd0940a3270 0x7fd0940a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:09:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.386+0000 7fd0a5d9b640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd0940a3270 0x7fd0940a5730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd0a819b540 tx=0x7fd090005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:09:13.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.390+0000 7fd09f7fe640 1 -- 172.21.15.157:0/3098103692 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd0a009c040 con 0x7fd0a8100b10 2024-09-19T08:09:13.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:13 smithi022 ceph-mon[30963]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T08:09:13.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:13 smithi022 ceph-mon[30963]: osdmap e32: 5 total, 4 up, 5 in 2024-09-19T08:09:13.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:13 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:09:13.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:13 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:09:13.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:13 smithi022 ceph-mon[30963]: Detected new or changed devices on smithi064 2024-09-19T08:09:13.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:13 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:13.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:13 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:13.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:13 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:09:13.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:13 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:09:13.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:13 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:13.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:13 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:13.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:13 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:09:13.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:13 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:13.566 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:13.565+0000 7fd0acbed640 1 -- 172.21.15.157:0/3098103692 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd0a80fe460 con 0x7fd0940a3270 2024-09-19T08:09:13.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:13 smithi064 ceph-mon[34900]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T08:09:13.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:13 smithi064 ceph-mon[34900]: osdmap e32: 5 total, 4 up, 5 in 2024-09-19T08:09:13.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:13 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:09:13.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:13 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:09:13.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:13 smithi064 ceph-mon[34900]: Detected new or changed devices on smithi064 2024-09-19T08:09:13.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:13 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:13.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:13 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:13.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:13 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:09:13.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:13 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:09:13.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:13 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:13.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:13 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:13.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:13 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:09:13.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:13 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:14 smithi157 ceph-mon[35143]: purged_snaps scrub starts 2024-09-19T08:09:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:14 smithi157 ceph-mon[35143]: purged_snaps scrub ok 2024-09-19T08:09:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:14 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi064 to 8544M 2024-09-19T08:09:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:14 smithi157 ceph-mon[35143]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:09:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:14 smithi157 ceph-mon[35143]: osd.4 [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423] boot 2024-09-19T08:09:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:14 smithi157 ceph-mon[35143]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T08:09:14.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:14 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:09:14.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:14 smithi157 ceph-mon[35143]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:09:14.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:14 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:09:14.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:14 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:09:14.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:14 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:14 smithi022 ceph-mon[30963]: purged_snaps scrub starts 2024-09-19T08:09:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:14 smithi022 ceph-mon[30963]: purged_snaps scrub ok 2024-09-19T08:09:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:14 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi064 to 8544M 2024-09-19T08:09:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:14 smithi022 ceph-mon[30963]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:09:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:14 smithi022 ceph-mon[30963]: osd.4 [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423] boot 2024-09-19T08:09:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:14 smithi022 ceph-mon[30963]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T08:09:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:14 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:09:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:14 smithi022 ceph-mon[30963]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:09:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:14 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:09:14.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:14 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:09:14.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:14 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:14.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:14 smithi064 ceph-mon[34900]: purged_snaps scrub starts 2024-09-19T08:09:14.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:14 smithi064 ceph-mon[34900]: purged_snaps scrub ok 2024-09-19T08:09:14.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:14 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi064 to 8544M 2024-09-19T08:09:14.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:14 smithi064 ceph-mon[34900]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:09:14.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:14 smithi064 ceph-mon[34900]: osd.4 [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423] boot 2024-09-19T08:09:14.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:14 smithi064 ceph-mon[34900]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T08:09:14.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:14 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:09:14.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:14 smithi064 ceph-mon[34900]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:09:14.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:14 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:09:14.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:14 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:09:14.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:14 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:15.246 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:15 smithi157 ceph-mon[35143]: osdmap e34: 5 total, 5 up, 5 in 2024-09-19T08:09:15.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:15 smithi022 ceph-mon[30963]: osdmap e34: 5 total, 5 up, 5 in 2024-09-19T08:09:15.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:15 smithi064 ceph-mon[34900]: osdmap e34: 5 total, 5 up, 5 in 2024-09-19T08:09:16.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:16 smithi157 ceph-mon[35143]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:16.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:16 smithi157 ceph-mon[35143]: from='client.? 172.21.15.157:0/484494604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "240a6c42-89a2-41c1-948c-b09914b5722e"}]: dispatch 2024-09-19T08:09:16.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:16 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "240a6c42-89a2-41c1-948c-b09914b5722e"}]: dispatch 2024-09-19T08:09:16.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:16 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "240a6c42-89a2-41c1-948c-b09914b5722e"}]': finished 2024-09-19T08:09:16.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:16 smithi157 ceph-mon[35143]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T08:09:16.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:16 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:16.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:16 smithi022 ceph-mon[30963]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:16.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:16 smithi022 ceph-mon[30963]: from='client.? 172.21.15.157:0/484494604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "240a6c42-89a2-41c1-948c-b09914b5722e"}]: dispatch 2024-09-19T08:09:16.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:16 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "240a6c42-89a2-41c1-948c-b09914b5722e"}]: dispatch 2024-09-19T08:09:16.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:16 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "240a6c42-89a2-41c1-948c-b09914b5722e"}]': finished 2024-09-19T08:09:16.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:16 smithi022 ceph-mon[30963]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T08:09:16.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:16 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:16.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:16 smithi064 ceph-mon[34900]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:16.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:16 smithi064 ceph-mon[34900]: from='client.? 172.21.15.157:0/484494604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "240a6c42-89a2-41c1-948c-b09914b5722e"}]: dispatch 2024-09-19T08:09:16.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:16 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "240a6c42-89a2-41c1-948c-b09914b5722e"}]: dispatch 2024-09-19T08:09:16.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:16 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "240a6c42-89a2-41c1-948c-b09914b5722e"}]': finished 2024-09-19T08:09:16.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:16 smithi064 ceph-mon[34900]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T08:09:16.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:16 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:17.488 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:17 smithi064 ceph-mon[34900]: from='client.? 172.21.15.157:0/2604535234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:09:17.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:17 smithi157 ceph-mon[35143]: from='client.? 172.21.15.157:0/2604535234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:09:17.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:17 smithi022 ceph-mon[30963]: from='client.? 172.21.15.157:0/2604535234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:09:18.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:18 smithi157 ceph-mon[35143]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:18.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:18 smithi022 ceph-mon[30963]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:18.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:18 smithi064 ceph-mon[34900]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:20.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:20 smithi157 ceph-mon[35143]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:20.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:20 smithi022 ceph-mon[30963]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:20.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:20 smithi064 ceph-mon[34900]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:22.440 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:22 smithi157 ceph-mon[35143]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:22.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:22 smithi022 ceph-mon[30963]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:22.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:22 smithi064 ceph-mon[34900]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:24.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:24 smithi157 ceph-mon[35143]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:24.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:24 smithi022 ceph-mon[30963]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:24.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:24 smithi064 ceph-mon[34900]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T08:09:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:25 smithi157 ceph-mon[35143]: Deploying daemon osd.5 on smithi157 2024-09-19T08:09:25.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T08:09:25.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:25.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:25 smithi022 ceph-mon[30963]: Deploying daemon osd.5 on smithi157 2024-09-19T08:09:25.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T08:09:25.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:25.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:25 smithi064 ceph-mon[34900]: Deploying daemon osd.5 on smithi157 2024-09-19T08:09:26.230 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:26 smithi157 ceph-mon[35143]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:26.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:26 smithi022 ceph-mon[30963]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:26.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:26 smithi064 ceph-mon[34900]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:28.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:28 smithi157 ceph-mon[35143]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:28.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:28 smithi022 ceph-mon[30963]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:28.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:28 smithi064 ceph-mon[34900]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:30.207 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:30 smithi157 ceph-mon[35143]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:30.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:30 smithi022 ceph-mon[30963]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:30.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:30 smithi064 ceph-mon[34900]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:31.406 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:31 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:31.406 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:31 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:31.406 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:31 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:31.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:31 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:31.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:31 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:31.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:31 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:31.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:31 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:31.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:31 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:31.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:31 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:32.419 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:32 smithi157 ceph-mon[35143]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:32.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:32 smithi022 ceph-mon[30963]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:32.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:32 smithi064 ceph-mon[34900]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:34.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:34.211+0000 7fd09f7fe640 1 -- 172.21.15.157:0/3098103692 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd0a80fe460 con 0x7fd0940a3270 2024-09-19T08:09:34.214 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 5 on host 'smithi157' 2024-09-19T08:09:34.215 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:34.213+0000 7fd0acbed640 1 -- 172.21.15.157:0/3098103692 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd0940a3270 msgr2=0x7fd0940a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:09:34.215 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:34.213+0000 7fd0acbed640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd0940a3270 0x7fd0940a5730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd0a819b540 tx=0x7fd090005d90 comp rx=0 tx=0).stop 2024-09-19T08:09:34.215 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:34.213+0000 7fd0acbed640 1 -- 172.21.15.157:0/3098103692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a8100b10 msgr2=0x7fd0a819a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:09:34.215 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:34.213+0000 7fd0acbed640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a8100b10 0x7fd0a819a020 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000d770 tx=0x7fd0a000dc30 comp rx=0 tx=0).stop 2024-09-19T08:09:34.215 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:34.214+0000 7fd0acbed640 1 -- 172.21.15.157:0/3098103692 shutdown_connections 2024-09-19T08:09:34.215 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:34.214+0000 7fd0acbed640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd0940a3270 0x7fd0940a5730 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:34.215 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:34.214+0000 7fd0acbed640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0a81026d0 0x7fd0a81a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:34.216 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:34.214+0000 7fd0acbed640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a8101d10 0x7fd0a819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:34.216 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:34.214+0000 7fd0acbed640 1 --2- 172.21.15.157:0/3098103692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0a8100b10 0x7fd0a819a020 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:34.216 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:34.214+0000 7fd0acbed640 1 -- 172.21.15.157:0/3098103692 >> 172.21.15.157:0/3098103692 conn(0x7fd0a80fc2a0 msgr2=0x7fd0a80fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:09:34.216 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:34.214+0000 7fd0acbed640 1 -- 172.21.15.157:0/3098103692 shutdown_connections 2024-09-19T08:09:34.216 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:09:34.214+0000 7fd0acbed640 1 -- 172.21.15.157:0/3098103692 wait complete. 2024-09-19T08:09:34.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:34 smithi157 ceph-mon[35143]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:34.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:34 smithi022 ceph-mon[30963]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:34.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:34 smithi064 ceph-mon[34900]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:34.853 DEBUG:teuthology.orchestra.run.smithi157:osd.5> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T08:09:34.855 INFO:tasks.cephadm:Deploying osd.6 on smithi022 with /dev/vg_nvme/lv_2... 2024-09-19T08:09:34.856 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T08:09:35.125 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:09:35.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:35 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:35.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:35 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:35.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:35 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:35.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:35 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:35.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:35 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:35.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:35 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:35.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:35 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:35.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:35 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:35.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:35 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:35.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:35 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:35.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:35 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:35.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:35 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:36.442 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:36 smithi157 ceph-mon[35143]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:36.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:36 smithi022 ceph-mon[30963]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:36.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:36 smithi064 ceph-mon[34900]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:37.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:37 smithi022 ceph-mon[30963]: from='osd.5 [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:09:37.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:37 smithi022 ceph-mon[30963]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:09:37.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:37 smithi157 ceph-mon[35143]: from='osd.5 [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:09:37.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:37 smithi157 ceph-mon[35143]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:09:37.503 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:09:37 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:09:37.176+0000 7f32db44c640 -1 osd.5 0 log_to_monitors true 2024-09-19T08:09:37.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:37 smithi064 ceph-mon[34900]: from='osd.5 [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:09:37.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:37 smithi064 ceph-mon[34900]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:09:37.798 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:09:37.826 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_2 2024-09-19T08:09:38.092 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:09:38.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:38.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T08:09:38.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='osd.5 [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:09:38.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T08:09:38.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:38.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:09:38.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:09:38.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:09:38.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:38.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:09:38.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:38.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:38.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:09:38.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:38.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T08:09:38.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='osd.5 [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:09:38.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T08:09:38.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:38.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:09:38.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:09:38.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:09:38.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:38.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:09:38.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:38.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:38.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:09:38.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:38 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:38.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T08:09:38.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='osd.5 [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:09:38.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T08:09:38.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:38.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:09:38.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:09:38.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:09:38.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:38.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:09:38.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:38.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:38.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:09:38.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:38 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:38.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.712+0000 7f581e6f3640 1 -- 172.21.15.22:0/2168436001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5818100ad0 msgr2=0x7f5818100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:09:38.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.712+0000 7f581e6f3640 1 --2- 172.21.15.22:0/2168436001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5818100ad0 0x7f5818100ed0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5808009900 tx=0x7f580802a410 comp rx=0 tx=0).stop 2024-09-19T08:09:38.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.712+0000 7f581e6f3640 1 -- 172.21.15.22:0/2168436001 shutdown_connections 2024-09-19T08:09:38.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.712+0000 7f581e6f3640 1 --2- 172.21.15.22:0/2168436001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5818102690 0x7f5818108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.712+0000 7f581e6f3640 1 --2- 172.21.15.22:0/2168436001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5818101cd0 0x7f5818102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.712+0000 7f581e6f3640 1 --2- 172.21.15.22:0/2168436001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5818100ad0 0x7f5818100ed0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.712+0000 7f581e6f3640 1 -- 172.21.15.22:0/2168436001 >> 172.21.15.22:0/2168436001 conn(0x7f58180fc280 msgr2=0x7f58180fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:09:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.712+0000 7f581e6f3640 1 -- 172.21.15.22:0/2168436001 shutdown_connections 2024-09-19T08:09:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.712+0000 7f581e6f3640 1 -- 172.21.15.22:0/2168436001 wait complete. 2024-09-19T08:09:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.712+0000 7f581e6f3640 1 Processor -- start 2024-09-19T08:09:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f581e6f3640 1 -- start start 2024-09-19T08:09:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f581e6f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5818100ad0 0x7f581806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:09:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f581e6f3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5818101cd0 0x7f581819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:09:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f581e6f3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5818102690 0x7f581819f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:09:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f581e6f3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f581810c730 con 0x7f5818101cd0 2024-09-19T08:09:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f581e6f3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f581810c5b0 con 0x7f5818102690 2024-09-19T08:09:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f581e6f3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f581810c8b0 con 0x7f5818100ad0 2024-09-19T08:09:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f58177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5818101cd0 0x7f581819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:09:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f58177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5818101cd0 0x7f581819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53960/0 (socket says 172.21.15.22:53960) 2024-09-19T08:09:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f58177fe640 1 -- 172.21.15.22:0/1323794582 learned_addr learned my addr 172.21.15.22:0/1323794582 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:09:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f581cc90640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5818102690 0x7f581819f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:09:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f5817fff640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5818100ad0 0x7f581806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:09:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f58177fe640 1 -- 172.21.15.22:0/1323794582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5818100ad0 msgr2=0x7f581806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:09:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f58177fe640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5818100ad0 0x7f581806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f58177fe640 1 -- 172.21.15.22:0/1323794582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5818102690 msgr2=0x7f581819f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:09:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f58177fe640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5818102690 0x7f581819f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f58177fe640 1 -- 172.21.15.22:0/1323794582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581819fbe0 con 0x7f5818101cd0 2024-09-19T08:09:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f5817fff640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5818100ad0 0x7f581806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:09:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.713+0000 7f58177fe640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5818101cd0 0x7f581819ccb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f580c00c890 tx=0x7f580c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:09:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.714+0000 7f58157fa640 1 -- 172.21.15.22:0/1323794582 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c004220 con 0x7f5818101cd0 2024-09-19T08:09:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.714+0000 7f58157fa640 1 -- 172.21.15.22:0/1323794582 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f580c0043c0 con 0x7f5818101cd0 2024-09-19T08:09:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.714+0000 7f581e6f3640 1 -- 172.21.15.22:0/1323794582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581819fe70 con 0x7f5818101cd0 2024-09-19T08:09:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.714+0000 7f58157fa640 1 -- 172.21.15.22:0/1323794582 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c0049e0 con 0x7f5818101cd0 2024-09-19T08:09:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.714+0000 7f581e6f3640 1 -- 172.21.15.22:0/1323794582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58181a0370 con 0x7f5818101cd0 2024-09-19T08:09:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.715+0000 7f581e6f3640 1 -- 172.21.15.22:0/1323794582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5818107f70 con 0x7f5818101cd0 2024-09-19T08:09:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.716+0000 7f58157fa640 1 -- 172.21.15.22:0/1323794582 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f580c0079b0 con 0x7f5818101cd0 2024-09-19T08:09:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.717+0000 7f58157fa640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f58000a32c0 0x7f58000a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:09:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.717+0000 7f5817fff640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f58000a32c0 0x7f58000a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:09:38.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.717+0000 7f58157fa640 1 -- 172.21.15.22:0/1323794582 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4253+0+0 (secure 0 0 0) 0x7f580c0ce920 con 0x7f5818101cd0 2024-09-19T08:09:38.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.717+0000 7f5817fff640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f58000a32c0 0x7f58000a5780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5808006010 tx=0x7f5808032dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:09:38.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.721+0000 7f58157fa640 1 -- 172.21.15.22:0/1323794582 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f580c014030 con 0x7f5818101cd0 2024-09-19T08:09:38.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:38.910+0000 7f581e6f3640 1 -- 172.21.15.22:0/1323794582 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5818062630 con 0x7f58000a32c0 2024-09-19T08:09:39.003 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:09:38.619+0000 7f32d73cf640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T08:09:39.003 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:09:38 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:09:38.627+0000 7f32cf1ef640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:09:39.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:39 smithi022 ceph-mon[30963]: Detected new or changed devices on smithi157 2024-09-19T08:09:39.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:39 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi157 to 8544M 2024-09-19T08:09:39.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:39 smithi022 ceph-mon[30963]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T08:09:39.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:39 smithi022 ceph-mon[30963]: osdmap e37: 6 total, 5 up, 6 in 2024-09-19T08:09:39.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:39 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:39.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:39 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:39.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:39 smithi022 ceph-mon[30963]: from='osd.5 ' entity='osd.5' 2024-09-19T08:09:39.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:39 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:09:39.480 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:39 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:09:39.480 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:39 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:39.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:39 smithi157 ceph-mon[35143]: Detected new or changed devices on smithi157 2024-09-19T08:09:39.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:39 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi157 to 8544M 2024-09-19T08:09:39.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:39 smithi157 ceph-mon[35143]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T08:09:39.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:39 smithi157 ceph-mon[35143]: osdmap e37: 6 total, 5 up, 6 in 2024-09-19T08:09:39.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:39 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:39.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:39 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:39.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:39 smithi157 ceph-mon[35143]: from='osd.5 ' entity='osd.5' 2024-09-19T08:09:39.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:39 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:09:39.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:39 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:09:39.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:39 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:39.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:39 smithi064 ceph-mon[34900]: Detected new or changed devices on smithi157 2024-09-19T08:09:39.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:39 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi157 to 8544M 2024-09-19T08:09:39.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:39 smithi064 ceph-mon[34900]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T08:09:39.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:39 smithi064 ceph-mon[34900]: osdmap e37: 6 total, 5 up, 6 in 2024-09-19T08:09:39.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:39 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:39.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:39 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:39.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:39 smithi064 ceph-mon[34900]: from='osd.5 ' entity='osd.5' 2024-09-19T08:09:39.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:39 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:09:39.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:39 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:09:39.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:39 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:40.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:40 smithi022 ceph-mon[30963]: purged_snaps scrub starts 2024-09-19T08:09:40.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:40 smithi022 ceph-mon[30963]: purged_snaps scrub ok 2024-09-19T08:09:40.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:40 smithi022 ceph-mon[30963]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:40.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:40 smithi022 ceph-mon[30963]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:09:40.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:40 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:40.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:40 smithi022 ceph-mon[30963]: osd.5 [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212] boot 2024-09-19T08:09:40.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:40 smithi022 ceph-mon[30963]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T08:09:40.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:40 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:40.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:40 smithi022 ceph-mon[30963]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T08:09:40.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:40 smithi157 ceph-mon[35143]: purged_snaps scrub starts 2024-09-19T08:09:40.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:40 smithi157 ceph-mon[35143]: purged_snaps scrub ok 2024-09-19T08:09:40.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:40 smithi157 ceph-mon[35143]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:40.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:40 smithi157 ceph-mon[35143]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:09:40.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:40 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:40.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:40 smithi157 ceph-mon[35143]: osd.5 [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212] boot 2024-09-19T08:09:40.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:40 smithi157 ceph-mon[35143]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T08:09:40.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:40 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:40.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:40 smithi157 ceph-mon[35143]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T08:09:40.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:40 smithi064 ceph-mon[34900]: purged_snaps scrub starts 2024-09-19T08:09:40.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:40 smithi064 ceph-mon[34900]: purged_snaps scrub ok 2024-09-19T08:09:40.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:40 smithi064 ceph-mon[34900]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:09:40.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:40 smithi064 ceph-mon[34900]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:09:40.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:40 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:40.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:40 smithi064 ceph-mon[34900]: osd.5 [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212] boot 2024-09-19T08:09:40.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:40 smithi064 ceph-mon[34900]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T08:09:40.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:40 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:09:40.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:40 smithi064 ceph-mon[34900]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T08:09:42.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:42 smithi157 ceph-mon[35143]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:09:42.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:42 smithi157 ceph-mon[35143]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T08:09:42.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:42 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1610469474' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80199030-109e-41c4-bc1c-731755464770"}]: dispatch 2024-09-19T08:09:42.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:42 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1610469474' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80199030-109e-41c4-bc1c-731755464770"}]': finished 2024-09-19T08:09:42.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:42 smithi157 ceph-mon[35143]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T08:09:42.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:42 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:09:42.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:42 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/427153668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:09:42.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:42 smithi022 ceph-mon[30963]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:09:42.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:42 smithi022 ceph-mon[30963]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T08:09:42.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:42 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1610469474' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80199030-109e-41c4-bc1c-731755464770"}]: dispatch 2024-09-19T08:09:42.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:42 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1610469474' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80199030-109e-41c4-bc1c-731755464770"}]': finished 2024-09-19T08:09:42.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:42 smithi022 ceph-mon[30963]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T08:09:42.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:42 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:09:42.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:42 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/427153668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:09:42.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:42 smithi064 ceph-mon[34900]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:09:42.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:42 smithi064 ceph-mon[34900]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T08:09:42.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:42 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1610469474' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80199030-109e-41c4-bc1c-731755464770"}]: dispatch 2024-09-19T08:09:42.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:42 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1610469474' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80199030-109e-41c4-bc1c-731755464770"}]': finished 2024-09-19T08:09:42.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:42 smithi064 ceph-mon[34900]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T08:09:42.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:42 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:09:42.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:42 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/427153668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:09:44.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:44 smithi157 ceph-mon[35143]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:09:44.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:44 smithi157 ceph-mon[35143]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T08:09:44.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:44 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:09:44.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:44 smithi022 ceph-mon[30963]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:09:44.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:44 smithi022 ceph-mon[30963]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T08:09:44.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:44 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:09:44.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:44 smithi064 ceph-mon[34900]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:09:44.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:44 smithi064 ceph-mon[34900]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T08:09:44.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:44 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:09:45.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:45 smithi157 ceph-mon[35143]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T08:09:45.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:09:45.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:45 smithi022 ceph-mon[30963]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T08:09:45.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:09:45.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:45 smithi064 ceph-mon[34900]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T08:09:45.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:09:46.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:46 smithi157 ceph-mon[35143]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T08:09:46.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:46 smithi022 ceph-mon[30963]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T08:09:46.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:46 smithi064 ceph-mon[34900]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T08:09:48.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:48 smithi022 ceph-mon[30963]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T08:09:48.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:48 smithi157 ceph-mon[35143]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T08:09:48.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:48 smithi064 ceph-mon[34900]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T08:09:50.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:50 smithi157 ceph-mon[35143]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-19T08:09:50.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:50 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T08:09:50.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:50 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:50.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:50 smithi022 ceph-mon[30963]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-19T08:09:50.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:50 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T08:09:50.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:50 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:50.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:50 smithi064 ceph-mon[34900]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-19T08:09:50.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T08:09:50.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:50 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:09:51.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:51 smithi157 ceph-mon[35143]: Deploying daemon osd.6 on smithi022 2024-09-19T08:09:51.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:51 smithi022 ceph-mon[30963]: Deploying daemon osd.6 on smithi022 2024-09-19T08:09:51.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:51 smithi064 ceph-mon[34900]: Deploying daemon osd.6 on smithi022 2024-09-19T08:09:52.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:52 smithi157 ceph-mon[35143]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:09:52.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:52 smithi022 ceph-mon[30963]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:09:52.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:52 smithi064 ceph-mon[34900]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:09:54.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:54 smithi157 ceph-mon[35143]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T08:09:54.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:54 smithi022 ceph-mon[30963]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T08:09:54.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:54 smithi064 ceph-mon[34900]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T08:09:56.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:56 smithi157 ceph-mon[35143]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:09:56.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:56 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:56.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:56 smithi022 ceph-mon[30963]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:09:56.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:56 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:56.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:56 smithi064 ceph-mon[34900]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:09:56.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:56 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:09:57.421 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:57 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:57.422 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:57 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:57.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:57 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:57.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:57 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:57.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:57.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:57 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:09:58.385 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:09:58 smithi022 ceph-mon[30963]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:09:58.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:09:58 smithi064 ceph-mon[34900]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:09:58.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:09:58 smithi157 ceph-mon[35143]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:09:59.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:59.668+0000 7f58157fa640 1 -- 172.21.15.22:0/1323794582 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5818062630 con 0x7f58000a32c0 2024-09-19T08:09:59.669 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 6 on host 'smithi022' 2024-09-19T08:09:59.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:59.671+0000 7f581e6f3640 1 -- 172.21.15.22:0/1323794582 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f58000a32c0 msgr2=0x7f58000a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:09:59.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:59.671+0000 7f581e6f3640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f58000a32c0 0x7f58000a5780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5808006010 tx=0x7f5808032dd0 comp rx=0 tx=0).stop 2024-09-19T08:09:59.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:59.671+0000 7f581e6f3640 1 -- 172.21.15.22:0/1323794582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5818101cd0 msgr2=0x7f581819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:09:59.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:59.671+0000 7f581e6f3640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5818101cd0 0x7f581819ccb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f580c00c890 tx=0x7f580c00cd50 comp rx=0 tx=0).stop 2024-09-19T08:09:59.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:59.671+0000 7f581e6f3640 1 -- 172.21.15.22:0/1323794582 shutdown_connections 2024-09-19T08:09:59.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:59.671+0000 7f581e6f3640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f58000a32c0 0x7f58000a5780 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:59.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:59.671+0000 7f581e6f3640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5818102690 0x7f581819f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:59.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:59.671+0000 7f581e6f3640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5818101cd0 0x7f581819ccb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:59.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:59.671+0000 7f581e6f3640 1 --2- 172.21.15.22:0/1323794582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5818100ad0 0x7f581806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:09:59.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:59.671+0000 7f581e6f3640 1 -- 172.21.15.22:0/1323794582 >> 172.21.15.22:0/1323794582 conn(0x7f58180fc280 msgr2=0x7f5818105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:09:59.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:59.671+0000 7f581e6f3640 1 -- 172.21.15.22:0/1323794582 shutdown_connections 2024-09-19T08:09:59.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:09:59.671+0000 7f581e6f3640 1 -- 172.21.15.22:0/1323794582 wait complete. 2024-09-19T08:10:00.326 DEBUG:teuthology.orchestra.run.smithi022:osd.6> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T08:10:00.329 INFO:tasks.cephadm:Deploying osd.7 on smithi064 with /dev/vg_nvme/lv_2... 2024-09-19T08:10:00.329 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T08:10:00.593 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:10:00.972 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:00 smithi022 ceph-mon[30963]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:10:00.972 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:00 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:00.973 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:00 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:00.973 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:00 smithi022 ceph-mon[30963]: overall HEALTH_OK 2024-09-19T08:10:00.973 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:00 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:00.973 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:00 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:00.976 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:00 smithi064 ceph-mon[34900]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:10:00.976 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:00 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:00.976 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:00 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:00.977 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:00 smithi064 ceph-mon[34900]: overall HEALTH_OK 2024-09-19T08:10:00.977 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:00 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:00.977 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:00 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:01.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:00 smithi157 ceph-mon[35143]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:10:01.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:00 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:01.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:00 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:01.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:00 smithi157 ceph-mon[35143]: overall HEALTH_OK 2024-09-19T08:10:01.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:00 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:01.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:00 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:02.881 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:02 smithi064 ceph-mon[34900]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:10:02.881 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:02 smithi064 ceph-mon[34900]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T08:10:02.957 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:02 smithi022 ceph-mon[30963]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:10:02.957 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:02 smithi022 ceph-mon[30963]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T08:10:02.957 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:10:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:10:02.576+0000 7f79d6829640 -1 osd.6 0 log_to_monitors true 2024-09-19T08:10:03.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:02 smithi157 ceph-mon[35143]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:10:03.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:02 smithi157 ceph-mon[35143]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T08:10:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:10:03.441 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_2 2024-09-19T08:10:03.690 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:10:03.945 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:03 smithi064 ceph-mon[34900]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T08:10:03.945 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:03 smithi064 ceph-mon[34900]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T08:10:03.945 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:03 smithi064 ceph-mon[34900]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-19T08:10:03.945 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:03 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:04.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:03 smithi157 ceph-mon[35143]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T08:10:04.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:03 smithi157 ceph-mon[35143]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T08:10:04.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:03 smithi157 ceph-mon[35143]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-19T08:10:04.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:03 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:04.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:03 smithi022 ceph-mon[30963]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T08:10:04.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:03 smithi022 ceph-mon[30963]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T08:10:04.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:03 smithi022 ceph-mon[30963]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-19T08:10:04.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:03 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:04.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.154+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/834297857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc59c102690 msgr2=0x7fc59c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:10:04.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.154+0000 7fc5a1ad6640 1 --2- 172.21.15.64:0/834297857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc59c102690 0x7fc59c108f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc594009ca0 tx=0x7fc59402a8c0 comp rx=0 tx=0).stop 2024-09-19T08:10:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.154+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/834297857 shutdown_connections 2024-09-19T08:10:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.154+0000 7fc5a1ad6640 1 --2- 172.21.15.64:0/834297857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc59c102690 0x7fc59c108f20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.154+0000 7fc5a1ad6640 1 --2- 172.21.15.64:0/834297857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc59c101cd0 0x7fc59c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.154+0000 7fc5a1ad6640 1 --2- 172.21.15.64:0/834297857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc59c100ad0 0x7fc59c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.154+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/834297857 >> 172.21.15.64:0/834297857 conn(0x7fc59c0fc280 msgr2=0x7fc59c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:10:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.154+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/834297857 shutdown_connections 2024-09-19T08:10:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.154+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/834297857 wait complete. 2024-09-19T08:10:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.154+0000 7fc5a1ad6640 1 Processor -- start 2024-09-19T08:10:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.154+0000 7fc5a1ad6640 1 -- start start 2024-09-19T08:10:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc5a1ad6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc59c100ad0 0x7fc59c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:10:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc5a1ad6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc59c101cd0 0x7fc59c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:10:04.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc5a1ad6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc59c102690 0x7fc59c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:10:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc5a1ad6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c10d880 con 0x7fc59c100ad0 2024-09-19T08:10:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc5a1ad6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c10d700 con 0x7fc59c101cd0 2024-09-19T08:10:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc5a1ad6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c10da00 con 0x7fc59c102690 2024-09-19T08:10:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc593fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc59c101cd0 0x7fc59c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:10:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc593fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc59c101cd0 0x7fc59c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60550/0 (socket says 172.21.15.64:60550) 2024-09-19T08:10:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc593fff640 1 -- 172.21.15.64:0/573558843 learned_addr learned my addr 172.21.15.64:0/573558843 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T08:10:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc59b7fe640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc59c100ad0 0x7fc59c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:10:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc59bfff640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc59c102690 0x7fc59c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:10:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc59b7fe640 1 -- 172.21.15.64:0/573558843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc59c102690 msgr2=0x7fc59c19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:10:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc59b7fe640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc59c102690 0x7fc59c19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc59b7fe640 1 -- 172.21.15.64:0/573558843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc59c101cd0 msgr2=0x7fc59c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:10:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc59b7fe640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc59c101cd0 0x7fc59c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc59b7fe640 1 -- 172.21.15.64:0/573558843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c19faa0 con 0x7fc59c100ad0 2024-09-19T08:10:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc593fff640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc59c101cd0 0x7fc59c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:10:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc59b7fe640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc59c100ad0 0x7fc59c197e70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00e9c0 tx=0x7fc58c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:10:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc5997fa640 1 -- 172.21.15.64:0/573558843 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c00ccc0 con 0x7fc59c100ad0 2024-09-19T08:10:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/573558843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59c19fd90 con 0x7fc59c100ad0 2024-09-19T08:10:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc5997fa640 1 -- 172.21.15.64:0/573558843 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc58c0035f0 con 0x7fc59c100ad0 2024-09-19T08:10:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.155+0000 7fc5997fa640 1 -- 172.21.15.64:0/573558843 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c00ce60 con 0x7fc59c100ad0 2024-09-19T08:10:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.156+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/573558843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc59c1a0210 con 0x7fc59c100ad0 2024-09-19T08:10:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.157+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/573558843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc59c107ed0 con 0x7fc59c100ad0 2024-09-19T08:10:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.158+0000 7fc5997fa640 1 -- 172.21.15.64:0/573558843 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc58c0037a0 con 0x7fc59c100ad0 2024-09-19T08:10:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.158+0000 7fc5997fa640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc5880a3270 0x7fc5880a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:10:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.158+0000 7fc5997fa640 1 -- 172.21.15.64:0/573558843 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4685+0+0 (secure 0 0 0) 0x7fc58c0cf210 con 0x7fc59c100ad0 2024-09-19T08:10:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.158+0000 7fc593fff640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc5880a3270 0x7fc5880a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:10:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.159+0000 7fc593fff640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc5880a3270 0x7fc5880a5730 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc59c199390 tx=0x7fc58400b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:10:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.162+0000 7fc5997fa640 1 -- 172.21.15.64:0/573558843 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc58c09cd40 con 0x7fc59c100ad0 2024-09-19T08:10:04.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:04.347+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/573558843 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc59c062630 con 0x7fc5880a3270 2024-09-19T08:10:04.507 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:10:04.114+0000 7f79d27ac640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T08:10:04.507 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:10:04.122+0000 7f79ca5cc640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:10:04.995 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:10:04.995 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-19T08:10:04.995 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: osdmap e45: 7 total, 6 up, 7 in 2024-09-19T08:10:04.995 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:04.995 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:04.995 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:04.996 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:04.996 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:04.996 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:04.996 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:04.996 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:04.996 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:04.996 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:04.996 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:04.996 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:04.996 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:04.996 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:04.996 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:04.997 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' 2024-09-19T08:10:04.997 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:10:04.997 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:10:04.997 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:04 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:05.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:10:05.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-19T08:10:05.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: osdmap e45: 7 total, 6 up, 7 in 2024-09-19T08:10:05.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:05.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:05.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:05.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:05.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:05.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:05.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:05.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:05.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:05.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:05.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:05.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:05.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:05.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:05.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:05.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' 2024-09-19T08:10:05.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:10:05.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:10:05.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:04 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:05.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:10:05.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-19T08:10:05.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: osdmap e45: 7 total, 6 up, 7 in 2024-09-19T08:10:05.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:05.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:05.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242]' entity='osd.6' 2024-09-19T08:10:05.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:10:05.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:10:05.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:04 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:05.748 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:05 smithi064 ceph-mon[34900]: purged_snaps scrub starts 2024-09-19T08:10:05.749 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:05 smithi064 ceph-mon[34900]: purged_snaps scrub ok 2024-09-19T08:10:05.749 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:05 smithi064 ceph-mon[34900]: Detected new or changed devices on smithi022 2024-09-19T08:10:05.749 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:05 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi022 to 5697M 2024-09-19T08:10:05.749 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:05 smithi064 ceph-mon[34900]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:10:05.749 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:05 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:05.749 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:05 smithi064 ceph-mon[34900]: osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242] boot 2024-09-19T08:10:05.749 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:05 smithi064 ceph-mon[34900]: osdmap e46: 7 total, 7 up, 7 in 2024-09-19T08:10:05.749 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:05 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:06.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:05 smithi157 ceph-mon[35143]: purged_snaps scrub starts 2024-09-19T08:10:06.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:05 smithi157 ceph-mon[35143]: purged_snaps scrub ok 2024-09-19T08:10:06.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:05 smithi157 ceph-mon[35143]: Detected new or changed devices on smithi022 2024-09-19T08:10:06.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:05 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi022 to 5697M 2024-09-19T08:10:06.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:05 smithi157 ceph-mon[35143]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:10:06.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:05 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:06.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:05 smithi157 ceph-mon[35143]: osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242] boot 2024-09-19T08:10:06.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:05 smithi157 ceph-mon[35143]: osdmap e46: 7 total, 7 up, 7 in 2024-09-19T08:10:06.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:05 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:06.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:05 smithi022 ceph-mon[30963]: purged_snaps scrub starts 2024-09-19T08:10:06.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:05 smithi022 ceph-mon[30963]: purged_snaps scrub ok 2024-09-19T08:10:06.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:05 smithi022 ceph-mon[30963]: Detected new or changed devices on smithi022 2024-09-19T08:10:06.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:05 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi022 to 5697M 2024-09-19T08:10:06.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:05 smithi022 ceph-mon[30963]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:10:06.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:05 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:06.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:05 smithi022 ceph-mon[30963]: osd.6 [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242] boot 2024-09-19T08:10:06.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:05 smithi022 ceph-mon[30963]: osdmap e46: 7 total, 7 up, 7 in 2024-09-19T08:10:06.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:05 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:10:06.996 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:06 smithi064 ceph-mon[34900]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:10:06.996 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:06 smithi064 ceph-mon[34900]: osdmap e47: 7 total, 7 up, 7 in 2024-09-19T08:10:07.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:06 smithi157 ceph-mon[35143]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:10:07.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:06 smithi157 ceph-mon[35143]: osdmap e47: 7 total, 7 up, 7 in 2024-09-19T08:10:07.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:06 smithi022 ceph-mon[30963]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:10:07.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:06 smithi022 ceph-mon[30963]: osdmap e47: 7 total, 7 up, 7 in 2024-09-19T08:10:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:07 smithi157 ceph-mon[35143]: from='client.? 172.21.15.64:0/1058516698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb"}]: dispatch 2024-09-19T08:10:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:07 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb"}]: dispatch 2024-09-19T08:10:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:07 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb"}]': finished 2024-09-19T08:10:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:07 smithi157 ceph-mon[35143]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T08:10:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:07 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:07 smithi157 ceph-mon[35143]: from='client.? 172.21.15.64:0/2887634367' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:10:08.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:07 smithi022 ceph-mon[30963]: from='client.? 172.21.15.64:0/1058516698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb"}]: dispatch 2024-09-19T08:10:08.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:07 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb"}]: dispatch 2024-09-19T08:10:08.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:07 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb"}]': finished 2024-09-19T08:10:08.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:07 smithi022 ceph-mon[30963]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T08:10:08.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:07 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:08.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:07 smithi022 ceph-mon[30963]: from='client.? 172.21.15.64:0/2887634367' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:10:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:07 smithi064 ceph-mon[34900]: from='client.? 172.21.15.64:0/1058516698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb"}]: dispatch 2024-09-19T08:10:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:07 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb"}]: dispatch 2024-09-19T08:10:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:07 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb"}]': finished 2024-09-19T08:10:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:07 smithi064 ceph-mon[34900]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T08:10:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:07 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:07 smithi064 ceph-mon[34900]: from='client.? 172.21.15.64:0/2887634367' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:10:09.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:08 smithi157 ceph-mon[35143]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:09.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:08 smithi022 ceph-mon[30963]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:09.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:08 smithi064 ceph-mon[34900]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:11.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:10 smithi157 ceph-mon[35143]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:11.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:10 smithi022 ceph-mon[30963]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:11.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:10 smithi064 ceph-mon[34900]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:12.947 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:12 smithi064 ceph-mon[34900]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:13.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:12 smithi157 ceph-mon[35143]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:13.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:12 smithi022 ceph-mon[30963]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:15.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:14 smithi157 ceph-mon[35143]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:15.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:14 smithi022 ceph-mon[30963]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:15.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:14 smithi064 ceph-mon[34900]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:15.942 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:15 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T08:10:15.942 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:15 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:16.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:15 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T08:10:16.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:15 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:16.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:15 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T08:10:16.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:15 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:16.950 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:16 smithi064 ceph-mon[34900]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:16.951 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:16 smithi064 ceph-mon[34900]: Deploying daemon osd.7 on smithi064 2024-09-19T08:10:17.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:16 smithi157 ceph-mon[35143]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:17.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:16 smithi157 ceph-mon[35143]: Deploying daemon osd.7 on smithi064 2024-09-19T08:10:17.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:16 smithi022 ceph-mon[30963]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:17.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:16 smithi022 ceph-mon[30963]: Deploying daemon osd.7 on smithi064 2024-09-19T08:10:18.974 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:18 smithi064 ceph-mon[34900]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:19.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:18 smithi157 ceph-mon[35143]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:19.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:18 smithi022 ceph-mon[30963]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:21.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:20 smithi157 ceph-mon[35143]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:21.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:20 smithi022 ceph-mon[30963]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:21.048 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:20 smithi064 ceph-mon[34900]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:22.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:21 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:22.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:21 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:22.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:21 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:22.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:21 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:22.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:21 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:22.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:21 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:22.051 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:21 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:22.051 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:21 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:22.051 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:21 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:23.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:22 smithi157 ceph-mon[35143]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:23.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:22 smithi022 ceph-mon[30963]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:23.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:22 smithi064 ceph-mon[34900]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:24.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:23 smithi157 ceph-mon[35143]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:24.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:23 smithi022 ceph-mon[30963]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:24.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:23 smithi064 ceph-mon[34900]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:24.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:24.899+0000 7fc5997fa640 1 -- 172.21.15.64:0/573558843 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc59c062630 con 0x7fc5880a3270 2024-09-19T08:10:24.899 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 7 on host 'smithi064' 2024-09-19T08:10:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:24.901+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/573558843 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc5880a3270 msgr2=0x7fc5880a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:10:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:24.901+0000 7fc5a1ad6640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc5880a3270 0x7fc5880a5730 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc59c199390 tx=0x7fc58400b480 comp rx=0 tx=0).stop 2024-09-19T08:10:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:24.901+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/573558843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc59c100ad0 msgr2=0x7fc59c197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:10:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:24.901+0000 7fc5a1ad6640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc59c100ad0 0x7fc59c197e70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00e9c0 tx=0x7fc58c00ee80 comp rx=0 tx=0).stop 2024-09-19T08:10:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:24.901+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/573558843 shutdown_connections 2024-09-19T08:10:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:24.901+0000 7fc5a1ad6640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc5880a3270 0x7fc5880a5730 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:24.901+0000 7fc5a1ad6640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc59c102690 0x7fc59c19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:24.901+0000 7fc5a1ad6640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc59c101cd0 0x7fc59c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:24.901+0000 7fc5a1ad6640 1 --2- 172.21.15.64:0/573558843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc59c100ad0 0x7fc59c197e70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:24.901+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/573558843 >> 172.21.15.64:0/573558843 conn(0x7fc59c0fc280 msgr2=0x7fc59c1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:10:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:24.901+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/573558843 shutdown_connections 2024-09-19T08:10:24.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:10:24.901+0000 7fc5a1ad6640 1 -- 172.21.15.64:0/573558843 wait complete. 2024-09-19T08:10:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:24 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:24 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:24 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:24 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:24 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:24 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:24 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:24 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:24 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:24 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:24 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:25.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:24 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:25.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:24 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:24 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:25.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:24 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:24 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:24 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:24 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:24 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:25.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:24 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:25.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:24 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:24 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:25.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:24 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:24 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:25.613 DEBUG:teuthology.orchestra.run.smithi064:osd.7> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T08:10:25.615 INFO:tasks.cephadm:Deploying osd.8 on smithi157 with /dev/vg_nvme/lv_2... 2024-09-19T08:10:25.615 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T08:10:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:10:26.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:25 smithi157 ceph-mon[35143]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:26.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:25 smithi022 ceph-mon[30963]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:26.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:25 smithi064 ceph-mon[34900]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:27.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:26 smithi157 ceph-mon[35143]: from='osd.7 [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:10:27.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:26 smithi157 ceph-mon[35143]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:10:27.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:26 smithi064 ceph-mon[34900]: from='osd.7 [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:10:27.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:26 smithi064 ceph-mon[34900]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:10:27.032 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:10:26 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:10:26.852+0000 7f759e114640 -1 osd.7 0 log_to_monitors true 2024-09-19T08:10:27.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:26 smithi022 ceph-mon[30963]: from='osd.7 [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:10:27.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:26 smithi022 ceph-mon[30963]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:10:28.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:27 smithi157 ceph-mon[35143]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:28.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:27 smithi157 ceph-mon[35143]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T08:10:28.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:27 smithi157 ceph-mon[35143]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T08:10:28.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:27 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:28.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:27 smithi157 ceph-mon[35143]: from='osd.7 [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:10:28.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:27 smithi157 ceph-mon[35143]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:10:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:27 smithi022 ceph-mon[30963]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:27 smithi022 ceph-mon[30963]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T08:10:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:27 smithi022 ceph-mon[30963]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T08:10:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:27 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:27 smithi022 ceph-mon[30963]: from='osd.7 [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:10:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:27 smithi022 ceph-mon[30963]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:10:28.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:27 smithi064 ceph-mon[34900]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:28.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:27 smithi064 ceph-mon[34900]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T08:10:28.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:27 smithi064 ceph-mon[34900]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T08:10:28.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:27 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:28.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:27 smithi064 ceph-mon[34900]: from='osd.7 [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:10:28.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:27 smithi064 ceph-mon[34900]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:10:28.337 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:10:27 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:10:27.991+0000 7f759a097640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T08:10:28.338 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:10:28.000+0000 7f7595ebf640 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:10:28.731 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:10:28.758 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_2 2024-09-19T08:10:29.026 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:10:29.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T08:10:29.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: osdmap e50: 8 total, 7 up, 8 in 2024-09-19T08:10:29.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:29.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:29.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: Detected new or changed devices on smithi064 2024-09-19T08:10:29.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:29.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:29.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:29.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:29.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:29.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi064 to 5696M 2024-09-19T08:10:29.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:29.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:29.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:29.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:28 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:29.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T08:10:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: osdmap e50: 8 total, 7 up, 8 in 2024-09-19T08:10:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: Detected new or changed devices on smithi064 2024-09-19T08:10:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi064 to 5696M 2024-09-19T08:10:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:29.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:29.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:28 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:29.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T08:10:29.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: osdmap e50: 8 total, 7 up, 8 in 2024-09-19T08:10:29.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:29.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:29.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: Detected new or changed devices on smithi064 2024-09-19T08:10:29.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:29.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:29.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:29.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:29.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:29.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi064 to 5696M 2024-09-19T08:10:29.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:29.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:29.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:29.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:28 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:29.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.524+0000 7f739ae81640 1 -- 172.21.15.157:0/878833987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7394102690 msgr2=0x7f7394108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:10:29.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.524+0000 7f739ae81640 1 --2- 172.21.15.157:0/878833987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7394102690 0x7f7394108f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f738c009900 tx=0x7f738c02a410 comp rx=0 tx=0).stop 2024-09-19T08:10:29.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.524+0000 7f739ae81640 1 -- 172.21.15.157:0/878833987 shutdown_connections 2024-09-19T08:10:29.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.524+0000 7f739ae81640 1 --2- 172.21.15.157:0/878833987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7394102690 0x7f7394108f20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:29.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.524+0000 7f739ae81640 1 --2- 172.21.15.157:0/878833987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7394101cd0 0x7f7394102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:29.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.524+0000 7f739ae81640 1 --2- 172.21.15.157:0/878833987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7394100ad0 0x7f7394100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:29.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.524+0000 7f739ae81640 1 -- 172.21.15.157:0/878833987 >> 172.21.15.157:0/878833987 conn(0x7f73940fc280 msgr2=0x7f73940fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:10:29.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.524+0000 7f739ae81640 1 -- 172.21.15.157:0/878833987 shutdown_connections 2024-09-19T08:10:29.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.524+0000 7f739ae81640 1 -- 172.21.15.157:0/878833987 wait complete. 2024-09-19T08:10:29.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.525+0000 7f739ae81640 1 Processor -- start 2024-09-19T08:10:29.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.525+0000 7f739ae81640 1 -- start start 2024-09-19T08:10:29.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.525+0000 7f739ae81640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7394100ad0 0x7f739406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:10:29.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.525+0000 7f739ae81640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7394101cd0 0x7f739419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:10:29.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.525+0000 7f739ae81640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7394102690 0x7f739419f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:10:29.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.525+0000 7f739ae81640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f739410c750 con 0x7f7394100ad0 2024-09-19T08:10:29.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.525+0000 7f739ae81640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f739410c5d0 con 0x7f7394101cd0 2024-09-19T08:10:29.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.525+0000 7f739ae81640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f739410c8d0 con 0x7f7394102690 2024-09-19T08:10:29.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.525+0000 7f739941e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7394102690 0x7f739419f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:10:29.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.525+0000 7f739941e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7394102690 0x7f739419f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54918/0 (socket says 172.21.15.157:54918) 2024-09-19T08:10:29.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f739941e640 1 -- 172.21.15.157:0/2861259466 learned_addr learned my addr 172.21.15.157:0/2861259466 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T08:10:29.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f7398c1d640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7394100ad0 0x7f739406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:10:29.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f7393fff640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7394101cd0 0x7f739419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:10:29.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f739941e640 1 -- 172.21.15.157:0/2861259466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7394101cd0 msgr2=0x7f739419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:10:29.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f739941e640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7394101cd0 0x7f739419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:29.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f739941e640 1 -- 172.21.15.157:0/2861259466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7394100ad0 msgr2=0x7f739406ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:10:29.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f739941e640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7394100ad0 0x7f739406ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:29.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f739941e640 1 -- 172.21.15.157:0/2861259466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739419faa0 con 0x7f7394102690 2024-09-19T08:10:29.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f7398c1d640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7394100ad0 0x7f739406ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:10:29.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f739941e640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7394102690 0x7f739419f1e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f738c0314e0 tx=0x7f738c031b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:10:29.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f7391ffb640 1 -- 172.21.15.157:0/2861259466 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c03f030 con 0x7f7394102690 2024-09-19T08:10:29.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f739ae81640 1 -- 172.21.15.157:0/2861259466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739419fd30 con 0x7f7394102690 2024-09-19T08:10:29.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f7391ffb640 1 -- 172.21.15.157:0/2861259466 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f738c003ad0 con 0x7f7394102690 2024-09-19T08:10:29.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f7391ffb640 1 -- 172.21.15.157:0/2861259466 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c035070 con 0x7f7394102690 2024-09-19T08:10:29.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.526+0000 7f739ae81640 1 -- 172.21.15.157:0/2861259466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73941a0200 con 0x7f7394102690 2024-09-19T08:10:29.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.528+0000 7f739ae81640 1 -- 172.21.15.157:0/2861259466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7394107e20 con 0x7f7394102690 2024-09-19T08:10:29.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.529+0000 7f7391ffb640 1 -- 172.21.15.157:0/2861259466 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f738c042030 con 0x7f7394102690 2024-09-19T08:10:29.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.529+0000 7f7391ffb640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f737c0a3240 0x7f737c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:10:29.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.530+0000 7f7391ffb640 1 -- 172.21.15.157:0/2861259466 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5430+0+0 (secure 0 0 0) 0x7f738c0ef4a0 con 0x7f7394102690 2024-09-19T08:10:29.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.530+0000 7f7398c1d640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f737c0a3240 0x7f737c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:10:29.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.530+0000 7f7398c1d640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f737c0a3240 0x7f737c0a5700 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f739406b050 tx=0x7f7384009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:10:29.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.534+0000 7f7391ffb640 1 -- 172.21.15.157:0/2861259466 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f738c0bcd00 con 0x7f7394102690 2024-09-19T08:10:29.707 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:29.707+0000 7f739ae81640 1 -- 172.21.15.157:0/2861259466 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7394062630 con 0x7f737c0a3240 2024-09-19T08:10:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:29 smithi157 ceph-mon[35143]: purged_snaps scrub starts 2024-09-19T08:10:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:29 smithi157 ceph-mon[35143]: purged_snaps scrub ok 2024-09-19T08:10:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:29 smithi157 ceph-mon[35143]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:29 smithi157 ceph-mon[35143]: osd.7 [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103] boot 2024-09-19T08:10:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:29 smithi157 ceph-mon[35143]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T08:10:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:30.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:29 smithi157 ceph-mon[35143]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:10:30.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:10:30.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:10:30.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:30.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:29 smithi022 ceph-mon[30963]: purged_snaps scrub starts 2024-09-19T08:10:30.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:29 smithi022 ceph-mon[30963]: purged_snaps scrub ok 2024-09-19T08:10:30.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:29 smithi022 ceph-mon[30963]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:30.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:30.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:29 smithi022 ceph-mon[30963]: osd.7 [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103] boot 2024-09-19T08:10:30.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:29 smithi022 ceph-mon[30963]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T08:10:30.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:30.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:29 smithi022 ceph-mon[30963]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:10:30.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:10:30.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:10:30.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:29 smithi064 ceph-mon[34900]: purged_snaps scrub starts 2024-09-19T08:10:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:29 smithi064 ceph-mon[34900]: purged_snaps scrub ok 2024-09-19T08:10:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:29 smithi064 ceph-mon[34900]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:10:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:29 smithi064 ceph-mon[34900]: osd.7 [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103] boot 2024-09-19T08:10:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:29 smithi064 ceph-mon[34900]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T08:10:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:10:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:29 smithi064 ceph-mon[34900]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:10:30.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:10:30.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:10:30.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:31.114 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:31 smithi157 ceph-mon[35143]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T08:10:31.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:31 smithi022 ceph-mon[30963]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T08:10:31.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:31 smithi064 ceph-mon[34900]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T08:10:32.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:32 smithi157 ceph-mon[35143]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:32.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:32 smithi157 ceph-mon[35143]: osdmap e53: 8 total, 8 up, 8 in 2024-09-19T08:10:32.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:32 smithi022 ceph-mon[30963]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:32.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:32 smithi022 ceph-mon[30963]: osdmap e53: 8 total, 8 up, 8 in 2024-09-19T08:10:32.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:32 smithi064 ceph-mon[34900]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:32.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:32 smithi064 ceph-mon[34900]: osdmap e53: 8 total, 8 up, 8 in 2024-09-19T08:10:33.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:33 smithi064 ceph-mon[34900]: osdmap e54: 8 total, 8 up, 8 in 2024-09-19T08:10:33.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:33 smithi064 ceph-mon[34900]: from='client.? 172.21.15.157:0/3085631473' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "764b1e6e-48c9-49f9-8124-bc13bcba3a08"}]: dispatch 2024-09-19T08:10:33.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:33 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "764b1e6e-48c9-49f9-8124-bc13bcba3a08"}]: dispatch 2024-09-19T08:10:33.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:33 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "764b1e6e-48c9-49f9-8124-bc13bcba3a08"}]': finished 2024-09-19T08:10:33.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:33 smithi064 ceph-mon[34900]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T08:10:33.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:33 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:33.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:33 smithi064 ceph-mon[34900]: from='client.? 172.21.15.157:0/2681858527' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:10:33.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:33 smithi157 ceph-mon[35143]: osdmap e54: 8 total, 8 up, 8 in 2024-09-19T08:10:33.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:33 smithi157 ceph-mon[35143]: from='client.? 172.21.15.157:0/3085631473' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "764b1e6e-48c9-49f9-8124-bc13bcba3a08"}]: dispatch 2024-09-19T08:10:33.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:33 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "764b1e6e-48c9-49f9-8124-bc13bcba3a08"}]: dispatch 2024-09-19T08:10:33.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:33 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "764b1e6e-48c9-49f9-8124-bc13bcba3a08"}]': finished 2024-09-19T08:10:33.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:33 smithi157 ceph-mon[35143]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T08:10:33.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:33 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:33.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:33 smithi157 ceph-mon[35143]: from='client.? 172.21.15.157:0/2681858527' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:10:33.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:33 smithi022 ceph-mon[30963]: osdmap e54: 8 total, 8 up, 8 in 2024-09-19T08:10:33.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:33 smithi022 ceph-mon[30963]: from='client.? 172.21.15.157:0/3085631473' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "764b1e6e-48c9-49f9-8124-bc13bcba3a08"}]: dispatch 2024-09-19T08:10:33.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:33 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "764b1e6e-48c9-49f9-8124-bc13bcba3a08"}]: dispatch 2024-09-19T08:10:33.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:33 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "764b1e6e-48c9-49f9-8124-bc13bcba3a08"}]': finished 2024-09-19T08:10:33.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:33 smithi022 ceph-mon[30963]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T08:10:33.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:33 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:33.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:33 smithi022 ceph-mon[30963]: from='client.? 172.21.15.157:0/2681858527' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:10:34.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:34 smithi157 ceph-mon[35143]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:34.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:34 smithi022 ceph-mon[30963]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:34.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:34 smithi064 ceph-mon[34900]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:36.498 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:36 smithi157 ceph-mon[35143]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:36.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:36 smithi022 ceph-mon[30963]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:36.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:36 smithi064 ceph-mon[34900]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:38.472 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:38 smithi157 ceph-mon[35143]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:38.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:38 smithi022 ceph-mon[30963]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:38.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:38 smithi064 ceph-mon[34900]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:40.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:40 smithi157 ceph-mon[35143]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:40.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:40 smithi022 ceph-mon[30963]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:40.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:40 smithi064 ceph-mon[34900]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:41.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:41 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T08:10:41.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:41 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:41.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:41 smithi157 ceph-mon[35143]: Deploying daemon osd.8 on smithi157 2024-09-19T08:10:41.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:41 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T08:10:41.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:41 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:41.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:41 smithi022 ceph-mon[30963]: Deploying daemon osd.8 on smithi157 2024-09-19T08:10:41.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:41 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T08:10:41.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:41 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:41.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:41 smithi064 ceph-mon[34900]: Deploying daemon osd.8 on smithi157 2024-09-19T08:10:42.169 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:42 smithi157 ceph-mon[35143]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:42.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:42 smithi022 ceph-mon[30963]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:42.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:42 smithi064 ceph-mon[34900]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:44.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:44 smithi157 ceph-mon[35143]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:44.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:44 smithi022 ceph-mon[30963]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:44.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:44 smithi064 ceph-mon[34900]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:46.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:46 smithi157 ceph-mon[35143]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:46.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:46 smithi022 ceph-mon[30963]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:46.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:46 smithi064 ceph-mon[34900]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:47.386 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:47 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:47.386 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:47 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:47.386 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:47 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:47.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:47 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:47.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:47 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:47.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:47 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:47.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:47 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:47.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:47 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:47.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:47 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:48.521 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:48 smithi157 ceph-mon[35143]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:48.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:48 smithi064 ceph-mon[34900]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:48.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:48 smithi022 ceph-mon[30963]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:49.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:49.922+0000 7f7391ffb640 1 -- 172.21.15.157:0/2861259466 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7394062630 con 0x7f737c0a3240 2024-09-19T08:10:49.922 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 8 on host 'smithi157' 2024-09-19T08:10:49.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:49.924+0000 7f739ae81640 1 -- 172.21.15.157:0/2861259466 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f737c0a3240 msgr2=0x7f737c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:10:49.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:49.924+0000 7f739ae81640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f737c0a3240 0x7f737c0a5700 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f739406b050 tx=0x7f7384009500 comp rx=0 tx=0).stop 2024-09-19T08:10:49.924 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:49.924+0000 7f739ae81640 1 -- 172.21.15.157:0/2861259466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7394102690 msgr2=0x7f739419f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:10:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:49.924+0000 7f739ae81640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7394102690 0x7f739419f1e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f738c0314e0 tx=0x7f738c031b60 comp rx=0 tx=0).stop 2024-09-19T08:10:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:49.924+0000 7f739ae81640 1 -- 172.21.15.157:0/2861259466 shutdown_connections 2024-09-19T08:10:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:49.924+0000 7f739ae81640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f737c0a3240 0x7f737c0a5700 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:49.924+0000 7f739ae81640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7394102690 0x7f739419f1e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:49.924+0000 7f739ae81640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7394101cd0 0x7f739419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:49.924+0000 7f739ae81640 1 --2- 172.21.15.157:0/2861259466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7394100ad0 0x7f739406ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:49.924+0000 7f739ae81640 1 -- 172.21.15.157:0/2861259466 >> 172.21.15.157:0/2861259466 conn(0x7f73940fc280 msgr2=0x7f7394104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:10:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:49.924+0000 7f739ae81640 1 -- 172.21.15.157:0/2861259466 shutdown_connections 2024-09-19T08:10:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:10:49.924+0000 7f739ae81640 1 -- 172.21.15.157:0/2861259466 wait complete. 2024-09-19T08:10:50.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:49 smithi157 ceph-mon[35143]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:50.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:50.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:50.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:50.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:49 smithi022 ceph-mon[30963]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:50.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:50.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:50.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:50.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.304 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:49 smithi064 ceph-mon[34900]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:50.304 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.304 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.304 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:50.305 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:50.305 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.305 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:10:50.305 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.305 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:50.622 DEBUG:teuthology.orchestra.run.smithi157:osd.8> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T08:10:50.624 INFO:tasks.cephadm:Deploying osd.9 on smithi022 with /dev/vg_nvme/lv_1... 2024-09-19T08:10:50.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T08:10:50.893 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:10:52.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:51 smithi157 ceph-mon[35143]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:52.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:51 smithi022 ceph-mon[30963]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:52.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:51 smithi064 ceph-mon[34900]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:52.503 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:10:52 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:10:52.117+0000 7fee8524f640 -1 osd.8 0 log_to_monitors true 2024-09-19T08:10:53.202 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:52 smithi157 ceph-mon[35143]: from='osd.8 [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:10:53.825 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:52 smithi157 ceph-mon[35143]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:10:53.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:52 smithi064 ceph-mon[34900]: from='osd.8 [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:10:53.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:52 smithi064 ceph-mon[34900]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:10:53.826 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:52 smithi022 ceph-mon[30963]: from='osd.8 [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:10:53.827 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:52 smithi022 ceph-mon[30963]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:10:53.827 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:10:53.828 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_1 2024-09-19T08:10:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:10:54.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:54.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T08:10:54.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T08:10:54.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:54.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: from='osd.8 [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:10:54.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:10:54.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: Detected new or changed devices on smithi157 2024-09-19T08:10:54.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:54.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:54.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:54.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:54.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:54.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi157 to 5696M 2024-09-19T08:10:54.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:54.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:54.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:54.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:54.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:54.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T08:10:54.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T08:10:54.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:54.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: from='osd.8 [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:10:54.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:10:54.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: Detected new or changed devices on smithi157 2024-09-19T08:10:54.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:54.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:54.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:54.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:54.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:54.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi157 to 5696M 2024-09-19T08:10:54.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:54.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:54.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:54.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:54.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:54.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T08:10:54.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T08:10:54.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:54.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: from='osd.8 [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:10:54.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:10:54.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: Detected new or changed devices on smithi157 2024-09-19T08:10:54.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:54.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:54.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:54.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:54.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:10:54.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi157 to 5696M 2024-09-19T08:10:54.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:54.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:54.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:10:54.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:10:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.550+0000 7f1887672640 1 -- 172.21.15.22:0/2744916759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1880102e00 msgr2=0x7f1880103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:10:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.550+0000 7f1887672640 1 --2- 172.21.15.22:0/2744916759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1880102e00 0x7f1880103280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1870009900 tx=0x7f187002a3f0 comp rx=0 tx=0).stop 2024-09-19T08:10:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.550+0000 7f1887672640 1 -- 172.21.15.22:0/2744916759 shutdown_connections 2024-09-19T08:10:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.550+0000 7f1887672640 1 --2- 172.21.15.22:0/2744916759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18801037c0 0x7f188010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.550+0000 7f1887672640 1 --2- 172.21.15.22:0/2744916759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1880102e00 0x7f1880103280 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.550+0000 7f1887672640 1 --2- 172.21.15.22:0/2744916759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1880101c00 0x7f1880102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.550+0000 7f1887672640 1 -- 172.21.15.22:0/2744916759 >> 172.21.15.22:0/2744916759 conn(0x7f18800fd390 msgr2=0x7f18800ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:10:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.550+0000 7f1887672640 1 -- 172.21.15.22:0/2744916759 shutdown_connections 2024-09-19T08:10:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.550+0000 7f1887672640 1 -- 172.21.15.22:0/2744916759 wait complete. 2024-09-19T08:10:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.551+0000 7f1887672640 1 Processor -- start 2024-09-19T08:10:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.551+0000 7f1887672640 1 -- start start 2024-09-19T08:10:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.551+0000 7f1887672640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1880101c00 0x7f1880197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:10:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.551+0000 7f1887672640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1880102e00 0x7f1880198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:10:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.551+0000 7f1887672640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18801037c0 0x7f188019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:10:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.551+0000 7f1887672640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f188010d860 con 0x7f1880102e00 2024-09-19T08:10:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.551+0000 7f1887672640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f188010d6e0 con 0x7f1880101c00 2024-09-19T08:10:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.551+0000 7f1887672640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f188010d9e0 con 0x7f18801037c0 2024-09-19T08:10:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.551+0000 7f1884c0d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1880102e00 0x7f1880198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:10:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.551+0000 7f1884c0d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1880102e00 0x7f1880198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49010/0 (socket says 172.21.15.22:49010) 2024-09-19T08:10:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.551+0000 7f1884c0d640 1 -- 172.21.15.22:0/3851520776 learned_addr learned my addr 172.21.15.22:0/3851520776 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:10:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f188540e640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1880101c00 0x7f1880197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:10:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f1884c0d640 1 -- 172.21.15.22:0/3851520776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18801037c0 msgr2=0x7f188019f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:10:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f1885c0f640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18801037c0 0x7f188019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:10:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f1884c0d640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18801037c0 0x7f188019f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f1884c0d640 1 -- 172.21.15.22:0/3851520776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1880101c00 msgr2=0x7f1880197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:10:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f1884c0d640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1880101c00 0x7f1880197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:10:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f1884c0d640 1 -- 172.21.15.22:0/3851520776 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188019fb10 con 0x7f1880102e00 2024-09-19T08:10:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f188540e640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1880101c00 0x7f1880197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:10:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f1885c0f640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18801037c0 0x7f188019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:10:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f1884c0d640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1880102e00 0x7f1880198420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1870009230 tx=0x7f1870031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:10:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f18767fc640 1 -- 172.21.15.22:0/3851520776 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187003f030 con 0x7f1880102e00 2024-09-19T08:10:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f1887672640 1 -- 172.21.15.22:0/3851520776 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188019fda0 con 0x7f1880102e00 2024-09-19T08:10:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f18767fc640 1 -- 172.21.15.22:0/3851520776 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1870003ad0 con 0x7f1880102e00 2024-09-19T08:10:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f18767fc640 1 -- 172.21.15.22:0/3851520776 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870035070 con 0x7f1880102e00 2024-09-19T08:10:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.552+0000 7f1887672640 1 -- 172.21.15.22:0/3851520776 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18801a0270 con 0x7f1880102e00 2024-09-19T08:10:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.553+0000 7f1887672640 1 -- 172.21.15.22:0/3851520776 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18801090a0 con 0x7f1880102e00 2024-09-19T08:10:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.554+0000 7f18767fc640 1 -- 172.21.15.22:0/3851520776 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1870042030 con 0x7f1880102e00 2024-09-19T08:10:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.555+0000 7f18767fc640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f186c0a3270 0x7f186c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:10:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.555+0000 7f18767fc640 1 -- 172.21.15.22:0/3851520776 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5549+0+0 (secure 0 0 0) 0x7f18700ef810 con 0x7f1880102e00 2024-09-19T08:10:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.555+0000 7f188540e640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f186c0a3270 0x7f186c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:10:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.555+0000 7f188540e640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f186c0a3270 0x7f186c0a5730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1880102c60 tx=0x7f187c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:10:54.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.559+0000 7f18767fc640 1 -- 172.21.15.22:0/3851520776 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f18700bd000 con 0x7f1880102e00 2024-09-19T08:10:54.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:10:54.744+0000 7f1887672640 1 -- 172.21.15.22:0/3851520776 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1880062630 con 0x7f186c0a3270 2024-09-19T08:10:54.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:10:54 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:10:54.438+0000 7fee811d2640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T08:10:54.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:10:54 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:10:54.447+0000 7fee78ff2640 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:10:55.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:54 smithi157 ceph-mon[35143]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T08:10:55.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:54 smithi157 ceph-mon[35143]: osdmap e57: 9 total, 8 up, 9 in 2024-09-19T08:10:55.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:54 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:55.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:54 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:55.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:54 smithi157 ceph-mon[35143]: from='osd.8 ' entity='osd.8' 2024-09-19T08:10:55.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:54 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:10:55.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:54 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:10:55.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:54 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:55.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:54 smithi022 ceph-mon[30963]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T08:10:55.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:54 smithi022 ceph-mon[30963]: osdmap e57: 9 total, 8 up, 9 in 2024-09-19T08:10:55.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:54 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:55.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:54 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:55.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:54 smithi022 ceph-mon[30963]: from='osd.8 ' entity='osd.8' 2024-09-19T08:10:55.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:54 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:10:55.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:54 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:10:55.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:54 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:55.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:54 smithi064 ceph-mon[34900]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T08:10:55.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:54 smithi064 ceph-mon[34900]: osdmap e57: 9 total, 8 up, 9 in 2024-09-19T08:10:55.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:54 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:55.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:54 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:55.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:54 smithi064 ceph-mon[34900]: from='osd.8 ' entity='osd.8' 2024-09-19T08:10:55.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:54 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:10:55.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:54 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:10:55.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:54 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:10:56.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:56 smithi157 ceph-mon[35143]: purged_snaps scrub starts 2024-09-19T08:10:56.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:56 smithi157 ceph-mon[35143]: purged_snaps scrub ok 2024-09-19T08:10:56.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:56 smithi157 ceph-mon[35143]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:56.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:56 smithi157 ceph-mon[35143]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:10:56.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:56 smithi157 ceph-mon[35143]: osd.8 [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477] boot 2024-09-19T08:10:56.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:56 smithi157 ceph-mon[35143]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T08:10:56.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:56 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:56.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:56 smithi157 ceph-mon[35143]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T08:10:56.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:55 smithi022 ceph-mon[30963]: purged_snaps scrub starts 2024-09-19T08:10:56.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:56 smithi022 ceph-mon[30963]: purged_snaps scrub ok 2024-09-19T08:10:56.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:56 smithi022 ceph-mon[30963]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:56.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:56 smithi022 ceph-mon[30963]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:10:56.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:56 smithi022 ceph-mon[30963]: osd.8 [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477] boot 2024-09-19T08:10:56.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:56 smithi022 ceph-mon[30963]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T08:10:56.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:56 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:56.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:56 smithi022 ceph-mon[30963]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T08:10:56.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:56 smithi064 ceph-mon[34900]: purged_snaps scrub starts 2024-09-19T08:10:56.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:56 smithi064 ceph-mon[34900]: purged_snaps scrub ok 2024-09-19T08:10:56.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:56 smithi064 ceph-mon[34900]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:10:56.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:56 smithi064 ceph-mon[34900]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:10:56.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:56 smithi064 ceph-mon[34900]: osd.8 [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477] boot 2024-09-19T08:10:56.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:56 smithi064 ceph-mon[34900]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T08:10:56.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:56 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:10:56.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:56 smithi064 ceph-mon[34900]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T08:10:57.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:57 smithi157 ceph-mon[35143]: osdmap e60: 9 total, 9 up, 9 in 2024-09-19T08:10:57.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:57 smithi022 ceph-mon[30963]: osdmap e60: 9 total, 9 up, 9 in 2024-09-19T08:10:57.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:57 smithi064 ceph-mon[34900]: osdmap e60: 9 total, 9 up, 9 in 2024-09-19T08:10:58.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:58 smithi157 ceph-mon[35143]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:10:58.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:58 smithi157 ceph-mon[35143]: osdmap e61: 9 total, 9 up, 9 in 2024-09-19T08:10:58.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:58 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3941264285' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82914818-df47-4374-9e3e-2f71fc76e78b"}]: dispatch 2024-09-19T08:10:58.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:58 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3941264285' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82914818-df47-4374-9e3e-2f71fc76e78b"}]': finished 2024-09-19T08:10:58.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:58 smithi157 ceph-mon[35143]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T08:10:58.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:58 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:10:58.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:10:58 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/952295605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:10:58.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:58 smithi022 ceph-mon[30963]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:10:58.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:58 smithi022 ceph-mon[30963]: osdmap e61: 9 total, 9 up, 9 in 2024-09-19T08:10:58.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:58 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3941264285' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82914818-df47-4374-9e3e-2f71fc76e78b"}]: dispatch 2024-09-19T08:10:58.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:58 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3941264285' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82914818-df47-4374-9e3e-2f71fc76e78b"}]': finished 2024-09-19T08:10:58.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:58 smithi022 ceph-mon[30963]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T08:10:58.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:58 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:10:58.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:10:58 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/952295605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:10:58.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:58 smithi064 ceph-mon[34900]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:10:58.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:58 smithi064 ceph-mon[34900]: osdmap e61: 9 total, 9 up, 9 in 2024-09-19T08:10:58.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:58 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3941264285' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82914818-df47-4374-9e3e-2f71fc76e78b"}]: dispatch 2024-09-19T08:10:58.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:58 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3941264285' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82914818-df47-4374-9e3e-2f71fc76e78b"}]': finished 2024-09-19T08:10:58.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:58 smithi064 ceph-mon[34900]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T08:10:58.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:58 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:10:58.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:10:58 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/952295605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:11:00.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:00 smithi157 ceph-mon[35143]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:00.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:00 smithi022 ceph-mon[30963]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:00.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:00 smithi064 ceph-mon[34900]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 283 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:02.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:02 smithi157 ceph-mon[35143]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T08:11:02.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:02 smithi022 ceph-mon[30963]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T08:11:02.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:02 smithi064 ceph-mon[34900]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T08:11:04.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:04 smithi022 ceph-mon[30963]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T08:11:04.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:04 smithi157 ceph-mon[35143]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T08:11:04.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:04 smithi064 ceph-mon[34900]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T08:11:06.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:06 smithi157 ceph-mon[35143]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:11:06.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T08:11:06.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:06.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:06 smithi022 ceph-mon[30963]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:11:06.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T08:11:06.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:06.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:06 smithi064 ceph-mon[34900]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:11:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T08:11:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:07.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:07 smithi157 ceph-mon[35143]: Deploying daemon osd.9 on smithi022 2024-09-19T08:11:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:07 smithi022 ceph-mon[30963]: Deploying daemon osd.9 on smithi022 2024-09-19T08:11:07.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:07 smithi064 ceph-mon[34900]: Deploying daemon osd.9 on smithi022 2024-09-19T08:11:08.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:08 smithi022 ceph-mon[30963]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T08:11:08.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:08 smithi157 ceph-mon[35143]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T08:11:08.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:08 smithi064 ceph-mon[34900]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T08:11:10.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:10 smithi022 ceph-mon[30963]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T08:11:10.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:10 smithi064 ceph-mon[34900]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T08:11:10.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:10 smithi157 ceph-mon[35143]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T08:11:12.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:12 smithi064 ceph-mon[34900]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:11:12.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:12 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:11:12.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:12 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:12.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:12 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:12.604 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:12 smithi022 ceph-mon[30963]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:11:12.605 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:12 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:11:12.605 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:12 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:12.605 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:12 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:12 smithi157 ceph-mon[35143]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:11:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:12 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:11:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:12 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:12 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:14.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:14 smithi022 ceph-mon[30963]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:14.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:14 smithi064 ceph-mon[34900]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:14.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:14 smithi157 ceph-mon[35143]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:15.779 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 9 on host 'smithi022' 2024-09-19T08:11:15.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:11:15.778+0000 7f18767fc640 1 -- 172.21.15.22:0/3851520776 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1880062630 con 0x7f186c0a3270 2024-09-19T08:11:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:11:15.781+0000 7f1887672640 1 -- 172.21.15.22:0/3851520776 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f186c0a3270 msgr2=0x7f186c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:11:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:11:15.781+0000 7f1887672640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f186c0a3270 0x7f186c0a5730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1880102c60 tx=0x7f187c005f90 comp rx=0 tx=0).stop 2024-09-19T08:11:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:11:15.781+0000 7f1887672640 1 -- 172.21.15.22:0/3851520776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1880102e00 msgr2=0x7f1880198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:11:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:11:15.781+0000 7f1887672640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1880102e00 0x7f1880198420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1870009230 tx=0x7f1870031b60 comp rx=0 tx=0).stop 2024-09-19T08:11:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:11:15.781+0000 7f1887672640 1 -- 172.21.15.22:0/3851520776 shutdown_connections 2024-09-19T08:11:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:11:15.781+0000 7f1887672640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f186c0a3270 0x7f186c0a5730 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:11:15.781+0000 7f1887672640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18801037c0 0x7f188019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:15.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:11:15.781+0000 7f1887672640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1880102e00 0x7f1880198420 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:15.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:11:15.781+0000 7f1887672640 1 --2- 172.21.15.22:0/3851520776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1880101c00 0x7f1880197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:15.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:11:15.781+0000 7f1887672640 1 -- 172.21.15.22:0/3851520776 >> 172.21.15.22:0/3851520776 conn(0x7f18800fd390 msgr2=0x7f1880106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:11:15.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:11:15.781+0000 7f1887672640 1 -- 172.21.15.22:0/3851520776 shutdown_connections 2024-09-19T08:11:15.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:11:15.781+0000 7f1887672640 1 -- 172.21.15.22:0/3851520776 wait complete. 2024-09-19T08:11:16.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:15 smithi022 ceph-mon[30963]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:16.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:15 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:15 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:15 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:16.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:15 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:11:16.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:15 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:15 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:11:16.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:15 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:15 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:15 smithi157 ceph-mon[35143]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:15 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:15 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:15 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:15 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:11:16.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:15 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:15 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:11:16.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:15 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:15 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:15 smithi064 ceph-mon[34900]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:16.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:15 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:15 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:15 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:16.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:15 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:11:16.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:15 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:15 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:11:16.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:15 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:15 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:16.410 DEBUG:teuthology.orchestra.run.smithi022:osd.9> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T08:11:16.413 INFO:tasks.cephadm:Deploying osd.10 on smithi064 with /dev/vg_nvme/lv_1... 2024-09-19T08:11:16.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T08:11:16.676 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:11:18.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:17 smithi064 ceph-mon[34900]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:18.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:17 smithi157 ceph-mon[35143]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:18.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:17 smithi022 ceph-mon[30963]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:18.257 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:11:17 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:11:17.971+0000 7fcfdeb1f640 -1 osd.9 0 log_to_monitors true 2024-09-19T08:11:19.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:18 smithi157 ceph-mon[35143]: from='osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T08:11:19.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:18 smithi022 ceph-mon[30963]: from='osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T08:11:19.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:18 smithi064 ceph-mon[34900]: from='osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T08:11:19.302 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T08:11:19.329 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_1 2024-09-19T08:11:19.596 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:11:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.090+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1042480740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f2c101cd0 msgr2=0x7f8f2c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:11:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.090+0000 7f8f31e4e640 1 --2- 172.21.15.64:0/1042480740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f2c101cd0 0x7f8f2c102150 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009900 tx=0x7f8f2002a410 comp rx=0 tx=0).stop 2024-09-19T08:11:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.091+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1042480740 shutdown_connections 2024-09-19T08:11:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.091+0000 7f8f31e4e640 1 --2- 172.21.15.64:0/1042480740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c102690 0x7f8f2c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.091+0000 7f8f31e4e640 1 --2- 172.21.15.64:0/1042480740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f2c101cd0 0x7f8f2c102150 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.091+0000 7f8f31e4e640 1 --2- 172.21.15.64:0/1042480740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f2c100ad0 0x7f8f2c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.091+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1042480740 >> 172.21.15.64:0/1042480740 conn(0x7f8f2c0fc280 msgr2=0x7f8f2c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:11:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.091+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1042480740 shutdown_connections 2024-09-19T08:11:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.091+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1042480740 wait complete. 2024-09-19T08:11:20.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.091+0000 7f8f31e4e640 1 Processor -- start 2024-09-19T08:11:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f31e4e640 1 -- start start 2024-09-19T08:11:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f31e4e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f2c100ad0 0x7f8f2c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:11:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f31e4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f2c101cd0 0x7f8f2c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:11:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f31e4e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c102690 0x7f8f2c19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:11:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f31e4e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c10c730 con 0x7f8f2c102690 2024-09-19T08:11:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f31e4e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c10c5b0 con 0x7f8f2c101cd0 2024-09-19T08:11:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f31e4e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c10c8b0 con 0x7f8f2c100ad0 2024-09-19T08:11:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f2affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f2c101cd0 0x7f8f2c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:11:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f2affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f2c101cd0 0x7f8f2c19cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46674/0 (socket says 172.21.15.64:46674) 2024-09-19T08:11:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f2affd640 1 -- 172.21.15.64:0/1828243364 learned_addr learned my addr 172.21.15.64:0/1828243364 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T08:11:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f2b7fe640 1 --2- 172.21.15.64:0/1828243364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f2c100ad0 0x7f8f2c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:11:20.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f2affd640 1 -- 172.21.15.64:0/1828243364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f2c100ad0 msgr2=0x7f8f2c06ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:11:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f2affd640 1 --2- 172.21.15.64:0/1828243364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f2c100ad0 0x7f8f2c06ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f2affd640 1 -- 172.21.15.64:0/1828243364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c102690 msgr2=0x7f8f2c19f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:11:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f2affd640 1 --2- 172.21.15.64:0/1828243364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c102690 0x7f8f2c19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.092+0000 7f8f2affd640 1 -- 172.21.15.64:0/1828243364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2c19faa0 con 0x7f8f2c101cd0 2024-09-19T08:11:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.093+0000 7f8f2affd640 1 --2- 172.21.15.64:0/1828243364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f2c101cd0 0x7f8f2c19cb90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009230 tx=0x7f8f20004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:11:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.093+0000 7f8f28ff9640 1 -- 172.21.15.64:0/1828243364 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2003f030 con 0x7f8f2c101cd0 2024-09-19T08:11:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.093+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1828243364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2c19fd30 con 0x7f8f2c101cd0 2024-09-19T08:11:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.093+0000 7f8f28ff9640 1 -- 172.21.15.64:0/1828243364 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f20003ce0 con 0x7f8f2c101cd0 2024-09-19T08:11:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.093+0000 7f8f28ff9640 1 -- 172.21.15.64:0/1828243364 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20035070 con 0x7f8f2c101cd0 2024-09-19T08:11:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.093+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1828243364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2c1a01c0 con 0x7f8f2c101cd0 2024-09-19T08:11:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.094+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1828243364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2c107e70 con 0x7f8f2c101cd0 2024-09-19T08:11:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.095+0000 7f8f28ff9640 1 -- 172.21.15.64:0/1828243364 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8f20042030 con 0x7f8f2c101cd0 2024-09-19T08:11:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.096+0000 7f8f28ff9640 1 --2- 172.21.15.64:0/1828243364 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8f140a3240 0x7f8f140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:11:20.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.096+0000 7f8f28ff9640 1 -- 172.21.15.64:0/1828243364 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5981+0+0 (secure 0 0 0) 0x7f8f200efa70 con 0x7f8f2c101cd0 2024-09-19T08:11:20.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.096+0000 7f8f2b7fe640 1 --2- 172.21.15.64:0/1828243364 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8f140a3240 0x7f8f140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:11:20.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.097+0000 7f8f2b7fe640 1 --2- 172.21.15.64:0/1828243364 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8f140a3240 0x7f8f140a5700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c005d00 tx=0x7f8f1c005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:11:20.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.099+0000 7f8f28ff9640 1 -- 172.21.15.64:0/1828243364 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8f200bd0b0 con 0x7f8f2c101cd0 2024-09-19T08:11:20.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:20.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: from='osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T08:11:20.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T08:11:20.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: from='osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-19T08:11:20.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:20.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: Detected new or changed devices on smithi022 2024-09-19T08:11:20.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:20.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:20.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:20.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:20.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:20.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:20.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi022 to 4273M 2024-09-19T08:11:20.256 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:20.256 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:20.256 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:11:20.256 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:20.257 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:11:19.926+0000 7fcfdaaa2640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T08:11:20.257 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:11:19.938+0000 7fcfd68ca640 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:11:20.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:20.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: from='osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T08:11:20.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T08:11:20.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: from='osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-19T08:11:20.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:20.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: Detected new or changed devices on smithi022 2024-09-19T08:11:20.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:20.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:20.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:20.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:20.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:20.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:20.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi022 to 4273M 2024-09-19T08:11:20.260 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:20.260 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:20.260 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:11:20.260 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:20.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:20.284+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1828243364 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8f2c062630 con 0x7f8f140a3240 2024-09-19T08:11:20.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:20.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: from='osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T08:11:20.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T08:11:20.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: from='osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-19T08:11:20.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:20.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: Detected new or changed devices on smithi022 2024-09-19T08:11:20.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:20.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:20.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:20.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:20.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:20.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:20.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi022 to 4273M 2024-09-19T08:11:20.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:20.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:20.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:11:20.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:21.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:20 smithi157 ceph-mon[35143]: from='osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-19T08:11:21.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:20 smithi157 ceph-mon[35143]: osdmap e64: 10 total, 9 up, 10 in 2024-09-19T08:11:21.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:20 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:21.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:20 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:21.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:20 smithi157 ceph-mon[35143]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:11:21.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:20 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:11:21.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:20 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:11:21.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:20 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:21.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:20 smithi022 ceph-mon[30963]: from='osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-19T08:11:21.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:20 smithi022 ceph-mon[30963]: osdmap e64: 10 total, 9 up, 10 in 2024-09-19T08:11:21.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:20 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:21.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:20 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:21.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:20 smithi022 ceph-mon[30963]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:11:21.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:20 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:11:21.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:20 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:11:21.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:20 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:21.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:20 smithi064 ceph-mon[34900]: from='osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-19T08:11:21.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:20 smithi064 ceph-mon[34900]: osdmap e64: 10 total, 9 up, 10 in 2024-09-19T08:11:21.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:20 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:21.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:20 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:21.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:20 smithi064 ceph-mon[34900]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:11:21.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:20 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:11:21.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:20 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:11:21.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:20 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:22.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:21 smithi064 ceph-mon[34900]: purged_snaps scrub starts 2024-09-19T08:11:22.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:21 smithi064 ceph-mon[34900]: purged_snaps scrub ok 2024-09-19T08:11:22.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:21 smithi064 ceph-mon[34900]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:22.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:21 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:22.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:21 smithi064 ceph-mon[34900]: osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990] boot 2024-09-19T08:11:22.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:21 smithi064 ceph-mon[34900]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T08:11:22.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:21 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:22.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:21 smithi157 ceph-mon[35143]: purged_snaps scrub starts 2024-09-19T08:11:22.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:21 smithi157 ceph-mon[35143]: purged_snaps scrub ok 2024-09-19T08:11:22.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:21 smithi157 ceph-mon[35143]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:22.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:21 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:22.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:21 smithi157 ceph-mon[35143]: osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990] boot 2024-09-19T08:11:22.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:21 smithi157 ceph-mon[35143]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T08:11:22.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:21 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:22.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:21 smithi022 ceph-mon[30963]: purged_snaps scrub starts 2024-09-19T08:11:22.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:21 smithi022 ceph-mon[30963]: purged_snaps scrub ok 2024-09-19T08:11:22.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:21 smithi022 ceph-mon[30963]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:11:22.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:21 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:22.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:21 smithi022 ceph-mon[30963]: osd.9 [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990] boot 2024-09-19T08:11:22.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:21 smithi022 ceph-mon[30963]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T08:11:22.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:21 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:11:23.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:22 smithi157 ceph-mon[35143]: osdmap e66: 10 total, 10 up, 10 in 2024-09-19T08:11:23.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:22 smithi157 ceph-mon[35143]: from='client.? 172.21.15.64:0/1477350952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ae177d-c454-4cba-ba21-cbeee0f10dbb"}]: dispatch 2024-09-19T08:11:23.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:22 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ae177d-c454-4cba-ba21-cbeee0f10dbb"}]: dispatch 2024-09-19T08:11:23.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:22 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ae177d-c454-4cba-ba21-cbeee0f10dbb"}]': finished 2024-09-19T08:11:23.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:22 smithi157 ceph-mon[35143]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T08:11:23.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:22 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:23.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:22 smithi022 ceph-mon[30963]: osdmap e66: 10 total, 10 up, 10 in 2024-09-19T08:11:23.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:22 smithi022 ceph-mon[30963]: from='client.? 172.21.15.64:0/1477350952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ae177d-c454-4cba-ba21-cbeee0f10dbb"}]: dispatch 2024-09-19T08:11:23.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:22 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ae177d-c454-4cba-ba21-cbeee0f10dbb"}]: dispatch 2024-09-19T08:11:23.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:22 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ae177d-c454-4cba-ba21-cbeee0f10dbb"}]': finished 2024-09-19T08:11:23.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:22 smithi022 ceph-mon[30963]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T08:11:23.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:22 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:23.317 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:22 smithi064 ceph-mon[34900]: osdmap e66: 10 total, 10 up, 10 in 2024-09-19T08:11:23.317 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:22 smithi064 ceph-mon[34900]: from='client.? 172.21.15.64:0/1477350952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ae177d-c454-4cba-ba21-cbeee0f10dbb"}]: dispatch 2024-09-19T08:11:23.317 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:22 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ae177d-c454-4cba-ba21-cbeee0f10dbb"}]: dispatch 2024-09-19T08:11:23.317 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:22 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ae177d-c454-4cba-ba21-cbeee0f10dbb"}]': finished 2024-09-19T08:11:23.317 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:22 smithi064 ceph-mon[34900]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T08:11:23.317 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:22 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:24.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:23 smithi157 ceph-mon[35143]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:24.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:23 smithi157 ceph-mon[35143]: from='client.? 172.21.15.64:0/2013570701' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:11:24.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:23 smithi022 ceph-mon[30963]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:24.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:23 smithi022 ceph-mon[30963]: from='client.? 172.21.15.64:0/2013570701' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:11:24.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:23 smithi064 ceph-mon[34900]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:24.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:23 smithi064 ceph-mon[34900]: from='client.? 172.21.15.64:0/2013570701' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:11:26.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:25 smithi157 ceph-mon[35143]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:26.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:25 smithi022 ceph-mon[30963]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:26.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:25 smithi064 ceph-mon[34900]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:28.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:27 smithi157 ceph-mon[35143]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:28.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:27 smithi022 ceph-mon[30963]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:28.319 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:27 smithi064 ceph-mon[34900]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:30.138 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:29 smithi064 ceph-mon[34900]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:29 smithi157 ceph-mon[35143]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:30.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:29 smithi022 ceph-mon[30963]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:32.199 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:31 smithi064 ceph-mon[34900]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:32.199 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:31 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T08:11:32.199 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:31 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:32.200 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:31 smithi064 ceph-mon[34900]: Deploying daemon osd.10 on smithi064 2024-09-19T08:11:32.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:31 smithi157 ceph-mon[35143]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:32.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:31 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T08:11:32.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:31 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:32.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:31 smithi157 ceph-mon[35143]: Deploying daemon osd.10 on smithi064 2024-09-19T08:11:32.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:31 smithi022 ceph-mon[30963]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:32.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:31 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T08:11:32.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:31 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:32.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:31 smithi022 ceph-mon[30963]: Deploying daemon osd.10 on smithi064 2024-09-19T08:11:34.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:33 smithi064 ceph-mon[34900]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:34.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:33 smithi157 ceph-mon[35143]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:34.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:33 smithi022 ceph-mon[30963]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:35.987 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:35 smithi064 ceph-mon[34900]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:36.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:35 smithi157 ceph-mon[35143]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:36.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:35 smithi022 ceph-mon[30963]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:37.994 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:37 smithi064 ceph-mon[34900]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:37.994 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:37 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:11:37.994 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:37 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:37.994 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:37 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:38.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:37 smithi157 ceph-mon[35143]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:38.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:37 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:11:38.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:37 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:38.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:37 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:38.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:37 smithi022 ceph-mon[30963]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:38.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:37 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:11:38.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:37 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:38.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:37 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:40.243 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:39 smithi064 ceph-mon[34900]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:40.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:39 smithi157 ceph-mon[35143]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:40.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:39 smithi022 ceph-mon[30963]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:40.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:40.594+0000 7f8f28ff9640 1 -- 172.21.15.64:0/1828243364 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8f2c062630 con 0x7f8f140a3240 2024-09-19T08:11:40.595 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 10 on host 'smithi064' 2024-09-19T08:11:40.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:40.596+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1828243364 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8f140a3240 msgr2=0x7f8f140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:11:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:40.596+0000 7f8f31e4e640 1 --2- 172.21.15.64:0/1828243364 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8f140a3240 0x7f8f140a5700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c005d00 tx=0x7f8f1c005c70 comp rx=0 tx=0).stop 2024-09-19T08:11:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:40.596+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1828243364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f2c101cd0 msgr2=0x7f8f2c19cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:11:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:40.596+0000 7f8f31e4e640 1 --2- 172.21.15.64:0/1828243364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f2c101cd0 0x7f8f2c19cb90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009230 tx=0x7f8f20004360 comp rx=0 tx=0).stop 2024-09-19T08:11:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:40.596+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1828243364 shutdown_connections 2024-09-19T08:11:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:40.596+0000 7f8f31e4e640 1 --2- 172.21.15.64:0/1828243364 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8f140a3240 0x7f8f140a5700 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:40.596+0000 7f8f31e4e640 1 --2- 172.21.15.64:0/1828243364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c102690 0x7f8f2c19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:40.596+0000 7f8f31e4e640 1 --2- 172.21.15.64:0/1828243364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f2c101cd0 0x7f8f2c19cb90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:40.596+0000 7f8f31e4e640 1 --2- 172.21.15.64:0/1828243364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f2c100ad0 0x7f8f2c06ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:40.596+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1828243364 >> 172.21.15.64:0/1828243364 conn(0x7f8f2c0fc280 msgr2=0x7f8f2c105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:11:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:40.597+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1828243364 shutdown_connections 2024-09-19T08:11:40.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T08:11:40.597+0000 7f8f31e4e640 1 -- 172.21.15.64:0/1828243364 wait complete. 2024-09-19T08:11:41.302 DEBUG:teuthology.orchestra.run.smithi064:osd.10> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T08:11:41.306 INFO:tasks.cephadm:Deploying osd.11 on smithi157 with /dev/vg_nvme/lv_1... 2024-09-19T08:11:41.306 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T08:11:41.565 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:11:41.863 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:41 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:41.863 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:41 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:41.863 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:41 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:41.864 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:41 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:41.918 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:41 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:41.918 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:41 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:41.918 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:41 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:41.918 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:41 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:42.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:41 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:42.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:41 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:42.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:41 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:42.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:41 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:42.876 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:42 smithi064 ceph-mon[34900]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:42.877 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:11:42 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:11:42.648+0000 7ff53b22a640 -1 osd.10 0 log_to_monitors true 2024-09-19T08:11:42.949 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:42 smithi157 ceph-mon[35143]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:43.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:42 smithi022 ceph-mon[30963]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:43.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:43 smithi064 ceph-mon[34900]: from='osd.10 [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:11:43.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:43 smithi064 ceph-mon[34900]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:11:43.871 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:43 smithi157 ceph-mon[35143]: from='osd.10 [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:11:43.872 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:43 smithi157 ceph-mon[35143]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:11:44.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:43 smithi022 ceph-mon[30963]: from='osd.10 [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:11:44.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:43 smithi022 ceph-mon[30963]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:11:44.142 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-19T08:11:44.168 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_1 2024-09-19T08:11:44.423 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:11:44.827 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:44 smithi157 ceph-mon[35143]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:44.827 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:44 smithi157 ceph-mon[35143]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T08:11:44.827 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:44 smithi157 ceph-mon[35143]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T08:11:44.827 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:44 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:44.827 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:44 smithi157 ceph-mon[35143]: from='osd.10 [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:11:44.827 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:44 smithi157 ceph-mon[35143]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:11:44.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:44 smithi064 ceph-mon[34900]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:44.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:44 smithi064 ceph-mon[34900]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T08:11:44.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:44 smithi064 ceph-mon[34900]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T08:11:44.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:44 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:44.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:44 smithi064 ceph-mon[34900]: from='osd.10 [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:11:44.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:44 smithi064 ceph-mon[34900]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:11:44.935 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.912+0000 7f8c3d48e640 1 -- 172.21.15.157:0/2492819363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c38101d10 msgr2=0x7f8c38102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:11:44.935 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.912+0000 7f8c3d48e640 1 --2- 172.21.15.157:0/2492819363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c38101d10 0x7f8c38102190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009900 tx=0x7f8c2002a410 comp rx=0 tx=0).stop 2024-09-19T08:11:44.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.912+0000 7f8c3d48e640 1 -- 172.21.15.157:0/2492819363 shutdown_connections 2024-09-19T08:11:44.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.912+0000 7f8c3d48e640 1 --2- 172.21.15.157:0/2492819363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c381026d0 0x7f8c38108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:44.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.912+0000 7f8c3d48e640 1 --2- 172.21.15.157:0/2492819363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c38101d10 0x7f8c38102190 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:44.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.912+0000 7f8c3d48e640 1 --2- 172.21.15.157:0/2492819363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c38100b10 0x7f8c38100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:44.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.912+0000 7f8c3d48e640 1 -- 172.21.15.157:0/2492819363 >> 172.21.15.157:0/2492819363 conn(0x7f8c380fc2a0 msgr2=0x7f8c380fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:11:44.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.912+0000 7f8c3d48e640 1 -- 172.21.15.157:0/2492819363 shutdown_connections 2024-09-19T08:11:44.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.912+0000 7f8c3d48e640 1 -- 172.21.15.157:0/2492819363 wait complete. 2024-09-19T08:11:44.936 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.913+0000 7f8c3d48e640 1 Processor -- start 2024-09-19T08:11:44.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.913+0000 7f8c3d48e640 1 -- start start 2024-09-19T08:11:44.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.913+0000 7f8c3d48e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c38100b10 0x7f8c38104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:11:44.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.913+0000 7f8c3d48e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c38101d10 0x7f8c381030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:11:44.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.913+0000 7f8c3d48e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c381026d0 0x7f8c38103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:11:44.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.913+0000 7f8c3d48e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3810c810 con 0x7f8c38101d10 2024-09-19T08:11:44.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.913+0000 7f8c3d48e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3810c690 con 0x7f8c38100b10 2024-09-19T08:11:44.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.913+0000 7f8c3d48e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3810c990 con 0x7f8c381026d0 2024-09-19T08:11:44.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.913+0000 7f8c377fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c381026d0 0x7f8c38103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:11:44.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.913+0000 7f8c36ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c38100b10 0x7f8c38104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:11:44.937 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.913+0000 7f8c377fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c381026d0 0x7f8c38103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38222/0 (socket says 172.21.15.157:38222) 2024-09-19T08:11:44.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.913+0000 7f8c377fe640 1 -- 172.21.15.157:0/3477962852 learned_addr learned my addr 172.21.15.157:0/3477962852 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T08:11:44.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.913+0000 7f8c2ffff640 1 --2- 172.21.15.157:0/3477962852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c38101d10 0x7f8c381030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:11:44.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.914+0000 7f8c377fe640 1 -- 172.21.15.157:0/3477962852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c38100b10 msgr2=0x7f8c38104aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:11:44.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.914+0000 7f8c377fe640 1 --2- 172.21.15.157:0/3477962852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c38100b10 0x7f8c38104aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:44.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.914+0000 7f8c377fe640 1 -- 172.21.15.157:0/3477962852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c38101d10 msgr2=0x7f8c381030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:11:44.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.914+0000 7f8c377fe640 1 --2- 172.21.15.157:0/3477962852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c38101d10 0x7f8c381030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:11:44.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.914+0000 7f8c377fe640 1 -- 172.21.15.157:0/3477962852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c38103ef0 con 0x7f8c381026d0 2024-09-19T08:11:44.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.914+0000 7f8c2ffff640 1 --2- 172.21.15.157:0/3477962852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c38101d10 0x7f8c381030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:11:44.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.914+0000 7f8c377fe640 1 --2- 172.21.15.157:0/3477962852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c381026d0 0x7f8c38103630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000eaa0 tx=0x7f8c3000eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:11:44.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.914+0000 7f8c34ff9640 1 -- 172.21.15.157:0/3477962852 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30004d20 con 0x7f8c381026d0 2024-09-19T08:11:44.938 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.914+0000 7f8c34ff9640 1 -- 172.21.15.157:0/3477962852 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c30005020 con 0x7f8c381026d0 2024-09-19T08:11:44.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.914+0000 7f8c3d48e640 1 -- 172.21.15.157:0/3477962852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c381a2140 con 0x7f8c381026d0 2024-09-19T08:11:44.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.914+0000 7f8c34ff9640 1 -- 172.21.15.157:0/3477962852 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c300116d0 con 0x7f8c381026d0 2024-09-19T08:11:44.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.914+0000 7f8c3d48e640 1 -- 172.21.15.157:0/3477962852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c381a26d0 con 0x7f8c381026d0 2024-09-19T08:11:44.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.915+0000 7f8c3d48e640 1 -- 172.21.15.157:0/3477962852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c38100ef0 con 0x7f8c381026d0 2024-09-19T08:11:44.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.916+0000 7f8c34ff9640 1 -- 172.21.15.157:0/3477962852 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8c300051d0 con 0x7f8c381026d0 2024-09-19T08:11:44.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.917+0000 7f8c34ff9640 1 --2- 172.21.15.157:0/3477962852 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8c240a32c0 0x7f8c240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:11:44.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.917+0000 7f8c34ff9640 1 -- 172.21.15.157:0/3477962852 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6414+0+0 (secure 0 0 0) 0x7f8c300cf360 con 0x7f8c381026d0 2024-09-19T08:11:44.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.917+0000 7f8c36ffd640 1 --2- 172.21.15.157:0/3477962852 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8c240a32c0 0x7f8c240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:11:44.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.918+0000 7f8c36ffd640 1 --2- 172.21.15.157:0/3477962852 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8c240a32c0 0x7f8c240a5780 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8c38101b70 tx=0x7f8c28009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:11:44.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:44.921+0000 7f8c34ff9640 1 -- 172.21.15.157:0/3477962852 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8c30016030 con 0x7f8c381026d0 2024-09-19T08:11:45.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:44 smithi022 ceph-mon[30963]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:45.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:44 smithi022 ceph-mon[30963]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T08:11:45.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:44 smithi022 ceph-mon[30963]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T08:11:45.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:44 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:45.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:44 smithi022 ceph-mon[30963]: from='osd.10 [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:11:45.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:44 smithi022 ceph-mon[30963]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T08:11:45.097 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:11:45.098+0000 7f8c3d48e640 1 -- 172.21.15.157:0/3477962852 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8c381a29c0 con 0x7f8c240a32c0 2024-09-19T08:11:45.336 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:11:45.019+0000 7ff5371ad640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T08:11:45.337 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:11:45.030+0000 7ff52efcd640 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:11:45.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T08:11:45.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: Detected new or changed devices on smithi064 2024-09-19T08:11:45.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: osdmap e69: 11 total, 10 up, 11 in 2024-09-19T08:11:45.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:45.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:45.760 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:45.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:45.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:45.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:45.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:45.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:45.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi064 to 4272M 2024-09-19T08:11:45.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:45.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:45.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:11:45.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:45.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:11:45.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:45.761 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:11:45.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:45.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='osd.10 ' entity='osd.10' 2024-09-19T08:11:45.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:11:45.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:11:45.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:45.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: osd.10 [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752] boot 2024-09-19T08:11:45.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T08:11:45.762 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:46.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T08:11:46.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: Detected new or changed devices on smithi064 2024-09-19T08:11:46.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: osdmap e69: 11 total, 10 up, 11 in 2024-09-19T08:11:46.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:46.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:46.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:46.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:46.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:46.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:46.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:46.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:46.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi064 to 4272M 2024-09-19T08:11:46.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:46.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:46.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:11:46.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:46.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:11:46.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:46.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:11:46.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:46.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='osd.10 ' entity='osd.10' 2024-09-19T08:11:46.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:11:46.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:11:46.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:46.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: osd.10 [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752] boot 2024-09-19T08:11:46.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T08:11:46.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:46.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T08:11:46.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: Detected new or changed devices on smithi064 2024-09-19T08:11:46.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: osdmap e69: 11 total, 10 up, 11 in 2024-09-19T08:11:46.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:46.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:46.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:46.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:46.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:46.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:46.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:46.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:11:46.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi064 to 4272M 2024-09-19T08:11:46.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:46.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:46.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:11:46.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:46.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:11:46.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:46.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:11:46.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:11:46.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='osd.10 ' entity='osd.10' 2024-09-19T08:11:46.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:11:46.089 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:11:46.089 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:46.089 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: osd.10 [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752] boot 2024-09-19T08:11:46.089 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T08:11:46.089 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:11:46.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:46 smithi157 ceph-mon[35143]: purged_snaps scrub starts 2024-09-19T08:11:46.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:46 smithi157 ceph-mon[35143]: purged_snaps scrub ok 2024-09-19T08:11:46.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:46 smithi157 ceph-mon[35143]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:46.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:46 smithi157 ceph-mon[35143]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:11:46.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:46 smithi157 ceph-mon[35143]: osdmap e71: 11 total, 11 up, 11 in 2024-09-19T08:11:47.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:46 smithi022 ceph-mon[30963]: purged_snaps scrub starts 2024-09-19T08:11:47.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:46 smithi022 ceph-mon[30963]: purged_snaps scrub ok 2024-09-19T08:11:47.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:46 smithi022 ceph-mon[30963]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:47.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:46 smithi022 ceph-mon[30963]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:11:47.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:46 smithi022 ceph-mon[30963]: osdmap e71: 11 total, 11 up, 11 in 2024-09-19T08:11:47.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:46 smithi064 ceph-mon[34900]: purged_snaps scrub starts 2024-09-19T08:11:47.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:46 smithi064 ceph-mon[34900]: purged_snaps scrub ok 2024-09-19T08:11:47.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:46 smithi064 ceph-mon[34900]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:11:47.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:46 smithi064 ceph-mon[34900]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:11:47.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:46 smithi064 ceph-mon[34900]: osdmap e71: 11 total, 11 up, 11 in 2024-09-19T08:11:48.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:48 smithi157 ceph-mon[35143]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 347 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:11:48.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:48 smithi157 ceph-mon[35143]: osdmap e72: 11 total, 11 up, 11 in 2024-09-19T08:11:48.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:48 smithi157 ceph-mon[35143]: from='client.? 172.21.15.157:0/3685384214' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf"}]: dispatch 2024-09-19T08:11:48.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:48 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf"}]: dispatch 2024-09-19T08:11:48.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:48 smithi157 ceph-mon[35143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf"}]': finished 2024-09-19T08:11:48.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:48 smithi157 ceph-mon[35143]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T08:11:48.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:11:48.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:48 smithi022 ceph-mon[30963]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 347 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:11:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:48 smithi022 ceph-mon[30963]: osdmap e72: 11 total, 11 up, 11 in 2024-09-19T08:11:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:48 smithi022 ceph-mon[30963]: from='client.? 172.21.15.157:0/3685384214' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf"}]: dispatch 2024-09-19T08:11:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:48 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf"}]: dispatch 2024-09-19T08:11:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:48 smithi022 ceph-mon[30963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf"}]': finished 2024-09-19T08:11:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:48 smithi022 ceph-mon[30963]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T08:11:48.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:11:48.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:48 smithi064 ceph-mon[34900]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 347 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:11:48.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:48 smithi064 ceph-mon[34900]: osdmap e72: 11 total, 11 up, 11 in 2024-09-19T08:11:48.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:48 smithi064 ceph-mon[34900]: from='client.? 172.21.15.157:0/3685384214' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf"}]: dispatch 2024-09-19T08:11:48.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:48 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf"}]: dispatch 2024-09-19T08:11:48.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:48 smithi064 ceph-mon[34900]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf"}]': finished 2024-09-19T08:11:48.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:48 smithi064 ceph-mon[34900]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T08:11:48.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:11:49.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:49 smithi157 ceph-mon[35143]: from='client.? 172.21.15.157:0/2395564822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:11:49.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:49 smithi157 ceph-mon[35143]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T08:11:49.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:11:49.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:49 smithi022 ceph-mon[30963]: from='client.? 172.21.15.157:0/2395564822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:11:49.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:49 smithi022 ceph-mon[30963]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T08:11:49.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:11:49.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:49 smithi064 ceph-mon[34900]: from='client.? 172.21.15.157:0/2395564822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:11:49.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:49 smithi064 ceph-mon[34900]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T08:11:49.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:11:51.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:50 smithi157 ceph-mon[35143]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 347 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:11:51.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:50 smithi022 ceph-mon[30963]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 347 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:11:51.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:50 smithi064 ceph-mon[34900]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 347 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:11:52.928 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:52 smithi157 ceph-mon[35143]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-19T08:11:53.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:52 smithi022 ceph-mon[30963]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-19T08:11:53.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:52 smithi064 ceph-mon[34900]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-19T08:11:54.995 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:54 smithi157 ceph-mon[35143]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:11:55.093 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:54 smithi064 ceph-mon[34900]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:11:55.094 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:54 smithi022 ceph-mon[30963]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:11:56.997 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:56 smithi157 ceph-mon[35143]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-19T08:11:56.997 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:56 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T08:11:56.997 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:56 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:57.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:56 smithi022 ceph-mon[30963]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-19T08:11:57.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:56 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T08:11:57.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:56 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:57.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:56 smithi064 ceph-mon[34900]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-19T08:11:57.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:56 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T08:11:57.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:56 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:11:58.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:57 smithi157 ceph-mon[35143]: Deploying daemon osd.11 on smithi157 2024-09-19T08:11:58.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:57 smithi022 ceph-mon[30963]: Deploying daemon osd.11 on smithi157 2024-09-19T08:11:58.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:57 smithi064 ceph-mon[34900]: Deploying daemon osd.11 on smithi157 2024-09-19T08:11:59.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:11:58 smithi157 ceph-mon[35143]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-19T08:11:59.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:11:58 smithi022 ceph-mon[30963]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-19T08:11:59.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:11:58 smithi064 ceph-mon[34900]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-19T08:12:00.957 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:00 smithi157 ceph-mon[35143]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T08:12:01.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:00 smithi022 ceph-mon[30963]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T08:12:01.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:00 smithi064 ceph-mon[34900]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T08:12:02.950 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:02 smithi157 ceph-mon[35143]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:12:02.950 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:02 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:02.950 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:02 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:02.950 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:02 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:03.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:02 smithi022 ceph-mon[30963]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:12:03.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:02 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:03.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:02 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:03.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:02 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:03.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:02 smithi064 ceph-mon[34900]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:12:03.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:02 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:03.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:02 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:03.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:02 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:05.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:04 smithi157 ceph-mon[35143]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:05.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:04 smithi022 ceph-mon[30963]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:05.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:04 smithi064 ceph-mon[34900]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:06.063 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 11 on host 'smithi157' 2024-09-19T08:12:06.064 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:06.062+0000 7f8c34ff9640 1 -- 172.21.15.157:0/3477962852 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8c381a29c0 con 0x7f8c240a32c0 2024-09-19T08:12:06.069 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:06.064+0000 7f8c3d48e640 1 -- 172.21.15.157:0/3477962852 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8c240a32c0 msgr2=0x7f8c240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:06.069 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:06.064+0000 7f8c3d48e640 1 --2- 172.21.15.157:0/3477962852 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8c240a32c0 0x7f8c240a5780 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8c38101b70 tx=0x7f8c28009110 comp rx=0 tx=0).stop 2024-09-19T08:12:06.069 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:06.065+0000 7f8c3d48e640 1 -- 172.21.15.157:0/3477962852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c381026d0 msgr2=0x7f8c38103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:06.069 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:06.065+0000 7f8c3d48e640 1 --2- 172.21.15.157:0/3477962852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c381026d0 0x7f8c38103630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000eaa0 tx=0x7f8c3000eea0 comp rx=0 tx=0).stop 2024-09-19T08:12:06.069 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:06.065+0000 7f8c3d48e640 1 -- 172.21.15.157:0/3477962852 shutdown_connections 2024-09-19T08:12:06.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:06.065+0000 7f8c3d48e640 1 --2- 172.21.15.157:0/3477962852 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8c240a32c0 0x7f8c240a5780 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:06.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:06.065+0000 7f8c3d48e640 1 --2- 172.21.15.157:0/3477962852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c381026d0 0x7f8c38103630 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:06.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:06.065+0000 7f8c3d48e640 1 --2- 172.21.15.157:0/3477962852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c38101d10 0x7f8c381030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:06.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:06.065+0000 7f8c3d48e640 1 --2- 172.21.15.157:0/3477962852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c38100b10 0x7f8c38104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:06.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:06.065+0000 7f8c3d48e640 1 -- 172.21.15.157:0/3477962852 >> 172.21.15.157:0/3477962852 conn(0x7f8c380fc2a0 msgr2=0x7f8c3806a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:06.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:06.065+0000 7f8c3d48e640 1 -- 172.21.15.157:0/3477962852 shutdown_connections 2024-09-19T08:12:06.070 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:06.065+0000 7f8c3d48e640 1 -- 172.21.15.157:0/3477962852 wait complete. 2024-09-19T08:12:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:06 smithi064 ceph-mon[34900]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.726 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:06 smithi157 ceph-mon[35143]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:06.727 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.727 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.727 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:06.727 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:06.727 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.727 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:06.727 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.727 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.754 DEBUG:teuthology.orchestra.run.smithi157:osd.11> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T08:12:06.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:06 smithi022 ceph-mon[30963]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:06.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:06.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:06.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:06.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:06.758 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T08:12:06.758 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T08:12:07.027 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:07.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.510+0000 7fb2796ec640 1 -- 172.21.15.22:0/3805947484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2741037c0 msgr2=0x7fb27410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:07.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.510+0000 7fb2796ec640 1 --2- 172.21.15.22:0/3805947484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2741037c0 0x7fb27410a050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009ca0 tx=0x7fb26c02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:12:07.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.510+0000 7fb2796ec640 1 -- 172.21.15.22:0/3805947484 shutdown_connections 2024-09-19T08:12:07.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.510+0000 7fb2796ec640 1 --2- 172.21.15.22:0/3805947484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2741037c0 0x7fb27410a050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:07.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.510+0000 7fb2796ec640 1 --2- 172.21.15.22:0/3805947484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb274102e00 0x7fb274103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:07.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.510+0000 7fb2796ec640 1 --2- 172.21.15.22:0/3805947484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb274101c00 0x7fb274102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.510+0000 7fb2796ec640 1 -- 172.21.15.22:0/3805947484 >> 172.21.15.22:0/3805947484 conn(0x7fb2740fd390 msgr2=0x7fb2740ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.510+0000 7fb2796ec640 1 -- 172.21.15.22:0/3805947484 shutdown_connections 2024-09-19T08:12:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.510+0000 7fb2796ec640 1 -- 172.21.15.22:0/3805947484 wait complete. 2024-09-19T08:12:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2796ec640 1 Processor -- start 2024-09-19T08:12:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2796ec640 1 -- start start 2024-09-19T08:12:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2796ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb274101c00 0x7fb274197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2796ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb274102e00 0x7fb274198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2796ec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2741037c0 0x7fb27419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2796ec640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb27410d860 con 0x7fb274102e00 2024-09-19T08:12:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2796ec640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb27410d6e0 con 0x7fb274101c00 2024-09-19T08:12:07.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2796ec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb27410d9e0 con 0x7fb2741037c0 2024-09-19T08:12:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2727fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb274102e00 0x7fb274198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2727fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb274102e00 0x7fb274198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39110/0 (socket says 172.21.15.22:39110) 2024-09-19T08:12:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2727fc640 1 -- 172.21.15.22:0/3373946426 learned_addr learned my addr 172.21.15.22:0/3373946426 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2727fc640 1 -- 172.21.15.22:0/3373946426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2741037c0 msgr2=0x7fb27419f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2727fc640 1 --2- 172.21.15.22:0/3373946426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2741037c0 0x7fb27419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2727fc640 1 -- 172.21.15.22:0/3373946426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb274101c00 msgr2=0x7fb274197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.511+0000 7fb2727fc640 1 --2- 172.21.15.22:0/3373946426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb274101c00 0x7fb274197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.512+0000 7fb2727fc640 1 -- 172.21.15.22:0/3373946426 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27419fa80 con 0x7fb274102e00 2024-09-19T08:12:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.512+0000 7fb2727fc640 1 --2- 172.21.15.22:0/3373946426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb274102e00 0x7fb274198390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb26800e9f0 tx=0x7fb26800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.512+0000 7fb25bfff640 1 -- 172.21.15.22:0/3373946426 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26800cd80 con 0x7fb274102e00 2024-09-19T08:12:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.512+0000 7fb25bfff640 1 -- 172.21.15.22:0/3373946426 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb268010910 con 0x7fb274102e00 2024-09-19T08:12:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.512+0000 7fb2796ec640 1 -- 172.21.15.22:0/3373946426 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27419fd70 con 0x7fb274102e00 2024-09-19T08:12:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.512+0000 7fb25bfff640 1 -- 172.21.15.22:0/3373946426 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268018990 con 0x7fb274102e00 2024-09-19T08:12:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.512+0000 7fb2796ec640 1 -- 172.21.15.22:0/3373946426 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2741a0250 con 0x7fb274102e00 2024-09-19T08:12:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.513+0000 7fb2796ec640 1 -- 172.21.15.22:0/3373946426 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb274108f50 con 0x7fb274102e00 2024-09-19T08:12:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.514+0000 7fb25bfff640 1 -- 172.21.15.22:0/3373946426 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb268010ac0 con 0x7fb274102e00 2024-09-19T08:12:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.515+0000 7fb25bfff640 1 --2- 172.21.15.22:0/3373946426 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb25c0a3270 0x7fb25c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.515+0000 7fb272ffd640 1 --2- 172.21.15.22:0/3373946426 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb25c0a3270 0x7fb25c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.515+0000 7fb25bfff640 1 -- 172.21.15.22:0/3373946426 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6809+0+0 (secure 0 0 0) 0x7fb2680cfb50 con 0x7fb274102e00 2024-09-19T08:12:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.516+0000 7fb272ffd640 1 --2- 172.21.15.22:0/3373946426 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb25c0a3270 0x7fb25c0a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb274102c60 tx=0x7fb264005c10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.519+0000 7fb25bfff640 1 -- 172.21.15.22:0/3373946426 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb26809ce50 con 0x7fb274102e00 2024-09-19T08:12:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.696+0000 7fb2796ec640 1 -- 172.21.15.22:0/3373946426 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb27402da70 con 0x7fb274102e00 2024-09-19T08:12:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.697+0000 7fb25bfff640 1 -- 172.21.15.22:0/3373946426 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7fb26809c750 con 0x7fb274102e00 2024-09-19T08:12:07.698 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:12:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.699+0000 7fb2796ec640 1 -- 172.21.15.22:0/3373946426 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb25c0a3270 msgr2=0x7fb25c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.699+0000 7fb2796ec640 1 --2- 172.21.15.22:0/3373946426 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb25c0a3270 0x7fb25c0a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb274102c60 tx=0x7fb264005c10 comp rx=0 tx=0).stop 2024-09-19T08:12:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.699+0000 7fb2796ec640 1 -- 172.21.15.22:0/3373946426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb274102e00 msgr2=0x7fb274198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.699+0000 7fb2796ec640 1 --2- 172.21.15.22:0/3373946426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb274102e00 0x7fb274198390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb26800e9f0 tx=0x7fb26800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:12:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.699+0000 7fb2796ec640 1 -- 172.21.15.22:0/3373946426 shutdown_connections 2024-09-19T08:12:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.699+0000 7fb2796ec640 1 --2- 172.21.15.22:0/3373946426 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb25c0a3270 0x7fb25c0a5730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.699+0000 7fb2796ec640 1 --2- 172.21.15.22:0/3373946426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2741037c0 0x7fb27419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.699+0000 7fb2796ec640 1 --2- 172.21.15.22:0/3373946426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb274102e00 0x7fb274198390 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.699+0000 7fb2796ec640 1 --2- 172.21.15.22:0/3373946426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb274101c00 0x7fb274197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.699+0000 7fb2796ec640 1 -- 172.21.15.22:0/3373946426 >> 172.21.15.22:0/3373946426 conn(0x7fb2740fd390 msgr2=0x7fb274106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.699+0000 7fb2796ec640 1 -- 172.21.15.22:0/3373946426 shutdown_connections 2024-09-19T08:12:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:07.699+0000 7fb2796ec640 1 -- 172.21.15.22:0/3373946426 wait complete. 2024-09-19T08:12:08.246 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:12:07 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:12:07.904+0000 7fcfcbc94640 -1 osd.11 0 log_to_monitors true 2024-09-19T08:12:08.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:08 smithi022 ceph-mon[30963]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:08.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:08 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3373946426' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:12:08.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:08 smithi022 ceph-mon[30963]: from='osd.11 [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:12:08.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:08 smithi022 ceph-mon[30963]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:12:08.480 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726733505,"num_in_osds":12,"osd_in_since":1726733507,"num_remapped_pgs":0} 2024-09-19T08:12:08.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:08 smithi157 ceph-mon[35143]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:08.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:08 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3373946426' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:12:08.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:08 smithi157 ceph-mon[35143]: from='osd.11 [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:12:08.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:08 smithi157 ceph-mon[35143]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:12:08.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:08 smithi064 ceph-mon[34900]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:08.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:08 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3373946426' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:12:08.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:08 smithi064 ceph-mon[34900]: from='osd.11 [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:12:08.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:08 smithi064 ceph-mon[34900]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:12:09.481 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T08:12:09.585 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:09 smithi157 ceph-mon[35143]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T08:12:09.585 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:09 smithi157 ceph-mon[35143]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T08:12:09.585 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:09 smithi157 ceph-mon[35143]: from='osd.11 [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:12:09.585 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:12:09.585 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:09 smithi157 ceph-mon[35143]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:12:09.586 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:12:09 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:12:09.340+0000 7fcfc7c17640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T08:12:09.586 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:12:09 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:12:09.352+0000 7fcfc3a3f640 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:12:09.756 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:09.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:09 smithi022 ceph-mon[30963]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T08:12:09.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:09 smithi022 ceph-mon[30963]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T08:12:09.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:09 smithi022 ceph-mon[30963]: from='osd.11 [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:12:09.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:12:09.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:09 smithi022 ceph-mon[30963]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:12:09.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:09 smithi064 ceph-mon[34900]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T08:12:09.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:09 smithi064 ceph-mon[34900]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T08:12:09.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:09 smithi064 ceph-mon[34900]: from='osd.11 [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:12:09.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:12:09.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:09 smithi064 ceph-mon[34900]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-19T08:12:10.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.208+0000 7f807d394640 1 -- 172.21.15.22:0/2257721744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80781037c0 msgr2=0x7f807810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:10.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.208+0000 7f807d394640 1 --2- 172.21.15.22:0/2257721744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80781037c0 0x7f807810a050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8070009ca0 tx=0x7f807002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:12:10.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.208+0000 7f807d394640 1 -- 172.21.15.22:0/2257721744 shutdown_connections 2024-09-19T08:12:10.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.208+0000 7f807d394640 1 --2- 172.21.15.22:0/2257721744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80781037c0 0x7f807810a050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:10.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.208+0000 7f807d394640 1 --2- 172.21.15.22:0/2257721744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8078102e00 0x7f8078103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:10.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.208+0000 7f807d394640 1 --2- 172.21.15.22:0/2257721744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8078101c00 0x7f8078102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:10.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.208+0000 7f807d394640 1 -- 172.21.15.22:0/2257721744 >> 172.21.15.22:0/2257721744 conn(0x7f80780fd390 msgr2=0x7f80780ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:10.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.208+0000 7f807d394640 1 -- 172.21.15.22:0/2257721744 shutdown_connections 2024-09-19T08:12:10.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.208+0000 7f807d394640 1 -- 172.21.15.22:0/2257721744 wait complete. 2024-09-19T08:12:10.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.208+0000 7f807d394640 1 Processor -- start 2024-09-19T08:12:10.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f807d394640 1 -- start start 2024-09-19T08:12:10.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f807d394640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8078101c00 0x7f8078197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:10.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f807d394640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8078102e00 0x7f80781983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:10.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f807d394640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80781037c0 0x7f807819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:10.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f807d394640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f807810d860 con 0x7f8078102e00 2024-09-19T08:12:10.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f807d394640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f807810d6e0 con 0x7f80781037c0 2024-09-19T08:12:10.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f807d394640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f807810d9e0 con 0x7f8078101c00 2024-09-19T08:12:10.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f80767fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8078102e00 0x7f80781983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:10.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f80767fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8078102e00 0x7f80781983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39134/0 (socket says 172.21.15.22:39134) 2024-09-19T08:12:10.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f80767fc640 1 -- 172.21.15.22:0/1577516718 learned_addr learned my addr 172.21.15.22:0/1577516718 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:10.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f8076ffd640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8078101c00 0x7f8078197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:10.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f80777fe640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80781037c0 0x7f807819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:10.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f80767fc640 1 -- 172.21.15.22:0/1577516718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8078101c00 msgr2=0x7f8078197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:10.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f80767fc640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8078101c00 0x7f8078197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:10.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f80767fc640 1 -- 172.21.15.22:0/1577516718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80781037c0 msgr2=0x7f807819f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:10.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f80767fc640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80781037c0 0x7f807819f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:10.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f80767fc640 1 -- 172.21.15.22:0/1577516718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807819fab0 con 0x7f8078102e00 2024-09-19T08:12:10.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f80777fe640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80781037c0 0x7f807819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:12:10.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.209+0000 7f8076ffd640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8078101c00 0x7f8078197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:12:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.210+0000 7f80767fc640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8078102e00 0x7f80781983c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f806800d950 tx=0x7f806800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.210+0000 7f806ffff640 1 -- 172.21.15.22:0/1577516718 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068017030 con 0x7f8078102e00 2024-09-19T08:12:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.210+0000 7f807d394640 1 -- 172.21.15.22:0/1577516718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807819fda0 con 0x7f8078102e00 2024-09-19T08:12:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.210+0000 7f806ffff640 1 -- 172.21.15.22:0/1577516718 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8068004020 con 0x7f8078102e00 2024-09-19T08:12:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.210+0000 7f806ffff640 1 -- 172.21.15.22:0/1577516718 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80680047a0 con 0x7f8078102e00 2024-09-19T08:12:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.210+0000 7f807d394640 1 -- 172.21.15.22:0/1577516718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80781a0260 con 0x7f8078102e00 2024-09-19T08:12:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.211+0000 7f807d394640 1 -- 172.21.15.22:0/1577516718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8078109000 con 0x7f8078102e00 2024-09-19T08:12:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.212+0000 7f806ffff640 1 -- 172.21.15.22:0/1577516718 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f80680041d0 con 0x7f8078102e00 2024-09-19T08:12:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.213+0000 7f806ffff640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f80640a3270 0x7f80640a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.213+0000 7f8076ffd640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f80640a3270 0x7f80640a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.213+0000 7f806ffff640 1 -- 172.21.15.22:0/1577516718 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6847+0+0 (secure 0 0 0) 0x7f80680cf4b0 con 0x7f8078102e00 2024-09-19T08:12:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.213+0000 7f8076ffd640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f80640a3270 0x7f80640a5730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f80600058c0 tx=0x7f8060005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.216+0000 7f806ffff640 1 -- 172.21.15.22:0/1577516718 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f806809c6f0 con 0x7f8078102e00 2024-09-19T08:12:10.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.397+0000 7f807d394640 1 -- 172.21.15.22:0/1577516718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f807802d9e0 con 0x7f8078102e00 2024-09-19T08:12:10.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.397+0000 7f806ffff640 1 -- 172.21.15.22:0/1577516718 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7f806809bff0 con 0x7f8078102e00 2024-09-19T08:12:10.397 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:12:10.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:10.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T08:12:10.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: osdmap e76: 12 total, 11 up, 12 in 2024-09-19T08:12:10.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:12:10.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:12:10.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: Detected new or changed devices on smithi157 2024-09-19T08:12:10.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:10.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:10.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:12:10.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:12:10.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:12:10.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:12:10.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi157 to 4272M 2024-09-19T08:12:10.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:10.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:10.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:10.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:10.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.399+0000 7f807d394640 1 -- 172.21.15.22:0/1577516718 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f80640a3270 msgr2=0x7f80640a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:10.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.399+0000 7f807d394640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f80640a3270 0x7f80640a5730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f80600058c0 tx=0x7f8060005850 comp rx=0 tx=0).stop 2024-09-19T08:12:10.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.399+0000 7f807d394640 1 -- 172.21.15.22:0/1577516718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8078102e00 msgr2=0x7f80781983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:10.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.399+0000 7f807d394640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8078102e00 0x7f80781983c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f806800d950 tx=0x7f806800de10 comp rx=0 tx=0).stop 2024-09-19T08:12:10.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.399+0000 7f807d394640 1 -- 172.21.15.22:0/1577516718 shutdown_connections 2024-09-19T08:12:10.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.399+0000 7f807d394640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f80640a3270 0x7f80640a5730 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:10.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.399+0000 7f807d394640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80781037c0 0x7f807819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:10.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.399+0000 7f807d394640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8078102e00 0x7f80781983c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:10.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.399+0000 7f807d394640 1 --2- 172.21.15.22:0/1577516718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8078101c00 0x7f8078197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:10.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.399+0000 7f807d394640 1 -- 172.21.15.22:0/1577516718 >> 172.21.15.22:0/1577516718 conn(0x7f80780fd390 msgr2=0x7f80781061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:10.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.399+0000 7f807d394640 1 -- 172.21.15.22:0/1577516718 shutdown_connections 2024-09-19T08:12:10.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:10.399+0000 7f807d394640 1 -- 172.21.15.22:0/1577516718 wait complete. 2024-09-19T08:12:10.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:10.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T08:12:10.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: osdmap e76: 12 total, 11 up, 12 in 2024-09-19T08:12:10.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:12:10.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:12:10.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: Detected new or changed devices on smithi157 2024-09-19T08:12:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:12:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:12:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:12:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:12:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi157 to 4272M 2024-09-19T08:12:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:10.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:10.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:10.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-19T08:12:10.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: osdmap e76: 12 total, 11 up, 12 in 2024-09-19T08:12:10.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:12:10.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:12:10.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: Detected new or changed devices on smithi157 2024-09-19T08:12:10.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:10.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:10.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:12:10.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:12:10.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:12:10.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:12:10.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi157 to 4272M 2024-09-19T08:12:10.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:10.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:10.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:10.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:11.089 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1726733530,"num_in_osds":12,"osd_in_since":1726733507,"num_remapped_pgs":0} 2024-09-19T08:12:11.090 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:12:11.348 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:11.675 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:11 smithi022 ceph-mon[30963]: purged_snaps scrub starts 2024-09-19T08:12:11.675 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:11 smithi022 ceph-mon[30963]: purged_snaps scrub ok 2024-09-19T08:12:11.675 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:11 smithi022 ceph-mon[30963]: osd.11 [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194] boot 2024-09-19T08:12:11.675 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:11 smithi022 ceph-mon[30963]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T08:12:11.675 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:11 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:12:11.675 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:11 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1577516718' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:12:11.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:11 smithi157 ceph-mon[35143]: purged_snaps scrub starts 2024-09-19T08:12:11.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:11 smithi157 ceph-mon[35143]: purged_snaps scrub ok 2024-09-19T08:12:11.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:11 smithi157 ceph-mon[35143]: osd.11 [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194] boot 2024-09-19T08:12:11.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:11 smithi157 ceph-mon[35143]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T08:12:11.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:11 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:12:11.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:11 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1577516718' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:12:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:11 smithi064 ceph-mon[34900]: purged_snaps scrub starts 2024-09-19T08:12:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:11 smithi064 ceph-mon[34900]: purged_snaps scrub ok 2024-09-19T08:12:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:11 smithi064 ceph-mon[34900]: osd.11 [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194] boot 2024-09-19T08:12:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:11 smithi064 ceph-mon[34900]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T08:12:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:11 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:12:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:11 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1577516718' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:12:11.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.885+0000 7fa4c56a0640 1 -- 172.21.15.22:0/101645365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4c0102d30 msgr2=0x7fa4c0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:11.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.885+0000 7fa4c56a0640 1 --2- 172.21.15.22:0/101645365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4c0102d30 0x7fa4c0103110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009cd0 tx=0x7fa4b002a8e0 comp rx=0 tx=0).stop 2024-09-19T08:12:11.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.886+0000 7fa4c56a0640 1 -- 172.21.15.22:0/101645365 shutdown_connections 2024-09-19T08:12:11.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.886+0000 7fa4c56a0640 1 --2- 172.21.15.22:0/101645365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c01082a0 0x7fa4c010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:11.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.886+0000 7fa4c56a0640 1 --2- 172.21.15.22:0/101645365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c01036e0 0x7fa4c0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:11.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.886+0000 7fa4c56a0640 1 --2- 172.21.15.22:0/101645365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4c0102d30 0x7fa4c0103110 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:11.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.886+0000 7fa4c56a0640 1 -- 172.21.15.22:0/101645365 >> 172.21.15.22:0/101645365 conn(0x7fa4c00fc7d0 msgr2=0x7fa4c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:11.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.886+0000 7fa4c56a0640 1 -- 172.21.15.22:0/101645365 shutdown_connections 2024-09-19T08:12:11.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.886+0000 7fa4c56a0640 1 -- 172.21.15.22:0/101645365 wait complete. 2024-09-19T08:12:11.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.886+0000 7fa4c56a0640 1 Processor -- start 2024-09-19T08:12:11.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.886+0000 7fa4c56a0640 1 -- start start 2024-09-19T08:12:11.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4c56a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4c0102d30 0x7fa4c0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:11.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4c56a0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c01036e0 0x7fa4c0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:11.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4c56a0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c01082a0 0x7fa4c0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:11.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4c56a0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c0112100 con 0x7fa4c0102d30 2024-09-19T08:12:11.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4c56a0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c0111f80 con 0x7fa4c01082a0 2024-09-19T08:12:11.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4c56a0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c0112280 con 0x7fa4c01036e0 2024-09-19T08:12:11.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4beffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4c0102d30 0x7fa4c0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:11.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4beffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4c0102d30 0x7fa4c0100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39162/0 (socket says 172.21.15.22:39162) 2024-09-19T08:12:11.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4beffd640 1 -- 172.21.15.22:0/1133402652 learned_addr learned my addr 172.21.15.22:0/1133402652 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:11.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4be7fc640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c01036e0 0x7fa4c0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:11.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4bf7fe640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c01082a0 0x7fa4c0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:11.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4beffd640 1 -- 172.21.15.22:0/1133402652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c01036e0 msgr2=0x7fa4c0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:11.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4beffd640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c01036e0 0x7fa4c0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:11.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4beffd640 1 -- 172.21.15.22:0/1133402652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c01082a0 msgr2=0x7fa4c0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:11.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4beffd640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c01082a0 0x7fa4c0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4beffd640 1 -- 172.21.15.22:0/1133402652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c01a6ac0 con 0x7fa4c0102d30 2024-09-19T08:12:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4bf7fe640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c01082a0 0x7fa4c0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:12:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4beffd640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4c0102d30 0x7fa4c0100990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa4b002ae00 tx=0x7fa4b0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4b7fff640 1 -- 172.21.15.22:0/1133402652 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0004040 con 0x7fa4c0102d30 2024-09-19T08:12:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4c56a0640 1 -- 172.21.15.22:0/1133402652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c01a6d50 con 0x7fa4c0102d30 2024-09-19T08:12:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4b7fff640 1 -- 172.21.15.22:0/1133402652 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4b00046c0 con 0x7fa4c0102d30 2024-09-19T08:12:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4b7fff640 1 -- 172.21.15.22:0/1133402652 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0046480 con 0x7fa4c0102d30 2024-09-19T08:12:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.887+0000 7fa4c56a0640 1 -- 172.21.15.22:0/1133402652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c01a7250 con 0x7fa4c0102d30 2024-09-19T08:12:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.889+0000 7fa4c56a0640 1 -- 172.21.15.22:0/1133402652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c0103110 con 0x7fa4c0102d30 2024-09-19T08:12:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.889+0000 7fa4b7fff640 1 -- 172.21.15.22:0/1133402652 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa4b00305a0 con 0x7fa4c0102d30 2024-09-19T08:12:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.890+0000 7fa4b7fff640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa4ac0a3240 0x7fa4ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.890+0000 7fa4be7fc640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa4ac0a3240 0x7fa4ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.890+0000 7fa4b7fff640 1 -- 172.21.15.22:0/1133402652 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fa4b002e030 con 0x7fa4c0102d30 2024-09-19T08:12:11.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.891+0000 7fa4be7fc640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa4ac0a3240 0x7fa4ac0a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0102420 tx=0x7fa4a800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:11.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:11.895+0000 7fa4b7fff640 1 -- 172.21.15.22:0/1133402652 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa4b004b060 con 0x7fa4c0102d30 2024-09-19T08:12:12.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:12.071+0000 7fa4c56a0640 1 -- 172.21.15.22:0/1133402652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4c010a2b0 con 0x7fa4c0102d30 2024-09-19T08:12:12.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:12.072+0000 7fa4b7fff640 1 -- 172.21.15.22:0/1133402652 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+18432 (secure 0 0 0) 0x7fa4b0030890 con 0x7fa4c0102d30 2024-09-19T08:12:12.073 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:12:12.073 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":78,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","created":"2024-09-19T08:05:19.476952+0000","modified":"2024-09-19T08:12:11.347952+0000","last_up_change":"2024-09-19T08:12:10.339586+0000","last_in_change":"2024-09-19T08:11:47.653576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:08:25.175759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ed67f2d-9b28-427a-b69f-676e7a2abcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6803","nonce":3442715307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6805","nonce":3442715307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6809","nonce":3442715307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6807","nonce":3442715307}]},"public_addr":"172.21.15.22:6803/3442715307","cluster_addr":"172.21.15.22:6805/3442715307","heartbeat_back_addr":"172.21.15.22:6809/3442715307","heartbeat_front_addr":"172.21.15.22:6807/3442715307","state":["exists","up"]},{"osd":1,"uuid":"bff3e283-4ab1-49d7-86d8-45d361995122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6801","nonce":2379700228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6803","nonce":2379700228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6807","nonce":2379700228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6805","nonce":2379700228}]},"public_addr":"172.21.15.64:6801/2379700228","cluster_addr":"172.21.15.64:6803/2379700228","heartbeat_back_addr":"172.21.15.64:6807/2379700228","heartbeat_front_addr":"172.21.15.64:6805/2379700228","state":["exists","up"]},{"osd":2,"uuid":"81421d08-7dc6-419b-a566-dee1e1e13b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6801","nonce":3363263867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6803","nonce":3363263867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6807","nonce":3363263867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6805","nonce":3363263867}]},"public_addr":"172.21.15.157:6801/3363263867","cluster_addr":"172.21.15.157:6803/3363263867","heartbeat_back_addr":"172.21.15.157:6807/3363263867","heartbeat_front_addr":"172.21.15.157:6805/3363263867","state":["exists","up"]},{"osd":3,"uuid":"34debfd0-3226-4d78-b9e6-e03dbc5190f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6811","nonce":4225229456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6813","nonce":4225229456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6817","nonce":4225229456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6815","nonce":4225229456}]},"public_addr":"172.21.15.22:6811/4225229456","cluster_addr":"172.21.15.22:6813/4225229456","heartbeat_back_addr":"172.21.15.22:6817/4225229456","heartbeat_front_addr":"172.21.15.22:6815/4225229456","state":["exists","up"]},{"osd":4,"uuid":"1293332c-d772-4ca9-bf76-79e0b29451ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6809","nonce":1858090423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6811","nonce":1858090423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6815","nonce":1858090423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6813","nonce":1858090423}]},"public_addr":"172.21.15.64:6809/1858090423","cluster_addr":"172.21.15.64:6811/1858090423","heartbeat_back_addr":"172.21.15.64:6815/1858090423","heartbeat_front_addr":"172.21.15.64:6813/1858090423","state":["exists","up"]},{"osd":5,"uuid":"240a6c42-89a2-41c1-948c-b09914b5722e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6809","nonce":2900701212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2900701212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2900701212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2900701212}]},"public_addr":"172.21.15.157:6809/2900701212","cluster_addr":"172.21.15.157:6811/2900701212","heartbeat_back_addr":"172.21.15.157:6815/2900701212","heartbeat_front_addr":"172.21.15.157:6813/2900701212","state":["exists","up"]},{"osd":6,"uuid":"80199030-109e-41c4-bc1c-731755464770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6819","nonce":4264426242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6821","nonce":4264426242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6825","nonce":4264426242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6823","nonce":4264426242}]},"public_addr":"172.21.15.22:6819/4264426242","cluster_addr":"172.21.15.22:6821/4264426242","heartbeat_back_addr":"172.21.15.22:6825/4264426242","heartbeat_front_addr":"172.21.15.22:6823/4264426242","state":["exists","up"]},{"osd":7,"uuid":"dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6817","nonce":3980224103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6819","nonce":3980224103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6823","nonce":3980224103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6821","nonce":3980224103}]},"public_addr":"172.21.15.64:6817/3980224103","cluster_addr":"172.21.15.64:6819/3980224103","heartbeat_back_addr":"172.21.15.64:6823/3980224103","heartbeat_front_addr":"172.21.15.64:6821/3980224103","state":["exists","up"]},{"osd":8,"uuid":"764b1e6e-48c9-49f9-8124-bc13bcba3a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6817","nonce":973678477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6819","nonce":973678477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6823","nonce":973678477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6821","nonce":973678477}]},"public_addr":"172.21.15.157:6817/973678477","cluster_addr":"172.21.15.157:6819/973678477","heartbeat_back_addr":"172.21.15.157:6823/973678477","heartbeat_front_addr":"172.21.15.157:6821/973678477","state":["exists","up"]},{"osd":9,"uuid":"82914818-df47-4374-9e3e-2f71fc76e78b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6827","nonce":710033990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6829","nonce":710033990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6833","nonce":710033990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6831","nonce":710033990}]},"public_addr":"172.21.15.22:6827/710033990","cluster_addr":"172.21.15.22:6829/710033990","heartbeat_back_addr":"172.21.15.22:6833/710033990","heartbeat_front_addr":"172.21.15.22:6831/710033990","state":["exists","up"]},{"osd":10,"uuid":"36ae177d-c454-4cba-ba21-cbeee0f10dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6825","nonce":373412752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6827","nonce":373412752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6831","nonce":373412752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6829","nonce":373412752}]},"public_addr":"172.21.15.64:6825/373412752","cluster_addr":"172.21.15.64:6827/373412752","heartbeat_back_addr":"172.21.15.64:6831/373412752","heartbeat_front_addr":"172.21.15.64:6829/373412752","state":["exists","up"]},{"osd":11,"uuid":"cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6825","nonce":1918889194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6827","nonce":1918889194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6831","nonce":1918889194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6829","nonce":1918889194}]},"public_addr":"172.21.15.157:6825/1918889194","cluster_addr":"172.21.15.157:6827/1918889194","heartbeat_back_addr":"172.21.15.157:6831/1918889194","heartbeat_front_addr":"172.21.15.157:6829/1918889194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:31.378218+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:56.794167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:20.790267+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:47.039106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:11.483842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:38.151466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:03.553308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:27.865321+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:53.086530+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:18.946661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:43.697243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:12:08.936588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3712426751":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/2133059277":"2024-09-20T08:05:42.354701+0000","172.21.15.22:6801/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2851750481":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/3623845502":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/384027596":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/586163474":"2024-09-20T08:05:42.354701+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:12:12.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:12.074+0000 7fa4c56a0640 1 -- 172.21.15.22:0/1133402652 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa4ac0a3240 msgr2=0x7fa4ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:12.074+0000 7fa4c56a0640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa4ac0a3240 0x7fa4ac0a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0102420 tx=0x7fa4a800b390 comp rx=0 tx=0).stop 2024-09-19T08:12:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:12.074+0000 7fa4c56a0640 1 -- 172.21.15.22:0/1133402652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4c0102d30 msgr2=0x7fa4c0100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:12.074+0000 7fa4c56a0640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4c0102d30 0x7fa4c0100990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa4b002ae00 tx=0x7fa4b0003970 comp rx=0 tx=0).stop 2024-09-19T08:12:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:12.074+0000 7fa4c56a0640 1 -- 172.21.15.22:0/1133402652 shutdown_connections 2024-09-19T08:12:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:12.074+0000 7fa4c56a0640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa4ac0a3240 0x7fa4ac0a5700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:12.074+0000 7fa4c56a0640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c01082a0 0x7fa4c0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:12.074+0000 7fa4c56a0640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4c01036e0 0x7fa4c0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:12.074+0000 7fa4c56a0640 1 --2- 172.21.15.22:0/1133402652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4c0102d30 0x7fa4c0100990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:12.074+0000 7fa4c56a0640 1 -- 172.21.15.22:0/1133402652 >> 172.21.15.22:0/1133402652 conn(0x7fa4c00fc7d0 msgr2=0x7fa4c0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:12.075+0000 7fa4c56a0640 1 -- 172.21.15.22:0/1133402652 shutdown_connections 2024-09-19T08:12:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:12.075+0000 7fa4c56a0640 1 -- 172.21.15.22:0/1133402652 wait complete. 2024-09-19T08:12:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:12 smithi157 ceph-mon[35143]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:12 smithi157 ceph-mon[35143]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T08:12:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:12 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1133402652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:12:12.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:12 smithi022 ceph-mon[30963]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:12.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:12 smithi022 ceph-mon[30963]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T08:12:12.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:12 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1133402652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:12:12.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:12 smithi064 ceph-mon[34900]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:12:12.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:12 smithi064 ceph-mon[34900]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T08:12:12.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:12 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1133402652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:12:12.840 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:08:25.175759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:12:12.841 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T08:12:13.097 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:13.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.587+0000 7f2ce2597640 1 -- 172.21.15.22:0/140723950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cdc10c180 msgr2=0x7f2cdc10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:13.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.587+0000 7f2ce2597640 1 --2- 172.21.15.22:0/140723950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cdc10c180 0x7f2cdc10e600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009ca0 tx=0x7f2cd402a950 comp rx=0 tx=0).stop 2024-09-19T08:12:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.587+0000 7f2ce2597640 1 -- 172.21.15.22:0/140723950 shutdown_connections 2024-09-19T08:12:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.587+0000 7f2ce2597640 1 --2- 172.21.15.22:0/140723950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cdc10c180 0x7f2cdc10e600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.587+0000 7f2ce2597640 1 --2- 172.21.15.22:0/140723950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc1036c0 0x7f2cdc10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.587+0000 7f2ce2597640 1 --2- 172.21.15.22:0/140723950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc102da0 0x7f2cdc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.587+0000 7f2ce2597640 1 -- 172.21.15.22:0/140723950 >> 172.21.15.22:0/140723950 conn(0x7f2cdc0fc7a0 msgr2=0x7f2cdc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.587+0000 7f2ce2597640 1 -- 172.21.15.22:0/140723950 shutdown_connections 2024-09-19T08:12:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.587+0000 7f2ce2597640 1 -- 172.21.15.22:0/140723950 wait complete. 2024-09-19T08:12:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.588+0000 7f2ce2597640 1 Processor -- start 2024-09-19T08:12:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.588+0000 7f2ce2597640 1 -- start start 2024-09-19T08:12:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.588+0000 7f2ce2597640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc102da0 0x7f2cdc1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.588+0000 7f2ce2597640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc1036c0 0x7f2cdc1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.588+0000 7f2ce2597640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cdc10c180 0x7f2cdc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.588+0000 7f2ce2597640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc111f20 con 0x7f2cdc10c180 2024-09-19T08:12:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.588+0000 7f2ce2597640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc111da0 con 0x7f2cdc1036c0 2024-09-19T08:12:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.588+0000 7f2ce2597640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc1120a0 con 0x7f2cdc102da0 2024-09-19T08:12:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.588+0000 7f2ce0b34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cdc10c180 0x7f2cdc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.588+0000 7f2ce0b34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cdc10c180 0x7f2cdc19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39190/0 (socket says 172.21.15.22:39190) 2024-09-19T08:12:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.588+0000 7f2ce0b34640 1 -- 172.21.15.22:0/1771034971 learned_addr learned my addr 172.21.15.22:0/1771034971 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.588+0000 7f2cdbfff640 1 --2- 172.21.15.22:0/1771034971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc102da0 0x7f2cdc1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.589+0000 7f2cd3fff640 1 --2- 172.21.15.22:0/1771034971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc1036c0 0x7f2cdc1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.589+0000 7f2ce0b34640 1 -- 172.21.15.22:0/1771034971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc102da0 msgr2=0x7f2cdc1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.589+0000 7f2ce0b34640 1 --2- 172.21.15.22:0/1771034971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc102da0 0x7f2cdc1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.589+0000 7f2ce0b34640 1 -- 172.21.15.22:0/1771034971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc1036c0 msgr2=0x7f2cdc1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.589+0000 7f2ce0b34640 1 --2- 172.21.15.22:0/1771034971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc1036c0 0x7f2cdc1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.589+0000 7f2ce0b34640 1 -- 172.21.15.22:0/1771034971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cdc19d360 con 0x7f2cdc10c180 2024-09-19T08:12:13.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.589+0000 7f2ce0b34640 1 --2- 172.21.15.22:0/1771034971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cdc10c180 0x7f2cdc19cc80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2cd402ae70 tx=0x7f2cd402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:13.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.589+0000 7f2cd9ffb640 1 -- 172.21.15.22:0/1771034971 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4034040 con 0x7f2cdc10c180 2024-09-19T08:12:13.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.589+0000 7f2cd9ffb640 1 -- 172.21.15.22:0/1771034971 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd4030920 con 0x7f2cdc10c180 2024-09-19T08:12:13.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.589+0000 7f2ce2597640 1 -- 172.21.15.22:0/1771034971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cdc19d5f0 con 0x7f2cdc10c180 2024-09-19T08:12:13.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.589+0000 7f2cd9ffb640 1 -- 172.21.15.22:0/1771034971 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4045480 con 0x7f2cdc10c180 2024-09-19T08:12:13.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.589+0000 7f2ce2597640 1 -- 172.21.15.22:0/1771034971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cdc1a4f30 con 0x7f2cdc10c180 2024-09-19T08:12:13.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.590+0000 7f2ce2597640 1 -- 172.21.15.22:0/1771034971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cdc103180 con 0x7f2cdc10c180 2024-09-19T08:12:13.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.591+0000 7f2cd9ffb640 1 -- 172.21.15.22:0/1771034971 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2cd4037020 con 0x7f2cdc10c180 2024-09-19T08:12:13.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.592+0000 7f2cd9ffb640 1 --2- 172.21.15.22:0/1771034971 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2cc80a3240 0x7f2cc80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:13.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.592+0000 7f2cdbfff640 1 --2- 172.21.15.22:0/1771034971 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2cc80a3240 0x7f2cc80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:13.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.592+0000 7f2cd9ffb640 1 -- 172.21.15.22:0/1771034971 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f2cd40f0430 con 0x7f2cdc10c180 2024-09-19T08:12:13.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.593+0000 7f2cdbfff640 1 --2- 172.21.15.22:0/1771034971 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2cc80a3240 0x7f2cc80a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc005d00 tx=0x7f2ccc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:13.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.597+0000 7f2cd9ffb640 1 -- 172.21.15.22:0/1771034971 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2cd40bd550 con 0x7f2cdc10c180 2024-09-19T08:12:13.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.772+0000 7f2ce2597640 1 -- 172.21.15.22:0/1771034971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2cdc107280 con 0x7f2cdc10c180 2024-09-19T08:12:13.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.772+0000 7f2cd9ffb640 1 -- 172.21.15.22:0/1771034971 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f2cd40bce50 con 0x7f2cdc10c180 2024-09-19T08:12:13.772 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-09-19T08:12:13.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.774+0000 7f2ce2597640 1 -- 172.21.15.22:0/1771034971 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2cc80a3240 msgr2=0x7f2cc80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:13.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.774+0000 7f2ce2597640 1 --2- 172.21.15.22:0/1771034971 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2cc80a3240 0x7f2cc80a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc005d00 tx=0x7f2ccc00b410 comp rx=0 tx=0).stop 2024-09-19T08:12:13.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.774+0000 7f2ce2597640 1 -- 172.21.15.22:0/1771034971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cdc10c180 msgr2=0x7f2cdc19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:13.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.774+0000 7f2ce2597640 1 --2- 172.21.15.22:0/1771034971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cdc10c180 0x7f2cdc19cc80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2cd402ae70 tx=0x7f2cd402f5b0 comp rx=0 tx=0).stop 2024-09-19T08:12:13.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.774+0000 7f2ce2597640 1 -- 172.21.15.22:0/1771034971 shutdown_connections 2024-09-19T08:12:13.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.774+0000 7f2ce2597640 1 --2- 172.21.15.22:0/1771034971 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2cc80a3240 0x7f2cc80a5700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:13.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.774+0000 7f2ce2597640 1 --2- 172.21.15.22:0/1771034971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cdc10c180 0x7f2cdc19cc80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:13.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.774+0000 7f2ce2597640 1 --2- 172.21.15.22:0/1771034971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cdc1036c0 0x7f2cdc1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:13.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.774+0000 7f2ce2597640 1 --2- 172.21.15.22:0/1771034971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc102da0 0x7f2cdc1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:13.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.774+0000 7f2ce2597640 1 -- 172.21.15.22:0/1771034971 >> 172.21.15.22:0/1771034971 conn(0x7f2cdc0fc7a0 msgr2=0x7f2cdc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:13.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.774+0000 7f2ce2597640 1 -- 172.21.15.22:0/1771034971 shutdown_connections 2024-09-19T08:12:13.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:13.774+0000 7f2ce2597640 1 -- 172.21.15.22:0/1771034971 wait complete. 2024-09-19T08:12:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:14 smithi022 ceph-mon[30963]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:14 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1771034971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:12:14.723 INFO:tasks.cephadm:Adding mds.a on smithi022 2024-09-19T08:12:14.723 INFO:tasks.cephadm:Adding mds.d on smithi022 2024-09-19T08:12:14.723 INFO:tasks.cephadm:Adding mds.g on smithi022 2024-09-19T08:12:14.723 INFO:tasks.cephadm:Adding mds.j on smithi022 2024-09-19T08:12:14.723 INFO:tasks.cephadm:Adding mds.b on smithi064 2024-09-19T08:12:14.724 INFO:tasks.cephadm:Adding mds.e on smithi064 2024-09-19T08:12:14.724 INFO:tasks.cephadm:Adding mds.h on smithi064 2024-09-19T08:12:14.724 INFO:tasks.cephadm:Adding mds.k on smithi064 2024-09-19T08:12:14.724 INFO:tasks.cephadm:Adding mds.c on smithi157 2024-09-19T08:12:14.724 INFO:tasks.cephadm:Adding mds.f on smithi157 2024-09-19T08:12:14.724 INFO:tasks.cephadm:Adding mds.i on smithi157 2024-09-19T08:12:14.724 INFO:tasks.cephadm:Adding mds.l on smithi157 2024-09-19T08:12:14.724 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l' 2024-09-19T08:12:14.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:14 smithi157 ceph-mon[35143]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:14.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:14 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1771034971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:12:14.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:14 smithi064 ceph-mon[34900]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:14.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:14 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1771034971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:12:14.987 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:12:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.489+0000 7f8be7eac640 1 -- 172.21.15.157:0/3690788610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be0102660 msgr2=0x7f8be0104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.489+0000 7f8be7eac640 1 --2- 172.21.15.157:0/3690788610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be0102660 0x7f8be0104a50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009900 tx=0x7f8bd002a410 comp rx=0 tx=0).stop 2024-09-19T08:12:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.489+0000 7f8be7eac640 1 -- 172.21.15.157:0/3690788610 shutdown_connections 2024-09-19T08:12:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.490+0000 7f8be7eac640 1 --2- 172.21.15.157:0/3690788610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be01078c0 0x7f8be0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.490+0000 7f8be7eac640 1 --2- 172.21.15.157:0/3690788610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8be0104f90 0x7f8be0107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.490+0000 7f8be7eac640 1 --2- 172.21.15.157:0/3690788610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be0102660 0x7f8be0104a50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.490+0000 7f8be7eac640 1 -- 172.21.15.157:0/3690788610 >> 172.21.15.157:0/3690788610 conn(0x7f8be00fc280 msgr2=0x7f8be00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.490+0000 7f8be7eac640 1 -- 172.21.15.157:0/3690788610 shutdown_connections 2024-09-19T08:12:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.490+0000 7f8be7eac640 1 -- 172.21.15.157:0/3690788610 wait complete. 2024-09-19T08:12:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.490+0000 7f8be7eac640 1 Processor -- start 2024-09-19T08:12:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.490+0000 7f8be7eac640 1 -- start start 2024-09-19T08:12:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.490+0000 7f8be7eac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be0102660 0x7f8be0101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.490+0000 7f8be7eac640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0104f90 0x7f8be0100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.490+0000 7f8be7eac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8be01078c0 0x7f8be0100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be7eac640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8be010d8e0 con 0x7f8be0104f90 2024-09-19T08:12:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be7eac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8be010d760 con 0x7f8be01078c0 2024-09-19T08:12:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be7eac640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8be010da60 con 0x7f8be0102660 2024-09-19T08:12:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be5c48640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be0102660 0x7f8be0101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be5c48640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be0102660 0x7f8be0101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60472/0 (socket says 172.21.15.157:60472) 2024-09-19T08:12:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be5c48640 1 -- 172.21.15.157:0/2747011478 learned_addr learned my addr 172.21.15.157:0/2747011478 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-19T08:12:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be6449640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8be01078c0 0x7f8be0100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be5447640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0104f90 0x7f8be0100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be5c48640 1 -- 172.21.15.157:0/2747011478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8be01078c0 msgr2=0x7f8be0100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be5c48640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8be01078c0 0x7f8be0100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be5c48640 1 -- 172.21.15.157:0/2747011478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0104f90 msgr2=0x7f8be0100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be5c48640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0104f90 0x7f8be0100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be5c48640 1 -- 172.21.15.157:0/2747011478 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be0101410 con 0x7f8be0102660 2024-09-19T08:12:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be5447640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0104f90 0x7f8be0100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:12:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be5c48640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be0102660 0x7f8be0101fc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009230 tx=0x7f8bd0004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8bd6ffd640 1 -- 172.21.15.157:0/2747011478 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd003f030 con 0x7f8be0102660 2024-09-19T08:12:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be7eac640 1 -- 172.21.15.157:0/2747011478 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be01a20a0 con 0x7f8be0102660 2024-09-19T08:12:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8bd6ffd640 1 -- 172.21.15.157:0/2747011478 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd0003ce0 con 0x7f8be0102660 2024-09-19T08:12:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8bd6ffd640 1 -- 172.21.15.157:0/2747011478 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0035070 con 0x7f8be0102660 2024-09-19T08:12:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.491+0000 7f8be7eac640 1 -- 172.21.15.157:0/2747011478 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be01a2620 con 0x7f8be0102660 2024-09-19T08:12:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.492+0000 7f8be7eac640 1 -- 172.21.15.157:0/2747011478 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be00fdd70 con 0x7f8be0102660 2024-09-19T08:12:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.494+0000 7f8bd6ffd640 1 -- 172.21.15.157:0/2747011478 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8bd0042030 con 0x7f8be0102660 2024-09-19T08:12:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.494+0000 7f8bd6ffd640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8bcc0a3240 0x7f8bcc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.495+0000 7f8bd6ffd640 1 -- 172.21.15.157:0/2747011478 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f8bd00efe20 con 0x7f8be0102660 2024-09-19T08:12:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.495+0000 7f8be5447640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8bcc0a3240 0x7f8bcc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.495+0000 7f8be5447640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8bcc0a3240 0x7f8bcc0a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8be0101d40 tx=0x7f8bdc005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.503+0000 7f8bd6ffd640 1 -- 172.21.15.157:0/2747011478 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8bd00bcfe0 con 0x7f8be0102660 2024-09-19T08:12:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.686+0000 7f8be7eac640 1 -- 172.21.15.157:0/2747011478 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}) -- 0x7f8be0062730 con 0x7f8bcc0a3240 2024-09-19T08:12:15.696 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.694+0000 7f8bd6ffd640 1 -- 172.21.15.157:0/2747011478 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8be0062730 con 0x7f8bcc0a3240 2024-09-19T08:12:15.696 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mds.all update... 2024-09-19T08:12:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.697+0000 7f8be7eac640 1 -- 172.21.15.157:0/2747011478 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8bcc0a3240 msgr2=0x7f8bcc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.697+0000 7f8be7eac640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8bcc0a3240 0x7f8bcc0a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8be0101d40 tx=0x7f8bdc005e50 comp rx=0 tx=0).stop 2024-09-19T08:12:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.697+0000 7f8be7eac640 1 -- 172.21.15.157:0/2747011478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be0102660 msgr2=0x7f8be0101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.697+0000 7f8be7eac640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be0102660 0x7f8be0101fc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009230 tx=0x7f8bd0004360 comp rx=0 tx=0).stop 2024-09-19T08:12:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.697+0000 7f8be7eac640 1 -- 172.21.15.157:0/2747011478 shutdown_connections 2024-09-19T08:12:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.697+0000 7f8be7eac640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8bcc0a3240 0x7f8bcc0a5700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.697+0000 7f8be7eac640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8be01078c0 0x7f8be0100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.697+0000 7f8be7eac640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0104f90 0x7f8be0100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.697+0000 7f8be7eac640 1 --2- 172.21.15.157:0/2747011478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be0102660 0x7f8be0101fc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.697+0000 7f8be7eac640 1 -- 172.21.15.157:0/2747011478 >> 172.21.15.157:0/2747011478 conn(0x7f8be00fc280 msgr2=0x7f8be0102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.697+0000 7f8be7eac640 1 -- 172.21.15.157:0/2747011478 shutdown_connections 2024-09-19T08:12:15.700 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-19T08:12:15.697+0000 7f8be7eac640 1 -- 172.21.15.157:0/2747011478 wait complete. 2024-09-19T08:12:16.536 DEBUG:teuthology.orchestra.run.smithi022:mds.a> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T08:12:16.540 DEBUG:teuthology.orchestra.run.smithi022:mds.d> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T08:12:16.583 DEBUG:teuthology.orchestra.run.smithi022:mds.g> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T08:12:16.626 DEBUG:teuthology.orchestra.run.smithi022:mds.j> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T08:12:16.669 DEBUG:teuthology.orchestra.run.smithi064:mds.b> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T08:12:16.671 DEBUG:teuthology.orchestra.run.smithi064:mds.e> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T08:12:16.715 DEBUG:teuthology.orchestra.run.smithi064:mds.h> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T08:12:16.759 DEBUG:teuthology.orchestra.run.smithi064:mds.k> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T08:12:16.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:16 smithi157 ceph-mon[35143]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:16.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:16 smithi157 ceph-mon[35143]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:12:16.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:16 smithi157 ceph-mon[35143]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-09-19T08:12:16.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:16 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:16.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:16 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:16.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:16 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:16.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:16 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:16.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:16 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:16.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:16 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:16.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:16 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:16.778 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:16 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:16.802 DEBUG:teuthology.orchestra.run.smithi157:mds.c> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T08:12:16.805 DEBUG:teuthology.orchestra.run.smithi157:mds.f> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T08:12:16.849 DEBUG:teuthology.orchestra.run.smithi157:mds.i> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T08:12:16.891 DEBUG:teuthology.orchestra.run.smithi157:mds.l> sudo journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T08:12:16.935 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T08:12:16.935 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:12:16.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:16 smithi022 ceph-mon[30963]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:16.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:16 smithi022 ceph-mon[30963]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:12:16.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:16 smithi022 ceph-mon[30963]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-09-19T08:12:16.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:16 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:16.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:16 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:16.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:16 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:16.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:16 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:16.962 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:16 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:16.962 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:16 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:16.962 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:16 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:16.962 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:16 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:17.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:16 smithi064 ceph-mon[34900]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:17.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:16 smithi064 ceph-mon[34900]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:12:17.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:16 smithi064 ceph-mon[34900]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-09-19T08:12:17.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:16 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:17.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:16 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:17.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:16 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:17.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:16 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:17.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:16 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:17.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:16 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:17.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:16 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:17.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:16 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:17.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.663+0000 7ff56e737640 1 -- 172.21.15.22:0/3979539586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff568108290 msgr2=0x7ff56810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:17.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.663+0000 7ff56e737640 1 --2- 172.21.15.22:0/3979539586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff568108290 0x7ff56810ec30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff560009cd0 tx=0x7ff56002a8e0 comp rx=0 tx=0).stop 2024-09-19T08:12:17.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.663+0000 7ff56e737640 1 -- 172.21.15.22:0/3979539586 shutdown_connections 2024-09-19T08:12:17.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.663+0000 7ff56e737640 1 --2- 172.21.15.22:0/3979539586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff568108290 0x7ff56810ec30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:17.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.663+0000 7ff56e737640 1 --2- 172.21.15.22:0/3979539586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff568103740 0x7ff568107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:17.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.663+0000 7ff56e737640 1 --2- 172.21.15.22:0/3979539586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff568102d90 0x7ff568103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:17.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.663+0000 7ff56e737640 1 -- 172.21.15.22:0/3979539586 >> 172.21.15.22:0/3979539586 conn(0x7ff5680fc7d0 msgr2=0x7ff5680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:17.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.663+0000 7ff56e737640 1 -- 172.21.15.22:0/3979539586 shutdown_connections 2024-09-19T08:12:17.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.663+0000 7ff56e737640 1 -- 172.21.15.22:0/3979539586 wait complete. 2024-09-19T08:12:17.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff56e737640 1 Processor -- start 2024-09-19T08:12:17.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff56e737640 1 -- start start 2024-09-19T08:12:17.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff56e737640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff568102d90 0x7ff568198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:17.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff56e737640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff568103740 0x7ff568198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:17.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff56e737640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff568108290 0x7ff56819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:17.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff56e737640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff568112090 con 0x7ff568103740 2024-09-19T08:12:17.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff56e737640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff568111f10 con 0x7ff568102d90 2024-09-19T08:12:17.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff56e737640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff568112210 con 0x7ff568108290 2024-09-19T08:12:17.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff55ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff568103740 0x7ff568198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff55ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff568103740 0x7ff568198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38332/0 (socket says 172.21.15.22:38332) 2024-09-19T08:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff55ffff640 1 -- 172.21.15.22:0/2147817105 learned_addr learned my addr 172.21.15.22:0/2147817105 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff567fff640 1 --2- 172.21.15.22:0/2147817105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff568102d90 0x7ff568198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff56ccd4640 1 --2- 172.21.15.22:0/2147817105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff568108290 0x7ff56819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff55ffff640 1 -- 172.21.15.22:0/2147817105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff568108290 msgr2=0x7ff56819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff55ffff640 1 --2- 172.21.15.22:0/2147817105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff568108290 0x7ff56819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff55ffff640 1 -- 172.21.15.22:0/2147817105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff568102d90 msgr2=0x7ff568198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff55ffff640 1 --2- 172.21.15.22:0/2147817105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff568102d90 0x7ff568198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.664+0000 7ff55ffff640 1 -- 172.21.15.22:0/2147817105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56819d5e0 con 0x7ff568103740 2024-09-19T08:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.665+0000 7ff55ffff640 1 --2- 172.21.15.22:0/2147817105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff568103740 0x7ff568198b70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff55000d770 tx=0x7ff55000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.665+0000 7ff565ffb640 1 -- 172.21.15.22:0/2147817105 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550017070 con 0x7ff568103740 2024-09-19T08:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.665+0000 7ff565ffb640 1 -- 172.21.15.22:0/2147817105 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff550009c30 con 0x7ff568103740 2024-09-19T08:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.665+0000 7ff56e737640 1 -- 172.21.15.22:0/2147817105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff56819d8d0 con 0x7ff568103740 2024-09-19T08:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.665+0000 7ff565ffb640 1 -- 172.21.15.22:0/2147817105 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55000f620 con 0x7ff568103740 2024-09-19T08:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.665+0000 7ff56e737640 1 -- 172.21.15.22:0/2147817105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5681a51b0 con 0x7ff568103740 2024-09-19T08:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.666+0000 7ff56e737640 1 -- 172.21.15.22:0/2147817105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff568103170 con 0x7ff568103740 2024-09-19T08:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.667+0000 7ff565ffb640 1 -- 172.21.15.22:0/2147817105 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff55001a070 con 0x7ff568103740 2024-09-19T08:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.668+0000 7ff565ffb640 1 --2- 172.21.15.22:0/2147817105 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff5540a3240 0x7ff5540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.668+0000 7ff567fff640 1 --2- 172.21.15.22:0/2147817105 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff5540a3240 0x7ff5540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.668+0000 7ff565ffb640 1 -- 172.21.15.22:0/2147817105 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7ff5500ceef0 con 0x7ff568103740 2024-09-19T08:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.668+0000 7ff567fff640 1 --2- 172.21.15.22:0/2147817105 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff5540a3240 0x7ff5540a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff55800bbe0 tx=0x7ff55800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.672+0000 7ff565ffb640 1 -- 172.21.15.22:0/2147817105 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff550009de0 con 0x7ff568103740 2024-09-19T08:12:17.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:17 smithi157 ceph-mon[35143]: Deploying daemon mds.l on smithi157 2024-09-19T08:12:17.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.887+0000 7ff56e737640 1 -- 172.21.15.22:0/2147817105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff56810a2a0 con 0x7ff568103740 2024-09-19T08:12:17.888 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:17 smithi022 ceph-mon[30963]: Deploying daemon mds.l on smithi157 2024-09-19T08:12:17.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.888+0000 7ff565ffb640 1 -- 172.21.15.22:0/2147817105 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7ff55009c0b0 con 0x7ff568103740 2024-09-19T08:12:17.889 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:12:17.889 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":1,"btime":"2024-09-19T08:05:19:476769+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T08:12:17.889 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 1 2024-09-19T08:12:17.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.890+0000 7ff56e737640 1 -- 172.21.15.22:0/2147817105 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff5540a3240 msgr2=0x7ff5540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:17.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.890+0000 7ff56e737640 1 --2- 172.21.15.22:0/2147817105 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff5540a3240 0x7ff5540a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff55800bbe0 tx=0x7ff55800b390 comp rx=0 tx=0).stop 2024-09-19T08:12:17.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.890+0000 7ff56e737640 1 -- 172.21.15.22:0/2147817105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff568103740 msgr2=0x7ff568198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:17.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.890+0000 7ff56e737640 1 --2- 172.21.15.22:0/2147817105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff568103740 0x7ff568198b70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff55000d770 tx=0x7ff55000dc30 comp rx=0 tx=0).stop 2024-09-19T08:12:17.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.891+0000 7ff56e737640 1 -- 172.21.15.22:0/2147817105 shutdown_connections 2024-09-19T08:12:17.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.891+0000 7ff56e737640 1 --2- 172.21.15.22:0/2147817105 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff5540a3240 0x7ff5540a5700 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:17.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.891+0000 7ff56e737640 1 --2- 172.21.15.22:0/2147817105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff568108290 0x7ff56819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:17.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.891+0000 7ff56e737640 1 --2- 172.21.15.22:0/2147817105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff568103740 0x7ff568198b70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:17.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.891+0000 7ff56e737640 1 --2- 172.21.15.22:0/2147817105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff568102d90 0x7ff568198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:17.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.891+0000 7ff56e737640 1 -- 172.21.15.22:0/2147817105 >> 172.21.15.22:0/2147817105 conn(0x7ff5680fc7d0 msgr2=0x7ff568105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:17.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.891+0000 7ff56e737640 1 -- 172.21.15.22:0/2147817105 shutdown_connections 2024-09-19T08:12:17.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:17.891+0000 7ff56e737640 1 -- 172.21.15.22:0/2147817105 wait complete. 2024-09-19T08:12:18.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:17 smithi064 ceph-mon[34900]: Deploying daemon mds.l on smithi157 2024-09-19T08:12:18.465 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:12:18 smithi157 systemd[1]: Starting Ceph mds.l for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:12:18.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:18 smithi157 ceph-mon[35143]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:18.731 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:18 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2147817105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:18.731 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:12:18 smithi157 podman[64748]: 2024-09-19 08:12:18.368000542 +0000 UTC m=+0.016104772 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:12:18.731 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:12:18 smithi157 podman[64748]: 2024-09-19 08:12:18.470441145 +0000 UTC m=+0.118545405 container create d09a90274cdad782e9eace4cf0dd852674d73624655044a5241db379df48eacb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-19T08:12:18.731 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:12:18 smithi157 podman[64748]: 2024-09-19 08:12:18.731782586 +0000 UTC m=+0.379886809 container init d09a90274cdad782e9eace4cf0dd852674d73624655044a5241db379df48eacb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:12:18.988 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:18 smithi064 ceph-mon[34900]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:18.988 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:18 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2147817105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:19.003 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:12:18 smithi157 podman[64748]: 2024-09-19 08:12:18.734901956 +0000 UTC m=+0.383006182 container start d09a90274cdad782e9eace4cf0dd852674d73624655044a5241db379df48eacb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T08:12:19.003 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:12:18 smithi157 ceph-mds[64767]: -- 172.21.15.157:0/247298091 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ab64615860 con 0x55ab645f3000 2024-09-19T08:12:19.003 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:12:18 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-l[64763]: starting mds.l at 2024-09-19T08:12:19.003 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:12:18 smithi157 bash[64748]: d09a90274cdad782e9eace4cf0dd852674d73624655044a5241db379df48eacb 2024-09-19T08:12:19.003 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:12:18 smithi157 systemd[1]: Started Ceph mds.l for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:12:19.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:18 smithi022 ceph-mon[30963]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:19.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:18 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2147817105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:20.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:19 smithi064 ceph-mon[34900]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:20.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:19 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.157:6832/2040635685,v1:172.21.15.157:6833/2040635685] up:boot 2024-09-19T08:12:20.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:19 smithi064 ceph-mon[34900]: fsmap 1 up:standby 2024-09-19T08:12:20.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:12:20.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:20.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:20.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:20.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:20.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:20.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:19 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:20.089 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:19 smithi064 ceph-mon[34900]: Deploying daemon mds.k on smithi064 2024-09-19T08:12:20.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:19 smithi157 ceph-mon[35143]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:20.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:19 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.157:6832/2040635685,v1:172.21.15.157:6833/2040635685] up:boot 2024-09-19T08:12:20.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:19 smithi157 ceph-mon[35143]: fsmap 1 up:standby 2024-09-19T08:12:20.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:12:20.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:20.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:20.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:20.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:20.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:20.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:19 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:20.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:19 smithi157 ceph-mon[35143]: Deploying daemon mds.k on smithi064 2024-09-19T08:12:20.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:19 smithi022 ceph-mon[30963]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:20.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:19 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.157:6832/2040635685,v1:172.21.15.157:6833/2040635685] up:boot 2024-09-19T08:12:20.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:19 smithi022 ceph-mon[30963]: fsmap 1 up:standby 2024-09-19T08:12:20.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:12:20.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:20.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:20.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:20.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:20.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:20.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:19 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:20.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:19 smithi022 ceph-mon[30963]: Deploying daemon mds.k on smithi064 2024-09-19T08:12:20.751 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:12:21.022 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:21.275 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:12:21 smithi064 systemd[1]: Starting Ceph mds.k for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:12:21.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.486+0000 7fd66f6ab640 1 -- 172.21.15.22:0/3298930851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd668101420 msgr2=0x7fd66810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:21.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.486+0000 7fd66f6ab640 1 --2- 172.21.15.22:0/3298930851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd668101420 0x7fd66810d950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd660009cd0 tx=0x7fd66002a8e0 comp rx=0 tx=0).stop 2024-09-19T08:12:21.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.486+0000 7fd66f6ab640 1 -- 172.21.15.22:0/3298930851 shutdown_connections 2024-09-19T08:12:21.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.486+0000 7fd66f6ab640 1 --2- 172.21.15.22:0/3298930851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd668101420 0x7fd66810d950 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:21.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.486+0000 7fd66f6ab640 1 --2- 172.21.15.22:0/3298930851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668100a80 0x7fd668100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:21.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.486+0000 7fd66f6ab640 1 --2- 172.21.15.22:0/3298930851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd668106a80 0x7fd668106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:21.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.486+0000 7fd66f6ab640 1 -- 172.21.15.22:0/3298930851 >> 172.21.15.22:0/3298930851 conn(0x7fd6680fc7a0 msgr2=0x7fd6680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:21.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.486+0000 7fd66f6ab640 1 -- 172.21.15.22:0/3298930851 shutdown_connections 2024-09-19T08:12:21.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.486+0000 7fd66f6ab640 1 -- 172.21.15.22:0/3298930851 wait complete. 2024-09-19T08:12:21.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66f6ab640 1 Processor -- start 2024-09-19T08:12:21.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66f6ab640 1 -- start start 2024-09-19T08:12:21.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66f6ab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd668100a80 0x7fd668197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:21.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66f6ab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668101420 0x7fd668197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:21.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66f6ab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd668106a80 0x7fd66819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:21.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66f6ab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd668110ed0 con 0x7fd668106a80 2024-09-19T08:12:21.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66f6ab640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd668110d50 con 0x7fd668100a80 2024-09-19T08:12:21.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66f6ab640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd668111050 con 0x7fd668101420 2024-09-19T08:12:21.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66dc48640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd668106a80 0x7fd66819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:21.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66dc48640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd668106a80 0x7fd66819bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38354/0 (socket says 172.21.15.22:38354) 2024-09-19T08:12:21.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66dc48640 1 -- 172.21.15.22:0/4274599797 learned_addr learned my addr 172.21.15.22:0/4274599797 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:21.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66d447640 1 --2- 172.21.15.22:0/4274599797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd668100a80 0x7fd668197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:21.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66cc46640 1 --2- 172.21.15.22:0/4274599797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668101420 0x7fd668197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66dc48640 1 -- 172.21.15.22:0/4274599797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668101420 msgr2=0x7fd668197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66dc48640 1 --2- 172.21.15.22:0/4274599797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668101420 0x7fd668197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.487+0000 7fd66dc48640 1 -- 172.21.15.22:0/4274599797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd668100a80 msgr2=0x7fd668197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.488+0000 7fd66dc48640 1 --2- 172.21.15.22:0/4274599797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd668100a80 0x7fd668197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.488+0000 7fd66dc48640 1 -- 172.21.15.22:0/4274599797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66819c2c0 con 0x7fd668106a80 2024-09-19T08:12:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.488+0000 7fd66dc48640 1 --2- 172.21.15.22:0/4274599797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd668106a80 0x7fd66819bbe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd660033040 tx=0x7fd660031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.488+0000 7fd65e7fc640 1 -- 172.21.15.22:0/4274599797 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660004040 con 0x7fd668106a80 2024-09-19T08:12:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.488+0000 7fd65e7fc640 1 -- 172.21.15.22:0/4274599797 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6600046c0 con 0x7fd668106a80 2024-09-19T08:12:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.488+0000 7fd66f6ab640 1 -- 172.21.15.22:0/4274599797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66819c550 con 0x7fd668106a80 2024-09-19T08:12:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.488+0000 7fd65e7fc640 1 -- 172.21.15.22:0/4274599797 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660046480 con 0x7fd668106a80 2024-09-19T08:12:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.488+0000 7fd66f6ab640 1 -- 172.21.15.22:0/4274599797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6681a3e90 con 0x7fd668106a80 2024-09-19T08:12:21.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.489+0000 7fd66f6ab640 1 -- 172.21.15.22:0/4274599797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd668100ee0 con 0x7fd668106a80 2024-09-19T08:12:21.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.490+0000 7fd65e7fc640 1 -- 172.21.15.22:0/4274599797 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd6600305a0 con 0x7fd668106a80 2024-09-19T08:12:21.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.491+0000 7fd65e7fc640 1 --2- 172.21.15.22:0/4274599797 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd6540a32c0 0x7fd6540a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:21.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.491+0000 7fd66d447640 1 --2- 172.21.15.22:0/4274599797 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd6540a32c0 0x7fd6540a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:21.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.491+0000 7fd65e7fc640 1 -- 172.21.15.22:0/4274599797 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fd66002e030 con 0x7fd668106a80 2024-09-19T08:12:21.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.491+0000 7fd66d447640 1 --2- 172.21.15.22:0/4274599797 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd6540a32c0 0x7fd6540a5780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd664009b50 tx=0x7fd664005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:21.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.495+0000 7fd65e7fc640 1 -- 172.21.15.22:0/4274599797 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd66004b060 con 0x7fd668106a80 2024-09-19T08:12:21.535 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:12:21 smithi064 podman[63581]: 2024-09-19 08:12:21.177547562 +0000 UTC m=+0.016175286 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:12:21.535 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:12:21 smithi064 podman[63581]: 2024-09-19 08:12:21.290874254 +0000 UTC m=+0.129502010 container create 32cd86275c8fa951bf9bfaad6f9952683376c44ee7e64c1887d92a247d40c2a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-k, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T08:12:21.535 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:12:21 smithi064 podman[63581]: 2024-09-19 08:12:21.535257487 +0000 UTC m=+0.373885212 container init 32cd86275c8fa951bf9bfaad6f9952683376c44ee7e64c1887d92a247d40c2a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-k, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T08:12:21.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.717+0000 7fd66f6ab640 1 -- 172.21.15.22:0/4274599797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6681a40f0 con 0x7fd668106a80 2024-09-19T08:12:21.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.718+0000 7fd65e7fc640 1 -- 172.21.15.22:0/4274599797 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fd6600bdaa0 con 0x7fd668106a80 2024-09-19T08:12:21.718 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:12:21.718 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":2,"btime":"2024-09-19T08:12:18:763094+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T08:12:21.719 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 2 2024-09-19T08:12:21.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.720+0000 7fd66f6ab640 1 -- 172.21.15.22:0/4274599797 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd6540a32c0 msgr2=0x7fd6540a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:21.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.720+0000 7fd66f6ab640 1 --2- 172.21.15.22:0/4274599797 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd6540a32c0 0x7fd6540a5780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd664009b50 tx=0x7fd664005f90 comp rx=0 tx=0).stop 2024-09-19T08:12:21.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.720+0000 7fd66f6ab640 1 -- 172.21.15.22:0/4274599797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd668106a80 msgr2=0x7fd66819bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:21.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.720+0000 7fd66f6ab640 1 --2- 172.21.15.22:0/4274599797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd668106a80 0x7fd66819bbe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd660033040 tx=0x7fd660031e60 comp rx=0 tx=0).stop 2024-09-19T08:12:21.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.720+0000 7fd66f6ab640 1 -- 172.21.15.22:0/4274599797 shutdown_connections 2024-09-19T08:12:21.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.720+0000 7fd66f6ab640 1 --2- 172.21.15.22:0/4274599797 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd6540a32c0 0x7fd6540a5780 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:21.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.720+0000 7fd66f6ab640 1 --2- 172.21.15.22:0/4274599797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd668106a80 0x7fd66819bbe0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:21.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.720+0000 7fd66f6ab640 1 --2- 172.21.15.22:0/4274599797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668101420 0x7fd668197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:21.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.720+0000 7fd66f6ab640 1 --2- 172.21.15.22:0/4274599797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd668100a80 0x7fd668197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:21.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.720+0000 7fd66f6ab640 1 -- 172.21.15.22:0/4274599797 >> 172.21.15.22:0/4274599797 conn(0x7fd6680fc7a0 msgr2=0x7fd6680fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:21.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.720+0000 7fd66f6ab640 1 -- 172.21.15.22:0/4274599797 shutdown_connections 2024-09-19T08:12:21.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:21.720+0000 7fd66f6ab640 1 -- 172.21.15.22:0/4274599797 wait complete. 2024-09-19T08:12:21.837 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:12:21 smithi064 podman[63581]: 2024-09-19 08:12:21.538296551 +0000 UTC m=+0.376924271 container start 32cd86275c8fa951bf9bfaad6f9952683376c44ee7e64c1887d92a247d40c2a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-k, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T08:12:21.837 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:12:21 smithi064 ceph-mds[63601]: -- 172.21.15.64:0/1339732777 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e3c5aa9860 con 0x55e3c5a86800 2024-09-19T08:12:21.837 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:12:21 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-k[63597]: starting mds.k at 2024-09-19T08:12:21.837 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:12:21 smithi064 bash[63581]: 32cd86275c8fa951bf9bfaad6f9952683376c44ee7e64c1887d92a247d40c2a8 2024-09-19T08:12:21.837 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:12:21 smithi064 systemd[1]: Started Ceph mds.k for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:12:22.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:22 smithi022 ceph-mon[30963]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:22.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:22 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:22.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:22 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:22.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:22 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:22.915 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:22 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:22.915 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:22 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:22.915 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:22 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:22.915 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:22 smithi022 ceph-mon[30963]: Deploying daemon mds.j on smithi022 2024-09-19T08:12:22.915 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:22 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/4274599797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:22.915 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:22 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.64:6832/138036781,v1:172.21.15.64:6833/138036781] up:boot 2024-09-19T08:12:22.915 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:22 smithi022 ceph-mon[30963]: fsmap 2 up:standby 2024-09-19T08:12:22.915 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:22 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:12:23.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:22 smithi157 ceph-mon[35143]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:23.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:22 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:23.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:22 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:23.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:22 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:23.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:22 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:23.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:22 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:23.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:22 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:23.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:22 smithi157 ceph-mon[35143]: Deploying daemon mds.j on smithi022 2024-09-19T08:12:23.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:22 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/4274599797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:23.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:22 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.64:6832/138036781,v1:172.21.15.64:6833/138036781] up:boot 2024-09-19T08:12:23.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:22 smithi157 ceph-mon[35143]: fsmap 2 up:standby 2024-09-19T08:12:23.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:22 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:12:23.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:22 smithi064 ceph-mon[34900]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:23.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:22 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:23.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:22 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:23.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:22 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:23.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:22 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:23.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:22 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:23.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:22 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:23.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:22 smithi064 ceph-mon[34900]: Deploying daemon mds.j on smithi022 2024-09-19T08:12:23.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:22 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/4274599797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:23.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:22 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.64:6832/138036781,v1:172.21.15.64:6833/138036781] up:boot 2024-09-19T08:12:23.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:22 smithi064 ceph-mon[34900]: fsmap 2 up:standby 2024-09-19T08:12:23.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:22 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:12:24.507 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:12:24 smithi022 systemd[1]: Starting Ceph mds.j for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:12:24.602 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:12:24.828 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:24 smithi022 ceph-mon[30963]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:24.828 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:12:24 smithi022 podman[69618]: 2024-09-19 08:12:24.46896898 +0000 UTC m=+0.016564697 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:12:24.828 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:12:24 smithi022 podman[69618]: 2024-09-19 08:12:24.59212505 +0000 UTC m=+0.139720798 container create d180067f1c3f4250d1ec3d0f14e74a419c5400bd4be3cd5d3b32a9f3ed5c656f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T08:12:24.828 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:12:24 smithi022 podman[69618]: 2024-09-19 08:12:24.828857484 +0000 UTC m=+0.376453197 container init d180067f1c3f4250d1ec3d0f14e74a419c5400bd4be3cd5d3b32a9f3ed5c656f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T08:12:24.957 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:24 smithi064 ceph-mon[34900]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:25.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:24 smithi157 ceph-mon[35143]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:25.048 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:25.171 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:12:24 smithi022 podman[69618]: 2024-09-19 08:12:24.832418843 +0000 UTC m=+0.380014550 container start d180067f1c3f4250d1ec3d0f14e74a419c5400bd4be3cd5d3b32a9f3ed5c656f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True) 2024-09-19T08:12:25.171 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:12:24 smithi022 ceph-mds[69666]: -- 172.21.15.22:0/1789651279 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e10047860 con 0x556e10025000 2024-09-19T08:12:25.171 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:12:24 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-j[69651]: starting mds.j at 2024-09-19T08:12:25.171 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:12:24 smithi022 bash[69618]: d180067f1c3f4250d1ec3d0f14e74a419c5400bd4be3cd5d3b32a9f3ed5c656f 2024-09-19T08:12:25.171 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:12:24 smithi022 systemd[1]: Started Ceph mds.j for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:12:25.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.601+0000 7fb09b317640 1 -- 172.21.15.22:0/4094591283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb094102df0 msgr2=0x7fb0941031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:25.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.601+0000 7fb09b317640 1 --2- 172.21.15.22:0/4094591283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb094102df0 0x7fb0941031d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb090009900 tx=0x7fb09002a410 comp rx=0 tx=0).stop 2024-09-19T08:12:25.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.601+0000 7fb09b317640 1 -- 172.21.15.22:0/4094591283 shutdown_connections 2024-09-19T08:12:25.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.601+0000 7fb09b317640 1 --2- 172.21.15.22:0/4094591283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb09410c2f0 0x7fb09410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:25.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.601+0000 7fb09b317640 1 --2- 172.21.15.22:0/4094591283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0941037a0 0x7fb09410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:25.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.601+0000 7fb09b317640 1 --2- 172.21.15.22:0/4094591283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb094102df0 0x7fb0941031d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:25.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.601+0000 7fb09b317640 1 -- 172.21.15.22:0/4094591283 >> 172.21.15.22:0/4094591283 conn(0x7fb0940fc7d0 msgr2=0x7fb0940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:25.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.601+0000 7fb09b317640 1 -- 172.21.15.22:0/4094591283 shutdown_connections 2024-09-19T08:12:25.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.601+0000 7fb09b317640 1 -- 172.21.15.22:0/4094591283 wait complete. 2024-09-19T08:12:25.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.602+0000 7fb09b317640 1 Processor -- start 2024-09-19T08:12:25.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.602+0000 7fb09b317640 1 -- start start 2024-09-19T08:12:25.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.602+0000 7fb09b317640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb094102df0 0x7fb094198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:25.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.602+0000 7fb09b317640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0941037a0 0x7fb094198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:25.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.602+0000 7fb09b317640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb09410c2f0 0x7fb09419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:25.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.602+0000 7fb09b317640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb094112110 con 0x7fb0941037a0 2024-09-19T08:12:25.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.602+0000 7fb09b317640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb094111f90 con 0x7fb09410c2f0 2024-09-19T08:12:25.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.602+0000 7fb09b317640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb094112290 con 0x7fb094102df0 2024-09-19T08:12:25.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.602+0000 7fb0988b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0941037a0 0x7fb094198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:25.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.602+0000 7fb0988b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0941037a0 0x7fb094198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38394/0 (socket says 172.21.15.22:38394) 2024-09-19T08:12:25.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.602+0000 7fb0988b2640 1 -- 172.21.15.22:0/722694684 learned_addr learned my addr 172.21.15.22:0/722694684 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:25.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.603+0000 7fb0990b3640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb094102df0 0x7fb094198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:25.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.603+0000 7fb0998b4640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb09410c2f0 0x7fb09419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:25.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.603+0000 7fb0988b2640 1 -- 172.21.15.22:0/722694684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb094102df0 msgr2=0x7fb094198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:25.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.603+0000 7fb0988b2640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb094102df0 0x7fb094198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:25.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.603+0000 7fb0988b2640 1 -- 172.21.15.22:0/722694684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb09410c2f0 msgr2=0x7fb09419cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:25.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.603+0000 7fb0988b2640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb09410c2f0 0x7fb09419cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:25.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.603+0000 7fb0988b2640 1 -- 172.21.15.22:0/722694684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09419d5e0 con 0x7fb0941037a0 2024-09-19T08:12:25.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.603+0000 7fb0998b4640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb09410c2f0 0x7fb09419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:12:25.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.603+0000 7fb0988b2640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0941037a0 0x7fb094198b70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb08400d750 tx=0x7fb08400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:25.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.603+0000 7fb08a7fc640 1 -- 172.21.15.22:0/722694684 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084017070 con 0x7fb0941037a0 2024-09-19T08:12:25.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.603+0000 7fb08a7fc640 1 -- 172.21.15.22:0/722694684 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb084005500 con 0x7fb0941037a0 2024-09-19T08:12:25.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.603+0000 7fb09b317640 1 -- 172.21.15.22:0/722694684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09419d8d0 con 0x7fb0941037a0 2024-09-19T08:12:25.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.603+0000 7fb08a7fc640 1 -- 172.21.15.22:0/722694684 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084004e00 con 0x7fb0941037a0 2024-09-19T08:12:25.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.603+0000 7fb09b317640 1 -- 172.21.15.22:0/722694684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0941a51b0 con 0x7fb0941037a0 2024-09-19T08:12:25.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.604+0000 7fb09b317640 1 -- 172.21.15.22:0/722694684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb09406a790 con 0x7fb0941037a0 2024-09-19T08:12:25.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.605+0000 7fb08a7fc640 1 -- 172.21.15.22:0/722694684 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb084004fa0 con 0x7fb0941037a0 2024-09-19T08:12:25.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.606+0000 7fb08a7fc640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb0800a32c0 0x7fb0800a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:25.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.606+0000 7fb0990b3640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb0800a32c0 0x7fb0800a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:25.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.606+0000 7fb08a7fc640 1 -- 172.21.15.22:0/722694684 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fb0840cedb0 con 0x7fb0941037a0 2024-09-19T08:12:25.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.606+0000 7fb0990b3640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb0800a32c0 0x7fb0800a5780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb090009230 tx=0x7fb0900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:25.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.610+0000 7fb08a7fc640 1 -- 172.21.15.22:0/722694684 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb08409bf70 con 0x7fb0941037a0 2024-09-19T08:12:25.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.831+0000 7fb09b317640 1 -- 172.21.15.22:0/722694684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0941072a0 con 0x7fb0941037a0 2024-09-19T08:12:25.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.831+0000 7fb08a7fc640 1 -- 172.21.15.22:0/722694684 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7fb08409b870 con 0x7fb0941037a0 2024-09-19T08:12:25.832 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:12:25.832 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":4,"btime":"2024-09-19T08:12:25:686670+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T08:12:25.832 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 4 2024-09-19T08:12:25.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.834+0000 7fb09b317640 1 -- 172.21.15.22:0/722694684 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb0800a32c0 msgr2=0x7fb0800a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:25.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.834+0000 7fb09b317640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb0800a32c0 0x7fb0800a5780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb090009230 tx=0x7fb0900058d0 comp rx=0 tx=0).stop 2024-09-19T08:12:25.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.834+0000 7fb09b317640 1 -- 172.21.15.22:0/722694684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0941037a0 msgr2=0x7fb094198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:25.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.834+0000 7fb09b317640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0941037a0 0x7fb094198b70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb08400d750 tx=0x7fb08400dc10 comp rx=0 tx=0).stop 2024-09-19T08:12:25.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.834+0000 7fb09b317640 1 -- 172.21.15.22:0/722694684 shutdown_connections 2024-09-19T08:12:25.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.834+0000 7fb09b317640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb0800a32c0 0x7fb0800a5780 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:25.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.834+0000 7fb09b317640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb09410c2f0 0x7fb09419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:25.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.834+0000 7fb09b317640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0941037a0 0x7fb094198b70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:25.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.834+0000 7fb09b317640 1 --2- 172.21.15.22:0/722694684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb094102df0 0x7fb094198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:25.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.834+0000 7fb09b317640 1 -- 172.21.15.22:0/722694684 >> 172.21.15.22:0/722694684 conn(0x7fb0940fc7d0 msgr2=0x7fb0940fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:25.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.834+0000 7fb09b317640 1 -- 172.21.15.22:0/722694684 shutdown_connections 2024-09-19T08:12:25.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:25.834+0000 7fb09b317640 1 -- 172.21.15.22:0/722694684 wait complete. 2024-09-19T08:12:26.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:25 smithi022 ceph-mon[30963]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:26.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:26.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:26.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:26.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:26.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:26.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:26.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:25 smithi022 ceph-mon[30963]: Deploying daemon mds.d on smithi022 2024-09-19T08:12:26.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:25 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.22:6834/3714062414,v1:172.21.15.22:6835/3714062414] up:boot 2024-09-19T08:12:26.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:25 smithi022 ceph-mon[30963]: fsmap 3 up:standby 2024-09-19T08:12:26.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:25 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T08:12:26.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:25 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/722694684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:26.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:25 smithi157 ceph-mon[35143]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:26.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:26.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:26.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:26.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:26.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:26.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:26.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:25 smithi157 ceph-mon[35143]: Deploying daemon mds.d on smithi022 2024-09-19T08:12:26.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:25 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.22:6834/3714062414,v1:172.21.15.22:6835/3714062414] up:boot 2024-09-19T08:12:26.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:25 smithi157 ceph-mon[35143]: fsmap 3 up:standby 2024-09-19T08:12:26.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:25 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T08:12:26.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:25 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/722694684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:26.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:25 smithi064 ceph-mon[34900]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:26.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:26.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:26.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:26.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:26.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:26.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:26.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:25 smithi064 ceph-mon[34900]: Deploying daemon mds.d on smithi022 2024-09-19T08:12:26.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:25 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.22:6834/3714062414,v1:172.21.15.22:6835/3714062414] up:boot 2024-09-19T08:12:26.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:25 smithi064 ceph-mon[34900]: fsmap 3 up:standby 2024-09-19T08:12:26.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:25 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T08:12:26.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:25 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/722694684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:28.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:27 smithi157 ceph-mon[35143]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:27 smithi022 ceph-mon[30963]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:28.257 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:12:28 smithi022 systemd[1]: Starting Ceph mds.d for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:12:28.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:27 smithi064 ceph-mon[34900]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:28.699 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:12:28 smithi022 podman[70041]: 2024-09-19 08:12:28.322734645 +0000 UTC m=+0.016965737 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:12:28.699 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:12:28 smithi022 podman[70041]: 2024-09-19 08:12:28.45077521 +0000 UTC m=+0.145006300 container create 9aa1f6ec73dc7e57b932f893b6badcad897e254d74ea99a59eb0495755be1c86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T08:12:29.257 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:12:28 smithi022 podman[70041]: 2024-09-19 08:12:28.820653139 +0000 UTC m=+0.514884231 container init 9aa1f6ec73dc7e57b932f893b6badcad897e254d74ea99a59eb0495755be1c86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-d, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:12:29.257 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:12:28 smithi022 podman[70041]: 2024-09-19 08:12:28.823683291 +0000 UTC m=+0.517914381 container start 9aa1f6ec73dc7e57b932f893b6badcad897e254d74ea99a59eb0495755be1c86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-d, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:12:29.257 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:12:28 smithi022 ceph-mds[70061]: -- 172.21.15.22:0/2126511748 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5638000c1860 con 0x56380009e800 2024-09-19T08:12:29.257 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:12:28 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-d[70057]: starting mds.d at 2024-09-19T08:12:29.257 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:12:28 smithi022 bash[70041]: 9aa1f6ec73dc7e57b932f893b6badcad897e254d74ea99a59eb0495755be1c86 2024-09-19T08:12:29.257 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:12:28 smithi022 systemd[1]: Started Ceph mds.d for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:12:29.267 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:12:29.528 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.001+0000 7efd88702640 1 -- 172.21.15.22:0/1422420632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80108300 msgr2=0x7efd8010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.001+0000 7efd88702640 1 --2- 172.21.15.22:0/1422420632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80108300 0x7efd8010ec70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd70009cd0 tx=0x7efd7002a8d0 comp rx=0 tx=0).stop 2024-09-19T08:12:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.001+0000 7efd88702640 1 -- 172.21.15.22:0/1422420632 shutdown_connections 2024-09-19T08:12:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.001+0000 7efd88702640 1 --2- 172.21.15.22:0/1422420632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80108300 0x7efd8010ec70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.001+0000 7efd88702640 1 --2- 172.21.15.22:0/1422420632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd80103780 0x7efd80107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.001+0000 7efd88702640 1 --2- 172.21.15.22:0/1422420632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd80102dd0 0x7efd801031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.001+0000 7efd88702640 1 -- 172.21.15.22:0/1422420632 >> 172.21.15.22:0/1422420632 conn(0x7efd800fc7d0 msgr2=0x7efd800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.001+0000 7efd88702640 1 -- 172.21.15.22:0/1422420632 shutdown_connections 2024-09-19T08:12:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.001+0000 7efd88702640 1 -- 172.21.15.22:0/1422420632 wait complete. 2024-09-19T08:12:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.001+0000 7efd88702640 1 Processor -- start 2024-09-19T08:12:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd88702640 1 -- start start 2024-09-19T08:12:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd88702640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd80102dd0 0x7efd801984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd88702640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd80103780 0x7efd80198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd88702640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80108300 0x7efd8019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd88702640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efd801120d0 con 0x7efd80108300 2024-09-19T08:12:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd88702640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efd80111f50 con 0x7efd80102dd0 2024-09-19T08:12:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd88702640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efd80112250 con 0x7efd80103780 2024-09-19T08:12:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd86c9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80108300 0x7efd8019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd86c9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80108300 0x7efd8019cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49582/0 (socket says 172.21.15.22:49582) 2024-09-19T08:12:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd86c9f640 1 -- 172.21.15.22:0/870999734 learned_addr learned my addr 172.21.15.22:0/870999734 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd85c9d640 1 --2- 172.21.15.22:0/870999734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd80103780 0x7efd80198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd8649e640 1 --2- 172.21.15.22:0/870999734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd80102dd0 0x7efd801984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd86c9f640 1 -- 172.21.15.22:0/870999734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd80103780 msgr2=0x7efd80198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd86c9f640 1 --2- 172.21.15.22:0/870999734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd80103780 0x7efd80198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd86c9f640 1 -- 172.21.15.22:0/870999734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd80102dd0 msgr2=0x7efd801984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd86c9f640 1 --2- 172.21.15.22:0/870999734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd80102dd0 0x7efd801984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd86c9f640 1 -- 172.21.15.22:0/870999734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8019d4a0 con 0x7efd80108300 2024-09-19T08:12:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.002+0000 7efd86c9f640 1 --2- 172.21.15.22:0/870999734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80108300 0x7efd8019cdc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efd70009ca0 tx=0x7efd7002f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.003+0000 7efd7f7fe640 1 -- 172.21.15.22:0/870999734 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd700317e0 con 0x7efd80108300 2024-09-19T08:12:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.003+0000 7efd7f7fe640 1 -- 172.21.15.22:0/870999734 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd70003e50 con 0x7efd80108300 2024-09-19T08:12:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.003+0000 7efd7f7fe640 1 -- 172.21.15.22:0/870999734 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70030590 con 0x7efd80108300 2024-09-19T08:12:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.003+0000 7efd88702640 1 -- 172.21.15.22:0/870999734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8019d730 con 0x7efd80108300 2024-09-19T08:12:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.003+0000 7efd88702640 1 -- 172.21.15.22:0/870999734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd801a5070 con 0x7efd80108300 2024-09-19T08:12:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.004+0000 7efd88702640 1 -- 172.21.15.22:0/870999734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd801031b0 con 0x7efd80108300 2024-09-19T08:12:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.005+0000 7efd7f7fe640 1 -- 172.21.15.22:0/870999734 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7efd70004040 con 0x7efd80108300 2024-09-19T08:12:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.006+0000 7efd7f7fe640 1 --2- 172.21.15.22:0/870999734 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efd680a32c0 0x7efd680a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:30.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.006+0000 7efd8649e640 1 --2- 172.21.15.22:0/870999734 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efd680a32c0 0x7efd680a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:30.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.006+0000 7efd7f7fe640 1 -- 172.21.15.22:0/870999734 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7efd700f03c0 con 0x7efd80108300 2024-09-19T08:12:30.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.006+0000 7efd8649e640 1 --2- 172.21.15.22:0/870999734 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efd680a32c0 0x7efd680a5780 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efd78009b60 tx=0x7efd78005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:30.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.010+0000 7efd7f7fe640 1 -- 172.21.15.22:0/870999734 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efd700bd580 con 0x7efd80108300 2024-09-19T08:12:30.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.219+0000 7efd88702640 1 -- 172.21.15.22:0/870999734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd801a5240 con 0x7efd80108300 2024-09-19T08:12:30.220 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:29 smithi022 ceph-mon[30963]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:30.220 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:30.220 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:30.220 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:30.220 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:30.220 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:30.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:30.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:29 smithi022 ceph-mon[30963]: Deploying daemon mds.e on smithi064 2024-09-19T08:12:30.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:29 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.22:6836/3668291316,v1:172.21.15.22:6837/3668291316] up:boot 2024-09-19T08:12:30.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:29 smithi022 ceph-mon[30963]: fsmap 4 up:standby 2024-09-19T08:12:30.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:29 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T08:12:30.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.220+0000 7efd7f7fe640 1 -- 172.21.15.22:0/870999734 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7efd700bce80 con 0x7efd80108300 2024-09-19T08:12:30.221 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:12:30.222 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":5,"btime":"2024-09-19T08:12:28:990489+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T08:12:30.223 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 5 2024-09-19T08:12:30.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.222+0000 7efd88702640 1 -- 172.21.15.22:0/870999734 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efd680a32c0 msgr2=0x7efd680a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:30.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.222+0000 7efd88702640 1 --2- 172.21.15.22:0/870999734 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efd680a32c0 0x7efd680a5780 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efd78009b60 tx=0x7efd78005f90 comp rx=0 tx=0).stop 2024-09-19T08:12:30.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.222+0000 7efd88702640 1 -- 172.21.15.22:0/870999734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80108300 msgr2=0x7efd8019cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:30.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.222+0000 7efd88702640 1 --2- 172.21.15.22:0/870999734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80108300 0x7efd8019cdc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efd70009ca0 tx=0x7efd7002f3b0 comp rx=0 tx=0).stop 2024-09-19T08:12:30.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.222+0000 7efd88702640 1 -- 172.21.15.22:0/870999734 shutdown_connections 2024-09-19T08:12:30.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.222+0000 7efd88702640 1 --2- 172.21.15.22:0/870999734 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efd680a32c0 0x7efd680a5780 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:30.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.222+0000 7efd88702640 1 --2- 172.21.15.22:0/870999734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd80108300 0x7efd8019cdc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:30.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.222+0000 7efd88702640 1 --2- 172.21.15.22:0/870999734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd80103780 0x7efd80198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:30.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.222+0000 7efd88702640 1 --2- 172.21.15.22:0/870999734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd80102dd0 0x7efd801984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:30.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.222+0000 7efd88702640 1 -- 172.21.15.22:0/870999734 >> 172.21.15.22:0/870999734 conn(0x7efd800fc7d0 msgr2=0x7efd80105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:30.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.222+0000 7efd88702640 1 -- 172.21.15.22:0/870999734 shutdown_connections 2024-09-19T08:12:30.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:30.222+0000 7efd88702640 1 -- 172.21.15.22:0/870999734 wait complete. 2024-09-19T08:12:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:29 smithi157 ceph-mon[35143]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:29 smithi157 ceph-mon[35143]: Deploying daemon mds.e on smithi064 2024-09-19T08:12:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:29 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.22:6836/3668291316,v1:172.21.15.22:6837/3668291316] up:boot 2024-09-19T08:12:30.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:29 smithi157 ceph-mon[35143]: fsmap 4 up:standby 2024-09-19T08:12:30.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:29 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T08:12:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:29 smithi064 ceph-mon[34900]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:29 smithi064 ceph-mon[34900]: Deploying daemon mds.e on smithi064 2024-09-19T08:12:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:29 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.22:6836/3668291316,v1:172.21.15.22:6837/3668291316] up:boot 2024-09-19T08:12:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:29 smithi064 ceph-mon[34900]: fsmap 4 up:standby 2024-09-19T08:12:30.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:29 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T08:12:31.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:30 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/870999734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:31.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:30 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/870999734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:31.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:30 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/870999734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:31.837 INFO:journalctl@ceph.mds.e.smithi064.stdout:Sep 19 08:12:31 smithi064 systemd[1]: Started Ceph mds.e for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:12:32.729 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:32 smithi157 ceph-mon[35143]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:32.729 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:32 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:32.729 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:32 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:32.729 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:32 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:32.729 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:32 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:32.729 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:32 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:32.729 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:32 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:32.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:32 smithi157 ceph-mon[35143]: Deploying daemon mds.f on smithi157 2024-09-19T08:12:32.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:32 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.64:6834/3333087560,v1:172.21.15.64:6835/3333087560] up:boot 2024-09-19T08:12:32.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:32 smithi157 ceph-mon[35143]: fsmap 5 up:standby 2024-09-19T08:12:32.730 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:32 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:12:33.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:32 smithi022 ceph-mon[30963]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:33.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:32 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:33.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:32 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:33.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:32 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:33.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:32 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:33.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:32 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:33.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:32 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:33.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:32 smithi022 ceph-mon[30963]: Deploying daemon mds.f on smithi157 2024-09-19T08:12:33.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:32 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.64:6834/3333087560,v1:172.21.15.64:6835/3333087560] up:boot 2024-09-19T08:12:33.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:32 smithi022 ceph-mon[30963]: fsmap 5 up:standby 2024-09-19T08:12:33.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:32 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:12:33.014 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:12:33.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:32 smithi064 ceph-mon[34900]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:32 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:32 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:32 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:32 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:32 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:32 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:32 smithi064 ceph-mon[34900]: Deploying daemon mds.f on smithi157 2024-09-19T08:12:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:32 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.64:6834/3333087560,v1:172.21.15.64:6835/3333087560] up:boot 2024-09-19T08:12:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:32 smithi064 ceph-mon[34900]: fsmap 5 up:standby 2024-09-19T08:12:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:32 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:12:33.282 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:33.717 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:12:33 smithi157 systemd[1]: Starting Ceph mds.f for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:12:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.891+0000 7fd3304a9640 1 -- 172.21.15.22:0/844199729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd328101de0 msgr2=0x7fd3281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.891+0000 7fd3304a9640 1 --2- 172.21.15.22:0/844199729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd328101de0 0x7fd3281021c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd324009900 tx=0x7fd32402a410 comp rx=0 tx=0).stop 2024-09-19T08:12:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.891+0000 7fd3304a9640 1 -- 172.21.15.22:0/844199729 shutdown_connections 2024-09-19T08:12:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.891+0000 7fd3304a9640 1 --2- 172.21.15.22:0/844199729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd328106f10 0x7fd32810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.891+0000 7fd3304a9640 1 --2- 172.21.15.22:0/844199729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd328102790 0x7fd3281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.891+0000 7fd3304a9640 1 --2- 172.21.15.22:0/844199729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd328101de0 0x7fd3281021c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.891+0000 7fd3304a9640 1 -- 172.21.15.22:0/844199729 >> 172.21.15.22:0/844199729 conn(0x7fd3280fd8c0 msgr2=0x7fd3280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.891+0000 7fd3304a9640 1 -- 172.21.15.22:0/844199729 shutdown_connections 2024-09-19T08:12:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.891+0000 7fd3304a9640 1 -- 172.21.15.22:0/844199729 wait complete. 2024-09-19T08:12:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.892+0000 7fd3304a9640 1 Processor -- start 2024-09-19T08:12:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.892+0000 7fd3304a9640 1 -- start start 2024-09-19T08:12:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.892+0000 7fd3304a9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd328101de0 0x7fd3280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.892+0000 7fd3304a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd328102790 0x7fd328074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.892+0000 7fd3304a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd328106f10 0x7fd328074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.892+0000 7fd3304a9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd328111fb0 con 0x7fd328102790 2024-09-19T08:12:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.892+0000 7fd3304a9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd328111e30 con 0x7fd328101de0 2024-09-19T08:12:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.892+0000 7fd3304a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd328112130 con 0x7fd328106f10 2024-09-19T08:12:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.892+0000 7fd32da44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd328102790 0x7fd328074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.892+0000 7fd32da44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd328102790 0x7fd328074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49586/0 (socket says 172.21.15.22:49586) 2024-09-19T08:12:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.892+0000 7fd32da44640 1 -- 172.21.15.22:0/4163471463 learned_addr learned my addr 172.21.15.22:0/4163471463 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.893+0000 7fd32e245640 1 --2- 172.21.15.22:0/4163471463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd328101de0 0x7fd3280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.893+0000 7fd32da44640 1 -- 172.21.15.22:0/4163471463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd328106f10 msgr2=0x7fd328074f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:12:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.893+0000 7fd32da44640 1 --2- 172.21.15.22:0/4163471463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd328106f10 0x7fd328074f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.893+0000 7fd32da44640 1 -- 172.21.15.22:0/4163471463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd328101de0 msgr2=0x7fd3280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.893+0000 7fd32da44640 1 --2- 172.21.15.22:0/4163471463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd328101de0 0x7fd3280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.893+0000 7fd32da44640 1 -- 172.21.15.22:0/4163471463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3281a69b0 con 0x7fd328102790 2024-09-19T08:12:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.893+0000 7fd32e245640 1 --2- 172.21.15.22:0/4163471463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd328101de0 0x7fd3280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:12:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.893+0000 7fd32da44640 1 --2- 172.21.15.22:0/4163471463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd328102790 0x7fd328074a30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd31800d950 tx=0x7fd31800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:33.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.893+0000 7fd31f7fe640 1 -- 172.21.15.22:0/4163471463 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318017030 con 0x7fd328102790 2024-09-19T08:12:33.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.893+0000 7fd3304a9640 1 -- 172.21.15.22:0/4163471463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3281a6ca0 con 0x7fd328102790 2024-09-19T08:12:33.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.893+0000 7fd31f7fe640 1 -- 172.21.15.22:0/4163471463 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd318004020 con 0x7fd328102790 2024-09-19T08:12:33.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.893+0000 7fd31f7fe640 1 -- 172.21.15.22:0/4163471463 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3180047a0 con 0x7fd328102790 2024-09-19T08:12:33.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.893+0000 7fd3304a9640 1 -- 172.21.15.22:0/4163471463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3281a71d0 con 0x7fd328102790 2024-09-19T08:12:33.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.894+0000 7fd3304a9640 1 -- 172.21.15.22:0/4163471463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3281021c0 con 0x7fd328102790 2024-09-19T08:12:33.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.895+0000 7fd31f7fe640 1 -- 172.21.15.22:0/4163471463 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd3180041d0 con 0x7fd328102790 2024-09-19T08:12:33.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.896+0000 7fd31f7fe640 1 --2- 172.21.15.22:0/4163471463 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd3140a3240 0x7fd3140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:33.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.896+0000 7fd32e245640 1 --2- 172.21.15.22:0/4163471463 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd3140a3240 0x7fd3140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:33.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.896+0000 7fd31f7fe640 1 -- 172.21.15.22:0/4163471463 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fd3180cf580 con 0x7fd328102790 2024-09-19T08:12:33.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.897+0000 7fd32e245640 1 --2- 172.21.15.22:0/4163471463 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd3140a3240 0x7fd3140a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd324005940 tx=0x7fd3240058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:33.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:33.899+0000 7fd31f7fe640 1 -- 172.21.15.22:0/4163471463 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd31809c740 con 0x7fd328102790 2024-09-19T08:12:34.003 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:12:33 smithi157 podman[65051]: 2024-09-19 08:12:33.620113381 +0000 UTC m=+0.016386309 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:12:34.003 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:12:33 smithi157 podman[65051]: 2024-09-19 08:12:33.732920523 +0000 UTC m=+0.129193452 container create b84a29081160548ce7faebf21acda2df756cfaf50fe3245341e6dd93ddacd371 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:12:34.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:34.109+0000 7fd3304a9640 1 -- 172.21.15.22:0/4163471463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd328075a20 con 0x7fd328102790 2024-09-19T08:12:34.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:34.110+0000 7fd31f7fe640 1 -- 172.21.15.22:0/4163471463 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7fd328075a20 con 0x7fd328102790 2024-09-19T08:12:34.110 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:12:34.110 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":6,"btime":"2024-09-19T08:12:31:978383+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-19T08:12:34.110 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 6 2024-09-19T08:12:34.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:34.112+0000 7fd3304a9640 1 -- 172.21.15.22:0/4163471463 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd3140a3240 msgr2=0x7fd3140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:34.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:34.112+0000 7fd3304a9640 1 --2- 172.21.15.22:0/4163471463 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd3140a3240 0x7fd3140a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd324005940 tx=0x7fd3240058b0 comp rx=0 tx=0).stop 2024-09-19T08:12:34.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:34.112+0000 7fd3304a9640 1 -- 172.21.15.22:0/4163471463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd328102790 msgr2=0x7fd328074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:34.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:34.112+0000 7fd3304a9640 1 --2- 172.21.15.22:0/4163471463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd328102790 0x7fd328074a30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd31800d950 tx=0x7fd31800de10 comp rx=0 tx=0).stop 2024-09-19T08:12:34.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:34.112+0000 7fd3304a9640 1 -- 172.21.15.22:0/4163471463 shutdown_connections 2024-09-19T08:12:34.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:34.112+0000 7fd3304a9640 1 --2- 172.21.15.22:0/4163471463 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd3140a3240 0x7fd3140a5700 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:34.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:34.112+0000 7fd3304a9640 1 --2- 172.21.15.22:0/4163471463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd328106f10 0x7fd328074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:34.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:34.112+0000 7fd3304a9640 1 --2- 172.21.15.22:0/4163471463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd328102790 0x7fd328074a30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:34.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:34.112+0000 7fd3304a9640 1 --2- 172.21.15.22:0/4163471463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd328101de0 0x7fd3280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:34.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:34.112+0000 7fd3304a9640 1 -- 172.21.15.22:0/4163471463 >> 172.21.15.22:0/4163471463 conn(0x7fd3280fd8c0 msgr2=0x7fd3280fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:34.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:34.112+0000 7fd3304a9640 1 -- 172.21.15.22:0/4163471463 shutdown_connections 2024-09-19T08:12:34.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:34.112+0000 7fd3304a9640 1 -- 172.21.15.22:0/4163471463 wait complete. 2024-09-19T08:12:34.503 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:12:34 smithi157 podman[65051]: 2024-09-19 08:12:34.119295136 +0000 UTC m=+0.515568064 container init b84a29081160548ce7faebf21acda2df756cfaf50fe3245341e6dd93ddacd371 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T08:12:34.503 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:12:34 smithi157 podman[65051]: 2024-09-19 08:12:34.122430298 +0000 UTC m=+0.518703227 container start b84a29081160548ce7faebf21acda2df756cfaf50fe3245341e6dd93ddacd371 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T08:12:34.503 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:12:34 smithi157 ceph-mds[65071]: -- 172.21.15.157:0/1667659672 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a48d395860 con 0x55a48d373000 2024-09-19T08:12:34.503 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:12:34 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-f[65067]: starting mds.f at 2024-09-19T08:12:34.503 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:12:34 smithi157 bash[65051]: b84a29081160548ce7faebf21acda2df756cfaf50fe3245341e6dd93ddacd371 2024-09-19T08:12:34.503 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:12:34 smithi157 systemd[1]: Started Ceph mds.f for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:12:34.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:34 smithi022 ceph-mon[30963]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:34.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:34 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/4163471463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:34.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:34.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:34.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:34.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:34.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:34.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:34 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:34.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:34 smithi064 ceph-mon[34900]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:34.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:34 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/4163471463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:34.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:34 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:34.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:34 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:34.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:34 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:34.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:34 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:34.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:34 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:34.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:34 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:34.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:34 smithi157 ceph-mon[35143]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:34.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:34 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/4163471463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:34.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:34 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:34.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:34 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:34.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:34 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:34.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:34 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:34.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:34 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:34.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:34 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:35.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:35 smithi157 ceph-mon[35143]: Deploying daemon mds.c on smithi157 2024-09-19T08:12:35.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:35 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.157:6834/4158350679,v1:172.21.15.157:6835/4158350679] up:boot 2024-09-19T08:12:35.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:35 smithi157 ceph-mon[35143]: fsmap 6 up:standby 2024-09-19T08:12:35.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:35 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:12:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:35 smithi022 ceph-mon[30963]: Deploying daemon mds.c on smithi157 2024-09-19T08:12:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:35 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.157:6834/4158350679,v1:172.21.15.157:6835/4158350679] up:boot 2024-09-19T08:12:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:35 smithi022 ceph-mon[30963]: fsmap 6 up:standby 2024-09-19T08:12:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:35 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:12:36.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:35 smithi064 ceph-mon[34900]: Deploying daemon mds.c on smithi157 2024-09-19T08:12:36.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:35 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.157:6834/4158350679,v1:172.21.15.157:6835/4158350679] up:boot 2024-09-19T08:12:36.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:35 smithi064 ceph-mon[34900]: fsmap 6 up:standby 2024-09-19T08:12:36.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:35 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:12:36.426 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:12:36 smithi157 systemd[1]: Starting Ceph mds.c for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:12:36.733 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:36 smithi157 ceph-mon[35143]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:36.733 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:12:36 smithi157 podman[65354]: 2024-09-19 08:12:36.329419815 +0000 UTC m=+0.015938612 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:12:36.734 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:12:36 smithi157 podman[65354]: 2024-09-19 08:12:36.480912315 +0000 UTC m=+0.167431112 container create 876264be95c99539ae64db5df7424ca3ae45d5f3efe6c94f207a088c6a6148db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T08:12:36.734 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:12:36 smithi157 podman[65354]: 2024-09-19 08:12:36.734312261 +0000 UTC m=+0.420831057 container init 876264be95c99539ae64db5df7424ca3ae45d5f3efe6c94f207a088c6a6148db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:12:36.904 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:12:36.931 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:36 smithi022 ceph-mon[30963]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:37.003 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:12:36 smithi157 podman[65354]: 2024-09-19 08:12:36.737220262 +0000 UTC m=+0.423739066 container start 876264be95c99539ae64db5df7424ca3ae45d5f3efe6c94f207a088c6a6148db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:12:37.003 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:12:36 smithi157 ceph-mds[65374]: -- 172.21.15.157:0/3866185962 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f7c3895860 con 0x55f7c3873400 2024-09-19T08:12:37.003 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:12:36 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-c[65370]: starting mds.c at 2024-09-19T08:12:37.003 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:12:36 smithi157 bash[65354]: 876264be95c99539ae64db5df7424ca3ae45d5f3efe6c94f207a088c6a6148db 2024-09-19T08:12:37.003 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:12:36 smithi157 systemd[1]: Started Ceph mds.c for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:12:37.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:36 smithi064 ceph-mon[34900]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:37.169 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:37.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.651+0000 7f29a09cb640 1 -- 172.21.15.22:0/3312453408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f299c106f20 msgr2=0x7f299c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:37.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.651+0000 7f29a09cb640 1 --2- 172.21.15.22:0/3312453408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f299c106f20 0x7f299c10eaa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2994009900 tx=0x7f299402a410 comp rx=0 tx=0).stop 2024-09-19T08:12:37.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.651+0000 7f29a09cb640 1 -- 172.21.15.22:0/3312453408 shutdown_connections 2024-09-19T08:12:37.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.651+0000 7f29a09cb640 1 --2- 172.21.15.22:0/3312453408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f299c106f20 0x7f299c10eaa0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:37.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.651+0000 7f29a09cb640 1 --2- 172.21.15.22:0/3312453408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c1027a0 0x7f299c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:37.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.651+0000 7f29a09cb640 1 --2- 172.21.15.22:0/3312453408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c101df0 0x7f299c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:37.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.651+0000 7f29a09cb640 1 -- 172.21.15.22:0/3312453408 >> 172.21.15.22:0/3312453408 conn(0x7f299c0fd8f0 msgr2=0x7f299c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:37.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.651+0000 7f29a09cb640 1 -- 172.21.15.22:0/3312453408 shutdown_connections 2024-09-19T08:12:37.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.651+0000 7f29a09cb640 1 -- 172.21.15.22:0/3312453408 wait complete. 2024-09-19T08:12:37.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.652+0000 7f29a09cb640 1 Processor -- start 2024-09-19T08:12:37.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.652+0000 7f29a09cb640 1 -- start start 2024-09-19T08:12:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.652+0000 7f29a09cb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c101df0 0x7f299c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.652+0000 7f29a09cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f299c1027a0 0x7f299c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.652+0000 7f29a09cb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c106f20 0x7f299c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.652+0000 7f29a09cb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f299c111f80 con 0x7f299c1027a0 2024-09-19T08:12:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.652+0000 7f29a09cb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f299c111e00 con 0x7f299c106f20 2024-09-19T08:12:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.652+0000 7f29a09cb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f299c112100 con 0x7f299c101df0 2024-09-19T08:12:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.652+0000 7f2999d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f299c1027a0 0x7f299c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.652+0000 7f2999d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f299c1027a0 0x7f299c1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53082/0 (socket says 172.21.15.22:53082) 2024-09-19T08:12:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.652+0000 7f2999d9b640 1 -- 172.21.15.22:0/111288269 learned_addr learned my addr 172.21.15.22:0/111288269 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.652+0000 7f299a59c640 1 --2- 172.21.15.22:0/111288269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c101df0 0x7f299c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.652+0000 7f299ad9d640 1 --2- 172.21.15.22:0/111288269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c106f20 0x7f299c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.652+0000 7f2999d9b640 1 -- 172.21.15.22:0/111288269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c101df0 msgr2=0x7f299c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.653+0000 7f2999d9b640 1 --2- 172.21.15.22:0/111288269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c101df0 0x7f299c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.653+0000 7f2999d9b640 1 -- 172.21.15.22:0/111288269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c106f20 msgr2=0x7f299c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.653+0000 7f2999d9b640 1 --2- 172.21.15.22:0/111288269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c106f20 0x7f299c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.653+0000 7f2999d9b640 1 -- 172.21.15.22:0/111288269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c19d430 con 0x7f299c1027a0 2024-09-19T08:12:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.653+0000 7f2999d9b640 1 --2- 172.21.15.22:0/111288269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f299c1027a0 0x7f299c1989c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f299000db80 tx=0x7f299000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.653+0000 7f298b7fe640 1 -- 172.21.15.22:0/111288269 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990017070 con 0x7f299c1027a0 2024-09-19T08:12:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.653+0000 7f298b7fe640 1 -- 172.21.15.22:0/111288269 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2990003b70 con 0x7f299c1027a0 2024-09-19T08:12:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.653+0000 7f29a09cb640 1 -- 172.21.15.22:0/111288269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299c19d720 con 0x7f299c1027a0 2024-09-19T08:12:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.653+0000 7f298b7fe640 1 -- 172.21.15.22:0/111288269 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29900106c0 con 0x7f299c1027a0 2024-09-19T08:12:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.653+0000 7f29a09cb640 1 -- 172.21.15.22:0/111288269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299c1a5000 con 0x7f299c1027a0 2024-09-19T08:12:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.654+0000 7f29a09cb640 1 -- 172.21.15.22:0/111288269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f299c1021d0 con 0x7f299c1027a0 2024-09-19T08:12:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.655+0000 7f298b7fe640 1 -- 172.21.15.22:0/111288269 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2990004020 con 0x7f299c1027a0 2024-09-19T08:12:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.656+0000 7f298b7fe640 1 --2- 172.21.15.22:0/111288269 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f29840a3240 0x7f29840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.656+0000 7f299a59c640 1 --2- 172.21.15.22:0/111288269 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f29840a3240 0x7f29840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.656+0000 7f298b7fe640 1 -- 172.21.15.22:0/111288269 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f29900cf210 con 0x7f299c1027a0 2024-09-19T08:12:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.656+0000 7f299a59c640 1 --2- 172.21.15.22:0/111288269 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f29840a3240 0x7f29840a5700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f298c009bd0 tx=0x7f298c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.660+0000 7f298b7fe640 1 -- 172.21.15.22:0/111288269 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2990020070 con 0x7f299c1027a0 2024-09-19T08:12:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.871+0000 7f29a09cb640 1 -- 172.21.15.22:0/111288269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f299c10a190 con 0x7f299c1027a0 2024-09-19T08:12:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.871+0000 7f298b7fe640 1 -- 172.21.15.22:0/111288269 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7f299009c330 con 0x7f299c1027a0 2024-09-19T08:12:37.873 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:12:37.873 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"btime":"2024-09-19T08:12:37:606120+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T08:12:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-09-19T08:12:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.873+0000 7f29a09cb640 1 -- 172.21.15.22:0/111288269 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f29840a3240 msgr2=0x7f29840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.873+0000 7f29a09cb640 1 --2- 172.21.15.22:0/111288269 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f29840a3240 0x7f29840a5700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f298c009bd0 tx=0x7f298c005f90 comp rx=0 tx=0).stop 2024-09-19T08:12:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.873+0000 7f29a09cb640 1 -- 172.21.15.22:0/111288269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f299c1027a0 msgr2=0x7f299c1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.874+0000 7f29a09cb640 1 --2- 172.21.15.22:0/111288269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f299c1027a0 0x7f299c1989c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f299000db80 tx=0x7f299000b630 comp rx=0 tx=0).stop 2024-09-19T08:12:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.874+0000 7f29a09cb640 1 -- 172.21.15.22:0/111288269 shutdown_connections 2024-09-19T08:12:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.874+0000 7f29a09cb640 1 --2- 172.21.15.22:0/111288269 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f29840a3240 0x7f29840a5700 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.874+0000 7f29a09cb640 1 --2- 172.21.15.22:0/111288269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f299c106f20 0x7f299c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.874+0000 7f29a09cb640 1 --2- 172.21.15.22:0/111288269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f299c1027a0 0x7f299c1989c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.874+0000 7f29a09cb640 1 --2- 172.21.15.22:0/111288269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c101df0 0x7f299c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.874+0000 7f29a09cb640 1 -- 172.21.15.22:0/111288269 >> 172.21.15.22:0/111288269 conn(0x7f299c0fd8f0 msgr2=0x7f299c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.874+0000 7f29a09cb640 1 -- 172.21.15.22:0/111288269 shutdown_connections 2024-09-19T08:12:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:37.874+0000 7f29a09cb640 1 -- 172.21.15.22:0/111288269 wait complete. 2024-09-19T08:12:38.189 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:37 smithi022 ceph-mon[30963]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:38.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:37 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:38.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:37 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:38.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:37 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:38.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:37 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:38.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:37 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:38.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:37 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:38.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:37 smithi022 ceph-mon[30963]: Deploying daemon mds.h on smithi064 2024-09-19T08:12:38.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:37 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] up:boot 2024-09-19T08:12:38.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:37 smithi022 ceph-mon[30963]: fsmap 7 up:standby 2024-09-19T08:12:38.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:37 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:12:38.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:37 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/111288269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:38.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:37 smithi157 ceph-mon[35143]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:38.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:37 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:38.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:37 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:38.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:37 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:38.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:37 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:38.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:37 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:38.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:37 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:38.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:37 smithi157 ceph-mon[35143]: Deploying daemon mds.h on smithi064 2024-09-19T08:12:38.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:37 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] up:boot 2024-09-19T08:12:38.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:37 smithi157 ceph-mon[35143]: fsmap 7 up:standby 2024-09-19T08:12:38.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:37 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:12:38.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:37 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/111288269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:38.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:37 smithi064 ceph-mon[34900]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:38.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:37 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:38.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:37 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:38.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:37 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:38.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:37 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:38.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:37 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:38.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:37 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:38.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:37 smithi064 ceph-mon[34900]: Deploying daemon mds.h on smithi064 2024-09-19T08:12:38.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:37 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] up:boot 2024-09-19T08:12:38.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:37 smithi064 ceph-mon[34900]: fsmap 7 up:standby 2024-09-19T08:12:38.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:37 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:12:38.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:37 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/111288269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:39.052 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:12:38 smithi064 systemd[1]: Starting Ceph mds.h for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:12:39.336 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:12:39 smithi064 podman[64199]: 2024-09-19 08:12:38.955016745 +0000 UTC m=+0.015976501 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:12:39.336 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:12:39 smithi064 podman[64199]: 2024-09-19 08:12:39.084037044 +0000 UTC m=+0.144996826 container create af25c49e9d7496012ac5681bb0d01f1290d3309c823e2829cf7760803cfc5917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T08:12:39.336 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:12:39 smithi064 podman[64199]: 2024-09-19 08:12:39.337092595 +0000 UTC m=+0.398052336 container init af25c49e9d7496012ac5681bb0d01f1290d3309c823e2829cf7760803cfc5917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T08:12:39.587 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:12:39 smithi064 podman[64199]: 2024-09-19 08:12:39.339871498 +0000 UTC m=+0.400831239 container start af25c49e9d7496012ac5681bb0d01f1290d3309c823e2829cf7760803cfc5917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-h, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T08:12:39.587 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:12:39 smithi064 ceph-mds[64218]: -- 172.21.15.64:0/3432913652 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5567f0be3860 con 0x5567f0bc1000 2024-09-19T08:12:39.587 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:12:39 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-h[64214]: starting mds.h at 2024-09-19T08:12:39.587 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:12:39 smithi064 bash[64199]: af25c49e9d7496012ac5681bb0d01f1290d3309c823e2829cf7760803cfc5917 2024-09-19T08:12:39.587 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:12:39 smithi064 systemd[1]: Started Ceph mds.h for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:12:40.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:12:40.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:40 smithi157 ceph-mon[35143]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:40.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:40 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:40.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:40 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:40.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:40 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:40.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:40 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:40.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:40 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:40.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:40 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:40.755 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:40 smithi157 ceph-mon[35143]: Deploying daemon mds.i on smithi157 2024-09-19T08:12:40.755 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:40 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.64:6836/2142121431,v1:172.21.15.64:6837/2142121431] up:boot 2024-09-19T08:12:40.755 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:40 smithi157 ceph-mon[35143]: fsmap 8 up:standby 2024-09-19T08:12:40.755 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:40 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:12:40.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:40 smithi022 ceph-mon[30963]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:40.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:40 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:40.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:40 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:40.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:40 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:40.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:40 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:40.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:40 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:40.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:40 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:40.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:40 smithi022 ceph-mon[30963]: Deploying daemon mds.i on smithi157 2024-09-19T08:12:40.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:40 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.64:6836/2142121431,v1:172.21.15.64:6837/2142121431] up:boot 2024-09-19T08:12:40.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:40 smithi022 ceph-mon[30963]: fsmap 8 up:standby 2024-09-19T08:12:40.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:40 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:12:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:40 smithi064 ceph-mon[34900]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:40 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:40 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:40 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:40 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:40 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:40 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:40 smithi064 ceph-mon[34900]: Deploying daemon mds.i on smithi157 2024-09-19T08:12:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:40 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.64:6836/2142121431,v1:172.21.15.64:6837/2142121431] up:boot 2024-09-19T08:12:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:40 smithi064 ceph-mon[34900]: fsmap 8 up:standby 2024-09-19T08:12:40.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:40 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:12:40.843 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:41.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.317+0000 7f413a5be640 1 -- 172.21.15.22:0/3264458944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4134100a80 msgr2=0x7f4134100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:41.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.317+0000 7f413a5be640 1 --2- 172.21.15.22:0/3264458944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4134100a80 0x7f4134100ee0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4128009930 tx=0x7f412802a420 comp rx=0 tx=0).stop 2024-09-19T08:12:41.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.317+0000 7f413a5be640 1 -- 172.21.15.22:0/3264458944 shutdown_connections 2024-09-19T08:12:41.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.317+0000 7f413a5be640 1 --2- 172.21.15.22:0/3264458944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4134101420 0x7f413410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:41.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.317+0000 7f413a5be640 1 --2- 172.21.15.22:0/3264458944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4134100a80 0x7f4134100ee0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:41.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.317+0000 7f413a5be640 1 --2- 172.21.15.22:0/3264458944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4134106a80 0x7f4134106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:41.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.317+0000 7f413a5be640 1 -- 172.21.15.22:0/3264458944 >> 172.21.15.22:0/3264458944 conn(0x7f41340fc7a0 msgr2=0x7f41340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:41.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.317+0000 7f413a5be640 1 -- 172.21.15.22:0/3264458944 shutdown_connections 2024-09-19T08:12:41.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.317+0000 7f413a5be640 1 -- 172.21.15.22:0/3264458944 wait complete. 2024-09-19T08:12:41.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.317+0000 7f413a5be640 1 Processor -- start 2024-09-19T08:12:41.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.317+0000 7f413a5be640 1 -- start start 2024-09-19T08:12:41.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f413a5be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4134100a80 0x7f413419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:41.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f413a5be640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4134101420 0x7f413419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:41.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f413a5be640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4134106a80 0x7f413419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:41.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f413a5be640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4134110e80 con 0x7f4134106a80 2024-09-19T08:12:41.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f413a5be640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4134110d00 con 0x7f4134100a80 2024-09-19T08:12:41.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f413a5be640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4134111000 con 0x7f4134101420 2024-09-19T08:12:41.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f4138b5b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4134106a80 0x7f413419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:41.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f4138b5b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4134106a80 0x7f413419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53100/0 (socket says 172.21.15.22:53100) 2024-09-19T08:12:41.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f4138b5b640 1 -- 172.21.15.22:0/2472090079 learned_addr learned my addr 172.21.15.22:0/2472090079 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f41337fe640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4134101420 0x7f413419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f4133fff640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4134100a80 0x7f413419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f4138b5b640 1 -- 172.21.15.22:0/2472090079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4134101420 msgr2=0x7f413419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f4138b5b640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4134101420 0x7f413419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f4138b5b640 1 -- 172.21.15.22:0/2472090079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4134100a80 msgr2=0x7f413419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f4138b5b640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4134100a80 0x7f413419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f4138b5b640 1 -- 172.21.15.22:0/2472090079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413419f630 con 0x7f4134106a80 2024-09-19T08:12:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f4133fff640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4134100a80 0x7f413419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:12:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f4138b5b640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4134106a80 0x7f413419ef50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f412c009fd0 tx=0x7f412c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.318+0000 7f41317fa640 1 -- 172.21.15.22:0/2472090079 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c017070 con 0x7f4134106a80 2024-09-19T08:12:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.319+0000 7f41317fa640 1 -- 172.21.15.22:0/2472090079 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412c003e50 con 0x7f4134106a80 2024-09-19T08:12:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.319+0000 7f413a5be640 1 -- 172.21.15.22:0/2472090079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413419f920 con 0x7f4134106a80 2024-09-19T08:12:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.319+0000 7f41317fa640 1 -- 172.21.15.22:0/2472090079 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c0106c0 con 0x7f4134106a80 2024-09-19T08:12:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.319+0000 7f413a5be640 1 -- 172.21.15.22:0/2472090079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41341a7200 con 0x7f4134106a80 2024-09-19T08:12:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.320+0000 7f413a5be640 1 -- 172.21.15.22:0/2472090079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4134100ee0 con 0x7f4134106a80 2024-09-19T08:12:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.321+0000 7f41317fa640 1 -- 172.21.15.22:0/2472090079 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f412c003970 con 0x7f4134106a80 2024-09-19T08:12:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.322+0000 7f41317fa640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f411c0a3240 0x7f411c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.322+0000 7f4133fff640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f411c0a3240 0x7f411c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.322+0000 7f41317fa640 1 -- 172.21.15.22:0/2472090079 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f412c0cf2d0 con 0x7f4134106a80 2024-09-19T08:12:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.322+0000 7f4133fff640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f411c0a3240 0x7f411c0a5700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4124009ad0 tx=0x7f4124005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:41.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.326+0000 7f41317fa640 1 -- 172.21.15.22:0/2472090079 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f412c09c3f0 con 0x7f4134106a80 2024-09-19T08:12:41.503 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:12:41 smithi157 systemd[1]: Starting Ceph mds.i for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:12:41.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.537+0000 7f413a5be640 1 -- 172.21.15.22:0/2472090079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41341a73d0 con 0x7f4134106a80 2024-09-19T08:12:41.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.538+0000 7f41317fa640 1 -- 172.21.15.22:0/2472090079 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7213 (secure 0 0 0) 0x7f412c09bcf0 con 0x7f4134106a80 2024-09-19T08:12:41.539 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:12:41.539 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"btime":"2024-09-19T08:12:39:886339+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T08:12:41.539 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2024-09-19T08:12:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.540+0000 7f413a5be640 1 -- 172.21.15.22:0/2472090079 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f411c0a3240 msgr2=0x7f411c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.540+0000 7f413a5be640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f411c0a3240 0x7f411c0a5700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4124009ad0 tx=0x7f4124005f90 comp rx=0 tx=0).stop 2024-09-19T08:12:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.540+0000 7f413a5be640 1 -- 172.21.15.22:0/2472090079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4134106a80 msgr2=0x7f413419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.540+0000 7f413a5be640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4134106a80 0x7f413419ef50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f412c009fd0 tx=0x7f412c00b260 comp rx=0 tx=0).stop 2024-09-19T08:12:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.540+0000 7f413a5be640 1 -- 172.21.15.22:0/2472090079 shutdown_connections 2024-09-19T08:12:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.540+0000 7f413a5be640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f411c0a3240 0x7f411c0a5700 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.540+0000 7f413a5be640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4134106a80 0x7f413419ef50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:41.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.540+0000 7f413a5be640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4134101420 0x7f413419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:41.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.540+0000 7f413a5be640 1 --2- 172.21.15.22:0/2472090079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4134100a80 0x7f413419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:41.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.540+0000 7f413a5be640 1 -- 172.21.15.22:0/2472090079 >> 172.21.15.22:0/2472090079 conn(0x7f41340fc7a0 msgr2=0x7f413406a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:41.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.540+0000 7f413a5be640 1 -- 172.21.15.22:0/2472090079 shutdown_connections 2024-09-19T08:12:41.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:41.540+0000 7f413a5be640 1 -- 172.21.15.22:0/2472090079 wait complete. 2024-09-19T08:12:41.857 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:12:41 smithi157 podman[65655]: 2024-09-19 08:12:41.491724382 +0000 UTC m=+0.016291053 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:12:41.857 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:12:41 smithi157 podman[65655]: 2024-09-19 08:12:41.603171172 +0000 UTC m=+0.127737868 container create b42f9d647701cf597db6eae403ecee54bc895ab1181855fc6237a00c2c232242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:12:41.857 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:12:41 smithi157 podman[65655]: 2024-09-19 08:12:41.830789987 +0000 UTC m=+0.355356641 container init b42f9d647701cf597db6eae403ecee54bc895ab1181855fc6237a00c2c232242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:12:41.858 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:12:41 smithi157 podman[65655]: 2024-09-19 08:12:41.833677039 +0000 UTC m=+0.358243693 container start b42f9d647701cf597db6eae403ecee54bc895ab1181855fc6237a00c2c232242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T08:12:42.253 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:12:41 smithi157 ceph-mds[65674]: -- 172.21.15.157:0/1949476704 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561b33a3b860 con 0x561b33a19400 2024-09-19T08:12:42.253 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:12:41 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i[65670]: starting mds.i at 2024-09-19T08:12:42.253 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:12:41 smithi157 bash[65655]: b42f9d647701cf597db6eae403ecee54bc895ab1181855fc6237a00c2c232242 2024-09-19T08:12:42.253 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:12:41 smithi157 systemd[1]: Started Ceph mds.i for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:12:42.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:42 smithi157 ceph-mon[35143]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:42.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:42 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2472090079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:42.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:42 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:42.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:42 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:42.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:42 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:42.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:42 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:42.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:42 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:42.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:42 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:42.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:42 smithi022 ceph-mon[30963]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:42.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:42 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2472090079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:42 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:42 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:42 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:42 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:42 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:42 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:42.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:42 smithi064 ceph-mon[34900]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:42.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:42 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2472090079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:42.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:42 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:42.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:42 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:42.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:42 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:42.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:42 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:42.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:42 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:42.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:42 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:43.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:43 smithi157 ceph-mon[35143]: Deploying daemon mds.g on smithi022 2024-09-19T08:12:43.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:43 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] up:boot 2024-09-19T08:12:43.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:43 smithi157 ceph-mon[35143]: fsmap 9 up:standby 2024-09-19T08:12:43.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:43 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:12:43.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:43 smithi022 ceph-mon[30963]: Deploying daemon mds.g on smithi022 2024-09-19T08:12:43.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:43 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] up:boot 2024-09-19T08:12:43.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:43 smithi022 ceph-mon[30963]: fsmap 9 up:standby 2024-09-19T08:12:43.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:43 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:12:43.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:43 smithi064 ceph-mon[34900]: Deploying daemon mds.g on smithi022 2024-09-19T08:12:43.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:43 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] up:boot 2024-09-19T08:12:43.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:43 smithi064 ceph-mon[34900]: fsmap 9 up:standby 2024-09-19T08:12:43.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:43 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:12:44.186 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:12:43 smithi022 systemd[1]: Starting Ceph mds.g for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:12:44.240 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:12:44.449 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:12:44 smithi022 podman[70805]: 2024-09-19 08:12:44.089323331 +0000 UTC m=+0.016454136 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:12:44.449 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:12:44 smithi022 podman[70805]: 2024-09-19 08:12:44.204522833 +0000 UTC m=+0.131653623 container create 5a88e318c867672fffd68803441a182964e50470cba4dc2fe873485c7e0065a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T08:12:44.449 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:12:44 smithi022 podman[70805]: 2024-09-19 08:12:44.449331877 +0000 UTC m=+0.376462667 container init 5a88e318c867672fffd68803441a182964e50470cba4dc2fe873485c7e0065a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T08:12:44.639 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:44.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:44 smithi157 ceph-mon[35143]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:44.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:44 smithi022 ceph-mon[30963]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:44.756 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:12:44 smithi022 podman[70805]: 2024-09-19 08:12:44.452801974 +0000 UTC m=+0.379932774 container start 5a88e318c867672fffd68803441a182964e50470cba4dc2fe873485c7e0065a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:12:44.757 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:12:44 smithi022 ceph-mds[70854]: -- 172.21.15.22:0/3820370858 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f201867860 con 0x55f201845400 2024-09-19T08:12:44.757 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:12:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-g[70838]: starting mds.g at 2024-09-19T08:12:44.757 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:12:44 smithi022 bash[70805]: 5a88e318c867672fffd68803441a182964e50470cba4dc2fe873485c7e0065a2 2024-09-19T08:12:44.757 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:12:44 smithi022 systemd[1]: Started Ceph mds.g for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:12:44.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:44 smithi064 ceph-mon[34900]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:45.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.137+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/2649121503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc98102d30 msgr2=0x7fdc98103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:45.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.137+0000 7fdc9ea6d640 1 --2- 172.21.15.22:0/2649121503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc98102d30 0x7fdc98103110 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009930 tx=0x7fdc8802a440 comp rx=0 tx=0).stop 2024-09-19T08:12:45.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.137+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/2649121503 shutdown_connections 2024-09-19T08:12:45.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.137+0000 7fdc9ea6d640 1 --2- 172.21.15.22:0/2649121503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc981082a0 0x7fdc9810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:45.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.137+0000 7fdc9ea6d640 1 --2- 172.21.15.22:0/2649121503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc981036e0 0x7fdc98107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:45.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.137+0000 7fdc9ea6d640 1 --2- 172.21.15.22:0/2649121503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc98102d30 0x7fdc98103110 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:45.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.137+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/2649121503 >> 172.21.15.22:0/2649121503 conn(0x7fdc980fc7d0 msgr2=0x7fdc980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:45.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.137+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/2649121503 shutdown_connections 2024-09-19T08:12:45.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.137+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/2649121503 wait complete. 2024-09-19T08:12:45.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.138+0000 7fdc9ea6d640 1 Processor -- start 2024-09-19T08:12:45.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.138+0000 7fdc9ea6d640 1 -- start start 2024-09-19T08:12:45.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.138+0000 7fdc9ea6d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc98102d30 0x7fdc98198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:45.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.138+0000 7fdc9ea6d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc981036e0 0x7fdc98198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:45.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.138+0000 7fdc9ea6d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc981082a0 0x7fdc9819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:45.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.138+0000 7fdc9ea6d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98112160 con 0x7fdc98102d30 2024-09-19T08:12:45.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.138+0000 7fdc9ea6d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98111fe0 con 0x7fdc981082a0 2024-09-19T08:12:45.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.138+0000 7fdc9ea6d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc981122e0 con 0x7fdc981036e0 2024-09-19T08:12:45.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.138+0000 7fdc9c809640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc98102d30 0x7fdc98198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:45.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.138+0000 7fdc9c809640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc98102d30 0x7fdc98198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53140/0 (socket says 172.21.15.22:53140) 2024-09-19T08:12:45.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.138+0000 7fdc9c809640 1 -- 172.21.15.22:0/3261745116 learned_addr learned my addr 172.21.15.22:0/3261745116 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:45.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.138+0000 7fdc97fff640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc981036e0 0x7fdc98198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:45.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.139+0000 7fdc9d00a640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc981082a0 0x7fdc9819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:45.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.139+0000 7fdc9c809640 1 -- 172.21.15.22:0/3261745116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc981036e0 msgr2=0x7fdc98198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.139+0000 7fdc9c809640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc981036e0 0x7fdc98198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.139+0000 7fdc9c809640 1 -- 172.21.15.22:0/3261745116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc981082a0 msgr2=0x7fdc9819cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.139+0000 7fdc9c809640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc981082a0 0x7fdc9819cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.139+0000 7fdc9c809640 1 -- 172.21.15.22:0/3261745116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9819d6d0 con 0x7fdc98102d30 2024-09-19T08:12:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.139+0000 7fdc9d00a640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc981082a0 0x7fdc9819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:12:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.139+0000 7fdc9c809640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc98102d30 0x7fdc98198720 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009a50 tx=0x7fdc88004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.139+0000 7fdc95ffb640 1 -- 172.21.15.22:0/3261745116 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8803f030 con 0x7fdc98102d30 2024-09-19T08:12:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.139+0000 7fdc95ffb640 1 -- 172.21.15.22:0/3261745116 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc88003ce0 con 0x7fdc98102d30 2024-09-19T08:12:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.139+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/3261745116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9819d960 con 0x7fdc98102d30 2024-09-19T08:12:45.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.139+0000 7fdc95ffb640 1 -- 172.21.15.22:0/3261745116 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88035070 con 0x7fdc98102d30 2024-09-19T08:12:45.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.139+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/3261745116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc981a52a0 con 0x7fdc98102d30 2024-09-19T08:12:45.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.140+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/3261745116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc98103190 con 0x7fdc98102d30 2024-09-19T08:12:45.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.141+0000 7fdc95ffb640 1 -- 172.21.15.22:0/3261745116 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fdc88042030 con 0x7fdc98102d30 2024-09-19T08:12:45.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.142+0000 7fdc95ffb640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdc800a32c0 0x7fdc800a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:45.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.142+0000 7fdc95ffb640 1 -- 172.21.15.22:0/3261745116 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fdc880eff50 con 0x7fdc98102d30 2024-09-19T08:12:45.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.142+0000 7fdc97fff640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdc800a32c0 0x7fdc800a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:45.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.143+0000 7fdc97fff640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdc800a32c0 0x7fdc800a5780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdc980feee0 tx=0x7fdc8c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:45.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.145+0000 7fdc95ffb640 1 -- 172.21.15.22:0/3261745116 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdc880bd070 con 0x7fdc98102d30 2024-09-19T08:12:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.361+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/3261745116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc9810a2b0 con 0x7fdc98102d30 2024-09-19T08:12:45.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.361+0000 7fdc95ffb640 1 -- 172.21.15.22:0/3261745116 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8869 (secure 0 0 0) 0x7fdc880bc970 con 0x7fdc98102d30 2024-09-19T08:12:45.363 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:12:45.363 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":11,"btime":"2024-09-19T08:12:44:493751+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T08:12:45.364 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 11 2024-09-19T08:12:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.364+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/3261745116 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdc800a32c0 msgr2=0x7fdc800a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.364+0000 7fdc9ea6d640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdc800a32c0 0x7fdc800a5780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdc980feee0 tx=0x7fdc8c008040 comp rx=0 tx=0).stop 2024-09-19T08:12:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.364+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/3261745116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc98102d30 msgr2=0x7fdc98198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.364+0000 7fdc9ea6d640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc98102d30 0x7fdc98198720 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009a50 tx=0x7fdc88004270 comp rx=0 tx=0).stop 2024-09-19T08:12:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.364+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/3261745116 shutdown_connections 2024-09-19T08:12:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.364+0000 7fdc9ea6d640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdc800a32c0 0x7fdc800a5780 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.364+0000 7fdc9ea6d640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc981082a0 0x7fdc9819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.364+0000 7fdc9ea6d640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc981036e0 0x7fdc98198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.364+0000 7fdc9ea6d640 1 --2- 172.21.15.22:0/3261745116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc98102d30 0x7fdc98198720 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.364+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/3261745116 >> 172.21.15.22:0/3261745116 conn(0x7fdc980fc7d0 msgr2=0x7fdc98105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.364+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/3261745116 shutdown_connections 2024-09-19T08:12:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:45.364+0000 7fdc9ea6d640 1 -- 172.21.15.22:0/3261745116 wait complete. 2024-09-19T08:12:45.717 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:45 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.22:6838/3762277518,v1:172.21.15.22:6839/3762277518] up:boot 2024-09-19T08:12:45.717 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:45 smithi022 ceph-mon[30963]: fsmap 10 up:standby 2024-09-19T08:12:45.717 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:12:45.717 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:45.717 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:45.717 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:45.718 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:45.718 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:45.718 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:45 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:45.718 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:45 smithi022 ceph-mon[30963]: Deploying daemon mds.a on smithi022 2024-09-19T08:12:45.718 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:45 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3261745116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:45.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:45 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.22:6838/3762277518,v1:172.21.15.22:6839/3762277518] up:boot 2024-09-19T08:12:45.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:45 smithi157 ceph-mon[35143]: fsmap 10 up:standby 2024-09-19T08:12:45.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:12:45.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:45.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:45.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:45.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:45.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:45.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:45 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:45.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:45 smithi157 ceph-mon[35143]: Deploying daemon mds.a on smithi022 2024-09-19T08:12:45.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:45 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3261745116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:45 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.22:6838/3762277518,v1:172.21.15.22:6839/3762277518] up:boot 2024-09-19T08:12:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:45 smithi064 ceph-mon[34900]: fsmap 10 up:standby 2024-09-19T08:12:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:12:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:45 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:45 smithi064 ceph-mon[34900]: Deploying daemon mds.a on smithi022 2024-09-19T08:12:45.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:45 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3261745116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:46.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:46 smithi022 ceph-mon[30963]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:47.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:46 smithi157 ceph-mon[35143]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:47.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:46 smithi064 ceph-mon[34900]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:47.507 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:12:47 smithi022 systemd[1]: Starting Ceph mds.a for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:12:47.957 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:12:47 smithi022 podman[71219]: 2024-09-19 08:12:47.537046058 +0000 UTC m=+0.016155206 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:12:47.957 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:12:47 smithi022 podman[71219]: 2024-09-19 08:12:47.712908718 +0000 UTC m=+0.192017895 container create df7590e175c906ac6c31adbc9d3793b9a20f79eb739553950a36b7869e0d7271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T08:12:47.957 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:12:47 smithi022 podman[71219]: 2024-09-19 08:12:47.957417435 +0000 UTC m=+0.436526580 container init df7590e175c906ac6c31adbc9d3793b9a20f79eb739553950a36b7869e0d7271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T08:12:48.257 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:12:47 smithi022 podman[71219]: 2024-09-19 08:12:47.960527171 +0000 UTC m=+0.439636323 container start df7590e175c906ac6c31adbc9d3793b9a20f79eb739553950a36b7869e0d7271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T08:12:48.257 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:12:47 smithi022 ceph-mds[71238]: -- 172.21.15.22:0/1378725215 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648a93e1860 con 0x5648a93bf400 2024-09-19T08:12:48.257 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:12:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-a[71234]: starting mds.a at 2024-09-19T08:12:48.257 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:12:48 smithi022 bash[71219]: df7590e175c906ac6c31adbc9d3793b9a20f79eb739553950a36b7869e0d7271 2024-09-19T08:12:48.257 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:12:48 smithi022 systemd[1]: Started Ceph mds.a for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:12:48.520 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:12:48.792 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:48.806 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:48 smithi064 ceph-mon[34900]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:48.806 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:48.806 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:48.806 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:48.806 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:48.807 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:48.807 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:48 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:48.913 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:48 smithi022 ceph-mon[30963]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:48.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:48.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:48.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:48.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:48.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:48.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:48 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:49.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:48 smithi157 ceph-mon[35143]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:49.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:49.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:49.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:49.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:12:49.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:12:49.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:48 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.293+0000 7f572787a640 1 -- 172.21.15.22:0/3762661094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5720102d30 msgr2=0x7f5720103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.293+0000 7f572787a640 1 --2- 172.21.15.22:0/3762661094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5720102d30 0x7f5720103110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f571c009930 tx=0x7f571c02a420 comp rx=0 tx=0).stop 2024-09-19T08:12:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.293+0000 7f572787a640 1 -- 172.21.15.22:0/3762661094 shutdown_connections 2024-09-19T08:12:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.293+0000 7f572787a640 1 --2- 172.21.15.22:0/3762661094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57201082a0 0x7f572010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.293+0000 7f572787a640 1 --2- 172.21.15.22:0/3762661094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57201036e0 0x7f5720107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:49.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.293+0000 7f572787a640 1 --2- 172.21.15.22:0/3762661094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5720102d30 0x7f5720103110 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:49.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.293+0000 7f572787a640 1 -- 172.21.15.22:0/3762661094 >> 172.21.15.22:0/3762661094 conn(0x7f57200fc7d0 msgr2=0x7f57200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:49.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.293+0000 7f572787a640 1 -- 172.21.15.22:0/3762661094 shutdown_connections 2024-09-19T08:12:49.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.293+0000 7f572787a640 1 -- 172.21.15.22:0/3762661094 wait complete. 2024-09-19T08:12:49.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f572787a640 1 Processor -- start 2024-09-19T08:12:49.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f572787a640 1 -- start start 2024-09-19T08:12:49.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f572787a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5720102d30 0x7f57201985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:49.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f572787a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57201036e0 0x7f5720198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:49.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f572787a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57201082a0 0x7f572019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:49.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f572787a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57201120a0 con 0x7f57201036e0 2024-09-19T08:12:49.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f572787a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5720111f20 con 0x7f57201082a0 2024-09-19T08:12:49.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f572787a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5720112220 con 0x7f5720102d30 2024-09-19T08:12:49.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f5724e15640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57201036e0 0x7f5720198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:49.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f5724e15640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57201036e0 0x7f5720198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52792/0 (socket says 172.21.15.22:52792) 2024-09-19T08:12:49.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f5724e15640 1 -- 172.21.15.22:0/4194784935 learned_addr learned my addr 172.21.15.22:0/4194784935 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:49.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f5725616640 1 --2- 172.21.15.22:0/4194784935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5720102d30 0x7f57201985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:49.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f5725e17640 1 --2- 172.21.15.22:0/4194784935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57201082a0 0x7f572019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:49.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f5724e15640 1 -- 172.21.15.22:0/4194784935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5720102d30 msgr2=0x7f57201985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:49.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f5724e15640 1 --2- 172.21.15.22:0/4194784935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5720102d30 0x7f57201985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:49.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f5724e15640 1 -- 172.21.15.22:0/4194784935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57201082a0 msgr2=0x7f572019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:49.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f5724e15640 1 --2- 172.21.15.22:0/4194784935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57201082a0 0x7f572019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:49.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.294+0000 7f5724e15640 1 -- 172.21.15.22:0/4194784935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572019d550 con 0x7f57201036e0 2024-09-19T08:12:49.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.295+0000 7f5724e15640 1 --2- 172.21.15.22:0/4194784935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57201036e0 0x7f5720198ae0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f571000d740 tx=0x7f571000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:49.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.295+0000 7f57167fc640 1 -- 172.21.15.22:0/4194784935 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710017070 con 0x7f57201036e0 2024-09-19T08:12:49.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.295+0000 7f57167fc640 1 -- 172.21.15.22:0/4194784935 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f571000de50 con 0x7f57201036e0 2024-09-19T08:12:49.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.295+0000 7f572787a640 1 -- 172.21.15.22:0/4194784935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572019d840 con 0x7f57201036e0 2024-09-19T08:12:49.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.295+0000 7f57167fc640 1 -- 172.21.15.22:0/4194784935 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571000f6b0 con 0x7f57201036e0 2024-09-19T08:12:49.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.295+0000 7f572787a640 1 -- 172.21.15.22:0/4194784935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57201a5120 con 0x7f57201036e0 2024-09-19T08:12:49.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.296+0000 7f572787a640 1 -- 172.21.15.22:0/4194784935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f572006a850 con 0x7f57201036e0 2024-09-19T08:12:49.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.297+0000 7f57167fc640 1 -- 172.21.15.22:0/4194784935 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f571001a070 con 0x7f57201036e0 2024-09-19T08:12:49.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.298+0000 7f57167fc640 1 --2- 172.21.15.22:0/4194784935 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f570c0a32c0 0x7f570c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:49.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.298+0000 7f5725616640 1 --2- 172.21.15.22:0/4194784935 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f570c0a32c0 0x7f570c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:49.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.298+0000 7f57167fc640 1 -- 172.21.15.22:0/4194784935 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f57100cedd0 con 0x7f57201036e0 2024-09-19T08:12:49.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.298+0000 7f5725616640 1 --2- 172.21.15.22:0/4194784935 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f570c0a32c0 0x7f570c0a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f571c009900 tx=0x7f571c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:49.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.301+0000 7f57167fc640 1 -- 172.21.15.22:0/4194784935 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f571009b920 con 0x7f57201036e0 2024-09-19T08:12:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.515+0000 7f572787a640 1 -- 172.21.15.22:0/4194784935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f572010a2b0 con 0x7f57201036e0 2024-09-19T08:12:49.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.515+0000 7f57167fc640 1 -- 172.21.15.22:0/4194784935 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7f571009fc20 con 0x7f57201036e0 2024-09-19T08:12:49.518 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:12:49.518 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":12,"btime":"2024-09-19T08:12:48:631242+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T08:12:49.518 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 12 2024-09-19T08:12:49.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.519+0000 7f572787a640 1 -- 172.21.15.22:0/4194784935 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f570c0a32c0 msgr2=0x7f570c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:49.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.519+0000 7f572787a640 1 --2- 172.21.15.22:0/4194784935 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f570c0a32c0 0x7f570c0a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f571c009900 tx=0x7f571c0058d0 comp rx=0 tx=0).stop 2024-09-19T08:12:49.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.519+0000 7f572787a640 1 -- 172.21.15.22:0/4194784935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57201036e0 msgr2=0x7f5720198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:49.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.519+0000 7f572787a640 1 --2- 172.21.15.22:0/4194784935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57201036e0 0x7f5720198ae0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f571000d740 tx=0x7f571000dc00 comp rx=0 tx=0).stop 2024-09-19T08:12:49.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.519+0000 7f572787a640 1 -- 172.21.15.22:0/4194784935 shutdown_connections 2024-09-19T08:12:49.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.519+0000 7f572787a640 1 --2- 172.21.15.22:0/4194784935 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f570c0a32c0 0x7f570c0a5780 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:49.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.519+0000 7f572787a640 1 --2- 172.21.15.22:0/4194784935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57201082a0 0x7f572019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:49.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.519+0000 7f572787a640 1 --2- 172.21.15.22:0/4194784935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57201036e0 0x7f5720198ae0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:49.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.519+0000 7f572787a640 1 --2- 172.21.15.22:0/4194784935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5720102d30 0x7f57201985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:49.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.519+0000 7f572787a640 1 -- 172.21.15.22:0/4194784935 >> 172.21.15.22:0/4194784935 conn(0x7f57200fc7d0 msgr2=0x7f5720105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:49.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.519+0000 7f572787a640 1 -- 172.21.15.22:0/4194784935 shutdown_connections 2024-09-19T08:12:49.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:49.519+0000 7f572787a640 1 -- 172.21.15.22:0/4194784935 wait complete. 2024-09-19T08:12:49.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:49 smithi064 ceph-mon[34900]: Deploying daemon mds.b on smithi064 2024-09-19T08:12:49.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:49 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.22:6840/1594373985,v1:172.21.15.22:6841/1594373985] up:boot 2024-09-19T08:12:49.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:49 smithi064 ceph-mon[34900]: fsmap 11 up:standby 2024-09-19T08:12:49.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:49 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:12:49.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:49 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/4194784935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:49.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:49 smithi022 ceph-mon[30963]: Deploying daemon mds.b on smithi064 2024-09-19T08:12:49.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:49 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.22:6840/1594373985,v1:172.21.15.22:6841/1594373985] up:boot 2024-09-19T08:12:49.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:49 smithi022 ceph-mon[30963]: fsmap 11 up:standby 2024-09-19T08:12:49.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:49 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:12:49.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:49 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/4194784935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:50.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:49 smithi157 ceph-mon[35143]: Deploying daemon mds.b on smithi064 2024-09-19T08:12:50.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:49 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.22:6840/1594373985,v1:172.21.15.22:6841/1594373985] up:boot 2024-09-19T08:12:50.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:49 smithi157 ceph-mon[35143]: fsmap 11 up:standby 2024-09-19T08:12:50.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:49 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:12:50.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:49 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/4194784935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:50.337 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:12:50 smithi064 systemd[1]: Starting Ceph mds.b for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:12:50.617 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:12:50 smithi064 podman[64512]: 2024-09-19 08:12:50.248653662 +0000 UTC m=+0.016273271 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:12:50.617 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:12:50 smithi064 podman[64512]: 2024-09-19 08:12:50.381206933 +0000 UTC m=+0.148826534 container create 702394fa37745be1348fd73692c211f02c5effa17dd0471cd98288e2132c06fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T08:12:51.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:50 smithi157 ceph-mon[35143]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:51.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:50 smithi022 ceph-mon[30963]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:51.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:50 smithi064 ceph-mon[34900]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:51.087 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:12:50 smithi064 podman[64512]: 2024-09-19 08:12:50.617561087 +0000 UTC m=+0.385180703 container init 702394fa37745be1348fd73692c211f02c5effa17dd0471cd98288e2132c06fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T08:12:51.087 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:12:50 smithi064 podman[64512]: 2024-09-19 08:12:50.620668494 +0000 UTC m=+0.388288096 container start 702394fa37745be1348fd73692c211f02c5effa17dd0471cd98288e2132c06fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:12:51.087 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:12:50 smithi064 ceph-mds[64531]: -- 172.21.15.64:0/1119217197 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa429f1c20 con 0x55fa429ce800 2024-09-19T08:12:51.087 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:12:50 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-b[64527]: starting mds.b at 2024-09-19T08:12:51.087 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:12:50 smithi064 bash[64512]: 702394fa37745be1348fd73692c211f02c5effa17dd0471cd98288e2132c06fb 2024-09-19T08:12:51.087 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:12:50 smithi064 systemd[1]: Started Ceph mds.b for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:12:52.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:51 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:51 smithi157 ceph-mon[35143]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:52.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:51 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:51 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:51 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:51 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:51 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:52.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:51 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] up:boot 2024-09-19T08:12:52.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:51 smithi157 ceph-mon[35143]: fsmap 12 up:standby 2024-09-19T08:12:52.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:51 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:12:52.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:51 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:51 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:51 smithi022 ceph-mon[30963]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:52.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:51 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:51 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:51 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:51 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:51 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:52.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:51 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] up:boot 2024-09-19T08:12:52.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:51 smithi022 ceph-mon[30963]: fsmap 12 up:standby 2024-09-19T08:12:52.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:51 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:12:52.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:51 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:51 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:51 smithi064 ceph-mon[34900]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:52.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:51 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:51 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:51 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:51 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:51 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:52.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:51 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] up:boot 2024-09-19T08:12:52.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:51 smithi064 ceph-mon[34900]: fsmap 12 up:standby 2024-09-19T08:12:52.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:51 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:12:52.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:51 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:52.216 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:12:52.518 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:53.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.010+0000 7f69ade8c640 1 -- 172.21.15.22:0/641071239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a8102570 msgr2=0x7f69a81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:53.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.010+0000 7f69ade8c640 1 --2- 172.21.15.22:0/641071239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a8102570 0x7f69a81029b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6990009900 tx=0x7f699002a480 comp rx=0 tx=0).stop 2024-09-19T08:12:53.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.010+0000 7f69ade8c640 1 -- 172.21.15.22:0/641071239 shutdown_connections 2024-09-19T08:12:53.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.010+0000 7f69ade8c640 1 --2- 172.21.15.22:0/641071239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69a8102ef0 0x7f69a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.010+0000 7f69ade8c640 1 --2- 172.21.15.22:0/641071239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a8102570 0x7f69a81029b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.010+0000 7f69ade8c640 1 --2- 172.21.15.22:0/641071239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69a8100e20 0x7f69a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.010+0000 7f69ade8c640 1 -- 172.21.15.22:0/641071239 >> 172.21.15.22:0/641071239 conn(0x7f69a80fc7d0 msgr2=0x7f69a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.010+0000 7f69ade8c640 1 -- 172.21.15.22:0/641071239 shutdown_connections 2024-09-19T08:12:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.010+0000 7f69ade8c640 1 -- 172.21.15.22:0/641071239 wait complete. 2024-09-19T08:12:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.011+0000 7f69ade8c640 1 Processor -- start 2024-09-19T08:12:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.011+0000 7f69ade8c640 1 -- start start 2024-09-19T08:12:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f69ade8c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69a8100e20 0x7f69a80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f69ade8c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a8102570 0x7f69a80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f69ade8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69a8102ef0 0x7f69a80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f69ade8c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8110f80 con 0x7f69a8102570 2024-09-19T08:12:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f69ade8c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8110e00 con 0x7f69a8102ef0 2024-09-19T08:12:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f69ade8c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8111100 con 0x7f69a8100e20 2024-09-19T08:12:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f699effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a8102570 0x7f69a80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f699effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a8102570 0x7f69a80fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52822/0 (socket says 172.21.15.22:52822) 2024-09-19T08:12:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f699effd640 1 -- 172.21.15.22:0/1204592771 learned_addr learned my addr 172.21.15.22:0/1204592771 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f69a77fe640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69a8100e20 0x7f69a80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f69a7fff640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69a8102ef0 0x7f69a80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f699effd640 1 -- 172.21.15.22:0/1204592771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69a8100e20 msgr2=0x7f69a80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f699effd640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69a8100e20 0x7f69a80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f699effd640 1 -- 172.21.15.22:0/1204592771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69a8102ef0 msgr2=0x7f69a80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f699effd640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69a8102ef0 0x7f69a80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f699effd640 1 -- 172.21.15.22:0/1204592771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a81a69c0 con 0x7f69a8102570 2024-09-19T08:12:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f69a7fff640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69a8102ef0 0x7f69a80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:12:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f699effd640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a8102570 0x7f69a80fd1c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6990006010 tx=0x7f6990003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.012+0000 7f69a57fa640 1 -- 172.21.15.22:0/1204592771 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699003f070 con 0x7f69a8102570 2024-09-19T08:12:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.013+0000 7f69a57fa640 1 -- 172.21.15.22:0/1204592771 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6990031c30 con 0x7f69a8102570 2024-09-19T08:12:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.013+0000 7f69ade8c640 1 -- 172.21.15.22:0/1204592771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a81a6c50 con 0x7f69a8102570 2024-09-19T08:12:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.013+0000 7f69a57fa640 1 -- 172.21.15.22:0/1204592771 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990035030 con 0x7f69a8102570 2024-09-19T08:12:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.013+0000 7f69ade8c640 1 -- 172.21.15.22:0/1204592771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a81a7130 con 0x7f69a8102570 2024-09-19T08:12:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.014+0000 7f69ade8c640 1 -- 172.21.15.22:0/1204592771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a8101200 con 0x7f69a8102570 2024-09-19T08:12:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.015+0000 7f69a57fa640 1 -- 172.21.15.22:0/1204592771 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6990042070 con 0x7f69a8102570 2024-09-19T08:12:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.016+0000 7f69a57fa640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f69940a3240 0x7f69940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.016+0000 7f69a77fe640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f69940a3240 0x7f69940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:53.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.016+0000 7f69a57fa640 1 -- 172.21.15.22:0/1204592771 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f69900eff50 con 0x7f69a8102570 2024-09-19T08:12:53.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.016+0000 7f69a77fe640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f69940a3240 0x7f69940a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6998009bd0 tx=0x7f6998005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:53.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.019+0000 7f69a57fa640 1 -- 172.21.15.22:0/1204592771 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f69900bcaa0 con 0x7f69a8102570 2024-09-19T08:12:53.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.247+0000 7f69ade8c640 1 -- 172.21.15.22:0/1204592771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69a810a0a0 con 0x7f69a8102570 2024-09-19T08:12:53.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.248+0000 7f69a57fa640 1 -- 172.21.15.22:0/1204592771 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10518 (secure 0 0 0) 0x7f69900c0da0 con 0x7f69a8102570 2024-09-19T08:12:53.248 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:12:53.248 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":13,"btime":"2024-09-19T08:12:51:642208+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T08:12:53.248 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 13 2024-09-19T08:12:53.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.250+0000 7f69ade8c640 1 -- 172.21.15.22:0/1204592771 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f69940a3240 msgr2=0x7f69940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:53.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.250+0000 7f69ade8c640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f69940a3240 0x7f69940a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6998009bd0 tx=0x7f6998005f90 comp rx=0 tx=0).stop 2024-09-19T08:12:53.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.250+0000 7f69ade8c640 1 -- 172.21.15.22:0/1204592771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a8102570 msgr2=0x7f69a80fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:53.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.250+0000 7f69ade8c640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a8102570 0x7f69a80fd1c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6990006010 tx=0x7f6990003c10 comp rx=0 tx=0).stop 2024-09-19T08:12:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.251+0000 7f69ade8c640 1 -- 172.21.15.22:0/1204592771 shutdown_connections 2024-09-19T08:12:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.251+0000 7f69ade8c640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f69940a3240 0x7f69940a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.251+0000 7f69ade8c640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69a8102ef0 0x7f69a80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.251+0000 7f69ade8c640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69a8102570 0x7f69a80fd1c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.251+0000 7f69ade8c640 1 --2- 172.21.15.22:0/1204592771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69a8100e20 0x7f69a80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.251+0000 7f69ade8c640 1 -- 172.21.15.22:0/1204592771 >> 172.21.15.22:0/1204592771 conn(0x7f69a80fc7d0 msgr2=0x7f69a806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.251+0000 7f69ade8c640 1 -- 172.21.15.22:0/1204592771 shutdown_connections 2024-09-19T08:12:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:53.251+0000 7f69ade8c640 1 -- 172.21.15.22:0/1204592771 wait complete. 2024-09-19T08:12:53.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:53.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:53.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:53 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:53.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:53 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1204592771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:53.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:53.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:53.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:53 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:53.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:53 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1204592771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:53.613 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.613 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.613 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.613 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.613 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.613 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.613 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:53.613 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:53.613 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:53.613 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:53 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:12:53.614 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:53 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1204592771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:12:53.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T08:12:54.563 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:54.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:54 smithi064 ceph-mon[34900]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:54.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:54 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:54.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:54 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:54.685 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:54 smithi022 ceph-mon[30963]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:54.685 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:54 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:54.685 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:54 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:54.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:54 smithi157 ceph-mon[35143]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:54.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:54 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:54.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:54 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:55.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.127+0000 7f9b5e274640 1 -- 172.21.15.22:0/712865925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b58102ef0 msgr2=0x7f9b5810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:55.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.127+0000 7f9b5e274640 1 --2- 172.21.15.22:0/712865925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b58102ef0 0x7f9b5810da40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009ca0 tx=0x7f9b5002a950 comp rx=0 tx=0).stop 2024-09-19T08:12:55.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.127+0000 7f9b5e274640 1 -- 172.21.15.22:0/712865925 shutdown_connections 2024-09-19T08:12:55.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.127+0000 7f9b5e274640 1 --2- 172.21.15.22:0/712865925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b58102ef0 0x7f9b5810da40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:55.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.127+0000 7f9b5e274640 1 --2- 172.21.15.22:0/712865925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b58102570 0x7f9b581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:55.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.127+0000 7f9b5e274640 1 --2- 172.21.15.22:0/712865925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b58100e20 0x7f9b58101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:55.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.127+0000 7f9b5e274640 1 -- 172.21.15.22:0/712865925 >> 172.21.15.22:0/712865925 conn(0x7f9b580fc7d0 msgr2=0x7f9b580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:55.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.127+0000 7f9b5e274640 1 -- 172.21.15.22:0/712865925 shutdown_connections 2024-09-19T08:12:55.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.127+0000 7f9b5e274640 1 -- 172.21.15.22:0/712865925 wait complete. 2024-09-19T08:12:55.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b5e274640 1 Processor -- start 2024-09-19T08:12:55.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b5e274640 1 -- start start 2024-09-19T08:12:55.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b5e274640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b58100e20 0x7f9b580fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:55.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b5e274640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b58102570 0x7f9b580fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:55.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b5e274640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b58102ef0 0x7f9b580fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:55.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b5e274640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9b58110fa0 con 0x7f9b58102570 2024-09-19T08:12:55.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b5e274640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b58110e20 con 0x7f9b58100e20 2024-09-19T08:12:55.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b5e274640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b58111120 con 0x7f9b58102ef0 2024-09-19T08:12:55.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b4ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b58102570 0x7f9b580fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:55.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b4ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b58102570 0x7f9b580fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52842/0 (socket says 172.21.15.22:52842) 2024-09-19T08:12:55.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b4ffff640 1 -- 172.21.15.22:0/351025065 learned_addr learned my addr 172.21.15.22:0/351025065 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:55.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b57fff640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b58100e20 0x7f9b580fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:55.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b5c811640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b58102ef0 0x7f9b580fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:55.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b4ffff640 1 -- 172.21.15.22:0/351025065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b58102ef0 msgr2=0x7f9b580fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:55.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b4ffff640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b58102ef0 0x7f9b580fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:55.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b4ffff640 1 -- 172.21.15.22:0/351025065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b58100e20 msgr2=0x7f9b580fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:55.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b4ffff640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b58100e20 0x7f9b580fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:55.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b4ffff640 1 -- 172.21.15.22:0/351025065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b581a6910 con 0x7f9b58102570 2024-09-19T08:12:55.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.128+0000 7f9b57fff640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b58100e20 0x7f9b580fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:12:55.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.129+0000 7f9b4ffff640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b58102570 0x7f9b580fd1c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000c920 tx=0x7f9b4000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:55.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.129+0000 7f9b55ffb640 1 -- 172.21.15.22:0/351025065 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40007c70 con 0x7f9b58102570 2024-09-19T08:12:55.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.129+0000 7f9b55ffb640 1 -- 172.21.15.22:0/351025065 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b40007e10 con 0x7f9b58102570 2024-09-19T08:12:55.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.129+0000 7f9b55ffb640 1 -- 172.21.15.22:0/351025065 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40018890 con 0x7f9b58102570 2024-09-19T08:12:55.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.129+0000 7f9b5e274640 1 -- 172.21.15.22:0/351025065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b581a6c00 con 0x7f9b58102570 2024-09-19T08:12:55.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.129+0000 7f9b5e274640 1 -- 172.21.15.22:0/351025065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b581a7190 con 0x7f9b58102570 2024-09-19T08:12:55.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.130+0000 7f9b5e274640 1 -- 172.21.15.22:0/351025065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b58101200 con 0x7f9b58102570 2024-09-19T08:12:55.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.131+0000 7f9b55ffb640 1 -- 172.21.15.22:0/351025065 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9b40018a70 con 0x7f9b58102570 2024-09-19T08:12:55.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.132+0000 7f9b55ffb640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f9b440a32c0 0x7f9b440a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:55.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.132+0000 7f9b57fff640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f9b440a32c0 0x7f9b440a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:55.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.132+0000 7f9b55ffb640 1 -- 172.21.15.22:0/351025065 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f9b400cf5d0 con 0x7f9b58102570 2024-09-19T08:12:55.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.132+0000 7f9b57fff640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f9b440a32c0 0x7f9b440a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009930 tx=0x7f9b48008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:55.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.136+0000 7f9b55ffb640 1 -- 172.21.15.22:0/351025065 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9b40014030 con 0x7f9b58102570 2024-09-19T08:12:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.331+0000 7f9b5e274640 1 -- 172.21.15.22:0/351025065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9b5802d7c0 con 0x7f9b58102570 2024-09-19T08:12:55.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.333+0000 7f9b55ffb640 1 -- 172.21.15.22:0/351025065 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7f9b4009c790 con 0x7f9b58102570 2024-09-19T08:12:55.334 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:12:55.334 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-09-19T08:12:55.334 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:12:55.334 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-09-19T08:12:55.334 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-09-19T08:12:55.334 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:12:55.334 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-09-19T08:12:55.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.337+0000 7f9b5e274640 1 -- 172.21.15.22:0/351025065 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f9b440a32c0 msgr2=0x7f9b440a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:55.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.337+0000 7f9b5e274640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f9b440a32c0 0x7f9b440a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009930 tx=0x7f9b48008040 comp rx=0 tx=0).stop 2024-09-19T08:12:55.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.337+0000 7f9b5e274640 1 -- 172.21.15.22:0/351025065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b58102570 msgr2=0x7f9b580fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:55.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.337+0000 7f9b5e274640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b58102570 0x7f9b580fd1c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000c920 tx=0x7f9b4000cde0 comp rx=0 tx=0).stop 2024-09-19T08:12:55.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.337+0000 7f9b5e274640 1 -- 172.21.15.22:0/351025065 shutdown_connections 2024-09-19T08:12:55.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.337+0000 7f9b5e274640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f9b440a32c0 0x7f9b440a5780 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:55.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.337+0000 7f9b5e274640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b58102ef0 0x7f9b580fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:55.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.337+0000 7f9b5e274640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b58102570 0x7f9b580fd1c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:55.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.337+0000 7f9b5e274640 1 --2- 172.21.15.22:0/351025065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b58100e20 0x7f9b580fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:55.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.337+0000 7f9b5e274640 1 -- 172.21.15.22:0/351025065 >> 172.21.15.22:0/351025065 conn(0x7f9b580fc7d0 msgr2=0x7f9b5806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:55.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.337+0000 7f9b5e274640 1 -- 172.21.15.22:0/351025065 shutdown_connections 2024-09-19T08:12:55.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:55.337+0000 7f9b5e274640 1 -- 172.21.15.22:0/351025065 wait complete. 2024-09-19T08:12:55.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:55 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:55.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:55 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:55.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:55 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:55.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:55 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:55.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:55 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:55.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:55 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:56.005 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T08:12:56.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T08:12:56.445 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:56.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:56 smithi022 ceph-mon[30963]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:56.567 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:56 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/351025065' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T08:12:56.567 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:56 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:56.567 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:56 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:56.567 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:56 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:56.567 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:56 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:56.567 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:56 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:56.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:56 smithi064 ceph-mon[34900]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:56.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:56 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/351025065' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T08:12:56.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:56 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:56.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:56 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:56.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:56 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:56.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:56 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:56.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:56 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:56.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:56 smithi157 ceph-mon[35143]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:56.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:56 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/351025065' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T08:12:56.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:56 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:56.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:56 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:56.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:56 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:12:56.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:56 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:12:56.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:56 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:12:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.985+0000 7f207e89f640 1 -- 172.21.15.22:0/547031216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2078101de0 msgr2=0x7f20781021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.985+0000 7f207e89f640 1 --2- 172.21.15.22:0/547031216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2078101de0 0x7f20781021c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2068009900 tx=0x7f206802a410 comp rx=0 tx=0).stop 2024-09-19T08:12:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.985+0000 7f207e89f640 1 -- 172.21.15.22:0/547031216 shutdown_connections 2024-09-19T08:12:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.985+0000 7f207e89f640 1 --2- 172.21.15.22:0/547031216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2078106f10 0x7f207810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.985+0000 7f207e89f640 1 --2- 172.21.15.22:0/547031216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2078102790 0x7f20781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.985+0000 7f207e89f640 1 --2- 172.21.15.22:0/547031216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2078101de0 0x7f20781021c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.985+0000 7f207e89f640 1 -- 172.21.15.22:0/547031216 >> 172.21.15.22:0/547031216 conn(0x7f20780fd8c0 msgr2=0x7f20780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.985+0000 7f207e89f640 1 -- 172.21.15.22:0/547031216 shutdown_connections 2024-09-19T08:12:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.985+0000 7f207e89f640 1 -- 172.21.15.22:0/547031216 wait complete. 2024-09-19T08:12:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.986+0000 7f207e89f640 1 Processor -- start 2024-09-19T08:12:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.986+0000 7f207e89f640 1 -- start start 2024-09-19T08:12:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.986+0000 7f207e89f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2078101de0 0x7f2078198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.986+0000 7f207e89f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2078102790 0x7f2078198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.986+0000 7f207e89f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2078106f10 0x7f207819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.986+0000 7f207e89f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2078111f70 con 0x7f2078102790 2024-09-19T08:12:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.986+0000 7f207e89f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2078111df0 con 0x7f2078106f10 2024-09-19T08:12:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.986+0000 7f207e89f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20781120f0 con 0x7f2078101de0 2024-09-19T08:12:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.986+0000 7f20777fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2078102790 0x7f2078198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.986+0000 7f20777fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2078102790 0x7f2078198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52864/0 (socket says 172.21.15.22:52864) 2024-09-19T08:12:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.986+0000 7f20777fe640 1 -- 172.21.15.22:0/2722576120 learned_addr learned my addr 172.21.15.22:0/2722576120 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.986+0000 7f2077fff640 1 --2- 172.21.15.22:0/2722576120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2078101de0 0x7f2078198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.986+0000 7f207ce3c640 1 --2- 172.21.15.22:0/2722576120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2078106f10 0x7f207819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.987+0000 7f20777fe640 1 -- 172.21.15.22:0/2722576120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2078101de0 msgr2=0x7f2078198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.987+0000 7f20777fe640 1 --2- 172.21.15.22:0/2722576120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2078101de0 0x7f2078198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.987+0000 7f20777fe640 1 -- 172.21.15.22:0/2722576120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2078106f10 msgr2=0x7f207819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.987+0000 7f20777fe640 1 --2- 172.21.15.22:0/2722576120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2078106f10 0x7f207819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.987+0000 7f20777fe640 1 -- 172.21.15.22:0/2722576120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207819d3e0 con 0x7f2078102790 2024-09-19T08:12:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.987+0000 7f20777fe640 1 --2- 172.21.15.22:0/2722576120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2078102790 0x7f2078198970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f206c00d950 tx=0x7f206c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.987+0000 7f20757fa640 1 -- 172.21.15.22:0/2722576120 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c017070 con 0x7f2078102790 2024-09-19T08:12:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.987+0000 7f20757fa640 1 -- 172.21.15.22:0/2722576120 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206c009c30 con 0x7f2078102790 2024-09-19T08:12:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.987+0000 7f207e89f640 1 -- 172.21.15.22:0/2722576120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207819d6d0 con 0x7f2078102790 2024-09-19T08:12:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.987+0000 7f20757fa640 1 -- 172.21.15.22:0/2722576120 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c00f6b0 con 0x7f2078102790 2024-09-19T08:12:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.987+0000 7f207e89f640 1 -- 172.21.15.22:0/2722576120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20781a4fb0 con 0x7f2078102790 2024-09-19T08:12:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.988+0000 7f207e89f640 1 -- 172.21.15.22:0/2722576120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20781021c0 con 0x7f2078102790 2024-09-19T08:12:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.989+0000 7f20757fa640 1 -- 172.21.15.22:0/2722576120 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f206c0109d0 con 0x7f2078102790 2024-09-19T08:12:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.990+0000 7f20757fa640 1 --2- 172.21.15.22:0/2722576120 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f20600a3240 0x7f20600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.990+0000 7f2077fff640 1 --2- 172.21.15.22:0/2722576120 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f20600a3240 0x7f20600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.990+0000 7f20757fa640 1 -- 172.21.15.22:0/2722576120 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f206c0cf0f0 con 0x7f2078102790 2024-09-19T08:12:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.990+0000 7f2077fff640 1 --2- 172.21.15.22:0/2722576120 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f20600a3240 0x7f20600a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2068009230 tx=0x7f20680058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:56.994+0000 7f20757fa640 1 -- 172.21.15.22:0/2722576120 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f206c009de0 con 0x7f2078102790 2024-09-19T08:12:57.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:57.174+0000 7f207e89f640 1 -- 172.21.15.22:0/2722576120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f207810a0f0 con 0x7f2078102790 2024-09-19T08:12:57.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:57.659+0000 7f20757fa640 1 -- 172.21.15.22:0/2722576120 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7f206c09bbc0 con 0x7f2078102790 2024-09-19T08:12:57.659 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-09-19T08:12:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:57.662+0000 7f207e89f640 1 -- 172.21.15.22:0/2722576120 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f20600a3240 msgr2=0x7f20600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:57.662+0000 7f207e89f640 1 --2- 172.21.15.22:0/2722576120 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f20600a3240 0x7f20600a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2068009230 tx=0x7f20680058d0 comp rx=0 tx=0).stop 2024-09-19T08:12:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:57.662+0000 7f207e89f640 1 -- 172.21.15.22:0/2722576120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2078102790 msgr2=0x7f2078198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:57.662+0000 7f207e89f640 1 --2- 172.21.15.22:0/2722576120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2078102790 0x7f2078198970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f206c00d950 tx=0x7f206c00de10 comp rx=0 tx=0).stop 2024-09-19T08:12:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:57.662+0000 7f207e89f640 1 -- 172.21.15.22:0/2722576120 shutdown_connections 2024-09-19T08:12:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:57.662+0000 7f207e89f640 1 --2- 172.21.15.22:0/2722576120 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f20600a3240 0x7f20600a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:57.662+0000 7f207e89f640 1 --2- 172.21.15.22:0/2722576120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2078106f10 0x7f207819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:57.662+0000 7f207e89f640 1 --2- 172.21.15.22:0/2722576120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2078102790 0x7f2078198970 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:57.662+0000 7f207e89f640 1 --2- 172.21.15.22:0/2722576120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2078101de0 0x7f2078198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:57.662+0000 7f207e89f640 1 -- 172.21.15.22:0/2722576120 >> 172.21.15.22:0/2722576120 conn(0x7f20780fd8c0 msgr2=0x7f20780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:57.662+0000 7f207e89f640 1 -- 172.21.15.22:0/2722576120 shutdown_connections 2024-09-19T08:12:57.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:57.662+0000 7f207e89f640 1 -- 172.21.15.22:0/2722576120 wait complete. 2024-09-19T08:12:57.957 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:57 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2722576120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T08:12:58.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:57 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2722576120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T08:12:58.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:57 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2722576120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T08:12:58.393 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T08:12:58.671 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:12:58.955 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:58 smithi022 ceph-mon[30963]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:58.955 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:58 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2722576120' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T08:12:58.955 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:58 smithi022 ceph-mon[30963]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T08:12:58.955 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:58 smithi022 ceph-mon[30963]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T08:12:59.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:58 smithi157 ceph-mon[35143]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:59.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:58 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2722576120' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T08:12:59.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:58 smithi157 ceph-mon[35143]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T08:12:59.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:58 smithi157 ceph-mon[35143]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T08:12:59.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:58 smithi064 ceph-mon[34900]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:12:59.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:58 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2722576120' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T08:12:59.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:58 smithi064 ceph-mon[34900]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T08:12:59.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:58 smithi064 ceph-mon[34900]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T08:12:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.199+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3779124268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3b4106f10 msgr2=0x7fe3b410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.199+0000 7fe3bbd5c640 1 --2- 172.21.15.22:0/3779124268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3b4106f10 0x7fe3b410ea90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009ca0 tx=0x7fe3ac02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:12:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.199+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3779124268 shutdown_connections 2024-09-19T08:12:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.199+0000 7fe3bbd5c640 1 --2- 172.21.15.22:0/3779124268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3b4106f10 0x7fe3b410ea90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.199+0000 7fe3bbd5c640 1 --2- 172.21.15.22:0/3779124268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b4102790 0x7fe3b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.199+0000 7fe3bbd5c640 1 --2- 172.21.15.22:0/3779124268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b4101de0 0x7fe3b41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.199+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3779124268 >> 172.21.15.22:0/3779124268 conn(0x7fe3b40fd8c0 msgr2=0x7fe3b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.199+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3779124268 shutdown_connections 2024-09-19T08:12:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.199+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3779124268 wait complete. 2024-09-19T08:12:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.200+0000 7fe3bbd5c640 1 Processor -- start 2024-09-19T08:12:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.200+0000 7fe3bbd5c640 1 -- start start 2024-09-19T08:12:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.200+0000 7fe3bbd5c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3b4101de0 0x7fe3b4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.200+0000 7fe3bbd5c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b4102790 0x7fe3b4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.200+0000 7fe3bbd5c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b4106f10 0x7fe3b419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.200+0000 7fe3bbd5c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b4111f90 con 0x7fe3b4101de0 2024-09-19T08:12:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.200+0000 7fe3bbd5c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b4111e10 con 0x7fe3b4106f10 2024-09-19T08:12:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.200+0000 7fe3bbd5c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b4112110 con 0x7fe3b4102790 2024-09-19T08:12:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.200+0000 7fe3b9af8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3b4101de0 0x7fe3b4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.200+0000 7fe3b9af8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3b4101de0 0x7fe3b4198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53548/0 (socket says 172.21.15.22:53548) 2024-09-19T08:12:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.200+0000 7fe3b9af8640 1 -- 172.21.15.22:0/3920339964 learned_addr learned my addr 172.21.15.22:0/3920339964 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:12:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.201+0000 7fe3b92f7640 1 --2- 172.21.15.22:0/3920339964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b4102790 0x7fe3b4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.201+0000 7fe3ba2f9640 1 --2- 172.21.15.22:0/3920339964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b4106f10 0x7fe3b419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.201+0000 7fe3b9af8640 1 -- 172.21.15.22:0/3920339964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b4102790 msgr2=0x7fe3b4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.201+0000 7fe3b9af8640 1 --2- 172.21.15.22:0/3920339964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b4102790 0x7fe3b4198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.201+0000 7fe3b9af8640 1 -- 172.21.15.22:0/3920339964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b4106f10 msgr2=0x7fe3b419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.201+0000 7fe3b9af8640 1 --2- 172.21.15.22:0/3920339964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b4106f10 0x7fe3b419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.201+0000 7fe3b9af8640 1 -- 172.21.15.22:0/3920339964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b419d4e0 con 0x7fe3b4101de0 2024-09-19T08:12:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.201+0000 7fe3b9af8640 1 --2- 172.21.15.22:0/3920339964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3b4101de0 0x7fe3b4198530 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000c890 tx=0x7fe3b000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.201+0000 7fe3aaffd640 1 -- 172.21.15.22:0/3920339964 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0004220 con 0x7fe3b4101de0 2024-09-19T08:12:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.201+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3920339964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b419d7d0 con 0x7fe3b4101de0 2024-09-19T08:12:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.201+0000 7fe3aaffd640 1 -- 172.21.15.22:0/3920339964 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3b00043c0 con 0x7fe3b4101de0 2024-09-19T08:12:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.201+0000 7fe3aaffd640 1 -- 172.21.15.22:0/3920339964 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0004a00 con 0x7fe3b4101de0 2024-09-19T08:12:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.201+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3920339964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b41a50b0 con 0x7fe3b4101de0 2024-09-19T08:12:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.202+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3920339964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b41021c0 con 0x7fe3b4101de0 2024-09-19T08:12:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.203+0000 7fe3aaffd640 1 -- 172.21.15.22:0/3920339964 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe3b00079b0 con 0x7fe3b4101de0 2024-09-19T08:12:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.204+0000 7fe3aaffd640 1 --2- 172.21.15.22:0/3920339964 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe3a00a3240 0x7fe3a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:12:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.204+0000 7fe3aaffd640 1 -- 172.21.15.22:0/3920339964 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7497+0+0 (secure 0 0 0) 0x7fe3b00cf790 con 0x7fe3b4101de0 2024-09-19T08:12:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.204+0000 7fe3b92f7640 1 --2- 172.21.15.22:0/3920339964 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe3a00a3240 0x7fe3a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:12:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.205+0000 7fe3b92f7640 1 --2- 172.21.15.22:0/3920339964 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe3a00a3240 0x7fe3a00a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe3b41087e0 tx=0x7fe3a4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:12:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.208+0000 7fe3aaffd640 1 -- 172.21.15.22:0/3920339964 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe3b0014030 con 0x7fe3b4101de0 2024-09-19T08:12:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.388+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3920339964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe3b410a120 con 0x7fe3b4101de0 2024-09-19T08:12:59.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.825+0000 7fe3aaffd640 1 -- 172.21.15.22:0/3920339964 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) ==== 170+0+0 (secure 0 0 0) 0x7fe3b009c7e0 con 0x7fe3b4101de0 2024-09-19T08:12:59.825 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-09-19T08:12:59.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.828+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3920339964 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe3a00a3240 msgr2=0x7fe3a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:59.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.828+0000 7fe3bbd5c640 1 --2- 172.21.15.22:0/3920339964 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe3a00a3240 0x7fe3a00a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe3b41087e0 tx=0x7fe3a4008040 comp rx=0 tx=0).stop 2024-09-19T08:12:59.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.828+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3920339964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3b4101de0 msgr2=0x7fe3b4198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:12:59.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.828+0000 7fe3bbd5c640 1 --2- 172.21.15.22:0/3920339964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3b4101de0 0x7fe3b4198530 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000c890 tx=0x7fe3b000cd50 comp rx=0 tx=0).stop 2024-09-19T08:12:59.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.828+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3920339964 shutdown_connections 2024-09-19T08:12:59.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.828+0000 7fe3bbd5c640 1 --2- 172.21.15.22:0/3920339964 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe3a00a3240 0x7fe3a00a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:59.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.828+0000 7fe3bbd5c640 1 --2- 172.21.15.22:0/3920339964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b4106f10 0x7fe3b419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:59.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.828+0000 7fe3bbd5c640 1 --2- 172.21.15.22:0/3920339964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b4102790 0x7fe3b4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:59.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.828+0000 7fe3bbd5c640 1 --2- 172.21.15.22:0/3920339964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3b4101de0 0x7fe3b4198530 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:12:59.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.828+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3920339964 >> 172.21.15.22:0/3920339964 conn(0x7fe3b40fd8c0 msgr2=0x7fe3b40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:12:59.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.828+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3920339964 shutdown_connections 2024-09-19T08:12:59.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:12:59.828+0000 7fe3bbd5c640 1 -- 172.21.15.22:0/3920339964 wait complete. 2024-09-19T08:13:00.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:59 smithi022 ceph-mon[30963]: pgmap v276: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:13:00.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:59 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3920339964' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T08:13:00.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:59 smithi022 ceph-mon[30963]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T08:13:00.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:59 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3920339964' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T08:13:00.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:12:59 smithi022 ceph-mon[30963]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T08:13:00.173 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:59 smithi064 ceph-mon[34900]: pgmap v276: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:13:00.173 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:59 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3920339964' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T08:13:00.173 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:59 smithi064 ceph-mon[34900]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T08:13:00.173 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:59 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3920339964' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T08:13:00.173 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:12:59 smithi064 ceph-mon[34900]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T08:13:00.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:59 smithi157 ceph-mon[35143]: pgmap v276: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:13:00.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:59 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3920339964' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T08:13:00.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:59 smithi157 ceph-mon[35143]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T08:13:00.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:59 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3920339964' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T08:13:00.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:12:59 smithi157 ceph-mon[35143]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T08:13:00.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T08:13:00.761 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.280+0000 7f8bdfd94640 1 -- 172.21.15.22:0/2186416765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bd8101420 msgr2=0x7f8bd810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.280+0000 7f8bdfd94640 1 --2- 172.21.15.22:0/2186416765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bd8101420 0x7f8bd810d950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009cd0 tx=0x7f8bd002a8e0 comp rx=0 tx=0).stop 2024-09-19T08:13:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.280+0000 7f8bdfd94640 1 -- 172.21.15.22:0/2186416765 shutdown_connections 2024-09-19T08:13:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.280+0000 7f8bdfd94640 1 --2- 172.21.15.22:0/2186416765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bd8101420 0x7f8bd810d950 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.280+0000 7f8bdfd94640 1 --2- 172.21.15.22:0/2186416765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd8100a80 0x7f8bd8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.280+0000 7f8bdfd94640 1 --2- 172.21.15.22:0/2186416765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8106a80 0x7f8bd8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:01.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.280+0000 7f8bdfd94640 1 -- 172.21.15.22:0/2186416765 >> 172.21.15.22:0/2186416765 conn(0x7f8bd80fc7a0 msgr2=0x7f8bd80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:01.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.280+0000 7f8bdfd94640 1 -- 172.21.15.22:0/2186416765 shutdown_connections 2024-09-19T08:13:01.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.280+0000 7f8bdfd94640 1 -- 172.21.15.22:0/2186416765 wait complete. 2024-09-19T08:13:01.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bdfd94640 1 Processor -- start 2024-09-19T08:13:01.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bdfd94640 1 -- start start 2024-09-19T08:13:01.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bdfd94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bd8100a80 0x7f8bd81972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:01.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bdfd94640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd8101420 0x7f8bd8197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bdfd94640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8106a80 0x7f8bd819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bdfd94640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8110ed0 con 0x7f8bd8100a80 2024-09-19T08:13:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bdfd94640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8110d50 con 0x7f8bd8106a80 2024-09-19T08:13:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bdfd94640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8111050 con 0x7f8bd8101420 2024-09-19T08:13:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bddb30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bd8100a80 0x7f8bd81972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bddb30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bd8100a80 0x7f8bd81972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53564/0 (socket says 172.21.15.22:53564) 2024-09-19T08:13:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bddb30640 1 -- 172.21.15.22:0/3563183484 learned_addr learned my addr 172.21.15.22:0/3563183484 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bde331640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8106a80 0x7f8bd819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bdd32f640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd8101420 0x7f8bd8197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bddb30640 1 -- 172.21.15.22:0/3563183484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd8101420 msgr2=0x7f8bd8197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.281+0000 7f8bddb30640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd8101420 0x7f8bd8197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.282+0000 7f8bddb30640 1 -- 172.21.15.22:0/3563183484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8106a80 msgr2=0x7f8bd819bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.282+0000 7f8bddb30640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8106a80 0x7f8bd819bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.282+0000 7f8bddb30640 1 -- 172.21.15.22:0/3563183484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd819c280 con 0x7f8bd8100a80 2024-09-19T08:13:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.282+0000 7f8bde331640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8106a80 0x7f8bd819bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:13:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.282+0000 7f8bdd32f640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd8101420 0x7f8bd8197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:13:01.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.282+0000 7f8bddb30640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bd8100a80 0x7f8bd81972d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400c890 tx=0x7f8bd400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:01.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.282+0000 7f8bceffd640 1 -- 172.21.15.22:0/3563183484 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4004220 con 0x7f8bd8100a80 2024-09-19T08:13:01.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.282+0000 7f8bdfd94640 1 -- 172.21.15.22:0/3563183484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd819c570 con 0x7f8bd8100a80 2024-09-19T08:13:01.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.282+0000 7f8bceffd640 1 -- 172.21.15.22:0/3563183484 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd40043c0 con 0x7f8bd8100a80 2024-09-19T08:13:01.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.282+0000 7f8bceffd640 1 -- 172.21.15.22:0/3563183484 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4004a00 con 0x7f8bd8100a80 2024-09-19T08:13:01.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.282+0000 7f8bdfd94640 1 -- 172.21.15.22:0/3563183484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd81a3e50 con 0x7f8bd8100a80 2024-09-19T08:13:01.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.283+0000 7f8bdfd94640 1 -- 172.21.15.22:0/3563183484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd8100ee0 con 0x7f8bd8100a80 2024-09-19T08:13:01.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.284+0000 7f8bceffd640 1 -- 172.21.15.22:0/3563183484 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8bd40079b0 con 0x7f8bd8100a80 2024-09-19T08:13:01.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.285+0000 7f8bceffd640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8bc40a3240 0x7f8bc40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:01.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.285+0000 7f8bdd32f640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8bc40a3240 0x7f8bc40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:01.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.285+0000 7f8bceffd640 1 -- 172.21.15.22:0/3563183484 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7879+0+0 (secure 0 0 0) 0x7f8bd40cf4a0 con 0x7f8bd8100a80 2024-09-19T08:13:01.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.285+0000 7f8bdd32f640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8bc40a3240 0x7f8bc40a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80fec90 tx=0x7f8bc8005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:01.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.289+0000 7f8bceffd640 1 -- 172.21.15.22:0/3563183484 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8bd4014030 con 0x7f8bd8100a80 2024-09-19T08:13:01.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:01.502+0000 7f8bdfd94640 1 -- 172.21.15.22:0/3563183484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8bd81a41f0 con 0x7f8bd8100a80 2024-09-19T08:13:02.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:01 smithi064 ceph-mon[34900]: pgmap v278: 97 pgs: 1 creating+activating, 26 creating+peering, 50 unknown, 20 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:13:02.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:01 smithi064 ceph-mon[34900]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T08:13:02.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:01 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3563183484' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T08:13:02.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:01 smithi064 ceph-mon[34900]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-19T08:13:02.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:01 smithi157 ceph-mon[35143]: pgmap v278: 97 pgs: 1 creating+activating, 26 creating+peering, 50 unknown, 20 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:13:02.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:01 smithi157 ceph-mon[35143]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T08:13:02.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:01 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3563183484' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T08:13:02.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:01 smithi157 ceph-mon[35143]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-19T08:13:02.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:01 smithi022 ceph-mon[30963]: pgmap v278: 97 pgs: 1 creating+activating, 26 creating+peering, 50 unknown, 20 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:13:02.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:01 smithi022 ceph-mon[30963]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T08:13:02.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:01 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3563183484' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T08:13:02.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:01 smithi022 ceph-mon[30963]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-19T08:13:02.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:02.258+0000 7f8bceffd640 1 -- 172.21.15.22:0/3563183484 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f8bd409c370 con 0x7f8bd8100a80 2024-09-19T08:13:02.258 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T08:13:02.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:02.262+0000 7f8bdfd94640 1 -- 172.21.15.22:0/3563183484 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8bc40a3240 msgr2=0x7f8bc40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:02.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:02.262+0000 7f8bdfd94640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8bc40a3240 0x7f8bc40a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80fec90 tx=0x7f8bc8005d90 comp rx=0 tx=0).stop 2024-09-19T08:13:02.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:02.262+0000 7f8bdfd94640 1 -- 172.21.15.22:0/3563183484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bd8100a80 msgr2=0x7f8bd81972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:02.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:02.262+0000 7f8bdfd94640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bd8100a80 0x7f8bd81972d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400c890 tx=0x7f8bd400cd50 comp rx=0 tx=0).stop 2024-09-19T08:13:02.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:02.262+0000 7f8bdfd94640 1 -- 172.21.15.22:0/3563183484 shutdown_connections 2024-09-19T08:13:02.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:02.262+0000 7f8bdfd94640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8bc40a3240 0x7f8bc40a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:02.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:02.262+0000 7f8bdfd94640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8106a80 0x7f8bd819bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:02.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:02.262+0000 7f8bdfd94640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd8101420 0x7f8bd8197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:02.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:02.262+0000 7f8bdfd94640 1 --2- 172.21.15.22:0/3563183484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bd8100a80 0x7f8bd81972d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:02.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:02.262+0000 7f8bdfd94640 1 -- 172.21.15.22:0/3563183484 >> 172.21.15.22:0/3563183484 conn(0x7f8bd80fc7a0 msgr2=0x7f8bd80fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:02.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:02.263+0000 7f8bdfd94640 1 -- 172.21.15.22:0/3563183484 shutdown_connections 2024-09-19T08:13:02.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:02.263+0000 7f8bdfd94640 1 -- 172.21.15.22:0/3563183484 wait complete. 2024-09-19T08:13:02.933 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:02 smithi022 ceph-mon[30963]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T08:13:02.933 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:02 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3563183484' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T08:13:02.933 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:02 smithi022 ceph-mon[30963]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-19T08:13:02.933 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:02 smithi022 ceph-mon[30963]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-19T08:13:02.968 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:13:03.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:02 smithi064 ceph-mon[34900]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T08:13:03.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:02 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3563183484' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T08:13:03.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:02 smithi064 ceph-mon[34900]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-19T08:13:03.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:02 smithi064 ceph-mon[34900]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-19T08:13:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:03.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:02 smithi157 ceph-mon[35143]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T08:13:03.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:02 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3563183484' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T08:13:03.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:02 smithi157 ceph-mon[35143]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-19T08:13:03.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:02 smithi157 ceph-mon[35143]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-19T08:13:03.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.718+0000 7f0cd919f640 1 -- 172.21.15.22:0/894916100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd4100e20 msgr2=0x7f0cd4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:03.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.718+0000 7f0cd919f640 1 --2- 172.21.15.22:0/894916100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd4100e20 0x7f0cd4101200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009900 tx=0x7f0cc402a480 comp rx=0 tx=0).stop 2024-09-19T08:13:03.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.718+0000 7f0cd919f640 1 -- 172.21.15.22:0/894916100 shutdown_connections 2024-09-19T08:13:03.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.718+0000 7f0cd919f640 1 --2- 172.21.15.22:0/894916100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cd4102ef0 0x7f0cd410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:03.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.718+0000 7f0cd919f640 1 --2- 172.21.15.22:0/894916100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd4102570 0x7f0cd41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.718+0000 7f0cd919f640 1 --2- 172.21.15.22:0/894916100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd4100e20 0x7f0cd4101200 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.718+0000 7f0cd919f640 1 -- 172.21.15.22:0/894916100 >> 172.21.15.22:0/894916100 conn(0x7f0cd40fc7d0 msgr2=0x7f0cd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.718+0000 7f0cd919f640 1 -- 172.21.15.22:0/894916100 shutdown_connections 2024-09-19T08:13:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.718+0000 7f0cd919f640 1 -- 172.21.15.22:0/894916100 wait complete. 2024-09-19T08:13:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.719+0000 7f0cd919f640 1 Processor -- start 2024-09-19T08:13:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.719+0000 7f0cd919f640 1 -- start start 2024-09-19T08:13:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.719+0000 7f0cd919f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd4100e20 0x7f0cd4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.719+0000 7f0cd919f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cd4102570 0x7f0cd4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.719+0000 7f0cd919f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd4102ef0 0x7f0cd419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.719+0000 7f0cd919f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4110fd0 con 0x7f0cd4100e20 2024-09-19T08:13:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.719+0000 7f0cd919f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4110e50 con 0x7f0cd4102570 2024-09-19T08:13:03.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.719+0000 7f0cd919f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4111150 con 0x7f0cd4102ef0 2024-09-19T08:13:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.719+0000 7f0cd2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd4100e20 0x7f0cd4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.719+0000 7f0cd2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd4100e20 0x7f0cd4198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53582/0 (socket says 172.21.15.22:53582) 2024-09-19T08:13:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.719+0000 7f0cd2d9d640 1 -- 172.21.15.22:0/2791720088 learned_addr learned my addr 172.21.15.22:0/2791720088 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.719+0000 7f0cd359e640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd4102ef0 0x7f0cd419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.720+0000 7f0cd259c640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cd4102570 0x7f0cd4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.720+0000 7f0cd2d9d640 1 -- 172.21.15.22:0/2791720088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd4102ef0 msgr2=0x7f0cd419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.720+0000 7f0cd2d9d640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd4102ef0 0x7f0cd419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.720+0000 7f0cd2d9d640 1 -- 172.21.15.22:0/2791720088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cd4102570 msgr2=0x7f0cd4198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.720+0000 7f0cd2d9d640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cd4102570 0x7f0cd4198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.720+0000 7f0cd2d9d640 1 -- 172.21.15.22:0/2791720088 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd419d530 con 0x7f0cd4100e20 2024-09-19T08:13:03.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.720+0000 7f0cd259c640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cd4102570 0x7f0cd4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:13:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.720+0000 7f0cd2d9d640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd4100e20 0x7f0cd4198580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0cc402aa70 tx=0x7f0cc4003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.720+0000 7f0cbbfff640 1 -- 172.21.15.22:0/2791720088 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc403f070 con 0x7f0cd4100e20 2024-09-19T08:13:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.720+0000 7f0cd919f640 1 -- 172.21.15.22:0/2791720088 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd419d7c0 con 0x7f0cd4100e20 2024-09-19T08:13:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.720+0000 7f0cbbfff640 1 -- 172.21.15.22:0/2791720088 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc4031c20 con 0x7f0cd4100e20 2024-09-19T08:13:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.720+0000 7f0cbbfff640 1 -- 172.21.15.22:0/2791720088 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4035030 con 0x7f0cd4100e20 2024-09-19T08:13:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.720+0000 7f0cd919f640 1 -- 172.21.15.22:0/2791720088 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd41a5100 con 0x7f0cd4100e20 2024-09-19T08:13:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.721+0000 7f0cd919f640 1 -- 172.21.15.22:0/2791720088 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd41021b0 con 0x7f0cd4100e20 2024-09-19T08:13:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.723+0000 7f0cbbfff640 1 -- 172.21.15.22:0/2791720088 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0cc4042070 con 0x7f0cd4100e20 2024-09-19T08:13:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.723+0000 7f0cbbfff640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f0cbc0a3240 0x7f0cbc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.723+0000 7f0cbbfff640 1 -- 172.21.15.22:0/2791720088 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f0cc40f05e0 con 0x7f0cd4100e20 2024-09-19T08:13:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.723+0000 7f0cd259c640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f0cbc0a3240 0x7f0cbc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.724+0000 7f0cd259c640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f0cbc0a3240 0x7f0cbc0a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40ffe70 tx=0x7f0cc8005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.727+0000 7f0cbbfff640 1 -- 172.21.15.22:0/2791720088 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0cc40bd450 con 0x7f0cd4100e20 2024-09-19T08:13:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.904+0000 7f0cd919f640 1 -- 172.21.15.22:0/2791720088 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cd4101200 con 0x7f0cd4100e20 2024-09-19T08:13:03.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:03 smithi022 ceph-mon[30963]: pgmap v281: 97 pgs: 1 creating+activating, 26 creating+peering, 28 unknown, 42 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:13:03.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:03 smithi022 ceph-mon[30963]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T08:13:03.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:03 smithi022 ceph-mon[30963]: Cluster is now healthy 2024-09-19T08:13:03.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:03 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] up:active 2024-09-19T08:13:03.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:03 smithi022 ceph-mon[30963]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T08:13:03.905 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:03 smithi022 ceph-mon[30963]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T08:13:03.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.905+0000 7f0cbbfff640 1 -- 172.21.15.22:0/2791720088 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22243 (secure 0 0 0) 0x7f0cd4101200 con 0x7f0cd4100e20 2024-09-19T08:13:03.906 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:13:03.906 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":83,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","created":"2024-09-19T08:05:19.476952+0000","modified":"2024-09-19T08:13:01.503182+0000","last_up_change":"2024-09-19T08:12:10.339586+0000","last_in_change":"2024-09-19T08:11:47.653576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:08:25.175759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:12:57.608787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:12:59.824327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ed67f2d-9b28-427a-b69f-676e7a2abcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6803","nonce":3442715307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6805","nonce":3442715307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6809","nonce":3442715307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6807","nonce":3442715307}]},"public_addr":"172.21.15.22:6803/3442715307","cluster_addr":"172.21.15.22:6805/3442715307","heartbeat_back_addr":"172.21.15.22:6809/3442715307","heartbeat_front_addr":"172.21.15.22:6807/3442715307","state":["exists","up"]},{"osd":1,"uuid":"bff3e283-4ab1-49d7-86d8-45d361995122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6801","nonce":2379700228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6803","nonce":2379700228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6807","nonce":2379700228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6805","nonce":2379700228}]},"public_addr":"172.21.15.64:6801/2379700228","cluster_addr":"172.21.15.64:6803/2379700228","heartbeat_back_addr":"172.21.15.64:6807/2379700228","heartbeat_front_addr":"172.21.15.64:6805/2379700228","state":["exists","up"]},{"osd":2,"uuid":"81421d08-7dc6-419b-a566-dee1e1e13b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6801","nonce":3363263867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6803","nonce":3363263867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6807","nonce":3363263867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6805","nonce":3363263867}]},"public_addr":"172.21.15.157:6801/3363263867","cluster_addr":"172.21.15.157:6803/3363263867","heartbeat_back_addr":"172.21.15.157:6807/3363263867","heartbeat_front_addr":"172.21.15.157:6805/3363263867","state":["exists","up"]},{"osd":3,"uuid":"34debfd0-3226-4d78-b9e6-e03dbc5190f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6811","nonce":4225229456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6813","nonce":4225229456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6817","nonce":4225229456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6815","nonce":4225229456}]},"public_addr":"172.21.15.22:6811/4225229456","cluster_addr":"172.21.15.22:6813/4225229456","heartbeat_back_addr":"172.21.15.22:6817/4225229456","heartbeat_front_addr":"172.21.15.22:6815/4225229456","state":["exists","up"]},{"osd":4,"uuid":"1293332c-d772-4ca9-bf76-79e0b29451ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6809","nonce":1858090423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6811","nonce":1858090423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6815","nonce":1858090423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6813","nonce":1858090423}]},"public_addr":"172.21.15.64:6809/1858090423","cluster_addr":"172.21.15.64:6811/1858090423","heartbeat_back_addr":"172.21.15.64:6815/1858090423","heartbeat_front_addr":"172.21.15.64:6813/1858090423","state":["exists","up"]},{"osd":5,"uuid":"240a6c42-89a2-41c1-948c-b09914b5722e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6809","nonce":2900701212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2900701212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2900701212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2900701212}]},"public_addr":"172.21.15.157:6809/2900701212","cluster_addr":"172.21.15.157:6811/2900701212","heartbeat_back_addr":"172.21.15.157:6815/2900701212","heartbeat_front_addr":"172.21.15.157:6813/2900701212","state":["exists","up"]},{"osd":6,"uuid":"80199030-109e-41c4-bc1c-731755464770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6819","nonce":4264426242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6821","nonce":4264426242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6825","nonce":4264426242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6823","nonce":4264426242}]},"public_addr":"172.21.15.22:6819/4264426242","cluster_addr":"172.21.15.22:6821/4264426242","heartbeat_back_addr":"172.21.15.22:6825/4264426242","heartbeat_front_addr":"172.21.15.22:6823/4264426242","state":["exists","up"]},{"osd":7,"uuid":"dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6817","nonce":3980224103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6819","nonce":3980224103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6823","nonce":3980224103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6821","nonce":3980224103}]},"public_addr":"172.21.15.64:6817/3980224103","cluster_addr":"172.21.15.64:6819/3980224103","heartbeat_back_addr":"172.21.15.64:6823/3980224103","heartbeat_front_addr":"172.21.15.64:6821/3980224103","state":["exists","up"]},{"osd":8,"uuid":"764b1e6e-48c9-49f9-8124-bc13bcba3a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6817","nonce":973678477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6819","nonce":973678477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6823","nonce":973678477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6821","nonce":973678477}]},"public_addr":"172.21.15.157:6817/973678477","cluster_addr":"172.21.15.157:6819/973678477","heartbeat_back_addr":"172.21.15.157:6823/973678477","heartbeat_front_addr":"172.21.15.157:6821/973678477","state":["exists","up"]},{"osd":9,"uuid":"82914818-df47-4374-9e3e-2f71fc76e78b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6827","nonce":710033990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6829","nonce":710033990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6833","nonce":710033990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6831","nonce":710033990}]},"public_addr":"172.21.15.22:6827/710033990","cluster_addr":"172.21.15.22:6829/710033990","heartbeat_back_addr":"172.21.15.22:6833/710033990","heartbeat_front_addr":"172.21.15.22:6831/710033990","state":["exists","up"]},{"osd":10,"uuid":"36ae177d-c454-4cba-ba21-cbeee0f10dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6825","nonce":373412752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6827","nonce":373412752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6831","nonce":373412752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6829","nonce":373412752}]},"public_addr":"172.21.15.64:6825/373412752","cluster_addr":"172.21.15.64:6827/373412752","heartbeat_back_addr":"172.21.15.64:6831/373412752","heartbeat_front_addr":"172.21.15.64:6829/373412752","state":["exists","up"]},{"osd":11,"uuid":"cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6825","nonce":1918889194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6827","nonce":1918889194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6831","nonce":1918889194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6829","nonce":1918889194}]},"public_addr":"172.21.15.157:6825/1918889194","cluster_addr":"172.21.15.157:6827/1918889194","heartbeat_back_addr":"172.21.15.157:6831/1918889194","heartbeat_front_addr":"172.21.15.157:6829/1918889194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:31.378218+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:56.794167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:20.790267+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:47.039106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:11.483842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:38.151466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:03.553308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:27.865321+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:53.086530+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:18.946661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:43.697243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:12:08.936588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3712426751":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/2133059277":"2024-09-20T08:05:42.354701+0000","172.21.15.22:6801/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2851750481":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/3623845502":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/384027596":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/586163474":"2024-09-20T08:05:42.354701+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:13:03.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.907+0000 7f0cd919f640 1 -- 172.21.15.22:0/2791720088 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f0cbc0a3240 msgr2=0x7f0cbc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:03.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.907+0000 7f0cd919f640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f0cbc0a3240 0x7f0cbc0a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40ffe70 tx=0x7f0cc8005d30 comp rx=0 tx=0).stop 2024-09-19T08:13:03.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.907+0000 7f0cd919f640 1 -- 172.21.15.22:0/2791720088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd4100e20 msgr2=0x7f0cd4198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:03.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.907+0000 7f0cd919f640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd4100e20 0x7f0cd4198580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0cc402aa70 tx=0x7f0cc4003b60 comp rx=0 tx=0).stop 2024-09-19T08:13:03.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.907+0000 7f0cd919f640 1 -- 172.21.15.22:0/2791720088 shutdown_connections 2024-09-19T08:13:03.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.907+0000 7f0cd919f640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f0cbc0a3240 0x7f0cbc0a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:03.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.907+0000 7f0cd919f640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cd4102ef0 0x7f0cd419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:03.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.907+0000 7f0cd919f640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cd4102570 0x7f0cd4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:03.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.907+0000 7f0cd919f640 1 --2- 172.21.15.22:0/2791720088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd4100e20 0x7f0cd4198580 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:03.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.907+0000 7f0cd919f640 1 -- 172.21.15.22:0/2791720088 >> 172.21.15.22:0/2791720088 conn(0x7f0cd40fc7d0 msgr2=0x7f0cd4105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:03.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.908+0000 7f0cd919f640 1 -- 172.21.15.22:0/2791720088 shutdown_connections 2024-09-19T08:13:03.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:03.908+0000 7f0cd919f640 1 -- 172.21.15.22:0/2791720088 wait complete. 2024-09-19T08:13:04.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:03 smithi064 ceph-mon[34900]: pgmap v281: 97 pgs: 1 creating+activating, 26 creating+peering, 28 unknown, 42 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:13:04.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:03 smithi064 ceph-mon[34900]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T08:13:04.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:03 smithi064 ceph-mon[34900]: Cluster is now healthy 2024-09-19T08:13:04.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:03 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] up:active 2024-09-19T08:13:04.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:03 smithi064 ceph-mon[34900]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T08:13:04.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:03 smithi064 ceph-mon[34900]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T08:13:04.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:03 smithi157 ceph-mon[35143]: pgmap v281: 97 pgs: 1 creating+activating, 26 creating+peering, 28 unknown, 42 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:13:04.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:03 smithi157 ceph-mon[35143]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T08:13:04.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:03 smithi157 ceph-mon[35143]: Cluster is now healthy 2024-09-19T08:13:04.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:03 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] up:active 2024-09-19T08:13:04.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:03 smithi157 ceph-mon[35143]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T08:13:04.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:03 smithi157 ceph-mon[35143]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T08:13:04.848 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:13:05.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:04 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2791720088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:05.113 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:05.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:04 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2791720088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:05.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:04 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2791720088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:05.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.612+0000 7f196ee4c640 1 -- 172.21.15.22:0/665539772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1968106f50 msgr2=0x7f196810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:05.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.612+0000 7f196ee4c640 1 --2- 172.21.15.22:0/665539772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1968106f50 0x7f196810eaf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1960009cd0 tx=0x7f196002a8e0 comp rx=0 tx=0).stop 2024-09-19T08:13:05.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.612+0000 7f196ee4c640 1 -- 172.21.15.22:0/665539772 shutdown_connections 2024-09-19T08:13:05.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.612+0000 7f196ee4c640 1 --2- 172.21.15.22:0/665539772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1968106f50 0x7f196810eaf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:05.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.612+0000 7f196ee4c640 1 --2- 172.21.15.22:0/665539772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1968100ca0 0x7f1968106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:05.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.612+0000 7f196ee4c640 1 --2- 172.21.15.22:0/665539772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19681002f0 0x7f19681006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:05.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.612+0000 7f196ee4c640 1 -- 172.21.15.22:0/665539772 >> 172.21.15.22:0/665539772 conn(0x7f1968077210 msgr2=0x7f19680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:05.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.612+0000 7f196ee4c640 1 -- 172.21.15.22:0/665539772 shutdown_connections 2024-09-19T08:13:05.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.612+0000 7f196ee4c640 1 -- 172.21.15.22:0/665539772 wait complete. 2024-09-19T08:13:05.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196ee4c640 1 Processor -- start 2024-09-19T08:13:05.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196ee4c640 1 -- start start 2024-09-19T08:13:05.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196ee4c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19681002f0 0x7f19681983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:05.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196ee4c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1968100ca0 0x7f19681988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:05.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196ee4c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1968106f50 0x7f196819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:05.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196ee4c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1968111fa0 con 0x7f19681002f0 2024-09-19T08:13:05.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196ee4c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1968111e20 con 0x7f1968106f50 2024-09-19T08:13:05.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196ee4c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1968112120 con 0x7f1968100ca0 2024-09-19T08:13:05.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196cbe8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19681002f0 0x7f19681983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:05.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196cbe8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19681002f0 0x7f19681983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53594/0 (socket says 172.21.15.22:53594) 2024-09-19T08:13:05.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196cbe8640 1 -- 172.21.15.22:0/166735674 learned_addr learned my addr 172.21.15.22:0/166735674 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:05.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f1967fff640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1968100ca0 0x7f19681988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:05.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196d3e9640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1968106f50 0x7f196819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:05.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196cbe8640 1 -- 172.21.15.22:0/166735674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1968100ca0 msgr2=0x7f19681988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:05.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196cbe8640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1968100ca0 0x7f19681988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:05.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196cbe8640 1 -- 172.21.15.22:0/166735674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1968106f50 msgr2=0x7f196819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:05.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196cbe8640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1968106f50 0x7f196819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:05.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196cbe8640 1 -- 172.21.15.22:0/166735674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196819d360 con 0x7f19681002f0 2024-09-19T08:13:05.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.613+0000 7f196d3e9640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1968106f50 0x7f196819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:13:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.614+0000 7f196cbe8640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19681002f0 0x7f19681983b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c920 tx=0x7f195c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.614+0000 7f1965ffb640 1 -- 172.21.15.22:0/166735674 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c007c70 con 0x7f19681002f0 2024-09-19T08:13:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.614+0000 7f1965ffb640 1 -- 172.21.15.22:0/166735674 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f195c007e10 con 0x7f19681002f0 2024-09-19T08:13:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.614+0000 7f196ee4c640 1 -- 172.21.15.22:0/166735674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196819d650 con 0x7f19681002f0 2024-09-19T08:13:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.614+0000 7f1965ffb640 1 -- 172.21.15.22:0/166735674 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c018890 con 0x7f19681002f0 2024-09-19T08:13:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.614+0000 7f196ee4c640 1 -- 172.21.15.22:0/166735674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19681a4f30 con 0x7f19681002f0 2024-09-19T08:13:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.615+0000 7f196ee4c640 1 -- 172.21.15.22:0/166735674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19681006d0 con 0x7f19681002f0 2024-09-19T08:13:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.616+0000 7f1965ffb640 1 -- 172.21.15.22:0/166735674 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f195c018a70 con 0x7f19681002f0 2024-09-19T08:13:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.617+0000 7f1965ffb640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f19580a3240 0x7f19580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.617+0000 7f1967fff640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f19580a3240 0x7f19580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.617+0000 7f1965ffb640 1 -- 172.21.15.22:0/166735674 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f195c0cf910 con 0x7f19681002f0 2024-09-19T08:13:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.618+0000 7f1967fff640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f19580a3240 0x7f19580a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f19681047c0 tx=0x7f1954005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.621+0000 7f1965ffb640 1 -- 172.21.15.22:0/166735674 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f195c014030 con 0x7f19681002f0 2024-09-19T08:13:05.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.799+0000 7f196ee4c640 1 -- 172.21.15.22:0/166735674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1968199710 con 0x7f19681002f0 2024-09-19T08:13:05.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.800+0000 7f1965ffb640 1 -- 172.21.15.22:0/166735674 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22243 (secure 0 0 0) 0x7f195c09c780 con 0x7f19681002f0 2024-09-19T08:13:05.801 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:13:05.801 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":83,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","created":"2024-09-19T08:05:19.476952+0000","modified":"2024-09-19T08:13:01.503182+0000","last_up_change":"2024-09-19T08:12:10.339586+0000","last_in_change":"2024-09-19T08:11:47.653576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:08:25.175759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:12:57.608787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:12:59.824327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ed67f2d-9b28-427a-b69f-676e7a2abcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6803","nonce":3442715307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6805","nonce":3442715307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6809","nonce":3442715307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6807","nonce":3442715307}]},"public_addr":"172.21.15.22:6803/3442715307","cluster_addr":"172.21.15.22:6805/3442715307","heartbeat_back_addr":"172.21.15.22:6809/3442715307","heartbeat_front_addr":"172.21.15.22:6807/3442715307","state":["exists","up"]},{"osd":1,"uuid":"bff3e283-4ab1-49d7-86d8-45d361995122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6801","nonce":2379700228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6803","nonce":2379700228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6807","nonce":2379700228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6805","nonce":2379700228}]},"public_addr":"172.21.15.64:6801/2379700228","cluster_addr":"172.21.15.64:6803/2379700228","heartbeat_back_addr":"172.21.15.64:6807/2379700228","heartbeat_front_addr":"172.21.15.64:6805/2379700228","state":["exists","up"]},{"osd":2,"uuid":"81421d08-7dc6-419b-a566-dee1e1e13b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6801","nonce":3363263867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6803","nonce":3363263867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6807","nonce":3363263867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6805","nonce":3363263867}]},"public_addr":"172.21.15.157:6801/3363263867","cluster_addr":"172.21.15.157:6803/3363263867","heartbeat_back_addr":"172.21.15.157:6807/3363263867","heartbeat_front_addr":"172.21.15.157:6805/3363263867","state":["exists","up"]},{"osd":3,"uuid":"34debfd0-3226-4d78-b9e6-e03dbc5190f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6811","nonce":4225229456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6813","nonce":4225229456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6817","nonce":4225229456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6815","nonce":4225229456}]},"public_addr":"172.21.15.22:6811/4225229456","cluster_addr":"172.21.15.22:6813/4225229456","heartbeat_back_addr":"172.21.15.22:6817/4225229456","heartbeat_front_addr":"172.21.15.22:6815/4225229456","state":["exists","up"]},{"osd":4,"uuid":"1293332c-d772-4ca9-bf76-79e0b29451ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6809","nonce":1858090423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6811","nonce":1858090423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6815","nonce":1858090423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6813","nonce":1858090423}]},"public_addr":"172.21.15.64:6809/1858090423","cluster_addr":"172.21.15.64:6811/1858090423","heartbeat_back_addr":"172.21.15.64:6815/1858090423","heartbeat_front_addr":"172.21.15.64:6813/1858090423","state":["exists","up"]},{"osd":5,"uuid":"240a6c42-89a2-41c1-948c-b09914b5722e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6809","nonce":2900701212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2900701212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2900701212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2900701212}]},"public_addr":"172.21.15.157:6809/2900701212","cluster_addr":"172.21.15.157:6811/2900701212","heartbeat_back_addr":"172.21.15.157:6815/2900701212","heartbeat_front_addr":"172.21.15.157:6813/2900701212","state":["exists","up"]},{"osd":6,"uuid":"80199030-109e-41c4-bc1c-731755464770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6819","nonce":4264426242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6821","nonce":4264426242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6825","nonce":4264426242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6823","nonce":4264426242}]},"public_addr":"172.21.15.22:6819/4264426242","cluster_addr":"172.21.15.22:6821/4264426242","heartbeat_back_addr":"172.21.15.22:6825/4264426242","heartbeat_front_addr":"172.21.15.22:6823/4264426242","state":["exists","up"]},{"osd":7,"uuid":"dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6817","nonce":3980224103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6819","nonce":3980224103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6823","nonce":3980224103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6821","nonce":3980224103}]},"public_addr":"172.21.15.64:6817/3980224103","cluster_addr":"172.21.15.64:6819/3980224103","heartbeat_back_addr":"172.21.15.64:6823/3980224103","heartbeat_front_addr":"172.21.15.64:6821/3980224103","state":["exists","up"]},{"osd":8,"uuid":"764b1e6e-48c9-49f9-8124-bc13bcba3a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6817","nonce":973678477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6819","nonce":973678477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6823","nonce":973678477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6821","nonce":973678477}]},"public_addr":"172.21.15.157:6817/973678477","cluster_addr":"172.21.15.157:6819/973678477","heartbeat_back_addr":"172.21.15.157:6823/973678477","heartbeat_front_addr":"172.21.15.157:6821/973678477","state":["exists","up"]},{"osd":9,"uuid":"82914818-df47-4374-9e3e-2f71fc76e78b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6827","nonce":710033990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6829","nonce":710033990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6833","nonce":710033990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6831","nonce":710033990}]},"public_addr":"172.21.15.22:6827/710033990","cluster_addr":"172.21.15.22:6829/710033990","heartbeat_back_addr":"172.21.15.22:6833/710033990","heartbeat_front_addr":"172.21.15.22:6831/710033990","state":["exists","up"]},{"osd":10,"uuid":"36ae177d-c454-4cba-ba21-cbeee0f10dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6825","nonce":373412752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6827","nonce":373412752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6831","nonce":373412752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6829","nonce":373412752}]},"public_addr":"172.21.15.64:6825/373412752","cluster_addr":"172.21.15.64:6827/373412752","heartbeat_back_addr":"172.21.15.64:6831/373412752","heartbeat_front_addr":"172.21.15.64:6829/373412752","state":["exists","up"]},{"osd":11,"uuid":"cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6825","nonce":1918889194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6827","nonce":1918889194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6831","nonce":1918889194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6829","nonce":1918889194}]},"public_addr":"172.21.15.157:6825/1918889194","cluster_addr":"172.21.15.157:6827/1918889194","heartbeat_back_addr":"172.21.15.157:6831/1918889194","heartbeat_front_addr":"172.21.15.157:6829/1918889194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:31.378218+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:56.794167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:20.790267+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:47.039106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:11.483842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:38.151466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:03.553308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:27.865321+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:53.086530+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:18.946661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:43.697243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:12:08.936588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3712426751":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/2133059277":"2024-09-20T08:05:42.354701+0000","172.21.15.22:6801/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2851750481":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/3623845502":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/384027596":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/586163474":"2024-09-20T08:05:42.354701+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:13:05.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.804+0000 7f196ee4c640 1 -- 172.21.15.22:0/166735674 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f19580a3240 msgr2=0x7f19580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:05.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.804+0000 7f196ee4c640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f19580a3240 0x7f19580a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f19681047c0 tx=0x7f1954005f90 comp rx=0 tx=0).stop 2024-09-19T08:13:05.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.804+0000 7f196ee4c640 1 -- 172.21.15.22:0/166735674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19681002f0 msgr2=0x7f19681983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:05.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.804+0000 7f196ee4c640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19681002f0 0x7f19681983b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c920 tx=0x7f195c00cde0 comp rx=0 tx=0).stop 2024-09-19T08:13:05.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.804+0000 7f196ee4c640 1 -- 172.21.15.22:0/166735674 shutdown_connections 2024-09-19T08:13:05.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.804+0000 7f196ee4c640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f19580a3240 0x7f19580a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:05.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.804+0000 7f196ee4c640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1968106f50 0x7f196819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:05.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.804+0000 7f196ee4c640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1968100ca0 0x7f19681988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:05.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.804+0000 7f196ee4c640 1 --2- 172.21.15.22:0/166735674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19681002f0 0x7f19681983b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:05.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.804+0000 7f196ee4c640 1 -- 172.21.15.22:0/166735674 >> 172.21.15.22:0/166735674 conn(0x7f1968077210 msgr2=0x7f19681057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:05.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.804+0000 7f196ee4c640 1 -- 172.21.15.22:0/166735674 shutdown_connections 2024-09-19T08:13:05.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:05.804+0000 7f196ee4c640 1 -- 172.21.15.22:0/166735674 wait complete. 2024-09-19T08:13:06.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:05 smithi022 ceph-mon[30963]: pgmap v282: 97 pgs: 23 creating+peering, 74 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-09-19T08:13:06.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:05 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/166735674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:06.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:05 smithi157 ceph-mon[35143]: pgmap v282: 97 pgs: 23 creating+peering, 74 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-09-19T08:13:06.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:05 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/166735674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:06.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:05 smithi064 ceph-mon[34900]: pgmap v282: 97 pgs: 23 creating+peering, 74 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-09-19T08:13:06.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:05 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/166735674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:06.492 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T08:13:06.754 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:07.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.268+0000 7f635650b640 1 -- 172.21.15.22:0/4025898287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6350103720 msgr2=0x7f635010bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:07.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.268+0000 7f635650b640 1 --2- 172.21.15.22:0/4025898287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6350103720 0x7f635010bc10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6344009900 tx=0x7f634402a480 comp rx=0 tx=0).stop 2024-09-19T08:13:07.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.268+0000 7f635650b640 1 -- 172.21.15.22:0/4025898287 shutdown_connections 2024-09-19T08:13:07.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.268+0000 7f635650b640 1 --2- 172.21.15.22:0/4025898287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635010c1a0 0x7f635010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:07.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.268+0000 7f635650b640 1 --2- 172.21.15.22:0/4025898287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6350103720 0x7f635010bc10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:07.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.268+0000 7f635650b640 1 --2- 172.21.15.22:0/4025898287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6350102e00 0x7f63501031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:07.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.268+0000 7f635650b640 1 -- 172.21.15.22:0/4025898287 >> 172.21.15.22:0/4025898287 conn(0x7f63500fc7a0 msgr2=0x7f63500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:07.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.268+0000 7f635650b640 1 -- 172.21.15.22:0/4025898287 shutdown_connections 2024-09-19T08:13:07.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.268+0000 7f635650b640 1 -- 172.21.15.22:0/4025898287 wait complete. 2024-09-19T08:13:07.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f635650b640 1 Processor -- start 2024-09-19T08:13:07.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f635650b640 1 -- start start 2024-09-19T08:13:07.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f635650b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6350102e00 0x7f63501984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:07.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f635650b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6350103720 0x7f63501989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:07.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f635650b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635010c1a0 0x7f635019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:07.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f635650b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6350111fd0 con 0x7f6350103720 2024-09-19T08:13:07.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f635650b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6350111e50 con 0x7f6350102e00 2024-09-19T08:13:07.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f635650b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6350112150 con 0x7f635010c1a0 2024-09-19T08:13:07.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f634f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6350103720 0x7f63501989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:07.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f634f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6350103720 0x7f63501989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53620/0 (socket says 172.21.15.22:53620) 2024-09-19T08:13:07.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f634f7fe640 1 -- 172.21.15.22:0/1491626763 learned_addr learned my addr 172.21.15.22:0/1491626763 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:07.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f634ffff640 1 --2- 172.21.15.22:0/1491626763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6350102e00 0x7f63501984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:07.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f6354aa8640 1 --2- 172.21.15.22:0/1491626763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635010c1a0 0x7f635019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:07.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f634f7fe640 1 -- 172.21.15.22:0/1491626763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635010c1a0 msgr2=0x7f635019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:07.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f634f7fe640 1 --2- 172.21.15.22:0/1491626763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635010c1a0 0x7f635019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:07.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f634f7fe640 1 -- 172.21.15.22:0/1491626763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6350102e00 msgr2=0x7f63501984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:07.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f634f7fe640 1 --2- 172.21.15.22:0/1491626763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6350102e00 0x7f63501984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:07.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.269+0000 7f634f7fe640 1 -- 172.21.15.22:0/1491626763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635019d460 con 0x7f6350103720 2024-09-19T08:13:07.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.270+0000 7f634f7fe640 1 --2- 172.21.15.22:0/1491626763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6350103720 0x7f63501989f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6344006010 tx=0x7f63440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:07.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.270+0000 7f634d7fa640 1 -- 172.21.15.22:0/1491626763 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634403f030 con 0x7f6350103720 2024-09-19T08:13:07.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.270+0000 7f634d7fa640 1 -- 172.21.15.22:0/1491626763 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6344003ce0 con 0x7f6350103720 2024-09-19T08:13:07.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.270+0000 7f635650b640 1 -- 172.21.15.22:0/1491626763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635019d6f0 con 0x7f6350103720 2024-09-19T08:13:07.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.270+0000 7f634d7fa640 1 -- 172.21.15.22:0/1491626763 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344035070 con 0x7f6350103720 2024-09-19T08:13:07.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.270+0000 7f635650b640 1 -- 172.21.15.22:0/1491626763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63501a5030 con 0x7f6350103720 2024-09-19T08:13:07.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.271+0000 7f635650b640 1 -- 172.21.15.22:0/1491626763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63501031e0 con 0x7f6350103720 2024-09-19T08:13:07.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.272+0000 7f634d7fa640 1 -- 172.21.15.22:0/1491626763 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6344042030 con 0x7f6350103720 2024-09-19T08:13:07.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.273+0000 7f634d7fa640 1 --2- 172.21.15.22:0/1491626763 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f63380a32c0 0x7f63380a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:07.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.273+0000 7f634ffff640 1 --2- 172.21.15.22:0/1491626763 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f63380a32c0 0x7f63380a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:07.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.273+0000 7f634d7fa640 1 -- 172.21.15.22:0/1491626763 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f63440efdf0 con 0x7f6350103720 2024-09-19T08:13:07.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.273+0000 7f634ffff640 1 --2- 172.21.15.22:0/1491626763 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f63380a32c0 0x7f63380a5780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6340005fd0 tx=0x7f634000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:07.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.277+0000 7f634d7fa640 1 -- 172.21.15.22:0/1491626763 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f63440bcc80 con 0x7f6350103720 2024-09-19T08:13:07.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.490+0000 7f635650b640 1 -- 172.21.15.22:0/1491626763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6350107220 con 0x7f6350103720 2024-09-19T08:13:07.753 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:13:07 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i[65670]: 2024-09-19T08:13:07.493+0000 7f7940556640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:13:07.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.845+0000 7f634d7fa640 1 -- 172.21.15.22:0/1491626763 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f63440bc580 con 0x7f6350103720 2024-09-19T08:13:07.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.848+0000 7f635650b640 1 -- 172.21.15.22:0/1491626763 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f63380a32c0 msgr2=0x7f63380a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:07.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.848+0000 7f635650b640 1 --2- 172.21.15.22:0/1491626763 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f63380a32c0 0x7f63380a5780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6340005fd0 tx=0x7f634000b390 comp rx=0 tx=0).stop 2024-09-19T08:13:07.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.848+0000 7f635650b640 1 -- 172.21.15.22:0/1491626763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6350103720 msgr2=0x7f63501989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:07.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.848+0000 7f635650b640 1 --2- 172.21.15.22:0/1491626763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6350103720 0x7f63501989f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6344006010 tx=0x7f63440042f0 comp rx=0 tx=0).stop 2024-09-19T08:13:07.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.848+0000 7f635650b640 1 -- 172.21.15.22:0/1491626763 shutdown_connections 2024-09-19T08:13:07.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.848+0000 7f635650b640 1 --2- 172.21.15.22:0/1491626763 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f63380a32c0 0x7f63380a5780 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:07.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.848+0000 7f635650b640 1 --2- 172.21.15.22:0/1491626763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f635010c1a0 0x7f635019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:07.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.848+0000 7f635650b640 1 --2- 172.21.15.22:0/1491626763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6350103720 0x7f63501989f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:07.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.848+0000 7f635650b640 1 --2- 172.21.15.22:0/1491626763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6350102e00 0x7f63501984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:07.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.848+0000 7f635650b640 1 -- 172.21.15.22:0/1491626763 >> 172.21.15.22:0/1491626763 conn(0x7f63500fc7a0 msgr2=0x7f63500fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:07.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.848+0000 7f635650b640 1 -- 172.21.15.22:0/1491626763 shutdown_connections 2024-09-19T08:13:07.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:07.848+0000 7f635650b640 1 -- 172.21.15.22:0/1491626763 wait complete. 2024-09-19T08:13:08.178 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:07 smithi022 ceph-mon[30963]: pgmap v283: 97 pgs: 97 active+clean; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-19T08:13:08.178 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:07 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1491626763' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T08:13:08.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:07 smithi157 ceph-mon[35143]: pgmap v283: 97 pgs: 97 active+clean; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-19T08:13:08.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:07 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1491626763' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T08:13:08.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:07 smithi064 ceph-mon[34900]: pgmap v283: 97 pgs: 97 active+clean; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-19T08:13:08.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:07 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1491626763' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T08:13:08.525 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-19T08:13:08.526 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-19T08:13:08.791 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:09.201 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:08 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1491626763' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T08:13:09.201 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:08 smithi022 ceph-mon[30963]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T08:13:09.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:08 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1491626763' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T08:13:09.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:08 smithi157 ceph-mon[35143]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T08:13:09.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.287+0000 7fe3db563640 1 -- 172.21.15.22:0/2276111134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d4074590 msgr2=0x7fe3d4074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:09.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.287+0000 7fe3db563640 1 --2- 172.21.15.22:0/2276111134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d4074590 0x7fe3d4074970 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009900 tx=0x7fe3d002a480 comp rx=0 tx=0).stop 2024-09-19T08:13:09.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.287+0000 7fe3db563640 1 -- 172.21.15.22:0/2276111134 shutdown_connections 2024-09-19T08:13:09.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.287+0000 7fe3db563640 1 --2- 172.21.15.22:0/2276111134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3d4112b90 0x7fe3d4114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:09.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.287+0000 7fe3db563640 1 --2- 172.21.15.22:0/2276111134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3d4074eb0 0x7fe3d4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:09.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.287+0000 7fe3db563640 1 --2- 172.21.15.22:0/2276111134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d4074590 0x7fe3d4074970 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:09.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.287+0000 7fe3db563640 1 -- 172.21.15.22:0/2276111134 >> 172.21.15.22:0/2276111134 conn(0x7fe3d40fd8c0 msgr2=0x7fe3d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:09.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.287+0000 7fe3db563640 1 -- 172.21.15.22:0/2276111134 shutdown_connections 2024-09-19T08:13:09.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.287+0000 7fe3db563640 1 -- 172.21.15.22:0/2276111134 wait complete. 2024-09-19T08:13:09.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.287+0000 7fe3db563640 1 Processor -- start 2024-09-19T08:13:09.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.287+0000 7fe3db563640 1 -- start start 2024-09-19T08:13:09.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3db563640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3d4074590 0x7fe3d41a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:09.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3db563640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d4074eb0 0x7fe3d41a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:09.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3db563640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3d4112b90 0x7fe3d419ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:09.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3db563640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d4118650 con 0x7fe3d4074eb0 2024-09-19T08:13:09.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3db563640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d41184d0 con 0x7fe3d4112b90 2024-09-19T08:13:09.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3db563640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d41187d0 con 0x7fe3d4074590 2024-09-19T08:13:09.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3d8afe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d4074eb0 0x7fe3d41a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:09.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3d8afe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d4074eb0 0x7fe3d41a4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44426/0 (socket says 172.21.15.22:44426) 2024-09-19T08:13:09.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3d8afe640 1 -- 172.21.15.22:0/1531005252 learned_addr learned my addr 172.21.15.22:0/1531005252 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:09.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3d9b00640 1 --2- 172.21.15.22:0/1531005252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3d4112b90 0x7fe3d419ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:09.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3d8afe640 1 -- 172.21.15.22:0/1531005252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3d4074590 msgr2=0x7fe3d41a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:09.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3d8afe640 1 --2- 172.21.15.22:0/1531005252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3d4074590 0x7fe3d41a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:09.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3d8afe640 1 -- 172.21.15.22:0/1531005252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3d4112b90 msgr2=0x7fe3d419ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:09.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3d8afe640 1 --2- 172.21.15.22:0/1531005252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3d4112b90 0x7fe3d419ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:09.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3d8afe640 1 -- 172.21.15.22:0/1531005252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d419f150 con 0x7fe3d4074eb0 2024-09-19T08:13:09.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3d8afe640 1 --2- 172.21.15.22:0/1531005252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d4074eb0 0x7fe3d41a4da0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400b4a0 tx=0x7fe3c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:09.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3ca7fc640 1 -- 172.21.15.22:0/1531005252 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4014070 con 0x7fe3d4074eb0 2024-09-19T08:13:09.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3ca7fc640 1 -- 172.21.15.22:0/1531005252 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c4010af0 con 0x7fe3d4074eb0 2024-09-19T08:13:09.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3db563640 1 -- 172.21.15.22:0/1531005252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d419f440 con 0x7fe3d4074eb0 2024-09-19T08:13:09.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3ca7fc640 1 -- 172.21.15.22:0/1531005252 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4017d10 con 0x7fe3d4074eb0 2024-09-19T08:13:09.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.288+0000 7fe3db563640 1 -- 172.21.15.22:0/1531005252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d41ab730 con 0x7fe3d4074eb0 2024-09-19T08:13:09.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.289+0000 7fe3db563640 1 -- 172.21.15.22:0/1531005252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d40749f0 con 0x7fe3d4074eb0 2024-09-19T08:13:09.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.291+0000 7fe3ca7fc640 1 -- 172.21.15.22:0/1531005252 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe3c4017070 con 0x7fe3d4074eb0 2024-09-19T08:13:09.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.292+0000 7fe3ca7fc640 1 --2- 172.21.15.22:0/1531005252 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe3c00a32c0 0x7fe3c00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:09.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.292+0000 7fe3d92ff640 1 --2- 172.21.15.22:0/1531005252 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe3c00a32c0 0x7fe3c00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:09.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.292+0000 7fe3ca7fc640 1 -- 172.21.15.22:0/1531005252 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fe3c40cef80 con 0x7fe3d4074eb0 2024-09-19T08:13:09.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.293+0000 7fe3d92ff640 1 --2- 172.21.15.22:0/1531005252 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe3c00a32c0 0x7fe3c00a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3d002a9a0 tx=0x7fe3d0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:09.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.295+0000 7fe3ca7fc640 1 -- 172.21.15.22:0/1531005252 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe3c409bdf0 con 0x7fe3d4074eb0 2024-09-19T08:13:09.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:08 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1491626763' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T08:13:09.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:08 smithi064 ceph-mon[34900]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T08:13:09.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.511+0000 7fe3db563640 1 -- 172.21.15.22:0/1531005252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fe3d402d7c0 con 0x7fe3d4074eb0 2024-09-19T08:13:09.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.857+0000 7fe3ca7fc640 1 -- 172.21.15.22:0/1531005252 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fe3c409b6f0 con 0x7fe3d4074eb0 2024-09-19T08:13:09.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.859+0000 7fe3db563640 1 -- 172.21.15.22:0/1531005252 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe3c00a32c0 msgr2=0x7fe3c00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:09.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.859+0000 7fe3db563640 1 --2- 172.21.15.22:0/1531005252 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe3c00a32c0 0x7fe3c00a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3d002a9a0 tx=0x7fe3d0000bc0 comp rx=0 tx=0).stop 2024-09-19T08:13:09.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.859+0000 7fe3db563640 1 -- 172.21.15.22:0/1531005252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d4074eb0 msgr2=0x7fe3d41a4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:09.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.859+0000 7fe3db563640 1 --2- 172.21.15.22:0/1531005252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d4074eb0 0x7fe3d41a4da0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400b4a0 tx=0x7fe3c400b960 comp rx=0 tx=0).stop 2024-09-19T08:13:09.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.859+0000 7fe3db563640 1 -- 172.21.15.22:0/1531005252 shutdown_connections 2024-09-19T08:13:09.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.859+0000 7fe3db563640 1 --2- 172.21.15.22:0/1531005252 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe3c00a32c0 0x7fe3c00a5780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:09.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.859+0000 7fe3db563640 1 --2- 172.21.15.22:0/1531005252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3d4112b90 0x7fe3d419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:09.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.859+0000 7fe3db563640 1 --2- 172.21.15.22:0/1531005252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3d4074eb0 0x7fe3d41a4da0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:09.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.859+0000 7fe3db563640 1 --2- 172.21.15.22:0/1531005252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3d4074590 0x7fe3d41a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:09.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.859+0000 7fe3db563640 1 -- 172.21.15.22:0/1531005252 >> 172.21.15.22:0/1531005252 conn(0x7fe3d40fd8c0 msgr2=0x7fe3d40ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:09.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.859+0000 7fe3db563640 1 -- 172.21.15.22:0/1531005252 shutdown_connections 2024-09-19T08:13:09.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:09.859+0000 7fe3db563640 1 -- 172.21.15.22:0/1531005252 wait complete. 2024-09-19T08:13:10.222 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:09 smithi022 ceph-mon[30963]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T08:13:10.222 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:09 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1531005252' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T08:13:10.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:09 smithi157 ceph-mon[35143]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T08:13:10.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:09 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1531005252' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T08:13:10.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:09 smithi064 ceph-mon[34900]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T08:13:10.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:09 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1531005252' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T08:13:10.601 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T08:13:10.868 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:11.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:10 smithi022 ceph-mon[30963]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T08:13:11.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:10 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1531005252' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T08:13:11.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:10 smithi022 ceph-mon[30963]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T08:13:11.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:10 smithi022 ceph-mon[30963]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T08:13:11.140 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:10 smithi022 ceph-mon[30963]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-19T08:13:11.140 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:10 smithi022 ceph-mon[30963]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-19T08:13:11.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:10 smithi157 ceph-mon[35143]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T08:13:11.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:10 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1531005252' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T08:13:11.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:10 smithi157 ceph-mon[35143]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T08:13:11.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:10 smithi157 ceph-mon[35143]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T08:13:11.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:10 smithi157 ceph-mon[35143]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-19T08:13:11.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:10 smithi157 ceph-mon[35143]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-19T08:13:11.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:10 smithi064 ceph-mon[34900]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T08:13:11.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:10 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1531005252' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T08:13:11.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:10 smithi064 ceph-mon[34900]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T08:13:11.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:10 smithi064 ceph-mon[34900]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T08:13:11.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:10 smithi064 ceph-mon[34900]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-19T08:13:11.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:10 smithi064 ceph-mon[34900]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-19T08:13:11.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.355+0000 7fcb3c64d640 1 -- 172.21.15.22:0/940639894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb34103660 msgr2=0x7fcb3410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:11.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.355+0000 7fcb3c64d640 1 --2- 172.21.15.22:0/940639894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb34103660 0x7fcb3410bb50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009900 tx=0x7fcb2402a480 comp rx=0 tx=0).stop 2024-09-19T08:13:11.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.355+0000 7fcb3c64d640 1 -- 172.21.15.22:0/940639894 shutdown_connections 2024-09-19T08:13:11.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.355+0000 7fcb3c64d640 1 --2- 172.21.15.22:0/940639894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb3410c170 0x7fcb3410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:11.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.355+0000 7fcb3c64d640 1 --2- 172.21.15.22:0/940639894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb34103660 0x7fcb3410bb50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:11.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.355+0000 7fcb3c64d640 1 --2- 172.21.15.22:0/940639894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34102d40 0x7fcb34103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.355+0000 7fcb3c64d640 1 -- 172.21.15.22:0/940639894 >> 172.21.15.22:0/940639894 conn(0x7fcb340fc7a0 msgr2=0x7fcb340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.355+0000 7fcb3c64d640 1 -- 172.21.15.22:0/940639894 shutdown_connections 2024-09-19T08:13:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.355+0000 7fcb3c64d640 1 -- 172.21.15.22:0/940639894 wait complete. 2024-09-19T08:13:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.356+0000 7fcb3c64d640 1 Processor -- start 2024-09-19T08:13:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.356+0000 7fcb3c64d640 1 -- start start 2024-09-19T08:13:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.356+0000 7fcb3c64d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34102d40 0x7fcb34198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.356+0000 7fcb3c64d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb34103660 0x7fcb34198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.356+0000 7fcb3c64d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb3410c170 0x7fcb3419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.356+0000 7fcb3c64d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcb34111f70 con 0x7fcb34103660 2024-09-19T08:13:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.356+0000 7fcb3c64d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcb34111df0 con 0x7fcb3410c170 2024-09-19T08:13:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.356+0000 7fcb3c64d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb341120f0 con 0x7fcb34102d40 2024-09-19T08:13:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.356+0000 7fcb39be8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb34103660 0x7fcb34198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.356+0000 7fcb39be8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb34103660 0x7fcb34198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44442/0 (socket says 172.21.15.22:44442) 2024-09-19T08:13:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.356+0000 7fcb39be8640 1 -- 172.21.15.22:0/3477460400 learned_addr learned my addr 172.21.15.22:0/3477460400 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.356+0000 7fcb3a3e9640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34102d40 0x7fcb34198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.357+0000 7fcb3abea640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb3410c170 0x7fcb3419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.357+0000 7fcb39be8640 1 -- 172.21.15.22:0/3477460400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34102d40 msgr2=0x7fcb34198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.357+0000 7fcb39be8640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34102d40 0x7fcb34198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.357+0000 7fcb39be8640 1 -- 172.21.15.22:0/3477460400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb3410c170 msgr2=0x7fcb3419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.357+0000 7fcb39be8640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb3410c170 0x7fcb3419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.357+0000 7fcb39be8640 1 -- 172.21.15.22:0/3477460400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3419d3e0 con 0x7fcb34103660 2024-09-19T08:13:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.357+0000 7fcb3abea640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb3410c170 0x7fcb3419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:13:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.357+0000 7fcb39be8640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb34103660 0x7fcb34198970 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcb24006010 tx=0x7fcb240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.357+0000 7fcb2b7fe640 1 -- 172.21.15.22:0/3477460400 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2403f030 con 0x7fcb34103660 2024-09-19T08:13:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.357+0000 7fcb2b7fe640 1 -- 172.21.15.22:0/3477460400 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb24003ce0 con 0x7fcb34103660 2024-09-19T08:13:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.357+0000 7fcb3c64d640 1 -- 172.21.15.22:0/3477460400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3419d670 con 0x7fcb34103660 2024-09-19T08:13:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.357+0000 7fcb2b7fe640 1 -- 172.21.15.22:0/3477460400 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24035070 con 0x7fcb34103660 2024-09-19T08:13:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.357+0000 7fcb3c64d640 1 -- 172.21.15.22:0/3477460400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb341a4fb0 con 0x7fcb34103660 2024-09-19T08:13:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.358+0000 7fcb3c64d640 1 -- 172.21.15.22:0/3477460400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb34103120 con 0x7fcb34103660 2024-09-19T08:13:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.360+0000 7fcb2b7fe640 1 -- 172.21.15.22:0/3477460400 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fcb24042030 con 0x7fcb34103660 2024-09-19T08:13:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.360+0000 7fcb2b7fe640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcb200a32c0 0x7fcb200a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.360+0000 7fcb3a3e9640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcb200a32c0 0x7fcb200a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.360+0000 7fcb2b7fe640 1 -- 172.21.15.22:0/3477460400 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fcb240effd0 con 0x7fcb34103660 2024-09-19T08:13:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.361+0000 7fcb3a3e9640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcb200a32c0 0x7fcb200a5780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcb30005fd0 tx=0x7fcb3000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.363+0000 7fcb2b7fe640 1 -- 172.21.15.22:0/3477460400 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcb240bce60 con 0x7fcb34103660 2024-09-19T08:13:11.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.576+0000 7fcb3c64d640 1 -- 172.21.15.22:0/3477460400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fcb341071e0 con 0x7fcb34103660 2024-09-19T08:13:11.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.871+0000 7fcb2b7fe640 1 -- 172.21.15.22:0/3477460400 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fcb240bc760 con 0x7fcb34103660 2024-09-19T08:13:11.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.874+0000 7fcb3c64d640 1 -- 172.21.15.22:0/3477460400 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcb200a32c0 msgr2=0x7fcb200a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:11.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.874+0000 7fcb3c64d640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcb200a32c0 0x7fcb200a5780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcb30005fd0 tx=0x7fcb3000b390 comp rx=0 tx=0).stop 2024-09-19T08:13:11.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.874+0000 7fcb3c64d640 1 -- 172.21.15.22:0/3477460400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb34103660 msgr2=0x7fcb34198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:11.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.874+0000 7fcb3c64d640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb34103660 0x7fcb34198970 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcb24006010 tx=0x7fcb240042f0 comp rx=0 tx=0).stop 2024-09-19T08:13:11.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.874+0000 7fcb3c64d640 1 -- 172.21.15.22:0/3477460400 shutdown_connections 2024-09-19T08:13:11.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.874+0000 7fcb3c64d640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fcb200a32c0 0x7fcb200a5780 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:11.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.874+0000 7fcb3c64d640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb3410c170 0x7fcb3419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:11.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.874+0000 7fcb3c64d640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb34103660 0x7fcb34198970 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:11.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.874+0000 7fcb3c64d640 1 --2- 172.21.15.22:0/3477460400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34102d40 0x7fcb34198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:11.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.874+0000 7fcb3c64d640 1 -- 172.21.15.22:0/3477460400 >> 172.21.15.22:0/3477460400 conn(0x7fcb340fc7a0 msgr2=0x7fcb340fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:11.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.874+0000 7fcb3c64d640 1 -- 172.21.15.22:0/3477460400 shutdown_connections 2024-09-19T08:13:11.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:11.874+0000 7fcb3c64d640 1 -- 172.21.15.22:0/3477460400 wait complete. 2024-09-19T08:13:12.199 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:11 smithi022 ceph-mon[30963]: pgmap v285: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-19T08:13:12.199 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:11 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] up:active 2024-09-19T08:13:12.199 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:11 smithi022 ceph-mon[30963]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T08:13:12.199 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:11 smithi022 ceph-mon[30963]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T08:13:12.199 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:11 smithi022 ceph-mon[30963]: Cluster is now healthy 2024-09-19T08:13:12.199 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:11 smithi022 ceph-mon[30963]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-19T08:13:12.199 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:11 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-19T08:13:12.199 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:11 smithi022 ceph-mon[30963]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T08:13:12.200 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:11 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3477460400' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T08:13:12.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:11 smithi157 ceph-mon[35143]: pgmap v285: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-19T08:13:12.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:11 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] up:active 2024-09-19T08:13:12.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:11 smithi157 ceph-mon[35143]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T08:13:12.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:11 smithi157 ceph-mon[35143]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T08:13:12.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:11 smithi157 ceph-mon[35143]: Cluster is now healthy 2024-09-19T08:13:12.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:11 smithi157 ceph-mon[35143]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-19T08:13:12.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:11 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-19T08:13:12.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:11 smithi157 ceph-mon[35143]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T08:13:12.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:11 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3477460400' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T08:13:12.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:11 smithi064 ceph-mon[34900]: pgmap v285: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-19T08:13:12.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:11 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] up:active 2024-09-19T08:13:12.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:11 smithi064 ceph-mon[34900]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T08:13:12.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:11 smithi064 ceph-mon[34900]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T08:13:12.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:11 smithi064 ceph-mon[34900]: Cluster is now healthy 2024-09-19T08:13:12.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:11 smithi064 ceph-mon[34900]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-19T08:13:12.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:11 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-19T08:13:12.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:11 smithi064 ceph-mon[34900]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T08:13:12.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:11 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3477460400' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T08:13:12.719 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T08:13:12.753 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:13:12 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i[65670]: 2024-09-19T08:13:12.495+0000 7f7940556640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T08:13:12.753 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:13:12 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i[65670]: 2024-09-19T08:13:12.495+0000 7f7940556640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T08:13:12.985 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:13.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:12 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3477460400' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T08:13:13.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:12 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] up:active 2024-09-19T08:13:13.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:12 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-19T08:13:13.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:12 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:13:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:12 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3477460400' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T08:13:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:12 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] up:active 2024-09-19T08:13:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:12 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-19T08:13:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:12 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:13:13.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:12 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3477460400' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T08:13:13.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:12 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] up:active 2024-09-19T08:13:13.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:12 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-19T08:13:13.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:12 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:13:13.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.540+0000 7fc82d65a640 1 -- 172.21.15.22:0/3697698123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc828109130 msgr2=0x7fc828110cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:13.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.540+0000 7fc82d65a640 1 --2- 172.21.15.22:0/3697698123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc828109130 0x7fc828110cb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc820009ca0 tx=0x7fc82002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:13:13.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.540+0000 7fc82d65a640 1 -- 172.21.15.22:0/3697698123 shutdown_connections 2024-09-19T08:13:13.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.540+0000 7fc82d65a640 1 --2- 172.21.15.22:0/3697698123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc828109130 0x7fc828110cb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:13.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.541+0000 7fc82d65a640 1 --2- 172.21.15.22:0/3697698123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8281049b0 0x7fc828108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:13.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.541+0000 7fc82d65a640 1 --2- 172.21.15.22:0/3697698123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc828104000 0x7fc8281043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:13.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.541+0000 7fc82d65a640 1 -- 172.21.15.22:0/3697698123 >> 172.21.15.22:0/3697698123 conn(0x7fc8280ffa20 msgr2=0x7fc828101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:13.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.541+0000 7fc82d65a640 1 -- 172.21.15.22:0/3697698123 shutdown_connections 2024-09-19T08:13:13.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.541+0000 7fc82d65a640 1 -- 172.21.15.22:0/3697698123 wait complete. 2024-09-19T08:13:13.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.541+0000 7fc82d65a640 1 Processor -- start 2024-09-19T08:13:13.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.541+0000 7fc82d65a640 1 -- start start 2024-09-19T08:13:13.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc82d65a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc828104000 0x7fc82819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:13.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc82d65a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8281049b0 0x7fc82819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:13.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc82d65a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc828109130 0x7fc82819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:13.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc82d65a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc828114160 con 0x7fc828104000 2024-09-19T08:13:13.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc82d65a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc828113fe0 con 0x7fc8281049b0 2024-09-19T08:13:13.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc82d65a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281142e0 con 0x7fc828109130 2024-09-19T08:13:13.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc826ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc828104000 0x7fc82819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:13.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc826ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc828104000 0x7fc82819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44472/0 (socket says 172.21.15.22:44472) 2024-09-19T08:13:13.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc826ffd640 1 -- 172.21.15.22:0/1101091342 learned_addr learned my addr 172.21.15.22:0/1101091342 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:13.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc8267fc640 1 --2- 172.21.15.22:0/1101091342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8281049b0 0x7fc82819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:13.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc8277fe640 1 --2- 172.21.15.22:0/1101091342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc828109130 0x7fc82819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:13.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc826ffd640 1 -- 172.21.15.22:0/1101091342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc828109130 msgr2=0x7fc82819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:13.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc826ffd640 1 --2- 172.21.15.22:0/1101091342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc828109130 0x7fc82819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:13.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc826ffd640 1 -- 172.21.15.22:0/1101091342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8281049b0 msgr2=0x7fc82819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:13.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc826ffd640 1 --2- 172.21.15.22:0/1101091342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8281049b0 0x7fc82819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:13.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc826ffd640 1 -- 172.21.15.22:0/1101091342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82819f5b0 con 0x7fc828104000 2024-09-19T08:13:13.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc826ffd640 1 --2- 172.21.15.22:0/1101091342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc828104000 0x7fc82819a600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc81800d770 tx=0x7fc81800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:13.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc80ffff640 1 -- 172.21.15.22:0/1101091342 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818017070 con 0x7fc828104000 2024-09-19T08:13:13.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc80ffff640 1 -- 172.21.15.22:0/1101091342 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc818005500 con 0x7fc828104000 2024-09-19T08:13:13.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc82d65a640 1 -- 172.21.15.22:0/1101091342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82819f8a0 con 0x7fc828104000 2024-09-19T08:13:13.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc80ffff640 1 -- 172.21.15.22:0/1101091342 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818004e00 con 0x7fc828104000 2024-09-19T08:13:13.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.542+0000 7fc82d65a640 1 -- 172.21.15.22:0/1101091342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8281a7180 con 0x7fc828104000 2024-09-19T08:13:13.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.543+0000 7fc82d65a640 1 -- 172.21.15.22:0/1101091342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8281043e0 con 0x7fc828104000 2024-09-19T08:13:13.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.545+0000 7fc80ffff640 1 -- 172.21.15.22:0/1101091342 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc818004fa0 con 0x7fc828104000 2024-09-19T08:13:13.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.545+0000 7fc80ffff640 1 --2- 172.21.15.22:0/1101091342 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc8100a3240 0x7fc8100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:13.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.545+0000 7fc8267fc640 1 --2- 172.21.15.22:0/1101091342 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc8100a3240 0x7fc8100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:13.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.545+0000 7fc80ffff640 1 -- 172.21.15.22:0/1101091342 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fc8180cf720 con 0x7fc828104000 2024-09-19T08:13:13.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.546+0000 7fc8267fc640 1 --2- 172.21.15.22:0/1101091342 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc8100a3240 0x7fc8100a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc81c005fd0 tx=0x7fc81c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:13.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.548+0000 7fc80ffff640 1 -- 172.21.15.22:0/1101091342 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc81809c590 con 0x7fc828104000 2024-09-19T08:13:13.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.763+0000 7fc82d65a640 1 -- 172.21.15.22:0/1101091342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc82810c370 con 0x7fc828104000 2024-09-19T08:13:13.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.878+0000 7fc80ffff640 1 -- 172.21.15.22:0/1101091342 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7fc81809be90 con 0x7fc828104000 2024-09-19T08:13:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.880+0000 7fc82d65a640 1 -- 172.21.15.22:0/1101091342 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc8100a3240 msgr2=0x7fc8100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.880+0000 7fc82d65a640 1 --2- 172.21.15.22:0/1101091342 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc8100a3240 0x7fc8100a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc81c005fd0 tx=0x7fc81c005850 comp rx=0 tx=0).stop 2024-09-19T08:13:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.880+0000 7fc82d65a640 1 -- 172.21.15.22:0/1101091342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc828104000 msgr2=0x7fc82819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.880+0000 7fc82d65a640 1 --2- 172.21.15.22:0/1101091342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc828104000 0x7fc82819a600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc81800d770 tx=0x7fc81800dc30 comp rx=0 tx=0).stop 2024-09-19T08:13:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.880+0000 7fc82d65a640 1 -- 172.21.15.22:0/1101091342 shutdown_connections 2024-09-19T08:13:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.880+0000 7fc82d65a640 1 --2- 172.21.15.22:0/1101091342 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc8100a3240 0x7fc8100a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.880+0000 7fc82d65a640 1 --2- 172.21.15.22:0/1101091342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc828109130 0x7fc82819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.880+0000 7fc82d65a640 1 --2- 172.21.15.22:0/1101091342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8281049b0 0x7fc82819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.880+0000 7fc82d65a640 1 --2- 172.21.15.22:0/1101091342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc828104000 0x7fc82819a600 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.880+0000 7fc82d65a640 1 -- 172.21.15.22:0/1101091342 >> 172.21.15.22:0/1101091342 conn(0x7fc8280ffa20 msgr2=0x7fc828100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.880+0000 7fc82d65a640 1 -- 172.21.15.22:0/1101091342 shutdown_connections 2024-09-19T08:13:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:13.880+0000 7fc82d65a640 1 -- 172.21.15.22:0/1101091342 wait complete. 2024-09-19T08:13:14.199 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:13 smithi022 ceph-mon[30963]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-19T08:13:14.200 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:13 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1101091342' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T08:13:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:13 smithi157 ceph-mon[35143]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-19T08:13:14.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:13 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1101091342' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T08:13:14.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:13 smithi064 ceph-mon[34900]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-19T08:13:14.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:13 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1101091342' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T08:13:14.557 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-19T08:13:14.557 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T08:13:14.818 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:15.196 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:14 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1101091342' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T08:13:15.197 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:14 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:13:15.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:14 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1101091342' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T08:13:15.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:14 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:13:15.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.281+0000 7f86e8866640 1 -- 172.21.15.22:0/2591113014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e4106a80 msgr2=0x7f86e4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:15.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.281+0000 7f86e8866640 1 --2- 172.21.15.22:0/2591113014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e4106a80 0x7f86e4106e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009930 tx=0x7f86d402a440 comp rx=0 tx=0).stop 2024-09-19T08:13:15.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.282+0000 7f86e8866640 1 -- 172.21.15.22:0/2591113014 shutdown_connections 2024-09-19T08:13:15.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.282+0000 7f86e8866640 1 --2- 172.21.15.22:0/2591113014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86e4101420 0x7f86e410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:15.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.282+0000 7f86e8866640 1 --2- 172.21.15.22:0/2591113014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e4100a80 0x7f86e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:15.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.282+0000 7f86e8866640 1 --2- 172.21.15.22:0/2591113014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e4106a80 0x7f86e4106e60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:15.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.282+0000 7f86e8866640 1 -- 172.21.15.22:0/2591113014 >> 172.21.15.22:0/2591113014 conn(0x7f86e40fc7a0 msgr2=0x7f86e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:15.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.282+0000 7f86e8866640 1 -- 172.21.15.22:0/2591113014 shutdown_connections 2024-09-19T08:13:15.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.282+0000 7f86e8866640 1 -- 172.21.15.22:0/2591113014 wait complete. 2024-09-19T08:13:15.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.282+0000 7f86e8866640 1 Processor -- start 2024-09-19T08:13:15.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e8866640 1 -- start start 2024-09-19T08:13:15.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e8866640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e4100a80 0x7f86e4199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:15.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e8866640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86e4101420 0x7f86e4199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:15.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e8866640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e4106a80 0x7f86e41a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:15.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e8866640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4110e40 con 0x7f86e4100a80 2024-09-19T08:13:15.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e8866640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4110cc0 con 0x7f86e4101420 2024-09-19T08:13:15.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e8866640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4110fc0 con 0x7f86e4106a80 2024-09-19T08:13:15.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e4100a80 0x7f86e4199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:15.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e4100a80 0x7f86e4199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44492/0 (socket says 172.21.15.22:44492) 2024-09-19T08:13:15.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e259c640 1 -- 172.21.15.22:0/1069396156 learned_addr learned my addr 172.21.15.22:0/1069396156 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:15.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e2d9d640 1 --2- 172.21.15.22:0/1069396156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e4106a80 0x7f86e41a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:15.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e259c640 1 -- 172.21.15.22:0/1069396156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e4106a80 msgr2=0x7f86e41a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:15.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e1d9b640 1 --2- 172.21.15.22:0/1069396156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86e4101420 0x7f86e4199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:15.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e259c640 1 --2- 172.21.15.22:0/1069396156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e4106a80 0x7f86e41a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:15.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e259c640 1 -- 172.21.15.22:0/1069396156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86e4101420 msgr2=0x7f86e4199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:15.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e259c640 1 --2- 172.21.15.22:0/1069396156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86e4101420 0x7f86e4199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:15.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e259c640 1 -- 172.21.15.22:0/1069396156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e41a47c0 con 0x7f86e4100a80 2024-09-19T08:13:15.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86e259c640 1 --2- 172.21.15.22:0/1069396156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e4100a80 0x7f86e4199200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009a50 tx=0x7f86d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:15.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.283+0000 7f86d37fe640 1 -- 172.21.15.22:0/1069396156 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d403f030 con 0x7f86e4100a80 2024-09-19T08:13:15.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.284+0000 7f86d37fe640 1 -- 172.21.15.22:0/1069396156 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d4003ce0 con 0x7f86e4100a80 2024-09-19T08:13:15.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.284+0000 7f86e8866640 1 -- 172.21.15.22:0/1069396156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e41a4990 con 0x7f86e4100a80 2024-09-19T08:13:15.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.284+0000 7f86d37fe640 1 -- 172.21.15.22:0/1069396156 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4035070 con 0x7f86e4100a80 2024-09-19T08:13:15.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.284+0000 7f86e8866640 1 -- 172.21.15.22:0/1069396156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e41a4de0 con 0x7f86e4100a80 2024-09-19T08:13:15.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.285+0000 7f86e8866640 1 -- 172.21.15.22:0/1069396156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e4100ee0 con 0x7f86e4100a80 2024-09-19T08:13:15.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.286+0000 7f86d37fe640 1 -- 172.21.15.22:0/1069396156 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f86d4042030 con 0x7f86e4100a80 2024-09-19T08:13:15.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.286+0000 7f86d37fe640 1 --2- 172.21.15.22:0/1069396156 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f86cc0a3240 0x7f86cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:15.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.286+0000 7f86e1d9b640 1 --2- 172.21.15.22:0/1069396156 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f86cc0a3240 0x7f86cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:15.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.286+0000 7f86d37fe640 1 -- 172.21.15.22:0/1069396156 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f86d40efe10 con 0x7f86e4100a80 2024-09-19T08:13:15.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.287+0000 7f86e1d9b640 1 --2- 172.21.15.22:0/1069396156 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f86cc0a3240 0x7f86cc0a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f86e406b2e0 tx=0x7f86d8005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:15.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.291+0000 7f86d37fe640 1 -- 172.21.15.22:0/1069396156 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f86d40bcc80 con 0x7f86e4100a80 2024-09-19T08:13:15.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:14 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1101091342' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T08:13:15.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:14 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:13:15.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.493+0000 7f86e8866640 1 -- 172.21.15.22:0/1069396156 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f86e40fd5d0 con 0x7f86cc0a3240 2024-09-19T08:13:15.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.520+0000 7f86d37fe640 1 -- 172.21.15.22:0/1069396156 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f86e40fd5d0 con 0x7f86cc0a3240 2024-09-19T08:13:15.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.522+0000 7f86e8866640 1 -- 172.21.15.22:0/1069396156 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f86cc0a3240 msgr2=0x7f86cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:15.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.522+0000 7f86e8866640 1 --2- 172.21.15.22:0/1069396156 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f86cc0a3240 0x7f86cc0a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f86e406b2e0 tx=0x7f86d8005cc0 comp rx=0 tx=0).stop 2024-09-19T08:13:15.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.522+0000 7f86e8866640 1 -- 172.21.15.22:0/1069396156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e4100a80 msgr2=0x7f86e4199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:15.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.522+0000 7f86e8866640 1 --2- 172.21.15.22:0/1069396156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e4100a80 0x7f86e4199200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009a50 tx=0x7f86d4004270 comp rx=0 tx=0).stop 2024-09-19T08:13:15.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.522+0000 7f86e8866640 1 -- 172.21.15.22:0/1069396156 shutdown_connections 2024-09-19T08:13:15.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.522+0000 7f86e8866640 1 --2- 172.21.15.22:0/1069396156 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f86cc0a3240 0x7f86cc0a5700 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:15.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.522+0000 7f86e8866640 1 --2- 172.21.15.22:0/1069396156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86e4106a80 0x7f86e41a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:15.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.522+0000 7f86e8866640 1 --2- 172.21.15.22:0/1069396156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86e4101420 0x7f86e4199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:15.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.522+0000 7f86e8866640 1 --2- 172.21.15.22:0/1069396156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e4100a80 0x7f86e4199200 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:15.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.522+0000 7f86e8866640 1 -- 172.21.15.22:0/1069396156 >> 172.21.15.22:0/1069396156 conn(0x7f86e40fc7a0 msgr2=0x7f86e40fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:15.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.522+0000 7f86e8866640 1 -- 172.21.15.22:0/1069396156 shutdown_connections 2024-09-19T08:13:15.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:15.522+0000 7f86e8866640 1 -- 172.21.15.22:0/1069396156 wait complete. 2024-09-19T08:13:15.757 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:13:15 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:13:15.498+0000 7efd44b11640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:13:15.757 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:13:15 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:13:15.498+0000 7efd44b11640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:13:15.757 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:13:15 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:13:15.498+0000 7efd44b11640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:13:15.757 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:13:15 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:13:15.498+0000 7efd44b11640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:13:15.757 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:13:15 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:13:15.498+0000 7efd44b11640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:13:16.201 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:15 smithi022 ceph-mon[30963]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.1 KiB/s wr, 10 op/s 2024-09-19T08:13:16.201 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:15 smithi022 ceph-mon[30963]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:16.232 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T08:13:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:15 smithi157 ceph-mon[35143]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.1 KiB/s wr, 10 op/s 2024-09-19T08:13:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:15 smithi157 ceph-mon[35143]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:16.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:15 smithi064 ceph-mon[34900]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.1 KiB/s wr, 10 op/s 2024-09-19T08:13:16.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:15 smithi064 ceph-mon[34900]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:16.491 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:16.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.971+0000 7f46827cc640 1 -- 172.21.15.22:0/1271334163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f467c10c170 msgr2=0x7f467c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:16.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.971+0000 7f46827cc640 1 --2- 172.21.15.22:0/1271334163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f467c10c170 0x7f467c10e610 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4674009ca0 tx=0x7f467402a950 comp rx=0 tx=0).stop 2024-09-19T08:13:16.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.972+0000 7f46827cc640 1 -- 172.21.15.22:0/1271334163 shutdown_connections 2024-09-19T08:13:16.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.972+0000 7f46827cc640 1 --2- 172.21.15.22:0/1271334163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f467c10c170 0x7f467c10e610 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:16.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.972+0000 7f46827cc640 1 --2- 172.21.15.22:0/1271334163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f467c103660 0x7f467c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:16.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.972+0000 7f46827cc640 1 --2- 172.21.15.22:0/1271334163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f467c102d40 0x7f467c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:16.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.972+0000 7f46827cc640 1 -- 172.21.15.22:0/1271334163 >> 172.21.15.22:0/1271334163 conn(0x7f467c0fc7a0 msgr2=0x7f467c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:16.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.972+0000 7f46827cc640 1 -- 172.21.15.22:0/1271334163 shutdown_connections 2024-09-19T08:13:16.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.972+0000 7f46827cc640 1 -- 172.21.15.22:0/1271334163 wait complete. 2024-09-19T08:13:16.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.972+0000 7f46827cc640 1 Processor -- start 2024-09-19T08:13:16.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.972+0000 7f46827cc640 1 -- start start 2024-09-19T08:13:16.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f46827cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f467c102d40 0x7f467c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:16.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f46827cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f467c103660 0x7f467c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:16.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f46827cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f467c10c170 0x7f467c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:16.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f46827cc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f467c111fb0 con 0x7f467c102d40 2024-09-19T08:13:16.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f46827cc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f467c111e30 con 0x7f467c10c170 2024-09-19T08:13:16.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f46827cc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f467c112130 con 0x7f467c103660 2024-09-19T08:13:16.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f467bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f467c102d40 0x7f467c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f467bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f467c102d40 0x7f467c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44520/0 (socket says 172.21.15.22:44520) 2024-09-19T08:13:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f467bfff640 1 -- 172.21.15.22:0/1158587501 learned_addr learned my addr 172.21.15.22:0/1158587501 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f4680d69640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f467c10c170 0x7f467c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f467b7fe640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f467c103660 0x7f467c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f467bfff640 1 -- 172.21.15.22:0/1158587501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f467c103660 msgr2=0x7f467c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f467bfff640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f467c103660 0x7f467c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f467bfff640 1 -- 172.21.15.22:0/1158587501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f467c10c170 msgr2=0x7f467c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f467bfff640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f467c10c170 0x7f467c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f467bfff640 1 -- 172.21.15.22:0/1158587501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467c1a6970 con 0x7f467c102d40 2024-09-19T08:13:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f467b7fe640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f467c103660 0x7f467c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:13:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.973+0000 7f467bfff640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f467c102d40 0x7f467c1009a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f466c00b4a0 tx=0x7f466c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.974+0000 7f46797fa640 1 -- 172.21.15.22:0/1158587501 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c014070 con 0x7f467c102d40 2024-09-19T08:13:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.974+0000 7f46797fa640 1 -- 172.21.15.22:0/1158587501 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466c010af0 con 0x7f467c102d40 2024-09-19T08:13:16.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.974+0000 7f46827cc640 1 -- 172.21.15.22:0/1158587501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467c1a6c60 con 0x7f467c102d40 2024-09-19T08:13:16.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.974+0000 7f46797fa640 1 -- 172.21.15.22:0/1158587501 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c017d10 con 0x7f467c102d40 2024-09-19T08:13:16.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.974+0000 7f46827cc640 1 -- 172.21.15.22:0/1158587501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467c1a71a0 con 0x7f467c102d40 2024-09-19T08:13:16.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.975+0000 7f46827cc640 1 -- 172.21.15.22:0/1158587501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f467c103120 con 0x7f467c102d40 2024-09-19T08:13:16.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.976+0000 7f46797fa640 1 -- 172.21.15.22:0/1158587501 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f466c017070 con 0x7f467c102d40 2024-09-19T08:13:16.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.976+0000 7f46797fa640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f46640a3350 0x7f46640a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:16.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.977+0000 7f467b7fe640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f46640a3350 0x7f46640a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:16.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.977+0000 7f46797fa640 1 -- 172.21.15.22:0/1158587501 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f466c0cead0 con 0x7f467c102d40 2024-09-19T08:13:16.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.977+0000 7f467b7fe640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f46640a3350 0x7f46640a5810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f467c102430 tx=0x7f4670009670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:16.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:16.981+0000 7f46797fa640 1 -- 172.21.15.22:0/1158587501 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f466c09b950 con 0x7f467c102d40 2024-09-19T08:13:17.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:17.183+0000 7f46827cc640 1 -- 172.21.15.22:0/1158587501 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f467c107280 con 0x7f46640a3350 2024-09-19T08:13:17.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:17.210+0000 7f46797fa640 1 -- 172.21.15.22:0/1158587501 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4670009880 con 0x7f46640a3350 2024-09-19T08:13:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:17.213+0000 7f46827cc640 1 -- 172.21.15.22:0/1158587501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f46640a3350 msgr2=0x7f46640a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:17.213+0000 7f46827cc640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f46640a3350 0x7f46640a5810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f467c102430 tx=0x7f4670009670 comp rx=0 tx=0).stop 2024-09-19T08:13:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:17.213+0000 7f46827cc640 1 -- 172.21.15.22:0/1158587501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f467c102d40 msgr2=0x7f467c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:17.213+0000 7f46827cc640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f467c102d40 0x7f467c1009a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f466c00b4a0 tx=0x7f466c00b960 comp rx=0 tx=0).stop 2024-09-19T08:13:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:17.213+0000 7f46827cc640 1 -- 172.21.15.22:0/1158587501 shutdown_connections 2024-09-19T08:13:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:17.213+0000 7f46827cc640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f46640a3350 0x7f46640a5810 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:17.213+0000 7f46827cc640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f467c10c170 0x7f467c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:17.213+0000 7f46827cc640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f467c103660 0x7f467c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:17.213+0000 7f46827cc640 1 --2- 172.21.15.22:0/1158587501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f467c102d40 0x7f467c1009a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:17.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:17.213+0000 7f46827cc640 1 -- 172.21.15.22:0/1158587501 >> 172.21.15.22:0/1158587501 conn(0x7f467c0fc7a0 msgr2=0x7f467c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:17.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:17.213+0000 7f46827cc640 1 -- 172.21.15.22:0/1158587501 shutdown_connections 2024-09-19T08:13:17.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:17.213+0000 7f46827cc640 1 -- 172.21.15.22:0/1158587501 wait complete. 2024-09-19T08:13:17.965 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T08:13:18.155 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:17 smithi064 ceph-mon[34900]: pgmap v288: 97 pgs: 97 active+clean; 588 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-19T08:13:18.155 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:17 smithi064 ceph-mon[34900]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T08:13:18.155 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:17 smithi064 ceph-mon[34900]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:18.220 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:18.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:17 smithi157 ceph-mon[35143]: pgmap v288: 97 pgs: 97 active+clean; 588 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-19T08:13:18.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:17 smithi157 ceph-mon[35143]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T08:13:18.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:17 smithi157 ceph-mon[35143]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:18.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:17 smithi022 ceph-mon[30963]: pgmap v288: 97 pgs: 97 active+clean; 588 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-19T08:13:18.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:17 smithi022 ceph-mon[30963]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T08:13:18.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:17 smithi022 ceph-mon[30963]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:18.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.704+0000 7f2dab257640 1 -- 172.21.15.22:0/3248740831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4102790 msgr2=0x7f2da41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:18.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.704+0000 7f2dab257640 1 --2- 172.21.15.22:0/3248740831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4102790 0x7f2da41067e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009900 tx=0x7f2d9802a410 comp rx=0 tx=0).stop 2024-09-19T08:13:18.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.704+0000 7f2dab257640 1 -- 172.21.15.22:0/3248740831 shutdown_connections 2024-09-19T08:13:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.704+0000 7f2dab257640 1 --2- 172.21.15.22:0/3248740831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da4106f10 0x7f2da410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.704+0000 7f2dab257640 1 --2- 172.21.15.22:0/3248740831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4102790 0x7f2da41067e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.704+0000 7f2dab257640 1 --2- 172.21.15.22:0/3248740831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da4101de0 0x7f2da41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.704+0000 7f2dab257640 1 -- 172.21.15.22:0/3248740831 >> 172.21.15.22:0/3248740831 conn(0x7f2da40fd8c0 msgr2=0x7f2da40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.704+0000 7f2dab257640 1 -- 172.21.15.22:0/3248740831 shutdown_connections 2024-09-19T08:13:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.704+0000 7f2dab257640 1 -- 172.21.15.22:0/3248740831 wait complete. 2024-09-19T08:13:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.704+0000 7f2dab257640 1 Processor -- start 2024-09-19T08:13:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.704+0000 7f2dab257640 1 -- start start 2024-09-19T08:13:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.704+0000 7f2dab257640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da4101de0 0x7f2da40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.704+0000 7f2dab257640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da4102790 0x7f2da4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.704+0000 7f2dab257640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4106f10 0x7f2da4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2dab257640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2da4111f30 con 0x7f2da4106f10 2024-09-19T08:13:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2dab257640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2da4111db0 con 0x7f2da4102790 2024-09-19T08:13:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2dab257640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2da41120b0 con 0x7f2da4101de0 2024-09-19T08:13:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da97f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4106f10 0x7f2da4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da4102790 0x7f2da4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da97f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4106f10 0x7f2da4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51710/0 (socket says 172.21.15.22:51710) 2024-09-19T08:13:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da97f4640 1 -- 172.21.15.22:0/1588161476 learned_addr learned my addr 172.21.15.22:0/1588161476 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da8ff3640 1 --2- 172.21.15.22:0/1588161476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da4101de0 0x7f2da40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da97f4640 1 -- 172.21.15.22:0/1588161476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da4101de0 msgr2=0x7f2da40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da97f4640 1 --2- 172.21.15.22:0/1588161476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da4101de0 0x7f2da40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da97f4640 1 -- 172.21.15.22:0/1588161476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da4102790 msgr2=0x7f2da4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da97f4640 1 --2- 172.21.15.22:0/1588161476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da4102790 0x7f2da4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da97f4640 1 -- 172.21.15.22:0/1588161476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da41a6840 con 0x7f2da4106f10 2024-09-19T08:13:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da3fff640 1 --2- 172.21.15.22:0/1588161476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da4102790 0x7f2da4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:13:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da97f4640 1 --2- 172.21.15.22:0/1588161476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4106f10 0x7f2da4074f70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00eb10 tx=0x7f2d9c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da1ffb640 1 -- 172.21.15.22:0/1588161476 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c00ce20 con 0x7f2da4106f10 2024-09-19T08:13:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da1ffb640 1 -- 172.21.15.22:0/1588161476 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d9c005020 con 0x7f2da4106f10 2024-09-19T08:13:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2dab257640 1 -- 172.21.15.22:0/1588161476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da41a6b30 con 0x7f2da4106f10 2024-09-19T08:13:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.705+0000 7f2da1ffb640 1 -- 172.21.15.22:0/1588161476 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c010690 con 0x7f2da4106f10 2024-09-19T08:13:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.706+0000 7f2dab257640 1 -- 172.21.15.22:0/1588161476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da41a6ff0 con 0x7f2da4106f10 2024-09-19T08:13:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.707+0000 7f2dab257640 1 -- 172.21.15.22:0/1588161476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da41021c0 con 0x7f2da4106f10 2024-09-19T08:13:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.707+0000 7f2da1ffb640 1 -- 172.21.15.22:0/1588161476 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2d9c005540 con 0x7f2da4106f10 2024-09-19T08:13:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.708+0000 7f2da1ffb640 1 --2- 172.21.15.22:0/1588161476 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2d8c0a32d0 0x7f2d8c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.708+0000 7f2da8ff3640 1 --2- 172.21.15.22:0/1588161476 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2d8c0a32d0 0x7f2d8c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.708+0000 7f2da1ffb640 1 -- 172.21.15.22:0/1588161476 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f2d9c0cfe10 con 0x7f2da4106f10 2024-09-19T08:13:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.709+0000 7f2da8ff3640 1 --2- 172.21.15.22:0/1588161476 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2d8c0a32d0 0x7f2d8c0a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400abe0 tx=0x7f2d9400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:18.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.712+0000 7f2da1ffb640 1 -- 172.21.15.22:0/1588161476 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2d9c09cb80 con 0x7f2da4106f10 2024-09-19T08:13:18.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.915+0000 7f2dab257640 1 -- 172.21.15.22:0/1588161476 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f2da4062630 con 0x7f2d8c0a32d0 2024-09-19T08:13:18.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.921+0000 7f2da1ffb640 1 -- 172.21.15.22:0/1588161476 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2da4062630 con 0x7f2d8c0a32d0 2024-09-19T08:13:18.921 INFO:teuthology.orchestra.run.smithi022.stdout:/volumes/qa/sv_0/0ae81cad-ab8d-423b-9aa8-141910c37b03 2024-09-19T08:13:18.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.923+0000 7f2dab257640 1 -- 172.21.15.22:0/1588161476 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2d8c0a32d0 msgr2=0x7f2d8c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:18.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.923+0000 7f2dab257640 1 --2- 172.21.15.22:0/1588161476 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2d8c0a32d0 0x7f2d8c0a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400abe0 tx=0x7f2d9400a2d0 comp rx=0 tx=0).stop 2024-09-19T08:13:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.923+0000 7f2dab257640 1 -- 172.21.15.22:0/1588161476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4106f10 msgr2=0x7f2da4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.923+0000 7f2dab257640 1 --2- 172.21.15.22:0/1588161476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4106f10 0x7f2da4074f70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00eb10 tx=0x7f2d9c00ef10 comp rx=0 tx=0).stop 2024-09-19T08:13:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.923+0000 7f2dab257640 1 -- 172.21.15.22:0/1588161476 shutdown_connections 2024-09-19T08:13:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.923+0000 7f2dab257640 1 --2- 172.21.15.22:0/1588161476 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2d8c0a32d0 0x7f2d8c0a5790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.923+0000 7f2dab257640 1 --2- 172.21.15.22:0/1588161476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2da4106f10 0x7f2da4074f70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.923+0000 7f2dab257640 1 --2- 172.21.15.22:0/1588161476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da4102790 0x7f2da4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.923+0000 7f2dab257640 1 --2- 172.21.15.22:0/1588161476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da4101de0 0x7f2da40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.923+0000 7f2dab257640 1 -- 172.21.15.22:0/1588161476 >> 172.21.15.22:0/1588161476 conn(0x7f2da40fd8c0 msgr2=0x7f2da40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.923+0000 7f2dab257640 1 -- 172.21.15.22:0/1588161476 shutdown_connections 2024-09-19T08:13:18.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:18.923+0000 7f2dab257640 1 -- 172.21.15.22:0/1588161476 wait complete. 2024-09-19T08:13:19.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T08:13:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:20.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:19 smithi022 ceph-mon[30963]: pgmap v289: 97 pgs: 97 active+clean; 588 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-19T08:13:20.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:19 smithi022 ceph-mon[30963]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:20.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:19 smithi157 ceph-mon[35143]: pgmap v289: 97 pgs: 97 active+clean; 588 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-19T08:13:20.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:19 smithi157 ceph-mon[35143]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:20.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:19 smithi064 ceph-mon[34900]: pgmap v289: 97 pgs: 97 active+clean; 588 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-19T08:13:20.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:19 smithi064 ceph-mon[34900]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:20.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.395+0000 7efca30e4640 1 -- 172.21.15.22:0/2172878656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc9c10c170 msgr2=0x7efc9c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:20.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.395+0000 7efca30e4640 1 --2- 172.21.15.22:0/2172878656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc9c10c170 0x7efc9c10e610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efc94009ca0 tx=0x7efc9402a950 comp rx=0 tx=0).stop 2024-09-19T08:13:20.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.395+0000 7efca30e4640 1 -- 172.21.15.22:0/2172878656 shutdown_connections 2024-09-19T08:13:20.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.395+0000 7efca30e4640 1 --2- 172.21.15.22:0/2172878656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc9c10c170 0x7efc9c10e610 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:20.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.395+0000 7efca30e4640 1 --2- 172.21.15.22:0/2172878656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9c103660 0x7efc9c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:20.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.395+0000 7efca30e4640 1 --2- 172.21.15.22:0/2172878656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc9c102d40 0x7efc9c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:20.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.395+0000 7efca30e4640 1 -- 172.21.15.22:0/2172878656 >> 172.21.15.22:0/2172878656 conn(0x7efc9c0fc7a0 msgr2=0x7efc9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:20.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.395+0000 7efca30e4640 1 -- 172.21.15.22:0/2172878656 shutdown_connections 2024-09-19T08:13:20.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.395+0000 7efca30e4640 1 -- 172.21.15.22:0/2172878656 wait complete. 2024-09-19T08:13:20.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efca30e4640 1 Processor -- start 2024-09-19T08:13:20.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efca30e4640 1 -- start start 2024-09-19T08:13:20.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efca30e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9c102d40 0x7efc9c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:20.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efca30e4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc9c103660 0x7efc9c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:20.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efca30e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc9c10c170 0x7efc9c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:20.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efca30e4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c111fb0 con 0x7efc9c103660 2024-09-19T08:13:20.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efca30e4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c111e30 con 0x7efc9c10c170 2024-09-19T08:13:20.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efca30e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c112130 con 0x7efc9c102d40 2024-09-19T08:13:20.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efc9bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc9c103660 0x7efc9c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:20.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efc9bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc9c103660 0x7efc9c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51724/0 (socket says 172.21.15.22:51724) 2024-09-19T08:13:20.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efc9bfff640 1 -- 172.21.15.22:0/910444676 learned_addr learned my addr 172.21.15.22:0/910444676 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:20.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efca0e80640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9c102d40 0x7efc9c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:20.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efca1681640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc9c10c170 0x7efc9c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:20.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efc9bfff640 1 -- 172.21.15.22:0/910444676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9c102d40 msgr2=0x7efc9c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:20.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efc9bfff640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9c102d40 0x7efc9c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:20.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efc9bfff640 1 -- 172.21.15.22:0/910444676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc9c10c170 msgr2=0x7efc9c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:20.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efc9bfff640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc9c10c170 0x7efc9c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:20.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efc9bfff640 1 -- 172.21.15.22:0/910444676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9c19d380 con 0x7efc9c103660 2024-09-19T08:13:20.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.396+0000 7efca0e80640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9c102d40 0x7efc9c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:13:20.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.397+0000 7efc9bfff640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc9c103660 0x7efc9c198910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efc8800b3f0 tx=0x7efc8800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:20.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.397+0000 7efc99ffb640 1 -- 172.21.15.22:0/910444676 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88015020 con 0x7efc9c103660 2024-09-19T08:13:20.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.397+0000 7efc99ffb640 1 -- 172.21.15.22:0/910444676 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc88011d10 con 0x7efc9c103660 2024-09-19T08:13:20.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.397+0000 7efca30e4640 1 -- 172.21.15.22:0/910444676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9c19d670 con 0x7efc9c103660 2024-09-19T08:13:20.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.397+0000 7efc99ffb640 1 -- 172.21.15.22:0/910444676 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8800f2e0 con 0x7efc9c103660 2024-09-19T08:13:20.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.397+0000 7efca30e4640 1 -- 172.21.15.22:0/910444676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc9c1a4f50 con 0x7efc9c103660 2024-09-19T08:13:20.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.398+0000 7efca30e4640 1 -- 172.21.15.22:0/910444676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc9c103120 con 0x7efc9c103660 2024-09-19T08:13:20.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.399+0000 7efc99ffb640 1 -- 172.21.15.22:0/910444676 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7efc88018020 con 0x7efc9c103660 2024-09-19T08:13:20.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.400+0000 7efc99ffb640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efc8c0a3350 0x7efc8c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:20.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.400+0000 7efca0e80640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efc8c0a3350 0x7efc8c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:20.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.400+0000 7efc99ffb640 1 -- 172.21.15.22:0/910444676 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7efc880cf150 con 0x7efc9c103660 2024-09-19T08:13:20.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.400+0000 7efca0e80640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efc8c0a3350 0x7efc8c0a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efc9000abe0 tx=0x7efc9000a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:20.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.404+0000 7efc99ffb640 1 -- 172.21.15.22:0/910444676 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efc8801f020 con 0x7efc9c103660 2024-09-19T08:13:20.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.607+0000 7efca30e4640 1 -- 172.21.15.22:0/910444676 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efc9c0feac0 con 0x7efc8c0a3350 2024-09-19T08:13:20.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.636+0000 7efc99ffb640 1 -- 172.21.15.22:0/910444676 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc9c0feac0 con 0x7efc8c0a3350 2024-09-19T08:13:20.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.638+0000 7efca30e4640 1 -- 172.21.15.22:0/910444676 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efc8c0a3350 msgr2=0x7efc8c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:20.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.638+0000 7efca30e4640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efc8c0a3350 0x7efc8c0a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efc9000abe0 tx=0x7efc9000a5c0 comp rx=0 tx=0).stop 2024-09-19T08:13:20.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.638+0000 7efca30e4640 1 -- 172.21.15.22:0/910444676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc9c103660 msgr2=0x7efc9c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:20.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.638+0000 7efca30e4640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc9c103660 0x7efc9c198910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efc8800b3f0 tx=0x7efc8800b8b0 comp rx=0 tx=0).stop 2024-09-19T08:13:20.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.639+0000 7efca30e4640 1 -- 172.21.15.22:0/910444676 shutdown_connections 2024-09-19T08:13:20.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.639+0000 7efca30e4640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efc8c0a3350 0x7efc8c0a5810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:20.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.639+0000 7efca30e4640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc9c10c170 0x7efc9c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:20.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.639+0000 7efca30e4640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc9c103660 0x7efc9c198910 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:20.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.639+0000 7efca30e4640 1 --2- 172.21.15.22:0/910444676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9c102d40 0x7efc9c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:20.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.639+0000 7efca30e4640 1 -- 172.21.15.22:0/910444676 >> 172.21.15.22:0/910444676 conn(0x7efc9c0fc7a0 msgr2=0x7efc9c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:20.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.639+0000 7efca30e4640 1 -- 172.21.15.22:0/910444676 shutdown_connections 2024-09-19T08:13:20.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:20.639+0000 7efca30e4640 1 -- 172.21.15.22:0/910444676 wait complete. 2024-09-19T08:13:21.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T08:13:21.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:21 smithi064 ceph-mon[34900]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:21.661 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:21.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:21 smithi157 ceph-mon[35143]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:21.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:21 smithi022 ceph-mon[30963]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.183+0000 7f91996eb640 1 -- 172.21.15.22:0/2116845933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9194103660 msgr2=0x7f919410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.183+0000 7f91996eb640 1 --2- 172.21.15.22:0/2116845933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9194103660 0x7f919410bb50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f917c009900 tx=0x7f917c02a480 comp rx=0 tx=0).stop 2024-09-19T08:13:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.183+0000 7f91996eb640 1 -- 172.21.15.22:0/2116845933 shutdown_connections 2024-09-19T08:13:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.183+0000 7f91996eb640 1 --2- 172.21.15.22:0/2116845933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919410c170 0x7f919410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.183+0000 7f91996eb640 1 --2- 172.21.15.22:0/2116845933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9194103660 0x7f919410bb50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.183+0000 7f91996eb640 1 --2- 172.21.15.22:0/2116845933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9194102d40 0x7f9194103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.183+0000 7f91996eb640 1 -- 172.21.15.22:0/2116845933 >> 172.21.15.22:0/2116845933 conn(0x7f91940fc7a0 msgr2=0x7f91940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.183+0000 7f91996eb640 1 -- 172.21.15.22:0/2116845933 shutdown_connections 2024-09-19T08:13:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.183+0000 7f91996eb640 1 -- 172.21.15.22:0/2116845933 wait complete. 2024-09-19T08:13:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.184+0000 7f91996eb640 1 Processor -- start 2024-09-19T08:13:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.184+0000 7f91996eb640 1 -- start start 2024-09-19T08:13:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.184+0000 7f91996eb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9194102d40 0x7f91941009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.184+0000 7f91996eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9194103660 0x7f9194100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.184+0000 7f91996eb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f919410c170 0x7f9194101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.184+0000 7f91996eb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9194112030 con 0x7f919410c170 2024-09-19T08:13:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.184+0000 7f91996eb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9194111eb0 con 0x7f9194102d40 2024-09-19T08:13:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.184+0000 7f91996eb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f91941121b0 con 0x7f9194103660 2024-09-19T08:13:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f91937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f919410c170 0x7f9194101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f91937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f919410c170 0x7f9194101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51738/0 (socket says 172.21.15.22:51738) 2024-09-19T08:13:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f91937fe640 1 -- 172.21.15.22:0/3716220624 learned_addr learned my addr 172.21.15.22:0/3716220624 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f9192ffd640 1 --2- 172.21.15.22:0/3716220624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9194102d40 0x7f91941009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f918a7fc640 1 --2- 172.21.15.22:0/3716220624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9194103660 0x7f9194100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f91937fe640 1 -- 172.21.15.22:0/3716220624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9194103660 msgr2=0x7f9194100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f91937fe640 1 --2- 172.21.15.22:0/3716220624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9194103660 0x7f9194100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f91937fe640 1 -- 172.21.15.22:0/3716220624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9194102d40 msgr2=0x7f91941009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f91937fe640 1 --2- 172.21.15.22:0/3716220624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9194102d40 0x7f91941009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f91937fe640 1 -- 172.21.15.22:0/3716220624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91941a6b00 con 0x7f919410c170 2024-09-19T08:13:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f91937fe640 1 --2- 172.21.15.22:0/3716220624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f919410c170 0x7f9194101420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f918c00b970 tx=0x7f918c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f9190ff9640 1 -- 172.21.15.22:0/3716220624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c019070 con 0x7f919410c170 2024-09-19T08:13:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f9190ff9640 1 -- 172.21.15.22:0/3716220624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f918c014440 con 0x7f919410c170 2024-09-19T08:13:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f91996eb640 1 -- 172.21.15.22:0/3716220624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91941a6df0 con 0x7f919410c170 2024-09-19T08:13:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f9190ff9640 1 -- 172.21.15.22:0/3716220624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c0075d0 con 0x7f919410c170 2024-09-19T08:13:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.185+0000 7f91996eb640 1 -- 172.21.15.22:0/3716220624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91941a72a0 con 0x7f919410c170 2024-09-19T08:13:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.187+0000 7f91996eb640 1 -- 172.21.15.22:0/3716220624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91941031a0 con 0x7f919410c170 2024-09-19T08:13:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.188+0000 7f9190ff9640 1 -- 172.21.15.22:0/3716220624 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f918c01c070 con 0x7f919410c170 2024-09-19T08:13:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.188+0000 7f9190ff9640 1 --2- 172.21.15.22:0/3716220624 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f91800a32d0 0x7f91800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.188+0000 7f9190ff9640 1 -- 172.21.15.22:0/3716220624 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f918c0ced90 con 0x7f919410c170 2024-09-19T08:13:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.188+0000 7f9192ffd640 1 --2- 172.21.15.22:0/3716220624 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f91800a32d0 0x7f91800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.189+0000 7f9192ffd640 1 --2- 172.21.15.22:0/3716220624 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f91800a32d0 0x7f91800a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9184009990 tx=0x7f9184009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.192+0000 7f9190ff9640 1 -- 172.21.15.22:0/3716220624 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f918c09bc10 con 0x7f919410c170 2024-09-19T08:13:22.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.397+0000 7f91996eb640 1 -- 172.21.15.22:0/3716220624 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f9194107250 con 0x7f91800a32d0 2024-09-19T08:13:22.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:22 smithi022 ceph-mon[30963]: pgmap v290: 97 pgs: 97 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-19T08:13:22.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.403+0000 7f9190ff9640 1 -- 172.21.15.22:0/3716220624 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f918400e850 con 0x7f91800a32d0 2024-09-19T08:13:22.403 INFO:teuthology.orchestra.run.smithi022.stdout:/volumes/qa/sv_1/c12f3609-5f1b-47c1-8ca8-c1ddc3b712c0 2024-09-19T08:13:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.405+0000 7f91996eb640 1 -- 172.21.15.22:0/3716220624 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f91800a32d0 msgr2=0x7f91800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.405+0000 7f91996eb640 1 --2- 172.21.15.22:0/3716220624 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f91800a32d0 0x7f91800a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9184009990 tx=0x7f9184009110 comp rx=0 tx=0).stop 2024-09-19T08:13:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.405+0000 7f91996eb640 1 -- 172.21.15.22:0/3716220624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f919410c170 msgr2=0x7f9194101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.405+0000 7f91996eb640 1 --2- 172.21.15.22:0/3716220624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f919410c170 0x7f9194101420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f918c00b970 tx=0x7f918c00bd70 comp rx=0 tx=0).stop 2024-09-19T08:13:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.405+0000 7f91996eb640 1 -- 172.21.15.22:0/3716220624 shutdown_connections 2024-09-19T08:13:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.405+0000 7f91996eb640 1 --2- 172.21.15.22:0/3716220624 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f91800a32d0 0x7f91800a5790 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.405+0000 7f91996eb640 1 --2- 172.21.15.22:0/3716220624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f919410c170 0x7f9194101420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.405+0000 7f91996eb640 1 --2- 172.21.15.22:0/3716220624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9194103660 0x7f9194100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.405+0000 7f91996eb640 1 --2- 172.21.15.22:0/3716220624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9194102d40 0x7f91941009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.405+0000 7f91996eb640 1 -- 172.21.15.22:0/3716220624 >> 172.21.15.22:0/3716220624 conn(0x7f91940fc7a0 msgr2=0x7f91940fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.405+0000 7f91996eb640 1 -- 172.21.15.22:0/3716220624 shutdown_connections 2024-09-19T08:13:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:22.405+0000 7f91996eb640 1 -- 172.21.15.22:0/3716220624 wait complete. 2024-09-19T08:13:22.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:22 smithi064 ceph-mon[34900]: pgmap v290: 97 pgs: 97 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-19T08:13:22.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:22 smithi157 ceph-mon[35143]: pgmap v290: 97 pgs: 97 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-19T08:13:23.081 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:13:23.338 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:23.460 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:23 smithi022 ceph-mon[30963]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:23.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:23 smithi064 ceph-mon[34900]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:23.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:23 smithi157 ceph-mon[35143]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:13:23.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.825+0000 7f07b142d640 1 -- 172.21.15.22:0/3981719928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ac106a80 msgr2=0x7f07ac106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:23.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.825+0000 7f07b142d640 1 --2- 172.21.15.22:0/3981719928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ac106a80 0x7f07ac106e60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f079c009930 tx=0x7f079c02a420 comp rx=0 tx=0).stop 2024-09-19T08:13:23.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.825+0000 7f07b142d640 1 -- 172.21.15.22:0/3981719928 shutdown_connections 2024-09-19T08:13:23.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.825+0000 7f07b142d640 1 --2- 172.21.15.22:0/3981719928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07ac101420 0x7f07ac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:23.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.825+0000 7f07b142d640 1 --2- 172.21.15.22:0/3981719928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07ac100a80 0x7f07ac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:23.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.825+0000 7f07b142d640 1 --2- 172.21.15.22:0/3981719928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ac106a80 0x7f07ac106e60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:23.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.825+0000 7f07b142d640 1 -- 172.21.15.22:0/3981719928 >> 172.21.15.22:0/3981719928 conn(0x7f07ac0fc7a0 msgr2=0x7f07ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:23.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.825+0000 7f07b142d640 1 -- 172.21.15.22:0/3981719928 shutdown_connections 2024-09-19T08:13:23.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.825+0000 7f07b142d640 1 -- 172.21.15.22:0/3981719928 wait complete. 2024-09-19T08:13:23.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.826+0000 7f07b142d640 1 Processor -- start 2024-09-19T08:13:23.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.826+0000 7f07b142d640 1 -- start start 2024-09-19T08:13:23.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.826+0000 7f07b142d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07ac100a80 0x7f07ac0fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:23.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.826+0000 7f07b142d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07ac101420 0x7f07ac0fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:23.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.826+0000 7f07b142d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ac106a80 0x7f07ac0fd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:23.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.826+0000 7f07b142d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac110e90 con 0x7f07ac106a80 2024-09-19T08:13:23.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.826+0000 7f07b142d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac110d10 con 0x7f07ac101420 2024-09-19T08:13:23.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.826+0000 7f07b142d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac111010 con 0x7f07ac100a80 2024-09-19T08:13:23.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.826+0000 7f07ab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ac106a80 0x7f07ac0fd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:23.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.826+0000 7f07ab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ac106a80 0x7f07ac0fd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51750/0 (socket says 172.21.15.22:51750) 2024-09-19T08:13:23.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.826+0000 7f07ab7fe640 1 -- 172.21.15.22:0/125568501 learned_addr learned my addr 172.21.15.22:0/125568501 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:23.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.827+0000 7f07aaffd640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07ac100a80 0x7f07ac0fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:23.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.827+0000 7f07aa7fc640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07ac101420 0x7f07ac0fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:23.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.827+0000 7f07ab7fe640 1 -- 172.21.15.22:0/125568501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07ac100a80 msgr2=0x7f07ac0fcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:23.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.827+0000 7f07ab7fe640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07ac100a80 0x7f07ac0fcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:23.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.827+0000 7f07ab7fe640 1 -- 172.21.15.22:0/125568501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07ac101420 msgr2=0x7f07ac0fd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.827+0000 7f07ab7fe640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07ac101420 0x7f07ac0fd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.827+0000 7f07ab7fe640 1 -- 172.21.15.22:0/125568501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ac1a68a0 con 0x7f07ac106a80 2024-09-19T08:13:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.827+0000 7f07aaffd640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07ac100a80 0x7f07ac0fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:13:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.827+0000 7f07ab7fe640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ac106a80 0x7f07ac0fd800 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f07a400cce0 tx=0x7f07a400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.827+0000 7f0793fff640 1 -- 172.21.15.22:0/125568501 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a400f450 con 0x7f07ac106a80 2024-09-19T08:13:23.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.827+0000 7f0793fff640 1 -- 172.21.15.22:0/125568501 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07a400fad0 con 0x7f07ac106a80 2024-09-19T08:13:23.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.827+0000 7f07b142d640 1 -- 172.21.15.22:0/125568501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ac1a6b90 con 0x7f07ac106a80 2024-09-19T08:13:23.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.827+0000 7f0793fff640 1 -- 172.21.15.22:0/125568501 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4018c10 con 0x7f07ac106a80 2024-09-19T08:13:23.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.827+0000 7f07b142d640 1 -- 172.21.15.22:0/125568501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ac1a70d0 con 0x7f07ac106a80 2024-09-19T08:13:23.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.828+0000 7f07b142d640 1 -- 172.21.15.22:0/125568501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07ac100ee0 con 0x7f07ac106a80 2024-09-19T08:13:23.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.829+0000 7f0793fff640 1 -- 172.21.15.22:0/125568501 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f07a400f5f0 con 0x7f07ac106a80 2024-09-19T08:13:23.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.830+0000 7f0793fff640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f07940a3350 0x7f07940a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:23.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.830+0000 7f07aaffd640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f07940a3350 0x7f07940a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:23.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.830+0000 7f0793fff640 1 -- 172.21.15.22:0/125568501 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f07a40cff60 con 0x7f07ac106a80 2024-09-19T08:13:23.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.830+0000 7f07aaffd640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f07940a3350 0x7f07940a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f079c006010 tx=0x7f079c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:23.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:23.834+0000 7f0793fff640 1 -- 172.21.15.22:0/125568501 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f07a4014030 con 0x7f07ac106a80 2024-09-19T08:13:24.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:24.046+0000 7f07b142d640 1 -- 172.21.15.22:0/125568501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07ac10a000 con 0x7f07ac106a80 2024-09-19T08:13:24.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:24.046+0000 7f0793fff640 1 -- 172.21.15.22:0/125568501 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11906 (secure 0 0 0) 0x7f07a409ccd0 con 0x7f07ac106a80 2024-09-19T08:13:24.047 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:13:24.047 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"btime":"2024-09-19T08:13:13:877693+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:13:13.764216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:13:24.047 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-09-19T08:13:24.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:24.048+0000 7f07b142d640 1 -- 172.21.15.22:0/125568501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f07940a3350 msgr2=0x7f07940a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:24.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:24.048+0000 7f07b142d640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f07940a3350 0x7f07940a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f079c006010 tx=0x7f079c005a20 comp rx=0 tx=0).stop 2024-09-19T08:13:24.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:24.048+0000 7f07b142d640 1 -- 172.21.15.22:0/125568501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ac106a80 msgr2=0x7f07ac0fd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:24.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:24.048+0000 7f07b142d640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ac106a80 0x7f07ac0fd800 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f07a400cce0 tx=0x7f07a400a620 comp rx=0 tx=0).stop 2024-09-19T08:13:24.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:24.049+0000 7f07b142d640 1 -- 172.21.15.22:0/125568501 shutdown_connections 2024-09-19T08:13:24.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:24.049+0000 7f07b142d640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f07940a3350 0x7f07940a5810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:24.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:24.049+0000 7f07b142d640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07ac106a80 0x7f07ac0fd800 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:24.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:24.049+0000 7f07b142d640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07ac101420 0x7f07ac0fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:24.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:24.049+0000 7f07b142d640 1 --2- 172.21.15.22:0/125568501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07ac100a80 0x7f07ac0fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:24.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:24.049+0000 7f07b142d640 1 -- 172.21.15.22:0/125568501 >> 172.21.15.22:0/125568501 conn(0x7f07ac0fc7a0 msgr2=0x7f07ac06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:24.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:24.049+0000 7f07b142d640 1 -- 172.21.15.22:0/125568501 shutdown_connections 2024-09-19T08:13:24.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:24.049+0000 7f07b142d640 1 -- 172.21.15.22:0/125568501 wait complete. 2024-09-19T08:13:24.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:24 smithi022 ceph-mon[30963]: pgmap v291: 97 pgs: 97 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.0 KiB/s wr, 13 op/s 2024-09-19T08:13:24.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:24 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/125568501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:13:24.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:24 smithi064 ceph-mon[34900]: pgmap v291: 97 pgs: 97 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.0 KiB/s wr, 13 op/s 2024-09-19T08:13:24.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:24 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/125568501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:13:24.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:24 smithi157 ceph-mon[35143]: pgmap v291: 97 pgs: 97 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.0 KiB/s wr, 13 op/s 2024-09-19T08:13:24.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:24 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/125568501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:13:24.835 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:13:25.094 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:25.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.577+0000 7f8ecb60a640 1 -- 172.21.15.22:0/199088385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ec4102d90 msgr2=0x7f8ec4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:25.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.577+0000 7f8ecb60a640 1 --2- 172.21.15.22:0/199088385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ec4102d90 0x7f8ec4103170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009930 tx=0x7f8ec002a420 comp rx=0 tx=0).stop 2024-09-19T08:13:25.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.578+0000 7f8ecb60a640 1 -- 172.21.15.22:0/199088385 shutdown_connections 2024-09-19T08:13:25.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.578+0000 7f8ecb60a640 1 --2- 172.21.15.22:0/199088385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ec4108290 0x7f8ec410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:25.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.578+0000 7f8ecb60a640 1 --2- 172.21.15.22:0/199088385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec4103740 0x7f8ec4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:25.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.578+0000 7f8ecb60a640 1 --2- 172.21.15.22:0/199088385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ec4102d90 0x7f8ec4103170 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:25.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.578+0000 7f8ecb60a640 1 -- 172.21.15.22:0/199088385 >> 172.21.15.22:0/199088385 conn(0x7f8ec40fc7d0 msgr2=0x7f8ec40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:25.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.578+0000 7f8ecb60a640 1 -- 172.21.15.22:0/199088385 shutdown_connections 2024-09-19T08:13:25.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.578+0000 7f8ecb60a640 1 -- 172.21.15.22:0/199088385 wait complete. 2024-09-19T08:13:25.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.578+0000 7f8ecb60a640 1 Processor -- start 2024-09-19T08:13:25.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.578+0000 7f8ecb60a640 1 -- start start 2024-09-19T08:13:25.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.578+0000 7f8ecb60a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec4102d90 0x7f8ec41009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:25.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ecb60a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ec4103740 0x7f8ec4100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:25.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ecb60a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ec4108290 0x7f8ec4101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:25.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ecb60a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec41120d0 con 0x7f8ec4108290 2024-09-19T08:13:25.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ecb60a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec4111f50 con 0x7f8ec4103740 2024-09-19T08:13:25.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ecb60a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec4112250 con 0x7f8ec4102d90 2024-09-19T08:13:25.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ec9ba7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ec4108290 0x7f8ec4101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:25.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ec9ba7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ec4108290 0x7f8ec4101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51764/0 (socket says 172.21.15.22:51764) 2024-09-19T08:13:25.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ec9ba7640 1 -- 172.21.15.22:0/3682374629 learned_addr learned my addr 172.21.15.22:0/3682374629 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:25.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ec93a6640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec4102d90 0x7f8ec41009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:25.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ec8ba5640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ec4103740 0x7f8ec4100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:25.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ec9ba7640 1 -- 172.21.15.22:0/3682374629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec4102d90 msgr2=0x7f8ec41009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:25.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ec9ba7640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec4102d90 0x7f8ec41009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:25.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ec9ba7640 1 -- 172.21.15.22:0/3682374629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ec4103740 msgr2=0x7f8ec4100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:25.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ec9ba7640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ec4103740 0x7f8ec4100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:25.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ec9ba7640 1 -- 172.21.15.22:0/3682374629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec41a6b00 con 0x7f8ec4108290 2024-09-19T08:13:25.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ec8ba5640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ec4103740 0x7f8ec4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:13:25.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ec93a6640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec4102d90 0x7f8ec41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:13:25.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ec9ba7640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ec4108290 0x7f8ec4101470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00cad0 tx=0x7f8ebc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:25.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8eba7fc640 1 -- 172.21.15.22:0/3682374629 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00ae20 con 0x7f8ec4108290 2024-09-19T08:13:25.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8eba7fc640 1 -- 172.21.15.22:0/3682374629 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ebc005020 con 0x7f8ec4108290 2024-09-19T08:13:25.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ecb60a640 1 -- 172.21.15.22:0/3682374629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec41a6df0 con 0x7f8ec4108290 2024-09-19T08:13:25.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8eba7fc640 1 -- 172.21.15.22:0/3682374629 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00f6f0 con 0x7f8ec4108290 2024-09-19T08:13:25.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.579+0000 7f8ecb60a640 1 -- 172.21.15.22:0/3682374629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec41a7300 con 0x7f8ec4108290 2024-09-19T08:13:25.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.580+0000 7f8ecb60a640 1 -- 172.21.15.22:0/3682374629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec406a7f0 con 0x7f8ec4108290 2024-09-19T08:13:25.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.582+0000 7f8eba7fc640 1 -- 172.21.15.22:0/3682374629 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8ebc0051d0 con 0x7f8ec4108290 2024-09-19T08:13:25.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.583+0000 7f8eba7fc640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8eb00a3350 0x7f8eb00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:25.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.583+0000 7f8ec93a6640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8eb00a3350 0x7f8eb00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:25.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.583+0000 7f8eba7fc640 1 -- 172.21.15.22:0/3682374629 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f8ebc0cfe20 con 0x7f8ec4108290 2024-09-19T08:13:25.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.584+0000 7f8ec93a6640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8eb00a3350 0x7f8eb00a5810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0000c00 tx=0x7f8ec0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:25.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.586+0000 7f8eba7fc640 1 -- 172.21.15.22:0/3682374629 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8ebc014030 con 0x7f8ec4108290 2024-09-19T08:13:25.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.761+0000 7f8ecb60a640 1 -- 172.21.15.22:0/3682374629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ec4103170 con 0x7f8ec4108290 2024-09-19T08:13:25.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.762+0000 7f8eba7fc640 1 -- 172.21.15.22:0/3682374629 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22243 (secure 0 0 0) 0x7f8ec4103170 con 0x7f8ec4108290 2024-09-19T08:13:25.763 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:13:25.763 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":83,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","created":"2024-09-19T08:05:19.476952+0000","modified":"2024-09-19T08:13:01.503182+0000","last_up_change":"2024-09-19T08:12:10.339586+0000","last_in_change":"2024-09-19T08:11:47.653576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:08:25.175759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:12:57.608787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:12:59.824327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ed67f2d-9b28-427a-b69f-676e7a2abcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6803","nonce":3442715307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6805","nonce":3442715307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6809","nonce":3442715307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6807","nonce":3442715307}]},"public_addr":"172.21.15.22:6803/3442715307","cluster_addr":"172.21.15.22:6805/3442715307","heartbeat_back_addr":"172.21.15.22:6809/3442715307","heartbeat_front_addr":"172.21.15.22:6807/3442715307","state":["exists","up"]},{"osd":1,"uuid":"bff3e283-4ab1-49d7-86d8-45d361995122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6801","nonce":2379700228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6803","nonce":2379700228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6807","nonce":2379700228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6805","nonce":2379700228}]},"public_addr":"172.21.15.64:6801/2379700228","cluster_addr":"172.21.15.64:6803/2379700228","heartbeat_back_addr":"172.21.15.64:6807/2379700228","heartbeat_front_addr":"172.21.15.64:6805/2379700228","state":["exists","up"]},{"osd":2,"uuid":"81421d08-7dc6-419b-a566-dee1e1e13b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6801","nonce":3363263867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6803","nonce":3363263867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6807","nonce":3363263867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6805","nonce":3363263867}]},"public_addr":"172.21.15.157:6801/3363263867","cluster_addr":"172.21.15.157:6803/3363263867","heartbeat_back_addr":"172.21.15.157:6807/3363263867","heartbeat_front_addr":"172.21.15.157:6805/3363263867","state":["exists","up"]},{"osd":3,"uuid":"34debfd0-3226-4d78-b9e6-e03dbc5190f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6811","nonce":4225229456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6813","nonce":4225229456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6817","nonce":4225229456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6815","nonce":4225229456}]},"public_addr":"172.21.15.22:6811/4225229456","cluster_addr":"172.21.15.22:6813/4225229456","heartbeat_back_addr":"172.21.15.22:6817/4225229456","heartbeat_front_addr":"172.21.15.22:6815/4225229456","state":["exists","up"]},{"osd":4,"uuid":"1293332c-d772-4ca9-bf76-79e0b29451ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6809","nonce":1858090423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6811","nonce":1858090423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6815","nonce":1858090423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6813","nonce":1858090423}]},"public_addr":"172.21.15.64:6809/1858090423","cluster_addr":"172.21.15.64:6811/1858090423","heartbeat_back_addr":"172.21.15.64:6815/1858090423","heartbeat_front_addr":"172.21.15.64:6813/1858090423","state":["exists","up"]},{"osd":5,"uuid":"240a6c42-89a2-41c1-948c-b09914b5722e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6809","nonce":2900701212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2900701212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2900701212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2900701212}]},"public_addr":"172.21.15.157:6809/2900701212","cluster_addr":"172.21.15.157:6811/2900701212","heartbeat_back_addr":"172.21.15.157:6815/2900701212","heartbeat_front_addr":"172.21.15.157:6813/2900701212","state":["exists","up"]},{"osd":6,"uuid":"80199030-109e-41c4-bc1c-731755464770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6819","nonce":4264426242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6821","nonce":4264426242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6825","nonce":4264426242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6823","nonce":4264426242}]},"public_addr":"172.21.15.22:6819/4264426242","cluster_addr":"172.21.15.22:6821/4264426242","heartbeat_back_addr":"172.21.15.22:6825/4264426242","heartbeat_front_addr":"172.21.15.22:6823/4264426242","state":["exists","up"]},{"osd":7,"uuid":"dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6817","nonce":3980224103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6819","nonce":3980224103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6823","nonce":3980224103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6821","nonce":3980224103}]},"public_addr":"172.21.15.64:6817/3980224103","cluster_addr":"172.21.15.64:6819/3980224103","heartbeat_back_addr":"172.21.15.64:6823/3980224103","heartbeat_front_addr":"172.21.15.64:6821/3980224103","state":["exists","up"]},{"osd":8,"uuid":"764b1e6e-48c9-49f9-8124-bc13bcba3a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6817","nonce":973678477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6819","nonce":973678477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6823","nonce":973678477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6821","nonce":973678477}]},"public_addr":"172.21.15.157:6817/973678477","cluster_addr":"172.21.15.157:6819/973678477","heartbeat_back_addr":"172.21.15.157:6823/973678477","heartbeat_front_addr":"172.21.15.157:6821/973678477","state":["exists","up"]},{"osd":9,"uuid":"82914818-df47-4374-9e3e-2f71fc76e78b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6827","nonce":710033990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6829","nonce":710033990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6833","nonce":710033990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6831","nonce":710033990}]},"public_addr":"172.21.15.22:6827/710033990","cluster_addr":"172.21.15.22:6829/710033990","heartbeat_back_addr":"172.21.15.22:6833/710033990","heartbeat_front_addr":"172.21.15.22:6831/710033990","state":["exists","up"]},{"osd":10,"uuid":"36ae177d-c454-4cba-ba21-cbeee0f10dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6825","nonce":373412752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6827","nonce":373412752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6831","nonce":373412752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6829","nonce":373412752}]},"public_addr":"172.21.15.64:6825/373412752","cluster_addr":"172.21.15.64:6827/373412752","heartbeat_back_addr":"172.21.15.64:6831/373412752","heartbeat_front_addr":"172.21.15.64:6829/373412752","state":["exists","up"]},{"osd":11,"uuid":"cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6825","nonce":1918889194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6827","nonce":1918889194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6831","nonce":1918889194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6829","nonce":1918889194}]},"public_addr":"172.21.15.157:6825/1918889194","cluster_addr":"172.21.15.157:6827/1918889194","heartbeat_back_addr":"172.21.15.157:6831/1918889194","heartbeat_front_addr":"172.21.15.157:6829/1918889194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:31.378218+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:56.794167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:20.790267+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:47.039106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:11.483842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:38.151466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:03.553308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:27.865321+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:53.086530+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:18.946661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:43.697243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:12:08.936588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3712426751":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/2133059277":"2024-09-20T08:05:42.354701+0000","172.21.15.22:6801/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2851750481":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/3623845502":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/384027596":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/586163474":"2024-09-20T08:05:42.354701+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:13:25.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.765+0000 7f8ecb60a640 1 -- 172.21.15.22:0/3682374629 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8eb00a3350 msgr2=0x7f8eb00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:25.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.765+0000 7f8ecb60a640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8eb00a3350 0x7f8eb00a5810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0000c00 tx=0x7f8ec0005a20 comp rx=0 tx=0).stop 2024-09-19T08:13:25.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.765+0000 7f8ecb60a640 1 -- 172.21.15.22:0/3682374629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ec4108290 msgr2=0x7f8ec4101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:25.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.765+0000 7f8ecb60a640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ec4108290 0x7f8ec4101470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00cad0 tx=0x7f8ebc00cf90 comp rx=0 tx=0).stop 2024-09-19T08:13:25.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.765+0000 7f8ecb60a640 1 -- 172.21.15.22:0/3682374629 shutdown_connections 2024-09-19T08:13:25.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.765+0000 7f8ecb60a640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f8eb00a3350 0x7f8eb00a5810 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:25.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.765+0000 7f8ecb60a640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ec4108290 0x7f8ec4101470 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:25.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.765+0000 7f8ecb60a640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ec4103740 0x7f8ec4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:25.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.765+0000 7f8ecb60a640 1 --2- 172.21.15.22:0/3682374629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec4102d90 0x7f8ec41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:25.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.765+0000 7f8ecb60a640 1 -- 172.21.15.22:0/3682374629 >> 172.21.15.22:0/3682374629 conn(0x7f8ec40fc7d0 msgr2=0x7f8ec4105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:25.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.765+0000 7f8ecb60a640 1 -- 172.21.15.22:0/3682374629 shutdown_connections 2024-09-19T08:13:25.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:25.765+0000 7f8ecb60a640 1 -- 172.21.15.22:0/3682374629 wait complete. 2024-09-19T08:13:26.469 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:26 smithi022 ceph-mon[30963]: pgmap v292: 97 pgs: 97 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-19T08:13:26.469 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:26 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3682374629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:26.503 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T08:13:26.504 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:13:26.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:26 smithi064 ceph-mon[34900]: pgmap v292: 97 pgs: 97 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-19T08:13:26.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:26 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3682374629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:26.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:26 smithi157 ceph-mon[35143]: pgmap v292: 97 pgs: 97 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-19T08:13:26.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:26 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3682374629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:26.764 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:27.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.257+0000 7f97d8368640 1 -- 172.21.15.22:0/2011149165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97d0101380 msgr2=0x7f97d010dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.257+0000 7f97d8368640 1 --2- 172.21.15.22:0/2011149165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97d0101380 0x7f97d010dfb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009900 tx=0x7f97c002a410 comp rx=0 tx=0).stop 2024-09-19T08:13:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.257+0000 7f97d8368640 1 -- 172.21.15.22:0/2011149165 shutdown_connections 2024-09-19T08:13:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.257+0000 7f97d8368640 1 --2- 172.21.15.22:0/2011149165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97d010e4f0 0x7f97d0110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.257+0000 7f97d8368640 1 --2- 172.21.15.22:0/2011149165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97d0101380 0x7f97d010dfb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.257+0000 7f97d8368640 1 --2- 172.21.15.22:0/2011149165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d01009d0 0x7f97d0100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.257+0000 7f97d8368640 1 -- 172.21.15.22:0/2011149165 >> 172.21.15.22:0/2011149165 conn(0x7f97d00fc7d0 msgr2=0x7f97d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.257+0000 7f97d8368640 1 -- 172.21.15.22:0/2011149165 shutdown_connections 2024-09-19T08:13:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.257+0000 7f97d8368640 1 -- 172.21.15.22:0/2011149165 wait complete. 2024-09-19T08:13:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.258+0000 7f97d8368640 1 Processor -- start 2024-09-19T08:13:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.258+0000 7f97d8368640 1 -- start start 2024-09-19T08:13:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.258+0000 7f97d8368640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d01009d0 0x7f97d01a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.258+0000 7f97d8368640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97d0101380 0x7f97d01a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.258+0000 7f97d8368640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97d010e4f0 0x7f97d019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.258+0000 7f97d8368640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f97d0114240 con 0x7f97d0101380 2024-09-19T08:13:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.258+0000 7f97d8368640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f97d01140c0 con 0x7f97d010e4f0 2024-09-19T08:13:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.258+0000 7f97d8368640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f97d01143c0 con 0x7f97d01009d0 2024-09-19T08:13:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97d5903640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97d0101380 0x7f97d01a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97d5903640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97d0101380 0x7f97d01a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51772/0 (socket says 172.21.15.22:51772) 2024-09-19T08:13:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97d5903640 1 -- 172.21.15.22:0/2724008092 learned_addr learned my addr 172.21.15.22:0/2724008092 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97d6104640 1 --2- 172.21.15.22:0/2724008092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d01009d0 0x7f97d01a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97d6905640 1 --2- 172.21.15.22:0/2724008092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97d010e4f0 0x7f97d019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97d5903640 1 -- 172.21.15.22:0/2724008092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d01009d0 msgr2=0x7f97d01a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97d5903640 1 --2- 172.21.15.22:0/2724008092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d01009d0 0x7f97d01a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97d5903640 1 -- 172.21.15.22:0/2724008092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97d010e4f0 msgr2=0x7f97d019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97d5903640 1 --2- 172.21.15.22:0/2724008092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97d010e4f0 0x7f97d019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97d5903640 1 -- 172.21.15.22:0/2724008092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d019ace0 con 0x7f97d0101380 2024-09-19T08:13:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97d5903640 1 --2- 172.21.15.22:0/2724008092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97d0101380 0x7f97d01a0930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f97c0006010 tx=0x7f97c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97c77fe640 1 -- 172.21.15.22:0/2724008092 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c003f030 con 0x7f97d0101380 2024-09-19T08:13:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97c77fe640 1 -- 172.21.15.22:0/2724008092 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97c0003ce0 con 0x7f97d0101380 2024-09-19T08:13:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97d8368640 1 -- 172.21.15.22:0/2724008092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d019af70 con 0x7f97d0101380 2024-09-19T08:13:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.259+0000 7f97d8368640 1 -- 172.21.15.22:0/2724008092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d01a72c0 con 0x7f97d0101380 2024-09-19T08:13:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.260+0000 7f97c77fe640 1 -- 172.21.15.22:0/2724008092 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0035070 con 0x7f97d0101380 2024-09-19T08:13:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.261+0000 7f97d8368640 1 -- 172.21.15.22:0/2724008092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d0100db0 con 0x7f97d0101380 2024-09-19T08:13:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.262+0000 7f97c77fe640 1 -- 172.21.15.22:0/2724008092 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f97c0042030 con 0x7f97d0101380 2024-09-19T08:13:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.262+0000 7f97c77fe640 1 --2- 172.21.15.22:0/2724008092 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f97bc0a32d0 0x7f97bc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.262+0000 7f97d6104640 1 --2- 172.21.15.22:0/2724008092 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f97bc0a32d0 0x7f97bc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.262+0000 7f97c77fe640 1 -- 172.21.15.22:0/2724008092 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f97c00eff00 con 0x7f97d0101380 2024-09-19T08:13:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.263+0000 7f97d6104640 1 --2- 172.21.15.22:0/2724008092 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f97bc0a32d0 0x7f97bc0a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00bbe0 tx=0x7f97cc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.267+0000 7f97c77fe640 1 -- 172.21.15.22:0/2724008092 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f97c00bc6e0 con 0x7f97d0101380 2024-09-19T08:13:27.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.444+0000 7f97d8368640 1 -- 172.21.15.22:0/2724008092 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f97d0062630 con 0x7f97bc0a32d0 2024-09-19T08:13:27.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.451+0000 7f97c77fe640 1 -- 172.21.15.22:0/2724008092 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258843 (secure 0 0 0) 0x7f97d0062630 con 0x7f97bc0a32d0 2024-09-19T08:13:27.451 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:13:27.456 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T08:13:27.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.454+0000 7f97d8368640 1 -- 172.21.15.22:0/2724008092 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f97bc0a32d0 msgr2=0x7f97bc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:27.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.454+0000 7f97d8368640 1 --2- 172.21.15.22:0/2724008092 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f97bc0a32d0 0x7f97bc0a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00bbe0 tx=0x7f97cc00b390 comp rx=0 tx=0).stop 2024-09-19T08:13:27.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.454+0000 7f97d8368640 1 -- 172.21.15.22:0/2724008092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97d0101380 msgr2=0x7f97d01a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:27.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.454+0000 7f97d8368640 1 --2- 172.21.15.22:0/2724008092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97d0101380 0x7f97d01a0930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f97c0006010 tx=0x7f97c00042f0 comp rx=0 tx=0).stop 2024-09-19T08:13:27.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.454+0000 7f97d8368640 1 -- 172.21.15.22:0/2724008092 shutdown_connections 2024-09-19T08:13:27.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.454+0000 7f97d8368640 1 --2- 172.21.15.22:0/2724008092 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f97bc0a32d0 0x7f97bc0a5790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:27.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.454+0000 7f97d8368640 1 --2- 172.21.15.22:0/2724008092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97d010e4f0 0x7f97d019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:27.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.454+0000 7f97d8368640 1 --2- 172.21.15.22:0/2724008092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97d0101380 0x7f97d01a0930 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:27.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.454+0000 7f97d8368640 1 --2- 172.21.15.22:0/2724008092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d01009d0 0x7f97d01a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:27.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.454+0000 7f97d8368640 1 -- 172.21.15.22:0/2724008092 >> 172.21.15.22:0/2724008092 conn(0x7f97d00fc7d0 msgr2=0x7f97d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:27.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.455+0000 7f97d8368640 1 -- 172.21.15.22:0/2724008092 shutdown_connections 2024-09-19T08:13:27.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:27.455+0000 7f97d8368640 1 -- 172.21.15.22:0/2724008092 wait complete. 2024-09-19T08:13:28.209 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-19T08:13:26.768563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641800,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":245,"num_read_kb":229,"num_write":330,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381004,"kb_used_data":10956,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560820,"statfs":{"total":1151940427776,"available":1151550279680,"internally_reserved":0,"allocated":11218944,"data_stored":7061724,"data_compressed":109176,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46300,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":97,"num_read_kb":101,"num_write":38,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002944"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948678+0000","last_change":"2024-09-19T08:13:00.853947+0000","last_active":"2024-09-19T08:13:01.948678+0000","last_peered":"2024-09-19T08:13:01.948678+0000","last_clean":"2024-09-19T08:13:01.948678+0000","last_became_active":"2024-09-19T08:13:00.852951+0000","last_became_peered":"2024-09-19T08:13:00.852951+0000","last_unstale":"2024-09-19T08:13:01.948678+0000","last_undegraded":"2024-09-19T08:13:01.948678+0000","last_fullsized":"2024-09-19T08:13:01.948678+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:25.786828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948757+0000","last_change":"2024-09-19T08:13:00.853389+0000","last_active":"2024-09-19T08:13:01.948757+0000","last_peered":"2024-09-19T08:13:01.948757+0000","last_clean":"2024-09-19T08:13:01.948757+0000","last_became_active":"2024-09-19T08:13:00.853102+0000","last_became_peered":"2024-09-19T08:13:00.853102+0000","last_unstale":"2024-09-19T08:13:01.948757+0000","last_undegraded":"2024-09-19T08:13:01.948757+0000","last_fullsized":"2024-09-19T08:13:01.948757+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:42.773917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948582+0000","last_change":"2024-09-19T08:13:00.846544+0000","last_active":"2024-09-19T08:13:01.948582+0000","last_peered":"2024-09-19T08:13:01.948582+0000","last_clean":"2024-09-19T08:13:01.948582+0000","last_became_active":"2024-09-19T08:13:00.846229+0000","last_became_peered":"2024-09-19T08:13:00.846229+0000","last_unstale":"2024-09-19T08:13:01.948582+0000","last_undegraded":"2024-09-19T08:13:01.948582+0000","last_fullsized":"2024-09-19T08:13:01.948582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:43.471684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948555+0000","last_change":"2024-09-19T08:13:00.848149+0000","last_active":"2024-09-19T08:13:01.948555+0000","last_peered":"2024-09-19T08:13:01.948555+0000","last_clean":"2024-09-19T08:13:01.948555+0000","last_became_active":"2024-09-19T08:13:00.847744+0000","last_became_peered":"2024-09-19T08:13:00.847744+0000","last_unstale":"2024-09-19T08:13:01.948555+0000","last_undegraded":"2024-09-19T08:13:01.948555+0000","last_fullsized":"2024-09-19T08:13:01.948555+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:01.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948347+0000","last_change":"2024-09-19T08:13:00.841862+0000","last_active":"2024-09-19T08:13:01.948347+0000","last_peered":"2024-09-19T08:13:01.948347+0000","last_clean":"2024-09-19T08:13:01.948347+0000","last_became_active":"2024-09-19T08:13:00.841331+0000","last_became_peered":"2024-09-19T08:13:00.841331+0000","last_unstale":"2024-09-19T08:13:01.948347+0000","last_undegraded":"2024-09-19T08:13:01.948347+0000","last_fullsized":"2024-09-19T08:13:01.948347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:54.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947492+0000","last_change":"2024-09-19T08:13:00.849626+0000","last_active":"2024-09-19T08:13:01.947492+0000","last_peered":"2024-09-19T08:13:01.947492+0000","last_clean":"2024-09-19T08:13:01.947492+0000","last_became_active":"2024-09-19T08:13:00.849034+0000","last_became_peered":"2024-09-19T08:13:00.849034+0000","last_unstale":"2024-09-19T08:13:01.947492+0000","last_undegraded":"2024-09-19T08:13:01.947492+0000","last_fullsized":"2024-09-19T08:13:01.947492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:39.118978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100394+0000","last_change":"2024-09-19T08:13:00.847522+0000","last_active":"2024-09-19T08:13:02.100394+0000","last_peered":"2024-09-19T08:13:02.100394+0000","last_clean":"2024-09-19T08:13:02.100394+0000","last_became_active":"2024-09-19T08:13:00.847117+0000","last_became_peered":"2024-09-19T08:13:00.847117+0000","last_unstale":"2024-09-19T08:13:02.100394+0000","last_undegraded":"2024-09-19T08:13:02.100394+0000","last_fullsized":"2024-09-19T08:13:02.100394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.836385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947191+0000","last_change":"2024-09-19T08:13:00.849544+0000","last_active":"2024-09-19T08:13:01.947191+0000","last_peered":"2024-09-19T08:13:01.947191+0000","last_clean":"2024-09-19T08:13:01.947191+0000","last_became_active":"2024-09-19T08:13:00.849170+0000","last_became_peered":"2024-09-19T08:13:00.849170+0000","last_unstale":"2024-09-19T08:13:01.947191+0000","last_undegraded":"2024-09-19T08:13:01.947191+0000","last_fullsized":"2024-09-19T08:13:01.947191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:28.371965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836582+0000","last_change":"2024-09-19T08:13:00.856052+0000","last_active":"2024-09-19T08:13:01.836582+0000","last_peered":"2024-09-19T08:13:01.836582+0000","last_clean":"2024-09-19T08:13:01.836582+0000","last_became_active":"2024-09-19T08:13:00.855291+0000","last_became_peered":"2024-09-19T08:13:00.855291+0000","last_unstale":"2024-09-19T08:13:01.836582+0000","last_undegraded":"2024-09-19T08:13:01.836582+0000","last_fullsized":"2024-09-19T08:13:01.836582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:18.581867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948289+0000","last_change":"2024-09-19T08:13:00.850145+0000","last_active":"2024-09-19T08:13:01.948289+0000","last_peered":"2024-09-19T08:13:01.948289+0000","last_clean":"2024-09-19T08:13:01.948289+0000","last_became_active":"2024-09-19T08:13:00.849798+0000","last_became_peered":"2024-09-19T08:13:00.849798+0000","last_unstale":"2024-09-19T08:13:01.948289+0000","last_undegraded":"2024-09-19T08:13:01.948289+0000","last_fullsized":"2024-09-19T08:13:01.948289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:21.400000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100790+0000","last_change":"2024-09-19T08:13:00.859153+0000","last_active":"2024-09-19T08:13:02.100790+0000","last_peered":"2024-09-19T08:13:02.100790+0000","last_clean":"2024-09-19T08:13:02.100790+0000","last_became_active":"2024-09-19T08:13:00.858473+0000","last_became_peered":"2024-09-19T08:13:00.858473+0000","last_unstale":"2024-09-19T08:13:02.100790+0000","last_undegraded":"2024-09-19T08:13:02.100790+0000","last_fullsized":"2024-09-19T08:13:02.100790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:07.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948660+0000","last_change":"2024-09-19T08:13:00.850797+0000","last_active":"2024-09-19T08:13:01.948660+0000","last_peered":"2024-09-19T08:13:01.948660+0000","last_clean":"2024-09-19T08:13:01.948660+0000","last_became_active":"2024-09-19T08:13:00.850393+0000","last_became_peered":"2024-09-19T08:13:00.850393+0000","last_unstale":"2024-09-19T08:13:01.948660+0000","last_undegraded":"2024-09-19T08:13:01.948660+0000","last_fullsized":"2024-09-19T08:13:01.948660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:59.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947687+0000","last_change":"2024-09-19T08:13:00.850835+0000","last_active":"2024-09-19T08:13:01.947687+0000","last_peered":"2024-09-19T08:13:01.947687+0000","last_clean":"2024-09-19T08:13:01.947687+0000","last_became_active":"2024-09-19T08:13:00.850518+0000","last_became_peered":"2024-09-19T08:13:00.850518+0000","last_unstale":"2024-09-19T08:13:01.947687+0000","last_undegraded":"2024-09-19T08:13:01.947687+0000","last_fullsized":"2024-09-19T08:13:01.947687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.178530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948530+0000","last_change":"2024-09-19T08:13:00.853934+0000","last_active":"2024-09-19T08:13:01.948530+0000","last_peered":"2024-09-19T08:13:01.948530+0000","last_clean":"2024-09-19T08:13:01.948530+0000","last_became_active":"2024-09-19T08:13:00.853604+0000","last_became_peered":"2024-09-19T08:13:00.853604+0000","last_unstale":"2024-09-19T08:13:01.948530+0000","last_undegraded":"2024-09-19T08:13:01.948530+0000","last_fullsized":"2024-09-19T08:13:01.948530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:43.622526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948350+0000","last_change":"2024-09-19T08:13:00.846026+0000","last_active":"2024-09-19T08:13:01.948350+0000","last_peered":"2024-09-19T08:13:01.948350+0000","last_clean":"2024-09-19T08:13:01.948350+0000","last_became_active":"2024-09-19T08:13:00.845691+0000","last_became_peered":"2024-09-19T08:13:00.845691+0000","last_unstale":"2024-09-19T08:13:01.948350+0000","last_undegraded":"2024-09-19T08:13:01.948350+0000","last_fullsized":"2024-09-19T08:13:01.948350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:55.724159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836019+0000","last_change":"2024-09-19T08:13:00.850266+0000","last_active":"2024-09-19T08:13:01.836019+0000","last_peered":"2024-09-19T08:13:01.836019+0000","last_clean":"2024-09-19T08:13:01.836019+0000","last_became_active":"2024-09-19T08:13:00.849928+0000","last_became_peered":"2024-09-19T08:13:00.849928+0000","last_unstale":"2024-09-19T08:13:01.836019+0000","last_undegraded":"2024-09-19T08:13:01.836019+0000","last_fullsized":"2024-09-19T08:13:01.836019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:38.231331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948706+0000","last_change":"2024-09-19T08:13:00.852551+0000","last_active":"2024-09-19T08:13:01.948706+0000","last_peered":"2024-09-19T08:13:01.948706+0000","last_clean":"2024-09-19T08:13:01.948706+0000","last_became_active":"2024-09-19T08:13:00.852225+0000","last_became_peered":"2024-09-19T08:13:00.852225+0000","last_unstale":"2024-09-19T08:13:01.948706+0000","last_undegraded":"2024-09-19T08:13:01.948706+0000","last_fullsized":"2024-09-19T08:13:01.948706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:33.115108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.831882+0000","last_change":"2024-09-19T08:13:00.854033+0000","last_active":"2024-09-19T08:13:01.831882+0000","last_peered":"2024-09-19T08:13:01.831882+0000","last_clean":"2024-09-19T08:13:01.831882+0000","last_became_active":"2024-09-19T08:13:00.853660+0000","last_became_peered":"2024-09-19T08:13:00.853660+0000","last_unstale":"2024-09-19T08:13:01.831882+0000","last_undegraded":"2024-09-19T08:13:01.831882+0000","last_fullsized":"2024-09-19T08:13:01.831882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:35.602420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.832908+0000","last_change":"2024-09-19T08:13:00.850658+0000","last_active":"2024-09-19T08:13:01.832908+0000","last_peered":"2024-09-19T08:13:01.832908+0000","last_clean":"2024-09-19T08:13:01.832908+0000","last_became_active":"2024-09-19T08:13:00.850324+0000","last_became_peered":"2024-09-19T08:13:00.850324+0000","last_unstale":"2024-09-19T08:13:01.832908+0000","last_undegraded":"2024-09-19T08:13:01.832908+0000","last_fullsized":"2024-09-19T08:13:01.832908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:59:49.004605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947899+0000","last_change":"2024-09-19T08:13:00.843421+0000","last_active":"2024-09-19T08:13:01.947899+0000","last_peered":"2024-09-19T08:13:01.947899+0000","last_clean":"2024-09-19T08:13:01.947899+0000","last_became_active":"2024-09-19T08:13:00.843109+0000","last_became_peered":"2024-09-19T08:13:00.843109+0000","last_unstale":"2024-09-19T08:13:01.947899+0000","last_undegraded":"2024-09-19T08:13:01.947899+0000","last_fullsized":"2024-09-19T08:13:01.947899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:50.522686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947846+0000","last_change":"2024-09-19T08:13:00.855215+0000","last_active":"2024-09-19T08:13:01.947846+0000","last_peered":"2024-09-19T08:13:01.947846+0000","last_clean":"2024-09-19T08:13:01.947846+0000","last_became_active":"2024-09-19T08:13:00.854878+0000","last_became_peered":"2024-09-19T08:13:00.854878+0000","last_unstale":"2024-09-19T08:13:01.947846+0000","last_undegraded":"2024-09-19T08:13:01.947846+0000","last_fullsized":"2024-09-19T08:13:01.947846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:48.216066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836201+0000","last_change":"2024-09-19T08:13:00.856995+0000","last_active":"2024-09-19T08:13:01.836201+0000","last_peered":"2024-09-19T08:13:01.836201+0000","last_clean":"2024-09-19T08:13:01.836201+0000","last_became_active":"2024-09-19T08:13:00.856645+0000","last_became_peered":"2024-09-19T08:13:00.856645+0000","last_unstale":"2024-09-19T08:13:01.836201+0000","last_undegraded":"2024-09-19T08:13:01.836201+0000","last_fullsized":"2024-09-19T08:13:01.836201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:20.699293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947354+0000","last_change":"2024-09-19T08:13:00.853362+0000","last_active":"2024-09-19T08:13:01.947354+0000","last_peered":"2024-09-19T08:13:01.947354+0000","last_clean":"2024-09-19T08:13:01.947354+0000","last_became_active":"2024-09-19T08:13:00.853046+0000","last_became_peered":"2024-09-19T08:13:00.853046+0000","last_unstale":"2024-09-19T08:13:01.947354+0000","last_undegraded":"2024-09-19T08:13:01.947354+0000","last_fullsized":"2024-09-19T08:13:01.947354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947080+0000","last_change":"2024-09-19T08:13:00.845175+0000","last_active":"2024-09-19T08:13:01.947080+0000","last_peered":"2024-09-19T08:13:01.947080+0000","last_clean":"2024-09-19T08:13:01.947080+0000","last_became_active":"2024-09-19T08:13:00.844857+0000","last_became_peered":"2024-09-19T08:13:00.844857+0000","last_unstale":"2024-09-19T08:13:01.947080+0000","last_undegraded":"2024-09-19T08:13:01.947080+0000","last_fullsized":"2024-09-19T08:13:01.947080+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:40.818956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947507+0000","last_change":"2024-09-19T08:13:00.847769+0000","last_active":"2024-09-19T08:13:01.947507+0000","last_peered":"2024-09-19T08:13:01.947507+0000","last_clean":"2024-09-19T08:13:01.947507+0000","last_became_active":"2024-09-19T08:13:00.847454+0000","last_became_peered":"2024-09-19T08:13:00.847454+0000","last_unstale":"2024-09-19T08:13:01.947507+0000","last_undegraded":"2024-09-19T08:13:01.947507+0000","last_fullsized":"2024-09-19T08:13:01.947507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.583051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.833005+0000","last_change":"2024-09-19T08:13:00.859882+0000","last_active":"2024-09-19T08:13:01.833005+0000","last_peered":"2024-09-19T08:13:01.833005+0000","last_clean":"2024-09-19T08:13:01.833005+0000","last_became_active":"2024-09-19T08:13:00.859491+0000","last_became_peered":"2024-09-19T08:13:00.859491+0000","last_unstale":"2024-09-19T08:13:01.833005+0000","last_undegraded":"2024-09-19T08:13:01.833005+0000","last_fullsized":"2024-09-19T08:13:01.833005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:13.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948381+0000","last_change":"2024-09-19T08:13:00.845584+0000","last_active":"2024-09-19T08:13:01.948381+0000","last_peered":"2024-09-19T08:13:01.948381+0000","last_clean":"2024-09-19T08:13:01.948381+0000","last_became_active":"2024-09-19T08:13:00.845166+0000","last_became_peered":"2024-09-19T08:13:00.845166+0000","last_unstale":"2024-09-19T08:13:01.948381+0000","last_undegraded":"2024-09-19T08:13:01.948381+0000","last_fullsized":"2024-09-19T08:13:01.948381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:02.705793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947956+0000","last_change":"2024-09-19T08:13:00.838069+0000","last_active":"2024-09-19T08:13:01.947956+0000","last_peered":"2024-09-19T08:13:01.947956+0000","last_clean":"2024-09-19T08:13:01.947956+0000","last_became_active":"2024-09-19T08:13:00.837736+0000","last_became_peered":"2024-09-19T08:13:00.837736+0000","last_unstale":"2024-09-19T08:13:01.947956+0000","last_undegraded":"2024-09-19T08:13:01.947956+0000","last_fullsized":"2024-09-19T08:13:01.947956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:01.388290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948680+0000","last_change":"2024-09-19T08:13:00.841507+0000","last_active":"2024-09-19T08:13:01.948680+0000","last_peered":"2024-09-19T08:13:01.948680+0000","last_clean":"2024-09-19T08:13:01.948680+0000","last_became_active":"2024-09-19T08:13:00.841161+0000","last_became_peered":"2024-09-19T08:13:00.841161+0000","last_unstale":"2024-09-19T08:13:01.948680+0000","last_undegraded":"2024-09-19T08:13:01.948680+0000","last_fullsized":"2024-09-19T08:13:01.948680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:41.862290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.832504+0000","last_change":"2024-09-19T08:13:00.847657+0000","last_active":"2024-09-19T08:13:01.832504+0000","last_peered":"2024-09-19T08:13:01.832504+0000","last_clean":"2024-09-19T08:13:01.832504+0000","last_became_active":"2024-09-19T08:13:00.846849+0000","last_became_peered":"2024-09-19T08:13:00.846849+0000","last_unstale":"2024-09-19T08:13:01.832504+0000","last_undegraded":"2024-09-19T08:13:01.832504+0000","last_fullsized":"2024-09-19T08:13:01.832504+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:35.219913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.832650+0000","last_change":"2024-09-19T08:13:00.845971+0000","last_active":"2024-09-19T08:13:01.832650+0000","last_peered":"2024-09-19T08:13:01.832650+0000","last_clean":"2024-09-19T08:13:01.832650+0000","last_became_active":"2024-09-19T08:13:00.845626+0000","last_became_peered":"2024-09-19T08:13:00.845626+0000","last_unstale":"2024-09-19T08:13:01.832650+0000","last_undegraded":"2024-09-19T08:13:01.832650+0000","last_fullsized":"2024-09-19T08:13:01.832650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:36.945627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947938+0000","last_change":"2024-09-19T08:13:00.853851+0000","last_active":"2024-09-19T08:13:01.947938+0000","last_peered":"2024-09-19T08:13:01.947938+0000","last_clean":"2024-09-19T08:13:01.947938+0000","last_became_active":"2024-09-19T08:13:00.853189+0000","last_became_peered":"2024-09-19T08:13:00.853189+0000","last_unstale":"2024-09-19T08:13:01.947938+0000","last_undegraded":"2024-09-19T08:13:01.947938+0000","last_fullsized":"2024-09-19T08:13:01.947938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:04.812522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.946963+0000","last_change":"2024-09-19T08:13:00.848325+0000","last_active":"2024-09-19T08:13:01.946963+0000","last_peered":"2024-09-19T08:13:01.946963+0000","last_clean":"2024-09-19T08:13:01.946963+0000","last_became_active":"2024-09-19T08:13:00.848015+0000","last_became_peered":"2024-09-19T08:13:00.848015+0000","last_unstale":"2024-09-19T08:13:01.946963+0000","last_undegraded":"2024-09-19T08:13:01.946963+0000","last_fullsized":"2024-09-19T08:13:01.946963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:10.948451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:23.901131+0000","last_change":"2024-09-19T08:12:58.681134+0000","last_active":"2024-09-19T08:13:23.901131+0000","last_peered":"2024-09-19T08:13:23.901131+0000","last_clean":"2024-09-19T08:13:23.901131+0000","last_became_active":"2024-09-19T08:12:58.680764+0000","last_became_peered":"2024-09-19T08:12:58.680764+0000","last_unstale":"2024-09-19T08:13:23.901131+0000","last_undegraded":"2024-09-19T08:13:23.901131+0000","last_fullsized":"2024-09-19T08:13:23.901131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:24.939263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947299+0000","last_change":"2024-09-19T08:13:00.844319+0000","last_active":"2024-09-19T08:13:01.947299+0000","last_peered":"2024-09-19T08:13:01.947299+0000","last_clean":"2024-09-19T08:13:01.947299+0000","last_became_active":"2024-09-19T08:13:00.843994+0000","last_became_peered":"2024-09-19T08:13:00.843994+0000","last_unstale":"2024-09-19T08:13:01.947299+0000","last_undegraded":"2024-09-19T08:13:01.947299+0000","last_fullsized":"2024-09-19T08:13:01.947299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:46.937466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.880088+0000","last_change":"2024-09-19T08:12:58.676541+0000","last_active":"2024-09-19T08:13:10.880088+0000","last_peered":"2024-09-19T08:13:10.880088+0000","last_clean":"2024-09-19T08:13:10.880088+0000","last_became_active":"2024-09-19T08:12:58.675988+0000","last_became_peered":"2024-09-19T08:12:58.675988+0000","last_unstale":"2024-09-19T08:13:10.880088+0000","last_undegraded":"2024-09-19T08:13:10.880088+0000","last_fullsized":"2024-09-19T08:13:10.880088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:01.434986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.880054+0000","last_change":"2024-09-19T08:12:58.677510+0000","last_active":"2024-09-19T08:13:10.880054+0000","last_peered":"2024-09-19T08:13:10.880054+0000","last_clean":"2024-09-19T08:13:10.880054+0000","last_became_active":"2024-09-19T08:12:58.677200+0000","last_became_peered":"2024-09-19T08:12:58.677200+0000","last_unstale":"2024-09-19T08:13:10.880054+0000","last_undegraded":"2024-09-19T08:13:10.880054+0000","last_fullsized":"2024-09-19T08:13:10.880054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:36.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947353+0000","last_change":"2024-09-19T08:13:00.853430+0000","last_active":"2024-09-19T08:13:01.947353+0000","last_peered":"2024-09-19T08:13:01.947353+0000","last_clean":"2024-09-19T08:13:01.947353+0000","last_became_active":"2024-09-19T08:13:00.853096+0000","last_became_peered":"2024-09-19T08:13:00.853096+0000","last_unstale":"2024-09-19T08:13:01.947353+0000","last_undegraded":"2024-09-19T08:13:01.947353+0000","last_fullsized":"2024-09-19T08:13:01.947353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:07.147021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947748+0000","last_change":"2024-09-19T08:12:58.680081+0000","last_active":"2024-09-19T08:13:01.947748+0000","last_peered":"2024-09-19T08:13:01.947748+0000","last_clean":"2024-09-19T08:13:01.947748+0000","last_became_active":"2024-09-19T08:12:58.679704+0000","last_became_peered":"2024-09-19T08:12:58.679704+0000","last_unstale":"2024-09-19T08:13:01.947748+0000","last_undegraded":"2024-09-19T08:13:01.947748+0000","last_fullsized":"2024-09-19T08:13:01.947748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:32.942586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.831528+0000","last_change":"2024-09-19T08:13:00.849394+0000","last_active":"2024-09-19T08:13:01.831528+0000","last_peered":"2024-09-19T08:13:01.831528+0000","last_clean":"2024-09-19T08:13:01.831528+0000","last_became_active":"2024-09-19T08:13:00.849051+0000","last_became_peered":"2024-09-19T08:13:00.849051+0000","last_unstale":"2024-09-19T08:13:01.831528+0000","last_undegraded":"2024-09-19T08:13:01.831528+0000","last_fullsized":"2024-09-19T08:13:01.831528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:23.012928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:23.900485+0000","last_change":"2024-09-19T08:12:58.676121+0000","last_active":"2024-09-19T08:13:23.900485+0000","last_peered":"2024-09-19T08:13:23.900485+0000","last_clean":"2024-09-19T08:13:23.900485+0000","last_became_active":"2024-09-19T08:12:58.675693+0000","last_became_peered":"2024-09-19T08:12:58.675693+0000","last_unstale":"2024-09-19T08:13:23.900485+0000","last_undegraded":"2024-09-19T08:13:23.900485+0000","last_fullsized":"2024-09-19T08:13:23.900485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:34.882190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836478+0000","last_change":"2024-09-19T08:13:00.858735+0000","last_active":"2024-09-19T08:13:01.836478+0000","last_peered":"2024-09-19T08:13:01.836478+0000","last_clean":"2024-09-19T08:13:01.836478+0000","last_became_active":"2024-09-19T08:13:00.858218+0000","last_became_peered":"2024-09-19T08:13:00.858218+0000","last_unstale":"2024-09-19T08:13:01.836478+0000","last_undegraded":"2024-09-19T08:13:01.836478+0000","last_fullsized":"2024-09-19T08:13:01.836478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:33.356810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872605+0000","last_change":"2024-09-19T08:12:58.679624+0000","last_active":"2024-09-19T08:13:09.872605+0000","last_peered":"2024-09-19T08:13:09.872605+0000","last_clean":"2024-09-19T08:13:09.872605+0000","last_became_active":"2024-09-19T08:12:58.679057+0000","last_became_peered":"2024-09-19T08:12:58.679057+0000","last_unstale":"2024-09-19T08:13:09.872605+0000","last_undegraded":"2024-09-19T08:13:09.872605+0000","last_fullsized":"2024-09-19T08:13:09.872605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:12.391663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948868+0000","last_change":"2024-09-19T08:13:00.843852+0000","last_active":"2024-09-19T08:13:01.948868+0000","last_peered":"2024-09-19T08:13:01.948868+0000","last_clean":"2024-09-19T08:13:01.948868+0000","last_became_active":"2024-09-19T08:13:00.843497+0000","last_became_peered":"2024-09-19T08:13:00.843497+0000","last_unstale":"2024-09-19T08:13:01.948868+0000","last_undegraded":"2024-09-19T08:13:01.948868+0000","last_fullsized":"2024-09-19T08:13:01.948868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:26.380024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.873411+0000","last_change":"2024-09-19T08:12:58.717225+0000","last_active":"2024-09-19T08:13:09.873411+0000","last_peered":"2024-09-19T08:13:09.873411+0000","last_clean":"2024-09-19T08:13:09.873411+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:13:09.873411+0000","last_undegraded":"2024-09-19T08:13:09.873411+0000","last_fullsized":"2024-09-19T08:13:09.873411+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:43.201907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836422+0000","last_change":"2024-09-19T08:13:00.845613+0000","last_active":"2024-09-19T08:13:01.836422+0000","last_peered":"2024-09-19T08:13:01.836422+0000","last_clean":"2024-09-19T08:13:01.836422+0000","last_became_active":"2024-09-19T08:13:00.845260+0000","last_became_peered":"2024-09-19T08:13:00.845260+0000","last_unstale":"2024-09-19T08:13:01.836422+0000","last_undegraded":"2024-09-19T08:13:01.836422+0000","last_fullsized":"2024-09-19T08:13:01.836422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:03:47.033972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'22","reported_seq":74,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:24.903105+0000","last_change":"2024-09-19T08:12:58.677308+0000","last_active":"2024-09-19T08:13:24.903105+0000","last_peered":"2024-09-19T08:13:24.903105+0000","last_clean":"2024-09-19T08:13:24.903105+0000","last_became_active":"2024-09-19T08:12:58.676925+0000","last_became_peered":"2024-09-19T08:12:58.676925+0000","last_unstale":"2024-09-19T08:13:24.903105+0000","last_undegraded":"2024-09-19T08:13:24.903105+0000","last_fullsized":"2024-09-19T08:13:24.903105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:50.873656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":62,"num_write":23,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947397+0000","last_change":"2024-09-19T08:13:00.855603+0000","last_active":"2024-09-19T08:13:01.947397+0000","last_peered":"2024-09-19T08:13:01.947397+0000","last_clean":"2024-09-19T08:13:01.947397+0000","last_became_active":"2024-09-19T08:13:00.855241+0000","last_became_peered":"2024-09-19T08:13:00.855241+0000","last_unstale":"2024-09-19T08:13:01.947397+0000","last_undegraded":"2024-09-19T08:13:01.947397+0000","last_fullsized":"2024-09-19T08:13:01.947397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:28.811847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.881619+0000","last_change":"2024-09-19T08:12:58.675897+0000","last_active":"2024-09-19T08:13:10.881619+0000","last_peered":"2024-09-19T08:13:10.881619+0000","last_clean":"2024-09-19T08:13:10.881619+0000","last_became_active":"2024-09-19T08:12:58.675525+0000","last_became_peered":"2024-09-19T08:12:58.675525+0000","last_unstale":"2024-09-19T08:13:10.881619+0000","last_undegraded":"2024-09-19T08:13:10.881619+0000","last_fullsized":"2024-09-19T08:13:10.881619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:54.518064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947169+0000","last_change":"2024-09-19T08:13:00.856420+0000","last_active":"2024-09-19T08:13:01.947169+0000","last_peered":"2024-09-19T08:13:01.947169+0000","last_clean":"2024-09-19T08:13:01.947169+0000","last_became_active":"2024-09-19T08:13:00.856100+0000","last_became_peered":"2024-09-19T08:13:00.856100+0000","last_unstale":"2024-09-19T08:13:01.947169+0000","last_undegraded":"2024-09-19T08:13:01.947169+0000","last_fullsized":"2024-09-19T08:13:01.947169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:58.684930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872838+0000","last_change":"2024-09-19T08:12:58.680778+0000","last_active":"2024-09-19T08:13:09.872838+0000","last_peered":"2024-09-19T08:13:09.872838+0000","last_clean":"2024-09-19T08:13:09.872838+0000","last_became_active":"2024-09-19T08:12:58.680364+0000","last_became_peered":"2024-09-19T08:12:58.680364+0000","last_unstale":"2024-09-19T08:13:09.872838+0000","last_undegraded":"2024-09-19T08:13:09.872838+0000","last_fullsized":"2024-09-19T08:13:09.872838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:10.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100651+0000","last_change":"2024-09-19T08:13:00.853415+0000","last_active":"2024-09-19T08:13:02.100651+0000","last_peered":"2024-09-19T08:13:02.100651+0000","last_clean":"2024-09-19T08:13:02.100651+0000","last_became_active":"2024-09-19T08:13:00.853101+0000","last_became_peered":"2024-09-19T08:13:00.853101+0000","last_unstale":"2024-09-19T08:13:02.100651+0000","last_undegraded":"2024-09-19T08:13:02.100651+0000","last_fullsized":"2024-09-19T08:13:02.100651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:42.375394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:22.899868+0000","last_change":"2024-09-19T08:12:58.680312+0000","last_active":"2024-09-19T08:13:22.899868+0000","last_peered":"2024-09-19T08:13:22.899868+0000","last_clean":"2024-09-19T08:13:22.899868+0000","last_became_active":"2024-09-19T08:12:58.679870+0000","last_became_peered":"2024-09-19T08:12:58.679870+0000","last_unstale":"2024-09-19T08:13:22.899868+0000","last_undegraded":"2024-09-19T08:13:22.899868+0000","last_fullsized":"2024-09-19T08:13:22.899868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:42.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100329+0000","last_change":"2024-09-19T08:13:00.853920+0000","last_active":"2024-09-19T08:13:02.100329+0000","last_peered":"2024-09-19T08:13:02.100329+0000","last_clean":"2024-09-19T08:13:02.100329+0000","last_became_active":"2024-09-19T08:13:00.853594+0000","last_became_peered":"2024-09-19T08:13:00.853594+0000","last_unstale":"2024-09-19T08:13:02.100329+0000","last_undegraded":"2024-09-19T08:13:02.100329+0000","last_fullsized":"2024-09-19T08:13:02.100329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:31.746133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.882028+0000","last_change":"2024-09-19T08:12:58.717487+0000","last_active":"2024-09-19T08:13:10.882028+0000","last_peered":"2024-09-19T08:13:10.882028+0000","last_clean":"2024-09-19T08:13:10.882028+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:13:10.882028+0000","last_undegraded":"2024-09-19T08:13:10.882028+0000","last_fullsized":"2024-09-19T08:13:10.882028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:38.506051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947060+0000","last_change":"2024-09-19T08:13:00.847061+0000","last_active":"2024-09-19T08:13:01.947060+0000","last_peered":"2024-09-19T08:13:01.947060+0000","last_clean":"2024-09-19T08:13:01.947060+0000","last_became_active":"2024-09-19T08:13:00.846742+0000","last_became_peered":"2024-09-19T08:13:00.846742+0000","last_unstale":"2024-09-19T08:13:01.947060+0000","last_undegraded":"2024-09-19T08:13:01.947060+0000","last_fullsized":"2024-09-19T08:13:01.947060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:21.785585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:11.877000+0000","last_change":"2024-09-19T08:12:58.679367+0000","last_active":"2024-09-19T08:13:11.877000+0000","last_peered":"2024-09-19T08:13:11.877000+0000","last_clean":"2024-09-19T08:13:11.877000+0000","last_became_active":"2024-09-19T08:12:58.679035+0000","last_became_peered":"2024-09-19T08:12:58.679035+0000","last_unstale":"2024-09-19T08:13:11.877000+0000","last_undegraded":"2024-09-19T08:13:11.877000+0000","last_fullsized":"2024-09-19T08:13:11.877000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:25.633215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948364+0000","last_change":"2024-09-19T08:13:00.844940+0000","last_active":"2024-09-19T08:13:01.948364+0000","last_peered":"2024-09-19T08:13:01.948364+0000","last_clean":"2024-09-19T08:13:01.948364+0000","last_became_active":"2024-09-19T08:13:00.844454+0000","last_became_peered":"2024-09-19T08:13:00.844454+0000","last_unstale":"2024-09-19T08:13:01.948364+0000","last_undegraded":"2024-09-19T08:13:01.948364+0000","last_fullsized":"2024-09-19T08:13:01.948364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:15.577624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.272636+0000","last_change":"2024-09-19T08:12:58.673301+0000","last_active":"2024-09-19T08:13:02.272636+0000","last_peered":"2024-09-19T08:13:02.272636+0000","last_clean":"2024-09-19T08:13:02.272636+0000","last_became_active":"2024-09-19T08:12:58.672981+0000","last_became_peered":"2024-09-19T08:12:58.672981+0000","last_unstale":"2024-09-19T08:13:02.272636+0000","last_undegraded":"2024-09-19T08:13:02.272636+0000","last_fullsized":"2024-09-19T08:13:02.272636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:03.963712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100834+0000","last_change":"2024-09-19T08:13:00.856620+0000","last_active":"2024-09-19T08:13:02.100834+0000","last_peered":"2024-09-19T08:13:02.100834+0000","last_clean":"2024-09-19T08:13:02.100834+0000","last_became_active":"2024-09-19T08:13:00.856310+0000","last_became_peered":"2024-09-19T08:13:00.856310+0000","last_unstale":"2024-09-19T08:13:02.100834+0000","last_undegraded":"2024-09-19T08:13:02.100834+0000","last_fullsized":"2024-09-19T08:13:02.100834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:16.918768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.271923+0000","last_change":"2024-09-19T08:12:58.672282+0000","last_active":"2024-09-19T08:13:02.271923+0000","last_peered":"2024-09-19T08:13:02.271923+0000","last_clean":"2024-09-19T08:13:02.271923+0000","last_became_active":"2024-09-19T08:12:58.671891+0000","last_became_peered":"2024-09-19T08:12:58.671891+0000","last_unstale":"2024-09-19T08:13:02.271923+0000","last_undegraded":"2024-09-19T08:13:02.271923+0000","last_fullsized":"2024-09-19T08:13:02.271923+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:41.731571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100814+0000","last_change":"2024-09-19T08:13:00.848107+0000","last_active":"2024-09-19T08:13:02.100814+0000","last_peered":"2024-09-19T08:13:02.100814+0000","last_clean":"2024-09-19T08:13:02.100814+0000","last_became_active":"2024-09-19T08:13:00.847632+0000","last_became_peered":"2024-09-19T08:13:00.847632+0000","last_unstale":"2024-09-19T08:13:02.100814+0000","last_undegraded":"2024-09-19T08:13:02.100814+0000","last_fullsized":"2024-09-19T08:13:02.100814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.491122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.880827+0000","last_change":"2024-09-19T08:12:59.179200+0000","last_active":"2024-09-19T08:13:10.880827+0000","last_peered":"2024-09-19T08:13:10.880827+0000","last_clean":"2024-09-19T08:13:10.880827+0000","last_became_active":"2024-09-19T08:12:59.178762+0000","last_became_peered":"2024-09-19T08:12:59.178762+0000","last_unstale":"2024-09-19T08:13:10.880827+0000","last_undegraded":"2024-09-19T08:13:10.880827+0000","last_fullsized":"2024-09-19T08:13:10.880827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:08.424106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100825+0000","last_change":"2024-09-19T08:13:00.855870+0000","last_active":"2024-09-19T08:13:02.100825+0000","last_peered":"2024-09-19T08:13:02.100825+0000","last_clean":"2024-09-19T08:13:02.100825+0000","last_became_active":"2024-09-19T08:13:00.855538+0000","last_became_peered":"2024-09-19T08:13:00.855538+0000","last_unstale":"2024-09-19T08:13:02.100825+0000","last_undegraded":"2024-09-19T08:13:02.100825+0000","last_fullsized":"2024-09-19T08:13:02.100825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100488+0000","last_change":"2024-09-19T08:11:48.661333+0000","last_active":"2024-09-19T08:13:02.100488+0000","last_peered":"2024-09-19T08:13:02.100488+0000","last_clean":"2024-09-19T08:13:02.100488+0000","last_became_active":"2024-09-19T08:11:48.660873+0000","last_became_peered":"2024-09-19T08:11:48.660873+0000","last_unstale":"2024-09-19T08:13:02.100488+0000","last_undegraded":"2024-09-19T08:13:02.100488+0000","last_fullsized":"2024-09-19T08:13:02.100488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_clean_scrub_stamp":"2024-09-19T08:08:25.372155+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:19.401292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:11.881883+0000","last_change":"2024-09-19T08:12:58.675237+0000","last_active":"2024-09-19T08:13:11.881883+0000","last_peered":"2024-09-19T08:13:11.881883+0000","last_clean":"2024-09-19T08:13:11.881883+0000","last_became_active":"2024-09-19T08:12:58.674910+0000","last_became_peered":"2024-09-19T08:12:58.674910+0000","last_unstale":"2024-09-19T08:13:11.881883+0000","last_undegraded":"2024-09-19T08:13:11.881883+0000","last_fullsized":"2024-09-19T08:13:11.881883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:47.038328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947516+0000","last_change":"2024-09-19T08:13:00.846754+0000","last_active":"2024-09-19T08:13:01.947516+0000","last_peered":"2024-09-19T08:13:01.947516+0000","last_clean":"2024-09-19T08:13:01.947516+0000","last_became_active":"2024-09-19T08:13:00.846452+0000","last_became_peered":"2024-09-19T08:13:00.846452+0000","last_unstale":"2024-09-19T08:13:01.947516+0000","last_undegraded":"2024-09-19T08:13:01.947516+0000","last_fullsized":"2024-09-19T08:13:01.947516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:22.929689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.881292+0000","last_change":"2024-09-19T08:12:58.678958+0000","last_active":"2024-09-19T08:13:10.881292+0000","last_peered":"2024-09-19T08:13:10.881292+0000","last_clean":"2024-09-19T08:13:10.881292+0000","last_became_active":"2024-09-19T08:12:58.678592+0000","last_became_peered":"2024-09-19T08:12:58.678592+0000","last_unstale":"2024-09-19T08:13:10.881292+0000","last_undegraded":"2024-09-19T08:13:10.881292+0000","last_fullsized":"2024-09-19T08:13:10.881292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:58.374606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947087+0000","last_change":"2024-09-19T08:13:00.840973+0000","last_active":"2024-09-19T08:13:01.947087+0000","last_peered":"2024-09-19T08:13:01.947087+0000","last_clean":"2024-09-19T08:13:01.947087+0000","last_became_active":"2024-09-19T08:13:00.840619+0000","last_became_peered":"2024-09-19T08:13:00.840619+0000","last_unstale":"2024-09-19T08:13:01.947087+0000","last_undegraded":"2024-09-19T08:13:01.947087+0000","last_fullsized":"2024-09-19T08:13:01.947087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:23.900583+0000","last_change":"2024-09-19T08:12:58.715972+0000","last_active":"2024-09-19T08:13:23.900583+0000","last_peered":"2024-09-19T08:13:23.900583+0000","last_clean":"2024-09-19T08:13:23.900583+0000","last_became_active":"2024-09-19T08:12:58.715630+0000","last_became_peered":"2024-09-19T08:12:58.715630+0000","last_unstale":"2024-09-19T08:13:23.900583+0000","last_undegraded":"2024-09-19T08:13:23.900583+0000","last_fullsized":"2024-09-19T08:13:23.900583+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:35.637121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947439+0000","last_change":"2024-09-19T08:13:00.847023+0000","last_active":"2024-09-19T08:13:01.947439+0000","last_peered":"2024-09-19T08:13:01.947439+0000","last_clean":"2024-09-19T08:13:01.947439+0000","last_became_active":"2024-09-19T08:13:00.846387+0000","last_became_peered":"2024-09-19T08:13:00.846387+0000","last_unstale":"2024-09-19T08:13:01.947439+0000","last_undegraded":"2024-09-19T08:13:01.947439+0000","last_fullsized":"2024-09-19T08:13:01.947439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:12.187929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100987+0000","last_change":"2024-09-19T08:12:58.681062+0000","last_active":"2024-09-19T08:13:02.100987+0000","last_peered":"2024-09-19T08:13:02.100987+0000","last_clean":"2024-09-19T08:13:02.100987+0000","last_became_active":"2024-09-19T08:12:58.680524+0000","last_became_peered":"2024-09-19T08:12:58.680524+0000","last_unstale":"2024-09-19T08:13:02.100987+0000","last_undegraded":"2024-09-19T08:13:02.100987+0000","last_fullsized":"2024-09-19T08:13:02.100987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947809+0000","last_change":"2024-09-19T08:13:00.851523+0000","last_active":"2024-09-19T08:13:01.947809+0000","last_peered":"2024-09-19T08:13:01.947809+0000","last_clean":"2024-09-19T08:13:01.947809+0000","last_became_active":"2024-09-19T08:13:00.851068+0000","last_became_peered":"2024-09-19T08:13:00.851068+0000","last_unstale":"2024-09-19T08:13:01.947809+0000","last_undegraded":"2024-09-19T08:13:01.947809+0000","last_fullsized":"2024-09-19T08:13:01.947809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:44.905469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.831886+0000","last_change":"2024-09-19T08:12:58.718234+0000","last_active":"2024-09-19T08:13:01.831886+0000","last_peered":"2024-09-19T08:13:01.831886+0000","last_clean":"2024-09-19T08:13:01.831886+0000","last_became_active":"2024-09-19T08:12:58.717840+0000","last_became_peered":"2024-09-19T08:12:58.717840+0000","last_unstale":"2024-09-19T08:13:01.831886+0000","last_undegraded":"2024-09-19T08:13:01.831886+0000","last_fullsized":"2024-09-19T08:13:01.831886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:05.283770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947784+0000","last_change":"2024-09-19T08:13:00.848305+0000","last_active":"2024-09-19T08:13:01.947784+0000","last_peered":"2024-09-19T08:13:01.947784+0000","last_clean":"2024-09-19T08:13:01.947784+0000","last_became_active":"2024-09-19T08:13:00.847981+0000","last_became_peered":"2024-09-19T08:13:00.847981+0000","last_unstale":"2024-09-19T08:13:01.947784+0000","last_undegraded":"2024-09-19T08:13:01.947784+0000","last_fullsized":"2024-09-19T08:13:01.947784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:04.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.871966+0000","last_change":"2024-09-19T08:12:58.715943+0000","last_active":"2024-09-19T08:13:09.871966+0000","last_peered":"2024-09-19T08:13:09.871966+0000","last_clean":"2024-09-19T08:13:09.871966+0000","last_became_active":"2024-09-19T08:12:58.715600+0000","last_became_peered":"2024-09-19T08:12:58.715600+0000","last_unstale":"2024-09-19T08:13:09.871966+0000","last_undegraded":"2024-09-19T08:13:09.871966+0000","last_fullsized":"2024-09-19T08:13:09.871966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:19.142271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948126+0000","last_change":"2024-09-19T08:13:00.840944+0000","last_active":"2024-09-19T08:13:01.948126+0000","last_peered":"2024-09-19T08:13:01.948126+0000","last_clean":"2024-09-19T08:13:01.948126+0000","last_became_active":"2024-09-19T08:13:00.840608+0000","last_became_peered":"2024-09-19T08:13:00.840608+0000","last_unstale":"2024-09-19T08:13:01.948126+0000","last_undegraded":"2024-09-19T08:13:01.948126+0000","last_fullsized":"2024-09-19T08:13:01.948126+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:25.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'6","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:17.267352+0000","last_change":"2024-09-19T08:12:58.672929+0000","last_active":"2024-09-19T08:13:17.267352+0000","last_peered":"2024-09-19T08:13:17.267352+0000","last_clean":"2024-09-19T08:13:17.267352+0000","last_became_active":"2024-09-19T08:12:58.672487+0000","last_became_peered":"2024-09-19T08:12:58.672487+0000","last_unstale":"2024-09-19T08:13:17.267352+0000","last_undegraded":"2024-09-19T08:13:17.267352+0000","last_fullsized":"2024-09-19T08:13:17.267352+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:33.042735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947672+0000","last_change":"2024-09-19T08:13:00.853457+0000","last_active":"2024-09-19T08:13:01.947672+0000","last_peered":"2024-09-19T08:13:01.947672+0000","last_clean":"2024-09-19T08:13:01.947672+0000","last_became_active":"2024-09-19T08:13:00.853159+0000","last_became_peered":"2024-09-19T08:13:00.853159+0000","last_unstale":"2024-09-19T08:13:01.947672+0000","last_undegraded":"2024-09-19T08:13:01.947672+0000","last_fullsized":"2024-09-19T08:13:01.947672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:33.899170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.869690+0000","last_change":"2024-09-19T08:12:59.179510+0000","last_active":"2024-09-19T08:13:09.869690+0000","last_peered":"2024-09-19T08:13:09.869690+0000","last_clean":"2024-09-19T08:13:09.869690+0000","last_became_active":"2024-09-19T08:12:59.179157+0000","last_became_peered":"2024-09-19T08:12:59.179157+0000","last_unstale":"2024-09-19T08:13:09.869690+0000","last_undegraded":"2024-09-19T08:13:09.869690+0000","last_fullsized":"2024-09-19T08:13:09.869690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:22.981289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948760+0000","last_change":"2024-09-19T08:13:00.841853+0000","last_active":"2024-09-19T08:13:01.948760+0000","last_peered":"2024-09-19T08:13:01.948760+0000","last_clean":"2024-09-19T08:13:01.948760+0000","last_became_active":"2024-09-19T08:13:00.841518+0000","last_became_peered":"2024-09-19T08:13:00.841518+0000","last_unstale":"2024-09-19T08:13:01.948760+0000","last_undegraded":"2024-09-19T08:13:01.948760+0000","last_fullsized":"2024-09-19T08:13:01.948760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:05.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872283+0000","last_change":"2024-09-19T08:12:58.716882+0000","last_active":"2024-09-19T08:13:09.872283+0000","last_peered":"2024-09-19T08:13:09.872283+0000","last_clean":"2024-09-19T08:13:09.872283+0000","last_became_active":"2024-09-19T08:12:58.716545+0000","last_became_peered":"2024-09-19T08:12:58.716545+0000","last_unstale":"2024-09-19T08:13:09.872283+0000","last_undegraded":"2024-09-19T08:13:09.872283+0000","last_fullsized":"2024-09-19T08:13:09.872283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:14.469109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.946790+0000","last_change":"2024-09-19T08:13:00.850484+0000","last_active":"2024-09-19T08:13:01.946790+0000","last_peered":"2024-09-19T08:13:01.946790+0000","last_clean":"2024-09-19T08:13:01.946790+0000","last_became_active":"2024-09-19T08:13:00.850049+0000","last_became_peered":"2024-09-19T08:13:00.850049+0000","last_unstale":"2024-09-19T08:13:01.946790+0000","last_undegraded":"2024-09-19T08:13:01.946790+0000","last_fullsized":"2024-09-19T08:13:01.946790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:24.424111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.870765+0000","last_change":"2024-09-19T08:12:58.669928+0000","last_active":"2024-09-19T08:13:09.870765+0000","last_peered":"2024-09-19T08:13:09.870765+0000","last_clean":"2024-09-19T08:13:09.870765+0000","last_became_active":"2024-09-19T08:12:58.669614+0000","last_became_peered":"2024-09-19T08:12:58.669614+0000","last_unstale":"2024-09-19T08:13:09.870765+0000","last_undegraded":"2024-09-19T08:13:09.870765+0000","last_fullsized":"2024-09-19T08:13:09.870765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:50.924779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948102+0000","last_change":"2024-09-19T08:13:00.853520+0000","last_active":"2024-09-19T08:13:01.948102+0000","last_peered":"2024-09-19T08:13:01.948102+0000","last_clean":"2024-09-19T08:13:01.948102+0000","last_became_active":"2024-09-19T08:13:00.853184+0000","last_became_peered":"2024-09-19T08:13:00.853184+0000","last_unstale":"2024-09-19T08:13:01.948102+0000","last_undegraded":"2024-09-19T08:13:01.948102+0000","last_fullsized":"2024-09-19T08:13:01.948102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:45.160439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.832515+0000","last_change":"2024-09-19T08:12:58.674359+0000","last_active":"2024-09-19T08:13:01.832515+0000","last_peered":"2024-09-19T08:13:01.832515+0000","last_clean":"2024-09-19T08:13:01.832515+0000","last_became_active":"2024-09-19T08:12:58.673890+0000","last_became_peered":"2024-09-19T08:12:58.673890+0000","last_unstale":"2024-09-19T08:13:01.832515+0000","last_undegraded":"2024-09-19T08:13:01.832515+0000","last_fullsized":"2024-09-19T08:13:01.832515+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:54.939959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947708+0000","last_change":"2024-09-19T08:13:00.851071+0000","last_active":"2024-09-19T08:13:01.947708+0000","last_peered":"2024-09-19T08:13:01.947708+0000","last_clean":"2024-09-19T08:13:01.947708+0000","last_became_active":"2024-09-19T08:13:00.850756+0000","last_became_peered":"2024-09-19T08:13:00.850756+0000","last_unstale":"2024-09-19T08:13:01.947708+0000","last_undegraded":"2024-09-19T08:13:01.947708+0000","last_fullsized":"2024-09-19T08:13:01.947708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:52.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872854+0000","last_change":"2024-09-19T08:12:58.681466+0000","last_active":"2024-09-19T08:13:09.872854+0000","last_peered":"2024-09-19T08:13:09.872854+0000","last_clean":"2024-09-19T08:13:09.872854+0000","last_became_active":"2024-09-19T08:12:58.681113+0000","last_became_peered":"2024-09-19T08:12:58.681113+0000","last_unstale":"2024-09-19T08:13:09.872854+0000","last_undegraded":"2024-09-19T08:13:09.872854+0000","last_fullsized":"2024-09-19T08:13:09.872854+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:49.610828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100583+0000","last_change":"2024-09-19T08:13:00.851272+0000","last_active":"2024-09-19T08:13:02.100583+0000","last_peered":"2024-09-19T08:13:02.100583+0000","last_clean":"2024-09-19T08:13:02.100583+0000","last_became_active":"2024-09-19T08:13:00.849098+0000","last_became_peered":"2024-09-19T08:13:00.849098+0000","last_unstale":"2024-09-19T08:13:02.100583+0000","last_undegraded":"2024-09-19T08:13:02.100583+0000","last_fullsized":"2024-09-19T08:13:02.100583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:05.643992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947777+0000","last_change":"2024-09-19T08:13:00.849945+0000","last_active":"2024-09-19T08:13:01.947777+0000","last_peered":"2024-09-19T08:13:01.947777+0000","last_clean":"2024-09-19T08:13:01.947777+0000","last_became_active":"2024-09-19T08:13:00.849591+0000","last_became_peered":"2024-09-19T08:13:00.849591+0000","last_unstale":"2024-09-19T08:13:01.947777+0000","last_undegraded":"2024-09-19T08:13:01.947777+0000","last_fullsized":"2024-09-19T08:13:01.947777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:27.434005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.272564+0000","last_change":"2024-09-19T08:12:58.674610+0000","last_active":"2024-09-19T08:13:02.272564+0000","last_peered":"2024-09-19T08:13:02.272564+0000","last_clean":"2024-09-19T08:13:02.272564+0000","last_became_active":"2024-09-19T08:12:58.674247+0000","last_became_peered":"2024-09-19T08:12:58.674247+0000","last_unstale":"2024-09-19T08:13:02.272564+0000","last_undegraded":"2024-09-19T08:13:02.272564+0000","last_fullsized":"2024-09-19T08:13:02.272564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:16.323574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836419+0000","last_change":"2024-09-19T08:13:00.858512+0000","last_active":"2024-09-19T08:13:01.836419+0000","last_peered":"2024-09-19T08:13:01.836419+0000","last_clean":"2024-09-19T08:13:01.836419+0000","last_became_active":"2024-09-19T08:13:00.857897+0000","last_became_peered":"2024-09-19T08:13:00.857897+0000","last_unstale":"2024-09-19T08:13:01.836419+0000","last_undegraded":"2024-09-19T08:13:01.836419+0000","last_fullsized":"2024-09-19T08:13:01.836419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.881153+0000","last_change":"2024-09-19T08:12:58.682207+0000","last_active":"2024-09-19T08:13:10.881153+0000","last_peered":"2024-09-19T08:13:10.881153+0000","last_clean":"2024-09-19T08:13:10.881153+0000","last_became_active":"2024-09-19T08:12:58.681842+0000","last_became_peered":"2024-09-19T08:12:58.681842+0000","last_unstale":"2024-09-19T08:13:10.881153+0000","last_undegraded":"2024-09-19T08:13:10.881153+0000","last_fullsized":"2024-09-19T08:13:10.881153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.825858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948835+0000","last_change":"2024-09-19T08:13:00.846748+0000","last_active":"2024-09-19T08:13:01.948835+0000","last_peered":"2024-09-19T08:13:01.948835+0000","last_clean":"2024-09-19T08:13:01.948835+0000","last_became_active":"2024-09-19T08:13:00.846368+0000","last_became_peered":"2024-09-19T08:13:00.846368+0000","last_unstale":"2024-09-19T08:13:01.948835+0000","last_undegraded":"2024-09-19T08:13:01.948835+0000","last_fullsized":"2024-09-19T08:13:01.948835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:59.526512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.271849+0000","last_change":"2024-09-19T08:12:58.716118+0000","last_active":"2024-09-19T08:13:02.271849+0000","last_peered":"2024-09-19T08:13:02.271849+0000","last_clean":"2024-09-19T08:13:02.271849+0000","last_became_active":"2024-09-19T08:12:58.715711+0000","last_became_peered":"2024-09-19T08:12:58.715711+0000","last_unstale":"2024-09-19T08:13:02.271849+0000","last_undegraded":"2024-09-19T08:13:02.271849+0000","last_fullsized":"2024-09-19T08:13:02.271849+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:29.439887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947595+0000","last_change":"2024-09-19T08:13:00.848729+0000","last_active":"2024-09-19T08:13:01.947595+0000","last_peered":"2024-09-19T08:13:01.947595+0000","last_clean":"2024-09-19T08:13:01.947595+0000","last_became_active":"2024-09-19T08:13:00.848470+0000","last_became_peered":"2024-09-19T08:13:00.848470+0000","last_unstale":"2024-09-19T08:13:01.947595+0000","last_undegraded":"2024-09-19T08:13:01.947595+0000","last_fullsized":"2024-09-19T08:13:01.947595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:30.653101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.882192+0000","last_change":"2024-09-19T08:12:58.681654+0000","last_active":"2024-09-19T08:13:10.882192+0000","last_peered":"2024-09-19T08:13:10.882192+0000","last_clean":"2024-09-19T08:13:10.882192+0000","last_became_active":"2024-09-19T08:12:58.681284+0000","last_became_peered":"2024-09-19T08:12:58.681284+0000","last_unstale":"2024-09-19T08:13:10.882192+0000","last_undegraded":"2024-09-19T08:13:10.882192+0000","last_fullsized":"2024-09-19T08:13:10.882192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.429930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":51198,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":109,"num_read_kb":110,"num_write":147,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":153594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251470,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481809,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":428609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710742,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":1067285,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874267,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":428845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":909312,"data_stored":428765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1021090,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":428234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":1020381,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":429064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":913408,"data_stored":428545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":476572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":474829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":429505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34755,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34755,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":46802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":48545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":48773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:13:28.214 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:13:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:28.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:28 smithi022 ceph-mon[30963]: pgmap v293: 97 pgs: 97 active+clean; 627 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T08:13:28.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:28 smithi022 ceph-mon[30963]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:13:28.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:28 smithi064 ceph-mon[34900]: pgmap v293: 97 pgs: 97 active+clean; 627 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T08:13:28.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:28 smithi064 ceph-mon[34900]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:13:28.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:28 smithi157 ceph-mon[35143]: pgmap v293: 97 pgs: 97 active+clean; 627 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T08:13:28.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:28 smithi157 ceph-mon[35143]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:13:29.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.983+0000 7fc570de2640 1 -- 172.21.15.22:0/3389724809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc56c100e20 msgr2=0x7fc56c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:29.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.983+0000 7fc570de2640 1 --2- 172.21.15.22:0/3389724809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc56c100e20 0x7fc56c101200 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc55c009900 tx=0x7fc55c02a480 comp rx=0 tx=0).stop 2024-09-19T08:13:29.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.984+0000 7fc570de2640 1 -- 172.21.15.22:0/3389724809 shutdown_connections 2024-09-19T08:13:29.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.984+0000 7fc570de2640 1 --2- 172.21.15.22:0/3389724809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc56c102ef0 0x7fc56c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:29.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.984+0000 7fc570de2640 1 --2- 172.21.15.22:0/3389724809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc56c102570 0x7fc56c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:29.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.984+0000 7fc570de2640 1 --2- 172.21.15.22:0/3389724809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc56c100e20 0x7fc56c101200 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:29.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.984+0000 7fc570de2640 1 -- 172.21.15.22:0/3389724809 >> 172.21.15.22:0/3389724809 conn(0x7fc56c0fc7d0 msgr2=0x7fc56c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:29.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.984+0000 7fc570de2640 1 -- 172.21.15.22:0/3389724809 shutdown_connections 2024-09-19T08:13:29.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.984+0000 7fc570de2640 1 -- 172.21.15.22:0/3389724809 wait complete. 2024-09-19T08:13:29.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.984+0000 7fc570de2640 1 Processor -- start 2024-09-19T08:13:29.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.984+0000 7fc570de2640 1 -- start start 2024-09-19T08:13:29.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc570de2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc56c100e20 0x7fc56c1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:29.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc570de2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc56c102570 0x7fc56c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:29.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc570de2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc56c102ef0 0x7fc56c1a4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:29.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc570de2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c111070 con 0x7fc56c102ef0 2024-09-19T08:13:29.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc570de2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c110ef0 con 0x7fc56c100e20 2024-09-19T08:13:29.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc570de2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c1111f0 con 0x7fc56c102570 2024-09-19T08:13:29.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc56ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc56c102ef0 0x7fc56c1a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:29.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc56ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc56c102ef0 0x7fc56c1a4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46518/0 (socket says 172.21.15.22:46518) 2024-09-19T08:13:29.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc56ad9d640 1 -- 172.21.15.22:0/2936162725 learned_addr learned my addr 172.21.15.22:0/2936162725 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:29.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc563fff640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc56c102570 0x7fc56c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:29.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc56a59c640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc56c100e20 0x7fc56c1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc56ad9d640 1 -- 172.21.15.22:0/2936162725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc56c102570 msgr2=0x7fc56c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc56ad9d640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc56c102570 0x7fc56c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc56ad9d640 1 -- 172.21.15.22:0/2936162725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc56c100e20 msgr2=0x7fc56c1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc56ad9d640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc56c100e20 0x7fc56c1994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc56ad9d640 1 -- 172.21.15.22:0/2936162725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56c1a4a60 con 0x7fc56c102ef0 2024-09-19T08:13:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc56a59c640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc56c100e20 0x7fc56c1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:13:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc56ad9d640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc56c102ef0 0x7fc56c1a4520 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b970 tx=0x7fc56400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc5637fe640 1 -- 172.21.15.22:0/2936162725 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564019070 con 0x7fc56c102ef0 2024-09-19T08:13:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.985+0000 7fc570de2640 1 -- 172.21.15.22:0/2936162725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc56c1a4c30 con 0x7fc56c102ef0 2024-09-19T08:13:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.986+0000 7fc5637fe640 1 -- 172.21.15.22:0/2936162725 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc564014440 con 0x7fc56c102ef0 2024-09-19T08:13:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.986+0000 7fc5637fe640 1 -- 172.21.15.22:0/2936162725 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5640075d0 con 0x7fc56c102ef0 2024-09-19T08:13:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.986+0000 7fc570de2640 1 -- 172.21.15.22:0/2936162725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc56c1a5140 con 0x7fc56c102ef0 2024-09-19T08:13:29.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.986+0000 7fc570de2640 1 -- 172.21.15.22:0/2936162725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc56c1021b0 con 0x7fc56c102ef0 2024-09-19T08:13:29.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.988+0000 7fc5637fe640 1 -- 172.21.15.22:0/2936162725 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc56401c070 con 0x7fc56c102ef0 2024-09-19T08:13:29.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.988+0000 7fc5637fe640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc5580a32d0 0x7fc5580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:29.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.988+0000 7fc56a59c640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc5580a32d0 0x7fc5580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:29.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.988+0000 7fc5637fe640 1 -- 172.21.15.22:0/2936162725 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fc5640cef20 con 0x7fc56c102ef0 2024-09-19T08:13:29.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.989+0000 7fc56a59c640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc5580a32d0 0x7fc5580a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc55c000c00 tx=0x7fc55c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:29.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:28.993+0000 7fc5637fe640 1 -- 172.21.15.22:0/2936162725 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc56409bca0 con 0x7fc56c102ef0 2024-09-19T08:13:29.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:29.164+0000 7fc570de2640 1 -- 172.21.15.22:0/2936162725 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc56c062630 con 0x7fc5580a32d0 2024-09-19T08:13:29.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:29.173+0000 7fc5637fe640 1 -- 172.21.15.22:0/2936162725 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258841 (secure 0 0 0) 0x7fc56c062630 con 0x7fc5580a32d0 2024-09-19T08:13:29.175 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:13:29.177 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T08:13:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:29.176+0000 7fc570de2640 1 -- 172.21.15.22:0/2936162725 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc5580a32d0 msgr2=0x7fc5580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:29.176+0000 7fc570de2640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc5580a32d0 0x7fc5580a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc55c000c00 tx=0x7fc55c005a20 comp rx=0 tx=0).stop 2024-09-19T08:13:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:29.176+0000 7fc570de2640 1 -- 172.21.15.22:0/2936162725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc56c102ef0 msgr2=0x7fc56c1a4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:29.176+0000 7fc570de2640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc56c102ef0 0x7fc56c1a4520 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b970 tx=0x7fc56400bd70 comp rx=0 tx=0).stop 2024-09-19T08:13:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:29.176+0000 7fc570de2640 1 -- 172.21.15.22:0/2936162725 shutdown_connections 2024-09-19T08:13:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:29.176+0000 7fc570de2640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc5580a32d0 0x7fc5580a5790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:29.176+0000 7fc570de2640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc56c102ef0 0x7fc56c1a4520 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:29.176+0000 7fc570de2640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc56c102570 0x7fc56c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:29.176+0000 7fc570de2640 1 --2- 172.21.15.22:0/2936162725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc56c100e20 0x7fc56c1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:29.176+0000 7fc570de2640 1 -- 172.21.15.22:0/2936162725 >> 172.21.15.22:0/2936162725 conn(0x7fc56c0fc7d0 msgr2=0x7fc56c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:29.176+0000 7fc570de2640 1 -- 172.21.15.22:0/2936162725 shutdown_connections 2024-09-19T08:13:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:29.176+0000 7fc570de2640 1 -- 172.21.15.22:0/2936162725 wait complete. 2024-09-19T08:13:29.899 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-19T08:13:28.769035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641800,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":245,"num_read_kb":229,"num_write":336,"num_write_kb":3033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381012,"kb_used_data":10964,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560812,"statfs":{"total":1151940427776,"available":1151550271488,"internally_reserved":0,"allocated":11227136,"data_stored":7069367,"data_compressed":109176,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39608,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":78,"num_write":27,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002984"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948678+0000","last_change":"2024-09-19T08:13:00.853947+0000","last_active":"2024-09-19T08:13:01.948678+0000","last_peered":"2024-09-19T08:13:01.948678+0000","last_clean":"2024-09-19T08:13:01.948678+0000","last_became_active":"2024-09-19T08:13:00.852951+0000","last_became_peered":"2024-09-19T08:13:00.852951+0000","last_unstale":"2024-09-19T08:13:01.948678+0000","last_undegraded":"2024-09-19T08:13:01.948678+0000","last_fullsized":"2024-09-19T08:13:01.948678+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:25.786828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948757+0000","last_change":"2024-09-19T08:13:00.853389+0000","last_active":"2024-09-19T08:13:01.948757+0000","last_peered":"2024-09-19T08:13:01.948757+0000","last_clean":"2024-09-19T08:13:01.948757+0000","last_became_active":"2024-09-19T08:13:00.853102+0000","last_became_peered":"2024-09-19T08:13:00.853102+0000","last_unstale":"2024-09-19T08:13:01.948757+0000","last_undegraded":"2024-09-19T08:13:01.948757+0000","last_fullsized":"2024-09-19T08:13:01.948757+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:42.773917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948582+0000","last_change":"2024-09-19T08:13:00.846544+0000","last_active":"2024-09-19T08:13:01.948582+0000","last_peered":"2024-09-19T08:13:01.948582+0000","last_clean":"2024-09-19T08:13:01.948582+0000","last_became_active":"2024-09-19T08:13:00.846229+0000","last_became_peered":"2024-09-19T08:13:00.846229+0000","last_unstale":"2024-09-19T08:13:01.948582+0000","last_undegraded":"2024-09-19T08:13:01.948582+0000","last_fullsized":"2024-09-19T08:13:01.948582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:43.471684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948555+0000","last_change":"2024-09-19T08:13:00.848149+0000","last_active":"2024-09-19T08:13:01.948555+0000","last_peered":"2024-09-19T08:13:01.948555+0000","last_clean":"2024-09-19T08:13:01.948555+0000","last_became_active":"2024-09-19T08:13:00.847744+0000","last_became_peered":"2024-09-19T08:13:00.847744+0000","last_unstale":"2024-09-19T08:13:01.948555+0000","last_undegraded":"2024-09-19T08:13:01.948555+0000","last_fullsized":"2024-09-19T08:13:01.948555+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:01.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948347+0000","last_change":"2024-09-19T08:13:00.841862+0000","last_active":"2024-09-19T08:13:01.948347+0000","last_peered":"2024-09-19T08:13:01.948347+0000","last_clean":"2024-09-19T08:13:01.948347+0000","last_became_active":"2024-09-19T08:13:00.841331+0000","last_became_peered":"2024-09-19T08:13:00.841331+0000","last_unstale":"2024-09-19T08:13:01.948347+0000","last_undegraded":"2024-09-19T08:13:01.948347+0000","last_fullsized":"2024-09-19T08:13:01.948347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:54.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947492+0000","last_change":"2024-09-19T08:13:00.849626+0000","last_active":"2024-09-19T08:13:01.947492+0000","last_peered":"2024-09-19T08:13:01.947492+0000","last_clean":"2024-09-19T08:13:01.947492+0000","last_became_active":"2024-09-19T08:13:00.849034+0000","last_became_peered":"2024-09-19T08:13:00.849034+0000","last_unstale":"2024-09-19T08:13:01.947492+0000","last_undegraded":"2024-09-19T08:13:01.947492+0000","last_fullsized":"2024-09-19T08:13:01.947492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:39.118978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100394+0000","last_change":"2024-09-19T08:13:00.847522+0000","last_active":"2024-09-19T08:13:02.100394+0000","last_peered":"2024-09-19T08:13:02.100394+0000","last_clean":"2024-09-19T08:13:02.100394+0000","last_became_active":"2024-09-19T08:13:00.847117+0000","last_became_peered":"2024-09-19T08:13:00.847117+0000","last_unstale":"2024-09-19T08:13:02.100394+0000","last_undegraded":"2024-09-19T08:13:02.100394+0000","last_fullsized":"2024-09-19T08:13:02.100394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.836385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947191+0000","last_change":"2024-09-19T08:13:00.849544+0000","last_active":"2024-09-19T08:13:01.947191+0000","last_peered":"2024-09-19T08:13:01.947191+0000","last_clean":"2024-09-19T08:13:01.947191+0000","last_became_active":"2024-09-19T08:13:00.849170+0000","last_became_peered":"2024-09-19T08:13:00.849170+0000","last_unstale":"2024-09-19T08:13:01.947191+0000","last_undegraded":"2024-09-19T08:13:01.947191+0000","last_fullsized":"2024-09-19T08:13:01.947191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:28.371965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836582+0000","last_change":"2024-09-19T08:13:00.856052+0000","last_active":"2024-09-19T08:13:01.836582+0000","last_peered":"2024-09-19T08:13:01.836582+0000","last_clean":"2024-09-19T08:13:01.836582+0000","last_became_active":"2024-09-19T08:13:00.855291+0000","last_became_peered":"2024-09-19T08:13:00.855291+0000","last_unstale":"2024-09-19T08:13:01.836582+0000","last_undegraded":"2024-09-19T08:13:01.836582+0000","last_fullsized":"2024-09-19T08:13:01.836582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:18.581867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948289+0000","last_change":"2024-09-19T08:13:00.850145+0000","last_active":"2024-09-19T08:13:01.948289+0000","last_peered":"2024-09-19T08:13:01.948289+0000","last_clean":"2024-09-19T08:13:01.948289+0000","last_became_active":"2024-09-19T08:13:00.849798+0000","last_became_peered":"2024-09-19T08:13:00.849798+0000","last_unstale":"2024-09-19T08:13:01.948289+0000","last_undegraded":"2024-09-19T08:13:01.948289+0000","last_fullsized":"2024-09-19T08:13:01.948289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:21.400000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100790+0000","last_change":"2024-09-19T08:13:00.859153+0000","last_active":"2024-09-19T08:13:02.100790+0000","last_peered":"2024-09-19T08:13:02.100790+0000","last_clean":"2024-09-19T08:13:02.100790+0000","last_became_active":"2024-09-19T08:13:00.858473+0000","last_became_peered":"2024-09-19T08:13:00.858473+0000","last_unstale":"2024-09-19T08:13:02.100790+0000","last_undegraded":"2024-09-19T08:13:02.100790+0000","last_fullsized":"2024-09-19T08:13:02.100790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:07.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948660+0000","last_change":"2024-09-19T08:13:00.850797+0000","last_active":"2024-09-19T08:13:01.948660+0000","last_peered":"2024-09-19T08:13:01.948660+0000","last_clean":"2024-09-19T08:13:01.948660+0000","last_became_active":"2024-09-19T08:13:00.850393+0000","last_became_peered":"2024-09-19T08:13:00.850393+0000","last_unstale":"2024-09-19T08:13:01.948660+0000","last_undegraded":"2024-09-19T08:13:01.948660+0000","last_fullsized":"2024-09-19T08:13:01.948660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:59.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947687+0000","last_change":"2024-09-19T08:13:00.850835+0000","last_active":"2024-09-19T08:13:01.947687+0000","last_peered":"2024-09-19T08:13:01.947687+0000","last_clean":"2024-09-19T08:13:01.947687+0000","last_became_active":"2024-09-19T08:13:00.850518+0000","last_became_peered":"2024-09-19T08:13:00.850518+0000","last_unstale":"2024-09-19T08:13:01.947687+0000","last_undegraded":"2024-09-19T08:13:01.947687+0000","last_fullsized":"2024-09-19T08:13:01.947687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.178530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948530+0000","last_change":"2024-09-19T08:13:00.853934+0000","last_active":"2024-09-19T08:13:01.948530+0000","last_peered":"2024-09-19T08:13:01.948530+0000","last_clean":"2024-09-19T08:13:01.948530+0000","last_became_active":"2024-09-19T08:13:00.853604+0000","last_became_peered":"2024-09-19T08:13:00.853604+0000","last_unstale":"2024-09-19T08:13:01.948530+0000","last_undegraded":"2024-09-19T08:13:01.948530+0000","last_fullsized":"2024-09-19T08:13:01.948530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:43.622526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948350+0000","last_change":"2024-09-19T08:13:00.846026+0000","last_active":"2024-09-19T08:13:01.948350+0000","last_peered":"2024-09-19T08:13:01.948350+0000","last_clean":"2024-09-19T08:13:01.948350+0000","last_became_active":"2024-09-19T08:13:00.845691+0000","last_became_peered":"2024-09-19T08:13:00.845691+0000","last_unstale":"2024-09-19T08:13:01.948350+0000","last_undegraded":"2024-09-19T08:13:01.948350+0000","last_fullsized":"2024-09-19T08:13:01.948350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:55.724159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836019+0000","last_change":"2024-09-19T08:13:00.850266+0000","last_active":"2024-09-19T08:13:01.836019+0000","last_peered":"2024-09-19T08:13:01.836019+0000","last_clean":"2024-09-19T08:13:01.836019+0000","last_became_active":"2024-09-19T08:13:00.849928+0000","last_became_peered":"2024-09-19T08:13:00.849928+0000","last_unstale":"2024-09-19T08:13:01.836019+0000","last_undegraded":"2024-09-19T08:13:01.836019+0000","last_fullsized":"2024-09-19T08:13:01.836019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:38.231331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948706+0000","last_change":"2024-09-19T08:13:00.852551+0000","last_active":"2024-09-19T08:13:01.948706+0000","last_peered":"2024-09-19T08:13:01.948706+0000","last_clean":"2024-09-19T08:13:01.948706+0000","last_became_active":"2024-09-19T08:13:00.852225+0000","last_became_peered":"2024-09-19T08:13:00.852225+0000","last_unstale":"2024-09-19T08:13:01.948706+0000","last_undegraded":"2024-09-19T08:13:01.948706+0000","last_fullsized":"2024-09-19T08:13:01.948706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:33.115108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.831882+0000","last_change":"2024-09-19T08:13:00.854033+0000","last_active":"2024-09-19T08:13:01.831882+0000","last_peered":"2024-09-19T08:13:01.831882+0000","last_clean":"2024-09-19T08:13:01.831882+0000","last_became_active":"2024-09-19T08:13:00.853660+0000","last_became_peered":"2024-09-19T08:13:00.853660+0000","last_unstale":"2024-09-19T08:13:01.831882+0000","last_undegraded":"2024-09-19T08:13:01.831882+0000","last_fullsized":"2024-09-19T08:13:01.831882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:35.602420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.832908+0000","last_change":"2024-09-19T08:13:00.850658+0000","last_active":"2024-09-19T08:13:01.832908+0000","last_peered":"2024-09-19T08:13:01.832908+0000","last_clean":"2024-09-19T08:13:01.832908+0000","last_became_active":"2024-09-19T08:13:00.850324+0000","last_became_peered":"2024-09-19T08:13:00.850324+0000","last_unstale":"2024-09-19T08:13:01.832908+0000","last_undegraded":"2024-09-19T08:13:01.832908+0000","last_fullsized":"2024-09-19T08:13:01.832908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:59:49.004605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947899+0000","last_change":"2024-09-19T08:13:00.843421+0000","last_active":"2024-09-19T08:13:01.947899+0000","last_peered":"2024-09-19T08:13:01.947899+0000","last_clean":"2024-09-19T08:13:01.947899+0000","last_became_active":"2024-09-19T08:13:00.843109+0000","last_became_peered":"2024-09-19T08:13:00.843109+0000","last_unstale":"2024-09-19T08:13:01.947899+0000","last_undegraded":"2024-09-19T08:13:01.947899+0000","last_fullsized":"2024-09-19T08:13:01.947899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:50.522686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947846+0000","last_change":"2024-09-19T08:13:00.855215+0000","last_active":"2024-09-19T08:13:01.947846+0000","last_peered":"2024-09-19T08:13:01.947846+0000","last_clean":"2024-09-19T08:13:01.947846+0000","last_became_active":"2024-09-19T08:13:00.854878+0000","last_became_peered":"2024-09-19T08:13:00.854878+0000","last_unstale":"2024-09-19T08:13:01.947846+0000","last_undegraded":"2024-09-19T08:13:01.947846+0000","last_fullsized":"2024-09-19T08:13:01.947846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:48.216066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836201+0000","last_change":"2024-09-19T08:13:00.856995+0000","last_active":"2024-09-19T08:13:01.836201+0000","last_peered":"2024-09-19T08:13:01.836201+0000","last_clean":"2024-09-19T08:13:01.836201+0000","last_became_active":"2024-09-19T08:13:00.856645+0000","last_became_peered":"2024-09-19T08:13:00.856645+0000","last_unstale":"2024-09-19T08:13:01.836201+0000","last_undegraded":"2024-09-19T08:13:01.836201+0000","last_fullsized":"2024-09-19T08:13:01.836201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:20.699293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947354+0000","last_change":"2024-09-19T08:13:00.853362+0000","last_active":"2024-09-19T08:13:01.947354+0000","last_peered":"2024-09-19T08:13:01.947354+0000","last_clean":"2024-09-19T08:13:01.947354+0000","last_became_active":"2024-09-19T08:13:00.853046+0000","last_became_peered":"2024-09-19T08:13:00.853046+0000","last_unstale":"2024-09-19T08:13:01.947354+0000","last_undegraded":"2024-09-19T08:13:01.947354+0000","last_fullsized":"2024-09-19T08:13:01.947354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947080+0000","last_change":"2024-09-19T08:13:00.845175+0000","last_active":"2024-09-19T08:13:01.947080+0000","last_peered":"2024-09-19T08:13:01.947080+0000","last_clean":"2024-09-19T08:13:01.947080+0000","last_became_active":"2024-09-19T08:13:00.844857+0000","last_became_peered":"2024-09-19T08:13:00.844857+0000","last_unstale":"2024-09-19T08:13:01.947080+0000","last_undegraded":"2024-09-19T08:13:01.947080+0000","last_fullsized":"2024-09-19T08:13:01.947080+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:40.818956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947507+0000","last_change":"2024-09-19T08:13:00.847769+0000","last_active":"2024-09-19T08:13:01.947507+0000","last_peered":"2024-09-19T08:13:01.947507+0000","last_clean":"2024-09-19T08:13:01.947507+0000","last_became_active":"2024-09-19T08:13:00.847454+0000","last_became_peered":"2024-09-19T08:13:00.847454+0000","last_unstale":"2024-09-19T08:13:01.947507+0000","last_undegraded":"2024-09-19T08:13:01.947507+0000","last_fullsized":"2024-09-19T08:13:01.947507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.583051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.833005+0000","last_change":"2024-09-19T08:13:00.859882+0000","last_active":"2024-09-19T08:13:01.833005+0000","last_peered":"2024-09-19T08:13:01.833005+0000","last_clean":"2024-09-19T08:13:01.833005+0000","last_became_active":"2024-09-19T08:13:00.859491+0000","last_became_peered":"2024-09-19T08:13:00.859491+0000","last_unstale":"2024-09-19T08:13:01.833005+0000","last_undegraded":"2024-09-19T08:13:01.833005+0000","last_fullsized":"2024-09-19T08:13:01.833005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:13.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948381+0000","last_change":"2024-09-19T08:13:00.845584+0000","last_active":"2024-09-19T08:13:01.948381+0000","last_peered":"2024-09-19T08:13:01.948381+0000","last_clean":"2024-09-19T08:13:01.948381+0000","last_became_active":"2024-09-19T08:13:00.845166+0000","last_became_peered":"2024-09-19T08:13:00.845166+0000","last_unstale":"2024-09-19T08:13:01.948381+0000","last_undegraded":"2024-09-19T08:13:01.948381+0000","last_fullsized":"2024-09-19T08:13:01.948381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:02.705793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947956+0000","last_change":"2024-09-19T08:13:00.838069+0000","last_active":"2024-09-19T08:13:01.947956+0000","last_peered":"2024-09-19T08:13:01.947956+0000","last_clean":"2024-09-19T08:13:01.947956+0000","last_became_active":"2024-09-19T08:13:00.837736+0000","last_became_peered":"2024-09-19T08:13:00.837736+0000","last_unstale":"2024-09-19T08:13:01.947956+0000","last_undegraded":"2024-09-19T08:13:01.947956+0000","last_fullsized":"2024-09-19T08:13:01.947956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:01.388290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948680+0000","last_change":"2024-09-19T08:13:00.841507+0000","last_active":"2024-09-19T08:13:01.948680+0000","last_peered":"2024-09-19T08:13:01.948680+0000","last_clean":"2024-09-19T08:13:01.948680+0000","last_became_active":"2024-09-19T08:13:00.841161+0000","last_became_peered":"2024-09-19T08:13:00.841161+0000","last_unstale":"2024-09-19T08:13:01.948680+0000","last_undegraded":"2024-09-19T08:13:01.948680+0000","last_fullsized":"2024-09-19T08:13:01.948680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:41.862290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.832504+0000","last_change":"2024-09-19T08:13:00.847657+0000","last_active":"2024-09-19T08:13:01.832504+0000","last_peered":"2024-09-19T08:13:01.832504+0000","last_clean":"2024-09-19T08:13:01.832504+0000","last_became_active":"2024-09-19T08:13:00.846849+0000","last_became_peered":"2024-09-19T08:13:00.846849+0000","last_unstale":"2024-09-19T08:13:01.832504+0000","last_undegraded":"2024-09-19T08:13:01.832504+0000","last_fullsized":"2024-09-19T08:13:01.832504+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:35.219913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.832650+0000","last_change":"2024-09-19T08:13:00.845971+0000","last_active":"2024-09-19T08:13:01.832650+0000","last_peered":"2024-09-19T08:13:01.832650+0000","last_clean":"2024-09-19T08:13:01.832650+0000","last_became_active":"2024-09-19T08:13:00.845626+0000","last_became_peered":"2024-09-19T08:13:00.845626+0000","last_unstale":"2024-09-19T08:13:01.832650+0000","last_undegraded":"2024-09-19T08:13:01.832650+0000","last_fullsized":"2024-09-19T08:13:01.832650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:36.945627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947938+0000","last_change":"2024-09-19T08:13:00.853851+0000","last_active":"2024-09-19T08:13:01.947938+0000","last_peered":"2024-09-19T08:13:01.947938+0000","last_clean":"2024-09-19T08:13:01.947938+0000","last_became_active":"2024-09-19T08:13:00.853189+0000","last_became_peered":"2024-09-19T08:13:00.853189+0000","last_unstale":"2024-09-19T08:13:01.947938+0000","last_undegraded":"2024-09-19T08:13:01.947938+0000","last_fullsized":"2024-09-19T08:13:01.947938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:04.812522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.946963+0000","last_change":"2024-09-19T08:13:00.848325+0000","last_active":"2024-09-19T08:13:01.946963+0000","last_peered":"2024-09-19T08:13:01.946963+0000","last_clean":"2024-09-19T08:13:01.946963+0000","last_became_active":"2024-09-19T08:13:00.848015+0000","last_became_peered":"2024-09-19T08:13:00.848015+0000","last_unstale":"2024-09-19T08:13:01.946963+0000","last_undegraded":"2024-09-19T08:13:01.946963+0000","last_fullsized":"2024-09-19T08:13:01.946963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:10.948451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:23.901131+0000","last_change":"2024-09-19T08:12:58.681134+0000","last_active":"2024-09-19T08:13:23.901131+0000","last_peered":"2024-09-19T08:13:23.901131+0000","last_clean":"2024-09-19T08:13:23.901131+0000","last_became_active":"2024-09-19T08:12:58.680764+0000","last_became_peered":"2024-09-19T08:12:58.680764+0000","last_unstale":"2024-09-19T08:13:23.901131+0000","last_undegraded":"2024-09-19T08:13:23.901131+0000","last_fullsized":"2024-09-19T08:13:23.901131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:24.939263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947299+0000","last_change":"2024-09-19T08:13:00.844319+0000","last_active":"2024-09-19T08:13:01.947299+0000","last_peered":"2024-09-19T08:13:01.947299+0000","last_clean":"2024-09-19T08:13:01.947299+0000","last_became_active":"2024-09-19T08:13:00.843994+0000","last_became_peered":"2024-09-19T08:13:00.843994+0000","last_unstale":"2024-09-19T08:13:01.947299+0000","last_undegraded":"2024-09-19T08:13:01.947299+0000","last_fullsized":"2024-09-19T08:13:01.947299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:46.937466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.880088+0000","last_change":"2024-09-19T08:12:58.676541+0000","last_active":"2024-09-19T08:13:10.880088+0000","last_peered":"2024-09-19T08:13:10.880088+0000","last_clean":"2024-09-19T08:13:10.880088+0000","last_became_active":"2024-09-19T08:12:58.675988+0000","last_became_peered":"2024-09-19T08:12:58.675988+0000","last_unstale":"2024-09-19T08:13:10.880088+0000","last_undegraded":"2024-09-19T08:13:10.880088+0000","last_fullsized":"2024-09-19T08:13:10.880088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:01.434986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.880054+0000","last_change":"2024-09-19T08:12:58.677510+0000","last_active":"2024-09-19T08:13:10.880054+0000","last_peered":"2024-09-19T08:13:10.880054+0000","last_clean":"2024-09-19T08:13:10.880054+0000","last_became_active":"2024-09-19T08:12:58.677200+0000","last_became_peered":"2024-09-19T08:12:58.677200+0000","last_unstale":"2024-09-19T08:13:10.880054+0000","last_undegraded":"2024-09-19T08:13:10.880054+0000","last_fullsized":"2024-09-19T08:13:10.880054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:36.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947353+0000","last_change":"2024-09-19T08:13:00.853430+0000","last_active":"2024-09-19T08:13:01.947353+0000","last_peered":"2024-09-19T08:13:01.947353+0000","last_clean":"2024-09-19T08:13:01.947353+0000","last_became_active":"2024-09-19T08:13:00.853096+0000","last_became_peered":"2024-09-19T08:13:00.853096+0000","last_unstale":"2024-09-19T08:13:01.947353+0000","last_undegraded":"2024-09-19T08:13:01.947353+0000","last_fullsized":"2024-09-19T08:13:01.947353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:07.147021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947748+0000","last_change":"2024-09-19T08:12:58.680081+0000","last_active":"2024-09-19T08:13:01.947748+0000","last_peered":"2024-09-19T08:13:01.947748+0000","last_clean":"2024-09-19T08:13:01.947748+0000","last_became_active":"2024-09-19T08:12:58.679704+0000","last_became_peered":"2024-09-19T08:12:58.679704+0000","last_unstale":"2024-09-19T08:13:01.947748+0000","last_undegraded":"2024-09-19T08:13:01.947748+0000","last_fullsized":"2024-09-19T08:13:01.947748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:32.942586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.831528+0000","last_change":"2024-09-19T08:13:00.849394+0000","last_active":"2024-09-19T08:13:01.831528+0000","last_peered":"2024-09-19T08:13:01.831528+0000","last_clean":"2024-09-19T08:13:01.831528+0000","last_became_active":"2024-09-19T08:13:00.849051+0000","last_became_peered":"2024-09-19T08:13:00.849051+0000","last_unstale":"2024-09-19T08:13:01.831528+0000","last_undegraded":"2024-09-19T08:13:01.831528+0000","last_fullsized":"2024-09-19T08:13:01.831528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:23.012928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:23.900485+0000","last_change":"2024-09-19T08:12:58.676121+0000","last_active":"2024-09-19T08:13:23.900485+0000","last_peered":"2024-09-19T08:13:23.900485+0000","last_clean":"2024-09-19T08:13:23.900485+0000","last_became_active":"2024-09-19T08:12:58.675693+0000","last_became_peered":"2024-09-19T08:12:58.675693+0000","last_unstale":"2024-09-19T08:13:23.900485+0000","last_undegraded":"2024-09-19T08:13:23.900485+0000","last_fullsized":"2024-09-19T08:13:23.900485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:34.882190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836478+0000","last_change":"2024-09-19T08:13:00.858735+0000","last_active":"2024-09-19T08:13:01.836478+0000","last_peered":"2024-09-19T08:13:01.836478+0000","last_clean":"2024-09-19T08:13:01.836478+0000","last_became_active":"2024-09-19T08:13:00.858218+0000","last_became_peered":"2024-09-19T08:13:00.858218+0000","last_unstale":"2024-09-19T08:13:01.836478+0000","last_undegraded":"2024-09-19T08:13:01.836478+0000","last_fullsized":"2024-09-19T08:13:01.836478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:33.356810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872605+0000","last_change":"2024-09-19T08:12:58.679624+0000","last_active":"2024-09-19T08:13:09.872605+0000","last_peered":"2024-09-19T08:13:09.872605+0000","last_clean":"2024-09-19T08:13:09.872605+0000","last_became_active":"2024-09-19T08:12:58.679057+0000","last_became_peered":"2024-09-19T08:12:58.679057+0000","last_unstale":"2024-09-19T08:13:09.872605+0000","last_undegraded":"2024-09-19T08:13:09.872605+0000","last_fullsized":"2024-09-19T08:13:09.872605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:12.391663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948868+0000","last_change":"2024-09-19T08:13:00.843852+0000","last_active":"2024-09-19T08:13:01.948868+0000","last_peered":"2024-09-19T08:13:01.948868+0000","last_clean":"2024-09-19T08:13:01.948868+0000","last_became_active":"2024-09-19T08:13:00.843497+0000","last_became_peered":"2024-09-19T08:13:00.843497+0000","last_unstale":"2024-09-19T08:13:01.948868+0000","last_undegraded":"2024-09-19T08:13:01.948868+0000","last_fullsized":"2024-09-19T08:13:01.948868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:26.380024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.873411+0000","last_change":"2024-09-19T08:12:58.717225+0000","last_active":"2024-09-19T08:13:09.873411+0000","last_peered":"2024-09-19T08:13:09.873411+0000","last_clean":"2024-09-19T08:13:09.873411+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:13:09.873411+0000","last_undegraded":"2024-09-19T08:13:09.873411+0000","last_fullsized":"2024-09-19T08:13:09.873411+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:43.201907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836422+0000","last_change":"2024-09-19T08:13:00.845613+0000","last_active":"2024-09-19T08:13:01.836422+0000","last_peered":"2024-09-19T08:13:01.836422+0000","last_clean":"2024-09-19T08:13:01.836422+0000","last_became_active":"2024-09-19T08:13:00.845260+0000","last_became_peered":"2024-09-19T08:13:00.845260+0000","last_unstale":"2024-09-19T08:13:01.836422+0000","last_undegraded":"2024-09-19T08:13:01.836422+0000","last_fullsized":"2024-09-19T08:13:01.836422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:03:47.033972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'22","reported_seq":74,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:24.903105+0000","last_change":"2024-09-19T08:12:58.677308+0000","last_active":"2024-09-19T08:13:24.903105+0000","last_peered":"2024-09-19T08:13:24.903105+0000","last_clean":"2024-09-19T08:13:24.903105+0000","last_became_active":"2024-09-19T08:12:58.676925+0000","last_became_peered":"2024-09-19T08:12:58.676925+0000","last_unstale":"2024-09-19T08:13:24.903105+0000","last_undegraded":"2024-09-19T08:13:24.903105+0000","last_fullsized":"2024-09-19T08:13:24.903105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:50.873656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":62,"num_write":23,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947397+0000","last_change":"2024-09-19T08:13:00.855603+0000","last_active":"2024-09-19T08:13:01.947397+0000","last_peered":"2024-09-19T08:13:01.947397+0000","last_clean":"2024-09-19T08:13:01.947397+0000","last_became_active":"2024-09-19T08:13:00.855241+0000","last_became_peered":"2024-09-19T08:13:00.855241+0000","last_unstale":"2024-09-19T08:13:01.947397+0000","last_undegraded":"2024-09-19T08:13:01.947397+0000","last_fullsized":"2024-09-19T08:13:01.947397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:28.811847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.881619+0000","last_change":"2024-09-19T08:12:58.675897+0000","last_active":"2024-09-19T08:13:10.881619+0000","last_peered":"2024-09-19T08:13:10.881619+0000","last_clean":"2024-09-19T08:13:10.881619+0000","last_became_active":"2024-09-19T08:12:58.675525+0000","last_became_peered":"2024-09-19T08:12:58.675525+0000","last_unstale":"2024-09-19T08:13:10.881619+0000","last_undegraded":"2024-09-19T08:13:10.881619+0000","last_fullsized":"2024-09-19T08:13:10.881619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:54.518064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947169+0000","last_change":"2024-09-19T08:13:00.856420+0000","last_active":"2024-09-19T08:13:01.947169+0000","last_peered":"2024-09-19T08:13:01.947169+0000","last_clean":"2024-09-19T08:13:01.947169+0000","last_became_active":"2024-09-19T08:13:00.856100+0000","last_became_peered":"2024-09-19T08:13:00.856100+0000","last_unstale":"2024-09-19T08:13:01.947169+0000","last_undegraded":"2024-09-19T08:13:01.947169+0000","last_fullsized":"2024-09-19T08:13:01.947169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:58.684930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872838+0000","last_change":"2024-09-19T08:12:58.680778+0000","last_active":"2024-09-19T08:13:09.872838+0000","last_peered":"2024-09-19T08:13:09.872838+0000","last_clean":"2024-09-19T08:13:09.872838+0000","last_became_active":"2024-09-19T08:12:58.680364+0000","last_became_peered":"2024-09-19T08:12:58.680364+0000","last_unstale":"2024-09-19T08:13:09.872838+0000","last_undegraded":"2024-09-19T08:13:09.872838+0000","last_fullsized":"2024-09-19T08:13:09.872838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:10.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100651+0000","last_change":"2024-09-19T08:13:00.853415+0000","last_active":"2024-09-19T08:13:02.100651+0000","last_peered":"2024-09-19T08:13:02.100651+0000","last_clean":"2024-09-19T08:13:02.100651+0000","last_became_active":"2024-09-19T08:13:00.853101+0000","last_became_peered":"2024-09-19T08:13:00.853101+0000","last_unstale":"2024-09-19T08:13:02.100651+0000","last_undegraded":"2024-09-19T08:13:02.100651+0000","last_fullsized":"2024-09-19T08:13:02.100651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:42.375394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:22.899868+0000","last_change":"2024-09-19T08:12:58.680312+0000","last_active":"2024-09-19T08:13:22.899868+0000","last_peered":"2024-09-19T08:13:22.899868+0000","last_clean":"2024-09-19T08:13:22.899868+0000","last_became_active":"2024-09-19T08:12:58.679870+0000","last_became_peered":"2024-09-19T08:12:58.679870+0000","last_unstale":"2024-09-19T08:13:22.899868+0000","last_undegraded":"2024-09-19T08:13:22.899868+0000","last_fullsized":"2024-09-19T08:13:22.899868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:42.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100329+0000","last_change":"2024-09-19T08:13:00.853920+0000","last_active":"2024-09-19T08:13:02.100329+0000","last_peered":"2024-09-19T08:13:02.100329+0000","last_clean":"2024-09-19T08:13:02.100329+0000","last_became_active":"2024-09-19T08:13:00.853594+0000","last_became_peered":"2024-09-19T08:13:00.853594+0000","last_unstale":"2024-09-19T08:13:02.100329+0000","last_undegraded":"2024-09-19T08:13:02.100329+0000","last_fullsized":"2024-09-19T08:13:02.100329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:31.746133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.882028+0000","last_change":"2024-09-19T08:12:58.717487+0000","last_active":"2024-09-19T08:13:10.882028+0000","last_peered":"2024-09-19T08:13:10.882028+0000","last_clean":"2024-09-19T08:13:10.882028+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:13:10.882028+0000","last_undegraded":"2024-09-19T08:13:10.882028+0000","last_fullsized":"2024-09-19T08:13:10.882028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:38.506051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947060+0000","last_change":"2024-09-19T08:13:00.847061+0000","last_active":"2024-09-19T08:13:01.947060+0000","last_peered":"2024-09-19T08:13:01.947060+0000","last_clean":"2024-09-19T08:13:01.947060+0000","last_became_active":"2024-09-19T08:13:00.846742+0000","last_became_peered":"2024-09-19T08:13:00.846742+0000","last_unstale":"2024-09-19T08:13:01.947060+0000","last_undegraded":"2024-09-19T08:13:01.947060+0000","last_fullsized":"2024-09-19T08:13:01.947060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:21.785585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:11.877000+0000","last_change":"2024-09-19T08:12:58.679367+0000","last_active":"2024-09-19T08:13:11.877000+0000","last_peered":"2024-09-19T08:13:11.877000+0000","last_clean":"2024-09-19T08:13:11.877000+0000","last_became_active":"2024-09-19T08:12:58.679035+0000","last_became_peered":"2024-09-19T08:12:58.679035+0000","last_unstale":"2024-09-19T08:13:11.877000+0000","last_undegraded":"2024-09-19T08:13:11.877000+0000","last_fullsized":"2024-09-19T08:13:11.877000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:25.633215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948364+0000","last_change":"2024-09-19T08:13:00.844940+0000","last_active":"2024-09-19T08:13:01.948364+0000","last_peered":"2024-09-19T08:13:01.948364+0000","last_clean":"2024-09-19T08:13:01.948364+0000","last_became_active":"2024-09-19T08:13:00.844454+0000","last_became_peered":"2024-09-19T08:13:00.844454+0000","last_unstale":"2024-09-19T08:13:01.948364+0000","last_undegraded":"2024-09-19T08:13:01.948364+0000","last_fullsized":"2024-09-19T08:13:01.948364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:15.577624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.272636+0000","last_change":"2024-09-19T08:12:58.673301+0000","last_active":"2024-09-19T08:13:02.272636+0000","last_peered":"2024-09-19T08:13:02.272636+0000","last_clean":"2024-09-19T08:13:02.272636+0000","last_became_active":"2024-09-19T08:12:58.672981+0000","last_became_peered":"2024-09-19T08:12:58.672981+0000","last_unstale":"2024-09-19T08:13:02.272636+0000","last_undegraded":"2024-09-19T08:13:02.272636+0000","last_fullsized":"2024-09-19T08:13:02.272636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:03.963712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100834+0000","last_change":"2024-09-19T08:13:00.856620+0000","last_active":"2024-09-19T08:13:02.100834+0000","last_peered":"2024-09-19T08:13:02.100834+0000","last_clean":"2024-09-19T08:13:02.100834+0000","last_became_active":"2024-09-19T08:13:00.856310+0000","last_became_peered":"2024-09-19T08:13:00.856310+0000","last_unstale":"2024-09-19T08:13:02.100834+0000","last_undegraded":"2024-09-19T08:13:02.100834+0000","last_fullsized":"2024-09-19T08:13:02.100834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:16.918768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.271923+0000","last_change":"2024-09-19T08:12:58.672282+0000","last_active":"2024-09-19T08:13:02.271923+0000","last_peered":"2024-09-19T08:13:02.271923+0000","last_clean":"2024-09-19T08:13:02.271923+0000","last_became_active":"2024-09-19T08:12:58.671891+0000","last_became_peered":"2024-09-19T08:12:58.671891+0000","last_unstale":"2024-09-19T08:13:02.271923+0000","last_undegraded":"2024-09-19T08:13:02.271923+0000","last_fullsized":"2024-09-19T08:13:02.271923+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:41.731571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100814+0000","last_change":"2024-09-19T08:13:00.848107+0000","last_active":"2024-09-19T08:13:02.100814+0000","last_peered":"2024-09-19T08:13:02.100814+0000","last_clean":"2024-09-19T08:13:02.100814+0000","last_became_active":"2024-09-19T08:13:00.847632+0000","last_became_peered":"2024-09-19T08:13:00.847632+0000","last_unstale":"2024-09-19T08:13:02.100814+0000","last_undegraded":"2024-09-19T08:13:02.100814+0000","last_fullsized":"2024-09-19T08:13:02.100814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.491122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.880827+0000","last_change":"2024-09-19T08:12:59.179200+0000","last_active":"2024-09-19T08:13:10.880827+0000","last_peered":"2024-09-19T08:13:10.880827+0000","last_clean":"2024-09-19T08:13:10.880827+0000","last_became_active":"2024-09-19T08:12:59.178762+0000","last_became_peered":"2024-09-19T08:12:59.178762+0000","last_unstale":"2024-09-19T08:13:10.880827+0000","last_undegraded":"2024-09-19T08:13:10.880827+0000","last_fullsized":"2024-09-19T08:13:10.880827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:08.424106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100825+0000","last_change":"2024-09-19T08:13:00.855870+0000","last_active":"2024-09-19T08:13:02.100825+0000","last_peered":"2024-09-19T08:13:02.100825+0000","last_clean":"2024-09-19T08:13:02.100825+0000","last_became_active":"2024-09-19T08:13:00.855538+0000","last_became_peered":"2024-09-19T08:13:00.855538+0000","last_unstale":"2024-09-19T08:13:02.100825+0000","last_undegraded":"2024-09-19T08:13:02.100825+0000","last_fullsized":"2024-09-19T08:13:02.100825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100488+0000","last_change":"2024-09-19T08:11:48.661333+0000","last_active":"2024-09-19T08:13:02.100488+0000","last_peered":"2024-09-19T08:13:02.100488+0000","last_clean":"2024-09-19T08:13:02.100488+0000","last_became_active":"2024-09-19T08:11:48.660873+0000","last_became_peered":"2024-09-19T08:11:48.660873+0000","last_unstale":"2024-09-19T08:13:02.100488+0000","last_undegraded":"2024-09-19T08:13:02.100488+0000","last_fullsized":"2024-09-19T08:13:02.100488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_clean_scrub_stamp":"2024-09-19T08:08:25.372155+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:19.401292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:11.881883+0000","last_change":"2024-09-19T08:12:58.675237+0000","last_active":"2024-09-19T08:13:11.881883+0000","last_peered":"2024-09-19T08:13:11.881883+0000","last_clean":"2024-09-19T08:13:11.881883+0000","last_became_active":"2024-09-19T08:12:58.674910+0000","last_became_peered":"2024-09-19T08:12:58.674910+0000","last_unstale":"2024-09-19T08:13:11.881883+0000","last_undegraded":"2024-09-19T08:13:11.881883+0000","last_fullsized":"2024-09-19T08:13:11.881883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:47.038328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947516+0000","last_change":"2024-09-19T08:13:00.846754+0000","last_active":"2024-09-19T08:13:01.947516+0000","last_peered":"2024-09-19T08:13:01.947516+0000","last_clean":"2024-09-19T08:13:01.947516+0000","last_became_active":"2024-09-19T08:13:00.846452+0000","last_became_peered":"2024-09-19T08:13:00.846452+0000","last_unstale":"2024-09-19T08:13:01.947516+0000","last_undegraded":"2024-09-19T08:13:01.947516+0000","last_fullsized":"2024-09-19T08:13:01.947516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:22.929689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.881292+0000","last_change":"2024-09-19T08:12:58.678958+0000","last_active":"2024-09-19T08:13:10.881292+0000","last_peered":"2024-09-19T08:13:10.881292+0000","last_clean":"2024-09-19T08:13:10.881292+0000","last_became_active":"2024-09-19T08:12:58.678592+0000","last_became_peered":"2024-09-19T08:12:58.678592+0000","last_unstale":"2024-09-19T08:13:10.881292+0000","last_undegraded":"2024-09-19T08:13:10.881292+0000","last_fullsized":"2024-09-19T08:13:10.881292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:58.374606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947087+0000","last_change":"2024-09-19T08:13:00.840973+0000","last_active":"2024-09-19T08:13:01.947087+0000","last_peered":"2024-09-19T08:13:01.947087+0000","last_clean":"2024-09-19T08:13:01.947087+0000","last_became_active":"2024-09-19T08:13:00.840619+0000","last_became_peered":"2024-09-19T08:13:00.840619+0000","last_unstale":"2024-09-19T08:13:01.947087+0000","last_undegraded":"2024-09-19T08:13:01.947087+0000","last_fullsized":"2024-09-19T08:13:01.947087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:23.900583+0000","last_change":"2024-09-19T08:12:58.715972+0000","last_active":"2024-09-19T08:13:23.900583+0000","last_peered":"2024-09-19T08:13:23.900583+0000","last_clean":"2024-09-19T08:13:23.900583+0000","last_became_active":"2024-09-19T08:12:58.715630+0000","last_became_peered":"2024-09-19T08:12:58.715630+0000","last_unstale":"2024-09-19T08:13:23.900583+0000","last_undegraded":"2024-09-19T08:13:23.900583+0000","last_fullsized":"2024-09-19T08:13:23.900583+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:35.637121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947439+0000","last_change":"2024-09-19T08:13:00.847023+0000","last_active":"2024-09-19T08:13:01.947439+0000","last_peered":"2024-09-19T08:13:01.947439+0000","last_clean":"2024-09-19T08:13:01.947439+0000","last_became_active":"2024-09-19T08:13:00.846387+0000","last_became_peered":"2024-09-19T08:13:00.846387+0000","last_unstale":"2024-09-19T08:13:01.947439+0000","last_undegraded":"2024-09-19T08:13:01.947439+0000","last_fullsized":"2024-09-19T08:13:01.947439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:12.187929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100987+0000","last_change":"2024-09-19T08:12:58.681062+0000","last_active":"2024-09-19T08:13:02.100987+0000","last_peered":"2024-09-19T08:13:02.100987+0000","last_clean":"2024-09-19T08:13:02.100987+0000","last_became_active":"2024-09-19T08:12:58.680524+0000","last_became_peered":"2024-09-19T08:12:58.680524+0000","last_unstale":"2024-09-19T08:13:02.100987+0000","last_undegraded":"2024-09-19T08:13:02.100987+0000","last_fullsized":"2024-09-19T08:13:02.100987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947809+0000","last_change":"2024-09-19T08:13:00.851523+0000","last_active":"2024-09-19T08:13:01.947809+0000","last_peered":"2024-09-19T08:13:01.947809+0000","last_clean":"2024-09-19T08:13:01.947809+0000","last_became_active":"2024-09-19T08:13:00.851068+0000","last_became_peered":"2024-09-19T08:13:00.851068+0000","last_unstale":"2024-09-19T08:13:01.947809+0000","last_undegraded":"2024-09-19T08:13:01.947809+0000","last_fullsized":"2024-09-19T08:13:01.947809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:44.905469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.831886+0000","last_change":"2024-09-19T08:12:58.718234+0000","last_active":"2024-09-19T08:13:01.831886+0000","last_peered":"2024-09-19T08:13:01.831886+0000","last_clean":"2024-09-19T08:13:01.831886+0000","last_became_active":"2024-09-19T08:12:58.717840+0000","last_became_peered":"2024-09-19T08:12:58.717840+0000","last_unstale":"2024-09-19T08:13:01.831886+0000","last_undegraded":"2024-09-19T08:13:01.831886+0000","last_fullsized":"2024-09-19T08:13:01.831886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:05.283770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947784+0000","last_change":"2024-09-19T08:13:00.848305+0000","last_active":"2024-09-19T08:13:01.947784+0000","last_peered":"2024-09-19T08:13:01.947784+0000","last_clean":"2024-09-19T08:13:01.947784+0000","last_became_active":"2024-09-19T08:13:00.847981+0000","last_became_peered":"2024-09-19T08:13:00.847981+0000","last_unstale":"2024-09-19T08:13:01.947784+0000","last_undegraded":"2024-09-19T08:13:01.947784+0000","last_fullsized":"2024-09-19T08:13:01.947784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:04.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.871966+0000","last_change":"2024-09-19T08:12:58.715943+0000","last_active":"2024-09-19T08:13:09.871966+0000","last_peered":"2024-09-19T08:13:09.871966+0000","last_clean":"2024-09-19T08:13:09.871966+0000","last_became_active":"2024-09-19T08:12:58.715600+0000","last_became_peered":"2024-09-19T08:12:58.715600+0000","last_unstale":"2024-09-19T08:13:09.871966+0000","last_undegraded":"2024-09-19T08:13:09.871966+0000","last_fullsized":"2024-09-19T08:13:09.871966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:19.142271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948126+0000","last_change":"2024-09-19T08:13:00.840944+0000","last_active":"2024-09-19T08:13:01.948126+0000","last_peered":"2024-09-19T08:13:01.948126+0000","last_clean":"2024-09-19T08:13:01.948126+0000","last_became_active":"2024-09-19T08:13:00.840608+0000","last_became_peered":"2024-09-19T08:13:00.840608+0000","last_unstale":"2024-09-19T08:13:01.948126+0000","last_undegraded":"2024-09-19T08:13:01.948126+0000","last_fullsized":"2024-09-19T08:13:01.948126+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:25.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'9","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:23.268827+0000","last_change":"2024-09-19T08:12:58.672929+0000","last_active":"2024-09-19T08:13:23.268827+0000","last_peered":"2024-09-19T08:13:23.268827+0000","last_clean":"2024-09-19T08:13:23.268827+0000","last_became_active":"2024-09-19T08:12:58.672487+0000","last_became_peered":"2024-09-19T08:12:58.672487+0000","last_unstale":"2024-09-19T08:13:23.268827+0000","last_undegraded":"2024-09-19T08:13:23.268827+0000","last_fullsized":"2024-09-19T08:13:23.268827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:33.042735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947672+0000","last_change":"2024-09-19T08:13:00.853457+0000","last_active":"2024-09-19T08:13:01.947672+0000","last_peered":"2024-09-19T08:13:01.947672+0000","last_clean":"2024-09-19T08:13:01.947672+0000","last_became_active":"2024-09-19T08:13:00.853159+0000","last_became_peered":"2024-09-19T08:13:00.853159+0000","last_unstale":"2024-09-19T08:13:01.947672+0000","last_undegraded":"2024-09-19T08:13:01.947672+0000","last_fullsized":"2024-09-19T08:13:01.947672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:33.899170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.869690+0000","last_change":"2024-09-19T08:12:59.179510+0000","last_active":"2024-09-19T08:13:09.869690+0000","last_peered":"2024-09-19T08:13:09.869690+0000","last_clean":"2024-09-19T08:13:09.869690+0000","last_became_active":"2024-09-19T08:12:59.179157+0000","last_became_peered":"2024-09-19T08:12:59.179157+0000","last_unstale":"2024-09-19T08:13:09.869690+0000","last_undegraded":"2024-09-19T08:13:09.869690+0000","last_fullsized":"2024-09-19T08:13:09.869690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:22.981289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948760+0000","last_change":"2024-09-19T08:13:00.841853+0000","last_active":"2024-09-19T08:13:01.948760+0000","last_peered":"2024-09-19T08:13:01.948760+0000","last_clean":"2024-09-19T08:13:01.948760+0000","last_became_active":"2024-09-19T08:13:00.841518+0000","last_became_peered":"2024-09-19T08:13:00.841518+0000","last_unstale":"2024-09-19T08:13:01.948760+0000","last_undegraded":"2024-09-19T08:13:01.948760+0000","last_fullsized":"2024-09-19T08:13:01.948760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:05.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872283+0000","last_change":"2024-09-19T08:12:58.716882+0000","last_active":"2024-09-19T08:13:09.872283+0000","last_peered":"2024-09-19T08:13:09.872283+0000","last_clean":"2024-09-19T08:13:09.872283+0000","last_became_active":"2024-09-19T08:12:58.716545+0000","last_became_peered":"2024-09-19T08:12:58.716545+0000","last_unstale":"2024-09-19T08:13:09.872283+0000","last_undegraded":"2024-09-19T08:13:09.872283+0000","last_fullsized":"2024-09-19T08:13:09.872283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:14.469109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.946790+0000","last_change":"2024-09-19T08:13:00.850484+0000","last_active":"2024-09-19T08:13:01.946790+0000","last_peered":"2024-09-19T08:13:01.946790+0000","last_clean":"2024-09-19T08:13:01.946790+0000","last_became_active":"2024-09-19T08:13:00.850049+0000","last_became_peered":"2024-09-19T08:13:00.850049+0000","last_unstale":"2024-09-19T08:13:01.946790+0000","last_undegraded":"2024-09-19T08:13:01.946790+0000","last_fullsized":"2024-09-19T08:13:01.946790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:24.424111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.870765+0000","last_change":"2024-09-19T08:12:58.669928+0000","last_active":"2024-09-19T08:13:09.870765+0000","last_peered":"2024-09-19T08:13:09.870765+0000","last_clean":"2024-09-19T08:13:09.870765+0000","last_became_active":"2024-09-19T08:12:58.669614+0000","last_became_peered":"2024-09-19T08:12:58.669614+0000","last_unstale":"2024-09-19T08:13:09.870765+0000","last_undegraded":"2024-09-19T08:13:09.870765+0000","last_fullsized":"2024-09-19T08:13:09.870765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:50.924779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948102+0000","last_change":"2024-09-19T08:13:00.853520+0000","last_active":"2024-09-19T08:13:01.948102+0000","last_peered":"2024-09-19T08:13:01.948102+0000","last_clean":"2024-09-19T08:13:01.948102+0000","last_became_active":"2024-09-19T08:13:00.853184+0000","last_became_peered":"2024-09-19T08:13:00.853184+0000","last_unstale":"2024-09-19T08:13:01.948102+0000","last_undegraded":"2024-09-19T08:13:01.948102+0000","last_fullsized":"2024-09-19T08:13:01.948102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:45.160439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.832515+0000","last_change":"2024-09-19T08:12:58.674359+0000","last_active":"2024-09-19T08:13:01.832515+0000","last_peered":"2024-09-19T08:13:01.832515+0000","last_clean":"2024-09-19T08:13:01.832515+0000","last_became_active":"2024-09-19T08:12:58.673890+0000","last_became_peered":"2024-09-19T08:12:58.673890+0000","last_unstale":"2024-09-19T08:13:01.832515+0000","last_undegraded":"2024-09-19T08:13:01.832515+0000","last_fullsized":"2024-09-19T08:13:01.832515+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:54.939959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947708+0000","last_change":"2024-09-19T08:13:00.851071+0000","last_active":"2024-09-19T08:13:01.947708+0000","last_peered":"2024-09-19T08:13:01.947708+0000","last_clean":"2024-09-19T08:13:01.947708+0000","last_became_active":"2024-09-19T08:13:00.850756+0000","last_became_peered":"2024-09-19T08:13:00.850756+0000","last_unstale":"2024-09-19T08:13:01.947708+0000","last_undegraded":"2024-09-19T08:13:01.947708+0000","last_fullsized":"2024-09-19T08:13:01.947708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:52.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872854+0000","last_change":"2024-09-19T08:12:58.681466+0000","last_active":"2024-09-19T08:13:09.872854+0000","last_peered":"2024-09-19T08:13:09.872854+0000","last_clean":"2024-09-19T08:13:09.872854+0000","last_became_active":"2024-09-19T08:12:58.681113+0000","last_became_peered":"2024-09-19T08:12:58.681113+0000","last_unstale":"2024-09-19T08:13:09.872854+0000","last_undegraded":"2024-09-19T08:13:09.872854+0000","last_fullsized":"2024-09-19T08:13:09.872854+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:49.610828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100583+0000","last_change":"2024-09-19T08:13:00.851272+0000","last_active":"2024-09-19T08:13:02.100583+0000","last_peered":"2024-09-19T08:13:02.100583+0000","last_clean":"2024-09-19T08:13:02.100583+0000","last_became_active":"2024-09-19T08:13:00.849098+0000","last_became_peered":"2024-09-19T08:13:00.849098+0000","last_unstale":"2024-09-19T08:13:02.100583+0000","last_undegraded":"2024-09-19T08:13:02.100583+0000","last_fullsized":"2024-09-19T08:13:02.100583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:05.643992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947777+0000","last_change":"2024-09-19T08:13:00.849945+0000","last_active":"2024-09-19T08:13:01.947777+0000","last_peered":"2024-09-19T08:13:01.947777+0000","last_clean":"2024-09-19T08:13:01.947777+0000","last_became_active":"2024-09-19T08:13:00.849591+0000","last_became_peered":"2024-09-19T08:13:00.849591+0000","last_unstale":"2024-09-19T08:13:01.947777+0000","last_undegraded":"2024-09-19T08:13:01.947777+0000","last_fullsized":"2024-09-19T08:13:01.947777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:27.434005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.272564+0000","last_change":"2024-09-19T08:12:58.674610+0000","last_active":"2024-09-19T08:13:02.272564+0000","last_peered":"2024-09-19T08:13:02.272564+0000","last_clean":"2024-09-19T08:13:02.272564+0000","last_became_active":"2024-09-19T08:12:58.674247+0000","last_became_peered":"2024-09-19T08:12:58.674247+0000","last_unstale":"2024-09-19T08:13:02.272564+0000","last_undegraded":"2024-09-19T08:13:02.272564+0000","last_fullsized":"2024-09-19T08:13:02.272564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:16.323574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836419+0000","last_change":"2024-09-19T08:13:00.858512+0000","last_active":"2024-09-19T08:13:01.836419+0000","last_peered":"2024-09-19T08:13:01.836419+0000","last_clean":"2024-09-19T08:13:01.836419+0000","last_became_active":"2024-09-19T08:13:00.857897+0000","last_became_peered":"2024-09-19T08:13:00.857897+0000","last_unstale":"2024-09-19T08:13:01.836419+0000","last_undegraded":"2024-09-19T08:13:01.836419+0000","last_fullsized":"2024-09-19T08:13:01.836419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.881153+0000","last_change":"2024-09-19T08:12:58.682207+0000","last_active":"2024-09-19T08:13:10.881153+0000","last_peered":"2024-09-19T08:13:10.881153+0000","last_clean":"2024-09-19T08:13:10.881153+0000","last_became_active":"2024-09-19T08:12:58.681842+0000","last_became_peered":"2024-09-19T08:12:58.681842+0000","last_unstale":"2024-09-19T08:13:10.881153+0000","last_undegraded":"2024-09-19T08:13:10.881153+0000","last_fullsized":"2024-09-19T08:13:10.881153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.825858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948835+0000","last_change":"2024-09-19T08:13:00.846748+0000","last_active":"2024-09-19T08:13:01.948835+0000","last_peered":"2024-09-19T08:13:01.948835+0000","last_clean":"2024-09-19T08:13:01.948835+0000","last_became_active":"2024-09-19T08:13:00.846368+0000","last_became_peered":"2024-09-19T08:13:00.846368+0000","last_unstale":"2024-09-19T08:13:01.948835+0000","last_undegraded":"2024-09-19T08:13:01.948835+0000","last_fullsized":"2024-09-19T08:13:01.948835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:59.526512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.271849+0000","last_change":"2024-09-19T08:12:58.716118+0000","last_active":"2024-09-19T08:13:02.271849+0000","last_peered":"2024-09-19T08:13:02.271849+0000","last_clean":"2024-09-19T08:13:02.271849+0000","last_became_active":"2024-09-19T08:12:58.715711+0000","last_became_peered":"2024-09-19T08:12:58.715711+0000","last_unstale":"2024-09-19T08:13:02.271849+0000","last_undegraded":"2024-09-19T08:13:02.271849+0000","last_fullsized":"2024-09-19T08:13:02.271849+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:29.439887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947595+0000","last_change":"2024-09-19T08:13:00.848729+0000","last_active":"2024-09-19T08:13:01.947595+0000","last_peered":"2024-09-19T08:13:01.947595+0000","last_clean":"2024-09-19T08:13:01.947595+0000","last_became_active":"2024-09-19T08:13:00.848470+0000","last_became_peered":"2024-09-19T08:13:00.848470+0000","last_unstale":"2024-09-19T08:13:01.947595+0000","last_undegraded":"2024-09-19T08:13:01.947595+0000","last_fullsized":"2024-09-19T08:13:01.947595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:30.653101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.882192+0000","last_change":"2024-09-19T08:12:58.681654+0000","last_active":"2024-09-19T08:13:10.882192+0000","last_peered":"2024-09-19T08:13:10.882192+0000","last_clean":"2024-09-19T08:13:10.882192+0000","last_became_active":"2024-09-19T08:12:58.681284+0000","last_became_peered":"2024-09-19T08:12:58.681284+0000","last_unstale":"2024-09-19T08:13:10.882192+0000","last_undegraded":"2024-09-19T08:13:10.882192+0000","last_fullsized":"2024-09-19T08:13:10.882192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.429930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":51198,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":109,"num_read_kb":110,"num_write":153,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":161237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251470,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481809,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":428609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710742,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":1067285,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874268,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":428845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":909312,"data_stored":428765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1021090,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":428234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":1020381,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":429064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":913408,"data_stored":428545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":484215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":474829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":429505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34755,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34755,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":46802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":48773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:13:29.903 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T08:13:29.904 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T08:13:29.904 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T08:13:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:30.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:30 smithi022 ceph-mon[30963]: pgmap v294: 97 pgs: 97 active+clean; 627 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-09-19T08:13:30.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:30 smithi022 ceph-mon[30963]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:13:30.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:30 smithi064 ceph-mon[34900]: pgmap v294: 97 pgs: 97 active+clean; 627 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-09-19T08:13:30.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:30 smithi064 ceph-mon[34900]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:13:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.661+0000 7f432fefd640 1 -- 172.21.15.22:0/1909749417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4328100b10 msgr2=0x7f4328100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.661+0000 7f432fefd640 1 --2- 172.21.15.22:0/1909749417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4328100b10 0x7f4328100f10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4324009900 tx=0x7f432402a410 comp rx=0 tx=0).stop 2024-09-19T08:13:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.661+0000 7f432fefd640 1 -- 172.21.15.22:0/1909749417 shutdown_connections 2024-09-19T08:13:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.661+0000 7f432fefd640 1 --2- 172.21.15.22:0/1909749417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43281026d0 0x7f4328108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.661+0000 7f432fefd640 1 --2- 172.21.15.22:0/1909749417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4328101d10 0x7f4328102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.661+0000 7f432fefd640 1 --2- 172.21.15.22:0/1909749417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4328100b10 0x7f4328100f10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:30.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.661+0000 7f432fefd640 1 -- 172.21.15.22:0/1909749417 >> 172.21.15.22:0/1909749417 conn(0x7f43280fc2a0 msgr2=0x7f43280fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.661+0000 7f432fefd640 1 -- 172.21.15.22:0/1909749417 shutdown_connections 2024-09-19T08:13:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.661+0000 7f432fefd640 1 -- 172.21.15.22:0/1909749417 wait complete. 2024-09-19T08:13:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.661+0000 7f432fefd640 1 Processor -- start 2024-09-19T08:13:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432fefd640 1 -- start start 2024-09-19T08:13:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432fefd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4328100b10 0x7f432810b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:30.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432fefd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4328101d10 0x7f4328109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:30.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432fefd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43281026d0 0x7f432810a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:30.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432fefd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f432810d980 con 0x7f43281026d0 2024-09-19T08:13:30.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432fefd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f432810d800 con 0x7f4328100b10 2024-09-19T08:13:30.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432fefd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f432810db00 con 0x7f4328101d10 2024-09-19T08:13:30.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432e49a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43281026d0 0x7f432810a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:30.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432e49a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43281026d0 0x7f432810a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46548/0 (socket says 172.21.15.22:46548) 2024-09-19T08:13:30.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432e49a640 1 -- 172.21.15.22:0/2694715338 learned_addr learned my addr 172.21.15.22:0/2694715338 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:30.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432dc99640 1 --2- 172.21.15.22:0/2694715338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4328100b10 0x7f432810b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:30.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432d498640 1 --2- 172.21.15.22:0/2694715338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4328101d10 0x7f4328109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:30.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432e49a640 1 -- 172.21.15.22:0/2694715338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4328101d10 msgr2=0x7f4328109ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:30.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432e49a640 1 --2- 172.21.15.22:0/2694715338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4328101d10 0x7f4328109ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:30.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432e49a640 1 -- 172.21.15.22:0/2694715338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4328100b10 msgr2=0x7f432810b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:30.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432e49a640 1 --2- 172.21.15.22:0/2694715338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4328100b10 0x7f432810b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:30.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432e49a640 1 -- 172.21.15.22:0/2694715338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432810a8e0 con 0x7f43281026d0 2024-09-19T08:13:30.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.662+0000 7f432e49a640 1 --2- 172.21.15.22:0/2694715338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43281026d0 0x7f432810a020 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f432000b970 tx=0x7f432000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:30.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.663+0000 7f431effd640 1 -- 172.21.15.22:0/2694715338 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320019070 con 0x7f43281026d0 2024-09-19T08:13:30.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.663+0000 7f432fefd640 1 -- 172.21.15.22:0/2694715338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43281a2220 con 0x7f43281026d0 2024-09-19T08:13:30.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.663+0000 7f431effd640 1 -- 172.21.15.22:0/2694715338 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f432000cb40 con 0x7f43281026d0 2024-09-19T08:13:30.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.663+0000 7f431effd640 1 -- 172.21.15.22:0/2694715338 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320007650 con 0x7f43281026d0 2024-09-19T08:13:30.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.663+0000 7f432fefd640 1 -- 172.21.15.22:0/2694715338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43281a2760 con 0x7f43281026d0 2024-09-19T08:13:30.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.664+0000 7f432fefd640 1 -- 172.21.15.22:0/2694715338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4328108000 con 0x7f43281026d0 2024-09-19T08:13:30.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.665+0000 7f431effd640 1 -- 172.21.15.22:0/2694715338 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f432001c070 con 0x7f43281026d0 2024-09-19T08:13:30.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.666+0000 7f431effd640 1 --2- 172.21.15.22:0/2694715338 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f43140a32d0 0x7f43140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:30.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.666+0000 7f432dc99640 1 --2- 172.21.15.22:0/2694715338 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f43140a32d0 0x7f43140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:30.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.666+0000 7f431effd640 1 -- 172.21.15.22:0/2694715338 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f43200ced80 con 0x7f43281026d0 2024-09-19T08:13:30.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.666+0000 7f432dc99640 1 --2- 172.21.15.22:0/2694715338 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f43140a32d0 0x7f43140a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4324005ab0 tx=0x7f4324000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:30.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.670+0000 7f431effd640 1 -- 172.21.15.22:0/2694715338 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f432009b5b0 con 0x7f43281026d0 2024-09-19T08:13:30.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:30 smithi157 ceph-mon[35143]: pgmap v294: 97 pgs: 97 active+clean; 627 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-09-19T08:13:30.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:30 smithi157 ceph-mon[35143]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:13:30.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.914+0000 7f432fefd640 1 -- 172.21.15.22:0/2694715338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f4328062df0 con 0x7f43281026d0 2024-09-19T08:13:30.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.916+0000 7f431effd640 1 -- 172.21.15.22:0/2694715338 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f43200d0050 con 0x7f43281026d0 2024-09-19T08:13:30.917 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-09-19T08:13:30.917 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQAq3etmDdyYNhAAdZ2mH9oV8YTykacLeEOSbw== 2024-09-19T08:13:30.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.918+0000 7f432fefd640 1 -- 172.21.15.22:0/2694715338 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f43140a32d0 msgr2=0x7f43140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:30.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.918+0000 7f432fefd640 1 --2- 172.21.15.22:0/2694715338 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f43140a32d0 0x7f43140a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4324005ab0 tx=0x7f4324000bc0 comp rx=0 tx=0).stop 2024-09-19T08:13:30.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.918+0000 7f432fefd640 1 -- 172.21.15.22:0/2694715338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43281026d0 msgr2=0x7f432810a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:30.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.918+0000 7f432fefd640 1 --2- 172.21.15.22:0/2694715338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43281026d0 0x7f432810a020 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f432000b970 tx=0x7f432000be30 comp rx=0 tx=0).stop 2024-09-19T08:13:30.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.919+0000 7f432fefd640 1 -- 172.21.15.22:0/2694715338 shutdown_connections 2024-09-19T08:13:30.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.919+0000 7f432fefd640 1 --2- 172.21.15.22:0/2694715338 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f43140a32d0 0x7f43140a5790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:30.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.919+0000 7f432fefd640 1 --2- 172.21.15.22:0/2694715338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43281026d0 0x7f432810a020 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:30.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.919+0000 7f432fefd640 1 --2- 172.21.15.22:0/2694715338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4328101d10 0x7f4328109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:30.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.919+0000 7f432fefd640 1 --2- 172.21.15.22:0/2694715338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4328100b10 0x7f432810b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:30.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.919+0000 7f432fefd640 1 -- 172.21.15.22:0/2694715338 >> 172.21.15.22:0/2694715338 conn(0x7f43280fc2a0 msgr2=0x7f43281051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:30.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.919+0000 7f432fefd640 1 -- 172.21.15.22:0/2694715338 shutdown_connections 2024-09-19T08:13:30.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:30.919+0000 7f432fefd640 1 -- 172.21.15.22:0/2694715338 wait complete. 2024-09-19T08:13:31.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:31 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2694715338' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T08:13:31.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:31 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2694715338' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T08:13:31.693 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:31 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2694715338' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T08:13:31.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:31 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2694715338' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T08:13:31.723 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:13:31.723 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T08:13:31.724 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T08:13:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:31 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2694715338' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T08:13:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:31 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2694715338' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T08:13:31.763 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T08:13:31.764 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T08:13:31.764 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T08:13:32.059 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:32.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:32 smithi022 ceph-mon[30963]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-19T08:13:32.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.530+0000 7fb41db71640 1 -- 172.21.15.22:0/3067274070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4181036c0 msgr2=0x7fb41810bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:32.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.530+0000 7fb41db71640 1 --2- 172.21.15.22:0/3067274070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4181036c0 0x7fb41810bbb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009900 tx=0x7fb40c02a480 comp rx=0 tx=0).stop 2024-09-19T08:13:32.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.530+0000 7fb41db71640 1 -- 172.21.15.22:0/3067274070 shutdown_connections 2024-09-19T08:13:32.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.530+0000 7fb41db71640 1 --2- 172.21.15.22:0/3067274070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb41810c180 0x7fb41810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:32.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.530+0000 7fb41db71640 1 --2- 172.21.15.22:0/3067274070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4181036c0 0x7fb41810bbb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:32.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.530+0000 7fb41db71640 1 --2- 172.21.15.22:0/3067274070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb418102da0 0x7fb418103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:32.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.530+0000 7fb41db71640 1 -- 172.21.15.22:0/3067274070 >> 172.21.15.22:0/3067274070 conn(0x7fb4180fc7a0 msgr2=0x7fb4180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:32.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.530+0000 7fb41db71640 1 -- 172.21.15.22:0/3067274070 shutdown_connections 2024-09-19T08:13:32.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.530+0000 7fb41db71640 1 -- 172.21.15.22:0/3067274070 wait complete. 2024-09-19T08:13:32.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb41db71640 1 Processor -- start 2024-09-19T08:13:32.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb41db71640 1 -- start start 2024-09-19T08:13:32.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb41db71640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb418102da0 0x7fb418100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:32.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb41db71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4181036c0 0x7fb418100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:32.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb41db71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb41810c180 0x7fb418101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:32.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb41db71640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb418111fa0 con 0x7fb4181036c0 2024-09-19T08:13:32.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb41db71640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb418111e20 con 0x7fb41810c180 2024-09-19T08:13:32.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb41db71640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb418112120 con 0x7fb418102da0 2024-09-19T08:13:32.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb416ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4181036c0 0x7fb418100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:32.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb416ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4181036c0 0x7fb418100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46566/0 (socket says 172.21.15.22:46566) 2024-09-19T08:13:32.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb416ffd640 1 -- 172.21.15.22:0/1435317931 learned_addr learned my addr 172.21.15.22:0/1435317931 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:32.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb417fff640 1 --2- 172.21.15.22:0/1435317931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb41810c180 0x7fb418101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:32.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb416ffd640 1 -- 172.21.15.22:0/1435317931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb418102da0 msgr2=0x7fb418100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:32.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb416ffd640 1 --2- 172.21.15.22:0/1435317931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb418102da0 0x7fb418100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:32.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb416ffd640 1 -- 172.21.15.22:0/1435317931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb41810c180 msgr2=0x7fb418101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:32.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb416ffd640 1 --2- 172.21.15.22:0/1435317931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb41810c180 0x7fb418101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:32.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.531+0000 7fb416ffd640 1 -- 172.21.15.22:0/1435317931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4181a6970 con 0x7fb4181036c0 2024-09-19T08:13:32.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.532+0000 7fb416ffd640 1 --2- 172.21.15.22:0/1435317931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4181036c0 0x7fb418100f40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb40c005ab0 tx=0x7fb40c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:32.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.532+0000 7fb414ff9640 1 -- 172.21.15.22:0/1435317931 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c03f030 con 0x7fb4181036c0 2024-09-19T08:13:32.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.532+0000 7fb414ff9640 1 -- 172.21.15.22:0/1435317931 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb40c003ce0 con 0x7fb4181036c0 2024-09-19T08:13:32.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.532+0000 7fb41db71640 1 -- 172.21.15.22:0/1435317931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4181a6c00 con 0x7fb4181036c0 2024-09-19T08:13:32.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.532+0000 7fb414ff9640 1 -- 172.21.15.22:0/1435317931 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c035070 con 0x7fb4181036c0 2024-09-19T08:13:32.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.532+0000 7fb41db71640 1 -- 172.21.15.22:0/1435317931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4181a7130 con 0x7fb4181036c0 2024-09-19T08:13:32.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.533+0000 7fb41db71640 1 -- 172.21.15.22:0/1435317931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb418103180 con 0x7fb4181036c0 2024-09-19T08:13:32.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.534+0000 7fb414ff9640 1 -- 172.21.15.22:0/1435317931 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb40c042030 con 0x7fb4181036c0 2024-09-19T08:13:32.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.535+0000 7fb414ff9640 1 --2- 172.21.15.22:0/1435317931 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb4000a3350 0x7fb4000a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:32.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.535+0000 7fb4177fe640 1 --2- 172.21.15.22:0/1435317931 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb4000a3350 0x7fb4000a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:32.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.535+0000 7fb414ff9640 1 -- 172.21.15.22:0/1435317931 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fb40c0f0230 con 0x7fb4181036c0 2024-09-19T08:13:32.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.535+0000 7fb4177fe640 1 --2- 172.21.15.22:0/1435317931 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb4000a3350 0x7fb4000a5810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb40800a8f0 tx=0x7fb408005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:32.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.539+0000 7fb414ff9640 1 -- 172.21.15.22:0/1435317931 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb40c0bcfa0 con 0x7fb4181036c0 2024-09-19T08:13:32.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:32 smithi064 ceph-mon[34900]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-19T08:13:32.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:32 smithi157 ceph-mon[35143]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-19T08:13:32.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.763+0000 7fb41db71640 1 -- 172.21.15.22:0/1435317931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb418106c30 con 0x7fb4181036c0 2024-09-19T08:13:32.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.768+0000 7fb414ff9640 1 -- 172.21.15.22:0/1435317931 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264048 (secure 0 0 0) 0x7fb40c0bc8a0 con 0x7fb4181036c0 2024-09-19T08:13:32.771 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:13:32.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.770+0000 7fb41db71640 1 -- 172.21.15.22:0/1435317931 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb4000a3350 msgr2=0x7fb4000a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:32.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.770+0000 7fb41db71640 1 --2- 172.21.15.22:0/1435317931 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb4000a3350 0x7fb4000a5810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb40800a8f0 tx=0x7fb408005bf0 comp rx=0 tx=0).stop 2024-09-19T08:13:32.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.770+0000 7fb41db71640 1 -- 172.21.15.22:0/1435317931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4181036c0 msgr2=0x7fb418100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:32.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.770+0000 7fb41db71640 1 --2- 172.21.15.22:0/1435317931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4181036c0 0x7fb418100f40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb40c005ab0 tx=0x7fb40c0042f0 comp rx=0 tx=0).stop 2024-09-19T08:13:32.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.771+0000 7fb41db71640 1 -- 172.21.15.22:0/1435317931 shutdown_connections 2024-09-19T08:13:32.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.771+0000 7fb41db71640 1 --2- 172.21.15.22:0/1435317931 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb4000a3350 0x7fb4000a5810 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:32.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.771+0000 7fb41db71640 1 --2- 172.21.15.22:0/1435317931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb41810c180 0x7fb418101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:32.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.771+0000 7fb41db71640 1 --2- 172.21.15.22:0/1435317931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4181036c0 0x7fb418100f40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:32.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.771+0000 7fb41db71640 1 --2- 172.21.15.22:0/1435317931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb418102da0 0x7fb418100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:32.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.771+0000 7fb41db71640 1 -- 172.21.15.22:0/1435317931 >> 172.21.15.22:0/1435317931 conn(0x7fb4180fc7a0 msgr2=0x7fb4180fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:32.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.771+0000 7fb41db71640 1 -- 172.21.15.22:0/1435317931 shutdown_connections 2024-09-19T08:13:32.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:32.771+0000 7fb41db71640 1 -- 172.21.15.22:0/1435317931 wait complete. 2024-09-19T08:13:33.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:33 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1435317931' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T08:13:33.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:33 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1435317931' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T08:13:33.648 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1866276413},{"type":"v1","addr":"172.21.15.22:6801","nonce":1866276413}]},"active_addr":"172.21.15.22:6801/1866276413","active_change":"2024-09-19T08:06:06.676586+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.22:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":1845420423}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":422860013}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":2837500116}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":155709052}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":2082791937}]}]} 2024-09-19T08:13:33.653 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T08:13:33.654 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T08:13:33.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:13:33.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:33 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1435317931' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T08:13:33.916 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:34.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.425+0000 7fb305ad4640 1 -- 172.21.15.22:0/28390905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb30010c150 msgr2=0x7fb30010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:34.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.425+0000 7fb305ad4640 1 --2- 172.21.15.22:0/28390905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb30010c150 0x7fb30010e5f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009ca0 tx=0x7fb2f802a950 comp rx=0 tx=0).stop 2024-09-19T08:13:34.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.425+0000 7fb305ad4640 1 -- 172.21.15.22:0/28390905 shutdown_connections 2024-09-19T08:13:34.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.425+0000 7fb305ad4640 1 --2- 172.21.15.22:0/28390905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb30010c150 0x7fb30010e5f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:34.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.425+0000 7fb305ad4640 1 --2- 172.21.15.22:0/28390905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb300103640 0x7fb30010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.425+0000 7fb305ad4640 1 --2- 172.21.15.22:0/28390905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb300102d20 0x7fb300103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.425+0000 7fb305ad4640 1 -- 172.21.15.22:0/28390905 >> 172.21.15.22:0/28390905 conn(0x7fb3000fc780 msgr2=0x7fb3000feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.425+0000 7fb305ad4640 1 -- 172.21.15.22:0/28390905 shutdown_connections 2024-09-19T08:13:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.425+0000 7fb305ad4640 1 -- 172.21.15.22:0/28390905 wait complete. 2024-09-19T08:13:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.425+0000 7fb305ad4640 1 Processor -- start 2024-09-19T08:13:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.425+0000 7fb305ad4640 1 -- start start 2024-09-19T08:13:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb305ad4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb300102d20 0x7fb300198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb305ad4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb300103640 0x7fb3001989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb305ad4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30010c150 0x7fb30019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb305ad4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb300111f50 con 0x7fb300102d20 2024-09-19T08:13:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb305ad4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb300111dd0 con 0x7fb30010c150 2024-09-19T08:13:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb305ad4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb3001120d0 con 0x7fb300103640 2024-09-19T08:13:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb2ff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb300102d20 0x7fb300198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb2ff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb300102d20 0x7fb300198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46580/0 (socket says 172.21.15.22:46580) 2024-09-19T08:13:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb2ff7fe640 1 -- 172.21.15.22:0/2500375900 learned_addr learned my addr 172.21.15.22:0/2500375900 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb2fffff640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30010c150 0x7fb30019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb2feffd640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb300103640 0x7fb3001989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb2ff7fe640 1 -- 172.21.15.22:0/2500375900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb300103640 msgr2=0x7fb3001989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb2ff7fe640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb300103640 0x7fb3001989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb2ff7fe640 1 -- 172.21.15.22:0/2500375900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30010c150 msgr2=0x7fb30019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb2ff7fe640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30010c150 0x7fb30019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb2ff7fe640 1 -- 172.21.15.22:0/2500375900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30019d420 con 0x7fb300102d20 2024-09-19T08:13:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb2feffd640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb300103640 0x7fb3001989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:13:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb2ff7fe640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb300102d20 0x7fb300198470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000d770 tx=0x7fb2f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.426+0000 7fb2fcff9640 1 -- 172.21.15.22:0/2500375900 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0017070 con 0x7fb300102d20 2024-09-19T08:13:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.427+0000 7fb305ad4640 1 -- 172.21.15.22:0/2500375900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30019d710 con 0x7fb300102d20 2024-09-19T08:13:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.427+0000 7fb2fcff9640 1 -- 172.21.15.22:0/2500375900 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2f0005500 con 0x7fb300102d20 2024-09-19T08:13:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.427+0000 7fb2fcff9640 1 -- 172.21.15.22:0/2500375900 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0004e00 con 0x7fb300102d20 2024-09-19T08:13:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.427+0000 7fb305ad4640 1 -- 172.21.15.22:0/2500375900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3001a4ff0 con 0x7fb300102d20 2024-09-19T08:13:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.427+0000 7fb305ad4640 1 -- 172.21.15.22:0/2500375900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb300103100 con 0x7fb300102d20 2024-09-19T08:13:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.429+0000 7fb2fcff9640 1 -- 172.21.15.22:0/2500375900 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb2f0004fa0 con 0x7fb300102d20 2024-09-19T08:13:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.429+0000 7fb2fcff9640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb2e80a32d0 0x7fb2e80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.430+0000 7fb2fcff9640 1 -- 172.21.15.22:0/2500375900 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fb2f00cf750 con 0x7fb300102d20 2024-09-19T08:13:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.430+0000 7fb2feffd640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb2e80a32d0 0x7fb2e80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.430+0000 7fb2feffd640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb2e80a32d0 0x7fb2e80a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4005fd0 tx=0x7fb2f4005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.433+0000 7fb2fcff9640 1 -- 172.21.15.22:0/2500375900 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb2f009c4c0 con 0x7fb300102d20 2024-09-19T08:13:34.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:34 smithi064 ceph-mon[34900]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-09-19T08:13:34.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.613+0000 7fb305ad4640 1 -- 172.21.15.22:0/2500375900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3001071f0 con 0x7fb300102d20 2024-09-19T08:13:34.613 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:34 smithi022 ceph-mon[30963]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-09-19T08:13:34.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.615+0000 7fb2fcff9640 1 -- 172.21.15.22:0/2500375900 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22243 (secure 0 0 0) 0x7fb2f009bdc0 con 0x7fb300102d20 2024-09-19T08:13:34.615 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:13:34.615 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":83,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","created":"2024-09-19T08:05:19.476952+0000","modified":"2024-09-19T08:13:01.503182+0000","last_up_change":"2024-09-19T08:12:10.339586+0000","last_in_change":"2024-09-19T08:11:47.653576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:08:25.175759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:12:57.608787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:12:59.824327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ed67f2d-9b28-427a-b69f-676e7a2abcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6803","nonce":3442715307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6805","nonce":3442715307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6809","nonce":3442715307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6807","nonce":3442715307}]},"public_addr":"172.21.15.22:6803/3442715307","cluster_addr":"172.21.15.22:6805/3442715307","heartbeat_back_addr":"172.21.15.22:6809/3442715307","heartbeat_front_addr":"172.21.15.22:6807/3442715307","state":["exists","up"]},{"osd":1,"uuid":"bff3e283-4ab1-49d7-86d8-45d361995122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6801","nonce":2379700228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6803","nonce":2379700228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6807","nonce":2379700228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6805","nonce":2379700228}]},"public_addr":"172.21.15.64:6801/2379700228","cluster_addr":"172.21.15.64:6803/2379700228","heartbeat_back_addr":"172.21.15.64:6807/2379700228","heartbeat_front_addr":"172.21.15.64:6805/2379700228","state":["exists","up"]},{"osd":2,"uuid":"81421d08-7dc6-419b-a566-dee1e1e13b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6801","nonce":3363263867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6803","nonce":3363263867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6807","nonce":3363263867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6805","nonce":3363263867}]},"public_addr":"172.21.15.157:6801/3363263867","cluster_addr":"172.21.15.157:6803/3363263867","heartbeat_back_addr":"172.21.15.157:6807/3363263867","heartbeat_front_addr":"172.21.15.157:6805/3363263867","state":["exists","up"]},{"osd":3,"uuid":"34debfd0-3226-4d78-b9e6-e03dbc5190f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6811","nonce":4225229456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6813","nonce":4225229456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6817","nonce":4225229456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6815","nonce":4225229456}]},"public_addr":"172.21.15.22:6811/4225229456","cluster_addr":"172.21.15.22:6813/4225229456","heartbeat_back_addr":"172.21.15.22:6817/4225229456","heartbeat_front_addr":"172.21.15.22:6815/4225229456","state":["exists","up"]},{"osd":4,"uuid":"1293332c-d772-4ca9-bf76-79e0b29451ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6809","nonce":1858090423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6811","nonce":1858090423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6815","nonce":1858090423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6813","nonce":1858090423}]},"public_addr":"172.21.15.64:6809/1858090423","cluster_addr":"172.21.15.64:6811/1858090423","heartbeat_back_addr":"172.21.15.64:6815/1858090423","heartbeat_front_addr":"172.21.15.64:6813/1858090423","state":["exists","up"]},{"osd":5,"uuid":"240a6c42-89a2-41c1-948c-b09914b5722e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6809","nonce":2900701212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2900701212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2900701212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2900701212}]},"public_addr":"172.21.15.157:6809/2900701212","cluster_addr":"172.21.15.157:6811/2900701212","heartbeat_back_addr":"172.21.15.157:6815/2900701212","heartbeat_front_addr":"172.21.15.157:6813/2900701212","state":["exists","up"]},{"osd":6,"uuid":"80199030-109e-41c4-bc1c-731755464770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6819","nonce":4264426242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6821","nonce":4264426242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6825","nonce":4264426242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6823","nonce":4264426242}]},"public_addr":"172.21.15.22:6819/4264426242","cluster_addr":"172.21.15.22:6821/4264426242","heartbeat_back_addr":"172.21.15.22:6825/4264426242","heartbeat_front_addr":"172.21.15.22:6823/4264426242","state":["exists","up"]},{"osd":7,"uuid":"dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6817","nonce":3980224103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6819","nonce":3980224103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6823","nonce":3980224103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6821","nonce":3980224103}]},"public_addr":"172.21.15.64:6817/3980224103","cluster_addr":"172.21.15.64:6819/3980224103","heartbeat_back_addr":"172.21.15.64:6823/3980224103","heartbeat_front_addr":"172.21.15.64:6821/3980224103","state":["exists","up"]},{"osd":8,"uuid":"764b1e6e-48c9-49f9-8124-bc13bcba3a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6817","nonce":973678477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6819","nonce":973678477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6823","nonce":973678477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6821","nonce":973678477}]},"public_addr":"172.21.15.157:6817/973678477","cluster_addr":"172.21.15.157:6819/973678477","heartbeat_back_addr":"172.21.15.157:6823/973678477","heartbeat_front_addr":"172.21.15.157:6821/973678477","state":["exists","up"]},{"osd":9,"uuid":"82914818-df47-4374-9e3e-2f71fc76e78b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6827","nonce":710033990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6829","nonce":710033990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6833","nonce":710033990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6831","nonce":710033990}]},"public_addr":"172.21.15.22:6827/710033990","cluster_addr":"172.21.15.22:6829/710033990","heartbeat_back_addr":"172.21.15.22:6833/710033990","heartbeat_front_addr":"172.21.15.22:6831/710033990","state":["exists","up"]},{"osd":10,"uuid":"36ae177d-c454-4cba-ba21-cbeee0f10dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6825","nonce":373412752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6827","nonce":373412752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6831","nonce":373412752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6829","nonce":373412752}]},"public_addr":"172.21.15.64:6825/373412752","cluster_addr":"172.21.15.64:6827/373412752","heartbeat_back_addr":"172.21.15.64:6831/373412752","heartbeat_front_addr":"172.21.15.64:6829/373412752","state":["exists","up"]},{"osd":11,"uuid":"cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6825","nonce":1918889194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6827","nonce":1918889194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6831","nonce":1918889194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6829","nonce":1918889194}]},"public_addr":"172.21.15.157:6825/1918889194","cluster_addr":"172.21.15.157:6827/1918889194","heartbeat_back_addr":"172.21.15.157:6831/1918889194","heartbeat_front_addr":"172.21.15.157:6829/1918889194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:31.378218+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:56.794167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:20.790267+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:47.039106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:11.483842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:38.151466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:03.553308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:27.865321+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:53.086530+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:18.946661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:43.697243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:12:08.936588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3712426751":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/2133059277":"2024-09-20T08:05:42.354701+0000","172.21.15.22:6801/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2851750481":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/3623845502":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/384027596":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/586163474":"2024-09-20T08:05:42.354701+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:13:34.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.617+0000 7fb305ad4640 1 -- 172.21.15.22:0/2500375900 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb2e80a32d0 msgr2=0x7fb2e80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:34.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.617+0000 7fb305ad4640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb2e80a32d0 0x7fb2e80a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4005fd0 tx=0x7fb2f4005f00 comp rx=0 tx=0).stop 2024-09-19T08:13:34.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.617+0000 7fb305ad4640 1 -- 172.21.15.22:0/2500375900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb300102d20 msgr2=0x7fb300198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:34.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.617+0000 7fb305ad4640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb300102d20 0x7fb300198470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000d770 tx=0x7fb2f000dc30 comp rx=0 tx=0).stop 2024-09-19T08:13:34.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.617+0000 7fb305ad4640 1 -- 172.21.15.22:0/2500375900 shutdown_connections 2024-09-19T08:13:34.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.617+0000 7fb305ad4640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb2e80a32d0 0x7fb2e80a5790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:34.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.617+0000 7fb305ad4640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30010c150 0x7fb30019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:34.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.617+0000 7fb305ad4640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb300103640 0x7fb3001989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:34.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.617+0000 7fb305ad4640 1 --2- 172.21.15.22:0/2500375900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb300102d20 0x7fb300198470 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:34.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.617+0000 7fb305ad4640 1 -- 172.21.15.22:0/2500375900 >> 172.21.15.22:0/2500375900 conn(0x7fb3000fc780 msgr2=0x7fb3000fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:34.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.617+0000 7fb305ad4640 1 -- 172.21.15.22:0/2500375900 shutdown_connections 2024-09-19T08:13:34.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:34.617+0000 7fb305ad4640 1 -- 172.21.15.22:0/2500375900 wait complete. 2024-09-19T08:13:34.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:34 smithi157 ceph-mon[35143]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-09-19T08:13:35.550 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:35 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2500375900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:35.586 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T08:13:35.587 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:13:35.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:35 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2500375900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:35.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:35 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2500375900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:35.851 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.345+0000 7f2e5936b640 1 -- 172.21.15.22:0/2970460966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e5410c170 msgr2=0x7f2e5410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.345+0000 7f2e5936b640 1 --2- 172.21.15.22:0/2970460966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e5410c170 0x7f2e5410e610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009ca0 tx=0x7f2e4c02a950 comp rx=0 tx=0).stop 2024-09-19T08:13:36.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.345+0000 7f2e5936b640 1 -- 172.21.15.22:0/2970460966 shutdown_connections 2024-09-19T08:13:36.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.345+0000 7f2e5936b640 1 --2- 172.21.15.22:0/2970460966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e5410c170 0x7f2e5410e610 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:36.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.345+0000 7f2e5936b640 1 --2- 172.21.15.22:0/2970460966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e54103660 0x7f2e5410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:36.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.345+0000 7f2e5936b640 1 --2- 172.21.15.22:0/2970460966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e54102d40 0x7f2e54103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:36.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.345+0000 7f2e5936b640 1 -- 172.21.15.22:0/2970460966 >> 172.21.15.22:0/2970460966 conn(0x7f2e540fc7a0 msgr2=0x7f2e540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:36.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.345+0000 7f2e5936b640 1 -- 172.21.15.22:0/2970460966 shutdown_connections 2024-09-19T08:13:36.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.345+0000 7f2e5936b640 1 -- 172.21.15.22:0/2970460966 wait complete. 2024-09-19T08:13:36.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e5936b640 1 Processor -- start 2024-09-19T08:13:36.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e5936b640 1 -- start start 2024-09-19T08:13:36.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e5936b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e54102d40 0x7f2e541009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e5936b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e54103660 0x7f2e54100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e5936b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e5410c170 0x7f2e54101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e5936b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2e54111f70 con 0x7f2e5410c170 2024-09-19T08:13:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e5936b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e54111df0 con 0x7f2e54103660 2024-09-19T08:13:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e5936b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e541120f0 con 0x7f2e54102d40 2024-09-19T08:13:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e5410c170 0x7f2e54101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e5410c170 0x7f2e54101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46598/0 (socket says 172.21.15.22:46598) 2024-09-19T08:13:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e537fe640 1 -- 172.21.15.22:0/369856227 learned_addr learned my addr 172.21.15.22:0/369856227 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e52ffd640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e54102d40 0x7f2e541009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:36.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e4bfff640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e54103660 0x7f2e54100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:36.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e537fe640 1 -- 172.21.15.22:0/369856227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e54102d40 msgr2=0x7f2e541009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:36.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e537fe640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e54102d40 0x7f2e541009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:36.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e537fe640 1 -- 172.21.15.22:0/369856227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e54103660 msgr2=0x7f2e54100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:36.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e537fe640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e54103660 0x7f2e54100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:36.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e537fe640 1 -- 172.21.15.22:0/369856227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e541a68b0 con 0x7f2e5410c170 2024-09-19T08:13:36.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.346+0000 7f2e52ffd640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e54102d40 0x7f2e541009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:13:36.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.347+0000 7f2e537fe640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e5410c170 0x7f2e54101420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c02ae70 tx=0x7f2e4c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:36.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.347+0000 7f2e50ff9640 1 -- 172.21.15.22:0/369856227 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c034040 con 0x7f2e5410c170 2024-09-19T08:13:36.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.347+0000 7f2e50ff9640 1 -- 172.21.15.22:0/369856227 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e4c030920 con 0x7f2e5410c170 2024-09-19T08:13:36.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.347+0000 7f2e5936b640 1 -- 172.21.15.22:0/369856227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e541a6b40 con 0x7f2e5410c170 2024-09-19T08:13:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.347+0000 7f2e50ff9640 1 -- 172.21.15.22:0/369856227 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c045480 con 0x7f2e5410c170 2024-09-19T08:13:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.347+0000 7f2e5936b640 1 -- 172.21.15.22:0/369856227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e541a7020 con 0x7f2e5410c170 2024-09-19T08:13:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.348+0000 7f2e5936b640 1 -- 172.21.15.22:0/369856227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e54103120 con 0x7f2e5410c170 2024-09-19T08:13:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.349+0000 7f2e50ff9640 1 -- 172.21.15.22:0/369856227 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2e4c037020 con 0x7f2e5410c170 2024-09-19T08:13:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.350+0000 7f2e50ff9640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2e400a32d0 0x7f2e400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.350+0000 7f2e52ffd640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2e400a32d0 0x7f2e400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.350+0000 7f2e50ff9640 1 -- 172.21.15.22:0/369856227 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f2e4c0f0900 con 0x7f2e5410c170 2024-09-19T08:13:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.350+0000 7f2e52ffd640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2e400a32d0 0x7f2e400a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2e440058c0 tx=0x7f2e44005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.354+0000 7f2e50ff9640 1 -- 172.21.15.22:0/369856227 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2e4c0bd680 con 0x7f2e5410c170 2024-09-19T08:13:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.529+0000 7f2e5936b640 1 -- 172.21.15.22:0/369856227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e541071e0 con 0x7f2e5410c170 2024-09-19T08:13:36.529 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:36 smithi022 ceph-mon[30963]: pgmap v297: 97 pgs: 97 active+clean; 634 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-19T08:13:36.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.531+0000 7f2e50ff9640 1 -- 172.21.15.22:0/369856227 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22243 (secure 0 0 0) 0x7f2e4c0bcf80 con 0x7f2e5410c170 2024-09-19T08:13:36.531 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:13:36.531 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":83,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","created":"2024-09-19T08:05:19.476952+0000","modified":"2024-09-19T08:13:01.503182+0000","last_up_change":"2024-09-19T08:12:10.339586+0000","last_in_change":"2024-09-19T08:11:47.653576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:08:25.175759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:12:57.608787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:12:59.824327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ed67f2d-9b28-427a-b69f-676e7a2abcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6803","nonce":3442715307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6805","nonce":3442715307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6809","nonce":3442715307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6807","nonce":3442715307}]},"public_addr":"172.21.15.22:6803/3442715307","cluster_addr":"172.21.15.22:6805/3442715307","heartbeat_back_addr":"172.21.15.22:6809/3442715307","heartbeat_front_addr":"172.21.15.22:6807/3442715307","state":["exists","up"]},{"osd":1,"uuid":"bff3e283-4ab1-49d7-86d8-45d361995122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6801","nonce":2379700228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6803","nonce":2379700228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6807","nonce":2379700228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6805","nonce":2379700228}]},"public_addr":"172.21.15.64:6801/2379700228","cluster_addr":"172.21.15.64:6803/2379700228","heartbeat_back_addr":"172.21.15.64:6807/2379700228","heartbeat_front_addr":"172.21.15.64:6805/2379700228","state":["exists","up"]},{"osd":2,"uuid":"81421d08-7dc6-419b-a566-dee1e1e13b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6801","nonce":3363263867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6803","nonce":3363263867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6807","nonce":3363263867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6805","nonce":3363263867}]},"public_addr":"172.21.15.157:6801/3363263867","cluster_addr":"172.21.15.157:6803/3363263867","heartbeat_back_addr":"172.21.15.157:6807/3363263867","heartbeat_front_addr":"172.21.15.157:6805/3363263867","state":["exists","up"]},{"osd":3,"uuid":"34debfd0-3226-4d78-b9e6-e03dbc5190f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6811","nonce":4225229456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6813","nonce":4225229456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6817","nonce":4225229456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6815","nonce":4225229456}]},"public_addr":"172.21.15.22:6811/4225229456","cluster_addr":"172.21.15.22:6813/4225229456","heartbeat_back_addr":"172.21.15.22:6817/4225229456","heartbeat_front_addr":"172.21.15.22:6815/4225229456","state":["exists","up"]},{"osd":4,"uuid":"1293332c-d772-4ca9-bf76-79e0b29451ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6809","nonce":1858090423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6811","nonce":1858090423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6815","nonce":1858090423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6813","nonce":1858090423}]},"public_addr":"172.21.15.64:6809/1858090423","cluster_addr":"172.21.15.64:6811/1858090423","heartbeat_back_addr":"172.21.15.64:6815/1858090423","heartbeat_front_addr":"172.21.15.64:6813/1858090423","state":["exists","up"]},{"osd":5,"uuid":"240a6c42-89a2-41c1-948c-b09914b5722e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6809","nonce":2900701212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2900701212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2900701212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2900701212}]},"public_addr":"172.21.15.157:6809/2900701212","cluster_addr":"172.21.15.157:6811/2900701212","heartbeat_back_addr":"172.21.15.157:6815/2900701212","heartbeat_front_addr":"172.21.15.157:6813/2900701212","state":["exists","up"]},{"osd":6,"uuid":"80199030-109e-41c4-bc1c-731755464770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6819","nonce":4264426242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6821","nonce":4264426242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6825","nonce":4264426242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6823","nonce":4264426242}]},"public_addr":"172.21.15.22:6819/4264426242","cluster_addr":"172.21.15.22:6821/4264426242","heartbeat_back_addr":"172.21.15.22:6825/4264426242","heartbeat_front_addr":"172.21.15.22:6823/4264426242","state":["exists","up"]},{"osd":7,"uuid":"dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6817","nonce":3980224103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6819","nonce":3980224103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6823","nonce":3980224103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6821","nonce":3980224103}]},"public_addr":"172.21.15.64:6817/3980224103","cluster_addr":"172.21.15.64:6819/3980224103","heartbeat_back_addr":"172.21.15.64:6823/3980224103","heartbeat_front_addr":"172.21.15.64:6821/3980224103","state":["exists","up"]},{"osd":8,"uuid":"764b1e6e-48c9-49f9-8124-bc13bcba3a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6817","nonce":973678477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6819","nonce":973678477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6823","nonce":973678477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6821","nonce":973678477}]},"public_addr":"172.21.15.157:6817/973678477","cluster_addr":"172.21.15.157:6819/973678477","heartbeat_back_addr":"172.21.15.157:6823/973678477","heartbeat_front_addr":"172.21.15.157:6821/973678477","state":["exists","up"]},{"osd":9,"uuid":"82914818-df47-4374-9e3e-2f71fc76e78b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6827","nonce":710033990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6829","nonce":710033990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6833","nonce":710033990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6831","nonce":710033990}]},"public_addr":"172.21.15.22:6827/710033990","cluster_addr":"172.21.15.22:6829/710033990","heartbeat_back_addr":"172.21.15.22:6833/710033990","heartbeat_front_addr":"172.21.15.22:6831/710033990","state":["exists","up"]},{"osd":10,"uuid":"36ae177d-c454-4cba-ba21-cbeee0f10dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6825","nonce":373412752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6827","nonce":373412752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6831","nonce":373412752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6829","nonce":373412752}]},"public_addr":"172.21.15.64:6825/373412752","cluster_addr":"172.21.15.64:6827/373412752","heartbeat_back_addr":"172.21.15.64:6831/373412752","heartbeat_front_addr":"172.21.15.64:6829/373412752","state":["exists","up"]},{"osd":11,"uuid":"cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6825","nonce":1918889194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6827","nonce":1918889194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6831","nonce":1918889194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6829","nonce":1918889194}]},"public_addr":"172.21.15.157:6825/1918889194","cluster_addr":"172.21.15.157:6827/1918889194","heartbeat_back_addr":"172.21.15.157:6831/1918889194","heartbeat_front_addr":"172.21.15.157:6829/1918889194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:31.378218+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:56.794167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:20.790267+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:47.039106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:11.483842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:38.151466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:03.553308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:27.865321+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:53.086530+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:18.946661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:43.697243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:12:08.936588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3712426751":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/2133059277":"2024-09-20T08:05:42.354701+0000","172.21.15.22:6801/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2851750481":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/3623845502":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/384027596":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/586163474":"2024-09-20T08:05:42.354701+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:13:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.533+0000 7f2e5936b640 1 -- 172.21.15.22:0/369856227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2e400a32d0 msgr2=0x7f2e400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.533+0000 7f2e5936b640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2e400a32d0 0x7f2e400a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2e440058c0 tx=0x7f2e44005d30 comp rx=0 tx=0).stop 2024-09-19T08:13:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.533+0000 7f2e5936b640 1 -- 172.21.15.22:0/369856227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e5410c170 msgr2=0x7f2e54101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.533+0000 7f2e5936b640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e5410c170 0x7f2e54101420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c02ae70 tx=0x7f2e4c02f5b0 comp rx=0 tx=0).stop 2024-09-19T08:13:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.533+0000 7f2e5936b640 1 -- 172.21.15.22:0/369856227 shutdown_connections 2024-09-19T08:13:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.533+0000 7f2e5936b640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2e400a32d0 0x7f2e400a5790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.533+0000 7f2e5936b640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e5410c170 0x7f2e54101420 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.533+0000 7f2e5936b640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e54103660 0x7f2e54100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.533+0000 7f2e5936b640 1 --2- 172.21.15.22:0/369856227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e54102d40 0x7f2e541009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.533+0000 7f2e5936b640 1 -- 172.21.15.22:0/369856227 >> 172.21.15.22:0/369856227 conn(0x7f2e540fc7a0 msgr2=0x7f2e540fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.533+0000 7f2e5936b640 1 -- 172.21.15.22:0/369856227 shutdown_connections 2024-09-19T08:13:36.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:36.533+0000 7f2e5936b640 1 -- 172.21.15.22:0/369856227 wait complete. 2024-09-19T08:13:36.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:36 smithi064 ceph-mon[34900]: pgmap v297: 97 pgs: 97 active+clean; 634 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-19T08:13:36.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:36 smithi157 ceph-mon[35143]: pgmap v297: 97 pgs: 97 active+clean; 634 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-19T08:13:37.230 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T08:13:37.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T08:13:37.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T08:13:37.232 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T08:13:37.232 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T08:13:37.232 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T08:13:37.233 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T08:13:37.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T08:13:37.235 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T08:13:37.235 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T08:13:37.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T08:13:37.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T08:13:37.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:37 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/369856227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:37.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:37 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/369856227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:37.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:37 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/369856227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:13:37.886 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:38.378 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:38 smithi022 ceph-mon[30963]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-19T08:13:38.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.482+0000 7f3abe538640 1 -- 172.21.15.22:0/3912061127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ab8100430 msgr2=0x7f3ab8100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.482+0000 7f3abe538640 1 --2- 172.21.15.22:0/3912061127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ab8100430 0x7f3ab8100810 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009900 tx=0x7f3aa802a410 comp rx=0 tx=0).stop 2024-09-19T08:13:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.482+0000 7f3abe538640 1 -- 172.21.15.22:0/3912061127 shutdown_connections 2024-09-19T08:13:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.482+0000 7f3abe538640 1 --2- 172.21.15.22:0/3912061127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8105560 0x7f3ab810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.482+0000 7f3abe538640 1 --2- 172.21.15.22:0/3912061127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab8100de0 0x7f3ab8104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.482+0000 7f3abe538640 1 --2- 172.21.15.22:0/3912061127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ab8100430 0x7f3ab8100810 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.482+0000 7f3abe538640 1 -- 172.21.15.22:0/3912061127 >> 172.21.15.22:0/3912061127 conn(0x7f3ab8077230 msgr2=0x7f3ab80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.482+0000 7f3abe538640 1 -- 172.21.15.22:0/3912061127 shutdown_connections 2024-09-19T08:13:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.482+0000 7f3abe538640 1 -- 172.21.15.22:0/3912061127 wait complete. 2024-09-19T08:13:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.483+0000 7f3abe538640 1 Processor -- start 2024-09-19T08:13:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.483+0000 7f3abe538640 1 -- start start 2024-09-19T08:13:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3abe538640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ab8100430 0x7f3ab8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3abe538640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8100de0 0x7f3ab8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3abe538640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab8105560 0x7f3ab819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3abe538640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8110560 con 0x7f3ab8100430 2024-09-19T08:13:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3abe538640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81103e0 con 0x7f3ab8100de0 2024-09-19T08:13:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3abe538640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81106e0 con 0x7f3ab8105560 2024-09-19T08:13:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ab8100430 0x7f3ab8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ab8100430 0x7f3ab8198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58770/0 (socket says 172.21.15.22:58770) 2024-09-19T08:13:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3ab7fff640 1 -- 172.21.15.22:0/576937003 learned_addr learned my addr 172.21.15.22:0/576937003 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3ab77fe640 1 --2- 172.21.15.22:0/576937003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8100de0 0x7f3ab8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3abcad5640 1 --2- 172.21.15.22:0/576937003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab8105560 0x7f3ab819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3ab7fff640 1 -- 172.21.15.22:0/576937003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab8105560 msgr2=0x7f3ab819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3ab7fff640 1 --2- 172.21.15.22:0/576937003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab8105560 0x7f3ab819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3ab7fff640 1 -- 172.21.15.22:0/576937003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8100de0 msgr2=0x7f3ab8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3ab7fff640 1 --2- 172.21.15.22:0/576937003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab8100de0 0x7f3ab8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3ab7fff640 1 -- 172.21.15.22:0/576937003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab819d3c0 con 0x7f3ab8100430 2024-09-19T08:13:38.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3ab7fff640 1 --2- 172.21.15.22:0/576937003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ab8100430 0x7f3ab8198410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8006010 tx=0x7f3aa80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:38.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.484+0000 7f3ab57fa640 1 -- 172.21.15.22:0/576937003 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa803f030 con 0x7f3ab8100430 2024-09-19T08:13:38.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.485+0000 7f3ab57fa640 1 -- 172.21.15.22:0/576937003 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa8003ce0 con 0x7f3ab8100430 2024-09-19T08:13:38.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.485+0000 7f3abe538640 1 -- 172.21.15.22:0/576937003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab819d650 con 0x7f3ab8100430 2024-09-19T08:13:38.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.485+0000 7f3ab57fa640 1 -- 172.21.15.22:0/576937003 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8035070 con 0x7f3ab8100430 2024-09-19T08:13:38.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.485+0000 7f3abe538640 1 -- 172.21.15.22:0/576937003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab81a4f90 con 0x7f3ab8100430 2024-09-19T08:13:38.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.486+0000 7f3abe538640 1 -- 172.21.15.22:0/576937003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ab8062980 con 0x7f3ab8100430 2024-09-19T08:13:38.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.487+0000 7f3ab57fa640 1 -- 172.21.15.22:0/576937003 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3aa8042030 con 0x7f3ab8100430 2024-09-19T08:13:38.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.487+0000 7f3ab57fa640 1 --2- 172.21.15.22:0/576937003 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f3aa00a32d0 0x7f3aa00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:38.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.488+0000 7f3ab77fe640 1 --2- 172.21.15.22:0/576937003 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f3aa00a32d0 0x7f3aa00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:38.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.488+0000 7f3ab57fa640 1 -- 172.21.15.22:0/576937003 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f3aa80f0550 con 0x7f3ab8100430 2024-09-19T08:13:38.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.488+0000 7f3ab57fa640 1 --2- 172.21.15.22:0/576937003 >> [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307] conn(0x7f3aa00acb20 0x7f3aa00aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:38.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.488+0000 7f3ab57fa640 1 -- 172.21.15.22:0/576937003 --> [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3aa8040e70 con 0x7f3aa00acb20 2024-09-19T08:13:38.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.488+0000 7f3ab57fa640 1 -- 172.21.15.22:0/576937003 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f3aa80f0a30 con 0x7f3ab8100430 2024-09-19T08:13:38.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.488+0000 7f3abcad5640 1 --2- 172.21.15.22:0/576937003 >> [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307] conn(0x7f3aa00acb20 0x7f3aa00aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:38.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.488+0000 7f3ab77fe640 1 --2- 172.21.15.22:0/576937003 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f3aa00a32d0 0x7f3aa00a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8102f60 tx=0x7f3aac00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:38.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.488+0000 7f3abcad5640 1 --2- 172.21.15.22:0/576937003 >> [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307] conn(0x7f3aa00acb20 0x7f3aa00aef40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:38.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.489+0000 7f3ab57fa640 1 -- 172.21.15.22:0/576937003 <== osd.0 v2:172.21.15.22:6802/3442715307 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3aa8040e70 con 0x7f3aa00acb20 2024-09-19T08:13:38.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.503+0000 7f3abe538640 1 -- 172.21.15.22:0/576937003 --> [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3ab8062e00 con 0x7f3aa00acb20 2024-09-19T08:13:38.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.503+0000 7f3ab57fa640 1 -- 172.21.15.22:0/576937003 <== osd.0 v2:172.21.15.22:6802/3442715307 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3ab8062e00 con 0x7f3aa00acb20 2024-09-19T08:13:38.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.504+0000 7f3abe538640 1 -- 172.21.15.22:0/576937003 >> [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307] conn(0x7f3aa00acb20 msgr2=0x7f3aa00aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:38.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.504+0000 7f3abe538640 1 --2- 172.21.15.22:0/576937003 >> [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307] conn(0x7f3aa00acb20 0x7f3aa00aef40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:38.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.504+0000 7f3abe538640 1 -- 172.21.15.22:0/576937003 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f3aa00a32d0 msgr2=0x7f3aa00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:38.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.504+0000 7f3abe538640 1 --2- 172.21.15.22:0/576937003 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f3aa00a32d0 0x7f3aa00a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8102f60 tx=0x7f3aac00a2d0 comp rx=0 tx=0).stop 2024-09-19T08:13:38.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.504+0000 7f3abe538640 1 -- 172.21.15.22:0/576937003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ab8100430 msgr2=0x7f3ab8198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:38.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.504+0000 7f3abe538640 1 --2- 172.21.15.22:0/576937003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ab8100430 0x7f3ab8198410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8006010 tx=0x7f3aa80042f0 comp rx=0 tx=0).stop 2024-09-19T08:13:38.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.504+0000 7f3abcad5640 1 -- 172.21.15.22:0/576937003 reap_dead start 2024-09-19T08:13:38.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.504+0000 7f3abe538640 1 -- 172.21.15.22:0/576937003 shutdown_connections 2024-09-19T08:13:38.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.504+0000 7f3abe538640 1 -- 172.21.15.22:0/576937003 >> 172.21.15.22:0/576937003 conn(0x7f3ab8077230 msgr2=0x7f3ab81034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:38.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.505+0000 7f3abe538640 1 -- 172.21.15.22:0/576937003 shutdown_connections 2024-09-19T08:13:38.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:38.505+0000 7f3abe538640 1 -- 172.21.15.22:0/576937003 wait complete. 2024-09-19T08:13:38.512 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:38.517 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:38.528 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:38.533 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:38.537 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:38.543 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:38.543 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:38.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:38 smithi064 ceph-mon[34900]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-19T08:13:38.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:38 smithi157 ceph-mon[35143]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-19T08:13:38.808 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:39.374 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:39.374 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:39.403 INFO:teuthology.orchestra.run.smithi022.stdout:34359738445 2024-09-19T08:13:39.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T08:13:40.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.107+0000 7f770a95b640 1 -- 172.21.15.22:0/3181457469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7704100a80 msgr2=0x7f7704100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.107+0000 7f770a95b640 1 --2- 172.21.15.22:0/3181457469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7704100a80 0x7f7704100ee0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009930 tx=0x7f76f802a420 comp rx=0 tx=0).stop 2024-09-19T08:13:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.107+0000 7f770a95b640 1 -- 172.21.15.22:0/3181457469 shutdown_connections 2024-09-19T08:13:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.107+0000 7f770a95b640 1 --2- 172.21.15.22:0/3181457469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7704101420 0x7f770410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.107+0000 7f770a95b640 1 --2- 172.21.15.22:0/3181457469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7704100a80 0x7f7704100ee0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.107+0000 7f770a95b640 1 --2- 172.21.15.22:0/3181457469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7704106a80 0x7f7704106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.107+0000 7f770a95b640 1 -- 172.21.15.22:0/3181457469 >> 172.21.15.22:0/3181457469 conn(0x7f77040fc7a0 msgr2=0x7f77040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.107+0000 7f770a95b640 1 -- 172.21.15.22:0/3181457469 shutdown_connections 2024-09-19T08:13:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.107+0000 7f770a95b640 1 -- 172.21.15.22:0/3181457469 wait complete. 2024-09-19T08:13:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f770a95b640 1 Processor -- start 2024-09-19T08:13:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f770a95b640 1 -- start start 2024-09-19T08:13:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f770a95b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7704100a80 0x7f7704199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f770a95b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7704101420 0x7f7704199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f770a95b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7704106a80 0x7f77041a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f770a95b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7704110ea0 con 0x7f7704101420 2024-09-19T08:13:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f770a95b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7704110d20 con 0x7f7704100a80 2024-09-19T08:13:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f770a95b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7704111020 con 0x7f7704106a80 2024-09-19T08:13:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f77037fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7704101420 0x7f7704199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f77037fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7704101420 0x7f7704199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58782/0 (socket says 172.21.15.22:58782) 2024-09-19T08:13:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f77037fe640 1 -- 172.21.15.22:0/3957817501 learned_addr learned my addr 172.21.15.22:0/3957817501 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f7703fff640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7704100a80 0x7f7704199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f7708ef8640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7704106a80 0x7f77041a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f77037fe640 1 -- 172.21.15.22:0/3957817501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7704106a80 msgr2=0x7f77041a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f77037fe640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7704106a80 0x7f77041a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f77037fe640 1 -- 172.21.15.22:0/3957817501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7704100a80 msgr2=0x7f7704199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f77037fe640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7704100a80 0x7f7704199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f77037fe640 1 -- 172.21.15.22:0/3957817501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77041a48d0 con 0x7f7704101420 2024-09-19T08:13:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.108+0000 7f7703fff640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7704100a80 0x7f7704199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:13:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.109+0000 7f77037fe640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7704101420 0x7f7704199850 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009a50 tx=0x7f76f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:40.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.109+0000 7f77017fa640 1 -- 172.21.15.22:0/3957817501 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f803f030 con 0x7f7704101420 2024-09-19T08:13:40.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.109+0000 7f77017fa640 1 -- 172.21.15.22:0/3957817501 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f8003ce0 con 0x7f7704101420 2024-09-19T08:13:40.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.109+0000 7f770a95b640 1 -- 172.21.15.22:0/3957817501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77041a4aa0 con 0x7f7704101420 2024-09-19T08:13:40.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.109+0000 7f77017fa640 1 -- 172.21.15.22:0/3957817501 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8035070 con 0x7f7704101420 2024-09-19T08:13:40.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.109+0000 7f770a95b640 1 -- 172.21.15.22:0/3957817501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77041a4f80 con 0x7f7704101420 2024-09-19T08:13:40.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.110+0000 7f770a95b640 1 -- 172.21.15.22:0/3957817501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7704062900 con 0x7f7704101420 2024-09-19T08:13:40.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.111+0000 7f77017fa640 1 -- 172.21.15.22:0/3957817501 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f76f8042030 con 0x7f7704101420 2024-09-19T08:13:40.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.112+0000 7f77017fa640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f76ec0a3350 0x7f76ec0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:40.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.112+0000 7f7703fff640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f76ec0a3350 0x7f76ec0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:40.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.112+0000 7f77017fa640 1 -- 172.21.15.22:0/3957817501 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f76f80f02f0 con 0x7f7704101420 2024-09-19T08:13:40.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.112+0000 7f77017fa640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867] conn(0x7f76ec0aca70 0x7f76ec0aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:40.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.112+0000 7f77017fa640 1 -- 172.21.15.22:0/3957817501 --> [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76ec0af540 con 0x7f76ec0aca70 2024-09-19T08:13:40.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.112+0000 7f77017fa640 1 -- 172.21.15.22:0/3957817501 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f76f80f07d0 con 0x7f7704101420 2024-09-19T08:13:40.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.112+0000 7f7703fff640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f76ec0a3350 0x7f76ec0a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009930 tx=0x7f76f4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:40.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.113+0000 7f7708ef8640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867] conn(0x7f76ec0aca70 0x7f76ec0aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:40.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.113+0000 7f7708ef8640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867] conn(0x7f76ec0aca70 0x7f76ec0aee90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:40.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.114+0000 7f77017fa640 1 -- 172.21.15.22:0/3957817501 <== osd.2 v2:172.21.15.157:6800/3363263867 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76ec0af540 con 0x7f76ec0aca70 2024-09-19T08:13:40.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.128+0000 7f770a95b640 1 -- 172.21.15.22:0/3957817501 --> [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7704062d80 con 0x7f76ec0aca70 2024-09-19T08:13:40.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.128+0000 7f77017fa640 1 -- 172.21.15.22:0/3957817501 <== osd.2 v2:172.21.15.157:6800/3363263867 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7704062d80 con 0x7f76ec0aca70 2024-09-19T08:13:40.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.128+0000 7f770a95b640 1 -- 172.21.15.22:0/3957817501 >> [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867] conn(0x7f76ec0aca70 msgr2=0x7f76ec0aee90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:40.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.128+0000 7f770a95b640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867] conn(0x7f76ec0aca70 0x7f76ec0aee90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:40.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.129+0000 7f770a95b640 1 -- 172.21.15.22:0/3957817501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f76ec0a3350 msgr2=0x7f76ec0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:40.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.129+0000 7f770a95b640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f76ec0a3350 0x7f76ec0a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009930 tx=0x7f76f4008040 comp rx=0 tx=0).stop 2024-09-19T08:13:40.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.129+0000 7f770a95b640 1 -- 172.21.15.22:0/3957817501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7704101420 msgr2=0x7f7704199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:40.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.129+0000 7f770a95b640 1 --2- 172.21.15.22:0/3957817501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7704101420 0x7f7704199850 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009a50 tx=0x7f76f8004270 comp rx=0 tx=0).stop 2024-09-19T08:13:40.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.129+0000 7f7708ef8640 1 -- 172.21.15.22:0/3957817501 reap_dead start 2024-09-19T08:13:40.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.129+0000 7f770a95b640 1 -- 172.21.15.22:0/3957817501 shutdown_connections 2024-09-19T08:13:40.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.129+0000 7f770a95b640 1 -- 172.21.15.22:0/3957817501 >> 172.21.15.22:0/3957817501 conn(0x7f77040fc7a0 msgr2=0x7f77040fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:40.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.129+0000 7f770a95b640 1 -- 172.21.15.22:0/3957817501 shutdown_connections 2024-09-19T08:13:40.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:40.129+0000 7f770a95b640 1 -- 172.21.15.22:0/3957817501 wait complete. 2024-09-19T08:13:40.469 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:40 smithi022 ceph-mon[30963]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T08:13:40.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:40 smithi064 ceph-mon[34900]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T08:13:40.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:40 smithi157 ceph-mon[35143]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T08:13:41.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.270+0000 7efbfd762640 1 -- 172.21.15.22:0/3487623062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efbf8102d40 msgr2=0x7efbf8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.270+0000 7efbfd762640 1 --2- 172.21.15.22:0/3487623062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efbf8102d40 0x7efbf8103120 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009ca0 tx=0x7efbe802a950 comp rx=0 tx=0).stop 2024-09-19T08:13:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.270+0000 7efbfd762640 1 -- 172.21.15.22:0/3487623062 shutdown_connections 2024-09-19T08:13:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.270+0000 7efbfd762640 1 --2- 172.21.15.22:0/3487623062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf810c170 0x7efbf810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.270+0000 7efbfd762640 1 --2- 172.21.15.22:0/3487623062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efbf8103660 0x7efbf810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.270+0000 7efbfd762640 1 --2- 172.21.15.22:0/3487623062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efbf8102d40 0x7efbf8103120 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.270+0000 7efbfd762640 1 -- 172.21.15.22:0/3487623062 >> 172.21.15.22:0/3487623062 conn(0x7efbf80fc7a0 msgr2=0x7efbf80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.270+0000 7efbfd762640 1 -- 172.21.15.22:0/3487623062 shutdown_connections 2024-09-19T08:13:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.270+0000 7efbfd762640 1 -- 172.21.15.22:0/3487623062 wait complete. 2024-09-19T08:13:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbfd762640 1 Processor -- start 2024-09-19T08:13:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbfd762640 1 -- start start 2024-09-19T08:13:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbfd762640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efbf8102d40 0x7efbf81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbfd762640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf8103660 0x7efbf8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbfd762640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efbf810c170 0x7efbf8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbfd762640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efbf8111f70 con 0x7efbf8102d40 2024-09-19T08:13:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbfd762640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efbf8111df0 con 0x7efbf810c170 2024-09-19T08:13:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbfd762640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efbf81120f0 con 0x7efbf8103660 2024-09-19T08:13:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbf6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efbf8102d40 0x7efbf81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbf6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efbf8102d40 0x7efbf81009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58820/0 (socket says 172.21.15.22:58820) 2024-09-19T08:13:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbf6ffd640 1 -- 172.21.15.22:0/3033311573 learned_addr learned my addr 172.21.15.22:0/3033311573 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbf67fc640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf8103660 0x7efbf8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbf77fe640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efbf810c170 0x7efbf8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbf6ffd640 1 -- 172.21.15.22:0/3033311573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf8103660 msgr2=0x7efbf8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbf6ffd640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf8103660 0x7efbf8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbf6ffd640 1 -- 172.21.15.22:0/3033311573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efbf810c170 msgr2=0x7efbf8101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbf6ffd640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efbf810c170 0x7efbf8101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.271+0000 7efbf6ffd640 1 -- 172.21.15.22:0/3033311573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf81a6950 con 0x7efbf8102d40 2024-09-19T08:13:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.272+0000 7efbf77fe640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efbf810c170 0x7efbf8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:13:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.272+0000 7efbf6ffd640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efbf8102d40 0x7efbf81009a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efbe8005cc0 tx=0x7efbe802f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.272+0000 7efbdffff640 1 -- 172.21.15.22:0/3033311573 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8034040 con 0x7efbf8102d40 2024-09-19T08:13:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.272+0000 7efbdffff640 1 -- 172.21.15.22:0/3033311573 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbe8030920 con 0x7efbf8102d40 2024-09-19T08:13:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.272+0000 7efbdffff640 1 -- 172.21.15.22:0/3033311573 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8004830 con 0x7efbf8102d40 2024-09-19T08:13:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.272+0000 7efbfd762640 1 -- 172.21.15.22:0/3033311573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf81a6be0 con 0x7efbf8102d40 2024-09-19T08:13:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.272+0000 7efbfd762640 1 -- 172.21.15.22:0/3033311573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf81a7070 con 0x7efbf8102d40 2024-09-19T08:13:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.273+0000 7efbfd762640 1 -- 172.21.15.22:0/3033311573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efbf80628c0 con 0x7efbf8102d40 2024-09-19T08:13:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.274+0000 7efbdffff640 1 -- 172.21.15.22:0/3033311573 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7efbe8037020 con 0x7efbf8102d40 2024-09-19T08:13:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.275+0000 7efbdffff640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efbe00a3350 0x7efbe00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.275+0000 7efbdffff640 1 -- 172.21.15.22:0/3033311573 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7efbe80f0ca0 con 0x7efbf8102d40 2024-09-19T08:13:41.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.275+0000 7efbf67fc640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efbe00a3350 0x7efbe00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:41.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.275+0000 7efbdffff640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103] conn(0x7efbe00acba0 0x7efbe00aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:41.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.275+0000 7efbdffff640 1 -- 172.21.15.22:0/3033311573 --> [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efbe00af670 con 0x7efbe00acba0 2024-09-19T08:13:41.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.275+0000 7efbdffff640 1 -- 172.21.15.22:0/3033311573 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7efbe80f1050 con 0x7efbf8102d40 2024-09-19T08:13:41.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.275+0000 7efbf67fc640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efbe00a3350 0x7efbe00a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efbf8102430 tx=0x7efbec00b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:41.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.276+0000 7efbf77fe640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103] conn(0x7efbe00acba0 0x7efbe00aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:41.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.276+0000 7efbf77fe640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103] conn(0x7efbe00acba0 0x7efbe00aefc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:41.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.277+0000 7efbdffff640 1 -- 172.21.15.22:0/3033311573 <== osd.7 v2:172.21.15.64:6816/3980224103 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efbe00af670 con 0x7efbe00acba0 2024-09-19T08:13:41.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.290+0000 7efbfd762640 1 -- 172.21.15.22:0/3033311573 --> [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efbf8062cc0 con 0x7efbe00acba0 2024-09-19T08:13:41.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.291+0000 7efbdffff640 1 -- 172.21.15.22:0/3033311573 <== osd.7 v2:172.21.15.64:6816/3980224103 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efbf8062cc0 con 0x7efbe00acba0 2024-09-19T08:13:41.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.291+0000 7efbfd762640 1 -- 172.21.15.22:0/3033311573 >> [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103] conn(0x7efbe00acba0 msgr2=0x7efbe00aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:41.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.291+0000 7efbfd762640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103] conn(0x7efbe00acba0 0x7efbe00aefc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:41.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.291+0000 7efbfd762640 1 -- 172.21.15.22:0/3033311573 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efbe00a3350 msgr2=0x7efbe00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:41.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.291+0000 7efbfd762640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7efbe00a3350 0x7efbe00a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efbf8102430 tx=0x7efbec00b2e0 comp rx=0 tx=0).stop 2024-09-19T08:13:41.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.291+0000 7efbfd762640 1 -- 172.21.15.22:0/3033311573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efbf8102d40 msgr2=0x7efbf81009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:41.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.291+0000 7efbfd762640 1 --2- 172.21.15.22:0/3033311573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efbf8102d40 0x7efbf81009a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efbe8005cc0 tx=0x7efbe802f630 comp rx=0 tx=0).stop 2024-09-19T08:13:41.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.291+0000 7efbf77fe640 1 -- 172.21.15.22:0/3033311573 reap_dead start 2024-09-19T08:13:41.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.291+0000 7efbfd762640 1 -- 172.21.15.22:0/3033311573 shutdown_connections 2024-09-19T08:13:41.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.291+0000 7efbfd762640 1 -- 172.21.15.22:0/3033311573 >> 172.21.15.22:0/3033311573 conn(0x7efbf80fc7a0 msgr2=0x7efbf80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:41.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.292+0000 7efbfd762640 1 -- 172.21.15.22:0/3033311573 shutdown_connections 2024-09-19T08:13:41.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.292+0000 7efbfd762640 1 -- 172.21.15.22:0/3033311573 wait complete. 2024-09-19T08:13:41.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.170+0000 7fd625a45640 1 -- 172.21.15.22:0/3238102932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6201037a0 msgr2=0x7fd62010bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:41.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.170+0000 7fd625a45640 1 --2- 172.21.15.22:0/3238102932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6201037a0 0x7fd62010bc00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd614009900 tx=0x7fd61402a410 comp rx=0 tx=0).stop 2024-09-19T08:13:41.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.170+0000 7fd625a45640 1 -- 172.21.15.22:0/3238102932 shutdown_connections 2024-09-19T08:13:41.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.170+0000 7fd625a45640 1 --2- 172.21.15.22:0/3238102932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd62010c2f0 0x7fd62010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:41.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.170+0000 7fd625a45640 1 --2- 172.21.15.22:0/3238102932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6201037a0 0x7fd62010bc00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:41.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.170+0000 7fd625a45640 1 --2- 172.21.15.22:0/3238102932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd620102df0 0x7fd6201031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:41.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.170+0000 7fd625a45640 1 -- 172.21.15.22:0/3238102932 >> 172.21.15.22:0/3238102932 conn(0x7fd6200fc7d0 msgr2=0x7fd6200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:41.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.170+0000 7fd625a45640 1 -- 172.21.15.22:0/3238102932 shutdown_connections 2024-09-19T08:13:41.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.170+0000 7fd625a45640 1 -- 172.21.15.22:0/3238102932 wait complete. 2024-09-19T08:13:41.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.170+0000 7fd625a45640 1 Processor -- start 2024-09-19T08:13:41.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd625a45640 1 -- start start 2024-09-19T08:13:41.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd625a45640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd620102df0 0x7fd6201a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:41.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd625a45640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6201037a0 0x7fd6201a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:41.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd625a45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd62010c2f0 0x7fd62019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:41.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd625a45640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd6201120b0 con 0x7fd620102df0 2024-09-19T08:13:41.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd625a45640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd620111f30 con 0x7fd62010c2f0 2024-09-19T08:13:41.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd625a45640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd620112230 con 0x7fd6201037a0 2024-09-19T08:13:41.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd61effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd620102df0 0x7fd6201a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:41.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd61effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd620102df0 0x7fd6201a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58794/0 (socket says 172.21.15.22:58794) 2024-09-19T08:13:41.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd61effd640 1 -- 172.21.15.22:0/592162924 learned_addr learned my addr 172.21.15.22:0/592162924 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:41.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd61e7fc640 1 --2- 172.21.15.22:0/592162924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6201037a0 0x7fd6201a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:41.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd61effd640 1 -- 172.21.15.22:0/592162924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6201037a0 msgr2=0x7fd6201a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:41.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd61f7fe640 1 --2- 172.21.15.22:0/592162924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd62010c2f0 0x7fd62019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:41.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd61effd640 1 --2- 172.21.15.22:0/592162924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6201037a0 0x7fd6201a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:41.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd61effd640 1 -- 172.21.15.22:0/592162924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd62010c2f0 msgr2=0x7fd62019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:41.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd61effd640 1 --2- 172.21.15.22:0/592162924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd62010c2f0 0x7fd62019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:41.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd61effd640 1 -- 172.21.15.22:0/592162924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62019aed0 con 0x7fd620102df0 2024-09-19T08:13:41.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.171+0000 7fd61effd640 1 --2- 172.21.15.22:0/592162924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd620102df0 0x7fd6201a0590 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd61000d950 tx=0x7fd61000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:41.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.172+0000 7fd624a43640 1 -- 172.21.15.22:0/592162924 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610017070 con 0x7fd620102df0 2024-09-19T08:13:41.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.172+0000 7fd625a45640 1 -- 172.21.15.22:0/592162924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62019b1c0 con 0x7fd620102df0 2024-09-19T08:13:41.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.172+0000 7fd624a43640 1 -- 172.21.15.22:0/592162924 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd610009c30 con 0x7fd620102df0 2024-09-19T08:13:41.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.172+0000 7fd624a43640 1 -- 172.21.15.22:0/592162924 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61000f6b0 con 0x7fd620102df0 2024-09-19T08:13:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.172+0000 7fd625a45640 1 -- 172.21.15.22:0/592162924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6201a7510 con 0x7fd620102df0 2024-09-19T08:13:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.173+0000 7fd625a45640 1 -- 172.21.15.22:0/592162924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd620102440 con 0x7fd620102df0 2024-09-19T08:13:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.174+0000 7fd624a43640 1 -- 172.21.15.22:0/592162924 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd6100109d0 con 0x7fd620102df0 2024-09-19T08:13:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.174+0000 7fd624a43640 1 --2- 172.21.15.22:0/592162924 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd6080a32d0 0x7fd6080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.175+0000 7fd61e7fc640 1 --2- 172.21.15.22:0/592162924 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd6080a32d0 0x7fd6080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.175+0000 7fd624a43640 1 -- 172.21.15.22:0/592162924 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fd6100cf960 con 0x7fd620102df0 2024-09-19T08:13:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.175+0000 7fd624a43640 1 --2- 172.21.15.22:0/592162924 >> [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228] conn(0x7fd6080acb20 0x7fd6080aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.175+0000 7fd624a43640 1 -- 172.21.15.22:0/592162924 --> [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6080af5f0 con 0x7fd6080acb20 2024-09-19T08:13:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.175+0000 7fd624a43640 1 -- 172.21.15.22:0/592162924 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fd6100cfe40 con 0x7fd620102df0 2024-09-19T08:13:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.175+0000 7fd61e7fc640 1 --2- 172.21.15.22:0/592162924 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd6080a32d0 0x7fd6080a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd614009230 tx=0x7fd6140314b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.175+0000 7fd61f7fe640 1 --2- 172.21.15.22:0/592162924 >> [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228] conn(0x7fd6080acb20 0x7fd6080aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.176+0000 7fd61f7fe640 1 --2- 172.21.15.22:0/592162924 >> [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228] conn(0x7fd6080acb20 0x7fd6080aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.176+0000 7fd624a43640 1 -- 172.21.15.22:0/592162924 <== osd.1 v2:172.21.15.64:6800/2379700228 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6080af5f0 con 0x7fd6080acb20 2024-09-19T08:13:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.191+0000 7fd625a45640 1 -- 172.21.15.22:0/592162924 --> [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd62006a790 con 0x7fd6080acb20 2024-09-19T08:13:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.192+0000 7fd624a43640 1 -- 172.21.15.22:0/592162924 <== osd.1 v2:172.21.15.64:6800/2379700228 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd62006a790 con 0x7fd6080acb20 2024-09-19T08:13:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.192+0000 7fd625a45640 1 -- 172.21.15.22:0/592162924 >> [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228] conn(0x7fd6080acb20 msgr2=0x7fd6080aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.192+0000 7fd625a45640 1 --2- 172.21.15.22:0/592162924 >> [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228] conn(0x7fd6080acb20 0x7fd6080aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.192+0000 7fd625a45640 1 -- 172.21.15.22:0/592162924 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd6080a32d0 msgr2=0x7fd6080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.192+0000 7fd625a45640 1 --2- 172.21.15.22:0/592162924 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fd6080a32d0 0x7fd6080a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd614009230 tx=0x7fd6140314b0 comp rx=0 tx=0).stop 2024-09-19T08:13:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.192+0000 7fd625a45640 1 -- 172.21.15.22:0/592162924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd620102df0 msgr2=0x7fd6201a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.192+0000 7fd625a45640 1 --2- 172.21.15.22:0/592162924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd620102df0 0x7fd6201a0590 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd61000d950 tx=0x7fd61000de10 comp rx=0 tx=0).stop 2024-09-19T08:13:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.192+0000 7fd61f7fe640 1 -- 172.21.15.22:0/592162924 reap_dead start 2024-09-19T08:13:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.192+0000 7fd625a45640 1 -- 172.21.15.22:0/592162924 shutdown_connections 2024-09-19T08:13:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.192+0000 7fd625a45640 1 -- 172.21.15.22:0/592162924 >> 172.21.15.22:0/592162924 conn(0x7fd6200fc7d0 msgr2=0x7fd62010d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.192+0000 7fd625a45640 1 -- 172.21.15.22:0/592162924 shutdown_connections 2024-09-19T08:13:41.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.192+0000 7fd625a45640 1 -- 172.21.15.22:0/592162924 wait complete. 2024-09-19T08:13:42.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.996+0000 7f274930c640 1 -- 172.21.15.22:0/433788109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2744102790 msgr2=0x7f27441067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:42.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.996+0000 7f274930c640 1 --2- 172.21.15.22:0/433788109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2744102790 0x7f27441067e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2738009900 tx=0x7f273802a410 comp rx=0 tx=0).stop 2024-09-19T08:13:42.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.996+0000 7f274930c640 1 -- 172.21.15.22:0/433788109 shutdown_connections 2024-09-19T08:13:42.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.996+0000 7f274930c640 1 --2- 172.21.15.22:0/433788109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2744106f10 0x7f274410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:42.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.996+0000 7f274930c640 1 --2- 172.21.15.22:0/433788109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2744102790 0x7f27441067e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:42.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.996+0000 7f274930c640 1 --2- 172.21.15.22:0/433788109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2744101de0 0x7f27441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:42.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.996+0000 7f274930c640 1 -- 172.21.15.22:0/433788109 >> 172.21.15.22:0/433788109 conn(0x7f27440fd8c0 msgr2=0x7f27440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:42.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.996+0000 7f274930c640 1 -- 172.21.15.22:0/433788109 shutdown_connections 2024-09-19T08:13:42.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.996+0000 7f274930c640 1 -- 172.21.15.22:0/433788109 wait complete. 2024-09-19T08:13:42.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.997+0000 7f274930c640 1 Processor -- start 2024-09-19T08:13:42.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.997+0000 7f274930c640 1 -- start start 2024-09-19T08:13:42.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.997+0000 7f274930c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2744101de0 0x7f2744198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:42.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.997+0000 7f274930c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2744102790 0x7f2744198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:42.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.997+0000 7f274930c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2744106f10 0x7f274419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:42.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.997+0000 7f274930c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2744111f90 con 0x7f2744102790 2024-09-19T08:13:42.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.997+0000 7f274930c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2744111e10 con 0x7f2744101de0 2024-09-19T08:13:42.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.997+0000 7f274930c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2744112110 con 0x7f2744106f10 2024-09-19T08:13:42.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.997+0000 7f27427fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2744102790 0x7f2744198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:42.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.997+0000 7f27427fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2744102790 0x7f2744198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58838/0 (socket says 172.21.15.22:58838) 2024-09-19T08:13:42.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.997+0000 7f27427fc640 1 -- 172.21.15.22:0/2419154439 learned_addr learned my addr 172.21.15.22:0/2419154439 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:42.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.997+0000 7f27437fe640 1 --2- 172.21.15.22:0/2419154439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2744106f10 0x7f274419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:42.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.997+0000 7f2742ffd640 1 --2- 172.21.15.22:0/2419154439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2744101de0 0x7f2744198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:42.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.998+0000 7f27427fc640 1 -- 172.21.15.22:0/2419154439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2744106f10 msgr2=0x7f274419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:42.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.998+0000 7f27427fc640 1 --2- 172.21.15.22:0/2419154439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2744106f10 0x7f274419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:42.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.998+0000 7f27427fc640 1 -- 172.21.15.22:0/2419154439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2744101de0 msgr2=0x7f2744198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:42.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.998+0000 7f27427fc640 1 --2- 172.21.15.22:0/2419154439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2744101de0 0x7f2744198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:42.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.998+0000 7f27427fc640 1 -- 172.21.15.22:0/2419154439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274419d3e0 con 0x7f2744102790 2024-09-19T08:13:42.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.998+0000 7f27427fc640 1 --2- 172.21.15.22:0/2419154439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2744102790 0x7f2744198970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2738005ab0 tx=0x7f27380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:42.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.998+0000 7f272bfff640 1 -- 172.21.15.22:0/2419154439 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273803f030 con 0x7f2744102790 2024-09-19T08:13:42.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.998+0000 7f274930c640 1 -- 172.21.15.22:0/2419154439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274419d670 con 0x7f2744102790 2024-09-19T08:13:42.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.998+0000 7f272bfff640 1 -- 172.21.15.22:0/2419154439 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2738003ce0 con 0x7f2744102790 2024-09-19T08:13:42.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.998+0000 7f272bfff640 1 -- 172.21.15.22:0/2419154439 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738035070 con 0x7f2744102790 2024-09-19T08:13:42.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.998+0000 7f274930c640 1 -- 172.21.15.22:0/2419154439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27441a4fb0 con 0x7f2744102790 2024-09-19T08:13:42.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:41.999+0000 7f274930c640 1 -- 172.21.15.22:0/2419154439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2744062980 con 0x7f2744102790 2024-09-19T08:13:42.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.000+0000 7f272bfff640 1 -- 172.21.15.22:0/2419154439 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2738042030 con 0x7f2744102790 2024-09-19T08:13:42.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.001+0000 7f272bfff640 1 --2- 172.21.15.22:0/2419154439 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f272c0a32d0 0x7f272c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:42.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.001+0000 7f2742ffd640 1 --2- 172.21.15.22:0/2419154439 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f272c0a32d0 0x7f272c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:42.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.001+0000 7f272bfff640 1 -- 172.21.15.22:0/2419154439 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f27380f0430 con 0x7f2744102790 2024-09-19T08:13:42.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.001+0000 7f272bfff640 1 --2- 172.21.15.22:0/2419154439 >> [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212] conn(0x7f272c0acb20 0x7f272c0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:42.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.001+0000 7f272bfff640 1 -- 172.21.15.22:0/2419154439 --> [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f273803ce50 con 0x7f272c0acb20 2024-09-19T08:13:42.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.001+0000 7f272bfff640 1 -- 172.21.15.22:0/2419154439 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f27380f0910 con 0x7f2744102790 2024-09-19T08:13:42.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.002+0000 7f2742ffd640 1 --2- 172.21.15.22:0/2419154439 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f272c0a32d0 0x7f272c0a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f273400a400 tx=0x7f273400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:42.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.002+0000 7f27437fe640 1 --2- 172.21.15.22:0/2419154439 >> [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212] conn(0x7f272c0acb20 0x7f272c0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:42.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.002+0000 7f27437fe640 1 --2- 172.21.15.22:0/2419154439 >> [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212] conn(0x7f272c0acb20 0x7f272c0aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:42.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.003+0000 7f272bfff640 1 -- 172.21.15.22:0/2419154439 <== osd.5 v2:172.21.15.157:6808/2900701212 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f273803ce50 con 0x7f272c0acb20 2024-09-19T08:13:42.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.017+0000 7f274930c640 1 -- 172.21.15.22:0/2419154439 --> [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2744062d80 con 0x7f272c0acb20 2024-09-19T08:13:42.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.017+0000 7f272bfff640 1 -- 172.21.15.22:0/2419154439 <== osd.5 v2:172.21.15.157:6808/2900701212 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2744062d80 con 0x7f272c0acb20 2024-09-19T08:13:42.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.018+0000 7f274930c640 1 -- 172.21.15.22:0/2419154439 >> [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212] conn(0x7f272c0acb20 msgr2=0x7f272c0aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:42.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.018+0000 7f274930c640 1 --2- 172.21.15.22:0/2419154439 >> [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212] conn(0x7f272c0acb20 0x7f272c0aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:42.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.018+0000 7f274930c640 1 -- 172.21.15.22:0/2419154439 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f272c0a32d0 msgr2=0x7f272c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:42.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.018+0000 7f274930c640 1 --2- 172.21.15.22:0/2419154439 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f272c0a32d0 0x7f272c0a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f273400a400 tx=0x7f273400d040 comp rx=0 tx=0).stop 2024-09-19T08:13:42.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.018+0000 7f274930c640 1 -- 172.21.15.22:0/2419154439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2744102790 msgr2=0x7f2744198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:42.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.018+0000 7f274930c640 1 --2- 172.21.15.22:0/2419154439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2744102790 0x7f2744198970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2738005ab0 tx=0x7f27380042f0 comp rx=0 tx=0).stop 2024-09-19T08:13:42.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.018+0000 7f27437fe640 1 -- 172.21.15.22:0/2419154439 reap_dead start 2024-09-19T08:13:42.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.018+0000 7f274930c640 1 -- 172.21.15.22:0/2419154439 shutdown_connections 2024-09-19T08:13:42.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.018+0000 7f274930c640 1 -- 172.21.15.22:0/2419154439 >> 172.21.15.22:0/2419154439 conn(0x7f27440fd8c0 msgr2=0x7f27440fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:42.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.018+0000 7f274930c640 1 -- 172.21.15.22:0/2419154439 shutdown_connections 2024-09-19T08:13:42.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.018+0000 7f274930c640 1 -- 172.21.15.22:0/2419154439 wait complete. 2024-09-19T08:13:42.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.452+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/2252430674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b58102790 msgr2=0x7f2b581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:42.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.452+0000 7f2b5d2b4640 1 --2- 172.21.15.22:0/2252430674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b58102790 0x7f2b581067e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009900 tx=0x7f2b4002a410 comp rx=0 tx=0).stop 2024-09-19T08:13:42.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.452+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/2252430674 shutdown_connections 2024-09-19T08:13:42.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.452+0000 7f2b5d2b4640 1 --2- 172.21.15.22:0/2252430674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b58106f10 0x7f2b5810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:42.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.452+0000 7f2b5d2b4640 1 --2- 172.21.15.22:0/2252430674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b58102790 0x7f2b581067e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:42.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.452+0000 7f2b5d2b4640 1 --2- 172.21.15.22:0/2252430674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b58101de0 0x7f2b581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:42.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.452+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/2252430674 >> 172.21.15.22:0/2252430674 conn(0x7f2b580fd8c0 msgr2=0x7f2b580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:42.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.452+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/2252430674 shutdown_connections 2024-09-19T08:13:42.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.452+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/2252430674 wait complete. 2024-09-19T08:13:42.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.453+0000 7f2b5d2b4640 1 Processor -- start 2024-09-19T08:13:42.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.453+0000 7f2b5d2b4640 1 -- start start 2024-09-19T08:13:42.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.453+0000 7f2b5d2b4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b58101de0 0x7f2b580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:42.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.453+0000 7f2b5d2b4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b58102790 0x7f2b58074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:42.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.453+0000 7f2b5d2b4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b58106f10 0x7f2b58074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:42.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.453+0000 7f2b5d2b4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58111f70 con 0x7f2b58102790 2024-09-19T08:13:42.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.453+0000 7f2b5d2b4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58111df0 con 0x7f2b58106f10 2024-09-19T08:13:42.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.453+0000 7f2b5d2b4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2b581120f0 con 0x7f2b58101de0 2024-09-19T08:13:42.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.453+0000 7f2b567fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b58102790 0x7f2b58074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:42.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.453+0000 7f2b567fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b58102790 0x7f2b58074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58860/0 (socket says 172.21.15.22:58860) 2024-09-19T08:13:42.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.453+0000 7f2b567fc640 1 -- 172.21.15.22:0/3947262445 learned_addr learned my addr 172.21.15.22:0/3947262445 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:42.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.453+0000 7f2b56ffd640 1 --2- 172.21.15.22:0/3947262445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b58101de0 0x7f2b580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:42.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.453+0000 7f2b577fe640 1 --2- 172.21.15.22:0/3947262445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b58106f10 0x7f2b58074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:42.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.454+0000 7f2b567fc640 1 -- 172.21.15.22:0/3947262445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b58101de0 msgr2=0x7f2b580744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:42.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.454+0000 7f2b567fc640 1 --2- 172.21.15.22:0/3947262445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b58101de0 0x7f2b580744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:42.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.454+0000 7f2b567fc640 1 -- 172.21.15.22:0/3947262445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b58106f10 msgr2=0x7f2b58074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:42.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.454+0000 7f2b567fc640 1 --2- 172.21.15.22:0/3947262445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b58106f10 0x7f2b58074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:42.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.454+0000 7f2b567fc640 1 -- 172.21.15.22:0/3947262445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b581a6940 con 0x7f2b58102790 2024-09-19T08:13:42.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.454+0000 7f2b567fc640 1 --2- 172.21.15.22:0/3947262445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b58102790 0x7f2b58074a30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2b40006010 tx=0x7f2b400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:42.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.454+0000 7f2b4ffff640 1 -- 172.21.15.22:0/3947262445 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4003f030 con 0x7f2b58102790 2024-09-19T08:13:42.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.454+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/3947262445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b581a6bd0 con 0x7f2b58102790 2024-09-19T08:13:42.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.454+0000 7f2b4ffff640 1 -- 172.21.15.22:0/3947262445 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b40003ce0 con 0x7f2b58102790 2024-09-19T08:13:42.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.454+0000 7f2b4ffff640 1 -- 172.21.15.22:0/3947262445 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40035070 con 0x7f2b58102790 2024-09-19T08:13:42.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.454+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/3947262445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b581a70b0 con 0x7f2b58102790 2024-09-19T08:13:42.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.455+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/3947262445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2b580628c0 con 0x7f2b58102790 2024-09-19T08:13:42.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.456+0000 7f2b4ffff640 1 -- 172.21.15.22:0/3947262445 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2b40042030 con 0x7f2b58102790 2024-09-19T08:13:42.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.457+0000 7f2b4ffff640 1 --2- 172.21.15.22:0/3947262445 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b440a32d0 0x7f2b440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:42.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.457+0000 7f2b56ffd640 1 --2- 172.21.15.22:0/3947262445 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b440a32d0 0x7f2b440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:42.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.457+0000 7f2b4ffff640 1 -- 172.21.15.22:0/3947262445 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f2b400f03a0 con 0x7f2b58102790 2024-09-19T08:13:42.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.457+0000 7f2b4ffff640 1 --2- 172.21.15.22:0/3947262445 >> [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990] conn(0x7f2b440acb20 0x7f2b440aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:42.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.457+0000 7f2b4ffff640 1 -- 172.21.15.22:0/3947262445 --> [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b40036e90 con 0x7f2b440acb20 2024-09-19T08:13:42.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.457+0000 7f2b4ffff640 1 -- 172.21.15.22:0/3947262445 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f2b400f0880 con 0x7f2b58102790 2024-09-19T08:13:42.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.457+0000 7f2b577fe640 1 --2- 172.21.15.22:0/3947262445 >> [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990] conn(0x7f2b440acb20 0x7f2b440aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:42.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.457+0000 7f2b56ffd640 1 --2- 172.21.15.22:0/3947262445 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b440a32d0 0x7f2b440a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009bd0 tx=0x7f2b48005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:42.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.458+0000 7f2b577fe640 1 --2- 172.21.15.22:0/3947262445 >> [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990] conn(0x7f2b440acb20 0x7f2b440aef40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:42.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.458+0000 7f2b4ffff640 1 -- 172.21.15.22:0/3947262445 <== osd.9 v2:172.21.15.22:6826/710033990 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b40036e90 con 0x7f2b440acb20 2024-09-19T08:13:42.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.472+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/3947262445 --> [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2b58062d40 con 0x7f2b440acb20 2024-09-19T08:13:42.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.472+0000 7f2b4ffff640 1 -- 172.21.15.22:0/3947262445 <== osd.9 v2:172.21.15.22:6826/710033990 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2b58062d40 con 0x7f2b440acb20 2024-09-19T08:13:42.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.473+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/3947262445 >> [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990] conn(0x7f2b440acb20 msgr2=0x7f2b440aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:42.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.473+0000 7f2b5d2b4640 1 --2- 172.21.15.22:0/3947262445 >> [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990] conn(0x7f2b440acb20 0x7f2b440aef40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:42.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.473+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/3947262445 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b440a32d0 msgr2=0x7f2b440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:42.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.473+0000 7f2b5d2b4640 1 --2- 172.21.15.22:0/3947262445 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b440a32d0 0x7f2b440a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009bd0 tx=0x7f2b48005f90 comp rx=0 tx=0).stop 2024-09-19T08:13:42.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.473+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/3947262445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b58102790 msgr2=0x7f2b58074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:42.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.473+0000 7f2b5d2b4640 1 --2- 172.21.15.22:0/3947262445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b58102790 0x7f2b58074a30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2b40006010 tx=0x7f2b400042f0 comp rx=0 tx=0).stop 2024-09-19T08:13:42.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.473+0000 7f2b577fe640 1 -- 172.21.15.22:0/3947262445 reap_dead start 2024-09-19T08:13:42.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.473+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/3947262445 shutdown_connections 2024-09-19T08:13:42.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.473+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/3947262445 >> 172.21.15.22:0/3947262445 conn(0x7f2b580fd8c0 msgr2=0x7f2b580fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:42.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.473+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/3947262445 shutdown_connections 2024-09-19T08:13:42.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:42.473+0000 7f2b5d2b4640 1 -- 172.21.15.22:0/3947262445 wait complete. 2024-09-19T08:13:42.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:42 smithi064 ceph-mon[34900]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-19T08:13:42.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:42 smithi157 ceph-mon[35143]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-19T08:13:42.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:42 smithi022 ceph-mon[30963]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-19T08:13:43.284 INFO:teuthology.orchestra.run.smithi022.stdout:279172874272 2024-09-19T08:13:43.285 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T08:13:44.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.052+0000 7f46ac58f640 1 -- 172.21.15.22:0/209642020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a410c170 msgr2=0x7f46a410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:44.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.052+0000 7f46ac58f640 1 --2- 172.21.15.22:0/209642020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a410c170 0x7f46a410e610 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f469c009ca0 tx=0x7f469c02a950 comp rx=0 tx=0).stop 2024-09-19T08:13:44.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 -- 172.21.15.22:0/209642020 shutdown_connections 2024-09-19T08:13:44.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 --2- 172.21.15.22:0/209642020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a410c170 0x7f46a410e610 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:44.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 --2- 172.21.15.22:0/209642020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46a4103660 0x7f46a410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:44.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 --2- 172.21.15.22:0/209642020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46a4102d40 0x7f46a4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:44.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 -- 172.21.15.22:0/209642020 >> 172.21.15.22:0/209642020 conn(0x7f46a40fc7a0 msgr2=0x7f46a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:44.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 -- 172.21.15.22:0/209642020 shutdown_connections 2024-09-19T08:13:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 -- 172.21.15.22:0/209642020 wait complete. 2024-09-19T08:13:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 Processor -- start 2024-09-19T08:13:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 -- start start 2024-09-19T08:13:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46a4102d40 0x7f46a4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a4103660 0x7f46a4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46a410c170 0x7f46a419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4112000 con 0x7f46a4103660 2024-09-19T08:13:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4111e80 con 0x7f46a4102d40 2024-09-19T08:13:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.053+0000 7f46ac58f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4112180 con 0x7f46a410c170 2024-09-19T08:13:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.054+0000 7f46a9b2a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a4103660 0x7f46a4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.054+0000 7f46a9b2a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a4103660 0x7f46a4198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58874/0 (socket says 172.21.15.22:58874) 2024-09-19T08:13:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.054+0000 7f46a9b2a640 1 -- 172.21.15.22:0/670256014 learned_addr learned my addr 172.21.15.22:0/670256014 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.054+0000 7f46aab2c640 1 --2- 172.21.15.22:0/670256014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46a410c170 0x7f46a419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.054+0000 7f46aa32b640 1 --2- 172.21.15.22:0/670256014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46a4102d40 0x7f46a4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.054+0000 7f46a9b2a640 1 -- 172.21.15.22:0/670256014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46a410c170 msgr2=0x7f46a419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.054+0000 7f46a9b2a640 1 --2- 172.21.15.22:0/670256014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46a410c170 0x7f46a419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.054+0000 7f46a9b2a640 1 -- 172.21.15.22:0/670256014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46a4102d40 msgr2=0x7f46a4198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.054+0000 7f46a9b2a640 1 --2- 172.21.15.22:0/670256014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46a4102d40 0x7f46a4198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.054+0000 7f46a9b2a640 1 -- 172.21.15.22:0/670256014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a419d3d0 con 0x7f46a4103660 2024-09-19T08:13:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.054+0000 7f46a9b2a640 1 --2- 172.21.15.22:0/670256014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a4103660 0x7f46a4198960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f469400b3f0 tx=0x7f469400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.054+0000 7f469b7fe640 1 -- 172.21.15.22:0/670256014 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694015020 con 0x7f46a4103660 2024-09-19T08:13:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.054+0000 7f469b7fe640 1 -- 172.21.15.22:0/670256014 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4694011d10 con 0x7f46a4103660 2024-09-19T08:13:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.054+0000 7f46ac58f640 1 -- 172.21.15.22:0/670256014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a419d6c0 con 0x7f46a4103660 2024-09-19T08:13:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.055+0000 7f469b7fe640 1 -- 172.21.15.22:0/670256014 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469400f2e0 con 0x7f46a4103660 2024-09-19T08:13:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.055+0000 7f46ac58f640 1 -- 172.21.15.22:0/670256014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a41a4fa0 con 0x7f46a4103660 2024-09-19T08:13:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.055+0000 7f46ac58f640 1 -- 172.21.15.22:0/670256014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46a4062940 con 0x7f46a4103660 2024-09-19T08:13:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.057+0000 7f469b7fe640 1 -- 172.21.15.22:0/670256014 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4694018020 con 0x7f46a4103660 2024-09-19T08:13:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.057+0000 7f469b7fe640 1 --2- 172.21.15.22:0/670256014 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f46900a3350 0x7f46900a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.057+0000 7f46aa32b640 1 --2- 172.21.15.22:0/670256014 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f46900a3350 0x7f46900a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.057+0000 7f469b7fe640 1 -- 172.21.15.22:0/670256014 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f46940cf530 con 0x7f46a4103660 2024-09-19T08:13:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.058+0000 7f469b7fe640 1 --2- 172.21.15.22:0/670256014 >> [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242] conn(0x7f46900acba0 0x7f46900aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.058+0000 7f469b7fe640 1 -- 172.21.15.22:0/670256014 --> [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4694013ec0 con 0x7f46900acba0 2024-09-19T08:13:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.058+0000 7f469b7fe640 1 -- 172.21.15.22:0/670256014 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f46940cfa10 con 0x7f46a4103660 2024-09-19T08:13:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.058+0000 7f46aab2c640 1 --2- 172.21.15.22:0/670256014 >> [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242] conn(0x7f46900acba0 0x7f46900aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.058+0000 7f46aa32b640 1 --2- 172.21.15.22:0/670256014 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f46900a3350 0x7f46900a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f46a000bbe0 tx=0x7f46a000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.058+0000 7f46aab2c640 1 --2- 172.21.15.22:0/670256014 >> [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242] conn(0x7f46900acba0 0x7f46900aefc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.059+0000 7f469b7fe640 1 -- 172.21.15.22:0/670256014 <== osd.6 v2:172.21.15.22:6818/4264426242 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4694013ec0 con 0x7f46900acba0 2024-09-19T08:13:44.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.072+0000 7f46ac58f640 1 -- 172.21.15.22:0/670256014 --> [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46a4062dc0 con 0x7f46900acba0 2024-09-19T08:13:44.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.073+0000 7f469b7fe640 1 -- 172.21.15.22:0/670256014 <== osd.6 v2:172.21.15.22:6818/4264426242 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f46a4062dc0 con 0x7f46900acba0 2024-09-19T08:13:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.073+0000 7f46ac58f640 1 -- 172.21.15.22:0/670256014 >> [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242] conn(0x7f46900acba0 msgr2=0x7f46900aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.073+0000 7f46ac58f640 1 --2- 172.21.15.22:0/670256014 >> [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242] conn(0x7f46900acba0 0x7f46900aefc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.073+0000 7f46ac58f640 1 -- 172.21.15.22:0/670256014 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f46900a3350 msgr2=0x7f46900a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.073+0000 7f46ac58f640 1 --2- 172.21.15.22:0/670256014 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f46900a3350 0x7f46900a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f46a000bbe0 tx=0x7f46a000b390 comp rx=0 tx=0).stop 2024-09-19T08:13:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.073+0000 7f46ac58f640 1 -- 172.21.15.22:0/670256014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a4103660 msgr2=0x7f46a4198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.073+0000 7f46ac58f640 1 --2- 172.21.15.22:0/670256014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a4103660 0x7f46a4198960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f469400b3f0 tx=0x7f469400b8b0 comp rx=0 tx=0).stop 2024-09-19T08:13:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.073+0000 7f46aab2c640 1 -- 172.21.15.22:0/670256014 reap_dead start 2024-09-19T08:13:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.073+0000 7f46ac58f640 1 -- 172.21.15.22:0/670256014 shutdown_connections 2024-09-19T08:13:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.073+0000 7f46ac58f640 1 -- 172.21.15.22:0/670256014 >> 172.21.15.22:0/670256014 conn(0x7f46a40fc7a0 msgr2=0x7f46a40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.073+0000 7f46ac58f640 1 -- 172.21.15.22:0/670256014 shutdown_connections 2024-09-19T08:13:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.073+0000 7f46ac58f640 1 -- 172.21.15.22:0/670256014 wait complete. 2024-09-19T08:13:44.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:44 smithi022 ceph-mon[30963]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-19T08:13:44.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:44 smithi064 ceph-mon[34900]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-19T08:13:44.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:44 smithi157 ceph-mon[35143]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-19T08:13:45.297 INFO:teuthology.orchestra.run.smithi022.stdout:219043332137 2024-09-19T08:13:45.298 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T08:13:45.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.515+0000 7fa6231fb640 1 -- 172.21.15.22:0/2811079982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa61c102570 msgr2=0x7fa61c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:45.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.515+0000 7fa6231fb640 1 --2- 172.21.15.22:0/2811079982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa61c102570 0x7fa61c1029b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa604009900 tx=0x7fa60402a480 comp rx=0 tx=0).stop 2024-09-19T08:13:45.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.515+0000 7fa6231fb640 1 -- 172.21.15.22:0/2811079982 shutdown_connections 2024-09-19T08:13:45.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.515+0000 7fa6231fb640 1 --2- 172.21.15.22:0/2811079982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa61c102ef0 0x7fa61c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:45.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.515+0000 7fa6231fb640 1 --2- 172.21.15.22:0/2811079982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa61c102570 0x7fa61c1029b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:45.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.515+0000 7fa6231fb640 1 --2- 172.21.15.22:0/2811079982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa61c100e20 0x7fa61c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:45.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.515+0000 7fa6231fb640 1 -- 172.21.15.22:0/2811079982 >> 172.21.15.22:0/2811079982 conn(0x7fa61c0fc7d0 msgr2=0x7fa61c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:45.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.515+0000 7fa6231fb640 1 -- 172.21.15.22:0/2811079982 shutdown_connections 2024-09-19T08:13:45.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.515+0000 7fa6231fb640 1 -- 172.21.15.22:0/2811079982 wait complete. 2024-09-19T08:13:45.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa6231fb640 1 Processor -- start 2024-09-19T08:13:45.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa6231fb640 1 -- start start 2024-09-19T08:13:45.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa6231fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa61c100e20 0x7fa61c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:45.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa6231fb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa61c102570 0x7fa61c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:45.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa6231fb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa61c102ef0 0x7fa61c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:45.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa6231fb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c110f50 con 0x7fa61c102ef0 2024-09-19T08:13:45.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa6231fb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c110dd0 con 0x7fa61c102570 2024-09-19T08:13:45.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa6231fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1110d0 con 0x7fa61c100e20 2024-09-19T08:13:45.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa621798640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa61c102ef0 0x7fa61c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:45.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa621798640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa61c102ef0 0x7fa61c0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58908/0 (socket says 172.21.15.22:58908) 2024-09-19T08:13:45.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa621798640 1 -- 172.21.15.22:0/4255209376 learned_addr learned my addr 172.21.15.22:0/4255209376 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:45.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa620f97640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa61c100e20 0x7fa61c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:45.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa61bfff640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa61c102570 0x7fa61c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:45.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa621798640 1 -- 172.21.15.22:0/4255209376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa61c100e20 msgr2=0x7fa61c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:45.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa621798640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa61c100e20 0x7fa61c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:45.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa621798640 1 -- 172.21.15.22:0/4255209376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa61c102570 msgr2=0x7fa61c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:45.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa621798640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa61c102570 0x7fa61c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:45.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa621798640 1 -- 172.21.15.22:0/4255209376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c1a6940 con 0x7fa61c102ef0 2024-09-19T08:13:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.516+0000 7fa61bfff640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa61c102570 0x7fa61c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:13:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.517+0000 7fa621798640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa61c102ef0 0x7fa61c0fd700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa61400cce0 tx=0x7fa61400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.517+0000 7fa619ffb640 1 -- 172.21.15.22:0/4255209376 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61400f450 con 0x7fa61c102ef0 2024-09-19T08:13:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.517+0000 7fa619ffb640 1 -- 172.21.15.22:0/4255209376 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa61400fad0 con 0x7fa61c102ef0 2024-09-19T08:13:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.517+0000 7fa6231fb640 1 -- 172.21.15.22:0/4255209376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61c1a6c30 con 0x7fa61c102ef0 2024-09-19T08:13:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.517+0000 7fa619ffb640 1 -- 172.21.15.22:0/4255209376 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614018b60 con 0x7fa61c102ef0 2024-09-19T08:13:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.517+0000 7fa6231fb640 1 -- 172.21.15.22:0/4255209376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61c1a7170 con 0x7fa61c102ef0 2024-09-19T08:13:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.518+0000 7fa6231fb640 1 -- 172.21.15.22:0/4255209376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa61c0628c0 con 0x7fa61c102ef0 2024-09-19T08:13:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.519+0000 7fa619ffb640 1 -- 172.21.15.22:0/4255209376 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa61400fc80 con 0x7fa61c102ef0 2024-09-19T08:13:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.520+0000 7fa619ffb640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa6080a3350 0x7fa6080a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.520+0000 7fa619ffb640 1 -- 172.21.15.22:0/4255209376 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fa6140d0240 con 0x7fa61c102ef0 2024-09-19T08:13:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.520+0000 7fa620f97640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa6080a3350 0x7fa6080a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.520+0000 7fa619ffb640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456] conn(0x7fa6080acba0 0x7fa6080aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:45.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.520+0000 7fa619ffb640 1 -- 172.21.15.22:0/4255209376 --> [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa614012ec0 con 0x7fa6080acba0 2024-09-19T08:13:45.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.520+0000 7fa619ffb640 1 -- 172.21.15.22:0/4255209376 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fa6140d0720 con 0x7fa61c102ef0 2024-09-19T08:13:45.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.520+0000 7fa61bfff640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456] conn(0x7fa6080acba0 0x7fa6080aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:45.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.520+0000 7fa620f97640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa6080a3350 0x7fa6080a5810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00abe0 tx=0x7fa60c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:45.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.521+0000 7fa61bfff640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456] conn(0x7fa6080acba0 0x7fa6080aefc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:45.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.521+0000 7fa619ffb640 1 -- 172.21.15.22:0/4255209376 <== osd.3 v2:172.21.15.22:6810/4225229456 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa614012ec0 con 0x7fa6080acba0 2024-09-19T08:13:45.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.535+0000 7fa6231fb640 1 -- 172.21.15.22:0/4255209376 --> [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa61c062d40 con 0x7fa6080acba0 2024-09-19T08:13:45.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.535+0000 7fa619ffb640 1 -- 172.21.15.22:0/4255209376 <== osd.3 v2:172.21.15.22:6810/4225229456 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa61c062d40 con 0x7fa6080acba0 2024-09-19T08:13:45.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.535+0000 7fa6231fb640 1 -- 172.21.15.22:0/4255209376 >> [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456] conn(0x7fa6080acba0 msgr2=0x7fa6080aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:45.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.535+0000 7fa6231fb640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456] conn(0x7fa6080acba0 0x7fa6080aefc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:45.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.536+0000 7fa6231fb640 1 -- 172.21.15.22:0/4255209376 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa6080a3350 msgr2=0x7fa6080a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:45.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.536+0000 7fa6231fb640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa6080a3350 0x7fa6080a5810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00abe0 tx=0x7fa60c00a2d0 comp rx=0 tx=0).stop 2024-09-19T08:13:45.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.536+0000 7fa6231fb640 1 -- 172.21.15.22:0/4255209376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa61c102ef0 msgr2=0x7fa61c0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:45.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.536+0000 7fa6231fb640 1 --2- 172.21.15.22:0/4255209376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa61c102ef0 0x7fa61c0fd700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa61400cce0 tx=0x7fa61400a620 comp rx=0 tx=0).stop 2024-09-19T08:13:45.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.536+0000 7fa621798640 1 -- 172.21.15.22:0/4255209376 reap_dead start 2024-09-19T08:13:45.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.536+0000 7fa6231fb640 1 -- 172.21.15.22:0/4255209376 shutdown_connections 2024-09-19T08:13:45.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.536+0000 7fa6231fb640 1 -- 172.21.15.22:0/4255209376 >> 172.21.15.22:0/4255209376 conn(0x7fa61c0fc7d0 msgr2=0x7fa61c06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:45.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.536+0000 7fa6231fb640 1 -- 172.21.15.22:0/4255209376 shutdown_connections 2024-09-19T08:13:45.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:45.536+0000 7fa6231fb640 1 -- 172.21.15.22:0/4255209376 wait complete. 2024-09-19T08:13:46.329 INFO:teuthology.orchestra.run.smithi022.stdout:107374182463 2024-09-19T08:13:46.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T08:13:46.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.271+0000 7f26615dc640 1 -- 172.21.15.22:0/2273109166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102d40 msgr2=0x7f265c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:46.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.271+0000 7f26615dc640 1 --2- 172.21.15.22:0/2273109166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102d40 0x7f265c103120 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f264c009900 tx=0x7f264c02a480 comp rx=0 tx=0).stop 2024-09-19T08:13:46.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.271+0000 7f26615dc640 1 -- 172.21.15.22:0/2273109166 shutdown_connections 2024-09-19T08:13:46.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.271+0000 7f26615dc640 1 --2- 172.21.15.22:0/2273109166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f265c10c170 0x7f265c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:46.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.271+0000 7f26615dc640 1 --2- 172.21.15.22:0/2273109166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f265c103660 0x7f265c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:46.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.271+0000 7f26615dc640 1 --2- 172.21.15.22:0/2273109166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102d40 0x7f265c103120 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:46.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.271+0000 7f26615dc640 1 -- 172.21.15.22:0/2273109166 >> 172.21.15.22:0/2273109166 conn(0x7f265c0fc7a0 msgr2=0x7f265c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:46.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.271+0000 7f26615dc640 1 -- 172.21.15.22:0/2273109166 shutdown_connections 2024-09-19T08:13:46.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.271+0000 7f26615dc640 1 -- 172.21.15.22:0/2273109166 wait complete. 2024-09-19T08:13:46.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.271+0000 7f26615dc640 1 Processor -- start 2024-09-19T08:13:46.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f26615dc640 1 -- start start 2024-09-19T08:13:46.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f26615dc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102d40 0x7f265c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:46.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f26615dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f265c103660 0x7f265c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:46.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f26615dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f265c10c170 0x7f265c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:46.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f26615dc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f265c111f70 con 0x7f265c102d40 2024-09-19T08:13:46.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f26615dc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f265c111df0 con 0x7f265c10c170 2024-09-19T08:13:46.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f26615dc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1120f0 con 0x7f265c103660 2024-09-19T08:13:46.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f265affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102d40 0x7f265c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f265affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102d40 0x7f265c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58898/0 (socket says 172.21.15.22:58898) 2024-09-19T08:13:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f265affd640 1 -- 172.21.15.22:0/1010769050 learned_addr learned my addr 172.21.15.22:0/1010769050 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f265b7fe640 1 --2- 172.21.15.22:0/1010769050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f265c10c170 0x7f265c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f265affd640 1 -- 172.21.15.22:0/1010769050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f265c103660 msgr2=0x7f265c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f265affd640 1 --2- 172.21.15.22:0/1010769050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f265c103660 0x7f265c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f265affd640 1 -- 172.21.15.22:0/1010769050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f265c10c170 msgr2=0x7f265c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f265affd640 1 --2- 172.21.15.22:0/1010769050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f265c10c170 0x7f265c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f265affd640 1 -- 172.21.15.22:0/1010769050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c1a68e0 con 0x7f265c102d40 2024-09-19T08:13:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f265b7fe640 1 --2- 172.21.15.22:0/1010769050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f265c10c170 0x7f265c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:13:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f265affd640 1 --2- 172.21.15.22:0/1010769050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102d40 0x7f265c1009a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f264c006010 tx=0x7f264c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.272+0000 7f2643fff640 1 -- 172.21.15.22:0/1010769050 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c03f030 con 0x7f265c102d40 2024-09-19T08:13:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.273+0000 7f26615dc640 1 -- 172.21.15.22:0/1010769050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265c1a6b70 con 0x7f265c102d40 2024-09-19T08:13:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.273+0000 7f2643fff640 1 -- 172.21.15.22:0/1010769050 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f264c003ce0 con 0x7f265c102d40 2024-09-19T08:13:46.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.273+0000 7f2643fff640 1 -- 172.21.15.22:0/1010769050 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c035070 con 0x7f265c102d40 2024-09-19T08:13:46.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.273+0000 7f26615dc640 1 -- 172.21.15.22:0/1010769050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265c1a7050 con 0x7f265c102d40 2024-09-19T08:13:46.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.273+0000 7f26615dc640 1 -- 172.21.15.22:0/1010769050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f265c0628c0 con 0x7f265c102d40 2024-09-19T08:13:46.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.275+0000 7f2643fff640 1 -- 172.21.15.22:0/1010769050 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f264c042030 con 0x7f265c102d40 2024-09-19T08:13:46.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.275+0000 7f2643fff640 1 --2- 172.21.15.22:0/1010769050 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f26440a32d0 0x7f26440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:46.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.276+0000 7f265a7fc640 1 --2- 172.21.15.22:0/1010769050 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f26440a32d0 0x7f26440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:46.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.276+0000 7f2643fff640 1 -- 172.21.15.22:0/1010769050 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f264c0f0740 con 0x7f265c102d40 2024-09-19T08:13:46.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.276+0000 7f2643fff640 1 --2- 172.21.15.22:0/1010769050 >> [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423] conn(0x7f26440acb20 0x7f26440aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:46.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.276+0000 7f2643fff640 1 -- 172.21.15.22:0/1010769050 --> [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f264c03be50 con 0x7f26440acb20 2024-09-19T08:13:46.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.276+0000 7f2643fff640 1 -- 172.21.15.22:0/1010769050 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f264c0f0c20 con 0x7f265c102d40 2024-09-19T08:13:46.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.276+0000 7f265a7fc640 1 --2- 172.21.15.22:0/1010769050 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f26440a32d0 0x7f26440a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f265c102430 tx=0x7f2650005d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:46.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.276+0000 7f265b7fe640 1 --2- 172.21.15.22:0/1010769050 >> [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423] conn(0x7f26440acb20 0x7f26440aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:46.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.277+0000 7f265b7fe640 1 --2- 172.21.15.22:0/1010769050 >> [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423] conn(0x7f26440acb20 0x7f26440aef40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:46.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.277+0000 7f2643fff640 1 -- 172.21.15.22:0/1010769050 <== osd.4 v2:172.21.15.64:6808/1858090423 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f264c03be50 con 0x7f26440acb20 2024-09-19T08:13:46.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.291+0000 7f26615dc640 1 -- 172.21.15.22:0/1010769050 --> [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f265c062cc0 con 0x7f26440acb20 2024-09-19T08:13:46.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.292+0000 7f2643fff640 1 -- 172.21.15.22:0/1010769050 <== osd.4 v2:172.21.15.64:6808/1858090423 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f265c062cc0 con 0x7f26440acb20 2024-09-19T08:13:46.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.292+0000 7f26615dc640 1 -- 172.21.15.22:0/1010769050 >> [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423] conn(0x7f26440acb20 msgr2=0x7f26440aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:46.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.292+0000 7f26615dc640 1 --2- 172.21.15.22:0/1010769050 >> [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423] conn(0x7f26440acb20 0x7f26440aef40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:46.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.292+0000 7f26615dc640 1 -- 172.21.15.22:0/1010769050 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f26440a32d0 msgr2=0x7f26440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:46.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.292+0000 7f26615dc640 1 --2- 172.21.15.22:0/1010769050 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f26440a32d0 0x7f26440a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f265c102430 tx=0x7f2650005d10 comp rx=0 tx=0).stop 2024-09-19T08:13:46.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.292+0000 7f26615dc640 1 -- 172.21.15.22:0/1010769050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102d40 msgr2=0x7f265c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:46.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.292+0000 7f26615dc640 1 --2- 172.21.15.22:0/1010769050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f265c102d40 0x7f265c1009a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f264c006010 tx=0x7f264c0042f0 comp rx=0 tx=0).stop 2024-09-19T08:13:46.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.292+0000 7f265b7fe640 1 -- 172.21.15.22:0/1010769050 reap_dead start 2024-09-19T08:13:46.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.292+0000 7f26615dc640 1 -- 172.21.15.22:0/1010769050 shutdown_connections 2024-09-19T08:13:46.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.292+0000 7f26615dc640 1 -- 172.21.15.22:0/1010769050 >> 172.21.15.22:0/1010769050 conn(0x7f265c0fc7a0 msgr2=0x7f265c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:46.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.292+0000 7f26615dc640 1 -- 172.21.15.22:0/1010769050 shutdown_connections 2024-09-19T08:13:46.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:44.292+0000 7f26615dc640 1 -- 172.21.15.22:0/1010769050 wait complete. 2024-09-19T08:13:46.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:46 smithi064 ceph-mon[34900]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-19T08:13:46.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:46 smithi157 ceph-mon[35143]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-19T08:13:46.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:46 smithi022 ceph-mon[30963]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-19T08:13:47.158 INFO:teuthology.orchestra.run.smithi022.stdout:141733920826 2024-09-19T08:13:47.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T08:13:48.269 INFO:teuthology.orchestra.run.smithi022.stdout:77309411395 2024-09-19T08:13:48.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T08:13:48.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:48 smithi022 ceph-mon[30963]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-19T08:13:48.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:48 smithi064 ceph-mon[34900]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-19T08:13:48.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:48 smithi157 ceph-mon[35143]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-19T08:13:48.868 INFO:teuthology.orchestra.run.smithi022.stdout:197568495662 2024-09-19T08:13:48.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T08:13:49.346 INFO:teuthology.orchestra.run.smithi022.stdout:55834574920 2024-09-19T08:13:49.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T08:13:49.745 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:49.787 INFO:teuthology.orchestra.run.smithi022.stdout:163208757300 2024-09-19T08:13:49.787 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T08:13:50.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.543+0000 7ffaee634640 1 -- 172.21.15.22:0/2934906685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffae8106f10 msgr2=0x7ffae810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:50.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.543+0000 7ffaee634640 1 --2- 172.21.15.22:0/2934906685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffae8106f10 0x7ffae810ea90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009ca0 tx=0x7ffae002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:13:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.543+0000 7ffaee634640 1 -- 172.21.15.22:0/2934906685 shutdown_connections 2024-09-19T08:13:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.543+0000 7ffaee634640 1 --2- 172.21.15.22:0/2934906685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffae8106f10 0x7ffae810ea90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.543+0000 7ffaee634640 1 --2- 172.21.15.22:0/2934906685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae8102790 0x7ffae81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.543+0000 7ffaee634640 1 --2- 172.21.15.22:0/2934906685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffae8101de0 0x7ffae81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.543+0000 7ffaee634640 1 -- 172.21.15.22:0/2934906685 >> 172.21.15.22:0/2934906685 conn(0x7ffae80fd8c0 msgr2=0x7ffae80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.543+0000 7ffaee634640 1 -- 172.21.15.22:0/2934906685 shutdown_connections 2024-09-19T08:13:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.543+0000 7ffaee634640 1 -- 172.21.15.22:0/2934906685 wait complete. 2024-09-19T08:13:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.543+0000 7ffaee634640 1 Processor -- start 2024-09-19T08:13:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffaee634640 1 -- start start 2024-09-19T08:13:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffaee634640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffae8101de0 0x7ffae80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffaee634640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffae8102790 0x7ffae8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffaee634640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae8106f10 0x7ffae8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffaee634640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8111fe0 con 0x7ffae8102790 2024-09-19T08:13:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffaee634640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8111e60 con 0x7ffae8101de0 2024-09-19T08:13:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffaee634640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8112160 con 0x7ffae8106f10 2024-09-19T08:13:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffae77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffae8102790 0x7ffae8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffae77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffae8102790 0x7ffae8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42230/0 (socket says 172.21.15.22:42230) 2024-09-19T08:13:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffae77fe640 1 -- 172.21.15.22:0/2142548863 learned_addr learned my addr 172.21.15.22:0/2142548863 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffaecbd1640 1 --2- 172.21.15.22:0/2142548863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae8106f10 0x7ffae8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffae7fff640 1 --2- 172.21.15.22:0/2142548863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffae8101de0 0x7ffae80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffae77fe640 1 -- 172.21.15.22:0/2142548863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae8106f10 msgr2=0x7ffae8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffae77fe640 1 --2- 172.21.15.22:0/2142548863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae8106f10 0x7ffae8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffae77fe640 1 -- 172.21.15.22:0/2142548863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffae8101de0 msgr2=0x7ffae80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffae77fe640 1 --2- 172.21.15.22:0/2142548863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffae8101de0 0x7ffae80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.544+0000 7ffae77fe640 1 -- 172.21.15.22:0/2142548863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae81a6920 con 0x7ffae8102790 2024-09-19T08:13:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.545+0000 7ffae77fe640 1 --2- 172.21.15.22:0/2142548863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffae8102790 0x7ffae8074a30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00b4a0 tx=0x7ffadc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.545+0000 7ffae57fa640 1 -- 172.21.15.22:0/2142548863 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc014070 con 0x7ffae8102790 2024-09-19T08:13:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.545+0000 7ffae57fa640 1 -- 172.21.15.22:0/2142548863 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffadc010af0 con 0x7ffae8102790 2024-09-19T08:13:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.545+0000 7ffaee634640 1 -- 172.21.15.22:0/2142548863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae81a6c10 con 0x7ffae8102790 2024-09-19T08:13:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.545+0000 7ffae57fa640 1 -- 172.21.15.22:0/2142548863 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc017d10 con 0x7ffae8102790 2024-09-19T08:13:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.545+0000 7ffaee634640 1 -- 172.21.15.22:0/2142548863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae81a7100 con 0x7ffae8102790 2024-09-19T08:13:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.546+0000 7ffaee634640 1 -- 172.21.15.22:0/2142548863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffae80628c0 con 0x7ffae8102790 2024-09-19T08:13:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.547+0000 7ffae57fa640 1 -- 172.21.15.22:0/2142548863 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ffadc017070 con 0x7ffae8102790 2024-09-19T08:13:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.548+0000 7ffae57fa640 1 --2- 172.21.15.22:0/2142548863 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ffad00a32d0 0x7ffad00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.548+0000 7ffae7fff640 1 --2- 172.21.15.22:0/2142548863 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ffad00a32d0 0x7ffad00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.548+0000 7ffae57fa640 1 -- 172.21.15.22:0/2142548863 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7ffadc0cf130 con 0x7ffae8102790 2024-09-19T08:13:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.548+0000 7ffae57fa640 1 --2- 172.21.15.22:0/2142548863 >> [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194] conn(0x7ffad00acb20 0x7ffad00aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.548+0000 7ffae57fa640 1 -- 172.21.15.22:0/2142548863 --> [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffad00af5f0 con 0x7ffad00acb20 2024-09-19T08:13:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.548+0000 7ffae57fa640 1 -- 172.21.15.22:0/2142548863 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7ffadc0cf4e0 con 0x7ffae8102790 2024-09-19T08:13:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.548+0000 7ffae7fff640 1 --2- 172.21.15.22:0/2142548863 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ffad00a32d0 0x7ffad00a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffad80058c0 tx=0x7ffad800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.548+0000 7ffaecbd1640 1 --2- 172.21.15.22:0/2142548863 >> [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194] conn(0x7ffad00acb20 0x7ffad00aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.549+0000 7ffaecbd1640 1 --2- 172.21.15.22:0/2142548863 >> [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194] conn(0x7ffad00acb20 0x7ffad00aef40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.550+0000 7ffae57fa640 1 -- 172.21.15.22:0/2142548863 <== osd.11 v2:172.21.15.157:6824/1918889194 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffad00af5f0 con 0x7ffad00acb20 2024-09-19T08:13:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.563+0000 7ffaee634640 1 -- 172.21.15.22:0/2142548863 --> [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffae8062cc0 con 0x7ffad00acb20 2024-09-19T08:13:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.564+0000 7ffae57fa640 1 -- 172.21.15.22:0/2142548863 <== osd.11 v2:172.21.15.157:6824/1918889194 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffae8062cc0 con 0x7ffad00acb20 2024-09-19T08:13:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.564+0000 7ffaee634640 1 -- 172.21.15.22:0/2142548863 >> [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194] conn(0x7ffad00acb20 msgr2=0x7ffad00aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.564+0000 7ffaee634640 1 --2- 172.21.15.22:0/2142548863 >> [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194] conn(0x7ffad00acb20 0x7ffad00aef40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.564+0000 7ffaee634640 1 -- 172.21.15.22:0/2142548863 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ffad00a32d0 msgr2=0x7ffad00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:50.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.564+0000 7ffaee634640 1 --2- 172.21.15.22:0/2142548863 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ffad00a32d0 0x7ffad00a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffad80058c0 tx=0x7ffad800a4c0 comp rx=0 tx=0).stop 2024-09-19T08:13:50.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.564+0000 7ffaee634640 1 -- 172.21.15.22:0/2142548863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffae8102790 msgr2=0x7ffae8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:50.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.564+0000 7ffaee634640 1 --2- 172.21.15.22:0/2142548863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffae8102790 0x7ffae8074a30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00b4a0 tx=0x7ffadc00b960 comp rx=0 tx=0).stop 2024-09-19T08:13:50.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.564+0000 7ffaecbd1640 1 -- 172.21.15.22:0/2142548863 reap_dead start 2024-09-19T08:13:50.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.564+0000 7ffaee634640 1 -- 172.21.15.22:0/2142548863 shutdown_connections 2024-09-19T08:13:50.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.564+0000 7ffaee634640 1 -- 172.21.15.22:0/2142548863 >> 172.21.15.22:0/2142548863 conn(0x7ffae80fd8c0 msgr2=0x7ffae80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:50.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.564+0000 7ffaee634640 1 -- 172.21.15.22:0/2142548863 shutdown_connections 2024-09-19T08:13:50.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:48.565+0000 7ffaee634640 1 -- 172.21.15.22:0/2142548863 wait complete. 2024-09-19T08:13:50.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:50 smithi064 ceph-mon[34900]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-19T08:13:50.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:50 smithi157 ceph-mon[35143]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-19T08:13:50.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:50 smithi022 ceph-mon[30963]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-19T08:13:50.890 INFO:teuthology.orchestra.run.smithi022.stdout:330712481814 2024-09-19T08:13:50.891 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T08:13:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.062+0000 7f2b451f3640 1 -- 172.21.15.22:0/232537005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b4010e500 msgr2=0x7f2b40110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.062+0000 7f2b451f3640 1 --2- 172.21.15.22:0/232537005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b4010e500 0x7f2b40110910 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009ca0 tx=0x7f2b3802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:13:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.062+0000 7f2b451f3640 1 -- 172.21.15.22:0/232537005 shutdown_connections 2024-09-19T08:13:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.062+0000 7f2b451f3640 1 --2- 172.21.15.22:0/232537005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b4010e500 0x7f2b40110910 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.062+0000 7f2b451f3640 1 --2- 172.21.15.22:0/232537005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b40101440 0x7f2b4010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.062+0000 7f2b451f3640 1 --2- 172.21.15.22:0/232537005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b40100b20 0x7f2b40100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.062+0000 7f2b451f3640 1 -- 172.21.15.22:0/232537005 >> 172.21.15.22:0/232537005 conn(0x7f2b400fc7d0 msgr2=0x7f2b400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.062+0000 7f2b451f3640 1 -- 172.21.15.22:0/232537005 shutdown_connections 2024-09-19T08:13:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.062+0000 7f2b451f3640 1 -- 172.21.15.22:0/232537005 wait complete. 2024-09-19T08:13:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.063+0000 7f2b451f3640 1 Processor -- start 2024-09-19T08:13:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.063+0000 7f2b451f3640 1 -- start start 2024-09-19T08:13:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b451f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b40100b20 0x7f2b4019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b451f3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b40101440 0x7f2b4019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b451f3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b4010e500 0x7f2b4019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b451f3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2b40114230 con 0x7f2b4010e500 2024-09-19T08:13:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b451f3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401140b0 con 0x7f2b40101440 2024-09-19T08:13:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b451f3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401143b0 con 0x7f2b40100b20 2024-09-19T08:13:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b3f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b4010e500 0x7f2b4019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b3f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b4010e500 0x7f2b4019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42250/0 (socket says 172.21.15.22:42250) 2024-09-19T08:13:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b3f59e640 1 -- 172.21.15.22:0/3832925793 learned_addr learned my addr 172.21.15.22:0/3832925793 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:51.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b3e59c640 1 --2- 172.21.15.22:0/3832925793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b40101440 0x7f2b4019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:51.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b3ed9d640 1 --2- 172.21.15.22:0/3832925793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b40100b20 0x7f2b4019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:51.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b3f59e640 1 -- 172.21.15.22:0/3832925793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b40100b20 msgr2=0x7f2b4019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b3f59e640 1 --2- 172.21.15.22:0/3832925793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b40100b20 0x7f2b4019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b3f59e640 1 -- 172.21.15.22:0/3832925793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b40101440 msgr2=0x7f2b4019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b3f59e640 1 --2- 172.21.15.22:0/3832925793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b40101440 0x7f2b4019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b3f59e640 1 -- 172.21.15.22:0/3832925793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4019f670 con 0x7f2b4010e500 2024-09-19T08:13:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b3f59e640 1 --2- 172.21.15.22:0/3832925793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b4010e500 0x7f2b4019ef90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2b38031f80 tx=0x7f2b38031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.064+0000 7f2b27fff640 1 -- 172.21.15.22:0/3832925793 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38003b70 con 0x7f2b4010e500 2024-09-19T08:13:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.065+0000 7f2b451f3640 1 -- 172.21.15.22:0/3832925793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4019f900 con 0x7f2b4010e500 2024-09-19T08:13:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.065+0000 7f2b27fff640 1 -- 172.21.15.22:0/3832925793 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b38003d10 con 0x7f2b4010e500 2024-09-19T08:13:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.065+0000 7f2b27fff640 1 -- 172.21.15.22:0/3832925793 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38004760 con 0x7f2b4010e500 2024-09-19T08:13:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.065+0000 7f2b451f3640 1 -- 172.21.15.22:0/3832925793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b401a7240 con 0x7f2b4010e500 2024-09-19T08:13:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.066+0000 7f2b451f3640 1 -- 172.21.15.22:0/3832925793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2b40062980 con 0x7f2b4010e500 2024-09-19T08:13:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.067+0000 7f2b27fff640 1 -- 172.21.15.22:0/3832925793 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2b3802f710 con 0x7f2b4010e500 2024-09-19T08:13:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.068+0000 7f2b27fff640 1 --2- 172.21.15.22:0/3832925793 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b280a32d0 0x7f2b280a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.068+0000 7f2b27fff640 1 -- 172.21.15.22:0/3832925793 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f2b380f0cb0 con 0x7f2b4010e500 2024-09-19T08:13:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.068+0000 7f2b3ed9d640 1 --2- 172.21.15.22:0/3832925793 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b280a32d0 0x7f2b280a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.068+0000 7f2b27fff640 1 --2- 172.21.15.22:0/3832925793 >> [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477] conn(0x7f2b280acb20 0x7f2b280aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.068+0000 7f2b27fff640 1 -- 172.21.15.22:0/3832925793 --> [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b280af5f0 con 0x7f2b280acb20 2024-09-19T08:13:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.068+0000 7f2b27fff640 1 -- 172.21.15.22:0/3832925793 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f2b380f1190 con 0x7f2b4010e500 2024-09-19T08:13:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.068+0000 7f2b3e59c640 1 --2- 172.21.15.22:0/3832925793 >> [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477] conn(0x7f2b280acb20 0x7f2b280aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.068+0000 7f2b3ed9d640 1 --2- 172.21.15.22:0/3832925793 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b280a32d0 0x7f2b280a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000abe0 tx=0x7f2b3000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:51.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.069+0000 7f2b3e59c640 1 --2- 172.21.15.22:0/3832925793 >> [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477] conn(0x7f2b280acb20 0x7f2b280aef40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:51.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.070+0000 7f2b27fff640 1 -- 172.21.15.22:0/3832925793 <== osd.8 v2:172.21.15.157:6816/973678477 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b280af5f0 con 0x7f2b280acb20 2024-09-19T08:13:51.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.095+0000 7f2b451f3640 1 -- 172.21.15.22:0/3832925793 --> [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2b40062e00 con 0x7f2b280acb20 2024-09-19T08:13:51.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.095+0000 7f2b27fff640 1 -- 172.21.15.22:0/3832925793 <== osd.8 v2:172.21.15.157:6816/973678477 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2b40062e00 con 0x7f2b280acb20 2024-09-19T08:13:51.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.096+0000 7f2b451f3640 1 -- 172.21.15.22:0/3832925793 >> [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477] conn(0x7f2b280acb20 msgr2=0x7f2b280aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:51.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.096+0000 7f2b451f3640 1 --2- 172.21.15.22:0/3832925793 >> [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477] conn(0x7f2b280acb20 0x7f2b280aef40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:51.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.096+0000 7f2b451f3640 1 -- 172.21.15.22:0/3832925793 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b280a32d0 msgr2=0x7f2b280a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:51.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.096+0000 7f2b451f3640 1 --2- 172.21.15.22:0/3832925793 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2b280a32d0 0x7f2b280a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000abe0 tx=0x7f2b3000a2d0 comp rx=0 tx=0).stop 2024-09-19T08:13:51.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.096+0000 7f2b451f3640 1 -- 172.21.15.22:0/3832925793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b4010e500 msgr2=0x7f2b4019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:51.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.096+0000 7f2b451f3640 1 --2- 172.21.15.22:0/3832925793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b4010e500 0x7f2b4019ef90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2b38031f80 tx=0x7f2b38031a60 comp rx=0 tx=0).stop 2024-09-19T08:13:51.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.096+0000 7f2b3f59e640 1 -- 172.21.15.22:0/3832925793 reap_dead start 2024-09-19T08:13:51.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.096+0000 7f2b451f3640 1 -- 172.21.15.22:0/3832925793 shutdown_connections 2024-09-19T08:13:51.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.096+0000 7f2b451f3640 1 -- 172.21.15.22:0/3832925793 >> 172.21.15.22:0/3832925793 conn(0x7f2b400fc7d0 msgr2=0x7f2b400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:51.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.096+0000 7f2b451f3640 1 -- 172.21.15.22:0/3832925793 shutdown_connections 2024-09-19T08:13:51.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.096+0000 7f2b451f3640 1 -- 172.21.15.22:0/3832925793 wait complete. 2024-09-19T08:13:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.398+0000 7fdae9645640 1 -- 172.21.15.22:0/1061475230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdae4102790 msgr2=0x7fdae41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.398+0000 7fdae9645640 1 --2- 172.21.15.22:0/1061475230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdae4102790 0x7fdae41067e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009900 tx=0x7fdad802a410 comp rx=0 tx=0).stop 2024-09-19T08:13:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.398+0000 7fdae9645640 1 -- 172.21.15.22:0/1061475230 shutdown_connections 2024-09-19T08:13:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.398+0000 7fdae9645640 1 --2- 172.21.15.22:0/1061475230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae4106f10 0x7fdae410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.398+0000 7fdae9645640 1 --2- 172.21.15.22:0/1061475230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdae4102790 0x7fdae41067e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.398+0000 7fdae9645640 1 --2- 172.21.15.22:0/1061475230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdae4101de0 0x7fdae41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.398+0000 7fdae9645640 1 -- 172.21.15.22:0/1061475230 >> 172.21.15.22:0/1061475230 conn(0x7fdae40fd8c0 msgr2=0x7fdae40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.398+0000 7fdae9645640 1 -- 172.21.15.22:0/1061475230 shutdown_connections 2024-09-19T08:13:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.398+0000 7fdae9645640 1 -- 172.21.15.22:0/1061475230 wait complete. 2024-09-19T08:13:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.399+0000 7fdae9645640 1 Processor -- start 2024-09-19T08:13:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.399+0000 7fdae9645640 1 -- start start 2024-09-19T08:13:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.399+0000 7fdae9645640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae4101de0 0x7fdae4198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.399+0000 7fdae9645640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdae4102790 0x7fdae4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.399+0000 7fdae9645640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdae4106f10 0x7fdae419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.399+0000 7fdae9645640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdae4111f70 con 0x7fdae4106f10 2024-09-19T08:13:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.399+0000 7fdae9645640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdae4111df0 con 0x7fdae4102790 2024-09-19T08:13:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.399+0000 7fdae9645640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdae41120f0 con 0x7fdae4101de0 2024-09-19T08:13:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.399+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdae4106f10 0x7fdae419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.399+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdae4106f10 0x7fdae419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42276/0 (socket says 172.21.15.22:42276) 2024-09-19T08:13:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.399+0000 7fdae37fe640 1 -- 172.21.15.22:0/3554056150 learned_addr learned my addr 172.21.15.22:0/3554056150 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.399+0000 7fdae27fc640 1 --2- 172.21.15.22:0/3554056150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdae4102790 0x7fdae4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.399+0000 7fdae2ffd640 1 --2- 172.21.15.22:0/3554056150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae4101de0 0x7fdae4198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.400+0000 7fdae37fe640 1 -- 172.21.15.22:0/3554056150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae4101de0 msgr2=0x7fdae4198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.400+0000 7fdae37fe640 1 --2- 172.21.15.22:0/3554056150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae4101de0 0x7fdae4198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.400+0000 7fdae37fe640 1 -- 172.21.15.22:0/3554056150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdae4102790 msgr2=0x7fdae4198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.400+0000 7fdae37fe640 1 --2- 172.21.15.22:0/3554056150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdae4102790 0x7fdae4198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.400+0000 7fdae37fe640 1 -- 172.21.15.22:0/3554056150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae419d400 con 0x7fdae4106f10 2024-09-19T08:13:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.400+0000 7fdae37fe640 1 --2- 172.21.15.22:0/3554056150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdae4106f10 0x7fdae419cd20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00eaa0 tx=0x7fdadc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.400+0000 7fdacbfff640 1 -- 172.21.15.22:0/3554056150 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc00cdb0 con 0x7fdae4106f10 2024-09-19T08:13:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.400+0000 7fdae9645640 1 -- 172.21.15.22:0/3554056150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae419d6f0 con 0x7fdae4106f10 2024-09-19T08:13:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.400+0000 7fdacbfff640 1 -- 172.21.15.22:0/3554056150 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdadc005020 con 0x7fdae4106f10 2024-09-19T08:13:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.400+0000 7fdacbfff640 1 -- 172.21.15.22:0/3554056150 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0116d0 con 0x7fdae4106f10 2024-09-19T08:13:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.400+0000 7fdae9645640 1 -- 172.21.15.22:0/3554056150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae41a4fd0 con 0x7fdae4106f10 2024-09-19T08:13:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.401+0000 7fdae9645640 1 -- 172.21.15.22:0/3554056150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdae4062980 con 0x7fdae4106f10 2024-09-19T08:13:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.402+0000 7fdacbfff640 1 -- 172.21.15.22:0/3554056150 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdadc0054d0 con 0x7fdae4106f10 2024-09-19T08:13:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.403+0000 7fdacbfff640 1 --2- 172.21.15.22:0/3554056150 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdacc0a32d0 0x7fdacc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.403+0000 7fdae2ffd640 1 --2- 172.21.15.22:0/3554056150 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdacc0a32d0 0x7fdacc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.403+0000 7fdacbfff640 1 -- 172.21.15.22:0/3554056150 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fdadc0cfa40 con 0x7fdae4106f10 2024-09-19T08:13:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.403+0000 7fdacbfff640 1 --2- 172.21.15.22:0/3554056150 >> [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752] conn(0x7fdacc0acb20 0x7fdacc0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.403+0000 7fdacbfff640 1 -- 172.21.15.22:0/3554056150 --> [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdadc018ec0 con 0x7fdacc0acb20 2024-09-19T08:13:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.403+0000 7fdacbfff640 1 -- 172.21.15.22:0/3554056150 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fdadc0d42c0 con 0x7fdae4106f10 2024-09-19T08:13:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.403+0000 7fdae2ffd640 1 --2- 172.21.15.22:0/3554056150 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdacc0a32d0 0x7fdacc0a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdad400bbe0 tx=0x7fdad400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.404+0000 7fdae27fc640 1 --2- 172.21.15.22:0/3554056150 >> [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752] conn(0x7fdacc0acb20 0x7fdacc0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.404+0000 7fdae27fc640 1 --2- 172.21.15.22:0/3554056150 >> [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752] conn(0x7fdacc0acb20 0x7fdacc0aef40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.405+0000 7fdacbfff640 1 -- 172.21.15.22:0/3554056150 <== osd.10 v2:172.21.15.64:6824/373412752 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdadc018ec0 con 0x7fdacc0acb20 2024-09-19T08:13:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.419+0000 7fdae9645640 1 -- 172.21.15.22:0/3554056150 --> [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdae4062e00 con 0x7fdacc0acb20 2024-09-19T08:13:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.419+0000 7fdacbfff640 1 -- 172.21.15.22:0/3554056150 <== osd.10 v2:172.21.15.64:6824/373412752 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdae4062e00 con 0x7fdacc0acb20 2024-09-19T08:13:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.420+0000 7fdae9645640 1 -- 172.21.15.22:0/3554056150 >> [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752] conn(0x7fdacc0acb20 msgr2=0x7fdacc0aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.420+0000 7fdae9645640 1 --2- 172.21.15.22:0/3554056150 >> [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752] conn(0x7fdacc0acb20 0x7fdacc0aef40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.420+0000 7fdae9645640 1 -- 172.21.15.22:0/3554056150 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdacc0a32d0 msgr2=0x7fdacc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.420+0000 7fdae9645640 1 --2- 172.21.15.22:0/3554056150 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdacc0a32d0 0x7fdacc0a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdad400bbe0 tx=0x7fdad400b390 comp rx=0 tx=0).stop 2024-09-19T08:13:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.420+0000 7fdae9645640 1 -- 172.21.15.22:0/3554056150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdae4106f10 msgr2=0x7fdae419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.420+0000 7fdae9645640 1 --2- 172.21.15.22:0/3554056150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdae4106f10 0x7fdae419cd20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00eaa0 tx=0x7fdadc00ef60 comp rx=0 tx=0).stop 2024-09-19T08:13:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.420+0000 7fdae37fe640 1 -- 172.21.15.22:0/3554056150 reap_dead start 2024-09-19T08:13:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.420+0000 7fdae9645640 1 -- 172.21.15.22:0/3554056150 shutdown_connections 2024-09-19T08:13:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.420+0000 7fdae9645640 1 -- 172.21.15.22:0/3554056150 >> 172.21.15.22:0/3554056150 conn(0x7fdae40fd8c0 msgr2=0x7fdae40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.420+0000 7fdae9645640 1 -- 172.21.15.22:0/3554056150 shutdown_connections 2024-09-19T08:13:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:51.420+0000 7fdae9645640 1 -- 172.21.15.22:0/3554056150 wait complete. 2024-09-19T08:13:52.329 INFO:teuthology.orchestra.run.smithi022.stdout:300647710748 2024-09-19T08:13:52.330 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T08:13:52.343 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:52.353 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:52.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:52 smithi064 ceph-mon[34900]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-19T08:13:52.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:52 smithi157 ceph-mon[35143]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-19T08:13:52.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:52 smithi022 ceph-mon[30963]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-19T08:13:52.872 INFO:teuthology.orchestra.run.smithi022.stdout:249108103207 2024-09-19T08:13:52.872 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T08:13:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.678+0000 7f7f215b1640 1 -- 172.21.15.22:0/3347285942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c102dd0 msgr2=0x7f7f1c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.678+0000 7f7f215b1640 1 --2- 172.21.15.22:0/3347285942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c102dd0 0x7f7f1c1031b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009930 tx=0x7f7f0c02a420 comp rx=0 tx=0).stop 2024-09-19T08:13:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.678+0000 7f7f215b1640 1 -- 172.21.15.22:0/3347285942 shutdown_connections 2024-09-19T08:13:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.678+0000 7f7f215b1640 1 --2- 172.21.15.22:0/3347285942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f1c108300 0x7f7f1c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.678+0000 7f7f215b1640 1 --2- 172.21.15.22:0/3347285942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f1c103780 0x7f7f1c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.678+0000 7f7f215b1640 1 --2- 172.21.15.22:0/3347285942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c102dd0 0x7f7f1c1031b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.678+0000 7f7f215b1640 1 -- 172.21.15.22:0/3347285942 >> 172.21.15.22:0/3347285942 conn(0x7f7f1c0fc7d0 msgr2=0x7f7f1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.678+0000 7f7f215b1640 1 -- 172.21.15.22:0/3347285942 shutdown_connections 2024-09-19T08:13:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.678+0000 7f7f215b1640 1 -- 172.21.15.22:0/3347285942 wait complete. 2024-09-19T08:13:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.679+0000 7f7f215b1640 1 Processor -- start 2024-09-19T08:13:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.679+0000 7f7f215b1640 1 -- start start 2024-09-19T08:13:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.679+0000 7f7f215b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c102dd0 0x7f7f1c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.679+0000 7f7f215b1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f1c103780 0x7f7f1c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.679+0000 7f7f215b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f1c108300 0x7f7f1c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.679+0000 7f7f215b1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c1120b0 con 0x7f7f1c102dd0 2024-09-19T08:13:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.679+0000 7f7f215b1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c111f30 con 0x7f7f1c108300 2024-09-19T08:13:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.679+0000 7f7f215b1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c112230 con 0x7f7f1c103780 2024-09-19T08:13:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f1affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c102dd0 0x7f7f1c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f1affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c102dd0 0x7f7f1c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42294/0 (socket says 172.21.15.22:42294) 2024-09-19T08:13:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f1affd640 1 -- 172.21.15.22:0/3042982910 learned_addr learned my addr 172.21.15.22:0/3042982910 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f1b7fe640 1 --2- 172.21.15.22:0/3042982910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f1c108300 0x7f7f1c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f1a7fc640 1 --2- 172.21.15.22:0/3042982910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f1c103780 0x7f7f1c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f1affd640 1 -- 172.21.15.22:0/3042982910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f1c103780 msgr2=0x7f7f1c198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f1affd640 1 --2- 172.21.15.22:0/3042982910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f1c103780 0x7f7f1c198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f1affd640 1 -- 172.21.15.22:0/3042982910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f1c108300 msgr2=0x7f7f1c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f1affd640 1 --2- 172.21.15.22:0/3042982910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f1c108300 0x7f7f1c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f1affd640 1 -- 172.21.15.22:0/3042982910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c19d5a0 con 0x7f7f1c102dd0 2024-09-19T08:13:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f1affd640 1 --2- 172.21.15.22:0/3042982910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c102dd0 0x7f7f1c1985f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009230 tx=0x7f7f0c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f03fff640 1 -- 172.21.15.22:0/3042982910 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c03f030 con 0x7f7f1c102dd0 2024-09-19T08:13:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f03fff640 1 -- 172.21.15.22:0/3042982910 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f0c003ad0 con 0x7f7f1c102dd0 2024-09-19T08:13:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f215b1640 1 -- 172.21.15.22:0/3042982910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1c19d830 con 0x7f7f1c102dd0 2024-09-19T08:13:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f03fff640 1 -- 172.21.15.22:0/3042982910 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c035070 con 0x7f7f1c102dd0 2024-09-19T08:13:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.680+0000 7f7f215b1640 1 -- 172.21.15.22:0/3042982910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1c1a5170 con 0x7f7f1c102dd0 2024-09-19T08:13:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.681+0000 7f7f215b1640 1 -- 172.21.15.22:0/3042982910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1c06a7b0 con 0x7f7f1c102dd0 2024-09-19T08:13:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.683+0000 7f7f03fff640 1 -- 172.21.15.22:0/3042982910 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7f0c042030 con 0x7f7f1c102dd0 2024-09-19T08:13:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.684+0000 7f7f03fff640 1 --2- 172.21.15.22:0/3042982910 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7f040a32d0 0x7f7f040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.684+0000 7f7f1a7fc640 1 --2- 172.21.15.22:0/3042982910 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7f040a32d0 0x7f7f040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.684+0000 7f7f03fff640 1 -- 172.21.15.22:0/3042982910 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f7f0c0f02b0 con 0x7f7f1c102dd0 2024-09-19T08:13:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.684+0000 7f7f1a7fc640 1 --2- 172.21.15.22:0/3042982910 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7f040a32d0 0x7f7f040a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0feef0 tx=0x7f7f1000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.687+0000 7f7f03fff640 1 -- 172.21.15.22:0/3042982910 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7f0c0bc8d0 con 0x7f7f1c102dd0 2024-09-19T08:13:53.757 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:53.758 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:53.758 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:53.759 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:53.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.891+0000 7f7f215b1640 1 -- 172.21.15.22:0/3042982910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f7f1c199980 con 0x7f7f1c102dd0 2024-09-19T08:13:53.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.891+0000 7f7f03fff640 1 -- 172.21.15.22:0/3042982910 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7f1c199980 con 0x7f7f1c102dd0 2024-09-19T08:13:53.892 INFO:teuthology.orchestra.run.smithi022.stdout:279172874274 2024-09-19T08:13:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.894+0000 7f7f215b1640 1 -- 172.21.15.22:0/3042982910 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7f040a32d0 msgr2=0x7f7f040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.894+0000 7f7f215b1640 1 --2- 172.21.15.22:0/3042982910 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7f040a32d0 0x7f7f040a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0feef0 tx=0x7f7f1000a250 comp rx=0 tx=0).stop 2024-09-19T08:13:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.894+0000 7f7f215b1640 1 -- 172.21.15.22:0/3042982910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c102dd0 msgr2=0x7f7f1c1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.894+0000 7f7f215b1640 1 --2- 172.21.15.22:0/3042982910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c102dd0 0x7f7f1c1985f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009230 tx=0x7f7f0c004270 comp rx=0 tx=0).stop 2024-09-19T08:13:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.894+0000 7f7f215b1640 1 -- 172.21.15.22:0/3042982910 shutdown_connections 2024-09-19T08:13:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.894+0000 7f7f215b1640 1 --2- 172.21.15.22:0/3042982910 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7f040a32d0 0x7f7f040a5790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.894+0000 7f7f215b1640 1 --2- 172.21.15.22:0/3042982910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f1c108300 0x7f7f1c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.894+0000 7f7f215b1640 1 --2- 172.21.15.22:0/3042982910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f1c103780 0x7f7f1c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.894+0000 7f7f215b1640 1 --2- 172.21.15.22:0/3042982910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c102dd0 0x7f7f1c1985f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.894+0000 7f7f215b1640 1 -- 172.21.15.22:0/3042982910 >> 172.21.15.22:0/3042982910 conn(0x7f7f1c0fc7d0 msgr2=0x7f7f1c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.894+0000 7f7f215b1640 1 -- 172.21.15.22:0/3042982910 shutdown_connections 2024-09-19T08:13:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.894+0000 7f7f215b1640 1 -- 172.21.15.22:0/3042982910 wait complete. 2024-09-19T08:13:54.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.991+0000 7f55815f8640 1 -- 172.21.15.22:0/3018717937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c10c180 msgr2=0x7f557c10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.991+0000 7f55815f8640 1 --2- 172.21.15.22:0/3018717937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c10c180 0x7f557c10e600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5574009ca0 tx=0x7f557402a950 comp rx=0 tx=0).stop 2024-09-19T08:13:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.991+0000 7f55815f8640 1 -- 172.21.15.22:0/3018717937 shutdown_connections 2024-09-19T08:13:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.991+0000 7f55815f8640 1 --2- 172.21.15.22:0/3018717937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c10c180 0x7f557c10e600 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.991+0000 7f55815f8640 1 --2- 172.21.15.22:0/3018717937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f557c1036c0 0x7f557c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.991+0000 7f55815f8640 1 --2- 172.21.15.22:0/3018717937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c102da0 0x7f557c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.991+0000 7f55815f8640 1 -- 172.21.15.22:0/3018717937 >> 172.21.15.22:0/3018717937 conn(0x7f557c0fc7a0 msgr2=0x7f557c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.991+0000 7f55815f8640 1 -- 172.21.15.22:0/3018717937 shutdown_connections 2024-09-19T08:13:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.991+0000 7f55815f8640 1 -- 172.21.15.22:0/3018717937 wait complete. 2024-09-19T08:13:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.991+0000 7f55815f8640 1 Processor -- start 2024-09-19T08:13:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f55815f8640 1 -- start start 2024-09-19T08:13:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f55815f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c102da0 0x7f557c100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f55815f8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f557c1036c0 0x7f557c100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f55815f8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c10c180 0x7f557c101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f55815f8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f557c111f20 con 0x7f557c10c180 2024-09-19T08:13:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f55815f8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f557c111da0 con 0x7f557c1036c0 2024-09-19T08:13:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f55815f8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f557c1120a0 con 0x7f557c102da0 2024-09-19T08:13:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f557b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c10c180 0x7f557c101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f557b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c10c180 0x7f557c101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42306/0 (socket says 172.21.15.22:42306) 2024-09-19T08:13:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f557b7fe640 1 -- 172.21.15.22:0/4062180879 learned_addr learned my addr 172.21.15.22:0/4062180879 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f557affd640 1 --2- 172.21.15.22:0/4062180879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c102da0 0x7f557c100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:54.262 INFO:teuthology.orchestra.run.smithi022.stdout:34359738447 2024-09-19T08:13:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f55727fc640 1 --2- 172.21.15.22:0/4062180879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f557c1036c0 0x7f557c100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f557b7fe640 1 -- 172.21.15.22:0/4062180879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c102da0 msgr2=0x7f557c100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f557b7fe640 1 --2- 172.21.15.22:0/4062180879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c102da0 0x7f557c100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f557b7fe640 1 -- 172.21.15.22:0/4062180879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f557c1036c0 msgr2=0x7f557c100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f557b7fe640 1 --2- 172.21.15.22:0/4062180879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f557c1036c0 0x7f557c100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.992+0000 7f557b7fe640 1 -- 172.21.15.22:0/4062180879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c1a67a0 con 0x7f557c10c180 2024-09-19T08:13:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.993+0000 7f557b7fe640 1 --2- 172.21.15.22:0/4062180879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c10c180 0x7f557c101480 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5574006010 tx=0x7f557402f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.993+0000 7f5578ff9640 1 -- 172.21.15.22:0/4062180879 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574034040 con 0x7f557c10c180 2024-09-19T08:13:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.993+0000 7f55815f8640 1 -- 172.21.15.22:0/4062180879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557c1a6a30 con 0x7f557c10c180 2024-09-19T08:13:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.993+0000 7f5578ff9640 1 -- 172.21.15.22:0/4062180879 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5574003e00 con 0x7f557c10c180 2024-09-19T08:13:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.993+0000 7f5578ff9640 1 -- 172.21.15.22:0/4062180879 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574045480 con 0x7f557c10c180 2024-09-19T08:13:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.993+0000 7f55815f8640 1 -- 172.21.15.22:0/4062180879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557c1a6f60 con 0x7f557c10c180 2024-09-19T08:13:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.994+0000 7f55815f8640 1 -- 172.21.15.22:0/4062180879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557c103180 con 0x7f557c10c180 2024-09-19T08:13:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.995+0000 7f5578ff9640 1 -- 172.21.15.22:0/4062180879 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5574037020 con 0x7f557c10c180 2024-09-19T08:13:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.996+0000 7f5578ff9640 1 --2- 172.21.15.22:0/4062180879 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f55680a32d0 0x7f55680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.996+0000 7f557affd640 1 --2- 172.21.15.22:0/4062180879 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f55680a32d0 0x7f55680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.996+0000 7f5578ff9640 1 -- 172.21.15.22:0/4062180879 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f55740f0990 con 0x7f557c10c180 2024-09-19T08:13:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:53.996+0000 7f557affd640 1 --2- 172.21.15.22:0/4062180879 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f55680a32d0 0x7f55680a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f556c00abe0 tx=0x7f556c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.000+0000 7f5578ff9640 1 -- 172.21.15.22:0/4062180879 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f55740bd710 con 0x7f557c10c180 2024-09-19T08:13:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.176+0000 7f55815f8640 1 -- 172.21.15.22:0/4062180879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f557c1a7190 con 0x7f557c10c180 2024-09-19T08:13:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.176+0000 7f5578ff9640 1 -- 172.21.15.22:0/4062180879 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f55740bd010 con 0x7f557c10c180 2024-09-19T08:13:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.178+0000 7f55815f8640 1 -- 172.21.15.22:0/4062180879 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f55680a32d0 msgr2=0x7f55680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.178+0000 7f55815f8640 1 --2- 172.21.15.22:0/4062180879 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f55680a32d0 0x7f55680a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f556c00abe0 tx=0x7f556c00a2d0 comp rx=0 tx=0).stop 2024-09-19T08:13:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.178+0000 7f55815f8640 1 -- 172.21.15.22:0/4062180879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c10c180 msgr2=0x7f557c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.178+0000 7f55815f8640 1 --2- 172.21.15.22:0/4062180879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c10c180 0x7f557c101480 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5574006010 tx=0x7f557402f710 comp rx=0 tx=0).stop 2024-09-19T08:13:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.178+0000 7f55815f8640 1 -- 172.21.15.22:0/4062180879 shutdown_connections 2024-09-19T08:13:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.178+0000 7f55815f8640 1 --2- 172.21.15.22:0/4062180879 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f55680a32d0 0x7f55680a5790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.178+0000 7f55815f8640 1 --2- 172.21.15.22:0/4062180879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c10c180 0x7f557c101480 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.178+0000 7f55815f8640 1 --2- 172.21.15.22:0/4062180879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f557c1036c0 0x7f557c100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:54.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.178+0000 7f55815f8640 1 --2- 172.21.15.22:0/4062180879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c102da0 0x7f557c100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:54.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.178+0000 7f55815f8640 1 -- 172.21.15.22:0/4062180879 >> 172.21.15.22:0/4062180879 conn(0x7f557c0fc7a0 msgr2=0x7f557c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:54.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.178+0000 7f55815f8640 1 -- 172.21.15.22:0/4062180879 shutdown_connections 2024-09-19T08:13:54.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:54.178+0000 7f55815f8640 1 -- 172.21.15.22:0/4062180879 wait complete. 2024-09-19T08:13:54.471 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:54.471 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:54.472 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:54.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:54 smithi022 ceph-mon[30963]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:13:54.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:54 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3042982910' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T08:13:54.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:54 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/4062180879' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T08:13:54.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:54 smithi157 ceph-mon[35143]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:13:54.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:54 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3042982910' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T08:13:54.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:54 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/4062180879' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T08:13:54.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:54 smithi064 ceph-mon[34900]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:13:54.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:54 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3042982910' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T08:13:54.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:54 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/4062180879' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T08:13:54.910 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874272 got 279172874274 for osd.9 2024-09-19T08:13:54.911 DEBUG:teuthology.parallel:result is None 2024-09-19T08:13:55.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.804+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1514467417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49c01037a0 msgr2=0x7f49c010bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:55.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.804+0000 7f49c4cd1640 1 --2- 172.21.15.22:0/1514467417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49c01037a0 0x7f49c010bc00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009900 tx=0x7f49b402a410 comp rx=0 tx=0).stop 2024-09-19T08:13:55.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.804+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1514467417 shutdown_connections 2024-09-19T08:13:55.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.804+0000 7f49c4cd1640 1 --2- 172.21.15.22:0/1514467417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c010c2f0 0x7f49c010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:55.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.804+0000 7f49c4cd1640 1 --2- 172.21.15.22:0/1514467417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49c01037a0 0x7f49c010bc00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:55.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.804+0000 7f49c4cd1640 1 --2- 172.21.15.22:0/1514467417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49c0102df0 0x7f49c01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:55.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.804+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1514467417 >> 172.21.15.22:0/1514467417 conn(0x7f49c00fc7d0 msgr2=0x7f49c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:55.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.805+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1514467417 shutdown_connections 2024-09-19T08:13:55.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.805+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1514467417 wait complete. 2024-09-19T08:13:55.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.805+0000 7f49c4cd1640 1 Processor -- start 2024-09-19T08:13:55.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.805+0000 7f49c4cd1640 1 -- start start 2024-09-19T08:13:55.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49c4cd1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c0102df0 0x7f49c01a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:55.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49c4cd1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49c01037a0 0x7f49c01a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:55.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49c4cd1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49c010c2f0 0x7f49c019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:55.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49c4cd1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49c01120b0 con 0x7f49c010c2f0 2024-09-19T08:13:55.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49c4cd1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0111f30 con 0x7f49c01037a0 2024-09-19T08:13:55.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49c4cd1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0112230 con 0x7f49c0102df0 2024-09-19T08:13:55.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49bed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49c010c2f0 0x7f49c019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:55.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49bed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49c010c2f0 0x7f49c019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42344/0 (socket says 172.21.15.22:42344) 2024-09-19T08:13:55.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49bed9d640 1 -- 172.21.15.22:0/1617689887 learned_addr learned my addr 172.21.15.22:0/1617689887 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49bdd9b640 1 --2- 172.21.15.22:0/1617689887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49c01037a0 0x7f49c01a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49be59c640 1 --2- 172.21.15.22:0/1617689887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c0102df0 0x7f49c01a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49bed9d640 1 -- 172.21.15.22:0/1617689887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c0102df0 msgr2=0x7f49c01a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49bed9d640 1 --2- 172.21.15.22:0/1617689887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c0102df0 0x7f49c01a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49bed9d640 1 -- 172.21.15.22:0/1617689887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49c01037a0 msgr2=0x7f49c01a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49bed9d640 1 --2- 172.21.15.22:0/1617689887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49c01037a0 0x7f49c01a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49bed9d640 1 -- 172.21.15.22:0/1617689887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c019ad10 con 0x7f49c010c2f0 2024-09-19T08:13:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49bed9d640 1 --2- 172.21.15.22:0/1617689887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49c010c2f0 0x7f49c019a5e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f49b800cce0 tx=0x7f49b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.806+0000 7f49af7fe640 1 -- 172.21.15.22:0/1617689887 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b800f450 con 0x7f49c010c2f0 2024-09-19T08:13:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.807+0000 7f49af7fe640 1 -- 172.21.15.22:0/1617689887 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b800fad0 con 0x7f49c010c2f0 2024-09-19T08:13:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.807+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1617689887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c019b000 con 0x7f49c010c2f0 2024-09-19T08:13:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.807+0000 7f49af7fe640 1 -- 172.21.15.22:0/1617689887 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8018c10 con 0x7f49c010c2f0 2024-09-19T08:13:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.807+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1617689887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c01a72f0 con 0x7f49c010c2f0 2024-09-19T08:13:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.808+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1617689887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c01031d0 con 0x7f49c010c2f0 2024-09-19T08:13:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.809+0000 7f49af7fe640 1 -- 172.21.15.22:0/1617689887 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f49b800f5f0 con 0x7f49c010c2f0 2024-09-19T08:13:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.809+0000 7f49af7fe640 1 --2- 172.21.15.22:0/1617689887 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f49a80a3350 0x7f49a80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.809+0000 7f49be59c640 1 --2- 172.21.15.22:0/1617689887 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f49a80a3350 0x7f49a80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.810+0000 7f49af7fe640 1 -- 172.21.15.22:0/1617689887 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f49b80cfaf0 con 0x7f49c010c2f0 2024-09-19T08:13:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.810+0000 7f49be59c640 1 --2- 172.21.15.22:0/1617689887 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f49a80a3350 0x7f49a80a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009bd0 tx=0x7f49b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:55.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.814+0000 7f49af7fe640 1 -- 172.21.15.22:0/1617689887 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f49b8014030 con 0x7f49c010c2f0 2024-09-19T08:13:56.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:56.004+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1617689887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f49c01a7520 con 0x7f49c010c2f0 2024-09-19T08:13:56.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:56.004+0000 7f49af7fe640 1 -- 172.21.15.22:0/1617689887 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f49b809c860 con 0x7f49c010c2f0 2024-09-19T08:13:56.005 INFO:teuthology.orchestra.run.smithi022.stdout:197568495664 2024-09-19T08:13:56.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:56.006+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1617689887 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f49a80a3350 msgr2=0x7f49a80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:56.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:56.006+0000 7f49c4cd1640 1 --2- 172.21.15.22:0/1617689887 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f49a80a3350 0x7f49a80a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009bd0 tx=0x7f49b0005f90 comp rx=0 tx=0).stop 2024-09-19T08:13:56.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:56.006+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1617689887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49c010c2f0 msgr2=0x7f49c019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:56.006+0000 7f49c4cd1640 1 --2- 172.21.15.22:0/1617689887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49c010c2f0 0x7f49c019a5e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f49b800cce0 tx=0x7f49b800a620 comp rx=0 tx=0).stop 2024-09-19T08:13:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:56.006+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1617689887 shutdown_connections 2024-09-19T08:13:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:56.006+0000 7f49c4cd1640 1 --2- 172.21.15.22:0/1617689887 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f49a80a3350 0x7f49a80a5810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:56.006+0000 7f49c4cd1640 1 --2- 172.21.15.22:0/1617689887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49c010c2f0 0x7f49c019a5e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:56.006+0000 7f49c4cd1640 1 --2- 172.21.15.22:0/1617689887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49c01037a0 0x7f49c01a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:56.006+0000 7f49c4cd1640 1 --2- 172.21.15.22:0/1617689887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c0102df0 0x7f49c01a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:56.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:56.006+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1617689887 >> 172.21.15.22:0/1617689887 conn(0x7f49c00fc7d0 msgr2=0x7f49c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:56.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:56.006+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1617689887 shutdown_connections 2024-09-19T08:13:56.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:56.006+0000 7f49c4cd1640 1 -- 172.21.15.22:0/1617689887 wait complete. 2024-09-19T08:13:56.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:56 smithi022 ceph-mon[30963]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:13:56.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:56 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/4188149586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T08:13:56.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:56 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1617689887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T08:13:56.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:56 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:13:56.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:56 smithi157 ceph-mon[35143]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:13:56.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:56 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/4188149586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T08:13:56.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:56 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1617689887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T08:13:56.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:56 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:13:56.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:56 smithi064 ceph-mon[34900]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:13:56.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:56 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/4188149586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T08:13:56.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:56 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1617689887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T08:13:56.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:56 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:13:56.864 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495662 got 197568495664 for osd.6 2024-09-19T08:13:56.864 DEBUG:teuthology.parallel:result is None 2024-09-19T08:13:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.712+0000 7f7912851640 1 -- 172.21.15.22:0/3492685898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f790c1082a0 msgr2=0x7f790c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.712+0000 7f7912851640 1 --2- 172.21.15.22:0/3492685898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f790c1082a0 0x7f790c10ec40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7904009cd0 tx=0x7f790402a910 comp rx=0 tx=0).stop 2024-09-19T08:13:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.712+0000 7f7912851640 1 -- 172.21.15.22:0/3492685898 shutdown_connections 2024-09-19T08:13:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.712+0000 7f7912851640 1 --2- 172.21.15.22:0/3492685898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f790c1082a0 0x7f790c10ec40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.712+0000 7f7912851640 1 --2- 172.21.15.22:0/3492685898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c1036e0 0x7f790c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.712+0000 7f7912851640 1 --2- 172.21.15.22:0/3492685898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c102d30 0x7f790c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.712+0000 7f7912851640 1 -- 172.21.15.22:0/3492685898 >> 172.21.15.22:0/3492685898 conn(0x7f790c0fc7d0 msgr2=0x7f790c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.712+0000 7f7912851640 1 -- 172.21.15.22:0/3492685898 shutdown_connections 2024-09-19T08:13:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.712+0000 7f7912851640 1 -- 172.21.15.22:0/3492685898 wait complete. 2024-09-19T08:13:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7912851640 1 Processor -- start 2024-09-19T08:13:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7912851640 1 -- start start 2024-09-19T08:13:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7912851640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c102d30 0x7f790c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7912851640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f790c1036e0 0x7f790c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7912851640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c1082a0 0x7f790c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7912851640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1120e0 con 0x7f790c1036e0 2024-09-19T08:13:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7912851640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f790c111f60 con 0x7f790c1082a0 2024-09-19T08:13:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7912851640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f790c112260 con 0x7f790c102d30 2024-09-19T08:13:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7903fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f790c1036e0 0x7f790c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7903fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f790c1036e0 0x7f790c198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42326/0 (socket says 172.21.15.22:42326) 2024-09-19T08:13:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7903fff640 1 -- 172.21.15.22:0/4188149586 learned_addr learned my addr 172.21.15.22:0/4188149586 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7910dee640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c1082a0 0x7f790c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f790bfff640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c102d30 0x7f790c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7903fff640 1 -- 172.21.15.22:0/4188149586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c102d30 msgr2=0x7f790c198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7903fff640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c102d30 0x7f790c198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7903fff640 1 -- 172.21.15.22:0/4188149586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c1082a0 msgr2=0x7f790c19cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7903fff640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c1082a0 0x7f790c19cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f7903fff640 1 -- 172.21.15.22:0/4188149586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790c19d5d0 con 0x7f790c1036e0 2024-09-19T08:13:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.713+0000 7f790bfff640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c102d30 0x7f790c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:13:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.714+0000 7f7903fff640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f790c1036e0 0x7f790c198b60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f78f4006fd0 tx=0x7f78f400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.714+0000 7f7909ffb640 1 -- 172.21.15.22:0/4188149586 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f4011830 con 0x7f790c1036e0 2024-09-19T08:13:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.714+0000 7f7909ffb640 1 -- 172.21.15.22:0/4188149586 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78f4016070 con 0x7f790c1036e0 2024-09-19T08:13:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.714+0000 7f7912851640 1 -- 172.21.15.22:0/4188149586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790c19d8c0 con 0x7f790c1036e0 2024-09-19T08:13:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.714+0000 7f7909ffb640 1 -- 172.21.15.22:0/4188149586 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f400f2e0 con 0x7f790c1036e0 2024-09-19T08:13:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.714+0000 7f7912851640 1 -- 172.21.15.22:0/4188149586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790c1a51a0 con 0x7f790c1036e0 2024-09-19T08:13:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.715+0000 7f7912851640 1 -- 172.21.15.22:0/4188149586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f790c103110 con 0x7f790c1036e0 2024-09-19T08:13:56.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.716+0000 7f7909ffb640 1 -- 172.21.15.22:0/4188149586 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f78f401b460 con 0x7f790c1036e0 2024-09-19T08:13:56.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.717+0000 7f7909ffb640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f78f80a32d0 0x7f78f80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:56.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.717+0000 7f790bfff640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f78f80a32d0 0x7f78f80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:56.968 INFO:teuthology.orchestra.run.smithi022.stdout:219043332139 2024-09-19T08:13:56.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.717+0000 7f7909ffb640 1 -- 172.21.15.22:0/4188149586 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f78f40ceea0 con 0x7f790c1036e0 2024-09-19T08:13:56.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.717+0000 7f790bfff640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f78f80a32d0 0x7f78f80a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0058c0 tx=0x7f78fc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:56.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.721+0000 7f7909ffb640 1 -- 172.21.15.22:0/4188149586 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f78f409bd20 con 0x7f790c1036e0 2024-09-19T08:13:56.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.915+0000 7f7912851640 1 -- 172.21.15.22:0/4188149586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f790c10a2b0 con 0x7f790c1036e0 2024-09-19T08:13:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.915+0000 7f7909ffb640 1 -- 172.21.15.22:0/4188149586 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f78f409b620 con 0x7f790c1036e0 2024-09-19T08:13:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.917+0000 7f7912851640 1 -- 172.21.15.22:0/4188149586 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f78f80a32d0 msgr2=0x7f78f80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.917+0000 7f7912851640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f78f80a32d0 0x7f78f80a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0058c0 tx=0x7f78fc005850 comp rx=0 tx=0).stop 2024-09-19T08:13:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.917+0000 7f7912851640 1 -- 172.21.15.22:0/4188149586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f790c1036e0 msgr2=0x7f790c198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.917+0000 7f7912851640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f790c1036e0 0x7f790c198b60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f78f4006fd0 tx=0x7f78f400b8e0 comp rx=0 tx=0).stop 2024-09-19T08:13:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.917+0000 7f7912851640 1 -- 172.21.15.22:0/4188149586 shutdown_connections 2024-09-19T08:13:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.917+0000 7f7912851640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f78f80a32d0 0x7f78f80a5790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.917+0000 7f7912851640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f790c1082a0 0x7f790c19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.917+0000 7f7912851640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f790c1036e0 0x7f790c198b60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.917+0000 7f7912851640 1 --2- 172.21.15.22:0/4188149586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f790c102d30 0x7f790c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.917+0000 7f7912851640 1 -- 172.21.15.22:0/4188149586 >> 172.21.15.22:0/4188149586 conn(0x7f790c0fc7d0 msgr2=0x7f790c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.917+0000 7f7912851640 1 -- 172.21.15.22:0/4188149586 shutdown_connections 2024-09-19T08:13:56.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:55.917+0000 7f7912851640 1 -- 172.21.15.22:0/4188149586 wait complete. 2024-09-19T08:13:57.622 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332137 got 219043332139 for osd.7 2024-09-19T08:13:57.623 DEBUG:teuthology.parallel:result is None 2024-09-19T08:13:58.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.025+0000 7fa12e09b640 1 -- 172.21.15.22:0/342797675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1281027a0 msgr2=0x7fa1281067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.025+0000 7fa12e09b640 1 --2- 172.21.15.22:0/342797675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1281027a0 0x7fa1281067f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009900 tx=0x7fa11c02a410 comp rx=0 tx=0).stop 2024-09-19T08:13:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.025+0000 7fa12e09b640 1 -- 172.21.15.22:0/342797675 shutdown_connections 2024-09-19T08:13:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.025+0000 7fa12e09b640 1 --2- 172.21.15.22:0/342797675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa128106f20 0x7fa12810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.025+0000 7fa12e09b640 1 --2- 172.21.15.22:0/342797675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1281027a0 0x7fa1281067f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.025+0000 7fa12e09b640 1 --2- 172.21.15.22:0/342797675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa128101df0 0x7fa1281021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.025+0000 7fa12e09b640 1 -- 172.21.15.22:0/342797675 >> 172.21.15.22:0/342797675 conn(0x7fa1280fd8f0 msgr2=0x7fa1280ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.025+0000 7fa12e09b640 1 -- 172.21.15.22:0/342797675 shutdown_connections 2024-09-19T08:13:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.025+0000 7fa12e09b640 1 -- 172.21.15.22:0/342797675 wait complete. 2024-09-19T08:13:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.025+0000 7fa12e09b640 1 Processor -- start 2024-09-19T08:13:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa12e09b640 1 -- start start 2024-09-19T08:13:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa12e09b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa128101df0 0x7fa128198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa12e09b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1281027a0 0x7fa128198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa12e09b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa128106f20 0x7fa12819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa12e09b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa128111f80 con 0x7fa1281027a0 2024-09-19T08:13:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa12e09b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa128111e00 con 0x7fa128106f20 2024-09-19T08:13:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa12e09b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa128112100 con 0x7fa128101df0 2024-09-19T08:13:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa126ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1281027a0 0x7fa128198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa126ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1281027a0 0x7fa128198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35874/0 (socket says 172.21.15.22:35874) 2024-09-19T08:13:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa126ffd640 1 -- 172.21.15.22:0/3956359167 learned_addr learned my addr 172.21.15.22:0/3956359167 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa127fff640 1 --2- 172.21.15.22:0/3956359167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa128106f20 0x7fa12819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa1277fe640 1 --2- 172.21.15.22:0/3956359167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa128101df0 0x7fa128198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa126ffd640 1 -- 172.21.15.22:0/3956359167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa128101df0 msgr2=0x7fa128198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa126ffd640 1 --2- 172.21.15.22:0/3956359167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa128101df0 0x7fa128198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa126ffd640 1 -- 172.21.15.22:0/3956359167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa128106f20 msgr2=0x7fa12819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:58.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa126ffd640 1 --2- 172.21.15.22:0/3956359167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa128106f20 0x7fa12819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.026+0000 7fa126ffd640 1 -- 172.21.15.22:0/3956359167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12819d400 con 0x7fa1281027a0 2024-09-19T08:13:58.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.027+0000 7fa126ffd640 1 --2- 172.21.15.22:0/3956359167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1281027a0 0x7fa128198990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006010 tx=0x7fa11c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:58.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.027+0000 7fa124ff9640 1 -- 172.21.15.22:0/3956359167 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c03f030 con 0x7fa1281027a0 2024-09-19T08:13:58.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.027+0000 7fa12e09b640 1 -- 172.21.15.22:0/3956359167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12819d690 con 0x7fa1281027a0 2024-09-19T08:13:58.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.027+0000 7fa124ff9640 1 -- 172.21.15.22:0/3956359167 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa11c003ce0 con 0x7fa1281027a0 2024-09-19T08:13:58.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.027+0000 7fa124ff9640 1 -- 172.21.15.22:0/3956359167 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c035070 con 0x7fa1281027a0 2024-09-19T08:13:58.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.027+0000 7fa12e09b640 1 -- 172.21.15.22:0/3956359167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa128074420 con 0x7fa1281027a0 2024-09-19T08:13:58.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.028+0000 7fa12e09b640 1 -- 172.21.15.22:0/3956359167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1281032d0 con 0x7fa1281027a0 2024-09-19T08:13:58.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.029+0000 7fa124ff9640 1 -- 172.21.15.22:0/3956359167 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa11c042030 con 0x7fa1281027a0 2024-09-19T08:13:58.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.030+0000 7fa124ff9640 1 --2- 172.21.15.22:0/3956359167 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa1100a32d0 0x7fa1100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:58.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.030+0000 7fa124ff9640 1 -- 172.21.15.22:0/3956359167 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fa11c0efe20 con 0x7fa1281027a0 2024-09-19T08:13:58.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.030+0000 7fa1277fe640 1 --2- 172.21.15.22:0/3956359167 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa1100a32d0 0x7fa1100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:58.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.031+0000 7fa1277fe640 1 --2- 172.21.15.22:0/3956359167 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa1100a32d0 0x7fa1100a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa118009bd0 tx=0x7fa118005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:58.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.034+0000 7fa124ff9640 1 -- 172.21.15.22:0/3956359167 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa11c0bc600 con 0x7fa1281027a0 2024-09-19T08:13:58.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.219+0000 7fa12e09b640 1 -- 172.21.15.22:0/3956359167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fa12810a160 con 0x7fa1281027a0 2024-09-19T08:13:58.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.220+0000 7fa124ff9640 1 -- 172.21.15.22:0/3956359167 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa11c0c0900 con 0x7fa1281027a0 2024-09-19T08:13:58.221 INFO:teuthology.orchestra.run.smithi022.stdout:141733920828 2024-09-19T08:13:58.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.222+0000 7fa12e09b640 1 -- 172.21.15.22:0/3956359167 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa1100a32d0 msgr2=0x7fa1100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:58.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.222+0000 7fa12e09b640 1 --2- 172.21.15.22:0/3956359167 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa1100a32d0 0x7fa1100a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa118009bd0 tx=0x7fa118005f90 comp rx=0 tx=0).stop 2024-09-19T08:13:58.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.222+0000 7fa12e09b640 1 -- 172.21.15.22:0/3956359167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1281027a0 msgr2=0x7fa128198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:58.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.222+0000 7fa12e09b640 1 --2- 172.21.15.22:0/3956359167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1281027a0 0x7fa128198990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006010 tx=0x7fa11c0042f0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.222+0000 7fa12e09b640 1 -- 172.21.15.22:0/3956359167 shutdown_connections 2024-09-19T08:13:58.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.222+0000 7fa12e09b640 1 --2- 172.21.15.22:0/3956359167 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fa1100a32d0 0x7fa1100a5790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.222+0000 7fa12e09b640 1 --2- 172.21.15.22:0/3956359167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa128106f20 0x7fa12819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.222+0000 7fa12e09b640 1 --2- 172.21.15.22:0/3956359167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1281027a0 0x7fa128198990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.222+0000 7fa12e09b640 1 --2- 172.21.15.22:0/3956359167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa128101df0 0x7fa128198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.222+0000 7fa12e09b640 1 -- 172.21.15.22:0/3956359167 >> 172.21.15.22:0/3956359167 conn(0x7fa1280fd8f0 msgr2=0x7fa1280fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:58.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.223+0000 7fa12e09b640 1 -- 172.21.15.22:0/3956359167 shutdown_connections 2024-09-19T08:13:58.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.223+0000 7fa12e09b640 1 -- 172.21.15.22:0/3956359167 wait complete. 2024-09-19T08:13:58.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.289+0000 7f7504789640 1 -- 172.21.15.22:0/2855460467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74fc102d30 msgr2=0x7f74fc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:58.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.289+0000 7f7504789640 1 --2- 172.21.15.22:0/2855460467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74fc102d30 0x7f74fc103110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009930 tx=0x7f74ec02a420 comp rx=0 tx=0).stop 2024-09-19T08:13:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.289+0000 7f7504789640 1 -- 172.21.15.22:0/2855460467 shutdown_connections 2024-09-19T08:13:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.289+0000 7f7504789640 1 --2- 172.21.15.22:0/2855460467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74fc1082a0 0x7f74fc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.289+0000 7f7504789640 1 --2- 172.21.15.22:0/2855460467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74fc1036e0 0x7f74fc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.289+0000 7f7504789640 1 --2- 172.21.15.22:0/2855460467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74fc102d30 0x7f74fc103110 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.289+0000 7f7504789640 1 -- 172.21.15.22:0/2855460467 >> 172.21.15.22:0/2855460467 conn(0x7f74fc0fc7d0 msgr2=0x7f74fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.289+0000 7f7504789640 1 -- 172.21.15.22:0/2855460467 shutdown_connections 2024-09-19T08:13:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.290+0000 7f7504789640 1 -- 172.21.15.22:0/2855460467 wait complete. 2024-09-19T08:13:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.290+0000 7f7504789640 1 Processor -- start 2024-09-19T08:13:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.290+0000 7f7504789640 1 -- start start 2024-09-19T08:13:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.290+0000 7f7504789640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74fc102d30 0x7f74fc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.290+0000 7f7504789640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74fc1036e0 0x7f74fc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:58.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.290+0000 7f7504789640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74fc1082a0 0x7f74fc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.290+0000 7f7504789640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc1120a0 con 0x7f74fc1036e0 2024-09-19T08:13:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.290+0000 7f7504789640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc111f20 con 0x7f74fc102d30 2024-09-19T08:13:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.290+0000 7f7504789640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc112220 con 0x7f74fc1082a0 2024-09-19T08:13:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f7501d24640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74fc1036e0 0x7f74fc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f7501d24640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74fc1036e0 0x7f74fc100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35888/0 (socket says 172.21.15.22:35888) 2024-09-19T08:13:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f7501d24640 1 -- 172.21.15.22:0/3332344945 learned_addr learned my addr 172.21.15.22:0/3332344945 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f7502525640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74fc102d30 0x7f74fc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f7502d26640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74fc1082a0 0x7f74fc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f7501d24640 1 -- 172.21.15.22:0/3332344945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74fc1082a0 msgr2=0x7f74fc101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f7501d24640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74fc1082a0 0x7f74fc101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f7501d24640 1 -- 172.21.15.22:0/3332344945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74fc102d30 msgr2=0x7f74fc100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:58.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f7501d24640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74fc102d30 0x7f74fc100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f7501d24640 1 -- 172.21.15.22:0/3332344945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74fc1a6a90 con 0x7f74fc1036e0 2024-09-19T08:13:58.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f7502d26640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74fc1082a0 0x7f74fc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:13:58.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f7501d24640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74fc1036e0 0x7f74fc100ed0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f74f800d950 tx=0x7f74f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:58.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f74f37fe640 1 -- 172.21.15.22:0/3332344945 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8017070 con 0x7f74fc1036e0 2024-09-19T08:13:58.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f7504789640 1 -- 172.21.15.22:0/3332344945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74fc1a6d80 con 0x7f74fc1036e0 2024-09-19T08:13:58.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f74f37fe640 1 -- 172.21.15.22:0/3332344945 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74f8009c30 con 0x7f74fc1036e0 2024-09-19T08:13:58.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f74f37fe640 1 -- 172.21.15.22:0/3332344945 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f80047a0 con 0x7f74fc1036e0 2024-09-19T08:13:58.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.291+0000 7f7504789640 1 -- 172.21.15.22:0/3332344945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74fc1a7240 con 0x7f74fc1036e0 2024-09-19T08:13:58.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.292+0000 7f7504789640 1 -- 172.21.15.22:0/3332344945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74fc06a850 con 0x7f74fc1036e0 2024-09-19T08:13:58.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.294+0000 7f74f37fe640 1 -- 172.21.15.22:0/3332344945 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f74f8004020 con 0x7f74fc1036e0 2024-09-19T08:13:58.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.294+0000 7f74f37fe640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f74e80a32d0 0x7f74e80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:58.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.294+0000 7f7502525640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f74e80a32d0 0x7f74e80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:58.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.294+0000 7f74f37fe640 1 -- 172.21.15.22:0/3332344945 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f74f80cf7d0 con 0x7f74fc1036e0 2024-09-19T08:13:58.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.295+0000 7f7502525640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f74e80a32d0 0x7f74e80a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009900 tx=0x7f74ec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:58.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.298+0000 7f74f37fe640 1 -- 172.21.15.22:0/3332344945 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f74f8020030 con 0x7f74fc1036e0 2024-09-19T08:13:58.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.477+0000 7f7504789640 1 -- 172.21.15.22:0/3332344945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f74fc103110 con 0x7f74fc1036e0 2024-09-19T08:13:58.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.477+0000 7f74f37fe640 1 -- 172.21.15.22:0/3332344945 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f74fc103110 con 0x7f74fc1036e0 2024-09-19T08:13:58.478 INFO:teuthology.orchestra.run.smithi022.stdout:330712481816 2024-09-19T08:13:58.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.480+0000 7f7504789640 1 -- 172.21.15.22:0/3332344945 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f74e80a32d0 msgr2=0x7f74e80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:58.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.480+0000 7f7504789640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f74e80a32d0 0x7f74e80a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009900 tx=0x7f74ec0058d0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.480+0000 7f7504789640 1 -- 172.21.15.22:0/3332344945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74fc1036e0 msgr2=0x7f74fc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:58.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.480+0000 7f7504789640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74fc1036e0 0x7f74fc100ed0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f74f800d950 tx=0x7f74f800de10 comp rx=0 tx=0).stop 2024-09-19T08:13:58.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.481+0000 7f7504789640 1 -- 172.21.15.22:0/3332344945 shutdown_connections 2024-09-19T08:13:58.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.481+0000 7f7504789640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f74e80a32d0 0x7f74e80a5790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.481+0000 7f7504789640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74fc1082a0 0x7f74fc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.481+0000 7f7504789640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74fc1036e0 0x7f74fc100ed0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.481+0000 7f7504789640 1 --2- 172.21.15.22:0/3332344945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74fc102d30 0x7f74fc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:58.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.481+0000 7f7504789640 1 -- 172.21.15.22:0/3332344945 >> 172.21.15.22:0/3332344945 conn(0x7f74fc0fc7d0 msgr2=0x7f74fc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:58.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.481+0000 7f7504789640 1 -- 172.21.15.22:0/3332344945 shutdown_connections 2024-09-19T08:13:58.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.481+0000 7f7504789640 1 -- 172.21.15.22:0/3332344945 wait complete. 2024-09-19T08:13:58.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:58 smithi022 ceph-mon[30963]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-19T08:13:58.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:58 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:13:58.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:58 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:13:58.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:58 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:13:58.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:58 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:13:58.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:58 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3956359167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T08:13:58.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:58 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3332344945' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T08:13:59.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:58 smithi157 ceph-mon[35143]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-19T08:13:59.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:58 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:13:59.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:58 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:13:59.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:58 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:13:59.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:58 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:13:59.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:58 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3956359167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T08:13:59.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:58 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3332344945' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T08:13:59.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:58 smithi064 ceph-mon[34900]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-19T08:13:59.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:58 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:13:59.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:58 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:13:59.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:58 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:13:59.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:58 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:13:59.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:58 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3956359167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T08:13:59.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:58 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3332344945' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T08:13:59.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.966+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/1800041934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fc4100e20 msgr2=0x7f6fc4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:59.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.966+0000 7f6fcb8d7640 1 --2- 172.21.15.22:0/1800041934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fc4100e20 0x7f6fc4101200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009900 tx=0x7f6fc002a480 comp rx=0 tx=0).stop 2024-09-19T08:13:59.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.966+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/1800041934 shutdown_connections 2024-09-19T08:13:59.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.966+0000 7f6fcb8d7640 1 --2- 172.21.15.22:0/1800041934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc4102ef0 0x7f6fc410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:59.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.966+0000 7f6fcb8d7640 1 --2- 172.21.15.22:0/1800041934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc4102570 0x7f6fc41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:59.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.966+0000 7f6fcb8d7640 1 --2- 172.21.15.22:0/1800041934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fc4100e20 0x7f6fc4101200 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:59.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.966+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/1800041934 >> 172.21.15.22:0/1800041934 conn(0x7f6fc40fc7d0 msgr2=0x7f6fc40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:59.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.966+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/1800041934 shutdown_connections 2024-09-19T08:13:59.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.966+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/1800041934 wait complete. 2024-09-19T08:13:59.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.967+0000 7f6fcb8d7640 1 Processor -- start 2024-09-19T08:13:59.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.967+0000 7f6fcb8d7640 1 -- start start 2024-09-19T08:13:59.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.967+0000 7f6fcb8d7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc4100e20 0x7f6fc4103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:59.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.967+0000 7f6fcb8d7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc4102570 0x7f6fc4103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:59.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.967+0000 7f6fcb8d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fc4102ef0 0x7f6fc41041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:59.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.967+0000 7f6fcb8d7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4110f80 con 0x7f6fc4102ef0 2024-09-19T08:13:59.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.967+0000 7f6fcb8d7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4110e00 con 0x7f6fc4100e20 2024-09-19T08:13:59.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.967+0000 7f6fcb8d7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4111100 con 0x7f6fc4102570 2024-09-19T08:13:59.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.967+0000 7f6fc9e74640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fc4102ef0 0x7f6fc41041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:59.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.967+0000 7f6fc9e74640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fc4102ef0 0x7f6fc41041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35900/0 (socket says 172.21.15.22:35900) 2024-09-19T08:13:59.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.967+0000 7f6fc9673640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc4100e20 0x7f6fc4103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:59.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.967+0000 7f6fc9e74640 1 -- 172.21.15.22:0/510724803 learned_addr learned my addr 172.21.15.22:0/510724803 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:13:59.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.968+0000 7f6fc8e72640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc4102570 0x7f6fc4103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:59.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.968+0000 7f6fc9e74640 1 -- 172.21.15.22:0/510724803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc4102570 msgr2=0x7f6fc4103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:59.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.968+0000 7f6fc9e74640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc4102570 0x7f6fc4103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:59.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.968+0000 7f6fc9e74640 1 -- 172.21.15.22:0/510724803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc4100e20 msgr2=0x7f6fc4103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:59.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.968+0000 7f6fc9e74640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc4100e20 0x7f6fc4103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:59.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.968+0000 7f6fc9e74640 1 -- 172.21.15.22:0/510724803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc41a69c0 con 0x7f6fc4102ef0 2024-09-19T08:13:59.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.968+0000 7f6fc9673640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc4100e20 0x7f6fc4103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:13:59.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.968+0000 7f6fc8e72640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc4102570 0x7f6fc4103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:13:59.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.968+0000 7f6fc9e74640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fc4102ef0 0x7f6fc41041d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00eb10 tx=0x7f6fbc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:59.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.968+0000 7f6fba7fc640 1 -- 172.21.15.22:0/510724803 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc004b20 con 0x7f6fc4102ef0 2024-09-19T08:13:59.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.968+0000 7f6fba7fc640 1 -- 172.21.15.22:0/510724803 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fbc004cc0 con 0x7f6fc4102ef0 2024-09-19T08:13:59.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.968+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/510724803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc41a6cb0 con 0x7f6fc4102ef0 2024-09-19T08:13:59.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.968+0000 7f6fba7fc640 1 -- 172.21.15.22:0/510724803 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc010760 con 0x7f6fc4102ef0 2024-09-19T08:13:59.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.968+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/510724803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc41a71a0 con 0x7f6fc4102ef0 2024-09-19T08:13:59.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.969+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/510724803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc4101200 con 0x7f6fc4102ef0 2024-09-19T08:13:59.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.971+0000 7f6fba7fc640 1 -- 172.21.15.22:0/510724803 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6fbc010900 con 0x7f6fc4102ef0 2024-09-19T08:13:59.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.972+0000 7f6fba7fc640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6fb00a32d0 0x7f6fb00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:13:59.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.972+0000 7f6fc9673640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6fb00a32d0 0x7f6fb00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:13:59.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.972+0000 7f6fba7fc640 1 -- 172.21.15.22:0/510724803 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f6fbc0d0290 con 0x7f6fc4102ef0 2024-09-19T08:13:59.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.972+0000 7f6fc9673640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6fb00a32d0 0x7f6fb00a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009230 tx=0x7f6fc00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:13:59.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:58.974+0000 7f6fba7fc640 1 -- 172.21.15.22:0/510724803 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6fbc09cf90 con 0x7f6fc4102ef0 2024-09-19T08:13:59.119 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:13:59.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:59.164+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/510724803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6fc4104c80 con 0x7f6fc4102ef0 2024-09-19T08:13:59.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:59.165+0000 7f6fba7fc640 1 -- 172.21.15.22:0/510724803 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6fc4104c80 con 0x7f6fc4102ef0 2024-09-19T08:13:59.166 INFO:teuthology.orchestra.run.smithi022.stdout:163208757303 2024-09-19T08:13:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:59.167+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/510724803 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6fb00a32d0 msgr2=0x7f6fb00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:59.167+0000 7f6fcb8d7640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6fb00a32d0 0x7f6fb00a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009230 tx=0x7f6fc00058d0 comp rx=0 tx=0).stop 2024-09-19T08:13:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:59.167+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/510724803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fc4102ef0 msgr2=0x7f6fc41041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:13:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:59.167+0000 7f6fcb8d7640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fc4102ef0 0x7f6fc41041d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00eb10 tx=0x7f6fbc00c2d0 comp rx=0 tx=0).stop 2024-09-19T08:13:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:59.167+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/510724803 shutdown_connections 2024-09-19T08:13:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:59.167+0000 7f6fcb8d7640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6fb00a32d0 0x7f6fb00a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:59.167+0000 7f6fcb8d7640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fc4102ef0 0x7f6fc41041d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:59.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:59.167+0000 7f6fcb8d7640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fc4102570 0x7f6fc4103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:59.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:59.167+0000 7f6fcb8d7640 1 --2- 172.21.15.22:0/510724803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc4100e20 0x7f6fc4103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:13:59.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:59.167+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/510724803 >> 172.21.15.22:0/510724803 conn(0x7f6fc40fc7d0 msgr2=0x7f6fc406a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:13:59.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:59.167+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/510724803 shutdown_connections 2024-09-19T08:13:59.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:13:59.167+0000 7f6fcb8d7640 1 -- 172.21.15.22:0/510724803 wait complete. 2024-09-19T08:13:59.574 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738447 for osd.0 2024-09-19T08:13:59.574 DEBUG:teuthology.parallel:result is None 2024-09-19T08:13:59.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:13:59 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/510724803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T08:14:00.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:13:59 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/510724803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T08:14:00.030 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920826 got 141733920828 for osd.4 2024-09-19T08:14:00.030 DEBUG:teuthology.parallel:result is None 2024-09-19T08:14:00.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:13:59 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/510724803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T08:14:00.786 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757300 got 163208757303 for osd.5 2024-09-19T08:14:00.786 DEBUG:teuthology.parallel:result is None 2024-09-19T08:14:01.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:00 smithi157 ceph-mon[35143]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:01.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:00 smithi022 ceph-mon[30963]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:01.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:00 smithi064 ceph-mon[34900]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:01.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.203+0000 7fc044619640 1 -- 172.21.15.22:0/3106836152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc03c103660 msgr2=0x7fc03c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:01.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.203+0000 7fc044619640 1 --2- 172.21.15.22:0/3106836152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc03c103660 0x7fc03c10bb50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc030009ca0 tx=0x7fc03002a950 comp rx=0 tx=0).stop 2024-09-19T08:14:01.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.203+0000 7fc044619640 1 -- 172.21.15.22:0/3106836152 shutdown_connections 2024-09-19T08:14:01.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.203+0000 7fc044619640 1 --2- 172.21.15.22:0/3106836152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc03c10c170 0x7fc03c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.203+0000 7fc044619640 1 --2- 172.21.15.22:0/3106836152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc03c103660 0x7fc03c10bb50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.203+0000 7fc044619640 1 --2- 172.21.15.22:0/3106836152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc03c102d40 0x7fc03c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.203+0000 7fc044619640 1 -- 172.21.15.22:0/3106836152 >> 172.21.15.22:0/3106836152 conn(0x7fc03c0fc7a0 msgr2=0x7fc03c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:01.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.203+0000 7fc044619640 1 -- 172.21.15.22:0/3106836152 shutdown_connections 2024-09-19T08:14:01.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.203+0000 7fc044619640 1 -- 172.21.15.22:0/3106836152 wait complete. 2024-09-19T08:14:01.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.204+0000 7fc044619640 1 Processor -- start 2024-09-19T08:14:01.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.204+0000 7fc044619640 1 -- start start 2024-09-19T08:14:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.204+0000 7fc044619640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc03c102d40 0x7fc03c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.204+0000 7fc044619640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc03c103660 0x7fc03c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.204+0000 7fc044619640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc03c10c170 0x7fc03c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.204+0000 7fc044619640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c111f70 con 0x7fc03c102d40 2024-09-19T08:14:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.204+0000 7fc044619640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c111df0 con 0x7fc03c103660 2024-09-19T08:14:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.204+0000 7fc044619640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c1120f0 con 0x7fc03c10c170 2024-09-19T08:14:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.204+0000 7fc0423b5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc03c102d40 0x7fc03c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.204+0000 7fc0423b5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc03c102d40 0x7fc03c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35920/0 (socket says 172.21.15.22:35920) 2024-09-19T08:14:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.204+0000 7fc0423b5640 1 -- 172.21.15.22:0/2898888116 learned_addr learned my addr 172.21.15.22:0/2898888116 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.204+0000 7fc041bb4640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc03c103660 0x7fc03c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.205+0000 7fc042bb6640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc03c10c170 0x7fc03c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.205+0000 7fc0423b5640 1 -- 172.21.15.22:0/2898888116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc03c10c170 msgr2=0x7fc03c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.205+0000 7fc0423b5640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc03c10c170 0x7fc03c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.205+0000 7fc0423b5640 1 -- 172.21.15.22:0/2898888116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc03c103660 msgr2=0x7fc03c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.205+0000 7fc0423b5640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc03c103660 0x7fc03c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.205+0000 7fc0423b5640 1 -- 172.21.15.22:0/2898888116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c19d440 con 0x7fc03c102d40 2024-09-19T08:14:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.205+0000 7fc042bb6640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc03c10c170 0x7fc03c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.205+0000 7fc0423b5640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc03c102d40 0x7fc03c198490 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc03800d950 tx=0x7fc03800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.205+0000 7fc02f7fe640 1 -- 172.21.15.22:0/2898888116 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038017070 con 0x7fc03c102d40 2024-09-19T08:14:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.205+0000 7fc02f7fe640 1 -- 172.21.15.22:0/2898888116 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc038009c30 con 0x7fc03c102d40 2024-09-19T08:14:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.205+0000 7fc044619640 1 -- 172.21.15.22:0/2898888116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03c19d730 con 0x7fc03c102d40 2024-09-19T08:14:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.205+0000 7fc02f7fe640 1 -- 172.21.15.22:0/2898888116 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03800f6b0 con 0x7fc03c102d40 2024-09-19T08:14:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.205+0000 7fc044619640 1 -- 172.21.15.22:0/2898888116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc03c1a5010 con 0x7fc03c102d40 2024-09-19T08:14:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.206+0000 7fc044619640 1 -- 172.21.15.22:0/2898888116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03c103120 con 0x7fc03c102d40 2024-09-19T08:14:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.207+0000 7fc02f7fe640 1 -- 172.21.15.22:0/2898888116 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc0380109d0 con 0x7fc03c102d40 2024-09-19T08:14:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.208+0000 7fc02f7fe640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc0280a32d0 0x7fc0280a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.208+0000 7fc02f7fe640 1 -- 172.21.15.22:0/2898888116 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fc0380cf530 con 0x7fc03c102d40 2024-09-19T08:14:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.208+0000 7fc041bb4640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc0280a32d0 0x7fc0280a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.208+0000 7fc041bb4640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc0280a32d0 0x7fc0280a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0feda0 tx=0x7fc030004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.211+0000 7fc02f7fe640 1 -- 172.21.15.22:0/2898888116 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc038009de0 con 0x7fc03c102d40 2024-09-19T08:14:01.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.388+0000 7fc044619640 1 -- 172.21.15.22:0/2898888116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc03c02d7c0 con 0x7fc03c102d40 2024-09-19T08:14:01.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.389+0000 7fc02f7fe640 1 -- 172.21.15.22:0/2898888116 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc03809c2a0 con 0x7fc03c102d40 2024-09-19T08:14:01.390 INFO:teuthology.orchestra.run.smithi022.stdout:77309411399 2024-09-19T08:14:01.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.391+0000 7fc044619640 1 -- 172.21.15.22:0/2898888116 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc0280a32d0 msgr2=0x7fc0280a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:01.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.391+0000 7fc044619640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc0280a32d0 0x7fc0280a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0feda0 tx=0x7fc030004a60 comp rx=0 tx=0).stop 2024-09-19T08:14:01.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.391+0000 7fc044619640 1 -- 172.21.15.22:0/2898888116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc03c102d40 msgr2=0x7fc03c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:01.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.391+0000 7fc044619640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc03c102d40 0x7fc03c198490 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc03800d950 tx=0x7fc03800de10 comp rx=0 tx=0).stop 2024-09-19T08:14:01.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.391+0000 7fc044619640 1 -- 172.21.15.22:0/2898888116 shutdown_connections 2024-09-19T08:14:01.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.391+0000 7fc044619640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc0280a32d0 0x7fc0280a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.391+0000 7fc044619640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc03c10c170 0x7fc03c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.391+0000 7fc044619640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc03c103660 0x7fc03c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.391+0000 7fc044619640 1 --2- 172.21.15.22:0/2898888116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc03c102d40 0x7fc03c198490 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.391+0000 7fc044619640 1 -- 172.21.15.22:0/2898888116 >> 172.21.15.22:0/2898888116 conn(0x7fc03c0fc7a0 msgr2=0x7fc03c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:01.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.391+0000 7fc044619640 1 -- 172.21.15.22:0/2898888116 shutdown_connections 2024-09-19T08:14:01.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.391+0000 7fc044619640 1 -- 172.21.15.22:0/2898888116 wait complete. 2024-09-19T08:14:01.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.445+0000 7f560c718640 1 -- 172.21.15.22:0/2430162567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5604102ef0 msgr2=0x7f560410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:01.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.445+0000 7f560c718640 1 --2- 172.21.15.22:0/2430162567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5604102ef0 0x7f560410da40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009ca0 tx=0x7f55fc02a950 comp rx=0 tx=0).stop 2024-09-19T08:14:01.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.445+0000 7f560c718640 1 -- 172.21.15.22:0/2430162567 shutdown_connections 2024-09-19T08:14:01.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.445+0000 7f560c718640 1 --2- 172.21.15.22:0/2430162567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5604102ef0 0x7f560410da40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.445+0000 7f560c718640 1 --2- 172.21.15.22:0/2430162567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5604102570 0x7f56041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.445+0000 7f560c718640 1 --2- 172.21.15.22:0/2430162567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5604100e20 0x7f5604101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.445+0000 7f560c718640 1 -- 172.21.15.22:0/2430162567 >> 172.21.15.22:0/2430162567 conn(0x7f56040fc7d0 msgr2=0x7f56040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:01.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.445+0000 7f560c718640 1 -- 172.21.15.22:0/2430162567 shutdown_connections 2024-09-19T08:14:01.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.445+0000 7f560c718640 1 -- 172.21.15.22:0/2430162567 wait complete. 2024-09-19T08:14:01.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.445+0000 7f560c718640 1 Processor -- start 2024-09-19T08:14:01.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.445+0000 7f560c718640 1 -- start start 2024-09-19T08:14:01.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.445+0000 7f560c718640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5604100e20 0x7f5604197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:01.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560c718640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5604102570 0x7f56041979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:01.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560c718640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5604102ef0 0x7f560419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:01.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560c718640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5604110f80 con 0x7f5604102ef0 2024-09-19T08:14:01.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560c718640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5604110e00 con 0x7f5604100e20 2024-09-19T08:14:01.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560c718640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5604111100 con 0x7f5604102570 2024-09-19T08:14:01.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560acb5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5604102ef0 0x7f560419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:01.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560acb5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5604102ef0 0x7f560419bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35938/0 (socket says 172.21.15.22:35938) 2024-09-19T08:14:01.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560a4b4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5604100e20 0x7f5604197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:01.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560acb5640 1 -- 172.21.15.22:0/4108422655 learned_addr learned my addr 172.21.15.22:0/4108422655 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f5609cb3640 1 --2- 172.21.15.22:0/4108422655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5604102570 0x7f56041979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560acb5640 1 -- 172.21.15.22:0/4108422655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5604102570 msgr2=0x7f56041979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560acb5640 1 --2- 172.21.15.22:0/4108422655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5604102570 0x7f56041979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560acb5640 1 -- 172.21.15.22:0/4108422655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5604100e20 msgr2=0x7f5604197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560acb5640 1 --2- 172.21.15.22:0/4108422655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5604100e20 0x7f5604197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560acb5640 1 -- 172.21.15.22:0/4108422655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560419c410 con 0x7f5604102ef0 2024-09-19T08:14:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560acb5640 1 --2- 172.21.15.22:0/4108422655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5604102ef0 0x7f560419bd30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f55fc004af0 tx=0x7f55fc02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f55f77fe640 1 -- 172.21.15.22:0/4108422655 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc034040 con 0x7f5604102ef0 2024-09-19T08:14:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f55f77fe640 1 -- 172.21.15.22:0/4108422655 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55fc030920 con 0x7f5604102ef0 2024-09-19T08:14:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f560c718640 1 -- 172.21.15.22:0/4108422655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560419c6a0 con 0x7f5604102ef0 2024-09-19T08:14:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.446+0000 7f55f77fe640 1 -- 172.21.15.22:0/4108422655 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc004820 con 0x7f5604102ef0 2024-09-19T08:14:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.447+0000 7f560c718640 1 -- 172.21.15.22:0/4108422655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56041a3fe0 con 0x7f5604102ef0 2024-09-19T08:14:01.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.447+0000 7f560c718640 1 -- 172.21.15.22:0/4108422655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56041021b0 con 0x7f5604102ef0 2024-09-19T08:14:01.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.449+0000 7f55f77fe640 1 -- 172.21.15.22:0/4108422655 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f55fc037020 con 0x7f5604102ef0 2024-09-19T08:14:01.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.451+0000 7f55f77fe640 1 --2- 172.21.15.22:0/4108422655 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f55f00a3350 0x7f55f00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:01.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.451+0000 7f560a4b4640 1 --2- 172.21.15.22:0/4108422655 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f55f00a3350 0x7f55f00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:01.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.451+0000 7f55f77fe640 1 -- 172.21.15.22:0/4108422655 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f55fc0f0510 con 0x7f5604102ef0 2024-09-19T08:14:01.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.451+0000 7f560a4b4640 1 --2- 172.21.15.22:0/4108422655 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f55f00a3350 0x7f55f00a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5600009bd0 tx=0x7f5600005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:01.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.453+0000 7f55f77fe640 1 -- 172.21.15.22:0/4108422655 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f55fc0bd420 con 0x7f5604102ef0 2024-09-19T08:14:01.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.631+0000 7f560c718640 1 -- 172.21.15.22:0/4108422655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f5604101200 con 0x7f5604102ef0 2024-09-19T08:14:01.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.632+0000 7f55f77fe640 1 -- 172.21.15.22:0/4108422655 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5604101200 con 0x7f5604102ef0 2024-09-19T08:14:01.633 INFO:teuthology.orchestra.run.smithi022.stdout:55834574923 2024-09-19T08:14:01.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.634+0000 7f560c718640 1 -- 172.21.15.22:0/4108422655 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f55f00a3350 msgr2=0x7f55f00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:01.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.634+0000 7f560c718640 1 --2- 172.21.15.22:0/4108422655 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f55f00a3350 0x7f55f00a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5600009bd0 tx=0x7f5600005f90 comp rx=0 tx=0).stop 2024-09-19T08:14:01.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.634+0000 7f560c718640 1 -- 172.21.15.22:0/4108422655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5604102ef0 msgr2=0x7f560419bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:01.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.634+0000 7f560c718640 1 --2- 172.21.15.22:0/4108422655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5604102ef0 0x7f560419bd30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f55fc004af0 tx=0x7f55fc02f5b0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.634+0000 7f560c718640 1 -- 172.21.15.22:0/4108422655 shutdown_connections 2024-09-19T08:14:01.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.634+0000 7f560c718640 1 --2- 172.21.15.22:0/4108422655 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f55f00a3350 0x7f55f00a5810 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.634+0000 7f560c718640 1 --2- 172.21.15.22:0/4108422655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5604102ef0 0x7f560419bd30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.634+0000 7f560c718640 1 --2- 172.21.15.22:0/4108422655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5604102570 0x7f56041979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.634+0000 7f560c718640 1 --2- 172.21.15.22:0/4108422655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5604100e20 0x7f5604197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:01.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.634+0000 7f560c718640 1 -- 172.21.15.22:0/4108422655 >> 172.21.15.22:0/4108422655 conn(0x7f56040fc7d0 msgr2=0x7f56041068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:01.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.634+0000 7f560c718640 1 -- 172.21.15.22:0/4108422655 shutdown_connections 2024-09-19T08:14:01.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:01.634+0000 7f560c718640 1 -- 172.21.15.22:0/4108422655 wait complete. 2024-09-19T08:14:02.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:01 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2898888116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T08:14:02.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:01 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/4108422655' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T08:14:02.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:01 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2898888116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T08:14:02.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:01 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/4108422655' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T08:14:02.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:01 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2898888116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T08:14:02.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:01 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/4108422655' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T08:14:02.194 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411399 for osd.2 2024-09-19T08:14:02.194 DEBUG:teuthology.parallel:result is None 2024-09-19T08:14:02.864 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:02 smithi022 ceph-mon[30963]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:14:03.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:02 smithi157 ceph-mon[35143]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:14:03.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:02 smithi064 ceph-mon[34900]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:14:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.436+0000 7f2633044640 1 -- 172.21.15.22:0/310182441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f262c101280 msgr2=0x7f262c10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.436+0000 7f2633044640 1 --2- 172.21.15.22:0/310182441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f262c101280 0x7f262c10df40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2618009900 tx=0x7f261802a410 comp rx=0 tx=0).stop 2024-09-19T08:14:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.436+0000 7f2633044640 1 -- 172.21.15.22:0/310182441 shutdown_connections 2024-09-19T08:14:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.436+0000 7f2633044640 1 --2- 172.21.15.22:0/310182441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c10e480 0x7f262c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.436+0000 7f2633044640 1 --2- 172.21.15.22:0/310182441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f262c101280 0x7f262c10df40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.436+0000 7f2633044640 1 --2- 172.21.15.22:0/310182441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262c1008d0 0x7f262c100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.436+0000 7f2633044640 1 -- 172.21.15.22:0/310182441 >> 172.21.15.22:0/310182441 conn(0x7f262c0fc7a0 msgr2=0x7f262c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.436+0000 7f2633044640 1 -- 172.21.15.22:0/310182441 shutdown_connections 2024-09-19T08:14:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.436+0000 7f2633044640 1 -- 172.21.15.22:0/310182441 wait complete. 2024-09-19T08:14:03.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.437+0000 7f2633044640 1 Processor -- start 2024-09-19T08:14:03.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2633044640 1 -- start start 2024-09-19T08:14:03.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2633044640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f262c1008d0 0x7f262c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:03.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2633044640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262c101280 0x7f262c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:03.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2633044640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c10e480 0x7f262c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:03.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2633044640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1141b0 con 0x7f262c1008d0 2024-09-19T08:14:03.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2633044640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f262c114030 con 0x7f262c10e480 2024-09-19T08:14:03.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2633044640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f262c114330 con 0x7f262c101280 2024-09-19T08:14:03.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2630de0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f262c1008d0 0x7f262c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:03.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2630de0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f262c1008d0 0x7f262c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35964/0 (socket says 172.21.15.22:35964) 2024-09-19T08:14:03.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2630de0640 1 -- 172.21.15.22:0/3109937931 learned_addr learned my addr 172.21.15.22:0/3109937931 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f262bfff640 1 --2- 172.21.15.22:0/3109937931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262c101280 0x7f262c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f26315e1640 1 --2- 172.21.15.22:0/3109937931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c10e480 0x7f262c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2630de0640 1 -- 172.21.15.22:0/3109937931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262c101280 msgr2=0x7f262c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2630de0640 1 --2- 172.21.15.22:0/3109937931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262c101280 0x7f262c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2630de0640 1 -- 172.21.15.22:0/3109937931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c10e480 msgr2=0x7f262c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2630de0640 1 --2- 172.21.15.22:0/3109937931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c10e480 0x7f262c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.438+0000 7f2630de0640 1 -- 172.21.15.22:0/3109937931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c19f6c0 con 0x7f262c1008d0 2024-09-19T08:14:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.439+0000 7f2630de0640 1 --2- 172.21.15.22:0/3109937931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f262c1008d0 0x7f262c19a710 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f262000d770 tx=0x7f262000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.439+0000 7f2629ffb640 1 -- 172.21.15.22:0/3109937931 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620017070 con 0x7f262c1008d0 2024-09-19T08:14:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.439+0000 7f2633044640 1 -- 172.21.15.22:0/3109937931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262c19f9b0 con 0x7f262c1008d0 2024-09-19T08:14:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.439+0000 7f2629ffb640 1 -- 172.21.15.22:0/3109937931 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2620005500 con 0x7f262c1008d0 2024-09-19T08:14:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.439+0000 7f2629ffb640 1 -- 172.21.15.22:0/3109937931 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620004e00 con 0x7f262c1008d0 2024-09-19T08:14:03.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.439+0000 7f2633044640 1 -- 172.21.15.22:0/3109937931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262c1a7290 con 0x7f262c1008d0 2024-09-19T08:14:03.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.440+0000 7f2633044640 1 -- 172.21.15.22:0/3109937931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f262c100cb0 con 0x7f262c1008d0 2024-09-19T08:14:03.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.441+0000 7f2629ffb640 1 -- 172.21.15.22:0/3109937931 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2620004fa0 con 0x7f262c1008d0 2024-09-19T08:14:03.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.442+0000 7f2629ffb640 1 --2- 172.21.15.22:0/3109937931 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f261c0a32d0 0x7f261c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:03.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.442+0000 7f2629ffb640 1 -- 172.21.15.22:0/3109937931 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f26200cf2d0 con 0x7f262c1008d0 2024-09-19T08:14:03.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.442+0000 7f262bfff640 1 --2- 172.21.15.22:0/3109937931 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f261c0a32d0 0x7f261c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:03.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.443+0000 7f262bfff640 1 --2- 172.21.15.22:0/3109937931 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f261c0a32d0 0x7f261c0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2618009230 tx=0x7f26180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:03.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.446+0000 7f2629ffb640 1 -- 172.21.15.22:0/3109937931 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f262009b9f0 con 0x7f262c1008d0 2024-09-19T08:14:03.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.623+0000 7f2633044640 1 -- 172.21.15.22:0/3109937931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f262c109640 con 0x7f262c1008d0 2024-09-19T08:14:03.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.623+0000 7f2629ffb640 1 -- 172.21.15.22:0/3109937931 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f262009fcf0 con 0x7f262c1008d0 2024-09-19T08:14:03.623 INFO:teuthology.orchestra.run.smithi022.stdout:300647710750 2024-09-19T08:14:03.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.625+0000 7f2633044640 1 -- 172.21.15.22:0/3109937931 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f261c0a32d0 msgr2=0x7f261c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:03.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.625+0000 7f2633044640 1 --2- 172.21.15.22:0/3109937931 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f261c0a32d0 0x7f261c0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2618009230 tx=0x7f26180058d0 comp rx=0 tx=0).stop 2024-09-19T08:14:03.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.625+0000 7f2633044640 1 -- 172.21.15.22:0/3109937931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f262c1008d0 msgr2=0x7f262c19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:03.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.625+0000 7f2633044640 1 --2- 172.21.15.22:0/3109937931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f262c1008d0 0x7f262c19a710 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f262000d770 tx=0x7f262000dc30 comp rx=0 tx=0).stop 2024-09-19T08:14:03.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.625+0000 7f2633044640 1 -- 172.21.15.22:0/3109937931 shutdown_connections 2024-09-19T08:14:03.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.625+0000 7f2633044640 1 --2- 172.21.15.22:0/3109937931 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f261c0a32d0 0x7f261c0a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:03.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.625+0000 7f2633044640 1 --2- 172.21.15.22:0/3109937931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c10e480 0x7f262c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:03.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.625+0000 7f2633044640 1 --2- 172.21.15.22:0/3109937931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f262c101280 0x7f262c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:03.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.625+0000 7f2633044640 1 --2- 172.21.15.22:0/3109937931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f262c1008d0 0x7f262c19a710 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:03.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.625+0000 7f2633044640 1 -- 172.21.15.22:0/3109937931 >> 172.21.15.22:0/3109937931 conn(0x7f262c0fc7a0 msgr2=0x7f262c10f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.625+0000 7f2633044640 1 -- 172.21.15.22:0/3109937931 shutdown_connections 2024-09-19T08:14:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:03.625+0000 7f2633044640 1 -- 172.21.15.22:0/3109937931 wait complete. 2024-09-19T08:14:03.835 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574923 for osd.1 2024-09-19T08:14:03.835 DEBUG:teuthology.parallel:result is None 2024-09-19T08:14:04.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:03 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3109937931' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T08:14:04.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:03 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3109937931' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T08:14:04.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:03 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3109937931' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T08:14:04.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.269+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/2626735863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f18106a80 msgr2=0x7f4f18106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:04.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.269+0000 7f4f1cd2d640 1 --2- 172.21.15.22:0/2626735863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f18106a80 0x7f4f18106e60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009930 tx=0x7f4f0802a420 comp rx=0 tx=0).stop 2024-09-19T08:14:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.269+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/2626735863 shutdown_connections 2024-09-19T08:14:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.269+0000 7f4f1cd2d640 1 --2- 172.21.15.22:0/2626735863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f18101420 0x7f4f1810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.269+0000 7f4f1cd2d640 1 --2- 172.21.15.22:0/2626735863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f18100a80 0x7f4f18100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.269+0000 7f4f1cd2d640 1 --2- 172.21.15.22:0/2626735863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f18106a80 0x7f4f18106e60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.269+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/2626735863 >> 172.21.15.22:0/2626735863 conn(0x7f4f180fc7a0 msgr2=0x7f4f180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.270+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/2626735863 shutdown_connections 2024-09-19T08:14:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.270+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/2626735863 wait complete. 2024-09-19T08:14:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.270+0000 7f4f1cd2d640 1 Processor -- start 2024-09-19T08:14:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.270+0000 7f4f1cd2d640 1 -- start start 2024-09-19T08:14:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.270+0000 7f4f1cd2d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f18100a80 0x7f4f18102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.270+0000 7f4f1cd2d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f18101420 0x7f4f18102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.270+0000 7f4f1cd2d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f18106a80 0x7f4f181030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.270+0000 7f4f1cd2d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18110ed0 con 0x7f4f18106a80 2024-09-19T08:14:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.270+0000 7f4f1cd2d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18110d50 con 0x7f4f18101420 2024-09-19T08:14:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.270+0000 7f4f1cd2d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18111050 con 0x7f4f18100a80 2024-09-19T08:14:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f16d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f18106a80 0x7f4f181030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f16d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f18106a80 0x7f4f181030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35980/0 (socket says 172.21.15.22:35980) 2024-09-19T08:14:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f16d9d640 1 -- 172.21.15.22:0/1705373479 learned_addr learned my addr 172.21.15.22:0/1705373479 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f1659c640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f18100a80 0x7f4f18102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f15d9b640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f18101420 0x7f4f18102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f16d9d640 1 -- 172.21.15.22:0/1705373479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f18100a80 msgr2=0x7f4f18102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f16d9d640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f18100a80 0x7f4f18102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:04.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f16d9d640 1 -- 172.21.15.22:0/1705373479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f18101420 msgr2=0x7f4f18102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:04.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f16d9d640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f18101420 0x7f4f18102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:04.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f16d9d640 1 -- 172.21.15.22:0/1705373479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f181a68a0 con 0x7f4f18106a80 2024-09-19T08:14:04.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f15d9b640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f18101420 0x7f4f18102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:04.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f16d9d640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f18106a80 0x7f4f181030d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000cce0 tx=0x7f4f1000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:04.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f077fe640 1 -- 172.21.15.22:0/1705373479 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1000f450 con 0x7f4f18106a80 2024-09-19T08:14:04.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f077fe640 1 -- 172.21.15.22:0/1705373479 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f1000fad0 con 0x7f4f18106a80 2024-09-19T08:14:04.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/1705373479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f181a6b90 con 0x7f4f18106a80 2024-09-19T08:14:04.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f077fe640 1 -- 172.21.15.22:0/1705373479 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10018c10 con 0x7f4f18106a80 2024-09-19T08:14:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.271+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/1705373479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f181a70d0 con 0x7f4f18106a80 2024-09-19T08:14:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.272+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/1705373479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f18100ee0 con 0x7f4f18106a80 2024-09-19T08:14:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.274+0000 7f4f077fe640 1 -- 172.21.15.22:0/1705373479 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4f1000f5f0 con 0x7f4f18106a80 2024-09-19T08:14:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.274+0000 7f4f077fe640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4f000a3350 0x7f4f000a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.274+0000 7f4f1659c640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4f000a3350 0x7f4f000a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.274+0000 7f4f077fe640 1 -- 172.21.15.22:0/1705373479 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f4f100cfea0 con 0x7f4f18106a80 2024-09-19T08:14:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.275+0000 7f4f1659c640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4f000a3350 0x7f4f000a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4f08006010 tx=0x7f4f08000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.278+0000 7f4f077fe640 1 -- 172.21.15.22:0/1705373479 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4f10014030 con 0x7f4f18106a80 2024-09-19T08:14:04.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.461+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/1705373479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4f181a7360 con 0x7f4f18106a80 2024-09-19T08:14:04.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.461+0000 7f4f077fe640 1 -- 172.21.15.22:0/1705373479 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4f1009cc10 con 0x7f4f18106a80 2024-09-19T08:14:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:107374182466 2024-09-19T08:14:04.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.464+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/1705373479 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4f000a3350 msgr2=0x7f4f000a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:04.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.464+0000 7f4f1cd2d640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4f000a3350 0x7f4f000a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4f08006010 tx=0x7f4f08000bc0 comp rx=0 tx=0).stop 2024-09-19T08:14:04.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.464+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/1705373479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f18106a80 msgr2=0x7f4f181030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:04.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.464+0000 7f4f1cd2d640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f18106a80 0x7f4f181030d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000cce0 tx=0x7f4f1000a620 comp rx=0 tx=0).stop 2024-09-19T08:14:04.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.464+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/1705373479 shutdown_connections 2024-09-19T08:14:04.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.464+0000 7f4f1cd2d640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4f000a3350 0x7f4f000a5810 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:04.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.464+0000 7f4f1cd2d640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f18106a80 0x7f4f181030d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:04.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.464+0000 7f4f1cd2d640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f18101420 0x7f4f18102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:04.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.464+0000 7f4f1cd2d640 1 --2- 172.21.15.22:0/1705373479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f18100a80 0x7f4f18102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:04.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.465+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/1705373479 >> 172.21.15.22:0/1705373479 conn(0x7f4f180fc7a0 msgr2=0x7f4f1806a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:04.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.465+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/1705373479 shutdown_connections 2024-09-19T08:14:04.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:04.465+0000 7f4f1cd2d640 1 -- 172.21.15.22:0/1705373479 wait complete. 2024-09-19T08:14:04.746 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481814 got 330712481816 for osd.11 2024-09-19T08:14:04.746 DEBUG:teuthology.parallel:result is None 2024-09-19T08:14:05.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:04 smithi157 ceph-mon[35143]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:05.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:04 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1705373479' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T08:14:05.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:04 smithi022 ceph-mon[30963]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:05.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:04 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1705373479' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T08:14:05.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:04 smithi064 ceph-mon[34900]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:05.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:04 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1705373479' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T08:14:05.179 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710748 got 300647710750 for osd.10 2024-09-19T08:14:05.179 DEBUG:teuthology.parallel:result is None 2024-09-19T08:14:05.447 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:05.922 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182463 got 107374182466 for osd.3 2024-09-19T08:14:05.922 DEBUG:teuthology.parallel:result is None 2024-09-19T08:14:06.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.410+0000 7f59d2733640 1 -- 172.21.15.22:0/1607013187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59cc102570 msgr2=0x7f59cc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:06.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.410+0000 7f59d2733640 1 --2- 172.21.15.22:0/1607013187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59cc102570 0x7f59cc1029b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009900 tx=0x7f59c002a480 comp rx=0 tx=0).stop 2024-09-19T08:14:06.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.410+0000 7f59d2733640 1 -- 172.21.15.22:0/1607013187 shutdown_connections 2024-09-19T08:14:06.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.410+0000 7f59d2733640 1 --2- 172.21.15.22:0/1607013187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59cc102ef0 0x7f59cc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:06.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.410+0000 7f59d2733640 1 --2- 172.21.15.22:0/1607013187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59cc102570 0x7f59cc1029b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:06.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.410+0000 7f59d2733640 1 --2- 172.21.15.22:0/1607013187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59cc100e20 0x7f59cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:06.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.410+0000 7f59d2733640 1 -- 172.21.15.22:0/1607013187 >> 172.21.15.22:0/1607013187 conn(0x7f59cc0fc7d0 msgr2=0x7f59cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.410+0000 7f59d2733640 1 -- 172.21.15.22:0/1607013187 shutdown_connections 2024-09-19T08:14:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.410+0000 7f59d2733640 1 -- 172.21.15.22:0/1607013187 wait complete. 2024-09-19T08:14:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.410+0000 7f59d2733640 1 Processor -- start 2024-09-19T08:14:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d2733640 1 -- start start 2024-09-19T08:14:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d2733640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59cc100e20 0x7f59cc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d2733640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59cc102570 0x7f59cc198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d2733640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59cc102ef0 0x7f59cc19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d2733640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc110f80 con 0x7f59cc102ef0 2024-09-19T08:14:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d2733640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc110e00 con 0x7f59cc102570 2024-09-19T08:14:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d2733640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc111100 con 0x7f59cc100e20 2024-09-19T08:14:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d0cd0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59cc102ef0 0x7f59cc19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d0cd0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59cc102ef0 0x7f59cc19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35990/0 (socket says 172.21.15.22:35990) 2024-09-19T08:14:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d0cd0640 1 -- 172.21.15.22:0/1058493360 learned_addr learned my addr 172.21.15.22:0/1058493360 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59cb7fe640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59cc102570 0x7f59cc198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59cbfff640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59cc100e20 0x7f59cc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d0cd0640 1 -- 172.21.15.22:0/1058493360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59cc100e20 msgr2=0x7f59cc1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d0cd0640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59cc100e20 0x7f59cc1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d0cd0640 1 -- 172.21.15.22:0/1058493360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59cc102570 msgr2=0x7f59cc198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d0cd0640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59cc102570 0x7f59cc198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.411+0000 7f59d0cd0640 1 -- 172.21.15.22:0/1058493360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc19d490 con 0x7f59cc102ef0 2024-09-19T08:14:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.412+0000 7f59cb7fe640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59cc102570 0x7f59cc198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.412+0000 7f59d0cd0640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59cc102ef0 0x7f59cc19cdb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f59c400cce0 tx=0x7f59c400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.412+0000 7f59c97fa640 1 -- 172.21.15.22:0/1058493360 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c400f450 con 0x7f59cc102ef0 2024-09-19T08:14:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.412+0000 7f59c97fa640 1 -- 172.21.15.22:0/1058493360 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59c400fad0 con 0x7f59cc102ef0 2024-09-19T08:14:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.412+0000 7f59d2733640 1 -- 172.21.15.22:0/1058493360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59cc19d780 con 0x7f59cc102ef0 2024-09-19T08:14:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.412+0000 7f59c97fa640 1 -- 172.21.15.22:0/1058493360 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4018c10 con 0x7f59cc102ef0 2024-09-19T08:14:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.412+0000 7f59d2733640 1 -- 172.21.15.22:0/1058493360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59cc1a5060 con 0x7f59cc102ef0 2024-09-19T08:14:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.413+0000 7f59d2733640 1 -- 172.21.15.22:0/1058493360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59cc101200 con 0x7f59cc102ef0 2024-09-19T08:14:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.414+0000 7f59c97fa640 1 -- 172.21.15.22:0/1058493360 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f59c400f5f0 con 0x7f59cc102ef0 2024-09-19T08:14:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.415+0000 7f59c97fa640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f59b40a32d0 0x7f59b40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.415+0000 7f59cbfff640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f59b40a32d0 0x7f59b40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.415+0000 7f59c97fa640 1 -- 172.21.15.22:0/1058493360 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f59c40d00a0 con 0x7f59cc102ef0 2024-09-19T08:14:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.416+0000 7f59cbfff640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f59b40a32d0 0x7f59b40a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0058c0 tx=0x7f59bc00a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.419+0000 7f59c97fa640 1 -- 172.21.15.22:0/1058493360 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59c4014030 con 0x7f59cc102ef0 2024-09-19T08:14:06.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.596+0000 7f59d2733640 1 -- 172.21.15.22:0/1058493360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f59cc199840 con 0x7f59cc102ef0 2024-09-19T08:14:06.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.597+0000 7f59c97fa640 1 -- 172.21.15.22:0/1058493360 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f59cc199840 con 0x7f59cc102ef0 2024-09-19T08:14:06.597 INFO:teuthology.orchestra.run.smithi022.stdout:249108103209 2024-09-19T08:14:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.599+0000 7f59d2733640 1 -- 172.21.15.22:0/1058493360 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f59b40a32d0 msgr2=0x7f59b40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.599+0000 7f59d2733640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f59b40a32d0 0x7f59b40a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0058c0 tx=0x7f59bc00a4c0 comp rx=0 tx=0).stop 2024-09-19T08:14:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.599+0000 7f59d2733640 1 -- 172.21.15.22:0/1058493360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59cc102ef0 msgr2=0x7f59cc19cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.599+0000 7f59d2733640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59cc102ef0 0x7f59cc19cdb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f59c400cce0 tx=0x7f59c400a620 comp rx=0 tx=0).stop 2024-09-19T08:14:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.599+0000 7f59d2733640 1 -- 172.21.15.22:0/1058493360 shutdown_connections 2024-09-19T08:14:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.599+0000 7f59d2733640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f59b40a32d0 0x7f59b40a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.599+0000 7f59d2733640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59cc102ef0 0x7f59cc19cdb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.599+0000 7f59d2733640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59cc102570 0x7f59cc198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.599+0000 7f59d2733640 1 --2- 172.21.15.22:0/1058493360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59cc100e20 0x7f59cc1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.599+0000 7f59d2733640 1 -- 172.21.15.22:0/1058493360 >> 172.21.15.22:0/1058493360 conn(0x7f59cc0fc7d0 msgr2=0x7f59cc06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.599+0000 7f59d2733640 1 -- 172.21.15.22:0/1058493360 shutdown_connections 2024-09-19T08:14:06.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:06.599+0000 7f59d2733640 1 -- 172.21.15.22:0/1058493360 wait complete. 2024-09-19T08:14:07.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:06 smithi157 ceph-mon[35143]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-09-19T08:14:07.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:06 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1058493360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T08:14:07.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-19T08:14:07.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-19T08:14:07.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-19T08:14:07.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T08:14:07.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-19T08:14:07.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-09-19T08:14:07.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-09-19T08:14:07.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T08:14:07.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:06 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-09-19T08:14:07.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:06 smithi022 ceph-mon[30963]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-09-19T08:14:07.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:06 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1058493360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T08:14:07.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-19T08:14:07.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-19T08:14:07.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-19T08:14:07.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T08:14:07.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-19T08:14:07.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-09-19T08:14:07.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-09-19T08:14:07.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T08:14:07.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:06 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-09-19T08:14:07.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:06 smithi064 ceph-mon[34900]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-09-19T08:14:07.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:06 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1058493360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T08:14:07.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-19T08:14:07.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-19T08:14:07.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-19T08:14:07.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T08:14:07.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-19T08:14:07.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-09-19T08:14:07.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-09-19T08:14:07.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T08:14:07.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:06 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-09-19T08:14:07.300 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103207 got 249108103209 for osd.8 2024-09-19T08:14:07.300 DEBUG:teuthology.parallel:result is None 2024-09-19T08:14:07.300 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T08:14:07.301 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:14:07.990 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:07 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-19T08:14:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:07 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-19T08:14:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:07 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-19T08:14:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:07 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T08:14:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:07 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-19T08:14:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:07 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-19T08:14:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:07 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-09-19T08:14:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:07 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T08:14:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:07 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-19T08:14:08.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:07 smithi157 ceph-mon[35143]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T08:14:08.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:07 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-19T08:14:08.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:07 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-19T08:14:08.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:07 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-19T08:14:08.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:07 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T08:14:08.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:07 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-19T08:14:08.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:07 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-19T08:14:08.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:07 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-09-19T08:14:08.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:07 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T08:14:08.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:07 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-19T08:14:08.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:07 smithi022 ceph-mon[30963]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T08:14:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:07 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-19T08:14:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:07 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-19T08:14:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:07 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-19T08:14:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:07 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T08:14:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:07 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-19T08:14:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:07 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-19T08:14:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:07 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-09-19T08:14:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:07 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T08:14:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:07 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-19T08:14:08.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:07 smithi064 ceph-mon[34900]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T08:14:08.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.479+0000 7f5d25108640 1 -- 172.21.15.22:0/2861756460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d20102570 msgr2=0x7f5d201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:08.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.479+0000 7f5d25108640 1 --2- 172.21.15.22:0/2861756460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d20102570 0x7f5d201029b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009900 tx=0x7f5d0802a480 comp rx=0 tx=0).stop 2024-09-19T08:14:08.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.479+0000 7f5d25108640 1 -- 172.21.15.22:0/2861756460 shutdown_connections 2024-09-19T08:14:08.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.479+0000 7f5d25108640 1 --2- 172.21.15.22:0/2861756460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d20102ef0 0x7f5d2010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:08.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.479+0000 7f5d25108640 1 --2- 172.21.15.22:0/2861756460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d20102570 0x7f5d201029b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:08.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.479+0000 7f5d25108640 1 --2- 172.21.15.22:0/2861756460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d20100e20 0x7f5d20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:08.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.479+0000 7f5d25108640 1 -- 172.21.15.22:0/2861756460 >> 172.21.15.22:0/2861756460 conn(0x7f5d200fc7d0 msgr2=0x7f5d200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:08.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.479+0000 7f5d25108640 1 -- 172.21.15.22:0/2861756460 shutdown_connections 2024-09-19T08:14:08.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.480+0000 7f5d25108640 1 -- 172.21.15.22:0/2861756460 wait complete. 2024-09-19T08:14:08.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.480+0000 7f5d25108640 1 Processor -- start 2024-09-19T08:14:08.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d25108640 1 -- start start 2024-09-19T08:14:08.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d25108640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d20100e20 0x7f5d200fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:08.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d25108640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d20102570 0x7f5d200fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:08.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d25108640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d20102ef0 0x7f5d200fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:08.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d25108640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20110f80 con 0x7f5d20100e20 2024-09-19T08:14:08.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d25108640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20110e00 con 0x7f5d20102ef0 2024-09-19T08:14:08.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d25108640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20111100 con 0x7f5d20102570 2024-09-19T08:14:08.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d1ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d20100e20 0x7f5d200fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:08.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d1ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d20100e20 0x7f5d200fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60044/0 (socket says 172.21.15.22:60044) 2024-09-19T08:14:08.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d1ed9d640 1 -- 172.21.15.22:0/3460150828 learned_addr learned my addr 172.21.15.22:0/3460150828 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:08.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d1f59e640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d20102ef0 0x7f5d200fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:08.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d1e59c640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d20102570 0x7f5d200fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:08.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d1ed9d640 1 -- 172.21.15.22:0/3460150828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d20102570 msgr2=0x7f5d200fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:08.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d1ed9d640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d20102570 0x7f5d200fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:08.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d1ed9d640 1 -- 172.21.15.22:0/3460150828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d20102ef0 msgr2=0x7f5d200fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:08.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d1ed9d640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d20102ef0 0x7f5d200fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:08.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d1ed9d640 1 -- 172.21.15.22:0/3460150828 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d201a69a0 con 0x7f5d20100e20 2024-09-19T08:14:08.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d1f59e640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d20102ef0 0x7f5d200fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:08.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.481+0000 7f5d1ed9d640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d20100e20 0x7f5d200fcc80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800e9f0 tx=0x7f5d1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:08.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.482+0000 7f5d17fff640 1 -- 172.21.15.22:0/3460150828 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1800cd80 con 0x7f5d20100e20 2024-09-19T08:14:08.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.482+0000 7f5d17fff640 1 -- 172.21.15.22:0/3460150828 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d18010910 con 0x7f5d20100e20 2024-09-19T08:14:08.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.482+0000 7f5d25108640 1 -- 172.21.15.22:0/3460150828 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d201a6c90 con 0x7f5d20100e20 2024-09-19T08:14:08.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.482+0000 7f5d17fff640 1 -- 172.21.15.22:0/3460150828 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18018990 con 0x7f5d20100e20 2024-09-19T08:14:08.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.482+0000 7f5d25108640 1 -- 172.21.15.22:0/3460150828 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d201a7180 con 0x7f5d20100e20 2024-09-19T08:14:08.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.483+0000 7f5d25108640 1 -- 172.21.15.22:0/3460150828 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d20101200 con 0x7f5d20100e20 2024-09-19T08:14:08.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.484+0000 7f5d17fff640 1 -- 172.21.15.22:0/3460150828 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5d18010ac0 con 0x7f5d20100e20 2024-09-19T08:14:08.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.485+0000 7f5d17fff640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f5d0c0a32d0 0x7f5d0c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:08.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.485+0000 7f5d1e59c640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f5d0c0a32d0 0x7f5d0c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:08.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.485+0000 7f5d17fff640 1 -- 172.21.15.22:0/3460150828 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5d180cff30 con 0x7f5d20100e20 2024-09-19T08:14:08.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.485+0000 7f5d1e59c640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f5d0c0a32d0 0x7f5d0c0a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5d200fe710 tx=0x7f5d08005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:08.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.489+0000 7f5d17fff640 1 -- 172.21.15.22:0/3460150828 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5d1809cab0 con 0x7f5d20100e20 2024-09-19T08:14:08.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.688+0000 7f5d25108640 1 -- 172.21.15.22:0/3460150828 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d20062630 con 0x7f5d0c0a32d0 2024-09-19T08:14:08.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.700+0000 7f5d17fff640 1 -- 172.21.15.22:0/3460150828 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258799 (secure 0 0 0) 0x7f5d20062630 con 0x7f5d0c0a32d0 2024-09-19T08:14:08.703 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:08.704 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T08:14:08.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.704+0000 7f5d25108640 1 -- 172.21.15.22:0/3460150828 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f5d0c0a32d0 msgr2=0x7f5d0c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:08.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.704+0000 7f5d25108640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f5d0c0a32d0 0x7f5d0c0a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5d200fe710 tx=0x7f5d08005a20 comp rx=0 tx=0).stop 2024-09-19T08:14:08.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.704+0000 7f5d25108640 1 -- 172.21.15.22:0/3460150828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d20100e20 msgr2=0x7f5d200fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:08.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.704+0000 7f5d25108640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d20100e20 0x7f5d200fcc80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800e9f0 tx=0x7f5d1800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:14:08.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.704+0000 7f5d25108640 1 -- 172.21.15.22:0/3460150828 shutdown_connections 2024-09-19T08:14:08.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.704+0000 7f5d25108640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f5d0c0a32d0 0x7f5d0c0a5790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:08.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.704+0000 7f5d25108640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d20102ef0 0x7f5d200fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:08.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.704+0000 7f5d25108640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d20102570 0x7f5d200fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:08.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.704+0000 7f5d25108640 1 --2- 172.21.15.22:0/3460150828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d20100e20 0x7f5d200fcc80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:08.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.704+0000 7f5d25108640 1 -- 172.21.15.22:0/3460150828 >> 172.21.15.22:0/3460150828 conn(0x7f5d200fc7d0 msgr2=0x7f5d2006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:08.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.704+0000 7f5d25108640 1 -- 172.21.15.22:0/3460150828 shutdown_connections 2024-09-19T08:14:08.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:08.704+0000 7f5d25108640 1 -- 172.21.15.22:0/3460150828 wait complete. 2024-09-19T08:14:09.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:08 smithi157 ceph-mon[35143]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T08:14:09.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:08 smithi157 ceph-mon[35143]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T08:14:09.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:08 smithi022 ceph-mon[30963]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T08:14:09.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:08 smithi022 ceph-mon[30963]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T08:14:09.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:08 smithi064 ceph-mon[34900]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T08:14:09.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:08 smithi064 ceph-mon[34900]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T08:14:09.391 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-19T08:14:07.730064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656170,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":488,"num_read_kb":364,"num_write":347,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381040,"kb_used_data":10992,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560784,"statfs":{"total":1151940427776,"available":1151550242816,"internally_reserved":0,"allocated":11255808,"data_stored":7104834,"data_compressed":109176,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.953662"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948678+0000","last_change":"2024-09-19T08:13:00.853947+0000","last_active":"2024-09-19T08:13:01.948678+0000","last_peered":"2024-09-19T08:13:01.948678+0000","last_clean":"2024-09-19T08:13:01.948678+0000","last_became_active":"2024-09-19T08:13:00.852951+0000","last_became_peered":"2024-09-19T08:13:00.852951+0000","last_unstale":"2024-09-19T08:13:01.948678+0000","last_undegraded":"2024-09-19T08:13:01.948678+0000","last_fullsized":"2024-09-19T08:13:01.948678+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:25.786828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948757+0000","last_change":"2024-09-19T08:13:00.853389+0000","last_active":"2024-09-19T08:13:01.948757+0000","last_peered":"2024-09-19T08:13:01.948757+0000","last_clean":"2024-09-19T08:13:01.948757+0000","last_became_active":"2024-09-19T08:13:00.853102+0000","last_became_peered":"2024-09-19T08:13:00.853102+0000","last_unstale":"2024-09-19T08:13:01.948757+0000","last_undegraded":"2024-09-19T08:13:01.948757+0000","last_fullsized":"2024-09-19T08:13:01.948757+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:42.773917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948582+0000","last_change":"2024-09-19T08:13:00.846544+0000","last_active":"2024-09-19T08:13:01.948582+0000","last_peered":"2024-09-19T08:13:01.948582+0000","last_clean":"2024-09-19T08:13:01.948582+0000","last_became_active":"2024-09-19T08:13:00.846229+0000","last_became_peered":"2024-09-19T08:13:00.846229+0000","last_unstale":"2024-09-19T08:13:01.948582+0000","last_undegraded":"2024-09-19T08:13:01.948582+0000","last_fullsized":"2024-09-19T08:13:01.948582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:43.471684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948555+0000","last_change":"2024-09-19T08:13:00.848149+0000","last_active":"2024-09-19T08:13:01.948555+0000","last_peered":"2024-09-19T08:13:01.948555+0000","last_clean":"2024-09-19T08:13:01.948555+0000","last_became_active":"2024-09-19T08:13:00.847744+0000","last_became_peered":"2024-09-19T08:13:00.847744+0000","last_unstale":"2024-09-19T08:13:01.948555+0000","last_undegraded":"2024-09-19T08:13:01.948555+0000","last_fullsized":"2024-09-19T08:13:01.948555+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:01.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948347+0000","last_change":"2024-09-19T08:13:00.841862+0000","last_active":"2024-09-19T08:13:01.948347+0000","last_peered":"2024-09-19T08:13:01.948347+0000","last_clean":"2024-09-19T08:13:01.948347+0000","last_became_active":"2024-09-19T08:13:00.841331+0000","last_became_peered":"2024-09-19T08:13:00.841331+0000","last_unstale":"2024-09-19T08:13:01.948347+0000","last_undegraded":"2024-09-19T08:13:01.948347+0000","last_fullsized":"2024-09-19T08:13:01.948347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:54.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947492+0000","last_change":"2024-09-19T08:13:00.849626+0000","last_active":"2024-09-19T08:13:01.947492+0000","last_peered":"2024-09-19T08:13:01.947492+0000","last_clean":"2024-09-19T08:13:01.947492+0000","last_became_active":"2024-09-19T08:13:00.849034+0000","last_became_peered":"2024-09-19T08:13:00.849034+0000","last_unstale":"2024-09-19T08:13:01.947492+0000","last_undegraded":"2024-09-19T08:13:01.947492+0000","last_fullsized":"2024-09-19T08:13:01.947492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:39.118978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100394+0000","last_change":"2024-09-19T08:13:00.847522+0000","last_active":"2024-09-19T08:13:02.100394+0000","last_peered":"2024-09-19T08:13:02.100394+0000","last_clean":"2024-09-19T08:13:02.100394+0000","last_became_active":"2024-09-19T08:13:00.847117+0000","last_became_peered":"2024-09-19T08:13:00.847117+0000","last_unstale":"2024-09-19T08:13:02.100394+0000","last_undegraded":"2024-09-19T08:13:02.100394+0000","last_fullsized":"2024-09-19T08:13:02.100394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.836385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947191+0000","last_change":"2024-09-19T08:13:00.849544+0000","last_active":"2024-09-19T08:13:01.947191+0000","last_peered":"2024-09-19T08:13:01.947191+0000","last_clean":"2024-09-19T08:13:01.947191+0000","last_became_active":"2024-09-19T08:13:00.849170+0000","last_became_peered":"2024-09-19T08:13:00.849170+0000","last_unstale":"2024-09-19T08:13:01.947191+0000","last_undegraded":"2024-09-19T08:13:01.947191+0000","last_fullsized":"2024-09-19T08:13:01.947191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:28.371965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836582+0000","last_change":"2024-09-19T08:13:00.856052+0000","last_active":"2024-09-19T08:13:01.836582+0000","last_peered":"2024-09-19T08:13:01.836582+0000","last_clean":"2024-09-19T08:13:01.836582+0000","last_became_active":"2024-09-19T08:13:00.855291+0000","last_became_peered":"2024-09-19T08:13:00.855291+0000","last_unstale":"2024-09-19T08:13:01.836582+0000","last_undegraded":"2024-09-19T08:13:01.836582+0000","last_fullsized":"2024-09-19T08:13:01.836582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:18.581867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948289+0000","last_change":"2024-09-19T08:13:00.850145+0000","last_active":"2024-09-19T08:13:01.948289+0000","last_peered":"2024-09-19T08:13:01.948289+0000","last_clean":"2024-09-19T08:13:01.948289+0000","last_became_active":"2024-09-19T08:13:00.849798+0000","last_became_peered":"2024-09-19T08:13:00.849798+0000","last_unstale":"2024-09-19T08:13:01.948289+0000","last_undegraded":"2024-09-19T08:13:01.948289+0000","last_fullsized":"2024-09-19T08:13:01.948289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:21.400000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100790+0000","last_change":"2024-09-19T08:13:00.859153+0000","last_active":"2024-09-19T08:13:02.100790+0000","last_peered":"2024-09-19T08:13:02.100790+0000","last_clean":"2024-09-19T08:13:02.100790+0000","last_became_active":"2024-09-19T08:13:00.858473+0000","last_became_peered":"2024-09-19T08:13:00.858473+0000","last_unstale":"2024-09-19T08:13:02.100790+0000","last_undegraded":"2024-09-19T08:13:02.100790+0000","last_fullsized":"2024-09-19T08:13:02.100790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:07.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948660+0000","last_change":"2024-09-19T08:13:00.850797+0000","last_active":"2024-09-19T08:13:01.948660+0000","last_peered":"2024-09-19T08:13:01.948660+0000","last_clean":"2024-09-19T08:13:01.948660+0000","last_became_active":"2024-09-19T08:13:00.850393+0000","last_became_peered":"2024-09-19T08:13:00.850393+0000","last_unstale":"2024-09-19T08:13:01.948660+0000","last_undegraded":"2024-09-19T08:13:01.948660+0000","last_fullsized":"2024-09-19T08:13:01.948660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:59.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947687+0000","last_change":"2024-09-19T08:13:00.850835+0000","last_active":"2024-09-19T08:13:01.947687+0000","last_peered":"2024-09-19T08:13:01.947687+0000","last_clean":"2024-09-19T08:13:01.947687+0000","last_became_active":"2024-09-19T08:13:00.850518+0000","last_became_peered":"2024-09-19T08:13:00.850518+0000","last_unstale":"2024-09-19T08:13:01.947687+0000","last_undegraded":"2024-09-19T08:13:01.947687+0000","last_fullsized":"2024-09-19T08:13:01.947687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.178530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948530+0000","last_change":"2024-09-19T08:13:00.853934+0000","last_active":"2024-09-19T08:13:01.948530+0000","last_peered":"2024-09-19T08:13:01.948530+0000","last_clean":"2024-09-19T08:13:01.948530+0000","last_became_active":"2024-09-19T08:13:00.853604+0000","last_became_peered":"2024-09-19T08:13:00.853604+0000","last_unstale":"2024-09-19T08:13:01.948530+0000","last_undegraded":"2024-09-19T08:13:01.948530+0000","last_fullsized":"2024-09-19T08:13:01.948530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:43.622526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948350+0000","last_change":"2024-09-19T08:13:00.846026+0000","last_active":"2024-09-19T08:13:01.948350+0000","last_peered":"2024-09-19T08:13:01.948350+0000","last_clean":"2024-09-19T08:13:01.948350+0000","last_became_active":"2024-09-19T08:13:00.845691+0000","last_became_peered":"2024-09-19T08:13:00.845691+0000","last_unstale":"2024-09-19T08:13:01.948350+0000","last_undegraded":"2024-09-19T08:13:01.948350+0000","last_fullsized":"2024-09-19T08:13:01.948350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:55.724159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836019+0000","last_change":"2024-09-19T08:13:00.850266+0000","last_active":"2024-09-19T08:13:01.836019+0000","last_peered":"2024-09-19T08:13:01.836019+0000","last_clean":"2024-09-19T08:13:01.836019+0000","last_became_active":"2024-09-19T08:13:00.849928+0000","last_became_peered":"2024-09-19T08:13:00.849928+0000","last_unstale":"2024-09-19T08:13:01.836019+0000","last_undegraded":"2024-09-19T08:13:01.836019+0000","last_fullsized":"2024-09-19T08:13:01.836019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:38.231331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948706+0000","last_change":"2024-09-19T08:13:00.852551+0000","last_active":"2024-09-19T08:13:01.948706+0000","last_peered":"2024-09-19T08:13:01.948706+0000","last_clean":"2024-09-19T08:13:01.948706+0000","last_became_active":"2024-09-19T08:13:00.852225+0000","last_became_peered":"2024-09-19T08:13:00.852225+0000","last_unstale":"2024-09-19T08:13:01.948706+0000","last_undegraded":"2024-09-19T08:13:01.948706+0000","last_fullsized":"2024-09-19T08:13:01.948706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:33.115108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726544+0000","last_change":"2024-09-19T08:13:00.854033+0000","last_active":"2024-09-19T08:14:06.726544+0000","last_peered":"2024-09-19T08:14:06.726544+0000","last_clean":"2024-09-19T08:14:06.726544+0000","last_became_active":"2024-09-19T08:13:00.853660+0000","last_became_peered":"2024-09-19T08:13:00.853660+0000","last_unstale":"2024-09-19T08:14:06.726544+0000","last_undegraded":"2024-09-19T08:14:06.726544+0000","last_fullsized":"2024-09-19T08:14:06.726544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:35.602420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.832908+0000","last_change":"2024-09-19T08:13:00.850658+0000","last_active":"2024-09-19T08:13:01.832908+0000","last_peered":"2024-09-19T08:13:01.832908+0000","last_clean":"2024-09-19T08:13:01.832908+0000","last_became_active":"2024-09-19T08:13:00.850324+0000","last_became_peered":"2024-09-19T08:13:00.850324+0000","last_unstale":"2024-09-19T08:13:01.832908+0000","last_undegraded":"2024-09-19T08:13:01.832908+0000","last_fullsized":"2024-09-19T08:13:01.832908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:59:49.004605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947899+0000","last_change":"2024-09-19T08:13:00.843421+0000","last_active":"2024-09-19T08:13:01.947899+0000","last_peered":"2024-09-19T08:13:01.947899+0000","last_clean":"2024-09-19T08:13:01.947899+0000","last_became_active":"2024-09-19T08:13:00.843109+0000","last_became_peered":"2024-09-19T08:13:00.843109+0000","last_unstale":"2024-09-19T08:13:01.947899+0000","last_undegraded":"2024-09-19T08:13:01.947899+0000","last_fullsized":"2024-09-19T08:13:01.947899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:50.522686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947846+0000","last_change":"2024-09-19T08:13:00.855215+0000","last_active":"2024-09-19T08:13:01.947846+0000","last_peered":"2024-09-19T08:13:01.947846+0000","last_clean":"2024-09-19T08:13:01.947846+0000","last_became_active":"2024-09-19T08:13:00.854878+0000","last_became_peered":"2024-09-19T08:13:00.854878+0000","last_unstale":"2024-09-19T08:13:01.947846+0000","last_undegraded":"2024-09-19T08:13:01.947846+0000","last_fullsized":"2024-09-19T08:13:01.947846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:48.216066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836201+0000","last_change":"2024-09-19T08:13:00.856995+0000","last_active":"2024-09-19T08:13:01.836201+0000","last_peered":"2024-09-19T08:13:01.836201+0000","last_clean":"2024-09-19T08:13:01.836201+0000","last_became_active":"2024-09-19T08:13:00.856645+0000","last_became_peered":"2024-09-19T08:13:00.856645+0000","last_unstale":"2024-09-19T08:13:01.836201+0000","last_undegraded":"2024-09-19T08:13:01.836201+0000","last_fullsized":"2024-09-19T08:13:01.836201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:20.699293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947354+0000","last_change":"2024-09-19T08:13:00.853362+0000","last_active":"2024-09-19T08:13:01.947354+0000","last_peered":"2024-09-19T08:13:01.947354+0000","last_clean":"2024-09-19T08:13:01.947354+0000","last_became_active":"2024-09-19T08:13:00.853046+0000","last_became_peered":"2024-09-19T08:13:00.853046+0000","last_unstale":"2024-09-19T08:13:01.947354+0000","last_undegraded":"2024-09-19T08:13:01.947354+0000","last_fullsized":"2024-09-19T08:13:01.947354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947080+0000","last_change":"2024-09-19T08:13:00.845175+0000","last_active":"2024-09-19T08:13:01.947080+0000","last_peered":"2024-09-19T08:13:01.947080+0000","last_clean":"2024-09-19T08:13:01.947080+0000","last_became_active":"2024-09-19T08:13:00.844857+0000","last_became_peered":"2024-09-19T08:13:00.844857+0000","last_unstale":"2024-09-19T08:13:01.947080+0000","last_undegraded":"2024-09-19T08:13:01.947080+0000","last_fullsized":"2024-09-19T08:13:01.947080+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:40.818956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947507+0000","last_change":"2024-09-19T08:13:00.847769+0000","last_active":"2024-09-19T08:13:01.947507+0000","last_peered":"2024-09-19T08:13:01.947507+0000","last_clean":"2024-09-19T08:13:01.947507+0000","last_became_active":"2024-09-19T08:13:00.847454+0000","last_became_peered":"2024-09-19T08:13:00.847454+0000","last_unstale":"2024-09-19T08:13:01.947507+0000","last_undegraded":"2024-09-19T08:13:01.947507+0000","last_fullsized":"2024-09-19T08:13:01.947507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.583051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.833005+0000","last_change":"2024-09-19T08:13:00.859882+0000","last_active":"2024-09-19T08:13:01.833005+0000","last_peered":"2024-09-19T08:13:01.833005+0000","last_clean":"2024-09-19T08:13:01.833005+0000","last_became_active":"2024-09-19T08:13:00.859491+0000","last_became_peered":"2024-09-19T08:13:00.859491+0000","last_unstale":"2024-09-19T08:13:01.833005+0000","last_undegraded":"2024-09-19T08:13:01.833005+0000","last_fullsized":"2024-09-19T08:13:01.833005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:13.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948381+0000","last_change":"2024-09-19T08:13:00.845584+0000","last_active":"2024-09-19T08:13:01.948381+0000","last_peered":"2024-09-19T08:13:01.948381+0000","last_clean":"2024-09-19T08:13:01.948381+0000","last_became_active":"2024-09-19T08:13:00.845166+0000","last_became_peered":"2024-09-19T08:13:00.845166+0000","last_unstale":"2024-09-19T08:13:01.948381+0000","last_undegraded":"2024-09-19T08:13:01.948381+0000","last_fullsized":"2024-09-19T08:13:01.948381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:02.705793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947956+0000","last_change":"2024-09-19T08:13:00.838069+0000","last_active":"2024-09-19T08:13:01.947956+0000","last_peered":"2024-09-19T08:13:01.947956+0000","last_clean":"2024-09-19T08:13:01.947956+0000","last_became_active":"2024-09-19T08:13:00.837736+0000","last_became_peered":"2024-09-19T08:13:00.837736+0000","last_unstale":"2024-09-19T08:13:01.947956+0000","last_undegraded":"2024-09-19T08:13:01.947956+0000","last_fullsized":"2024-09-19T08:13:01.947956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:01.388290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948680+0000","last_change":"2024-09-19T08:13:00.841507+0000","last_active":"2024-09-19T08:13:01.948680+0000","last_peered":"2024-09-19T08:13:01.948680+0000","last_clean":"2024-09-19T08:13:01.948680+0000","last_became_active":"2024-09-19T08:13:00.841161+0000","last_became_peered":"2024-09-19T08:13:00.841161+0000","last_unstale":"2024-09-19T08:13:01.948680+0000","last_undegraded":"2024-09-19T08:13:01.948680+0000","last_fullsized":"2024-09-19T08:13:01.948680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:41.862290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.832504+0000","last_change":"2024-09-19T08:13:00.847657+0000","last_active":"2024-09-19T08:13:01.832504+0000","last_peered":"2024-09-19T08:13:01.832504+0000","last_clean":"2024-09-19T08:13:01.832504+0000","last_became_active":"2024-09-19T08:13:00.846849+0000","last_became_peered":"2024-09-19T08:13:00.846849+0000","last_unstale":"2024-09-19T08:13:01.832504+0000","last_undegraded":"2024-09-19T08:13:01.832504+0000","last_fullsized":"2024-09-19T08:13:01.832504+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:35.219913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.832650+0000","last_change":"2024-09-19T08:13:00.845971+0000","last_active":"2024-09-19T08:13:01.832650+0000","last_peered":"2024-09-19T08:13:01.832650+0000","last_clean":"2024-09-19T08:13:01.832650+0000","last_became_active":"2024-09-19T08:13:00.845626+0000","last_became_peered":"2024-09-19T08:13:00.845626+0000","last_unstale":"2024-09-19T08:13:01.832650+0000","last_undegraded":"2024-09-19T08:13:01.832650+0000","last_fullsized":"2024-09-19T08:13:01.832650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:36.945627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947938+0000","last_change":"2024-09-19T08:13:00.853851+0000","last_active":"2024-09-19T08:13:01.947938+0000","last_peered":"2024-09-19T08:13:01.947938+0000","last_clean":"2024-09-19T08:13:01.947938+0000","last_became_active":"2024-09-19T08:13:00.853189+0000","last_became_peered":"2024-09-19T08:13:00.853189+0000","last_unstale":"2024-09-19T08:13:01.947938+0000","last_undegraded":"2024-09-19T08:13:01.947938+0000","last_fullsized":"2024-09-19T08:13:01.947938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:04.812522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.946963+0000","last_change":"2024-09-19T08:13:00.848325+0000","last_active":"2024-09-19T08:13:01.946963+0000","last_peered":"2024-09-19T08:13:01.946963+0000","last_clean":"2024-09-19T08:13:01.946963+0000","last_became_active":"2024-09-19T08:13:00.848015+0000","last_became_peered":"2024-09-19T08:13:00.848015+0000","last_unstale":"2024-09-19T08:13:01.946963+0000","last_undegraded":"2024-09-19T08:13:01.946963+0000","last_fullsized":"2024-09-19T08:13:01.946963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:10.948451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:14:03.954679+0000","last_change":"2024-09-19T08:12:58.681134+0000","last_active":"2024-09-19T08:14:03.954679+0000","last_peered":"2024-09-19T08:14:03.954679+0000","last_clean":"2024-09-19T08:14:03.954679+0000","last_became_active":"2024-09-19T08:12:58.680764+0000","last_became_peered":"2024-09-19T08:12:58.680764+0000","last_unstale":"2024-09-19T08:14:03.954679+0000","last_undegraded":"2024-09-19T08:14:03.954679+0000","last_fullsized":"2024-09-19T08:14:03.954679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:24.939263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947299+0000","last_change":"2024-09-19T08:13:00.844319+0000","last_active":"2024-09-19T08:13:01.947299+0000","last_peered":"2024-09-19T08:13:01.947299+0000","last_clean":"2024-09-19T08:13:01.947299+0000","last_became_active":"2024-09-19T08:13:00.843994+0000","last_became_peered":"2024-09-19T08:13:00.843994+0000","last_unstale":"2024-09-19T08:13:01.947299+0000","last_undegraded":"2024-09-19T08:13:01.947299+0000","last_fullsized":"2024-09-19T08:13:01.947299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:46.937466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.880088+0000","last_change":"2024-09-19T08:12:58.676541+0000","last_active":"2024-09-19T08:13:10.880088+0000","last_peered":"2024-09-19T08:13:10.880088+0000","last_clean":"2024-09-19T08:13:10.880088+0000","last_became_active":"2024-09-19T08:12:58.675988+0000","last_became_peered":"2024-09-19T08:12:58.675988+0000","last_unstale":"2024-09-19T08:13:10.880088+0000","last_undegraded":"2024-09-19T08:13:10.880088+0000","last_fullsized":"2024-09-19T08:13:10.880088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:01.434986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.880054+0000","last_change":"2024-09-19T08:12:58.677510+0000","last_active":"2024-09-19T08:13:10.880054+0000","last_peered":"2024-09-19T08:13:10.880054+0000","last_clean":"2024-09-19T08:13:10.880054+0000","last_became_active":"2024-09-19T08:12:58.677200+0000","last_became_peered":"2024-09-19T08:12:58.677200+0000","last_unstale":"2024-09-19T08:13:10.880054+0000","last_undegraded":"2024-09-19T08:13:10.880054+0000","last_fullsized":"2024-09-19T08:13:10.880054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:36.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947353+0000","last_change":"2024-09-19T08:13:00.853430+0000","last_active":"2024-09-19T08:13:01.947353+0000","last_peered":"2024-09-19T08:13:01.947353+0000","last_clean":"2024-09-19T08:13:01.947353+0000","last_became_active":"2024-09-19T08:13:00.853096+0000","last_became_peered":"2024-09-19T08:13:00.853096+0000","last_unstale":"2024-09-19T08:13:01.947353+0000","last_undegraded":"2024-09-19T08:13:01.947353+0000","last_fullsized":"2024-09-19T08:13:01.947353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:07.147021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947748+0000","last_change":"2024-09-19T08:12:58.680081+0000","last_active":"2024-09-19T08:13:01.947748+0000","last_peered":"2024-09-19T08:13:01.947748+0000","last_clean":"2024-09-19T08:13:01.947748+0000","last_became_active":"2024-09-19T08:12:58.679704+0000","last_became_peered":"2024-09-19T08:12:58.679704+0000","last_unstale":"2024-09-19T08:13:01.947748+0000","last_undegraded":"2024-09-19T08:13:01.947748+0000","last_fullsized":"2024-09-19T08:13:01.947748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:32.942586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726249+0000","last_change":"2024-09-19T08:13:00.849394+0000","last_active":"2024-09-19T08:14:06.726249+0000","last_peered":"2024-09-19T08:14:06.726249+0000","last_clean":"2024-09-19T08:14:06.726249+0000","last_became_active":"2024-09-19T08:13:00.849051+0000","last_became_peered":"2024-09-19T08:13:00.849051+0000","last_unstale":"2024-09-19T08:14:06.726249+0000","last_undegraded":"2024-09-19T08:14:06.726249+0000","last_fullsized":"2024-09-19T08:14:06.726249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:23.012928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":75,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:14:03.953993+0000","last_change":"2024-09-19T08:12:58.676121+0000","last_active":"2024-09-19T08:14:03.953993+0000","last_peered":"2024-09-19T08:14:03.953993+0000","last_clean":"2024-09-19T08:14:03.953993+0000","last_became_active":"2024-09-19T08:12:58.675693+0000","last_became_peered":"2024-09-19T08:12:58.675693+0000","last_unstale":"2024-09-19T08:14:03.953993+0000","last_undegraded":"2024-09-19T08:14:03.953993+0000","last_fullsized":"2024-09-19T08:14:03.953993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:34.882190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836478+0000","last_change":"2024-09-19T08:13:00.858735+0000","last_active":"2024-09-19T08:13:01.836478+0000","last_peered":"2024-09-19T08:13:01.836478+0000","last_clean":"2024-09-19T08:13:01.836478+0000","last_became_active":"2024-09-19T08:13:00.858218+0000","last_became_peered":"2024-09-19T08:13:00.858218+0000","last_unstale":"2024-09-19T08:13:01.836478+0000","last_undegraded":"2024-09-19T08:13:01.836478+0000","last_fullsized":"2024-09-19T08:13:01.836478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:33.356810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872605+0000","last_change":"2024-09-19T08:12:58.679624+0000","last_active":"2024-09-19T08:13:09.872605+0000","last_peered":"2024-09-19T08:13:09.872605+0000","last_clean":"2024-09-19T08:13:09.872605+0000","last_became_active":"2024-09-19T08:12:58.679057+0000","last_became_peered":"2024-09-19T08:12:58.679057+0000","last_unstale":"2024-09-19T08:13:09.872605+0000","last_undegraded":"2024-09-19T08:13:09.872605+0000","last_fullsized":"2024-09-19T08:13:09.872605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:12.391663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948868+0000","last_change":"2024-09-19T08:13:00.843852+0000","last_active":"2024-09-19T08:13:01.948868+0000","last_peered":"2024-09-19T08:13:01.948868+0000","last_clean":"2024-09-19T08:13:01.948868+0000","last_became_active":"2024-09-19T08:13:00.843497+0000","last_became_peered":"2024-09-19T08:13:00.843497+0000","last_unstale":"2024-09-19T08:13:01.948868+0000","last_undegraded":"2024-09-19T08:13:01.948868+0000","last_fullsized":"2024-09-19T08:13:01.948868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:26.380024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.873411+0000","last_change":"2024-09-19T08:12:58.717225+0000","last_active":"2024-09-19T08:13:09.873411+0000","last_peered":"2024-09-19T08:13:09.873411+0000","last_clean":"2024-09-19T08:13:09.873411+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:13:09.873411+0000","last_undegraded":"2024-09-19T08:13:09.873411+0000","last_fullsized":"2024-09-19T08:13:09.873411+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:43.201907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836422+0000","last_change":"2024-09-19T08:13:00.845613+0000","last_active":"2024-09-19T08:13:01.836422+0000","last_peered":"2024-09-19T08:13:01.836422+0000","last_clean":"2024-09-19T08:13:01.836422+0000","last_became_active":"2024-09-19T08:13:00.845260+0000","last_became_peered":"2024-09-19T08:13:00.845260+0000","last_unstale":"2024-09-19T08:13:01.836422+0000","last_undegraded":"2024-09-19T08:13:01.836422+0000","last_fullsized":"2024-09-19T08:13:01.836422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:03:47.033972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'29","reported_seq":164,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:14:04.955077+0000","last_change":"2024-09-19T08:12:58.677308+0000","last_active":"2024-09-19T08:14:04.955077+0000","last_peered":"2024-09-19T08:14:04.955077+0000","last_clean":"2024-09-19T08:14:04.955077+0000","last_became_active":"2024-09-19T08:12:58.676925+0000","last_became_peered":"2024-09-19T08:12:58.676925+0000","last_unstale":"2024-09-19T08:14:04.955077+0000","last_undegraded":"2024-09-19T08:14:04.955077+0000","last_fullsized":"2024-09-19T08:14:04.955077+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:50.873656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61172,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947397+0000","last_change":"2024-09-19T08:13:00.855603+0000","last_active":"2024-09-19T08:13:01.947397+0000","last_peered":"2024-09-19T08:13:01.947397+0000","last_clean":"2024-09-19T08:13:01.947397+0000","last_became_active":"2024-09-19T08:13:00.855241+0000","last_became_peered":"2024-09-19T08:13:00.855241+0000","last_unstale":"2024-09-19T08:13:01.947397+0000","last_undegraded":"2024-09-19T08:13:01.947397+0000","last_fullsized":"2024-09-19T08:13:01.947397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:28.811847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.881619+0000","last_change":"2024-09-19T08:12:58.675897+0000","last_active":"2024-09-19T08:13:10.881619+0000","last_peered":"2024-09-19T08:13:10.881619+0000","last_clean":"2024-09-19T08:13:10.881619+0000","last_became_active":"2024-09-19T08:12:58.675525+0000","last_became_peered":"2024-09-19T08:12:58.675525+0000","last_unstale":"2024-09-19T08:13:10.881619+0000","last_undegraded":"2024-09-19T08:13:10.881619+0000","last_fullsized":"2024-09-19T08:13:10.881619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:54.518064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947169+0000","last_change":"2024-09-19T08:13:00.856420+0000","last_active":"2024-09-19T08:13:01.947169+0000","last_peered":"2024-09-19T08:13:01.947169+0000","last_clean":"2024-09-19T08:13:01.947169+0000","last_became_active":"2024-09-19T08:13:00.856100+0000","last_became_peered":"2024-09-19T08:13:00.856100+0000","last_unstale":"2024-09-19T08:13:01.947169+0000","last_undegraded":"2024-09-19T08:13:01.947169+0000","last_fullsized":"2024-09-19T08:13:01.947169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:58.684930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872838+0000","last_change":"2024-09-19T08:12:58.680778+0000","last_active":"2024-09-19T08:13:09.872838+0000","last_peered":"2024-09-19T08:13:09.872838+0000","last_clean":"2024-09-19T08:13:09.872838+0000","last_became_active":"2024-09-19T08:12:58.680364+0000","last_became_peered":"2024-09-19T08:12:58.680364+0000","last_unstale":"2024-09-19T08:13:09.872838+0000","last_undegraded":"2024-09-19T08:13:09.872838+0000","last_fullsized":"2024-09-19T08:13:09.872838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:10.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100651+0000","last_change":"2024-09-19T08:13:00.853415+0000","last_active":"2024-09-19T08:13:02.100651+0000","last_peered":"2024-09-19T08:13:02.100651+0000","last_clean":"2024-09-19T08:13:02.100651+0000","last_became_active":"2024-09-19T08:13:00.853101+0000","last_became_peered":"2024-09-19T08:13:00.853101+0000","last_unstale":"2024-09-19T08:13:02.100651+0000","last_undegraded":"2024-09-19T08:13:02.100651+0000","last_fullsized":"2024-09-19T08:13:02.100651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:42.375394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":78,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:14:02.952897+0000","last_change":"2024-09-19T08:12:58.680312+0000","last_active":"2024-09-19T08:14:02.952897+0000","last_peered":"2024-09-19T08:14:02.952897+0000","last_clean":"2024-09-19T08:14:02.952897+0000","last_became_active":"2024-09-19T08:12:58.679870+0000","last_became_peered":"2024-09-19T08:12:58.679870+0000","last_unstale":"2024-09-19T08:14:02.952897+0000","last_undegraded":"2024-09-19T08:14:02.952897+0000","last_fullsized":"2024-09-19T08:14:02.952897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:42.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100329+0000","last_change":"2024-09-19T08:13:00.853920+0000","last_active":"2024-09-19T08:13:02.100329+0000","last_peered":"2024-09-19T08:13:02.100329+0000","last_clean":"2024-09-19T08:13:02.100329+0000","last_became_active":"2024-09-19T08:13:00.853594+0000","last_became_peered":"2024-09-19T08:13:00.853594+0000","last_unstale":"2024-09-19T08:13:02.100329+0000","last_undegraded":"2024-09-19T08:13:02.100329+0000","last_fullsized":"2024-09-19T08:13:02.100329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:31.746133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.882028+0000","last_change":"2024-09-19T08:12:58.717487+0000","last_active":"2024-09-19T08:13:10.882028+0000","last_peered":"2024-09-19T08:13:10.882028+0000","last_clean":"2024-09-19T08:13:10.882028+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:13:10.882028+0000","last_undegraded":"2024-09-19T08:13:10.882028+0000","last_fullsized":"2024-09-19T08:13:10.882028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:38.506051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947060+0000","last_change":"2024-09-19T08:13:00.847061+0000","last_active":"2024-09-19T08:13:01.947060+0000","last_peered":"2024-09-19T08:13:01.947060+0000","last_clean":"2024-09-19T08:13:01.947060+0000","last_became_active":"2024-09-19T08:13:00.846742+0000","last_became_peered":"2024-09-19T08:13:00.846742+0000","last_unstale":"2024-09-19T08:13:01.947060+0000","last_undegraded":"2024-09-19T08:13:01.947060+0000","last_fullsized":"2024-09-19T08:13:01.947060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:21.785585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:11.877000+0000","last_change":"2024-09-19T08:12:58.679367+0000","last_active":"2024-09-19T08:13:11.877000+0000","last_peered":"2024-09-19T08:13:11.877000+0000","last_clean":"2024-09-19T08:13:11.877000+0000","last_became_active":"2024-09-19T08:12:58.679035+0000","last_became_peered":"2024-09-19T08:12:58.679035+0000","last_unstale":"2024-09-19T08:13:11.877000+0000","last_undegraded":"2024-09-19T08:13:11.877000+0000","last_fullsized":"2024-09-19T08:13:11.877000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:25.633215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948364+0000","last_change":"2024-09-19T08:13:00.844940+0000","last_active":"2024-09-19T08:13:01.948364+0000","last_peered":"2024-09-19T08:13:01.948364+0000","last_clean":"2024-09-19T08:13:01.948364+0000","last_became_active":"2024-09-19T08:13:00.844454+0000","last_became_peered":"2024-09-19T08:13:00.844454+0000","last_unstale":"2024-09-19T08:13:01.948364+0000","last_undegraded":"2024-09-19T08:13:01.948364+0000","last_fullsized":"2024-09-19T08:13:01.948364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:15.577624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.272636+0000","last_change":"2024-09-19T08:12:58.673301+0000","last_active":"2024-09-19T08:13:02.272636+0000","last_peered":"2024-09-19T08:13:02.272636+0000","last_clean":"2024-09-19T08:13:02.272636+0000","last_became_active":"2024-09-19T08:12:58.672981+0000","last_became_peered":"2024-09-19T08:12:58.672981+0000","last_unstale":"2024-09-19T08:13:02.272636+0000","last_undegraded":"2024-09-19T08:13:02.272636+0000","last_fullsized":"2024-09-19T08:13:02.272636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:03.963712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100834+0000","last_change":"2024-09-19T08:13:00.856620+0000","last_active":"2024-09-19T08:13:02.100834+0000","last_peered":"2024-09-19T08:13:02.100834+0000","last_clean":"2024-09-19T08:13:02.100834+0000","last_became_active":"2024-09-19T08:13:00.856310+0000","last_became_peered":"2024-09-19T08:13:00.856310+0000","last_unstale":"2024-09-19T08:13:02.100834+0000","last_undegraded":"2024-09-19T08:13:02.100834+0000","last_fullsized":"2024-09-19T08:13:02.100834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:16.918768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.271923+0000","last_change":"2024-09-19T08:12:58.672282+0000","last_active":"2024-09-19T08:13:02.271923+0000","last_peered":"2024-09-19T08:13:02.271923+0000","last_clean":"2024-09-19T08:13:02.271923+0000","last_became_active":"2024-09-19T08:12:58.671891+0000","last_became_peered":"2024-09-19T08:12:58.671891+0000","last_unstale":"2024-09-19T08:13:02.271923+0000","last_undegraded":"2024-09-19T08:13:02.271923+0000","last_fullsized":"2024-09-19T08:13:02.271923+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:41.731571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100814+0000","last_change":"2024-09-19T08:13:00.848107+0000","last_active":"2024-09-19T08:13:02.100814+0000","last_peered":"2024-09-19T08:13:02.100814+0000","last_clean":"2024-09-19T08:13:02.100814+0000","last_became_active":"2024-09-19T08:13:00.847632+0000","last_became_peered":"2024-09-19T08:13:00.847632+0000","last_unstale":"2024-09-19T08:13:02.100814+0000","last_undegraded":"2024-09-19T08:13:02.100814+0000","last_fullsized":"2024-09-19T08:13:02.100814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.491122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.880827+0000","last_change":"2024-09-19T08:12:59.179200+0000","last_active":"2024-09-19T08:13:10.880827+0000","last_peered":"2024-09-19T08:13:10.880827+0000","last_clean":"2024-09-19T08:13:10.880827+0000","last_became_active":"2024-09-19T08:12:59.178762+0000","last_became_peered":"2024-09-19T08:12:59.178762+0000","last_unstale":"2024-09-19T08:13:10.880827+0000","last_undegraded":"2024-09-19T08:13:10.880827+0000","last_fullsized":"2024-09-19T08:13:10.880827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:08.424106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100825+0000","last_change":"2024-09-19T08:13:00.855870+0000","last_active":"2024-09-19T08:13:02.100825+0000","last_peered":"2024-09-19T08:13:02.100825+0000","last_clean":"2024-09-19T08:13:02.100825+0000","last_became_active":"2024-09-19T08:13:00.855538+0000","last_became_peered":"2024-09-19T08:13:00.855538+0000","last_unstale":"2024-09-19T08:13:02.100825+0000","last_undegraded":"2024-09-19T08:13:02.100825+0000","last_fullsized":"2024-09-19T08:13:02.100825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100488+0000","last_change":"2024-09-19T08:11:48.661333+0000","last_active":"2024-09-19T08:13:02.100488+0000","last_peered":"2024-09-19T08:13:02.100488+0000","last_clean":"2024-09-19T08:13:02.100488+0000","last_became_active":"2024-09-19T08:11:48.660873+0000","last_became_peered":"2024-09-19T08:11:48.660873+0000","last_unstale":"2024-09-19T08:13:02.100488+0000","last_undegraded":"2024-09-19T08:13:02.100488+0000","last_fullsized":"2024-09-19T08:13:02.100488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_clean_scrub_stamp":"2024-09-19T08:08:25.372155+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:19.401292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:11.881883+0000","last_change":"2024-09-19T08:12:58.675237+0000","last_active":"2024-09-19T08:13:11.881883+0000","last_peered":"2024-09-19T08:13:11.881883+0000","last_clean":"2024-09-19T08:13:11.881883+0000","last_became_active":"2024-09-19T08:12:58.674910+0000","last_became_peered":"2024-09-19T08:12:58.674910+0000","last_unstale":"2024-09-19T08:13:11.881883+0000","last_undegraded":"2024-09-19T08:13:11.881883+0000","last_fullsized":"2024-09-19T08:13:11.881883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:47.038328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947516+0000","last_change":"2024-09-19T08:13:00.846754+0000","last_active":"2024-09-19T08:13:01.947516+0000","last_peered":"2024-09-19T08:13:01.947516+0000","last_clean":"2024-09-19T08:13:01.947516+0000","last_became_active":"2024-09-19T08:13:00.846452+0000","last_became_peered":"2024-09-19T08:13:00.846452+0000","last_unstale":"2024-09-19T08:13:01.947516+0000","last_undegraded":"2024-09-19T08:13:01.947516+0000","last_fullsized":"2024-09-19T08:13:01.947516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:22.929689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.881292+0000","last_change":"2024-09-19T08:12:58.678958+0000","last_active":"2024-09-19T08:13:10.881292+0000","last_peered":"2024-09-19T08:13:10.881292+0000","last_clean":"2024-09-19T08:13:10.881292+0000","last_became_active":"2024-09-19T08:12:58.678592+0000","last_became_peered":"2024-09-19T08:12:58.678592+0000","last_unstale":"2024-09-19T08:13:10.881292+0000","last_undegraded":"2024-09-19T08:13:10.881292+0000","last_fullsized":"2024-09-19T08:13:10.881292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:58.374606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947087+0000","last_change":"2024-09-19T08:13:00.840973+0000","last_active":"2024-09-19T08:13:01.947087+0000","last_peered":"2024-09-19T08:13:01.947087+0000","last_clean":"2024-09-19T08:13:01.947087+0000","last_became_active":"2024-09-19T08:13:00.840619+0000","last_became_peered":"2024-09-19T08:13:00.840619+0000","last_unstale":"2024-09-19T08:13:01.947087+0000","last_undegraded":"2024-09-19T08:13:01.947087+0000","last_fullsized":"2024-09-19T08:13:01.947087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:14:03.953890+0000","last_change":"2024-09-19T08:12:58.715972+0000","last_active":"2024-09-19T08:14:03.953890+0000","last_peered":"2024-09-19T08:14:03.953890+0000","last_clean":"2024-09-19T08:14:03.953890+0000","last_became_active":"2024-09-19T08:12:58.715630+0000","last_became_peered":"2024-09-19T08:12:58.715630+0000","last_unstale":"2024-09-19T08:14:03.953890+0000","last_undegraded":"2024-09-19T08:14:03.953890+0000","last_fullsized":"2024-09-19T08:14:03.953890+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:35.637121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947439+0000","last_change":"2024-09-19T08:13:00.847023+0000","last_active":"2024-09-19T08:13:01.947439+0000","last_peered":"2024-09-19T08:13:01.947439+0000","last_clean":"2024-09-19T08:13:01.947439+0000","last_became_active":"2024-09-19T08:13:00.846387+0000","last_became_peered":"2024-09-19T08:13:00.846387+0000","last_unstale":"2024-09-19T08:13:01.947439+0000","last_undegraded":"2024-09-19T08:13:01.947439+0000","last_fullsized":"2024-09-19T08:13:01.947439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:12.187929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100987+0000","last_change":"2024-09-19T08:12:58.681062+0000","last_active":"2024-09-19T08:13:02.100987+0000","last_peered":"2024-09-19T08:13:02.100987+0000","last_clean":"2024-09-19T08:13:02.100987+0000","last_became_active":"2024-09-19T08:12:58.680524+0000","last_became_peered":"2024-09-19T08:12:58.680524+0000","last_unstale":"2024-09-19T08:13:02.100987+0000","last_undegraded":"2024-09-19T08:13:02.100987+0000","last_fullsized":"2024-09-19T08:13:02.100987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947809+0000","last_change":"2024-09-19T08:13:00.851523+0000","last_active":"2024-09-19T08:13:01.947809+0000","last_peered":"2024-09-19T08:13:01.947809+0000","last_clean":"2024-09-19T08:13:01.947809+0000","last_became_active":"2024-09-19T08:13:00.851068+0000","last_became_peered":"2024-09-19T08:13:00.851068+0000","last_unstale":"2024-09-19T08:13:01.947809+0000","last_undegraded":"2024-09-19T08:13:01.947809+0000","last_fullsized":"2024-09-19T08:13:01.947809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:44.905469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726509+0000","last_change":"2024-09-19T08:12:58.718234+0000","last_active":"2024-09-19T08:14:06.726509+0000","last_peered":"2024-09-19T08:14:06.726509+0000","last_clean":"2024-09-19T08:14:06.726509+0000","last_became_active":"2024-09-19T08:12:58.717840+0000","last_became_peered":"2024-09-19T08:12:58.717840+0000","last_unstale":"2024-09-19T08:14:06.726509+0000","last_undegraded":"2024-09-19T08:14:06.726509+0000","last_fullsized":"2024-09-19T08:14:06.726509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:05.283770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947784+0000","last_change":"2024-09-19T08:13:00.848305+0000","last_active":"2024-09-19T08:13:01.947784+0000","last_peered":"2024-09-19T08:13:01.947784+0000","last_clean":"2024-09-19T08:13:01.947784+0000","last_became_active":"2024-09-19T08:13:00.847981+0000","last_became_peered":"2024-09-19T08:13:00.847981+0000","last_unstale":"2024-09-19T08:13:01.947784+0000","last_undegraded":"2024-09-19T08:13:01.947784+0000","last_fullsized":"2024-09-19T08:13:01.947784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:04.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.871966+0000","last_change":"2024-09-19T08:12:58.715943+0000","last_active":"2024-09-19T08:13:09.871966+0000","last_peered":"2024-09-19T08:13:09.871966+0000","last_clean":"2024-09-19T08:13:09.871966+0000","last_became_active":"2024-09-19T08:12:58.715600+0000","last_became_peered":"2024-09-19T08:12:58.715600+0000","last_unstale":"2024-09-19T08:13:09.871966+0000","last_undegraded":"2024-09-19T08:13:09.871966+0000","last_fullsized":"2024-09-19T08:13:09.871966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:19.142271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948126+0000","last_change":"2024-09-19T08:13:00.840944+0000","last_active":"2024-09-19T08:13:01.948126+0000","last_peered":"2024-09-19T08:13:01.948126+0000","last_clean":"2024-09-19T08:13:01.948126+0000","last_became_active":"2024-09-19T08:13:00.840608+0000","last_became_peered":"2024-09-19T08:13:00.840608+0000","last_unstale":"2024-09-19T08:13:01.948126+0000","last_undegraded":"2024-09-19T08:13:01.948126+0000","last_fullsized":"2024-09-19T08:13:01.948126+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:25.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'11","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726648+0000","last_change":"2024-09-19T08:12:58.672929+0000","last_active":"2024-09-19T08:14:06.726648+0000","last_peered":"2024-09-19T08:14:06.726648+0000","last_clean":"2024-09-19T08:14:06.726648+0000","last_became_active":"2024-09-19T08:12:58.672487+0000","last_became_peered":"2024-09-19T08:12:58.672487+0000","last_unstale":"2024-09-19T08:14:06.726648+0000","last_undegraded":"2024-09-19T08:14:06.726648+0000","last_fullsized":"2024-09-19T08:14:06.726648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:33.042735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":2,"reported_epoch":84,"state":"peering","last_fresh":"2024-09-19T08:13:04.156818+0000","last_change":"2024-09-19T08:14:06.939357+0000","last_active":"2024-09-19T08:14:06.939357+0000","last_peered":"2024-09-19T08:13:04.156818+0000","last_clean":"2024-09-19T08:13:04.156818+0000","last_became_active":"2024-09-19T08:13:00.853159+0000","last_became_peered":"2024-09-19T08:13:00.853159+0000","last_unstale":"2024-09-19T08:13:04.156818+0000","last_undegraded":"2024-09-19T08:13:04.156818+0000","last_fullsized":"2024-09-19T08:13:04.156818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.869690+0000","last_change":"2024-09-19T08:12:59.179510+0000","last_active":"2024-09-19T08:13:09.869690+0000","last_peered":"2024-09-19T08:13:09.869690+0000","last_clean":"2024-09-19T08:13:09.869690+0000","last_became_active":"2024-09-19T08:12:59.179157+0000","last_became_peered":"2024-09-19T08:12:59.179157+0000","last_unstale":"2024-09-19T08:13:09.869690+0000","last_undegraded":"2024-09-19T08:13:09.869690+0000","last_fullsized":"2024-09-19T08:13:09.869690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:22.981289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948760+0000","last_change":"2024-09-19T08:13:00.841853+0000","last_active":"2024-09-19T08:13:01.948760+0000","last_peered":"2024-09-19T08:13:01.948760+0000","last_clean":"2024-09-19T08:13:01.948760+0000","last_became_active":"2024-09-19T08:13:00.841518+0000","last_became_peered":"2024-09-19T08:13:00.841518+0000","last_unstale":"2024-09-19T08:13:01.948760+0000","last_undegraded":"2024-09-19T08:13:01.948760+0000","last_fullsized":"2024-09-19T08:13:01.948760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:05.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872283+0000","last_change":"2024-09-19T08:12:58.716882+0000","last_active":"2024-09-19T08:13:09.872283+0000","last_peered":"2024-09-19T08:13:09.872283+0000","last_clean":"2024-09-19T08:13:09.872283+0000","last_became_active":"2024-09-19T08:12:58.716545+0000","last_became_peered":"2024-09-19T08:12:58.716545+0000","last_unstale":"2024-09-19T08:13:09.872283+0000","last_undegraded":"2024-09-19T08:13:09.872283+0000","last_fullsized":"2024-09-19T08:13:09.872283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:14.469109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.946790+0000","last_change":"2024-09-19T08:13:00.850484+0000","last_active":"2024-09-19T08:13:01.946790+0000","last_peered":"2024-09-19T08:13:01.946790+0000","last_clean":"2024-09-19T08:13:01.946790+0000","last_became_active":"2024-09-19T08:13:00.850049+0000","last_became_peered":"2024-09-19T08:13:00.850049+0000","last_unstale":"2024-09-19T08:13:01.946790+0000","last_undegraded":"2024-09-19T08:13:01.946790+0000","last_fullsized":"2024-09-19T08:13:01.946790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:24.424111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.870765+0000","last_change":"2024-09-19T08:12:58.669928+0000","last_active":"2024-09-19T08:13:09.870765+0000","last_peered":"2024-09-19T08:13:09.870765+0000","last_clean":"2024-09-19T08:13:09.870765+0000","last_became_active":"2024-09-19T08:12:58.669614+0000","last_became_peered":"2024-09-19T08:12:58.669614+0000","last_unstale":"2024-09-19T08:13:09.870765+0000","last_undegraded":"2024-09-19T08:13:09.870765+0000","last_fullsized":"2024-09-19T08:13:09.870765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:50.924779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948102+0000","last_change":"2024-09-19T08:13:00.853520+0000","last_active":"2024-09-19T08:13:01.948102+0000","last_peered":"2024-09-19T08:13:01.948102+0000","last_clean":"2024-09-19T08:13:01.948102+0000","last_became_active":"2024-09-19T08:13:00.853184+0000","last_became_peered":"2024-09-19T08:13:00.853184+0000","last_unstale":"2024-09-19T08:13:01.948102+0000","last_undegraded":"2024-09-19T08:13:01.948102+0000","last_fullsized":"2024-09-19T08:13:01.948102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:45.160439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.832515+0000","last_change":"2024-09-19T08:12:58.674359+0000","last_active":"2024-09-19T08:13:01.832515+0000","last_peered":"2024-09-19T08:13:01.832515+0000","last_clean":"2024-09-19T08:13:01.832515+0000","last_became_active":"2024-09-19T08:12:58.673890+0000","last_became_peered":"2024-09-19T08:12:58.673890+0000","last_unstale":"2024-09-19T08:13:01.832515+0000","last_undegraded":"2024-09-19T08:13:01.832515+0000","last_fullsized":"2024-09-19T08:13:01.832515+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:54.939959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947708+0000","last_change":"2024-09-19T08:13:00.851071+0000","last_active":"2024-09-19T08:13:01.947708+0000","last_peered":"2024-09-19T08:13:01.947708+0000","last_clean":"2024-09-19T08:13:01.947708+0000","last_became_active":"2024-09-19T08:13:00.850756+0000","last_became_peered":"2024-09-19T08:13:00.850756+0000","last_unstale":"2024-09-19T08:13:01.947708+0000","last_undegraded":"2024-09-19T08:13:01.947708+0000","last_fullsized":"2024-09-19T08:13:01.947708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:52.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872854+0000","last_change":"2024-09-19T08:12:58.681466+0000","last_active":"2024-09-19T08:13:09.872854+0000","last_peered":"2024-09-19T08:13:09.872854+0000","last_clean":"2024-09-19T08:13:09.872854+0000","last_became_active":"2024-09-19T08:12:58.681113+0000","last_became_peered":"2024-09-19T08:12:58.681113+0000","last_unstale":"2024-09-19T08:13:09.872854+0000","last_undegraded":"2024-09-19T08:13:09.872854+0000","last_fullsized":"2024-09-19T08:13:09.872854+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:49.610828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100583+0000","last_change":"2024-09-19T08:13:00.851272+0000","last_active":"2024-09-19T08:13:02.100583+0000","last_peered":"2024-09-19T08:13:02.100583+0000","last_clean":"2024-09-19T08:13:02.100583+0000","last_became_active":"2024-09-19T08:13:00.849098+0000","last_became_peered":"2024-09-19T08:13:00.849098+0000","last_unstale":"2024-09-19T08:13:02.100583+0000","last_undegraded":"2024-09-19T08:13:02.100583+0000","last_fullsized":"2024-09-19T08:13:02.100583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:05.643992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947777+0000","last_change":"2024-09-19T08:13:00.849945+0000","last_active":"2024-09-19T08:13:01.947777+0000","last_peered":"2024-09-19T08:13:01.947777+0000","last_clean":"2024-09-19T08:13:01.947777+0000","last_became_active":"2024-09-19T08:13:00.849591+0000","last_became_peered":"2024-09-19T08:13:00.849591+0000","last_unstale":"2024-09-19T08:13:01.947777+0000","last_undegraded":"2024-09-19T08:13:01.947777+0000","last_fullsized":"2024-09-19T08:13:01.947777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:27.434005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.272564+0000","last_change":"2024-09-19T08:12:58.674610+0000","last_active":"2024-09-19T08:13:02.272564+0000","last_peered":"2024-09-19T08:13:02.272564+0000","last_clean":"2024-09-19T08:13:02.272564+0000","last_became_active":"2024-09-19T08:12:58.674247+0000","last_became_peered":"2024-09-19T08:12:58.674247+0000","last_unstale":"2024-09-19T08:13:02.272564+0000","last_undegraded":"2024-09-19T08:13:02.272564+0000","last_fullsized":"2024-09-19T08:13:02.272564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:16.323574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836419+0000","last_change":"2024-09-19T08:13:00.858512+0000","last_active":"2024-09-19T08:13:01.836419+0000","last_peered":"2024-09-19T08:13:01.836419+0000","last_clean":"2024-09-19T08:13:01.836419+0000","last_became_active":"2024-09-19T08:13:00.857897+0000","last_became_peered":"2024-09-19T08:13:00.857897+0000","last_unstale":"2024-09-19T08:13:01.836419+0000","last_undegraded":"2024-09-19T08:13:01.836419+0000","last_fullsized":"2024-09-19T08:13:01.836419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.881153+0000","last_change":"2024-09-19T08:12:58.682207+0000","last_active":"2024-09-19T08:13:10.881153+0000","last_peered":"2024-09-19T08:13:10.881153+0000","last_clean":"2024-09-19T08:13:10.881153+0000","last_became_active":"2024-09-19T08:12:58.681842+0000","last_became_peered":"2024-09-19T08:12:58.681842+0000","last_unstale":"2024-09-19T08:13:10.881153+0000","last_undegraded":"2024-09-19T08:13:10.881153+0000","last_fullsized":"2024-09-19T08:13:10.881153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.825858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948835+0000","last_change":"2024-09-19T08:13:00.846748+0000","last_active":"2024-09-19T08:13:01.948835+0000","last_peered":"2024-09-19T08:13:01.948835+0000","last_clean":"2024-09-19T08:13:01.948835+0000","last_became_active":"2024-09-19T08:13:00.846368+0000","last_became_peered":"2024-09-19T08:13:00.846368+0000","last_unstale":"2024-09-19T08:13:01.948835+0000","last_undegraded":"2024-09-19T08:13:01.948835+0000","last_fullsized":"2024-09-19T08:13:01.948835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:59.526512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.271849+0000","last_change":"2024-09-19T08:12:58.716118+0000","last_active":"2024-09-19T08:13:02.271849+0000","last_peered":"2024-09-19T08:13:02.271849+0000","last_clean":"2024-09-19T08:13:02.271849+0000","last_became_active":"2024-09-19T08:12:58.715711+0000","last_became_peered":"2024-09-19T08:12:58.715711+0000","last_unstale":"2024-09-19T08:13:02.271849+0000","last_undegraded":"2024-09-19T08:13:02.271849+0000","last_fullsized":"2024-09-19T08:13:02.271849+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:29.439887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947595+0000","last_change":"2024-09-19T08:13:00.848729+0000","last_active":"2024-09-19T08:13:01.947595+0000","last_peered":"2024-09-19T08:13:01.947595+0000","last_clean":"2024-09-19T08:13:01.947595+0000","last_became_active":"2024-09-19T08:13:00.848470+0000","last_became_peered":"2024-09-19T08:13:00.848470+0000","last_unstale":"2024-09-19T08:13:01.947595+0000","last_undegraded":"2024-09-19T08:13:01.947595+0000","last_fullsized":"2024-09-19T08:13:01.947595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:30.653101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726774+0000","last_change":"2024-09-19T08:12:58.681654+0000","last_active":"2024-09-19T08:14:06.726774+0000","last_peered":"2024-09-19T08:14:06.726774+0000","last_clean":"2024-09-19T08:14:06.726774+0000","last_became_active":"2024-09-19T08:12:58.681284+0000","last_became_peered":"2024-09-19T08:12:58.681284+0000","last_unstale":"2024-09-19T08:14:06.726774+0000","last_undegraded":"2024-09-19T08:14:06.726774+0000","last_fullsized":"2024-09-19T08:14:06.726774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.429930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65568,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":352,"num_read_kb":245,"num_write":164,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251470,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481818,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":428609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710751,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":1081655,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874277,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":428845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":909312,"data_stored":428765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1021090,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495667,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":428234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":1020381,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":429064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":913408,"data_stored":428545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":490942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":489199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":429505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34755,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34755,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:14:09.396 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:14:09.668 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:10.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:09 smithi157 ceph-mon[35143]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:10.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:14:10.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:09 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:14:10.004 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:09 smithi022 ceph-mon[30963]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:10.004 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:14:10.004 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:09 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:14:10.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:09 smithi064 ceph-mon[34900]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:10.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:14:10.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:09 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:14:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.214+0000 7f0e00b24640 1 -- 172.21.15.22:0/2126012002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dfc1082a0 msgr2=0x7f0dfc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.214+0000 7f0e00b24640 1 --2- 172.21.15.22:0/2126012002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dfc1082a0 0x7f0dfc10ec40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009930 tx=0x7f0df402a420 comp rx=0 tx=0).stop 2024-09-19T08:14:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.214+0000 7f0e00b24640 1 -- 172.21.15.22:0/2126012002 shutdown_connections 2024-09-19T08:14:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.214+0000 7f0e00b24640 1 --2- 172.21.15.22:0/2126012002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dfc1082a0 0x7f0dfc10ec40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.214+0000 7f0e00b24640 1 --2- 172.21.15.22:0/2126012002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc1036e0 0x7f0dfc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.214+0000 7f0e00b24640 1 --2- 172.21.15.22:0/2126012002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dfc102d30 0x7f0dfc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.214+0000 7f0e00b24640 1 -- 172.21.15.22:0/2126012002 >> 172.21.15.22:0/2126012002 conn(0x7f0dfc0fc7d0 msgr2=0x7f0dfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.214+0000 7f0e00b24640 1 -- 172.21.15.22:0/2126012002 shutdown_connections 2024-09-19T08:14:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.214+0000 7f0e00b24640 1 -- 172.21.15.22:0/2126012002 wait complete. 2024-09-19T08:14:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.215+0000 7f0e00b24640 1 Processor -- start 2024-09-19T08:14:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.215+0000 7f0e00b24640 1 -- start start 2024-09-19T08:14:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.215+0000 7f0e00b24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dfc102d30 0x7f0dfc198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.215+0000 7f0e00b24640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc1036e0 0x7f0dfc198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.215+0000 7f0e00b24640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dfc1082a0 0x7f0dfc19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.215+0000 7f0e00b24640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc1120a0 con 0x7f0dfc1082a0 2024-09-19T08:14:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.215+0000 7f0e00b24640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc111f20 con 0x7f0dfc102d30 2024-09-19T08:14:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.215+0000 7f0e00b24640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc112220 con 0x7f0dfc1036e0 2024-09-19T08:14:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.215+0000 7f0dfad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dfc1082a0 0x7f0dfc19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.215+0000 7f0dfad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dfc1082a0 0x7f0dfc19ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60062/0 (socket says 172.21.15.22:60062) 2024-09-19T08:14:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.215+0000 7f0dfad9d640 1 -- 172.21.15.22:0/2188600721 learned_addr learned my addr 172.21.15.22:0/2188600721 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.215+0000 7f0df9d9b640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc1036e0 0x7f0dfc198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.216+0000 7f0dfa59c640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dfc102d30 0x7f0dfc198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.216+0000 7f0dfad9d640 1 -- 172.21.15.22:0/2188600721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc1036e0 msgr2=0x7f0dfc198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.216+0000 7f0dfad9d640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc1036e0 0x7f0dfc198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.216+0000 7f0dfad9d640 1 -- 172.21.15.22:0/2188600721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dfc102d30 msgr2=0x7f0dfc198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.216+0000 7f0dfad9d640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dfc102d30 0x7f0dfc198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.216+0000 7f0dfad9d640 1 -- 172.21.15.22:0/2188600721 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc19d4f0 con 0x7f0dfc1082a0 2024-09-19T08:14:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.216+0000 7f0dfa59c640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dfc102d30 0x7f0dfc198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:14:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.216+0000 7f0dfad9d640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dfc1082a0 0x7f0dfc19ce10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0df4005ab0 tx=0x7f0df40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.216+0000 7f0deb7fe640 1 -- 172.21.15.22:0/2188600721 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df403f030 con 0x7f0dfc1082a0 2024-09-19T08:14:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.216+0000 7f0deb7fe640 1 -- 172.21.15.22:0/2188600721 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0df4003ce0 con 0x7f0dfc1082a0 2024-09-19T08:14:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.216+0000 7f0e00b24640 1 -- 172.21.15.22:0/2188600721 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dfc19d780 con 0x7f0dfc1082a0 2024-09-19T08:14:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.216+0000 7f0deb7fe640 1 -- 172.21.15.22:0/2188600721 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4035070 con 0x7f0dfc1082a0 2024-09-19T08:14:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.216+0000 7f0e00b24640 1 -- 172.21.15.22:0/2188600721 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dfc1a50c0 con 0x7f0dfc1082a0 2024-09-19T08:14:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.217+0000 7f0e00b24640 1 -- 172.21.15.22:0/2188600721 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dfc103110 con 0x7f0dfc1082a0 2024-09-19T08:14:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.219+0000 7f0deb7fe640 1 -- 172.21.15.22:0/2188600721 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0df4042030 con 0x7f0dfc1082a0 2024-09-19T08:14:10.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.220+0000 7f0deb7fe640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f0de40a32d0 0x7f0de40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:10.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.220+0000 7f0dfa59c640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f0de40a32d0 0x7f0de40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:10.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.220+0000 7f0deb7fe640 1 -- 172.21.15.22:0/2188600721 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0df40f0340 con 0x7f0dfc1082a0 2024-09-19T08:14:10.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.220+0000 7f0dfa59c640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f0de40a32d0 0x7f0de40a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0dec005fd0 tx=0x7f0dec0094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:10.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.223+0000 7f0deb7fe640 1 -- 172.21.15.22:0/2188600721 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0df40bcec0 con 0x7f0dfc1082a0 2024-09-19T08:14:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.396+0000 7f0e00b24640 1 -- 172.21.15.22:0/2188600721 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0dfc107420 con 0x7f0de40a32d0 2024-09-19T08:14:10.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.402+0000 7f0deb7fe640 1 -- 172.21.15.22:0/2188600721 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258818 (secure 0 0 0) 0x7f0dfc107420 con 0x7f0de40a32d0 2024-09-19T08:14:10.405 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:10.406 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T08:14:10.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.405+0000 7f0e00b24640 1 -- 172.21.15.22:0/2188600721 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f0de40a32d0 msgr2=0x7f0de40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:10.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.405+0000 7f0e00b24640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f0de40a32d0 0x7f0de40a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0dec005fd0 tx=0x7f0dec0094d0 comp rx=0 tx=0).stop 2024-09-19T08:14:10.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.405+0000 7f0e00b24640 1 -- 172.21.15.22:0/2188600721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dfc1082a0 msgr2=0x7f0dfc19ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:10.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.405+0000 7f0e00b24640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dfc1082a0 0x7f0dfc19ce10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0df4005ab0 tx=0x7f0df40042f0 comp rx=0 tx=0).stop 2024-09-19T08:14:10.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.405+0000 7f0e00b24640 1 -- 172.21.15.22:0/2188600721 shutdown_connections 2024-09-19T08:14:10.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.405+0000 7f0e00b24640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f0de40a32d0 0x7f0de40a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:10.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.405+0000 7f0e00b24640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dfc1082a0 0x7f0dfc19ce10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:10.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.405+0000 7f0e00b24640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc1036e0 0x7f0dfc198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:10.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.405+0000 7f0e00b24640 1 --2- 172.21.15.22:0/2188600721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dfc102d30 0x7f0dfc198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:10.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.405+0000 7f0e00b24640 1 -- 172.21.15.22:0/2188600721 >> 172.21.15.22:0/2188600721 conn(0x7f0dfc0fc7d0 msgr2=0x7f0dfc105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:10.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.405+0000 7f0e00b24640 1 -- 172.21.15.22:0/2188600721 shutdown_connections 2024-09-19T08:14:10.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:10.405+0000 7f0e00b24640 1 -- 172.21.15.22:0/2188600721 wait complete. 2024-09-19T08:14:11.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:10 smithi157 ceph-mon[35143]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-19T08:14:11.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:10 smithi157 ceph-mon[35143]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-19T08:14:11.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:14:11.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:14:11.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:10 smithi157 ceph-mon[35143]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:14:11.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:10 smithi022 ceph-mon[30963]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-19T08:14:11.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:10 smithi022 ceph-mon[30963]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-19T08:14:11.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:14:11.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:14:11.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:10 smithi022 ceph-mon[30963]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:14:11.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:10 smithi064 ceph-mon[34900]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-19T08:14:11.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:10 smithi064 ceph-mon[34900]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-19T08:14:11.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:14:11.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:14:11.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:10 smithi064 ceph-mon[34900]: from='mgr.14150 172.21.15.22:0/3617362323' entity='mgr.x' 2024-09-19T08:14:11.136 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-19T08:14:08.778972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656170,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":488,"num_read_kb":364,"num_write":347,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381080,"kb_used_data":11028,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560744,"statfs":{"total":1151940427776,"available":1151550201856,"internally_reserved":0,"allocated":11292672,"data_stored":7130724,"data_compressed":117889,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002136"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948678+0000","last_change":"2024-09-19T08:13:00.853947+0000","last_active":"2024-09-19T08:13:01.948678+0000","last_peered":"2024-09-19T08:13:01.948678+0000","last_clean":"2024-09-19T08:13:01.948678+0000","last_became_active":"2024-09-19T08:13:00.852951+0000","last_became_peered":"2024-09-19T08:13:00.852951+0000","last_unstale":"2024-09-19T08:13:01.948678+0000","last_undegraded":"2024-09-19T08:13:01.948678+0000","last_fullsized":"2024-09-19T08:13:01.948678+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:25.786828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733811+0000","last_change":"2024-09-19T08:13:00.853389+0000","last_active":"2024-09-19T08:14:07.733811+0000","last_peered":"2024-09-19T08:14:07.733811+0000","last_clean":"2024-09-19T08:14:07.733811+0000","last_became_active":"2024-09-19T08:13:00.853102+0000","last_became_peered":"2024-09-19T08:13:00.853102+0000","last_unstale":"2024-09-19T08:14:07.733811+0000","last_undegraded":"2024-09-19T08:14:07.733811+0000","last_fullsized":"2024-09-19T08:14:07.733811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:42.773917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733742+0000","last_change":"2024-09-19T08:13:00.846544+0000","last_active":"2024-09-19T08:14:07.733742+0000","last_peered":"2024-09-19T08:14:07.733742+0000","last_clean":"2024-09-19T08:14:07.733742+0000","last_became_active":"2024-09-19T08:13:00.846229+0000","last_became_peered":"2024-09-19T08:13:00.846229+0000","last_unstale":"2024-09-19T08:14:07.733742+0000","last_undegraded":"2024-09-19T08:14:07.733742+0000","last_fullsized":"2024-09-19T08:14:07.733742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:43.471684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948555+0000","last_change":"2024-09-19T08:13:00.848149+0000","last_active":"2024-09-19T08:13:01.948555+0000","last_peered":"2024-09-19T08:13:01.948555+0000","last_clean":"2024-09-19T08:13:01.948555+0000","last_became_active":"2024-09-19T08:13:00.847744+0000","last_became_peered":"2024-09-19T08:13:00.847744+0000","last_unstale":"2024-09-19T08:13:01.948555+0000","last_undegraded":"2024-09-19T08:13:01.948555+0000","last_fullsized":"2024-09-19T08:13:01.948555+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:01.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948347+0000","last_change":"2024-09-19T08:13:00.841862+0000","last_active":"2024-09-19T08:13:01.948347+0000","last_peered":"2024-09-19T08:13:01.948347+0000","last_clean":"2024-09-19T08:13:01.948347+0000","last_became_active":"2024-09-19T08:13:00.841331+0000","last_became_peered":"2024-09-19T08:13:00.841331+0000","last_unstale":"2024-09-19T08:13:01.948347+0000","last_undegraded":"2024-09-19T08:13:01.948347+0000","last_fullsized":"2024-09-19T08:13:01.948347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:54.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947492+0000","last_change":"2024-09-19T08:13:00.849626+0000","last_active":"2024-09-19T08:13:01.947492+0000","last_peered":"2024-09-19T08:13:01.947492+0000","last_clean":"2024-09-19T08:13:01.947492+0000","last_became_active":"2024-09-19T08:13:00.849034+0000","last_became_peered":"2024-09-19T08:13:00.849034+0000","last_unstale":"2024-09-19T08:13:01.947492+0000","last_undegraded":"2024-09-19T08:13:01.947492+0000","last_fullsized":"2024-09-19T08:13:01.947492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:39.118978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100394+0000","last_change":"2024-09-19T08:13:00.847522+0000","last_active":"2024-09-19T08:13:02.100394+0000","last_peered":"2024-09-19T08:13:02.100394+0000","last_clean":"2024-09-19T08:13:02.100394+0000","last_became_active":"2024-09-19T08:13:00.847117+0000","last_became_peered":"2024-09-19T08:13:00.847117+0000","last_unstale":"2024-09-19T08:13:02.100394+0000","last_undegraded":"2024-09-19T08:13:02.100394+0000","last_fullsized":"2024-09-19T08:13:02.100394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.836385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947191+0000","last_change":"2024-09-19T08:13:00.849544+0000","last_active":"2024-09-19T08:13:01.947191+0000","last_peered":"2024-09-19T08:13:01.947191+0000","last_clean":"2024-09-19T08:13:01.947191+0000","last_became_active":"2024-09-19T08:13:00.849170+0000","last_became_peered":"2024-09-19T08:13:00.849170+0000","last_unstale":"2024-09-19T08:13:01.947191+0000","last_undegraded":"2024-09-19T08:13:01.947191+0000","last_fullsized":"2024-09-19T08:13:01.947191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:28.371965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836582+0000","last_change":"2024-09-19T08:13:00.856052+0000","last_active":"2024-09-19T08:13:01.836582+0000","last_peered":"2024-09-19T08:13:01.836582+0000","last_clean":"2024-09-19T08:13:01.836582+0000","last_became_active":"2024-09-19T08:13:00.855291+0000","last_became_peered":"2024-09-19T08:13:00.855291+0000","last_unstale":"2024-09-19T08:13:01.836582+0000","last_undegraded":"2024-09-19T08:13:01.836582+0000","last_fullsized":"2024-09-19T08:13:01.836582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:18.581867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948289+0000","last_change":"2024-09-19T08:13:00.850145+0000","last_active":"2024-09-19T08:13:01.948289+0000","last_peered":"2024-09-19T08:13:01.948289+0000","last_clean":"2024-09-19T08:13:01.948289+0000","last_became_active":"2024-09-19T08:13:00.849798+0000","last_became_peered":"2024-09-19T08:13:00.849798+0000","last_unstale":"2024-09-19T08:13:01.948289+0000","last_undegraded":"2024-09-19T08:13:01.948289+0000","last_fullsized":"2024-09-19T08:13:01.948289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:21.400000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100790+0000","last_change":"2024-09-19T08:13:00.859153+0000","last_active":"2024-09-19T08:13:02.100790+0000","last_peered":"2024-09-19T08:13:02.100790+0000","last_clean":"2024-09-19T08:13:02.100790+0000","last_became_active":"2024-09-19T08:13:00.858473+0000","last_became_peered":"2024-09-19T08:13:00.858473+0000","last_unstale":"2024-09-19T08:13:02.100790+0000","last_undegraded":"2024-09-19T08:13:02.100790+0000","last_fullsized":"2024-09-19T08:13:02.100790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:07.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948660+0000","last_change":"2024-09-19T08:13:00.850797+0000","last_active":"2024-09-19T08:13:01.948660+0000","last_peered":"2024-09-19T08:13:01.948660+0000","last_clean":"2024-09-19T08:13:01.948660+0000","last_became_active":"2024-09-19T08:13:00.850393+0000","last_became_peered":"2024-09-19T08:13:00.850393+0000","last_unstale":"2024-09-19T08:13:01.948660+0000","last_undegraded":"2024-09-19T08:13:01.948660+0000","last_fullsized":"2024-09-19T08:13:01.948660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:59.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947687+0000","last_change":"2024-09-19T08:13:00.850835+0000","last_active":"2024-09-19T08:13:01.947687+0000","last_peered":"2024-09-19T08:13:01.947687+0000","last_clean":"2024-09-19T08:13:01.947687+0000","last_became_active":"2024-09-19T08:13:00.850518+0000","last_became_peered":"2024-09-19T08:13:00.850518+0000","last_unstale":"2024-09-19T08:13:01.947687+0000","last_undegraded":"2024-09-19T08:13:01.947687+0000","last_fullsized":"2024-09-19T08:13:01.947687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.178530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733525+0000","last_change":"2024-09-19T08:13:00.853934+0000","last_active":"2024-09-19T08:14:07.733525+0000","last_peered":"2024-09-19T08:14:07.733525+0000","last_clean":"2024-09-19T08:14:07.733525+0000","last_became_active":"2024-09-19T08:13:00.853604+0000","last_became_peered":"2024-09-19T08:13:00.853604+0000","last_unstale":"2024-09-19T08:14:07.733525+0000","last_undegraded":"2024-09-19T08:14:07.733525+0000","last_fullsized":"2024-09-19T08:14:07.733525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:43.622526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.741370+0000","last_change":"2024-09-19T08:14:07.741370+0000","last_active":"2024-09-19T08:14:07.741370+0000","last_peered":"2024-09-19T08:14:07.741370+0000","last_clean":"2024-09-19T08:14:07.741370+0000","last_became_active":"2024-09-19T08:14:07.741116+0000","last_became_peered":"2024-09-19T08:14:07.741116+0000","last_unstale":"2024-09-19T08:14:07.741370+0000","last_undegraded":"2024-09-19T08:14:07.741370+0000","last_fullsized":"2024-09-19T08:14:07.741370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:45.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836019+0000","last_change":"2024-09-19T08:13:00.850266+0000","last_active":"2024-09-19T08:13:01.836019+0000","last_peered":"2024-09-19T08:13:01.836019+0000","last_clean":"2024-09-19T08:13:01.836019+0000","last_became_active":"2024-09-19T08:13:00.849928+0000","last_became_peered":"2024-09-19T08:13:00.849928+0000","last_unstale":"2024-09-19T08:13:01.836019+0000","last_undegraded":"2024-09-19T08:13:01.836019+0000","last_fullsized":"2024-09-19T08:13:01.836019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:38.231331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948706+0000","last_change":"2024-09-19T08:13:00.852551+0000","last_active":"2024-09-19T08:13:01.948706+0000","last_peered":"2024-09-19T08:13:01.948706+0000","last_clean":"2024-09-19T08:13:01.948706+0000","last_became_active":"2024-09-19T08:13:00.852225+0000","last_became_peered":"2024-09-19T08:13:00.852225+0000","last_unstale":"2024-09-19T08:13:01.948706+0000","last_undegraded":"2024-09-19T08:13:01.948706+0000","last_fullsized":"2024-09-19T08:13:01.948706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:33.115108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726544+0000","last_change":"2024-09-19T08:13:00.854033+0000","last_active":"2024-09-19T08:14:06.726544+0000","last_peered":"2024-09-19T08:14:06.726544+0000","last_clean":"2024-09-19T08:14:06.726544+0000","last_became_active":"2024-09-19T08:13:00.853660+0000","last_became_peered":"2024-09-19T08:13:00.853660+0000","last_unstale":"2024-09-19T08:14:06.726544+0000","last_undegraded":"2024-09-19T08:14:06.726544+0000","last_fullsized":"2024-09-19T08:14:06.726544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:35.602420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732530+0000","last_change":"2024-09-19T08:13:00.850658+0000","last_active":"2024-09-19T08:14:07.732530+0000","last_peered":"2024-09-19T08:14:07.732530+0000","last_clean":"2024-09-19T08:14:07.732530+0000","last_became_active":"2024-09-19T08:13:00.850324+0000","last_became_peered":"2024-09-19T08:13:00.850324+0000","last_unstale":"2024-09-19T08:14:07.732530+0000","last_undegraded":"2024-09-19T08:14:07.732530+0000","last_fullsized":"2024-09-19T08:14:07.732530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:59:49.004605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947899+0000","last_change":"2024-09-19T08:13:00.843421+0000","last_active":"2024-09-19T08:13:01.947899+0000","last_peered":"2024-09-19T08:13:01.947899+0000","last_clean":"2024-09-19T08:13:01.947899+0000","last_became_active":"2024-09-19T08:13:00.843109+0000","last_became_peered":"2024-09-19T08:13:00.843109+0000","last_unstale":"2024-09-19T08:13:01.947899+0000","last_undegraded":"2024-09-19T08:13:01.947899+0000","last_fullsized":"2024-09-19T08:13:01.947899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:50.522686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947846+0000","last_change":"2024-09-19T08:13:00.855215+0000","last_active":"2024-09-19T08:13:01.947846+0000","last_peered":"2024-09-19T08:13:01.947846+0000","last_clean":"2024-09-19T08:13:01.947846+0000","last_became_active":"2024-09-19T08:13:00.854878+0000","last_became_peered":"2024-09-19T08:13:00.854878+0000","last_unstale":"2024-09-19T08:13:01.947846+0000","last_undegraded":"2024-09-19T08:13:01.947846+0000","last_fullsized":"2024-09-19T08:13:01.947846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:48.216066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836201+0000","last_change":"2024-09-19T08:13:00.856995+0000","last_active":"2024-09-19T08:13:01.836201+0000","last_peered":"2024-09-19T08:13:01.836201+0000","last_clean":"2024-09-19T08:13:01.836201+0000","last_became_active":"2024-09-19T08:13:00.856645+0000","last_became_peered":"2024-09-19T08:13:00.856645+0000","last_unstale":"2024-09-19T08:13:01.836201+0000","last_undegraded":"2024-09-19T08:13:01.836201+0000","last_fullsized":"2024-09-19T08:13:01.836201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:20.699293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947354+0000","last_change":"2024-09-19T08:13:00.853362+0000","last_active":"2024-09-19T08:13:01.947354+0000","last_peered":"2024-09-19T08:13:01.947354+0000","last_clean":"2024-09-19T08:13:01.947354+0000","last_became_active":"2024-09-19T08:13:00.853046+0000","last_became_peered":"2024-09-19T08:13:00.853046+0000","last_unstale":"2024-09-19T08:13:01.947354+0000","last_undegraded":"2024-09-19T08:13:01.947354+0000","last_fullsized":"2024-09-19T08:13:01.947354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947080+0000","last_change":"2024-09-19T08:13:00.845175+0000","last_active":"2024-09-19T08:13:01.947080+0000","last_peered":"2024-09-19T08:13:01.947080+0000","last_clean":"2024-09-19T08:13:01.947080+0000","last_became_active":"2024-09-19T08:13:00.844857+0000","last_became_peered":"2024-09-19T08:13:00.844857+0000","last_unstale":"2024-09-19T08:13:01.947080+0000","last_undegraded":"2024-09-19T08:13:01.947080+0000","last_fullsized":"2024-09-19T08:13:01.947080+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:40.818956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947507+0000","last_change":"2024-09-19T08:13:00.847769+0000","last_active":"2024-09-19T08:13:01.947507+0000","last_peered":"2024-09-19T08:13:01.947507+0000","last_clean":"2024-09-19T08:13:01.947507+0000","last_became_active":"2024-09-19T08:13:00.847454+0000","last_became_peered":"2024-09-19T08:13:00.847454+0000","last_unstale":"2024-09-19T08:13:01.947507+0000","last_undegraded":"2024-09-19T08:13:01.947507+0000","last_fullsized":"2024-09-19T08:13:01.947507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.583051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732671+0000","last_change":"2024-09-19T08:13:00.859882+0000","last_active":"2024-09-19T08:14:07.732671+0000","last_peered":"2024-09-19T08:14:07.732671+0000","last_clean":"2024-09-19T08:14:07.732671+0000","last_became_active":"2024-09-19T08:13:00.859491+0000","last_became_peered":"2024-09-19T08:13:00.859491+0000","last_unstale":"2024-09-19T08:14:07.732671+0000","last_undegraded":"2024-09-19T08:14:07.732671+0000","last_fullsized":"2024-09-19T08:14:07.732671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:13.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948381+0000","last_change":"2024-09-19T08:13:00.845584+0000","last_active":"2024-09-19T08:13:01.948381+0000","last_peered":"2024-09-19T08:13:01.948381+0000","last_clean":"2024-09-19T08:13:01.948381+0000","last_became_active":"2024-09-19T08:13:00.845166+0000","last_became_peered":"2024-09-19T08:13:00.845166+0000","last_unstale":"2024-09-19T08:13:01.948381+0000","last_undegraded":"2024-09-19T08:13:01.948381+0000","last_fullsized":"2024-09-19T08:13:01.948381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:02.705793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947956+0000","last_change":"2024-09-19T08:13:00.838069+0000","last_active":"2024-09-19T08:13:01.947956+0000","last_peered":"2024-09-19T08:13:01.947956+0000","last_clean":"2024-09-19T08:13:01.947956+0000","last_became_active":"2024-09-19T08:13:00.837736+0000","last_became_peered":"2024-09-19T08:13:00.837736+0000","last_unstale":"2024-09-19T08:13:01.947956+0000","last_undegraded":"2024-09-19T08:13:01.947956+0000","last_fullsized":"2024-09-19T08:13:01.947956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:01.388290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733793+0000","last_change":"2024-09-19T08:13:00.841507+0000","last_active":"2024-09-19T08:14:07.733793+0000","last_peered":"2024-09-19T08:14:07.733793+0000","last_clean":"2024-09-19T08:14:07.733793+0000","last_became_active":"2024-09-19T08:13:00.841161+0000","last_became_peered":"2024-09-19T08:13:00.841161+0000","last_unstale":"2024-09-19T08:14:07.733793+0000","last_undegraded":"2024-09-19T08:14:07.733793+0000","last_fullsized":"2024-09-19T08:14:07.733793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:41.862290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732148+0000","last_change":"2024-09-19T08:13:00.847657+0000","last_active":"2024-09-19T08:14:07.732148+0000","last_peered":"2024-09-19T08:14:07.732148+0000","last_clean":"2024-09-19T08:14:07.732148+0000","last_became_active":"2024-09-19T08:13:00.846849+0000","last_became_peered":"2024-09-19T08:13:00.846849+0000","last_unstale":"2024-09-19T08:14:07.732148+0000","last_undegraded":"2024-09-19T08:14:07.732148+0000","last_fullsized":"2024-09-19T08:14:07.732148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:35.219913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732183+0000","last_change":"2024-09-19T08:13:00.845971+0000","last_active":"2024-09-19T08:14:07.732183+0000","last_peered":"2024-09-19T08:14:07.732183+0000","last_clean":"2024-09-19T08:14:07.732183+0000","last_became_active":"2024-09-19T08:13:00.845626+0000","last_became_peered":"2024-09-19T08:13:00.845626+0000","last_unstale":"2024-09-19T08:14:07.732183+0000","last_undegraded":"2024-09-19T08:14:07.732183+0000","last_fullsized":"2024-09-19T08:14:07.732183+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:36.945627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947938+0000","last_change":"2024-09-19T08:13:00.853851+0000","last_active":"2024-09-19T08:13:01.947938+0000","last_peered":"2024-09-19T08:13:01.947938+0000","last_clean":"2024-09-19T08:13:01.947938+0000","last_became_active":"2024-09-19T08:13:00.853189+0000","last_became_peered":"2024-09-19T08:13:00.853189+0000","last_unstale":"2024-09-19T08:13:01.947938+0000","last_undegraded":"2024-09-19T08:13:01.947938+0000","last_fullsized":"2024-09-19T08:13:01.947938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:04.812522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.946963+0000","last_change":"2024-09-19T08:13:00.848325+0000","last_active":"2024-09-19T08:13:01.946963+0000","last_peered":"2024-09-19T08:13:01.946963+0000","last_clean":"2024-09-19T08:13:01.946963+0000","last_became_active":"2024-09-19T08:13:00.848015+0000","last_became_peered":"2024-09-19T08:13:00.848015+0000","last_unstale":"2024-09-19T08:13:01.946963+0000","last_undegraded":"2024-09-19T08:13:01.946963+0000","last_fullsized":"2024-09-19T08:13:01.946963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:10.948451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:14:03.954679+0000","last_change":"2024-09-19T08:12:58.681134+0000","last_active":"2024-09-19T08:14:03.954679+0000","last_peered":"2024-09-19T08:14:03.954679+0000","last_clean":"2024-09-19T08:14:03.954679+0000","last_became_active":"2024-09-19T08:12:58.680764+0000","last_became_peered":"2024-09-19T08:12:58.680764+0000","last_unstale":"2024-09-19T08:14:03.954679+0000","last_undegraded":"2024-09-19T08:14:03.954679+0000","last_fullsized":"2024-09-19T08:14:03.954679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:24.939263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947299+0000","last_change":"2024-09-19T08:13:00.844319+0000","last_active":"2024-09-19T08:13:01.947299+0000","last_peered":"2024-09-19T08:13:01.947299+0000","last_clean":"2024-09-19T08:13:01.947299+0000","last_became_active":"2024-09-19T08:13:00.843994+0000","last_became_peered":"2024-09-19T08:13:00.843994+0000","last_unstale":"2024-09-19T08:13:01.947299+0000","last_undegraded":"2024-09-19T08:13:01.947299+0000","last_fullsized":"2024-09-19T08:13:01.947299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:46.937466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.880088+0000","last_change":"2024-09-19T08:12:58.676541+0000","last_active":"2024-09-19T08:13:10.880088+0000","last_peered":"2024-09-19T08:13:10.880088+0000","last_clean":"2024-09-19T08:13:10.880088+0000","last_became_active":"2024-09-19T08:12:58.675988+0000","last_became_peered":"2024-09-19T08:12:58.675988+0000","last_unstale":"2024-09-19T08:13:10.880088+0000","last_undegraded":"2024-09-19T08:13:10.880088+0000","last_fullsized":"2024-09-19T08:13:10.880088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:01.434986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.880054+0000","last_change":"2024-09-19T08:12:58.677510+0000","last_active":"2024-09-19T08:13:10.880054+0000","last_peered":"2024-09-19T08:13:10.880054+0000","last_clean":"2024-09-19T08:13:10.880054+0000","last_became_active":"2024-09-19T08:12:58.677200+0000","last_became_peered":"2024-09-19T08:12:58.677200+0000","last_unstale":"2024-09-19T08:13:10.880054+0000","last_undegraded":"2024-09-19T08:13:10.880054+0000","last_fullsized":"2024-09-19T08:13:10.880054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:36.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947353+0000","last_change":"2024-09-19T08:13:00.853430+0000","last_active":"2024-09-19T08:13:01.947353+0000","last_peered":"2024-09-19T08:13:01.947353+0000","last_clean":"2024-09-19T08:13:01.947353+0000","last_became_active":"2024-09-19T08:13:00.853096+0000","last_became_peered":"2024-09-19T08:13:00.853096+0000","last_unstale":"2024-09-19T08:13:01.947353+0000","last_undegraded":"2024-09-19T08:13:01.947353+0000","last_fullsized":"2024-09-19T08:13:01.947353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:07.147021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947748+0000","last_change":"2024-09-19T08:12:58.680081+0000","last_active":"2024-09-19T08:13:01.947748+0000","last_peered":"2024-09-19T08:13:01.947748+0000","last_clean":"2024-09-19T08:13:01.947748+0000","last_became_active":"2024-09-19T08:12:58.679704+0000","last_became_peered":"2024-09-19T08:12:58.679704+0000","last_unstale":"2024-09-19T08:13:01.947748+0000","last_undegraded":"2024-09-19T08:13:01.947748+0000","last_fullsized":"2024-09-19T08:13:01.947748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:32.942586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726249+0000","last_change":"2024-09-19T08:13:00.849394+0000","last_active":"2024-09-19T08:14:06.726249+0000","last_peered":"2024-09-19T08:14:06.726249+0000","last_clean":"2024-09-19T08:14:06.726249+0000","last_became_active":"2024-09-19T08:13:00.849051+0000","last_became_peered":"2024-09-19T08:13:00.849051+0000","last_unstale":"2024-09-19T08:14:06.726249+0000","last_undegraded":"2024-09-19T08:14:06.726249+0000","last_fullsized":"2024-09-19T08:14:06.726249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:23.012928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":75,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:14:03.953993+0000","last_change":"2024-09-19T08:12:58.676121+0000","last_active":"2024-09-19T08:14:03.953993+0000","last_peered":"2024-09-19T08:14:03.953993+0000","last_clean":"2024-09-19T08:14:03.953993+0000","last_became_active":"2024-09-19T08:12:58.675693+0000","last_became_peered":"2024-09-19T08:12:58.675693+0000","last_unstale":"2024-09-19T08:14:03.953993+0000","last_undegraded":"2024-09-19T08:14:03.953993+0000","last_fullsized":"2024-09-19T08:14:03.953993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:34.882190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836478+0000","last_change":"2024-09-19T08:13:00.858735+0000","last_active":"2024-09-19T08:13:01.836478+0000","last_peered":"2024-09-19T08:13:01.836478+0000","last_clean":"2024-09-19T08:13:01.836478+0000","last_became_active":"2024-09-19T08:13:00.858218+0000","last_became_peered":"2024-09-19T08:13:00.858218+0000","last_unstale":"2024-09-19T08:13:01.836478+0000","last_undegraded":"2024-09-19T08:13:01.836478+0000","last_fullsized":"2024-09-19T08:13:01.836478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:33.356810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872605+0000","last_change":"2024-09-19T08:12:58.679624+0000","last_active":"2024-09-19T08:13:09.872605+0000","last_peered":"2024-09-19T08:13:09.872605+0000","last_clean":"2024-09-19T08:13:09.872605+0000","last_became_active":"2024-09-19T08:12:58.679057+0000","last_became_peered":"2024-09-19T08:12:58.679057+0000","last_unstale":"2024-09-19T08:13:09.872605+0000","last_undegraded":"2024-09-19T08:13:09.872605+0000","last_fullsized":"2024-09-19T08:13:09.872605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:12.391663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733696+0000","last_change":"2024-09-19T08:13:00.843852+0000","last_active":"2024-09-19T08:14:07.733696+0000","last_peered":"2024-09-19T08:14:07.733696+0000","last_clean":"2024-09-19T08:14:07.733696+0000","last_became_active":"2024-09-19T08:13:00.843497+0000","last_became_peered":"2024-09-19T08:13:00.843497+0000","last_unstale":"2024-09-19T08:14:07.733696+0000","last_undegraded":"2024-09-19T08:14:07.733696+0000","last_fullsized":"2024-09-19T08:14:07.733696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:26.380024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.873411+0000","last_change":"2024-09-19T08:12:58.717225+0000","last_active":"2024-09-19T08:13:09.873411+0000","last_peered":"2024-09-19T08:13:09.873411+0000","last_clean":"2024-09-19T08:13:09.873411+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:13:09.873411+0000","last_undegraded":"2024-09-19T08:13:09.873411+0000","last_fullsized":"2024-09-19T08:13:09.873411+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:43.201907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836422+0000","last_change":"2024-09-19T08:13:00.845613+0000","last_active":"2024-09-19T08:13:01.836422+0000","last_peered":"2024-09-19T08:13:01.836422+0000","last_clean":"2024-09-19T08:13:01.836422+0000","last_became_active":"2024-09-19T08:13:00.845260+0000","last_became_peered":"2024-09-19T08:13:00.845260+0000","last_unstale":"2024-09-19T08:13:01.836422+0000","last_undegraded":"2024-09-19T08:13:01.836422+0000","last_fullsized":"2024-09-19T08:13:01.836422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:03:47.033972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'29","reported_seq":164,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:14:04.955077+0000","last_change":"2024-09-19T08:12:58.677308+0000","last_active":"2024-09-19T08:14:04.955077+0000","last_peered":"2024-09-19T08:14:04.955077+0000","last_clean":"2024-09-19T08:14:04.955077+0000","last_became_active":"2024-09-19T08:12:58.676925+0000","last_became_peered":"2024-09-19T08:12:58.676925+0000","last_unstale":"2024-09-19T08:14:04.955077+0000","last_undegraded":"2024-09-19T08:14:04.955077+0000","last_fullsized":"2024-09-19T08:14:04.955077+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:50.873656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61172,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947397+0000","last_change":"2024-09-19T08:13:00.855603+0000","last_active":"2024-09-19T08:13:01.947397+0000","last_peered":"2024-09-19T08:13:01.947397+0000","last_clean":"2024-09-19T08:13:01.947397+0000","last_became_active":"2024-09-19T08:13:00.855241+0000","last_became_peered":"2024-09-19T08:13:00.855241+0000","last_unstale":"2024-09-19T08:13:01.947397+0000","last_undegraded":"2024-09-19T08:13:01.947397+0000","last_fullsized":"2024-09-19T08:13:01.947397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:28.811847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732313+0000","last_change":"2024-09-19T08:12:58.675897+0000","last_active":"2024-09-19T08:14:07.732313+0000","last_peered":"2024-09-19T08:14:07.732313+0000","last_clean":"2024-09-19T08:14:07.732313+0000","last_became_active":"2024-09-19T08:12:58.675525+0000","last_became_peered":"2024-09-19T08:12:58.675525+0000","last_unstale":"2024-09-19T08:14:07.732313+0000","last_undegraded":"2024-09-19T08:14:07.732313+0000","last_fullsized":"2024-09-19T08:14:07.732313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:54.518064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947169+0000","last_change":"2024-09-19T08:13:00.856420+0000","last_active":"2024-09-19T08:13:01.947169+0000","last_peered":"2024-09-19T08:13:01.947169+0000","last_clean":"2024-09-19T08:13:01.947169+0000","last_became_active":"2024-09-19T08:13:00.856100+0000","last_became_peered":"2024-09-19T08:13:00.856100+0000","last_unstale":"2024-09-19T08:13:01.947169+0000","last_undegraded":"2024-09-19T08:13:01.947169+0000","last_fullsized":"2024-09-19T08:13:01.947169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:58.684930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872838+0000","last_change":"2024-09-19T08:12:58.680778+0000","last_active":"2024-09-19T08:13:09.872838+0000","last_peered":"2024-09-19T08:13:09.872838+0000","last_clean":"2024-09-19T08:13:09.872838+0000","last_became_active":"2024-09-19T08:12:58.680364+0000","last_became_peered":"2024-09-19T08:12:58.680364+0000","last_unstale":"2024-09-19T08:13:09.872838+0000","last_undegraded":"2024-09-19T08:13:09.872838+0000","last_fullsized":"2024-09-19T08:13:09.872838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:10.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100651+0000","last_change":"2024-09-19T08:13:00.853415+0000","last_active":"2024-09-19T08:13:02.100651+0000","last_peered":"2024-09-19T08:13:02.100651+0000","last_clean":"2024-09-19T08:13:02.100651+0000","last_became_active":"2024-09-19T08:13:00.853101+0000","last_became_peered":"2024-09-19T08:13:00.853101+0000","last_unstale":"2024-09-19T08:13:02.100651+0000","last_undegraded":"2024-09-19T08:13:02.100651+0000","last_fullsized":"2024-09-19T08:13:02.100651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:42.375394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":78,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:14:02.952897+0000","last_change":"2024-09-19T08:12:58.680312+0000","last_active":"2024-09-19T08:14:02.952897+0000","last_peered":"2024-09-19T08:14:02.952897+0000","last_clean":"2024-09-19T08:14:02.952897+0000","last_became_active":"2024-09-19T08:12:58.679870+0000","last_became_peered":"2024-09-19T08:12:58.679870+0000","last_unstale":"2024-09-19T08:14:02.952897+0000","last_undegraded":"2024-09-19T08:14:02.952897+0000","last_fullsized":"2024-09-19T08:14:02.952897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:42.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100329+0000","last_change":"2024-09-19T08:13:00.853920+0000","last_active":"2024-09-19T08:13:02.100329+0000","last_peered":"2024-09-19T08:13:02.100329+0000","last_clean":"2024-09-19T08:13:02.100329+0000","last_became_active":"2024-09-19T08:13:00.853594+0000","last_became_peered":"2024-09-19T08:13:00.853594+0000","last_unstale":"2024-09-19T08:13:02.100329+0000","last_undegraded":"2024-09-19T08:13:02.100329+0000","last_fullsized":"2024-09-19T08:13:02.100329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:31.746133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.882028+0000","last_change":"2024-09-19T08:12:58.717487+0000","last_active":"2024-09-19T08:13:10.882028+0000","last_peered":"2024-09-19T08:13:10.882028+0000","last_clean":"2024-09-19T08:13:10.882028+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:13:10.882028+0000","last_undegraded":"2024-09-19T08:13:10.882028+0000","last_fullsized":"2024-09-19T08:13:10.882028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:38.506051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947060+0000","last_change":"2024-09-19T08:13:00.847061+0000","last_active":"2024-09-19T08:13:01.947060+0000","last_peered":"2024-09-19T08:13:01.947060+0000","last_clean":"2024-09-19T08:13:01.947060+0000","last_became_active":"2024-09-19T08:13:00.846742+0000","last_became_peered":"2024-09-19T08:13:00.846742+0000","last_unstale":"2024-09-19T08:13:01.947060+0000","last_undegraded":"2024-09-19T08:13:01.947060+0000","last_fullsized":"2024-09-19T08:13:01.947060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:21.785585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:11.877000+0000","last_change":"2024-09-19T08:12:58.679367+0000","last_active":"2024-09-19T08:13:11.877000+0000","last_peered":"2024-09-19T08:13:11.877000+0000","last_clean":"2024-09-19T08:13:11.877000+0000","last_became_active":"2024-09-19T08:12:58.679035+0000","last_became_peered":"2024-09-19T08:12:58.679035+0000","last_unstale":"2024-09-19T08:13:11.877000+0000","last_undegraded":"2024-09-19T08:13:11.877000+0000","last_fullsized":"2024-09-19T08:13:11.877000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:25.633215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948364+0000","last_change":"2024-09-19T08:13:00.844940+0000","last_active":"2024-09-19T08:13:01.948364+0000","last_peered":"2024-09-19T08:13:01.948364+0000","last_clean":"2024-09-19T08:13:01.948364+0000","last_became_active":"2024-09-19T08:13:00.844454+0000","last_became_peered":"2024-09-19T08:13:00.844454+0000","last_unstale":"2024-09-19T08:13:01.948364+0000","last_undegraded":"2024-09-19T08:13:01.948364+0000","last_fullsized":"2024-09-19T08:13:01.948364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:15.577624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.272636+0000","last_change":"2024-09-19T08:12:58.673301+0000","last_active":"2024-09-19T08:13:02.272636+0000","last_peered":"2024-09-19T08:13:02.272636+0000","last_clean":"2024-09-19T08:13:02.272636+0000","last_became_active":"2024-09-19T08:12:58.672981+0000","last_became_peered":"2024-09-19T08:12:58.672981+0000","last_unstale":"2024-09-19T08:13:02.272636+0000","last_undegraded":"2024-09-19T08:13:02.272636+0000","last_fullsized":"2024-09-19T08:13:02.272636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:03.963712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100834+0000","last_change":"2024-09-19T08:13:00.856620+0000","last_active":"2024-09-19T08:13:02.100834+0000","last_peered":"2024-09-19T08:13:02.100834+0000","last_clean":"2024-09-19T08:13:02.100834+0000","last_became_active":"2024-09-19T08:13:00.856310+0000","last_became_peered":"2024-09-19T08:13:00.856310+0000","last_unstale":"2024-09-19T08:13:02.100834+0000","last_undegraded":"2024-09-19T08:13:02.100834+0000","last_fullsized":"2024-09-19T08:13:02.100834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:16.918768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.271923+0000","last_change":"2024-09-19T08:12:58.672282+0000","last_active":"2024-09-19T08:13:02.271923+0000","last_peered":"2024-09-19T08:13:02.271923+0000","last_clean":"2024-09-19T08:13:02.271923+0000","last_became_active":"2024-09-19T08:12:58.671891+0000","last_became_peered":"2024-09-19T08:12:58.671891+0000","last_unstale":"2024-09-19T08:13:02.271923+0000","last_undegraded":"2024-09-19T08:13:02.271923+0000","last_fullsized":"2024-09-19T08:13:02.271923+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:41.731571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100814+0000","last_change":"2024-09-19T08:13:00.848107+0000","last_active":"2024-09-19T08:13:02.100814+0000","last_peered":"2024-09-19T08:13:02.100814+0000","last_clean":"2024-09-19T08:13:02.100814+0000","last_became_active":"2024-09-19T08:13:00.847632+0000","last_became_peered":"2024-09-19T08:13:00.847632+0000","last_unstale":"2024-09-19T08:13:02.100814+0000","last_undegraded":"2024-09-19T08:13:02.100814+0000","last_fullsized":"2024-09-19T08:13:02.100814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.491122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732531+0000","last_change":"2024-09-19T08:12:59.179200+0000","last_active":"2024-09-19T08:14:07.732531+0000","last_peered":"2024-09-19T08:14:07.732531+0000","last_clean":"2024-09-19T08:14:07.732531+0000","last_became_active":"2024-09-19T08:12:59.178762+0000","last_became_peered":"2024-09-19T08:12:59.178762+0000","last_unstale":"2024-09-19T08:14:07.732531+0000","last_undegraded":"2024-09-19T08:14:07.732531+0000","last_fullsized":"2024-09-19T08:14:07.732531+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:08.424106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100825+0000","last_change":"2024-09-19T08:13:00.855870+0000","last_active":"2024-09-19T08:13:02.100825+0000","last_peered":"2024-09-19T08:13:02.100825+0000","last_clean":"2024-09-19T08:13:02.100825+0000","last_became_active":"2024-09-19T08:13:00.855538+0000","last_became_peered":"2024-09-19T08:13:00.855538+0000","last_unstale":"2024-09-19T08:13:02.100825+0000","last_undegraded":"2024-09-19T08:13:02.100825+0000","last_fullsized":"2024-09-19T08:13:02.100825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100488+0000","last_change":"2024-09-19T08:11:48.661333+0000","last_active":"2024-09-19T08:13:02.100488+0000","last_peered":"2024-09-19T08:13:02.100488+0000","last_clean":"2024-09-19T08:13:02.100488+0000","last_became_active":"2024-09-19T08:11:48.660873+0000","last_became_peered":"2024-09-19T08:11:48.660873+0000","last_unstale":"2024-09-19T08:13:02.100488+0000","last_undegraded":"2024-09-19T08:13:02.100488+0000","last_fullsized":"2024-09-19T08:13:02.100488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_clean_scrub_stamp":"2024-09-19T08:08:25.372155+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:19.401292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:11.881883+0000","last_change":"2024-09-19T08:12:58.675237+0000","last_active":"2024-09-19T08:13:11.881883+0000","last_peered":"2024-09-19T08:13:11.881883+0000","last_clean":"2024-09-19T08:13:11.881883+0000","last_became_active":"2024-09-19T08:12:58.674910+0000","last_became_peered":"2024-09-19T08:12:58.674910+0000","last_unstale":"2024-09-19T08:13:11.881883+0000","last_undegraded":"2024-09-19T08:13:11.881883+0000","last_fullsized":"2024-09-19T08:13:11.881883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:47.038328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947516+0000","last_change":"2024-09-19T08:13:00.846754+0000","last_active":"2024-09-19T08:13:01.947516+0000","last_peered":"2024-09-19T08:13:01.947516+0000","last_clean":"2024-09-19T08:13:01.947516+0000","last_became_active":"2024-09-19T08:13:00.846452+0000","last_became_peered":"2024-09-19T08:13:00.846452+0000","last_unstale":"2024-09-19T08:13:01.947516+0000","last_undegraded":"2024-09-19T08:13:01.947516+0000","last_fullsized":"2024-09-19T08:13:01.947516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:22.929689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.881292+0000","last_change":"2024-09-19T08:12:58.678958+0000","last_active":"2024-09-19T08:13:10.881292+0000","last_peered":"2024-09-19T08:13:10.881292+0000","last_clean":"2024-09-19T08:13:10.881292+0000","last_became_active":"2024-09-19T08:12:58.678592+0000","last_became_peered":"2024-09-19T08:12:58.678592+0000","last_unstale":"2024-09-19T08:13:10.881292+0000","last_undegraded":"2024-09-19T08:13:10.881292+0000","last_fullsized":"2024-09-19T08:13:10.881292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:58.374606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947087+0000","last_change":"2024-09-19T08:13:00.840973+0000","last_active":"2024-09-19T08:13:01.947087+0000","last_peered":"2024-09-19T08:13:01.947087+0000","last_clean":"2024-09-19T08:13:01.947087+0000","last_became_active":"2024-09-19T08:13:00.840619+0000","last_became_peered":"2024-09-19T08:13:00.840619+0000","last_unstale":"2024-09-19T08:13:01.947087+0000","last_undegraded":"2024-09-19T08:13:01.947087+0000","last_fullsized":"2024-09-19T08:13:01.947087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:14:03.953890+0000","last_change":"2024-09-19T08:12:58.715972+0000","last_active":"2024-09-19T08:14:03.953890+0000","last_peered":"2024-09-19T08:14:03.953890+0000","last_clean":"2024-09-19T08:14:03.953890+0000","last_became_active":"2024-09-19T08:12:58.715630+0000","last_became_peered":"2024-09-19T08:12:58.715630+0000","last_unstale":"2024-09-19T08:14:03.953890+0000","last_undegraded":"2024-09-19T08:14:03.953890+0000","last_fullsized":"2024-09-19T08:14:03.953890+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:35.637121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947439+0000","last_change":"2024-09-19T08:13:00.847023+0000","last_active":"2024-09-19T08:13:01.947439+0000","last_peered":"2024-09-19T08:13:01.947439+0000","last_clean":"2024-09-19T08:13:01.947439+0000","last_became_active":"2024-09-19T08:13:00.846387+0000","last_became_peered":"2024-09-19T08:13:00.846387+0000","last_unstale":"2024-09-19T08:13:01.947439+0000","last_undegraded":"2024-09-19T08:13:01.947439+0000","last_fullsized":"2024-09-19T08:13:01.947439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:12.187929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100987+0000","last_change":"2024-09-19T08:12:58.681062+0000","last_active":"2024-09-19T08:13:02.100987+0000","last_peered":"2024-09-19T08:13:02.100987+0000","last_clean":"2024-09-19T08:13:02.100987+0000","last_became_active":"2024-09-19T08:12:58.680524+0000","last_became_peered":"2024-09-19T08:12:58.680524+0000","last_unstale":"2024-09-19T08:13:02.100987+0000","last_undegraded":"2024-09-19T08:13:02.100987+0000","last_fullsized":"2024-09-19T08:13:02.100987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947809+0000","last_change":"2024-09-19T08:13:00.851523+0000","last_active":"2024-09-19T08:13:01.947809+0000","last_peered":"2024-09-19T08:13:01.947809+0000","last_clean":"2024-09-19T08:13:01.947809+0000","last_became_active":"2024-09-19T08:13:00.851068+0000","last_became_peered":"2024-09-19T08:13:00.851068+0000","last_unstale":"2024-09-19T08:13:01.947809+0000","last_undegraded":"2024-09-19T08:13:01.947809+0000","last_fullsized":"2024-09-19T08:13:01.947809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:44.905469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726509+0000","last_change":"2024-09-19T08:12:58.718234+0000","last_active":"2024-09-19T08:14:06.726509+0000","last_peered":"2024-09-19T08:14:06.726509+0000","last_clean":"2024-09-19T08:14:06.726509+0000","last_became_active":"2024-09-19T08:12:58.717840+0000","last_became_peered":"2024-09-19T08:12:58.717840+0000","last_unstale":"2024-09-19T08:14:06.726509+0000","last_undegraded":"2024-09-19T08:14:06.726509+0000","last_fullsized":"2024-09-19T08:14:06.726509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:05.283770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947784+0000","last_change":"2024-09-19T08:13:00.848305+0000","last_active":"2024-09-19T08:13:01.947784+0000","last_peered":"2024-09-19T08:13:01.947784+0000","last_clean":"2024-09-19T08:13:01.947784+0000","last_became_active":"2024-09-19T08:13:00.847981+0000","last_became_peered":"2024-09-19T08:13:00.847981+0000","last_unstale":"2024-09-19T08:13:01.947784+0000","last_undegraded":"2024-09-19T08:13:01.947784+0000","last_fullsized":"2024-09-19T08:13:01.947784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:04.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.871966+0000","last_change":"2024-09-19T08:12:58.715943+0000","last_active":"2024-09-19T08:13:09.871966+0000","last_peered":"2024-09-19T08:13:09.871966+0000","last_clean":"2024-09-19T08:13:09.871966+0000","last_became_active":"2024-09-19T08:12:58.715600+0000","last_became_peered":"2024-09-19T08:12:58.715600+0000","last_unstale":"2024-09-19T08:13:09.871966+0000","last_undegraded":"2024-09-19T08:13:09.871966+0000","last_fullsized":"2024-09-19T08:13:09.871966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:19.142271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948126+0000","last_change":"2024-09-19T08:13:00.840944+0000","last_active":"2024-09-19T08:13:01.948126+0000","last_peered":"2024-09-19T08:13:01.948126+0000","last_clean":"2024-09-19T08:13:01.948126+0000","last_became_active":"2024-09-19T08:13:00.840608+0000","last_became_peered":"2024-09-19T08:13:00.840608+0000","last_unstale":"2024-09-19T08:13:01.948126+0000","last_undegraded":"2024-09-19T08:13:01.948126+0000","last_fullsized":"2024-09-19T08:13:01.948126+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:25.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'11","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726648+0000","last_change":"2024-09-19T08:12:58.672929+0000","last_active":"2024-09-19T08:14:06.726648+0000","last_peered":"2024-09-19T08:14:06.726648+0000","last_clean":"2024-09-19T08:14:06.726648+0000","last_became_active":"2024-09-19T08:12:58.672487+0000","last_became_peered":"2024-09-19T08:12:58.672487+0000","last_unstale":"2024-09-19T08:14:06.726648+0000","last_undegraded":"2024-09-19T08:14:06.726648+0000","last_fullsized":"2024-09-19T08:14:06.726648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:33.042735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":2,"reported_epoch":84,"state":"peering","last_fresh":"2024-09-19T08:13:04.156818+0000","last_change":"2024-09-19T08:14:06.939357+0000","last_active":"2024-09-19T08:14:06.939357+0000","last_peered":"2024-09-19T08:13:04.156818+0000","last_clean":"2024-09-19T08:13:04.156818+0000","last_became_active":"2024-09-19T08:13:00.853159+0000","last_became_peered":"2024-09-19T08:13:00.853159+0000","last_unstale":"2024-09-19T08:13:04.156818+0000","last_undegraded":"2024-09-19T08:13:04.156818+0000","last_fullsized":"2024-09-19T08:13:04.156818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.869690+0000","last_change":"2024-09-19T08:12:59.179510+0000","last_active":"2024-09-19T08:13:09.869690+0000","last_peered":"2024-09-19T08:13:09.869690+0000","last_clean":"2024-09-19T08:13:09.869690+0000","last_became_active":"2024-09-19T08:12:59.179157+0000","last_became_peered":"2024-09-19T08:12:59.179157+0000","last_unstale":"2024-09-19T08:13:09.869690+0000","last_undegraded":"2024-09-19T08:13:09.869690+0000","last_fullsized":"2024-09-19T08:13:09.869690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:22.981289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733930+0000","last_change":"2024-09-19T08:13:00.841853+0000","last_active":"2024-09-19T08:14:07.733930+0000","last_peered":"2024-09-19T08:14:07.733930+0000","last_clean":"2024-09-19T08:14:07.733930+0000","last_became_active":"2024-09-19T08:13:00.841518+0000","last_became_peered":"2024-09-19T08:13:00.841518+0000","last_unstale":"2024-09-19T08:14:07.733930+0000","last_undegraded":"2024-09-19T08:14:07.733930+0000","last_fullsized":"2024-09-19T08:14:07.733930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:05.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872283+0000","last_change":"2024-09-19T08:12:58.716882+0000","last_active":"2024-09-19T08:13:09.872283+0000","last_peered":"2024-09-19T08:13:09.872283+0000","last_clean":"2024-09-19T08:13:09.872283+0000","last_became_active":"2024-09-19T08:12:58.716545+0000","last_became_peered":"2024-09-19T08:12:58.716545+0000","last_unstale":"2024-09-19T08:13:09.872283+0000","last_undegraded":"2024-09-19T08:13:09.872283+0000","last_fullsized":"2024-09-19T08:13:09.872283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:14.469109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.946790+0000","last_change":"2024-09-19T08:13:00.850484+0000","last_active":"2024-09-19T08:13:01.946790+0000","last_peered":"2024-09-19T08:13:01.946790+0000","last_clean":"2024-09-19T08:13:01.946790+0000","last_became_active":"2024-09-19T08:13:00.850049+0000","last_became_peered":"2024-09-19T08:13:00.850049+0000","last_unstale":"2024-09-19T08:13:01.946790+0000","last_undegraded":"2024-09-19T08:13:01.946790+0000","last_fullsized":"2024-09-19T08:13:01.946790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:24.424111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.870765+0000","last_change":"2024-09-19T08:12:58.669928+0000","last_active":"2024-09-19T08:13:09.870765+0000","last_peered":"2024-09-19T08:13:09.870765+0000","last_clean":"2024-09-19T08:13:09.870765+0000","last_became_active":"2024-09-19T08:12:58.669614+0000","last_became_peered":"2024-09-19T08:12:58.669614+0000","last_unstale":"2024-09-19T08:13:09.870765+0000","last_undegraded":"2024-09-19T08:13:09.870765+0000","last_fullsized":"2024-09-19T08:13:09.870765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:50.924779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948102+0000","last_change":"2024-09-19T08:13:00.853520+0000","last_active":"2024-09-19T08:13:01.948102+0000","last_peered":"2024-09-19T08:13:01.948102+0000","last_clean":"2024-09-19T08:13:01.948102+0000","last_became_active":"2024-09-19T08:13:00.853184+0000","last_became_peered":"2024-09-19T08:13:00.853184+0000","last_unstale":"2024-09-19T08:13:01.948102+0000","last_undegraded":"2024-09-19T08:13:01.948102+0000","last_fullsized":"2024-09-19T08:13:01.948102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:45.160439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732133+0000","last_change":"2024-09-19T08:12:58.674359+0000","last_active":"2024-09-19T08:14:07.732133+0000","last_peered":"2024-09-19T08:14:07.732133+0000","last_clean":"2024-09-19T08:14:07.732133+0000","last_became_active":"2024-09-19T08:12:58.673890+0000","last_became_peered":"2024-09-19T08:12:58.673890+0000","last_unstale":"2024-09-19T08:14:07.732133+0000","last_undegraded":"2024-09-19T08:14:07.732133+0000","last_fullsized":"2024-09-19T08:14:07.732133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:54.939959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947708+0000","last_change":"2024-09-19T08:13:00.851071+0000","last_active":"2024-09-19T08:13:01.947708+0000","last_peered":"2024-09-19T08:13:01.947708+0000","last_clean":"2024-09-19T08:13:01.947708+0000","last_became_active":"2024-09-19T08:13:00.850756+0000","last_became_peered":"2024-09-19T08:13:00.850756+0000","last_unstale":"2024-09-19T08:13:01.947708+0000","last_undegraded":"2024-09-19T08:13:01.947708+0000","last_fullsized":"2024-09-19T08:13:01.947708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:52.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872854+0000","last_change":"2024-09-19T08:12:58.681466+0000","last_active":"2024-09-19T08:13:09.872854+0000","last_peered":"2024-09-19T08:13:09.872854+0000","last_clean":"2024-09-19T08:13:09.872854+0000","last_became_active":"2024-09-19T08:12:58.681113+0000","last_became_peered":"2024-09-19T08:12:58.681113+0000","last_unstale":"2024-09-19T08:13:09.872854+0000","last_undegraded":"2024-09-19T08:13:09.872854+0000","last_fullsized":"2024-09-19T08:13:09.872854+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:49.610828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.100583+0000","last_change":"2024-09-19T08:13:00.851272+0000","last_active":"2024-09-19T08:13:02.100583+0000","last_peered":"2024-09-19T08:13:02.100583+0000","last_clean":"2024-09-19T08:13:02.100583+0000","last_became_active":"2024-09-19T08:13:00.849098+0000","last_became_peered":"2024-09-19T08:13:00.849098+0000","last_unstale":"2024-09-19T08:13:02.100583+0000","last_undegraded":"2024-09-19T08:13:02.100583+0000","last_fullsized":"2024-09-19T08:13:02.100583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:05.643992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947777+0000","last_change":"2024-09-19T08:13:00.849945+0000","last_active":"2024-09-19T08:13:01.947777+0000","last_peered":"2024-09-19T08:13:01.947777+0000","last_clean":"2024-09-19T08:13:01.947777+0000","last_became_active":"2024-09-19T08:13:00.849591+0000","last_became_peered":"2024-09-19T08:13:00.849591+0000","last_unstale":"2024-09-19T08:13:01.947777+0000","last_undegraded":"2024-09-19T08:13:01.947777+0000","last_fullsized":"2024-09-19T08:13:01.947777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:27.434005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.272564+0000","last_change":"2024-09-19T08:12:58.674610+0000","last_active":"2024-09-19T08:13:02.272564+0000","last_peered":"2024-09-19T08:13:02.272564+0000","last_clean":"2024-09-19T08:13:02.272564+0000","last_became_active":"2024-09-19T08:12:58.674247+0000","last_became_peered":"2024-09-19T08:12:58.674247+0000","last_unstale":"2024-09-19T08:13:02.272564+0000","last_undegraded":"2024-09-19T08:13:02.272564+0000","last_fullsized":"2024-09-19T08:13:02.272564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:16.323574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.836419+0000","last_change":"2024-09-19T08:13:00.858512+0000","last_active":"2024-09-19T08:13:01.836419+0000","last_peered":"2024-09-19T08:13:01.836419+0000","last_clean":"2024-09-19T08:13:01.836419+0000","last_became_active":"2024-09-19T08:13:00.857897+0000","last_became_peered":"2024-09-19T08:13:00.857897+0000","last_unstale":"2024-09-19T08:13:01.836419+0000","last_undegraded":"2024-09-19T08:13:01.836419+0000","last_fullsized":"2024-09-19T08:13:01.836419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.881153+0000","last_change":"2024-09-19T08:12:58.682207+0000","last_active":"2024-09-19T08:13:10.881153+0000","last_peered":"2024-09-19T08:13:10.881153+0000","last_clean":"2024-09-19T08:13:10.881153+0000","last_became_active":"2024-09-19T08:12:58.681842+0000","last_became_peered":"2024-09-19T08:12:58.681842+0000","last_unstale":"2024-09-19T08:13:10.881153+0000","last_undegraded":"2024-09-19T08:13:10.881153+0000","last_fullsized":"2024-09-19T08:13:10.881153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.825858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733650+0000","last_change":"2024-09-19T08:13:00.846748+0000","last_active":"2024-09-19T08:14:07.733650+0000","last_peered":"2024-09-19T08:14:07.733650+0000","last_clean":"2024-09-19T08:14:07.733650+0000","last_became_active":"2024-09-19T08:13:00.846368+0000","last_became_peered":"2024-09-19T08:13:00.846368+0000","last_unstale":"2024-09-19T08:14:07.733650+0000","last_undegraded":"2024-09-19T08:14:07.733650+0000","last_fullsized":"2024-09-19T08:14:07.733650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:59.526512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.271849+0000","last_change":"2024-09-19T08:12:58.716118+0000","last_active":"2024-09-19T08:13:02.271849+0000","last_peered":"2024-09-19T08:13:02.271849+0000","last_clean":"2024-09-19T08:13:02.271849+0000","last_became_active":"2024-09-19T08:12:58.715711+0000","last_became_peered":"2024-09-19T08:12:58.715711+0000","last_unstale":"2024-09-19T08:13:02.271849+0000","last_undegraded":"2024-09-19T08:13:02.271849+0000","last_fullsized":"2024-09-19T08:13:02.271849+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:29.439887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947595+0000","last_change":"2024-09-19T08:13:00.848729+0000","last_active":"2024-09-19T08:13:01.947595+0000","last_peered":"2024-09-19T08:13:01.947595+0000","last_clean":"2024-09-19T08:13:01.947595+0000","last_became_active":"2024-09-19T08:13:00.848470+0000","last_became_peered":"2024-09-19T08:13:00.848470+0000","last_unstale":"2024-09-19T08:13:01.947595+0000","last_undegraded":"2024-09-19T08:13:01.947595+0000","last_fullsized":"2024-09-19T08:13:01.947595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:30.653101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726774+0000","last_change":"2024-09-19T08:12:58.681654+0000","last_active":"2024-09-19T08:14:06.726774+0000","last_peered":"2024-09-19T08:14:06.726774+0000","last_clean":"2024-09-19T08:14:06.726774+0000","last_became_active":"2024-09-19T08:12:58.681284+0000","last_became_peered":"2024-09-19T08:12:58.681284+0000","last_unstale":"2024-09-19T08:14:06.726774+0000","last_undegraded":"2024-09-19T08:14:06.726774+0000","last_fullsized":"2024-09-19T08:14:06.726774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.429930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65568,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":352,"num_read_kb":245,"num_write":164,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":348160,"data_stored":135532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251470,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481818,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":428609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710751,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":1081655,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874277,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":428845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":909312,"data_stored":428765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1021090,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495667,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":428234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":1020381,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":429064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":913408,"data_stored":428545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":508131,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":489199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":438206,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34755,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34755,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:14:11.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:14:11.401 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:11.824 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:11 smithi022 ceph-mon[30963]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.908+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/85078060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc99c101420 msgr2=0x7fc99c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.908+0000 7fc9a0b4f640 1 --2- 172.21.15.22:0/85078060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc99c101420 0x7fc99c10d950 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc994009cd0 tx=0x7fc99402a8e0 comp rx=0 tx=0).stop 2024-09-19T08:14:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.908+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/85078060 shutdown_connections 2024-09-19T08:14:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.908+0000 7fc9a0b4f640 1 --2- 172.21.15.22:0/85078060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc99c101420 0x7fc99c10d950 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.908+0000 7fc9a0b4f640 1 --2- 172.21.15.22:0/85078060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc99c100a80 0x7fc99c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.908+0000 7fc9a0b4f640 1 --2- 172.21.15.22:0/85078060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc99c106a80 0x7fc99c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.908+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/85078060 >> 172.21.15.22:0/85078060 conn(0x7fc99c0fc7a0 msgr2=0x7fc99c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.908+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/85078060 shutdown_connections 2024-09-19T08:14:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.908+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/85078060 wait complete. 2024-09-19T08:14:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.908+0000 7fc9a0b4f640 1 Processor -- start 2024-09-19T08:14:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc9a0b4f640 1 -- start start 2024-09-19T08:14:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc9a0b4f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc99c100a80 0x7fc99c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc9a0b4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc99c101420 0x7fc99c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc9a0b4f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc99c106a80 0x7fc99c19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc9a0b4f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c110e80 con 0x7fc99c101420 2024-09-19T08:14:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc9a0b4f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c110d00 con 0x7fc99c100a80 2024-09-19T08:14:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc9a0b4f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c111000 con 0x7fc99c106a80 2024-09-19T08:14:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc993fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc99c101420 0x7fc99c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:11.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc993fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc99c101420 0x7fc99c197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60086/0 (socket says 172.21.15.22:60086) 2024-09-19T08:14:11.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc993fff640 1 -- 172.21.15.22:0/772136039 learned_addr learned my addr 172.21.15.22:0/772136039 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:11.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc99a59c640 1 --2- 172.21.15.22:0/772136039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc99c100a80 0x7fc99c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:11.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc99ad9d640 1 --2- 172.21.15.22:0/772136039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc99c106a80 0x7fc99c19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:11.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc993fff640 1 -- 172.21.15.22:0/772136039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc99c106a80 msgr2=0x7fc99c19bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:11.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc993fff640 1 --2- 172.21.15.22:0/772136039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc99c106a80 0x7fc99c19bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:11.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc993fff640 1 -- 172.21.15.22:0/772136039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc99c100a80 msgr2=0x7fc99c1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:11.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc993fff640 1 --2- 172.21.15.22:0/772136039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc99c100a80 0x7fc99c1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:11.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc993fff640 1 -- 172.21.15.22:0/772136039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c19c290 con 0x7fc99c101420 2024-09-19T08:14:11.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.909+0000 7fc993fff640 1 --2- 172.21.15.22:0/772136039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc99c101420 0x7fc99c197820 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc98400d950 tx=0x7fc98400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:11.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.910+0000 7fc9937fe640 1 -- 172.21.15.22:0/772136039 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984017070 con 0x7fc99c101420 2024-09-19T08:14:11.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.910+0000 7fc9937fe640 1 -- 172.21.15.22:0/772136039 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc984009c30 con 0x7fc99c101420 2024-09-19T08:14:11.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.910+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/772136039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99c19c580 con 0x7fc99c101420 2024-09-19T08:14:11.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.910+0000 7fc9937fe640 1 -- 172.21.15.22:0/772136039 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98400f6b0 con 0x7fc99c101420 2024-09-19T08:14:11.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.910+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/772136039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99c1a3e60 con 0x7fc99c101420 2024-09-19T08:14:11.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.911+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/772136039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99c100ee0 con 0x7fc99c101420 2024-09-19T08:14:11.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.912+0000 7fc9937fe640 1 -- 172.21.15.22:0/772136039 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc9840109d0 con 0x7fc99c101420 2024-09-19T08:14:11.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.913+0000 7fc9937fe640 1 --2- 172.21.15.22:0/772136039 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc9880a3350 0x7fc9880a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:11.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.913+0000 7fc99a59c640 1 --2- 172.21.15.22:0/772136039 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc9880a3350 0x7fc9880a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:11.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.913+0000 7fc9937fe640 1 -- 172.21.15.22:0/772136039 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc9840cf5e0 con 0x7fc99c101420 2024-09-19T08:14:11.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.914+0000 7fc99a59c640 1 --2- 172.21.15.22:0/772136039 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc9880a3350 0x7fc9880a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc98c005d00 tx=0x7fc98c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:11.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:11.918+0000 7fc9937fe640 1 -- 172.21.15.22:0/772136039 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc984009de0 con 0x7fc99c101420 2024-09-19T08:14:12.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:11 smithi064 ceph-mon[34900]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:12.091+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/772136039 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc99c0fe5c0 con 0x7fc9880a3350 2024-09-19T08:14:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:12.097+0000 7fc9937fe640 1 -- 172.21.15.22:0/772136039 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258895 (secure 0 0 0) 0x7fc99c0fe5c0 con 0x7fc9880a3350 2024-09-19T08:14:12.101 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T08:14:12.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:12.100+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/772136039 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc9880a3350 msgr2=0x7fc9880a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:12.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:12.100+0000 7fc9a0b4f640 1 --2- 172.21.15.22:0/772136039 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc9880a3350 0x7fc9880a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc98c005d00 tx=0x7fc98c00b410 comp rx=0 tx=0).stop 2024-09-19T08:14:12.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:12.100+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/772136039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc99c101420 msgr2=0x7fc99c197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:12.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:12.100+0000 7fc9a0b4f640 1 --2- 172.21.15.22:0/772136039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc99c101420 0x7fc99c197820 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc98400d950 tx=0x7fc98400de10 comp rx=0 tx=0).stop 2024-09-19T08:14:12.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:12.100+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/772136039 shutdown_connections 2024-09-19T08:14:12.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:12.100+0000 7fc9a0b4f640 1 --2- 172.21.15.22:0/772136039 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc9880a3350 0x7fc9880a5810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:12.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:12.100+0000 7fc9a0b4f640 1 --2- 172.21.15.22:0/772136039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc99c106a80 0x7fc99c19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:12.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:12.100+0000 7fc9a0b4f640 1 --2- 172.21.15.22:0/772136039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc99c101420 0x7fc99c197820 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:12.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:12.100+0000 7fc9a0b4f640 1 --2- 172.21.15.22:0/772136039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc99c100a80 0x7fc99c1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:12.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:12.100+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/772136039 >> 172.21.15.22:0/772136039 conn(0x7fc99c0fc7a0 msgr2=0x7fc99c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:12.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:12.100+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/772136039 shutdown_connections 2024-09-19T08:14:12.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:12.100+0000 7fc9a0b4f640 1 -- 172.21.15.22:0/772136039 wait complete. 2024-09-19T08:14:12.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:11 smithi157 ceph-mon[35143]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:12.736 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-19T08:14:10.779418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656170,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":508,"num_read_kb":374,"num_write":347,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1241908,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381308,"kb_used_data":11208,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560516,"statfs":{"total":1151940427776,"available":1151549968384,"internally_reserved":0,"allocated":11476992,"data_stored":7303731,"data_compressed":165268,"data_compressed_allocated":212992,"data_compressed_original":1957888,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":25,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":61172,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001896"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948678+0000","last_change":"2024-09-19T08:13:00.853947+0000","last_active":"2024-09-19T08:13:01.948678+0000","last_peered":"2024-09-19T08:13:01.948678+0000","last_clean":"2024-09-19T08:13:01.948678+0000","last_became_active":"2024-09-19T08:13:00.852951+0000","last_became_peered":"2024-09-19T08:13:00.852951+0000","last_unstale":"2024-09-19T08:13:01.948678+0000","last_undegraded":"2024-09-19T08:13:01.948678+0000","last_fullsized":"2024-09-19T08:13:01.948678+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:25.786828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733811+0000","last_change":"2024-09-19T08:13:00.853389+0000","last_active":"2024-09-19T08:14:07.733811+0000","last_peered":"2024-09-19T08:14:07.733811+0000","last_clean":"2024-09-19T08:14:07.733811+0000","last_became_active":"2024-09-19T08:13:00.853102+0000","last_became_peered":"2024-09-19T08:13:00.853102+0000","last_unstale":"2024-09-19T08:14:07.733811+0000","last_undegraded":"2024-09-19T08:14:07.733811+0000","last_fullsized":"2024-09-19T08:14:07.733811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:42.773917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733742+0000","last_change":"2024-09-19T08:13:00.846544+0000","last_active":"2024-09-19T08:14:07.733742+0000","last_peered":"2024-09-19T08:14:07.733742+0000","last_clean":"2024-09-19T08:14:07.733742+0000","last_became_active":"2024-09-19T08:13:00.846229+0000","last_became_peered":"2024-09-19T08:13:00.846229+0000","last_unstale":"2024-09-19T08:14:07.733742+0000","last_undegraded":"2024-09-19T08:14:07.733742+0000","last_fullsized":"2024-09-19T08:14:07.733742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:43.471684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.739325+0000","last_change":"2024-09-19T08:13:00.848149+0000","last_active":"2024-09-19T08:14:07.739325+0000","last_peered":"2024-09-19T08:14:07.739325+0000","last_clean":"2024-09-19T08:14:07.739325+0000","last_became_active":"2024-09-19T08:13:00.847744+0000","last_became_peered":"2024-09-19T08:13:00.847744+0000","last_unstale":"2024-09-19T08:14:07.739325+0000","last_undegraded":"2024-09-19T08:14:07.739325+0000","last_fullsized":"2024-09-19T08:14:07.739325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:01.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948347+0000","last_change":"2024-09-19T08:13:00.841862+0000","last_active":"2024-09-19T08:13:01.948347+0000","last_peered":"2024-09-19T08:13:01.948347+0000","last_clean":"2024-09-19T08:13:01.948347+0000","last_became_active":"2024-09-19T08:13:00.841331+0000","last_became_peered":"2024-09-19T08:13:00.841331+0000","last_unstale":"2024-09-19T08:13:01.948347+0000","last_undegraded":"2024-09-19T08:13:01.948347+0000","last_fullsized":"2024-09-19T08:13:01.948347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:54.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735492+0000","last_change":"2024-09-19T08:13:00.849626+0000","last_active":"2024-09-19T08:14:07.735492+0000","last_peered":"2024-09-19T08:14:07.735492+0000","last_clean":"2024-09-19T08:14:07.735492+0000","last_became_active":"2024-09-19T08:13:00.849034+0000","last_became_peered":"2024-09-19T08:13:00.849034+0000","last_unstale":"2024-09-19T08:14:07.735492+0000","last_undegraded":"2024-09-19T08:14:07.735492+0000","last_fullsized":"2024-09-19T08:14:07.735492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:39.118978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734312+0000","last_change":"2024-09-19T08:13:00.847522+0000","last_active":"2024-09-19T08:14:07.734312+0000","last_peered":"2024-09-19T08:14:07.734312+0000","last_clean":"2024-09-19T08:14:07.734312+0000","last_became_active":"2024-09-19T08:13:00.847117+0000","last_became_peered":"2024-09-19T08:13:00.847117+0000","last_unstale":"2024-09-19T08:14:07.734312+0000","last_undegraded":"2024-09-19T08:14:07.734312+0000","last_fullsized":"2024-09-19T08:14:07.734312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.836385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399370+0000","last_change":"2024-09-19T08:13:00.849544+0000","last_active":"2024-09-19T08:14:08.399370+0000","last_peered":"2024-09-19T08:14:08.399370+0000","last_clean":"2024-09-19T08:14:08.399370+0000","last_became_active":"2024-09-19T08:13:00.849170+0000","last_became_peered":"2024-09-19T08:13:00.849170+0000","last_unstale":"2024-09-19T08:14:08.399370+0000","last_undegraded":"2024-09-19T08:14:08.399370+0000","last_fullsized":"2024-09-19T08:14:08.399370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:28.371965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733158+0000","last_change":"2024-09-19T08:13:00.856052+0000","last_active":"2024-09-19T08:14:07.733158+0000","last_peered":"2024-09-19T08:14:07.733158+0000","last_clean":"2024-09-19T08:14:07.733158+0000","last_became_active":"2024-09-19T08:13:00.855291+0000","last_became_peered":"2024-09-19T08:13:00.855291+0000","last_unstale":"2024-09-19T08:14:07.733158+0000","last_undegraded":"2024-09-19T08:14:07.733158+0000","last_fullsized":"2024-09-19T08:14:07.733158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:18.581867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735684+0000","last_change":"2024-09-19T08:13:00.850145+0000","last_active":"2024-09-19T08:14:07.735684+0000","last_peered":"2024-09-19T08:14:07.735684+0000","last_clean":"2024-09-19T08:14:07.735684+0000","last_became_active":"2024-09-19T08:13:00.849798+0000","last_became_peered":"2024-09-19T08:13:00.849798+0000","last_unstale":"2024-09-19T08:14:07.735684+0000","last_undegraded":"2024-09-19T08:14:07.735684+0000","last_fullsized":"2024-09-19T08:14:07.735684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:21.400000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734734+0000","last_change":"2024-09-19T08:13:00.859153+0000","last_active":"2024-09-19T08:14:07.734734+0000","last_peered":"2024-09-19T08:14:07.734734+0000","last_clean":"2024-09-19T08:14:07.734734+0000","last_became_active":"2024-09-19T08:13:00.858473+0000","last_became_peered":"2024-09-19T08:13:00.858473+0000","last_unstale":"2024-09-19T08:14:07.734734+0000","last_undegraded":"2024-09-19T08:14:07.734734+0000","last_fullsized":"2024-09-19T08:14:07.734734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:07.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.739385+0000","last_change":"2024-09-19T08:13:00.850797+0000","last_active":"2024-09-19T08:14:07.739385+0000","last_peered":"2024-09-19T08:14:07.739385+0000","last_clean":"2024-09-19T08:14:07.739385+0000","last_became_active":"2024-09-19T08:13:00.850393+0000","last_became_peered":"2024-09-19T08:13:00.850393+0000","last_unstale":"2024-09-19T08:14:07.739385+0000","last_undegraded":"2024-09-19T08:14:07.739385+0000","last_fullsized":"2024-09-19T08:14:07.739385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:59.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731520+0000","last_change":"2024-09-19T08:13:00.850835+0000","last_active":"2024-09-19T08:14:07.731520+0000","last_peered":"2024-09-19T08:14:07.731520+0000","last_clean":"2024-09-19T08:14:07.731520+0000","last_became_active":"2024-09-19T08:13:00.850518+0000","last_became_peered":"2024-09-19T08:13:00.850518+0000","last_unstale":"2024-09-19T08:14:07.731520+0000","last_undegraded":"2024-09-19T08:14:07.731520+0000","last_fullsized":"2024-09-19T08:14:07.731520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.178530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733525+0000","last_change":"2024-09-19T08:13:00.853934+0000","last_active":"2024-09-19T08:14:07.733525+0000","last_peered":"2024-09-19T08:14:07.733525+0000","last_clean":"2024-09-19T08:14:07.733525+0000","last_became_active":"2024-09-19T08:13:00.853604+0000","last_became_peered":"2024-09-19T08:13:00.853604+0000","last_unstale":"2024-09-19T08:14:07.733525+0000","last_undegraded":"2024-09-19T08:14:07.733525+0000","last_fullsized":"2024-09-19T08:14:07.733525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:43.622526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.741370+0000","last_change":"2024-09-19T08:14:07.741370+0000","last_active":"2024-09-19T08:14:07.741370+0000","last_peered":"2024-09-19T08:14:07.741370+0000","last_clean":"2024-09-19T08:14:07.741370+0000","last_became_active":"2024-09-19T08:14:07.741116+0000","last_became_peered":"2024-09-19T08:14:07.741116+0000","last_unstale":"2024-09-19T08:14:07.741370+0000","last_undegraded":"2024-09-19T08:14:07.741370+0000","last_fullsized":"2024-09-19T08:14:07.741370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:45.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732701+0000","last_change":"2024-09-19T08:13:00.850266+0000","last_active":"2024-09-19T08:14:07.732701+0000","last_peered":"2024-09-19T08:14:07.732701+0000","last_clean":"2024-09-19T08:14:07.732701+0000","last_became_active":"2024-09-19T08:13:00.849928+0000","last_became_peered":"2024-09-19T08:13:00.849928+0000","last_unstale":"2024-09-19T08:14:07.732701+0000","last_undegraded":"2024-09-19T08:14:07.732701+0000","last_fullsized":"2024-09-19T08:14:07.732701+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:38.231331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948706+0000","last_change":"2024-09-19T08:13:00.852551+0000","last_active":"2024-09-19T08:13:01.948706+0000","last_peered":"2024-09-19T08:13:01.948706+0000","last_clean":"2024-09-19T08:13:01.948706+0000","last_became_active":"2024-09-19T08:13:00.852225+0000","last_became_peered":"2024-09-19T08:13:00.852225+0000","last_unstale":"2024-09-19T08:13:01.948706+0000","last_undegraded":"2024-09-19T08:13:01.948706+0000","last_fullsized":"2024-09-19T08:13:01.948706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:33.115108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726544+0000","last_change":"2024-09-19T08:13:00.854033+0000","last_active":"2024-09-19T08:14:06.726544+0000","last_peered":"2024-09-19T08:14:06.726544+0000","last_clean":"2024-09-19T08:14:06.726544+0000","last_became_active":"2024-09-19T08:13:00.853660+0000","last_became_peered":"2024-09-19T08:13:00.853660+0000","last_unstale":"2024-09-19T08:14:06.726544+0000","last_undegraded":"2024-09-19T08:14:06.726544+0000","last_fullsized":"2024-09-19T08:14:06.726544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:35.602420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732530+0000","last_change":"2024-09-19T08:13:00.850658+0000","last_active":"2024-09-19T08:14:07.732530+0000","last_peered":"2024-09-19T08:14:07.732530+0000","last_clean":"2024-09-19T08:14:07.732530+0000","last_became_active":"2024-09-19T08:13:00.850324+0000","last_became_peered":"2024-09-19T08:13:00.850324+0000","last_unstale":"2024-09-19T08:14:07.732530+0000","last_undegraded":"2024-09-19T08:14:07.732530+0000","last_fullsized":"2024-09-19T08:14:07.732530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:59:49.004605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947899+0000","last_change":"2024-09-19T08:13:00.843421+0000","last_active":"2024-09-19T08:13:01.947899+0000","last_peered":"2024-09-19T08:13:01.947899+0000","last_clean":"2024-09-19T08:13:01.947899+0000","last_became_active":"2024-09-19T08:13:00.843109+0000","last_became_peered":"2024-09-19T08:13:00.843109+0000","last_unstale":"2024-09-19T08:13:01.947899+0000","last_undegraded":"2024-09-19T08:13:01.947899+0000","last_fullsized":"2024-09-19T08:13:01.947899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:50.522686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731582+0000","last_change":"2024-09-19T08:13:00.855215+0000","last_active":"2024-09-19T08:14:07.731582+0000","last_peered":"2024-09-19T08:14:07.731582+0000","last_clean":"2024-09-19T08:14:07.731582+0000","last_became_active":"2024-09-19T08:13:00.854878+0000","last_became_peered":"2024-09-19T08:13:00.854878+0000","last_unstale":"2024-09-19T08:14:07.731582+0000","last_undegraded":"2024-09-19T08:14:07.731582+0000","last_fullsized":"2024-09-19T08:14:07.731582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:48.216066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733505+0000","last_change":"2024-09-19T08:13:00.856995+0000","last_active":"2024-09-19T08:14:07.733505+0000","last_peered":"2024-09-19T08:14:07.733505+0000","last_clean":"2024-09-19T08:14:07.733505+0000","last_became_active":"2024-09-19T08:13:00.856645+0000","last_became_peered":"2024-09-19T08:13:00.856645+0000","last_unstale":"2024-09-19T08:14:07.733505+0000","last_undegraded":"2024-09-19T08:14:07.733505+0000","last_fullsized":"2024-09-19T08:14:07.733505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:20.699293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399539+0000","last_change":"2024-09-19T08:13:00.853362+0000","last_active":"2024-09-19T08:14:08.399539+0000","last_peered":"2024-09-19T08:14:08.399539+0000","last_clean":"2024-09-19T08:14:08.399539+0000","last_became_active":"2024-09-19T08:13:00.853046+0000","last_became_peered":"2024-09-19T08:13:00.853046+0000","last_unstale":"2024-09-19T08:14:08.399539+0000","last_undegraded":"2024-09-19T08:14:08.399539+0000","last_fullsized":"2024-09-19T08:14:08.399539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736509+0000","last_change":"2024-09-19T08:13:00.845175+0000","last_active":"2024-09-19T08:14:07.736509+0000","last_peered":"2024-09-19T08:14:07.736509+0000","last_clean":"2024-09-19T08:14:07.736509+0000","last_became_active":"2024-09-19T08:13:00.844857+0000","last_became_peered":"2024-09-19T08:13:00.844857+0000","last_unstale":"2024-09-19T08:14:07.736509+0000","last_undegraded":"2024-09-19T08:14:07.736509+0000","last_fullsized":"2024-09-19T08:14:07.736509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:40.818956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399481+0000","last_change":"2024-09-19T08:13:00.847769+0000","last_active":"2024-09-19T08:14:08.399481+0000","last_peered":"2024-09-19T08:14:08.399481+0000","last_clean":"2024-09-19T08:14:08.399481+0000","last_became_active":"2024-09-19T08:13:00.847454+0000","last_became_peered":"2024-09-19T08:13:00.847454+0000","last_unstale":"2024-09-19T08:14:08.399481+0000","last_undegraded":"2024-09-19T08:14:08.399481+0000","last_fullsized":"2024-09-19T08:14:08.399481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.583051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732671+0000","last_change":"2024-09-19T08:13:00.859882+0000","last_active":"2024-09-19T08:14:07.732671+0000","last_peered":"2024-09-19T08:14:07.732671+0000","last_clean":"2024-09-19T08:14:07.732671+0000","last_became_active":"2024-09-19T08:13:00.859491+0000","last_became_peered":"2024-09-19T08:13:00.859491+0000","last_unstale":"2024-09-19T08:14:07.732671+0000","last_undegraded":"2024-09-19T08:14:07.732671+0000","last_fullsized":"2024-09-19T08:14:07.732671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:13.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948381+0000","last_change":"2024-09-19T08:13:00.845584+0000","last_active":"2024-09-19T08:13:01.948381+0000","last_peered":"2024-09-19T08:13:01.948381+0000","last_clean":"2024-09-19T08:13:01.948381+0000","last_became_active":"2024-09-19T08:13:00.845166+0000","last_became_peered":"2024-09-19T08:13:00.845166+0000","last_unstale":"2024-09-19T08:13:01.948381+0000","last_undegraded":"2024-09-19T08:13:01.948381+0000","last_fullsized":"2024-09-19T08:13:01.948381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:02.705793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731667+0000","last_change":"2024-09-19T08:13:00.838069+0000","last_active":"2024-09-19T08:14:07.731667+0000","last_peered":"2024-09-19T08:14:07.731667+0000","last_clean":"2024-09-19T08:14:07.731667+0000","last_became_active":"2024-09-19T08:13:00.837736+0000","last_became_peered":"2024-09-19T08:13:00.837736+0000","last_unstale":"2024-09-19T08:14:07.731667+0000","last_undegraded":"2024-09-19T08:14:07.731667+0000","last_fullsized":"2024-09-19T08:14:07.731667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:01.388290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733793+0000","last_change":"2024-09-19T08:13:00.841507+0000","last_active":"2024-09-19T08:14:07.733793+0000","last_peered":"2024-09-19T08:14:07.733793+0000","last_clean":"2024-09-19T08:14:07.733793+0000","last_became_active":"2024-09-19T08:13:00.841161+0000","last_became_peered":"2024-09-19T08:13:00.841161+0000","last_unstale":"2024-09-19T08:14:07.733793+0000","last_undegraded":"2024-09-19T08:14:07.733793+0000","last_fullsized":"2024-09-19T08:14:07.733793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:41.862290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732148+0000","last_change":"2024-09-19T08:13:00.847657+0000","last_active":"2024-09-19T08:14:07.732148+0000","last_peered":"2024-09-19T08:14:07.732148+0000","last_clean":"2024-09-19T08:14:07.732148+0000","last_became_active":"2024-09-19T08:13:00.846849+0000","last_became_peered":"2024-09-19T08:13:00.846849+0000","last_unstale":"2024-09-19T08:14:07.732148+0000","last_undegraded":"2024-09-19T08:14:07.732148+0000","last_fullsized":"2024-09-19T08:14:07.732148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:35.219913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732183+0000","last_change":"2024-09-19T08:13:00.845971+0000","last_active":"2024-09-19T08:14:07.732183+0000","last_peered":"2024-09-19T08:14:07.732183+0000","last_clean":"2024-09-19T08:14:07.732183+0000","last_became_active":"2024-09-19T08:13:00.845626+0000","last_became_peered":"2024-09-19T08:13:00.845626+0000","last_unstale":"2024-09-19T08:14:07.732183+0000","last_undegraded":"2024-09-19T08:14:07.732183+0000","last_fullsized":"2024-09-19T08:14:07.732183+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:36.945627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947938+0000","last_change":"2024-09-19T08:13:00.853851+0000","last_active":"2024-09-19T08:13:01.947938+0000","last_peered":"2024-09-19T08:13:01.947938+0000","last_clean":"2024-09-19T08:13:01.947938+0000","last_became_active":"2024-09-19T08:13:00.853189+0000","last_became_peered":"2024-09-19T08:13:00.853189+0000","last_unstale":"2024-09-19T08:13:01.947938+0000","last_undegraded":"2024-09-19T08:13:01.947938+0000","last_fullsized":"2024-09-19T08:13:01.947938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:04.812522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.946963+0000","last_change":"2024-09-19T08:13:00.848325+0000","last_active":"2024-09-19T08:13:01.946963+0000","last_peered":"2024-09-19T08:13:01.946963+0000","last_clean":"2024-09-19T08:13:01.946963+0000","last_became_active":"2024-09-19T08:13:00.848015+0000","last_became_peered":"2024-09-19T08:13:00.848015+0000","last_unstale":"2024-09-19T08:13:01.946963+0000","last_undegraded":"2024-09-19T08:13:01.946963+0000","last_fullsized":"2024-09-19T08:13:01.946963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:10.948451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":90,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.961140+0000","last_change":"2024-09-19T08:12:58.681134+0000","last_active":"2024-09-19T08:14:08.961140+0000","last_peered":"2024-09-19T08:14:08.961140+0000","last_clean":"2024-09-19T08:14:08.961140+0000","last_became_active":"2024-09-19T08:12:58.680764+0000","last_became_peered":"2024-09-19T08:12:58.680764+0000","last_unstale":"2024-09-19T08:14:08.961140+0000","last_undegraded":"2024-09-19T08:14:08.961140+0000","last_fullsized":"2024-09-19T08:14:08.961140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:24.939263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736894+0000","last_change":"2024-09-19T08:13:00.844319+0000","last_active":"2024-09-19T08:14:07.736894+0000","last_peered":"2024-09-19T08:14:07.736894+0000","last_clean":"2024-09-19T08:14:07.736894+0000","last_became_active":"2024-09-19T08:13:00.843994+0000","last_became_peered":"2024-09-19T08:13:00.843994+0000","last_unstale":"2024-09-19T08:14:07.736894+0000","last_undegraded":"2024-09-19T08:14:07.736894+0000","last_fullsized":"2024-09-19T08:14:07.736894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:46.937466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735608+0000","last_change":"2024-09-19T08:12:58.676541+0000","last_active":"2024-09-19T08:14:07.735608+0000","last_peered":"2024-09-19T08:14:07.735608+0000","last_clean":"2024-09-19T08:14:07.735608+0000","last_became_active":"2024-09-19T08:12:58.675988+0000","last_became_peered":"2024-09-19T08:12:58.675988+0000","last_unstale":"2024-09-19T08:14:07.735608+0000","last_undegraded":"2024-09-19T08:14:07.735608+0000","last_fullsized":"2024-09-19T08:14:07.735608+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:01.434986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731256+0000","last_change":"2024-09-19T08:12:58.677510+0000","last_active":"2024-09-19T08:14:07.731256+0000","last_peered":"2024-09-19T08:14:07.731256+0000","last_clean":"2024-09-19T08:14:07.731256+0000","last_became_active":"2024-09-19T08:12:58.677200+0000","last_became_peered":"2024-09-19T08:12:58.677200+0000","last_unstale":"2024-09-19T08:14:07.731256+0000","last_undegraded":"2024-09-19T08:14:07.731256+0000","last_fullsized":"2024-09-19T08:14:07.731256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:36.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399481+0000","last_change":"2024-09-19T08:13:00.853430+0000","last_active":"2024-09-19T08:14:08.399481+0000","last_peered":"2024-09-19T08:14:08.399481+0000","last_clean":"2024-09-19T08:14:08.399481+0000","last_became_active":"2024-09-19T08:13:00.853096+0000","last_became_peered":"2024-09-19T08:13:00.853096+0000","last_unstale":"2024-09-19T08:14:08.399481+0000","last_undegraded":"2024-09-19T08:14:08.399481+0000","last_fullsized":"2024-09-19T08:14:08.399481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:07.147021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736819+0000","last_change":"2024-09-19T08:12:58.680081+0000","last_active":"2024-09-19T08:14:07.736819+0000","last_peered":"2024-09-19T08:14:07.736819+0000","last_clean":"2024-09-19T08:14:07.736819+0000","last_became_active":"2024-09-19T08:12:58.679704+0000","last_became_peered":"2024-09-19T08:12:58.679704+0000","last_unstale":"2024-09-19T08:14:07.736819+0000","last_undegraded":"2024-09-19T08:14:07.736819+0000","last_fullsized":"2024-09-19T08:14:07.736819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:32.942586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726249+0000","last_change":"2024-09-19T08:13:00.849394+0000","last_active":"2024-09-19T08:14:06.726249+0000","last_peered":"2024-09-19T08:14:06.726249+0000","last_clean":"2024-09-19T08:14:06.726249+0000","last_became_active":"2024-09-19T08:13:00.849051+0000","last_became_peered":"2024-09-19T08:13:00.849051+0000","last_unstale":"2024-09-19T08:14:06.726249+0000","last_undegraded":"2024-09-19T08:14:06.726249+0000","last_fullsized":"2024-09-19T08:14:06.726249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:23.012928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":84,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.960591+0000","last_change":"2024-09-19T08:12:58.676121+0000","last_active":"2024-09-19T08:14:08.960591+0000","last_peered":"2024-09-19T08:14:08.960591+0000","last_clean":"2024-09-19T08:14:08.960591+0000","last_became_active":"2024-09-19T08:12:58.675693+0000","last_became_peered":"2024-09-19T08:12:58.675693+0000","last_unstale":"2024-09-19T08:14:08.960591+0000","last_undegraded":"2024-09-19T08:14:08.960591+0000","last_fullsized":"2024-09-19T08:14:08.960591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:34.882190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733023+0000","last_change":"2024-09-19T08:13:00.858735+0000","last_active":"2024-09-19T08:14:07.733023+0000","last_peered":"2024-09-19T08:14:07.733023+0000","last_clean":"2024-09-19T08:14:07.733023+0000","last_became_active":"2024-09-19T08:13:00.858218+0000","last_became_peered":"2024-09-19T08:13:00.858218+0000","last_unstale":"2024-09-19T08:14:07.733023+0000","last_undegraded":"2024-09-19T08:14:07.733023+0000","last_fullsized":"2024-09-19T08:14:07.733023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:33.356810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.743785+0000","last_change":"2024-09-19T08:14:07.743785+0000","last_active":"2024-09-19T08:14:07.743785+0000","last_peered":"2024-09-19T08:14:07.743785+0000","last_clean":"2024-09-19T08:14:07.743785+0000","last_became_active":"2024-09-19T08:14:07.739531+0000","last_became_peered":"2024-09-19T08:14:07.739531+0000","last_unstale":"2024-09-19T08:14:07.743785+0000","last_undegraded":"2024-09-19T08:14:07.743785+0000","last_fullsized":"2024-09-19T08:14:07.743785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:43.367704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733696+0000","last_change":"2024-09-19T08:13:00.843852+0000","last_active":"2024-09-19T08:14:07.733696+0000","last_peered":"2024-09-19T08:14:07.733696+0000","last_clean":"2024-09-19T08:14:07.733696+0000","last_became_active":"2024-09-19T08:13:00.843497+0000","last_became_peered":"2024-09-19T08:13:00.843497+0000","last_unstale":"2024-09-19T08:14:07.733696+0000","last_undegraded":"2024-09-19T08:14:07.733696+0000","last_fullsized":"2024-09-19T08:14:07.733696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:26.380024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.737669+0000","last_change":"2024-09-19T08:12:58.717225+0000","last_active":"2024-09-19T08:14:07.737669+0000","last_peered":"2024-09-19T08:14:07.737669+0000","last_clean":"2024-09-19T08:14:07.737669+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:14:07.737669+0000","last_undegraded":"2024-09-19T08:14:07.737669+0000","last_fullsized":"2024-09-19T08:14:07.737669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:43.201907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733016+0000","last_change":"2024-09-19T08:13:00.845613+0000","last_active":"2024-09-19T08:14:07.733016+0000","last_peered":"2024-09-19T08:14:07.733016+0000","last_clean":"2024-09-19T08:14:07.733016+0000","last_became_active":"2024-09-19T08:13:00.845260+0000","last_became_peered":"2024-09-19T08:13:00.845260+0000","last_unstale":"2024-09-19T08:14:07.733016+0000","last_undegraded":"2024-09-19T08:14:07.733016+0000","last_fullsized":"2024-09-19T08:14:07.733016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:03:47.033972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'29","reported_seq":194,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:09.746400+0000","last_change":"2024-09-19T08:14:07.749616+0000","last_active":"2024-09-19T08:14:09.746400+0000","last_peered":"2024-09-19T08:14:09.746400+0000","last_clean":"2024-09-19T08:14:09.746400+0000","last_became_active":"2024-09-19T08:14:07.740298+0000","last_became_peered":"2024-09-19T08:14:07.740298+0000","last_unstale":"2024-09-19T08:14:09.746400+0000","last_undegraded":"2024-09-19T08:14:09.746400+0000","last_fullsized":"2024-09-19T08:14:09.746400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:28.796855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61172,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61172,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.947397+0000","last_change":"2024-09-19T08:13:00.855603+0000","last_active":"2024-09-19T08:13:01.947397+0000","last_peered":"2024-09-19T08:13:01.947397+0000","last_clean":"2024-09-19T08:13:01.947397+0000","last_became_active":"2024-09-19T08:13:00.855241+0000","last_became_peered":"2024-09-19T08:13:00.855241+0000","last_unstale":"2024-09-19T08:13:01.947397+0000","last_undegraded":"2024-09-19T08:13:01.947397+0000","last_fullsized":"2024-09-19T08:13:01.947397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:28.811847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732313+0000","last_change":"2024-09-19T08:12:58.675897+0000","last_active":"2024-09-19T08:14:07.732313+0000","last_peered":"2024-09-19T08:14:07.732313+0000","last_clean":"2024-09-19T08:14:07.732313+0000","last_became_active":"2024-09-19T08:12:58.675525+0000","last_became_peered":"2024-09-19T08:12:58.675525+0000","last_unstale":"2024-09-19T08:14:07.732313+0000","last_undegraded":"2024-09-19T08:14:07.732313+0000","last_fullsized":"2024-09-19T08:14:07.732313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:54.518064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731271+0000","last_change":"2024-09-19T08:13:00.856420+0000","last_active":"2024-09-19T08:14:07.731271+0000","last_peered":"2024-09-19T08:14:07.731271+0000","last_clean":"2024-09-19T08:14:07.731271+0000","last_became_active":"2024-09-19T08:13:00.856100+0000","last_became_peered":"2024-09-19T08:13:00.856100+0000","last_unstale":"2024-09-19T08:14:07.731271+0000","last_undegraded":"2024-09-19T08:14:07.731271+0000","last_fullsized":"2024-09-19T08:14:07.731271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:58.684930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.737378+0000","last_change":"2024-09-19T08:12:58.680778+0000","last_active":"2024-09-19T08:14:07.737378+0000","last_peered":"2024-09-19T08:14:07.737378+0000","last_clean":"2024-09-19T08:14:07.737378+0000","last_became_active":"2024-09-19T08:12:58.680364+0000","last_became_peered":"2024-09-19T08:12:58.680364+0000","last_unstale":"2024-09-19T08:14:07.737378+0000","last_undegraded":"2024-09-19T08:14:07.737378+0000","last_fullsized":"2024-09-19T08:14:07.737378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:10.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734418+0000","last_change":"2024-09-19T08:13:00.853415+0000","last_active":"2024-09-19T08:14:07.734418+0000","last_peered":"2024-09-19T08:14:07.734418+0000","last_clean":"2024-09-19T08:14:07.734418+0000","last_became_active":"2024-09-19T08:13:00.853101+0000","last_became_peered":"2024-09-19T08:13:00.853101+0000","last_unstale":"2024-09-19T08:14:07.734418+0000","last_undegraded":"2024-09-19T08:14:07.734418+0000","last_fullsized":"2024-09-19T08:14:07.734418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:42.375394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":87,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.959690+0000","last_change":"2024-09-19T08:12:58.680312+0000","last_active":"2024-09-19T08:14:07.959690+0000","last_peered":"2024-09-19T08:14:07.959690+0000","last_clean":"2024-09-19T08:14:07.959690+0000","last_became_active":"2024-09-19T08:12:58.679870+0000","last_became_peered":"2024-09-19T08:12:58.679870+0000","last_unstale":"2024-09-19T08:14:07.959690+0000","last_undegraded":"2024-09-19T08:14:07.959690+0000","last_fullsized":"2024-09-19T08:14:07.959690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:42.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734408+0000","last_change":"2024-09-19T08:13:00.853920+0000","last_active":"2024-09-19T08:14:07.734408+0000","last_peered":"2024-09-19T08:14:07.734408+0000","last_clean":"2024-09-19T08:14:07.734408+0000","last_became_active":"2024-09-19T08:13:00.853594+0000","last_became_peered":"2024-09-19T08:13:00.853594+0000","last_unstale":"2024-09-19T08:14:07.734408+0000","last_undegraded":"2024-09-19T08:14:07.734408+0000","last_fullsized":"2024-09-19T08:14:07.734408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:31.746133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.882028+0000","last_change":"2024-09-19T08:12:58.717487+0000","last_active":"2024-09-19T08:13:10.882028+0000","last_peered":"2024-09-19T08:13:10.882028+0000","last_clean":"2024-09-19T08:13:10.882028+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:13:10.882028+0000","last_undegraded":"2024-09-19T08:13:10.882028+0000","last_fullsized":"2024-09-19T08:13:10.882028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:38.506051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.737002+0000","last_change":"2024-09-19T08:13:00.847061+0000","last_active":"2024-09-19T08:14:07.737002+0000","last_peered":"2024-09-19T08:14:07.737002+0000","last_clean":"2024-09-19T08:14:07.737002+0000","last_became_active":"2024-09-19T08:13:00.846742+0000","last_became_peered":"2024-09-19T08:13:00.846742+0000","last_unstale":"2024-09-19T08:14:07.737002+0000","last_undegraded":"2024-09-19T08:14:07.737002+0000","last_fullsized":"2024-09-19T08:14:07.737002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:21.785585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:11.877000+0000","last_change":"2024-09-19T08:12:58.679367+0000","last_active":"2024-09-19T08:13:11.877000+0000","last_peered":"2024-09-19T08:13:11.877000+0000","last_clean":"2024-09-19T08:13:11.877000+0000","last_became_active":"2024-09-19T08:12:58.679035+0000","last_became_peered":"2024-09-19T08:12:58.679035+0000","last_unstale":"2024-09-19T08:13:11.877000+0000","last_undegraded":"2024-09-19T08:13:11.877000+0000","last_fullsized":"2024-09-19T08:13:11.877000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:25.633215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948364+0000","last_change":"2024-09-19T08:13:00.844940+0000","last_active":"2024-09-19T08:13:01.948364+0000","last_peered":"2024-09-19T08:13:01.948364+0000","last_clean":"2024-09-19T08:13:01.948364+0000","last_became_active":"2024-09-19T08:13:00.844454+0000","last_became_peered":"2024-09-19T08:13:00.844454+0000","last_unstale":"2024-09-19T08:13:01.948364+0000","last_undegraded":"2024-09-19T08:13:01.948364+0000","last_fullsized":"2024-09-19T08:13:01.948364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:15.577624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731644+0000","last_change":"2024-09-19T08:12:58.673301+0000","last_active":"2024-09-19T08:14:07.731644+0000","last_peered":"2024-09-19T08:14:07.731644+0000","last_clean":"2024-09-19T08:14:07.731644+0000","last_became_active":"2024-09-19T08:12:58.672981+0000","last_became_peered":"2024-09-19T08:12:58.672981+0000","last_unstale":"2024-09-19T08:14:07.731644+0000","last_undegraded":"2024-09-19T08:14:07.731644+0000","last_fullsized":"2024-09-19T08:14:07.731644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:03.963712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735037+0000","last_change":"2024-09-19T08:13:00.856620+0000","last_active":"2024-09-19T08:14:07.735037+0000","last_peered":"2024-09-19T08:14:07.735037+0000","last_clean":"2024-09-19T08:14:07.735037+0000","last_became_active":"2024-09-19T08:13:00.856310+0000","last_became_peered":"2024-09-19T08:13:00.856310+0000","last_unstale":"2024-09-19T08:14:07.735037+0000","last_undegraded":"2024-09-19T08:14:07.735037+0000","last_fullsized":"2024-09-19T08:14:07.735037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:16.918768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.737741+0000","last_change":"2024-09-19T08:12:58.672282+0000","last_active":"2024-09-19T08:14:07.737741+0000","last_peered":"2024-09-19T08:14:07.737741+0000","last_clean":"2024-09-19T08:14:07.737741+0000","last_became_active":"2024-09-19T08:12:58.671891+0000","last_became_peered":"2024-09-19T08:12:58.671891+0000","last_unstale":"2024-09-19T08:14:07.737741+0000","last_undegraded":"2024-09-19T08:14:07.737741+0000","last_fullsized":"2024-09-19T08:14:07.737741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:41.731571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734550+0000","last_change":"2024-09-19T08:13:00.848107+0000","last_active":"2024-09-19T08:14:07.734550+0000","last_peered":"2024-09-19T08:14:07.734550+0000","last_clean":"2024-09-19T08:14:07.734550+0000","last_became_active":"2024-09-19T08:13:00.847632+0000","last_became_peered":"2024-09-19T08:13:00.847632+0000","last_unstale":"2024-09-19T08:14:07.734550+0000","last_undegraded":"2024-09-19T08:14:07.734550+0000","last_fullsized":"2024-09-19T08:14:07.734550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.491122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732531+0000","last_change":"2024-09-19T08:12:59.179200+0000","last_active":"2024-09-19T08:14:07.732531+0000","last_peered":"2024-09-19T08:14:07.732531+0000","last_clean":"2024-09-19T08:14:07.732531+0000","last_became_active":"2024-09-19T08:12:59.178762+0000","last_became_peered":"2024-09-19T08:12:59.178762+0000","last_unstale":"2024-09-19T08:14:07.732531+0000","last_undegraded":"2024-09-19T08:14:07.732531+0000","last_fullsized":"2024-09-19T08:14:07.732531+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:08.424106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734839+0000","last_change":"2024-09-19T08:13:00.855870+0000","last_active":"2024-09-19T08:14:07.734839+0000","last_peered":"2024-09-19T08:14:07.734839+0000","last_clean":"2024-09-19T08:14:07.734839+0000","last_became_active":"2024-09-19T08:13:00.855538+0000","last_became_peered":"2024-09-19T08:13:00.855538+0000","last_unstale":"2024-09-19T08:14:07.734839+0000","last_undegraded":"2024-09-19T08:14:07.734839+0000","last_fullsized":"2024-09-19T08:14:07.734839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734771+0000","last_change":"2024-09-19T08:11:48.661333+0000","last_active":"2024-09-19T08:14:07.734771+0000","last_peered":"2024-09-19T08:14:07.734771+0000","last_clean":"2024-09-19T08:14:07.734771+0000","last_became_active":"2024-09-19T08:11:48.660873+0000","last_became_peered":"2024-09-19T08:11:48.660873+0000","last_unstale":"2024-09-19T08:14:07.734771+0000","last_undegraded":"2024-09-19T08:14:07.734771+0000","last_fullsized":"2024-09-19T08:14:07.734771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_clean_scrub_stamp":"2024-09-19T08:08:25.372155+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:19.401292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738561+0000","last_change":"2024-09-19T08:12:58.675237+0000","last_active":"2024-09-19T08:14:07.738561+0000","last_peered":"2024-09-19T08:14:07.738561+0000","last_clean":"2024-09-19T08:14:07.738561+0000","last_became_active":"2024-09-19T08:12:58.674910+0000","last_became_peered":"2024-09-19T08:12:58.674910+0000","last_unstale":"2024-09-19T08:14:07.738561+0000","last_undegraded":"2024-09-19T08:14:07.738561+0000","last_fullsized":"2024-09-19T08:14:07.738561+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:47.038328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399776+0000","last_change":"2024-09-19T08:13:00.846754+0000","last_active":"2024-09-19T08:14:08.399776+0000","last_peered":"2024-09-19T08:14:08.399776+0000","last_clean":"2024-09-19T08:14:08.399776+0000","last_became_active":"2024-09-19T08:13:00.846452+0000","last_became_peered":"2024-09-19T08:13:00.846452+0000","last_unstale":"2024-09-19T08:14:08.399776+0000","last_undegraded":"2024-09-19T08:14:08.399776+0000","last_fullsized":"2024-09-19T08:14:08.399776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:22.929689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734639+0000","last_change":"2024-09-19T08:12:58.678958+0000","last_active":"2024-09-19T08:14:07.734639+0000","last_peered":"2024-09-19T08:14:07.734639+0000","last_clean":"2024-09-19T08:14:07.734639+0000","last_became_active":"2024-09-19T08:12:58.678592+0000","last_became_peered":"2024-09-19T08:12:58.678592+0000","last_unstale":"2024-09-19T08:14:07.734639+0000","last_undegraded":"2024-09-19T08:14:07.734639+0000","last_fullsized":"2024-09-19T08:14:07.734639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:58.374606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731205+0000","last_change":"2024-09-19T08:13:00.840973+0000","last_active":"2024-09-19T08:14:07.731205+0000","last_peered":"2024-09-19T08:14:07.731205+0000","last_clean":"2024-09-19T08:14:07.731205+0000","last_became_active":"2024-09-19T08:13:00.840619+0000","last_became_peered":"2024-09-19T08:13:00.840619+0000","last_unstale":"2024-09-19T08:14:07.731205+0000","last_undegraded":"2024-09-19T08:14:07.731205+0000","last_fullsized":"2024-09-19T08:14:07.731205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":85,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.960649+0000","last_change":"2024-09-19T08:12:58.715972+0000","last_active":"2024-09-19T08:14:08.960649+0000","last_peered":"2024-09-19T08:14:08.960649+0000","last_clean":"2024-09-19T08:14:08.960649+0000","last_became_active":"2024-09-19T08:12:58.715630+0000","last_became_peered":"2024-09-19T08:12:58.715630+0000","last_unstale":"2024-09-19T08:14:08.960649+0000","last_undegraded":"2024-09-19T08:14:08.960649+0000","last_fullsized":"2024-09-19T08:14:08.960649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:35.637121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736616+0000","last_change":"2024-09-19T08:13:00.847023+0000","last_active":"2024-09-19T08:14:07.736616+0000","last_peered":"2024-09-19T08:14:07.736616+0000","last_clean":"2024-09-19T08:14:07.736616+0000","last_became_active":"2024-09-19T08:13:00.846387+0000","last_became_peered":"2024-09-19T08:13:00.846387+0000","last_unstale":"2024-09-19T08:14:07.736616+0000","last_undegraded":"2024-09-19T08:14:07.736616+0000","last_fullsized":"2024-09-19T08:14:07.736616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:12.187929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735037+0000","last_change":"2024-09-19T08:12:58.681062+0000","last_active":"2024-09-19T08:14:07.735037+0000","last_peered":"2024-09-19T08:14:07.735037+0000","last_clean":"2024-09-19T08:14:07.735037+0000","last_became_active":"2024-09-19T08:12:58.680524+0000","last_became_peered":"2024-09-19T08:12:58.680524+0000","last_unstale":"2024-09-19T08:14:07.735037+0000","last_undegraded":"2024-09-19T08:14:07.735037+0000","last_fullsized":"2024-09-19T08:14:07.735037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735930+0000","last_change":"2024-09-19T08:13:00.851523+0000","last_active":"2024-09-19T08:14:07.735930+0000","last_peered":"2024-09-19T08:14:07.735930+0000","last_clean":"2024-09-19T08:14:07.735930+0000","last_became_active":"2024-09-19T08:13:00.851068+0000","last_became_peered":"2024-09-19T08:13:00.851068+0000","last_unstale":"2024-09-19T08:14:07.735930+0000","last_undegraded":"2024-09-19T08:14:07.735930+0000","last_fullsized":"2024-09-19T08:14:07.735930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:44.905469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726509+0000","last_change":"2024-09-19T08:12:58.718234+0000","last_active":"2024-09-19T08:14:06.726509+0000","last_peered":"2024-09-19T08:14:06.726509+0000","last_clean":"2024-09-19T08:14:06.726509+0000","last_became_active":"2024-09-19T08:12:58.717840+0000","last_became_peered":"2024-09-19T08:12:58.717840+0000","last_unstale":"2024-09-19T08:14:06.726509+0000","last_undegraded":"2024-09-19T08:14:06.726509+0000","last_fullsized":"2024-09-19T08:14:06.726509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:05.283770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736201+0000","last_change":"2024-09-19T08:13:00.848305+0000","last_active":"2024-09-19T08:14:07.736201+0000","last_peered":"2024-09-19T08:14:07.736201+0000","last_clean":"2024-09-19T08:14:07.736201+0000","last_became_active":"2024-09-19T08:13:00.847981+0000","last_became_peered":"2024-09-19T08:13:00.847981+0000","last_unstale":"2024-09-19T08:14:07.736201+0000","last_undegraded":"2024-09-19T08:14:07.736201+0000","last_fullsized":"2024-09-19T08:14:07.736201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:04.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.871966+0000","last_change":"2024-09-19T08:12:58.715943+0000","last_active":"2024-09-19T08:13:09.871966+0000","last_peered":"2024-09-19T08:13:09.871966+0000","last_clean":"2024-09-19T08:13:09.871966+0000","last_became_active":"2024-09-19T08:12:58.715600+0000","last_became_peered":"2024-09-19T08:12:58.715600+0000","last_unstale":"2024-09-19T08:13:09.871966+0000","last_undegraded":"2024-09-19T08:13:09.871966+0000","last_fullsized":"2024-09-19T08:13:09.871966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:19.142271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731759+0000","last_change":"2024-09-19T08:13:00.840944+0000","last_active":"2024-09-19T08:14:07.731759+0000","last_peered":"2024-09-19T08:14:07.731759+0000","last_clean":"2024-09-19T08:14:07.731759+0000","last_became_active":"2024-09-19T08:13:00.840608+0000","last_became_peered":"2024-09-19T08:13:00.840608+0000","last_unstale":"2024-09-19T08:14:07.731759+0000","last_undegraded":"2024-09-19T08:14:07.731759+0000","last_fullsized":"2024-09-19T08:14:07.731759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:25.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'11","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726648+0000","last_change":"2024-09-19T08:12:58.672929+0000","last_active":"2024-09-19T08:14:06.726648+0000","last_peered":"2024-09-19T08:14:06.726648+0000","last_clean":"2024-09-19T08:14:06.726648+0000","last_became_active":"2024-09-19T08:12:58.672487+0000","last_became_peered":"2024-09-19T08:12:58.672487+0000","last_unstale":"2024-09-19T08:14:06.726648+0000","last_undegraded":"2024-09-19T08:14:06.726648+0000","last_fullsized":"2024-09-19T08:14:06.726648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:33.042735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":2,"reported_epoch":84,"state":"peering","last_fresh":"2024-09-19T08:13:04.156818+0000","last_change":"2024-09-19T08:14:06.939357+0000","last_active":"2024-09-19T08:14:06.939357+0000","last_peered":"2024-09-19T08:13:04.156818+0000","last_clean":"2024-09-19T08:13:04.156818+0000","last_became_active":"2024-09-19T08:13:00.853159+0000","last_became_peered":"2024-09-19T08:13:00.853159+0000","last_unstale":"2024-09-19T08:13:04.156818+0000","last_undegraded":"2024-09-19T08:13:04.156818+0000","last_fullsized":"2024-09-19T08:13:04.156818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.739005+0000","last_change":"2024-09-19T08:12:59.179510+0000","last_active":"2024-09-19T08:14:07.739005+0000","last_peered":"2024-09-19T08:14:07.739005+0000","last_clean":"2024-09-19T08:14:07.739005+0000","last_became_active":"2024-09-19T08:12:59.179157+0000","last_became_peered":"2024-09-19T08:12:59.179157+0000","last_unstale":"2024-09-19T08:14:07.739005+0000","last_undegraded":"2024-09-19T08:14:07.739005+0000","last_fullsized":"2024-09-19T08:14:07.739005+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:22.981289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733930+0000","last_change":"2024-09-19T08:13:00.841853+0000","last_active":"2024-09-19T08:14:07.733930+0000","last_peered":"2024-09-19T08:14:07.733930+0000","last_clean":"2024-09-19T08:14:07.733930+0000","last_became_active":"2024-09-19T08:13:00.841518+0000","last_became_peered":"2024-09-19T08:13:00.841518+0000","last_unstale":"2024-09-19T08:14:07.733930+0000","last_undegraded":"2024-09-19T08:14:07.733930+0000","last_fullsized":"2024-09-19T08:14:07.733930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:05.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733280+0000","last_change":"2024-09-19T08:12:58.716882+0000","last_active":"2024-09-19T08:14:07.733280+0000","last_peered":"2024-09-19T08:14:07.733280+0000","last_clean":"2024-09-19T08:14:07.733280+0000","last_became_active":"2024-09-19T08:12:58.716545+0000","last_became_peered":"2024-09-19T08:12:58.716545+0000","last_unstale":"2024-09-19T08:14:07.733280+0000","last_undegraded":"2024-09-19T08:14:07.733280+0000","last_fullsized":"2024-09-19T08:14:07.733280+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:14.469109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.946790+0000","last_change":"2024-09-19T08:13:00.850484+0000","last_active":"2024-09-19T08:13:01.946790+0000","last_peered":"2024-09-19T08:13:01.946790+0000","last_clean":"2024-09-19T08:13:01.946790+0000","last_became_active":"2024-09-19T08:13:00.850049+0000","last_became_peered":"2024-09-19T08:13:00.850049+0000","last_unstale":"2024-09-19T08:13:01.946790+0000","last_undegraded":"2024-09-19T08:13:01.946790+0000","last_fullsized":"2024-09-19T08:13:01.946790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:24.424111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399703+0000","last_change":"2024-09-19T08:12:58.669928+0000","last_active":"2024-09-19T08:14:08.399703+0000","last_peered":"2024-09-19T08:14:08.399703+0000","last_clean":"2024-09-19T08:14:08.399703+0000","last_became_active":"2024-09-19T08:12:58.669614+0000","last_became_peered":"2024-09-19T08:12:58.669614+0000","last_unstale":"2024-09-19T08:14:08.399703+0000","last_undegraded":"2024-09-19T08:14:08.399703+0000","last_fullsized":"2024-09-19T08:14:08.399703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:50.924779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:01.948102+0000","last_change":"2024-09-19T08:13:00.853520+0000","last_active":"2024-09-19T08:13:01.948102+0000","last_peered":"2024-09-19T08:13:01.948102+0000","last_clean":"2024-09-19T08:13:01.948102+0000","last_became_active":"2024-09-19T08:13:00.853184+0000","last_became_peered":"2024-09-19T08:13:00.853184+0000","last_unstale":"2024-09-19T08:13:01.948102+0000","last_undegraded":"2024-09-19T08:13:01.948102+0000","last_fullsized":"2024-09-19T08:13:01.948102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:45.160439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732133+0000","last_change":"2024-09-19T08:12:58.674359+0000","last_active":"2024-09-19T08:14:07.732133+0000","last_peered":"2024-09-19T08:14:07.732133+0000","last_clean":"2024-09-19T08:14:07.732133+0000","last_became_active":"2024-09-19T08:12:58.673890+0000","last_became_peered":"2024-09-19T08:12:58.673890+0000","last_unstale":"2024-09-19T08:14:07.732133+0000","last_undegraded":"2024-09-19T08:14:07.732133+0000","last_fullsized":"2024-09-19T08:14:07.732133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:54.939959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399978+0000","last_change":"2024-09-19T08:13:00.851071+0000","last_active":"2024-09-19T08:14:08.399978+0000","last_peered":"2024-09-19T08:14:08.399978+0000","last_clean":"2024-09-19T08:14:08.399978+0000","last_became_active":"2024-09-19T08:13:00.850756+0000","last_became_peered":"2024-09-19T08:13:00.850756+0000","last_unstale":"2024-09-19T08:14:08.399978+0000","last_undegraded":"2024-09-19T08:14:08.399978+0000","last_fullsized":"2024-09-19T08:14:08.399978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:52.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:09.872854+0000","last_change":"2024-09-19T08:12:58.681466+0000","last_active":"2024-09-19T08:13:09.872854+0000","last_peered":"2024-09-19T08:13:09.872854+0000","last_clean":"2024-09-19T08:13:09.872854+0000","last_became_active":"2024-09-19T08:12:58.681113+0000","last_became_peered":"2024-09-19T08:12:58.681113+0000","last_unstale":"2024-09-19T08:13:09.872854+0000","last_undegraded":"2024-09-19T08:13:09.872854+0000","last_fullsized":"2024-09-19T08:13:09.872854+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:49.610828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734635+0000","last_change":"2024-09-19T08:13:00.851272+0000","last_active":"2024-09-19T08:14:07.734635+0000","last_peered":"2024-09-19T08:14:07.734635+0000","last_clean":"2024-09-19T08:14:07.734635+0000","last_became_active":"2024-09-19T08:13:00.849098+0000","last_became_peered":"2024-09-19T08:13:00.849098+0000","last_unstale":"2024-09-19T08:14:07.734635+0000","last_undegraded":"2024-09-19T08:14:07.734635+0000","last_fullsized":"2024-09-19T08:14:07.734635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:05.643992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738913+0000","last_change":"2024-09-19T08:14:07.738913+0000","last_active":"2024-09-19T08:14:07.738913+0000","last_peered":"2024-09-19T08:14:07.738913+0000","last_clean":"2024-09-19T08:14:07.738913+0000","last_became_active":"2024-09-19T08:14:07.738560+0000","last_became_peered":"2024-09-19T08:14:07.738560+0000","last_unstale":"2024-09-19T08:14:07.738913+0000","last_undegraded":"2024-09-19T08:14:07.738913+0000","last_fullsized":"2024-09-19T08:14:07.738913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:35.731290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:02.272564+0000","last_change":"2024-09-19T08:12:58.674610+0000","last_active":"2024-09-19T08:13:02.272564+0000","last_peered":"2024-09-19T08:13:02.272564+0000","last_clean":"2024-09-19T08:13:02.272564+0000","last_became_active":"2024-09-19T08:12:58.674247+0000","last_became_peered":"2024-09-19T08:12:58.674247+0000","last_unstale":"2024-09-19T08:13:02.272564+0000","last_undegraded":"2024-09-19T08:13:02.272564+0000","last_fullsized":"2024-09-19T08:13:02.272564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:16.323574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732898+0000","last_change":"2024-09-19T08:13:00.858512+0000","last_active":"2024-09-19T08:14:07.732898+0000","last_peered":"2024-09-19T08:14:07.732898+0000","last_clean":"2024-09-19T08:14:07.732898+0000","last_became_active":"2024-09-19T08:13:00.857897+0000","last_became_peered":"2024-09-19T08:13:00.857897+0000","last_unstale":"2024-09-19T08:14:07.732898+0000","last_undegraded":"2024-09-19T08:14:07.732898+0000","last_fullsized":"2024-09-19T08:14:07.732898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:13:10.881153+0000","last_change":"2024-09-19T08:12:58.682207+0000","last_active":"2024-09-19T08:13:10.881153+0000","last_peered":"2024-09-19T08:13:10.881153+0000","last_clean":"2024-09-19T08:13:10.881153+0000","last_became_active":"2024-09-19T08:12:58.681842+0000","last_became_peered":"2024-09-19T08:12:58.681842+0000","last_unstale":"2024-09-19T08:13:10.881153+0000","last_undegraded":"2024-09-19T08:13:10.881153+0000","last_fullsized":"2024-09-19T08:13:10.881153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.825858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733650+0000","last_change":"2024-09-19T08:13:00.846748+0000","last_active":"2024-09-19T08:14:07.733650+0000","last_peered":"2024-09-19T08:14:07.733650+0000","last_clean":"2024-09-19T08:14:07.733650+0000","last_became_active":"2024-09-19T08:13:00.846368+0000","last_became_peered":"2024-09-19T08:13:00.846368+0000","last_unstale":"2024-09-19T08:14:07.733650+0000","last_undegraded":"2024-09-19T08:14:07.733650+0000","last_fullsized":"2024-09-19T08:14:07.733650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:59.526512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.400239+0000","last_change":"2024-09-19T08:12:58.716118+0000","last_active":"2024-09-19T08:14:08.400239+0000","last_peered":"2024-09-19T08:14:08.400239+0000","last_clean":"2024-09-19T08:14:08.400239+0000","last_became_active":"2024-09-19T08:12:58.715711+0000","last_became_peered":"2024-09-19T08:12:58.715711+0000","last_unstale":"2024-09-19T08:14:08.400239+0000","last_undegraded":"2024-09-19T08:14:08.400239+0000","last_fullsized":"2024-09-19T08:14:08.400239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:29.439887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.400041+0000","last_change":"2024-09-19T08:13:00.848729+0000","last_active":"2024-09-19T08:14:08.400041+0000","last_peered":"2024-09-19T08:14:08.400041+0000","last_clean":"2024-09-19T08:14:08.400041+0000","last_became_active":"2024-09-19T08:13:00.848470+0000","last_became_peered":"2024-09-19T08:13:00.848470+0000","last_unstale":"2024-09-19T08:14:08.400041+0000","last_undegraded":"2024-09-19T08:14:08.400041+0000","last_fullsized":"2024-09-19T08:14:08.400041+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:30.653101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T08:14:06.726774+0000","last_change":"2024-09-19T08:12:58.681654+0000","last_active":"2024-09-19T08:14:06.726774+0000","last_peered":"2024-09-19T08:14:06.726774+0000","last_clean":"2024-09-19T08:14:06.726774+0000","last_became_active":"2024-09-19T08:12:58.681284+0000","last_became_peered":"2024-09-19T08:12:58.681284+0000","last_unstale":"2024-09-19T08:14:06.726774+0000","last_undegraded":"2024-09-19T08:14:06.726774+0000","last_fullsized":"2024-09-19T08:14:06.726774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.429930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65568,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":372,"num_read_kb":255,"num_write":164,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":61172,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":196704,"data_compressed":9611,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251470,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481818,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":428609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710752,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1069056,"data_stored":1098844,"data_compressed":42203,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874277,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":428845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":909312,"data_stored":428765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332143,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":983040,"data_stored":1038279,"data_compressed":42203,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495668,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":921600,"data_stored":445423,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":995328,"data_stored":1037570,"data_compressed":42203,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":950272,"data_stored":507425,"data_compressed":15422,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":925696,"data_stored":437246,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":508131,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":983040,"data_stored":506388,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":438206,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34755,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34755,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":62209,"data_compressed":9611,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:14:12.741 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.7 is not active+clean 2024-09-19T08:14:12.741 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 84, 'state': 'peering', 'last_fresh': '2024-09-19T08:13:04.156818+0000', 'last_change': '2024-09-19T08:14:06.939357+0000', 'last_active': '2024-09-19T08:14:06.939357+0000', 'last_peered': '2024-09-19T08:13:04.156818+0000', 'last_clean': '2024-09-19T08:13:04.156818+0000', 'last_became_active': '2024-09-19T08:13:00.853159+0000', 'last_became_peered': '2024-09-19T08:13:00.853159+0000', 'last_unstale': '2024-09-19T08:13:04.156818+0000', 'last_undegraded': '2024-09-19T08:13:04.156818+0000', 'last_fullsized': '2024-09-19T08:13:04.156818+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 82, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:12:59.824436+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:12:59.824436+0000', 'last_clean_scrub_stamp': '2024-09-19T08:12:59.824436+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [9, 11, 7], 'acting': [9, 11, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 9, 'acting_primary': 9, 'purged_snaps': []} 2024-09-19T08:14:12.741 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph status --format=json 2024-09-19T08:14:13.005 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:13.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:12 smithi022 ceph-mon[30963]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 7.5 KiB/s, 0 objects/s recovering 2024-09-19T08:14:13.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:12 smithi022 ceph-mon[30963]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:13.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:12 smithi064 ceph-mon[34900]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 7.5 KiB/s, 0 objects/s recovering 2024-09-19T08:14:13.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:12 smithi064 ceph-mon[34900]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:12 smithi157 ceph-mon[35143]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 7.5 KiB/s, 0 objects/s recovering 2024-09-19T08:14:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:12 smithi157 ceph-mon[35143]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:13.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.464+0000 7ff06ca47640 1 -- 172.21.15.22:0/3111958228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068102790 msgr2=0x7ff0681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:13.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.464+0000 7ff06ca47640 1 --2- 172.21.15.22:0/3111958228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068102790 0x7ff0681067e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff050009900 tx=0x7ff05002a410 comp rx=0 tx=0).stop 2024-09-19T08:14:13.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.464+0000 7ff06ca47640 1 -- 172.21.15.22:0/3111958228 shutdown_connections 2024-09-19T08:14:13.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.464+0000 7ff06ca47640 1 --2- 172.21.15.22:0/3111958228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff068106f10 0x7ff06810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:13.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.464+0000 7ff06ca47640 1 --2- 172.21.15.22:0/3111958228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068102790 0x7ff0681067e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:13.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.464+0000 7ff06ca47640 1 --2- 172.21.15.22:0/3111958228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff068101de0 0x7ff0681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:13.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.464+0000 7ff06ca47640 1 -- 172.21.15.22:0/3111958228 >> 172.21.15.22:0/3111958228 conn(0x7ff0680fd8c0 msgr2=0x7ff0680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:13.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.464+0000 7ff06ca47640 1 -- 172.21.15.22:0/3111958228 shutdown_connections 2024-09-19T08:14:13.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.464+0000 7ff06ca47640 1 -- 172.21.15.22:0/3111958228 wait complete. 2024-09-19T08:14:13.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.465+0000 7ff06ca47640 1 Processor -- start 2024-09-19T08:14:13.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.465+0000 7ff06ca47640 1 -- start start 2024-09-19T08:14:13.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.465+0000 7ff06ca47640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068101de0 0x7ff0680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:13.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.465+0000 7ff06ca47640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff068102790 0x7ff068074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:13.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.465+0000 7ff06ca47640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff068106f10 0x7ff068074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:13.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.465+0000 7ff06ca47640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff068111f90 con 0x7ff068101de0 2024-09-19T08:14:13.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.465+0000 7ff06ca47640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff068111e10 con 0x7ff068106f10 2024-09-19T08:14:13.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.465+0000 7ff06ca47640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff068112110 con 0x7ff068102790 2024-09-19T08:14:13.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.465+0000 7ff06659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068101de0 0x7ff0680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:13.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff06659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068101de0 0x7ff0680744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60104/0 (socket says 172.21.15.22:60104) 2024-09-19T08:14:13.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff06659c640 1 -- 172.21.15.22:0/2081709351 learned_addr learned my addr 172.21.15.22:0/2081709351 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:13.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff066d9d640 1 --2- 172.21.15.22:0/2081709351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff068106f10 0x7ff068074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:13.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff065d9b640 1 --2- 172.21.15.22:0/2081709351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff068102790 0x7ff068074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff06659c640 1 -- 172.21.15.22:0/2081709351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff068102790 msgr2=0x7ff068074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff06659c640 1 --2- 172.21.15.22:0/2081709351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff068102790 0x7ff068074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff06659c640 1 -- 172.21.15.22:0/2081709351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff068106f10 msgr2=0x7ff068074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff06659c640 1 --2- 172.21.15.22:0/2081709351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff068106f10 0x7ff068074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff06659c640 1 -- 172.21.15.22:0/2081709351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0681a6950 con 0x7ff068101de0 2024-09-19T08:14:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff06659c640 1 --2- 172.21.15.22:0/2081709351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068101de0 0x7ff0680744f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff06000b4a0 tx=0x7ff06000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff05f7fe640 1 -- 172.21.15.22:0/2081709351 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060014070 con 0x7ff068101de0 2024-09-19T08:14:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff06ca47640 1 -- 172.21.15.22:0/2081709351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0681a6c40 con 0x7ff068101de0 2024-09-19T08:14:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff05f7fe640 1 -- 172.21.15.22:0/2081709351 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff060010af0 con 0x7ff068101de0 2024-09-19T08:14:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff05f7fe640 1 -- 172.21.15.22:0/2081709351 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060017d10 con 0x7ff068101de0 2024-09-19T08:14:13.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.466+0000 7ff06ca47640 1 -- 172.21.15.22:0/2081709351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0681a7130 con 0x7ff068101de0 2024-09-19T08:14:13.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.467+0000 7ff06ca47640 1 -- 172.21.15.22:0/2081709351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0681021c0 con 0x7ff068101de0 2024-09-19T08:14:13.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.469+0000 7ff05f7fe640 1 -- 172.21.15.22:0/2081709351 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff060017070 con 0x7ff068101de0 2024-09-19T08:14:13.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.469+0000 7ff05f7fe640 1 --2- 172.21.15.22:0/2081709351 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff0540a32d0 0x7ff0540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:13.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.470+0000 7ff065d9b640 1 --2- 172.21.15.22:0/2081709351 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff0540a32d0 0x7ff0540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:13.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.470+0000 7ff05f7fe640 1 -- 172.21.15.22:0/2081709351 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff0600ced60 con 0x7ff068101de0 2024-09-19T08:14:13.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.470+0000 7ff065d9b640 1 --2- 172.21.15.22:0/2081709351 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff0540a32d0 0x7ff0540a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff068075f80 tx=0x7ff050005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:13.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.474+0000 7ff05f7fe640 1 -- 172.21.15.22:0/2081709351 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff06009ba30 con 0x7ff068101de0 2024-09-19T08:14:13.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.718+0000 7ff06ca47640 1 -- 172.21.15.22:0/2081709351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ff06810a0f0 con 0x7ff068101de0 2024-09-19T08:14:13.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.718+0000 7ff05f7fe640 1 -- 172.21.15.22:0/2081709351 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2569 (secure 0 0 0) 0x7ff06009b330 con 0x7ff068101de0 2024-09-19T08:14:13.718 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:13.719 INFO:teuthology.orchestra.run.smithi022.stdout:{"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 1 pg peering","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":436,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":85,"num_osds":12,"num_up_osds":12,"osd_up_since":1726733530,"num_in_osds":12,"osd_in_since":1726733507,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":96},{"state_name":"peering","count":1}],"num_pgs":97,"num_pools":3,"num_objects":64,"data_bytes":656170,"bytes_used":390459392,"bytes_avail":1151549968384,"bytes_total":1151940427776,"inactive_pgs_ratio":0.010309278033673763,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":7644,"recovering_keys_per_sec":0,"num_objects_recovered":6,"num_bytes_recovered":61172,"num_keys_recovered":0,"read_bytes_sec":3199,"read_op_per_sec":6,"write_op_per_sec":0},"fsmap":{"epoch":24,"btime":"2024-09-19T08:13:13:877693+0000","id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24428},{"filesystem_id":1,"rank":1,"name":"f","status":"up:standby-replay","gid":24454},{"filesystem_id":1,"rank":0,"name":"h","status":"up:standby-replay","gid":24455},{"filesystem_id":1,"rank":2,"name":"c","status":"up:active","gid":24460},{"filesystem_id":1,"rank":1,"name":"b","status":"up:active","gid":24479},{"filesystem_id":1,"rank":0,"name":"i","status":"up:active","gid":24481}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.22:8443/"}},"servicemap":{"epoch":25,"modified":"2024-09-19T08:14:08.779062+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T08:14:13.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.721+0000 7ff06ca47640 1 -- 172.21.15.22:0/2081709351 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff0540a32d0 msgr2=0x7ff0540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:13.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.721+0000 7ff06ca47640 1 --2- 172.21.15.22:0/2081709351 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff0540a32d0 0x7ff0540a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff068075f80 tx=0x7ff050005a20 comp rx=0 tx=0).stop 2024-09-19T08:14:13.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.721+0000 7ff06ca47640 1 -- 172.21.15.22:0/2081709351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068101de0 msgr2=0x7ff0680744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:13.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.721+0000 7ff06ca47640 1 --2- 172.21.15.22:0/2081709351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068101de0 0x7ff0680744f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff06000b4a0 tx=0x7ff06000b960 comp rx=0 tx=0).stop 2024-09-19T08:14:13.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.721+0000 7ff06ca47640 1 -- 172.21.15.22:0/2081709351 shutdown_connections 2024-09-19T08:14:13.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.721+0000 7ff06ca47640 1 --2- 172.21.15.22:0/2081709351 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff0540a32d0 0x7ff0540a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:13.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.721+0000 7ff06ca47640 1 --2- 172.21.15.22:0/2081709351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff068106f10 0x7ff068074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:13.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.721+0000 7ff06ca47640 1 --2- 172.21.15.22:0/2081709351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff068102790 0x7ff068074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:13.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.721+0000 7ff06ca47640 1 --2- 172.21.15.22:0/2081709351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068101de0 0x7ff0680744f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:13.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.721+0000 7ff06ca47640 1 -- 172.21.15.22:0/2081709351 >> 172.21.15.22:0/2081709351 conn(0x7ff0680fd8c0 msgr2=0x7ff0680fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:13.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.721+0000 7ff06ca47640 1 -- 172.21.15.22:0/2081709351 shutdown_connections 2024-09-19T08:14:13.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:13.721+0000 7ff06ca47640 1 -- 172.21.15.22:0/2081709351 wait complete. 2024-09-19T08:14:14.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:13 smithi022 ceph-mon[30963]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 7.5 KiB/s, 1 objects/s recovering 2024-09-19T08:14:14.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:13 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2081709351' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-19T08:14:14.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:13 smithi064 ceph-mon[34900]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 7.5 KiB/s, 1 objects/s recovering 2024-09-19T08:14:14.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:13 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2081709351' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-19T08:14:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:13 smithi157 ceph-mon[35143]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 7.5 KiB/s, 1 objects/s recovering 2024-09-19T08:14:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:13 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2081709351' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-19T08:14:14.406 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-19T08:14:14.406 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:14:14.669 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:15.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:14 smithi064 ceph-mon[34900]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-19T08:14:15.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:14 smithi064 ceph-mon[34900]: Cluster is now healthy 2024-09-19T08:14:15.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:14 smithi022 ceph-mon[30963]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-19T08:14:15.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:14 smithi022 ceph-mon[30963]: Cluster is now healthy 2024-09-19T08:14:15.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.175+0000 7f6f39d85640 1 -- 172.21.15.22:0/1490407234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f34102570 msgr2=0x7f6f341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:15.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.175+0000 7f6f39d85640 1 --2- 172.21.15.22:0/1490407234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f34102570 0x7f6f341029b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009900 tx=0x7f6f2802a480 comp rx=0 tx=0).stop 2024-09-19T08:14:15.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.175+0000 7f6f39d85640 1 -- 172.21.15.22:0/1490407234 shutdown_connections 2024-09-19T08:14:15.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.175+0000 7f6f39d85640 1 --2- 172.21.15.22:0/1490407234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f34102ef0 0x7f6f3410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:15.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.175+0000 7f6f39d85640 1 --2- 172.21.15.22:0/1490407234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f34102570 0x7f6f341029b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:15.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.175+0000 7f6f39d85640 1 --2- 172.21.15.22:0/1490407234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f34100e20 0x7f6f34101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:15.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.175+0000 7f6f39d85640 1 -- 172.21.15.22:0/1490407234 >> 172.21.15.22:0/1490407234 conn(0x7f6f340fc7d0 msgr2=0x7f6f340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:15.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.175+0000 7f6f39d85640 1 -- 172.21.15.22:0/1490407234 shutdown_connections 2024-09-19T08:14:15.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.175+0000 7f6f39d85640 1 -- 172.21.15.22:0/1490407234 wait complete. 2024-09-19T08:14:15.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.175+0000 7f6f39d85640 1 Processor -- start 2024-09-19T08:14:15.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f39d85640 1 -- start start 2024-09-19T08:14:15.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f39d85640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f34100e20 0x7f6f3419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:15.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f39d85640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f34102570 0x7f6f3419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:15.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f39d85640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f34102ef0 0x7f6f3419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:15.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f39d85640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34110f80 con 0x7f6f34102ef0 2024-09-19T08:14:15.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f39d85640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34110e00 con 0x7f6f34102570 2024-09-19T08:14:15.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f39d85640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34111100 con 0x7f6f34100e20 2024-09-19T08:14:15.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f33fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f34102ef0 0x7f6f3419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:15.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f33fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f34102ef0 0x7f6f3419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60126/0 (socket says 172.21.15.22:60126) 2024-09-19T08:14:15.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f33fff640 1 -- 172.21.15.22:0/3634240404 learned_addr learned my addr 172.21.15.22:0/3634240404 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:15.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f33fff640 1 -- 172.21.15.22:0/3634240404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f34100e20 msgr2=0x7f6f3419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:15.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f337fe640 1 --2- 172.21.15.22:0/3634240404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f34100e20 0x7f6f3419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:15.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f32ffd640 1 --2- 172.21.15.22:0/3634240404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f34102570 0x7f6f3419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:15.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f33fff640 1 --2- 172.21.15.22:0/3634240404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f34100e20 0x7f6f3419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:15.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f33fff640 1 -- 172.21.15.22:0/3634240404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f34102570 msgr2=0x7f6f3419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:15.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f33fff640 1 --2- 172.21.15.22:0/3634240404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f34102570 0x7f6f3419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:15.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f33fff640 1 -- 172.21.15.22:0/3634240404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3419f640 con 0x7f6f34102ef0 2024-09-19T08:14:15.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.176+0000 7f6f33fff640 1 --2- 172.21.15.22:0/3634240404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f34102ef0 0x7f6f3419ef60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00eaa0 tx=0x7f6f2c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:15.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.177+0000 7f6f30ff9640 1 -- 172.21.15.22:0/3634240404 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c00cdb0 con 0x7f6f34102ef0 2024-09-19T08:14:15.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.177+0000 7f6f39d85640 1 -- 172.21.15.22:0/3634240404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3419f930 con 0x7f6f34102ef0 2024-09-19T08:14:15.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.177+0000 7f6f30ff9640 1 -- 172.21.15.22:0/3634240404 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f2c005020 con 0x7f6f34102ef0 2024-09-19T08:14:15.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.177+0000 7f6f30ff9640 1 -- 172.21.15.22:0/3634240404 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0116b0 con 0x7f6f34102ef0 2024-09-19T08:14:15.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.177+0000 7f6f39d85640 1 -- 172.21.15.22:0/3634240404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f341a7210 con 0x7f6f34102ef0 2024-09-19T08:14:15.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.178+0000 7f6f39d85640 1 -- 172.21.15.22:0/3634240404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f34101200 con 0x7f6f34102ef0 2024-09-19T08:14:15.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.179+0000 7f6f30ff9640 1 -- 172.21.15.22:0/3634240404 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6f2c005560 con 0x7f6f34102ef0 2024-09-19T08:14:15.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.180+0000 7f6f30ff9640 1 --2- 172.21.15.22:0/3634240404 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6f1c0a3350 0x7f6f1c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:15.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.180+0000 7f6f337fe640 1 --2- 172.21.15.22:0/3634240404 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6f1c0a3350 0x7f6f1c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:15.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.180+0000 7f6f30ff9640 1 -- 172.21.15.22:0/3634240404 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6f2c0cfd70 con 0x7f6f34102ef0 2024-09-19T08:14:15.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.180+0000 7f6f337fe640 1 --2- 172.21.15.22:0/3634240404 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6f1c0a3350 0x7f6f1c0a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f24005fd0 tx=0x7f6f24005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:15.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.184+0000 7f6f30ff9640 1 -- 172.21.15.22:0/3634240404 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6f2c016030 con 0x7f6f34102ef0 2024-09-19T08:14:15.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:14 smithi157 ceph-mon[35143]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-19T08:14:15.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:14 smithi157 ceph-mon[35143]: Cluster is now healthy 2024-09-19T08:14:15.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.357+0000 7f6f39d85640 1 -- 172.21.15.22:0/3634240404 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f3410cda0 con 0x7f6f1c0a3350 2024-09-19T08:14:15.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.366+0000 7f6f30ff9640 1 -- 172.21.15.22:0/3634240404 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258982 (secure 0 0 0) 0x7f6f3410cda0 con 0x7f6f1c0a3350 2024-09-19T08:14:15.369 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:15.369 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T08:14:15.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.369+0000 7f6f39d85640 1 -- 172.21.15.22:0/3634240404 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6f1c0a3350 msgr2=0x7f6f1c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:15.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.369+0000 7f6f39d85640 1 --2- 172.21.15.22:0/3634240404 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6f1c0a3350 0x7f6f1c0a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f24005fd0 tx=0x7f6f24005850 comp rx=0 tx=0).stop 2024-09-19T08:14:15.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.369+0000 7f6f39d85640 1 -- 172.21.15.22:0/3634240404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f34102ef0 msgr2=0x7f6f3419ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:15.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.369+0000 7f6f39d85640 1 --2- 172.21.15.22:0/3634240404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f34102ef0 0x7f6f3419ef60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00eaa0 tx=0x7f6f2c00ef60 comp rx=0 tx=0).stop 2024-09-19T08:14:15.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.369+0000 7f6f39d85640 1 -- 172.21.15.22:0/3634240404 shutdown_connections 2024-09-19T08:14:15.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.369+0000 7f6f39d85640 1 --2- 172.21.15.22:0/3634240404 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6f1c0a3350 0x7f6f1c0a5810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:15.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.369+0000 7f6f39d85640 1 --2- 172.21.15.22:0/3634240404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f34102ef0 0x7f6f3419ef60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:15.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.369+0000 7f6f39d85640 1 --2- 172.21.15.22:0/3634240404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f34102570 0x7f6f3419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:15.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.369+0000 7f6f39d85640 1 --2- 172.21.15.22:0/3634240404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f34100e20 0x7f6f3419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:15.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.369+0000 7f6f39d85640 1 -- 172.21.15.22:0/3634240404 >> 172.21.15.22:0/3634240404 conn(0x7f6f340fc7d0 msgr2=0x7f6f3406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:15.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.369+0000 7f6f39d85640 1 -- 172.21.15.22:0/3634240404 shutdown_connections 2024-09-19T08:14:15.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:15.369+0000 7f6f39d85640 1 -- 172.21.15.22:0/3634240404 wait complete. 2024-09-19T08:14:16.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:15 smithi022 ceph-mon[30963]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1016 B/s rd, 2 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-09-19T08:14:16.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:15 smithi022 ceph-mon[30963]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:16.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:15 smithi064 ceph-mon[34900]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1016 B/s rd, 2 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-09-19T08:14:16.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:15 smithi064 ceph-mon[34900]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:16.113 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-19T08:14:14.780415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656170,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":511,"num_read_kb":372,"num_write":347,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1242042,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381360,"kb_used_data":11236,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560464,"statfs":{"total":1151940427776,"available":1151549915136,"internally_reserved":0,"allocated":11505664,"data_stored":7311102,"data_compressed":188519,"data_compressed_allocated":245760,"data_compressed_original":2023424,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":8,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":61306,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.059985"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735251+0000","last_change":"2024-09-19T08:13:00.853947+0000","last_active":"2024-09-19T08:14:07.735251+0000","last_peered":"2024-09-19T08:14:07.735251+0000","last_clean":"2024-09-19T08:14:07.735251+0000","last_became_active":"2024-09-19T08:13:00.852951+0000","last_became_peered":"2024-09-19T08:13:00.852951+0000","last_unstale":"2024-09-19T08:14:07.735251+0000","last_undegraded":"2024-09-19T08:14:07.735251+0000","last_fullsized":"2024-09-19T08:14:07.735251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:25.786828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733811+0000","last_change":"2024-09-19T08:13:00.853389+0000","last_active":"2024-09-19T08:14:07.733811+0000","last_peered":"2024-09-19T08:14:07.733811+0000","last_clean":"2024-09-19T08:14:07.733811+0000","last_became_active":"2024-09-19T08:13:00.853102+0000","last_became_peered":"2024-09-19T08:13:00.853102+0000","last_unstale":"2024-09-19T08:14:07.733811+0000","last_undegraded":"2024-09-19T08:14:07.733811+0000","last_fullsized":"2024-09-19T08:14:07.733811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:42.773917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733742+0000","last_change":"2024-09-19T08:13:00.846544+0000","last_active":"2024-09-19T08:14:07.733742+0000","last_peered":"2024-09-19T08:14:07.733742+0000","last_clean":"2024-09-19T08:14:07.733742+0000","last_became_active":"2024-09-19T08:13:00.846229+0000","last_became_peered":"2024-09-19T08:13:00.846229+0000","last_unstale":"2024-09-19T08:14:07.733742+0000","last_undegraded":"2024-09-19T08:14:07.733742+0000","last_fullsized":"2024-09-19T08:14:07.733742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:43.471684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.739325+0000","last_change":"2024-09-19T08:13:00.848149+0000","last_active":"2024-09-19T08:14:07.739325+0000","last_peered":"2024-09-19T08:14:07.739325+0000","last_clean":"2024-09-19T08:14:07.739325+0000","last_became_active":"2024-09-19T08:13:00.847744+0000","last_became_peered":"2024-09-19T08:13:00.847744+0000","last_unstale":"2024-09-19T08:14:07.739325+0000","last_undegraded":"2024-09-19T08:14:07.739325+0000","last_fullsized":"2024-09-19T08:14:07.739325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:01.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738770+0000","last_change":"2024-09-19T08:13:00.841862+0000","last_active":"2024-09-19T08:14:07.738770+0000","last_peered":"2024-09-19T08:14:07.738770+0000","last_clean":"2024-09-19T08:14:07.738770+0000","last_became_active":"2024-09-19T08:13:00.841331+0000","last_became_peered":"2024-09-19T08:13:00.841331+0000","last_unstale":"2024-09-19T08:14:07.738770+0000","last_undegraded":"2024-09-19T08:14:07.738770+0000","last_fullsized":"2024-09-19T08:14:07.738770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:54.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735492+0000","last_change":"2024-09-19T08:13:00.849626+0000","last_active":"2024-09-19T08:14:07.735492+0000","last_peered":"2024-09-19T08:14:07.735492+0000","last_clean":"2024-09-19T08:14:07.735492+0000","last_became_active":"2024-09-19T08:13:00.849034+0000","last_became_peered":"2024-09-19T08:13:00.849034+0000","last_unstale":"2024-09-19T08:14:07.735492+0000","last_undegraded":"2024-09-19T08:14:07.735492+0000","last_fullsized":"2024-09-19T08:14:07.735492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:39.118978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734312+0000","last_change":"2024-09-19T08:13:00.847522+0000","last_active":"2024-09-19T08:14:07.734312+0000","last_peered":"2024-09-19T08:14:07.734312+0000","last_clean":"2024-09-19T08:14:07.734312+0000","last_became_active":"2024-09-19T08:13:00.847117+0000","last_became_peered":"2024-09-19T08:13:00.847117+0000","last_unstale":"2024-09-19T08:14:07.734312+0000","last_undegraded":"2024-09-19T08:14:07.734312+0000","last_fullsized":"2024-09-19T08:14:07.734312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.836385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399370+0000","last_change":"2024-09-19T08:13:00.849544+0000","last_active":"2024-09-19T08:14:08.399370+0000","last_peered":"2024-09-19T08:14:08.399370+0000","last_clean":"2024-09-19T08:14:08.399370+0000","last_became_active":"2024-09-19T08:13:00.849170+0000","last_became_peered":"2024-09-19T08:13:00.849170+0000","last_unstale":"2024-09-19T08:14:08.399370+0000","last_undegraded":"2024-09-19T08:14:08.399370+0000","last_fullsized":"2024-09-19T08:14:08.399370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:28.371965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733158+0000","last_change":"2024-09-19T08:13:00.856052+0000","last_active":"2024-09-19T08:14:07.733158+0000","last_peered":"2024-09-19T08:14:07.733158+0000","last_clean":"2024-09-19T08:14:07.733158+0000","last_became_active":"2024-09-19T08:13:00.855291+0000","last_became_peered":"2024-09-19T08:13:00.855291+0000","last_unstale":"2024-09-19T08:14:07.733158+0000","last_undegraded":"2024-09-19T08:14:07.733158+0000","last_fullsized":"2024-09-19T08:14:07.733158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:18.581867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735684+0000","last_change":"2024-09-19T08:13:00.850145+0000","last_active":"2024-09-19T08:14:07.735684+0000","last_peered":"2024-09-19T08:14:07.735684+0000","last_clean":"2024-09-19T08:14:07.735684+0000","last_became_active":"2024-09-19T08:13:00.849798+0000","last_became_peered":"2024-09-19T08:13:00.849798+0000","last_unstale":"2024-09-19T08:14:07.735684+0000","last_undegraded":"2024-09-19T08:14:07.735684+0000","last_fullsized":"2024-09-19T08:14:07.735684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:21.400000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734734+0000","last_change":"2024-09-19T08:13:00.859153+0000","last_active":"2024-09-19T08:14:07.734734+0000","last_peered":"2024-09-19T08:14:07.734734+0000","last_clean":"2024-09-19T08:14:07.734734+0000","last_became_active":"2024-09-19T08:13:00.858473+0000","last_became_peered":"2024-09-19T08:13:00.858473+0000","last_unstale":"2024-09-19T08:14:07.734734+0000","last_undegraded":"2024-09-19T08:14:07.734734+0000","last_fullsized":"2024-09-19T08:14:07.734734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:07.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.739385+0000","last_change":"2024-09-19T08:13:00.850797+0000","last_active":"2024-09-19T08:14:07.739385+0000","last_peered":"2024-09-19T08:14:07.739385+0000","last_clean":"2024-09-19T08:14:07.739385+0000","last_became_active":"2024-09-19T08:13:00.850393+0000","last_became_peered":"2024-09-19T08:13:00.850393+0000","last_unstale":"2024-09-19T08:14:07.739385+0000","last_undegraded":"2024-09-19T08:14:07.739385+0000","last_fullsized":"2024-09-19T08:14:07.739385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:59.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731520+0000","last_change":"2024-09-19T08:13:00.850835+0000","last_active":"2024-09-19T08:14:07.731520+0000","last_peered":"2024-09-19T08:14:07.731520+0000","last_clean":"2024-09-19T08:14:07.731520+0000","last_became_active":"2024-09-19T08:13:00.850518+0000","last_became_peered":"2024-09-19T08:13:00.850518+0000","last_unstale":"2024-09-19T08:14:07.731520+0000","last_undegraded":"2024-09-19T08:14:07.731520+0000","last_fullsized":"2024-09-19T08:14:07.731520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.178530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733525+0000","last_change":"2024-09-19T08:13:00.853934+0000","last_active":"2024-09-19T08:14:07.733525+0000","last_peered":"2024-09-19T08:14:07.733525+0000","last_clean":"2024-09-19T08:14:07.733525+0000","last_became_active":"2024-09-19T08:13:00.853604+0000","last_became_peered":"2024-09-19T08:13:00.853604+0000","last_unstale":"2024-09-19T08:14:07.733525+0000","last_undegraded":"2024-09-19T08:14:07.733525+0000","last_fullsized":"2024-09-19T08:14:07.733525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:43.622526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.741370+0000","last_change":"2024-09-19T08:14:07.741370+0000","last_active":"2024-09-19T08:14:07.741370+0000","last_peered":"2024-09-19T08:14:07.741370+0000","last_clean":"2024-09-19T08:14:07.741370+0000","last_became_active":"2024-09-19T08:14:07.741116+0000","last_became_peered":"2024-09-19T08:14:07.741116+0000","last_unstale":"2024-09-19T08:14:07.741370+0000","last_undegraded":"2024-09-19T08:14:07.741370+0000","last_fullsized":"2024-09-19T08:14:07.741370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:45.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732701+0000","last_change":"2024-09-19T08:13:00.850266+0000","last_active":"2024-09-19T08:14:07.732701+0000","last_peered":"2024-09-19T08:14:07.732701+0000","last_clean":"2024-09-19T08:14:07.732701+0000","last_became_active":"2024-09-19T08:13:00.849928+0000","last_became_peered":"2024-09-19T08:13:00.849928+0000","last_unstale":"2024-09-19T08:14:07.732701+0000","last_undegraded":"2024-09-19T08:14:07.732701+0000","last_fullsized":"2024-09-19T08:14:07.732701+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:38.231331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735367+0000","last_change":"2024-09-19T08:13:00.852551+0000","last_active":"2024-09-19T08:14:07.735367+0000","last_peered":"2024-09-19T08:14:07.735367+0000","last_clean":"2024-09-19T08:14:07.735367+0000","last_became_active":"2024-09-19T08:13:00.852225+0000","last_became_peered":"2024-09-19T08:13:00.852225+0000","last_unstale":"2024-09-19T08:14:07.735367+0000","last_undegraded":"2024-09-19T08:14:07.735367+0000","last_fullsized":"2024-09-19T08:14:07.735367+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:33.115108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735168+0000","last_change":"2024-09-19T08:13:00.854033+0000","last_active":"2024-09-19T08:14:07.735168+0000","last_peered":"2024-09-19T08:14:07.735168+0000","last_clean":"2024-09-19T08:14:07.735168+0000","last_became_active":"2024-09-19T08:13:00.853660+0000","last_became_peered":"2024-09-19T08:13:00.853660+0000","last_unstale":"2024-09-19T08:14:07.735168+0000","last_undegraded":"2024-09-19T08:14:07.735168+0000","last_fullsized":"2024-09-19T08:14:07.735168+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:35.602420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732530+0000","last_change":"2024-09-19T08:13:00.850658+0000","last_active":"2024-09-19T08:14:07.732530+0000","last_peered":"2024-09-19T08:14:07.732530+0000","last_clean":"2024-09-19T08:14:07.732530+0000","last_became_active":"2024-09-19T08:13:00.850324+0000","last_became_peered":"2024-09-19T08:13:00.850324+0000","last_unstale":"2024-09-19T08:14:07.732530+0000","last_undegraded":"2024-09-19T08:14:07.732530+0000","last_fullsized":"2024-09-19T08:14:07.732530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:59:49.004605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.741837+0000","last_change":"2024-09-19T08:14:07.741837+0000","last_active":"2024-09-19T08:14:07.741837+0000","last_peered":"2024-09-19T08:14:07.741837+0000","last_clean":"2024-09-19T08:14:07.741837+0000","last_became_active":"2024-09-19T08:14:07.741398+0000","last_became_peered":"2024-09-19T08:14:07.741398+0000","last_unstale":"2024-09-19T08:14:07.741837+0000","last_undegraded":"2024-09-19T08:14:07.741837+0000","last_fullsized":"2024-09-19T08:14:07.741837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:56.716663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731582+0000","last_change":"2024-09-19T08:13:00.855215+0000","last_active":"2024-09-19T08:14:07.731582+0000","last_peered":"2024-09-19T08:14:07.731582+0000","last_clean":"2024-09-19T08:14:07.731582+0000","last_became_active":"2024-09-19T08:13:00.854878+0000","last_became_peered":"2024-09-19T08:13:00.854878+0000","last_unstale":"2024-09-19T08:14:07.731582+0000","last_undegraded":"2024-09-19T08:14:07.731582+0000","last_fullsized":"2024-09-19T08:14:07.731582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:48.216066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733505+0000","last_change":"2024-09-19T08:13:00.856995+0000","last_active":"2024-09-19T08:14:07.733505+0000","last_peered":"2024-09-19T08:14:07.733505+0000","last_clean":"2024-09-19T08:14:07.733505+0000","last_became_active":"2024-09-19T08:13:00.856645+0000","last_became_peered":"2024-09-19T08:13:00.856645+0000","last_unstale":"2024-09-19T08:14:07.733505+0000","last_undegraded":"2024-09-19T08:14:07.733505+0000","last_fullsized":"2024-09-19T08:14:07.733505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:20.699293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399539+0000","last_change":"2024-09-19T08:13:00.853362+0000","last_active":"2024-09-19T08:14:08.399539+0000","last_peered":"2024-09-19T08:14:08.399539+0000","last_clean":"2024-09-19T08:14:08.399539+0000","last_became_active":"2024-09-19T08:13:00.853046+0000","last_became_peered":"2024-09-19T08:13:00.853046+0000","last_unstale":"2024-09-19T08:14:08.399539+0000","last_undegraded":"2024-09-19T08:14:08.399539+0000","last_fullsized":"2024-09-19T08:14:08.399539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736509+0000","last_change":"2024-09-19T08:13:00.845175+0000","last_active":"2024-09-19T08:14:07.736509+0000","last_peered":"2024-09-19T08:14:07.736509+0000","last_clean":"2024-09-19T08:14:07.736509+0000","last_became_active":"2024-09-19T08:13:00.844857+0000","last_became_peered":"2024-09-19T08:13:00.844857+0000","last_unstale":"2024-09-19T08:14:07.736509+0000","last_undegraded":"2024-09-19T08:14:07.736509+0000","last_fullsized":"2024-09-19T08:14:07.736509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:40.818956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399481+0000","last_change":"2024-09-19T08:13:00.847769+0000","last_active":"2024-09-19T08:14:08.399481+0000","last_peered":"2024-09-19T08:14:08.399481+0000","last_clean":"2024-09-19T08:14:08.399481+0000","last_became_active":"2024-09-19T08:13:00.847454+0000","last_became_peered":"2024-09-19T08:13:00.847454+0000","last_unstale":"2024-09-19T08:14:08.399481+0000","last_undegraded":"2024-09-19T08:14:08.399481+0000","last_fullsized":"2024-09-19T08:14:08.399481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.583051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732671+0000","last_change":"2024-09-19T08:13:00.859882+0000","last_active":"2024-09-19T08:14:07.732671+0000","last_peered":"2024-09-19T08:14:07.732671+0000","last_clean":"2024-09-19T08:14:07.732671+0000","last_became_active":"2024-09-19T08:13:00.859491+0000","last_became_peered":"2024-09-19T08:13:00.859491+0000","last_unstale":"2024-09-19T08:14:07.732671+0000","last_undegraded":"2024-09-19T08:14:07.732671+0000","last_fullsized":"2024-09-19T08:14:07.732671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:13.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734271+0000","last_change":"2024-09-19T08:13:00.845584+0000","last_active":"2024-09-19T08:14:07.734271+0000","last_peered":"2024-09-19T08:14:07.734271+0000","last_clean":"2024-09-19T08:14:07.734271+0000","last_became_active":"2024-09-19T08:13:00.845166+0000","last_became_peered":"2024-09-19T08:13:00.845166+0000","last_unstale":"2024-09-19T08:14:07.734271+0000","last_undegraded":"2024-09-19T08:14:07.734271+0000","last_fullsized":"2024-09-19T08:14:07.734271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:02.705793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731667+0000","last_change":"2024-09-19T08:13:00.838069+0000","last_active":"2024-09-19T08:14:07.731667+0000","last_peered":"2024-09-19T08:14:07.731667+0000","last_clean":"2024-09-19T08:14:07.731667+0000","last_became_active":"2024-09-19T08:13:00.837736+0000","last_became_peered":"2024-09-19T08:13:00.837736+0000","last_unstale":"2024-09-19T08:14:07.731667+0000","last_undegraded":"2024-09-19T08:14:07.731667+0000","last_fullsized":"2024-09-19T08:14:07.731667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:01.388290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733793+0000","last_change":"2024-09-19T08:13:00.841507+0000","last_active":"2024-09-19T08:14:07.733793+0000","last_peered":"2024-09-19T08:14:07.733793+0000","last_clean":"2024-09-19T08:14:07.733793+0000","last_became_active":"2024-09-19T08:13:00.841161+0000","last_became_peered":"2024-09-19T08:13:00.841161+0000","last_unstale":"2024-09-19T08:14:07.733793+0000","last_undegraded":"2024-09-19T08:14:07.733793+0000","last_fullsized":"2024-09-19T08:14:07.733793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:41.862290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732148+0000","last_change":"2024-09-19T08:13:00.847657+0000","last_active":"2024-09-19T08:14:07.732148+0000","last_peered":"2024-09-19T08:14:07.732148+0000","last_clean":"2024-09-19T08:14:07.732148+0000","last_became_active":"2024-09-19T08:13:00.846849+0000","last_became_peered":"2024-09-19T08:13:00.846849+0000","last_unstale":"2024-09-19T08:14:07.732148+0000","last_undegraded":"2024-09-19T08:14:07.732148+0000","last_fullsized":"2024-09-19T08:14:07.732148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:35.219913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732183+0000","last_change":"2024-09-19T08:13:00.845971+0000","last_active":"2024-09-19T08:14:07.732183+0000","last_peered":"2024-09-19T08:14:07.732183+0000","last_clean":"2024-09-19T08:14:07.732183+0000","last_became_active":"2024-09-19T08:13:00.845626+0000","last_became_peered":"2024-09-19T08:13:00.845626+0000","last_unstale":"2024-09-19T08:14:07.732183+0000","last_undegraded":"2024-09-19T08:14:07.732183+0000","last_fullsized":"2024-09-19T08:14:07.732183+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:36.945627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738450+0000","last_change":"2024-09-19T08:14:07.738450+0000","last_active":"2024-09-19T08:14:07.738450+0000","last_peered":"2024-09-19T08:14:07.738450+0000","last_clean":"2024-09-19T08:14:07.738450+0000","last_became_active":"2024-09-19T08:14:07.738028+0000","last_became_peered":"2024-09-19T08:14:07.738028+0000","last_unstale":"2024-09-19T08:14:07.738450+0000","last_undegraded":"2024-09-19T08:14:07.738450+0000","last_fullsized":"2024-09-19T08:14:07.738450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:08.659158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738529+0000","last_change":"2024-09-19T08:13:00.848325+0000","last_active":"2024-09-19T08:14:07.738529+0000","last_peered":"2024-09-19T08:14:07.738529+0000","last_clean":"2024-09-19T08:14:07.738529+0000","last_became_active":"2024-09-19T08:13:00.848015+0000","last_became_peered":"2024-09-19T08:13:00.848015+0000","last_unstale":"2024-09-19T08:14:07.738529+0000","last_undegraded":"2024-09-19T08:14:07.738529+0000","last_fullsized":"2024-09-19T08:14:07.738529+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:10.948451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":90,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.961140+0000","last_change":"2024-09-19T08:12:58.681134+0000","last_active":"2024-09-19T08:14:08.961140+0000","last_peered":"2024-09-19T08:14:08.961140+0000","last_clean":"2024-09-19T08:14:08.961140+0000","last_became_active":"2024-09-19T08:12:58.680764+0000","last_became_peered":"2024-09-19T08:12:58.680764+0000","last_unstale":"2024-09-19T08:14:08.961140+0000","last_undegraded":"2024-09-19T08:14:08.961140+0000","last_fullsized":"2024-09-19T08:14:08.961140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:24.939263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736894+0000","last_change":"2024-09-19T08:13:00.844319+0000","last_active":"2024-09-19T08:14:07.736894+0000","last_peered":"2024-09-19T08:14:07.736894+0000","last_clean":"2024-09-19T08:14:07.736894+0000","last_became_active":"2024-09-19T08:13:00.843994+0000","last_became_peered":"2024-09-19T08:13:00.843994+0000","last_unstale":"2024-09-19T08:14:07.736894+0000","last_undegraded":"2024-09-19T08:14:07.736894+0000","last_fullsized":"2024-09-19T08:14:07.736894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:46.937466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735608+0000","last_change":"2024-09-19T08:12:58.676541+0000","last_active":"2024-09-19T08:14:07.735608+0000","last_peered":"2024-09-19T08:14:07.735608+0000","last_clean":"2024-09-19T08:14:07.735608+0000","last_became_active":"2024-09-19T08:12:58.675988+0000","last_became_peered":"2024-09-19T08:12:58.675988+0000","last_unstale":"2024-09-19T08:14:07.735608+0000","last_undegraded":"2024-09-19T08:14:07.735608+0000","last_fullsized":"2024-09-19T08:14:07.735608+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:01.434986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731256+0000","last_change":"2024-09-19T08:12:58.677510+0000","last_active":"2024-09-19T08:14:07.731256+0000","last_peered":"2024-09-19T08:14:07.731256+0000","last_clean":"2024-09-19T08:14:07.731256+0000","last_became_active":"2024-09-19T08:12:58.677200+0000","last_became_peered":"2024-09-19T08:12:58.677200+0000","last_unstale":"2024-09-19T08:14:07.731256+0000","last_undegraded":"2024-09-19T08:14:07.731256+0000","last_fullsized":"2024-09-19T08:14:07.731256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:36.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399481+0000","last_change":"2024-09-19T08:13:00.853430+0000","last_active":"2024-09-19T08:14:08.399481+0000","last_peered":"2024-09-19T08:14:08.399481+0000","last_clean":"2024-09-19T08:14:08.399481+0000","last_became_active":"2024-09-19T08:13:00.853096+0000","last_became_peered":"2024-09-19T08:13:00.853096+0000","last_unstale":"2024-09-19T08:14:08.399481+0000","last_undegraded":"2024-09-19T08:14:08.399481+0000","last_fullsized":"2024-09-19T08:14:08.399481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:07.147021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736819+0000","last_change":"2024-09-19T08:12:58.680081+0000","last_active":"2024-09-19T08:14:07.736819+0000","last_peered":"2024-09-19T08:14:07.736819+0000","last_clean":"2024-09-19T08:14:07.736819+0000","last_became_active":"2024-09-19T08:12:58.679704+0000","last_became_peered":"2024-09-19T08:12:58.679704+0000","last_unstale":"2024-09-19T08:14:07.736819+0000","last_undegraded":"2024-09-19T08:14:07.736819+0000","last_fullsized":"2024-09-19T08:14:07.736819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:32.942586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735021+0000","last_change":"2024-09-19T08:13:00.849394+0000","last_active":"2024-09-19T08:14:07.735021+0000","last_peered":"2024-09-19T08:14:07.735021+0000","last_clean":"2024-09-19T08:14:07.735021+0000","last_became_active":"2024-09-19T08:13:00.849051+0000","last_became_peered":"2024-09-19T08:13:00.849051+0000","last_unstale":"2024-09-19T08:14:07.735021+0000","last_undegraded":"2024-09-19T08:14:07.735021+0000","last_fullsized":"2024-09-19T08:14:07.735021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:23.012928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":84,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.960591+0000","last_change":"2024-09-19T08:12:58.676121+0000","last_active":"2024-09-19T08:14:08.960591+0000","last_peered":"2024-09-19T08:14:08.960591+0000","last_clean":"2024-09-19T08:14:08.960591+0000","last_became_active":"2024-09-19T08:12:58.675693+0000","last_became_peered":"2024-09-19T08:12:58.675693+0000","last_unstale":"2024-09-19T08:14:08.960591+0000","last_undegraded":"2024-09-19T08:14:08.960591+0000","last_fullsized":"2024-09-19T08:14:08.960591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:34.882190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733023+0000","last_change":"2024-09-19T08:13:00.858735+0000","last_active":"2024-09-19T08:14:07.733023+0000","last_peered":"2024-09-19T08:14:07.733023+0000","last_clean":"2024-09-19T08:14:07.733023+0000","last_became_active":"2024-09-19T08:13:00.858218+0000","last_became_peered":"2024-09-19T08:13:00.858218+0000","last_unstale":"2024-09-19T08:14:07.733023+0000","last_undegraded":"2024-09-19T08:14:07.733023+0000","last_fullsized":"2024-09-19T08:14:07.733023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:33.356810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.743785+0000","last_change":"2024-09-19T08:14:07.743785+0000","last_active":"2024-09-19T08:14:07.743785+0000","last_peered":"2024-09-19T08:14:07.743785+0000","last_clean":"2024-09-19T08:14:07.743785+0000","last_became_active":"2024-09-19T08:14:07.739531+0000","last_became_peered":"2024-09-19T08:14:07.739531+0000","last_unstale":"2024-09-19T08:14:07.743785+0000","last_undegraded":"2024-09-19T08:14:07.743785+0000","last_fullsized":"2024-09-19T08:14:07.743785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:43.367704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733696+0000","last_change":"2024-09-19T08:13:00.843852+0000","last_active":"2024-09-19T08:14:07.733696+0000","last_peered":"2024-09-19T08:14:07.733696+0000","last_clean":"2024-09-19T08:14:07.733696+0000","last_became_active":"2024-09-19T08:13:00.843497+0000","last_became_peered":"2024-09-19T08:13:00.843497+0000","last_unstale":"2024-09-19T08:14:07.733696+0000","last_undegraded":"2024-09-19T08:14:07.733696+0000","last_fullsized":"2024-09-19T08:14:07.733696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:26.380024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.737669+0000","last_change":"2024-09-19T08:12:58.717225+0000","last_active":"2024-09-19T08:14:07.737669+0000","last_peered":"2024-09-19T08:14:07.737669+0000","last_clean":"2024-09-19T08:14:07.737669+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:14:07.737669+0000","last_undegraded":"2024-09-19T08:14:07.737669+0000","last_fullsized":"2024-09-19T08:14:07.737669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:43.201907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733016+0000","last_change":"2024-09-19T08:13:00.845613+0000","last_active":"2024-09-19T08:14:07.733016+0000","last_peered":"2024-09-19T08:14:07.733016+0000","last_clean":"2024-09-19T08:14:07.733016+0000","last_became_active":"2024-09-19T08:13:00.845260+0000","last_became_peered":"2024-09-19T08:13:00.845260+0000","last_unstale":"2024-09-19T08:14:07.733016+0000","last_undegraded":"2024-09-19T08:14:07.733016+0000","last_fullsized":"2024-09-19T08:14:07.733016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:03:47.033972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'29","reported_seq":194,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:09.746400+0000","last_change":"2024-09-19T08:14:07.749616+0000","last_active":"2024-09-19T08:14:09.746400+0000","last_peered":"2024-09-19T08:14:09.746400+0000","last_clean":"2024-09-19T08:14:09.746400+0000","last_became_active":"2024-09-19T08:14:07.740298+0000","last_became_peered":"2024-09-19T08:14:07.740298+0000","last_unstale":"2024-09-19T08:14:09.746400+0000","last_undegraded":"2024-09-19T08:14:09.746400+0000","last_fullsized":"2024-09-19T08:14:09.746400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:28.796855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61172,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61172,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738939+0000","last_change":"2024-09-19T08:13:00.855603+0000","last_active":"2024-09-19T08:14:07.738939+0000","last_peered":"2024-09-19T08:14:07.738939+0000","last_clean":"2024-09-19T08:14:07.738939+0000","last_became_active":"2024-09-19T08:13:00.855241+0000","last_became_peered":"2024-09-19T08:13:00.855241+0000","last_unstale":"2024-09-19T08:14:07.738939+0000","last_undegraded":"2024-09-19T08:14:07.738939+0000","last_fullsized":"2024-09-19T08:14:07.738939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:28.811847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732313+0000","last_change":"2024-09-19T08:12:58.675897+0000","last_active":"2024-09-19T08:14:07.732313+0000","last_peered":"2024-09-19T08:14:07.732313+0000","last_clean":"2024-09-19T08:14:07.732313+0000","last_became_active":"2024-09-19T08:12:58.675525+0000","last_became_peered":"2024-09-19T08:12:58.675525+0000","last_unstale":"2024-09-19T08:14:07.732313+0000","last_undegraded":"2024-09-19T08:14:07.732313+0000","last_fullsized":"2024-09-19T08:14:07.732313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:54.518064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731271+0000","last_change":"2024-09-19T08:13:00.856420+0000","last_active":"2024-09-19T08:14:07.731271+0000","last_peered":"2024-09-19T08:14:07.731271+0000","last_clean":"2024-09-19T08:14:07.731271+0000","last_became_active":"2024-09-19T08:13:00.856100+0000","last_became_peered":"2024-09-19T08:13:00.856100+0000","last_unstale":"2024-09-19T08:14:07.731271+0000","last_undegraded":"2024-09-19T08:14:07.731271+0000","last_fullsized":"2024-09-19T08:14:07.731271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:58.684930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.737378+0000","last_change":"2024-09-19T08:12:58.680778+0000","last_active":"2024-09-19T08:14:07.737378+0000","last_peered":"2024-09-19T08:14:07.737378+0000","last_clean":"2024-09-19T08:14:07.737378+0000","last_became_active":"2024-09-19T08:12:58.680364+0000","last_became_peered":"2024-09-19T08:12:58.680364+0000","last_unstale":"2024-09-19T08:14:07.737378+0000","last_undegraded":"2024-09-19T08:14:07.737378+0000","last_fullsized":"2024-09-19T08:14:07.737378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:10.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734418+0000","last_change":"2024-09-19T08:13:00.853415+0000","last_active":"2024-09-19T08:14:07.734418+0000","last_peered":"2024-09-19T08:14:07.734418+0000","last_clean":"2024-09-19T08:14:07.734418+0000","last_became_active":"2024-09-19T08:13:00.853101+0000","last_became_peered":"2024-09-19T08:13:00.853101+0000","last_unstale":"2024-09-19T08:14:07.734418+0000","last_undegraded":"2024-09-19T08:14:07.734418+0000","last_fullsized":"2024-09-19T08:14:07.734418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:42.375394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":92,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:12.966309+0000","last_change":"2024-09-19T08:12:58.680312+0000","last_active":"2024-09-19T08:14:12.966309+0000","last_peered":"2024-09-19T08:14:12.966309+0000","last_clean":"2024-09-19T08:14:12.966309+0000","last_became_active":"2024-09-19T08:12:58.679870+0000","last_became_peered":"2024-09-19T08:12:58.679870+0000","last_unstale":"2024-09-19T08:14:12.966309+0000","last_undegraded":"2024-09-19T08:14:12.966309+0000","last_fullsized":"2024-09-19T08:14:12.966309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:42.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734408+0000","last_change":"2024-09-19T08:13:00.853920+0000","last_active":"2024-09-19T08:14:07.734408+0000","last_peered":"2024-09-19T08:14:07.734408+0000","last_clean":"2024-09-19T08:14:07.734408+0000","last_became_active":"2024-09-19T08:13:00.853594+0000","last_became_peered":"2024-09-19T08:13:00.853594+0000","last_unstale":"2024-09-19T08:14:07.734408+0000","last_undegraded":"2024-09-19T08:14:07.734408+0000","last_fullsized":"2024-09-19T08:14:07.734408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:31.746133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.752282+0000","last_change":"2024-09-19T08:14:07.751888+0000","last_active":"2024-09-19T08:14:07.752282+0000","last_peered":"2024-09-19T08:14:07.752282+0000","last_clean":"2024-09-19T08:14:07.752282+0000","last_became_active":"2024-09-19T08:14:07.737201+0000","last_became_peered":"2024-09-19T08:14:07.737201+0000","last_unstale":"2024-09-19T08:14:07.752282+0000","last_undegraded":"2024-09-19T08:14:07.752282+0000","last_fullsized":"2024-09-19T08:14:07.752282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:44.095987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.737002+0000","last_change":"2024-09-19T08:13:00.847061+0000","last_active":"2024-09-19T08:14:07.737002+0000","last_peered":"2024-09-19T08:14:07.737002+0000","last_clean":"2024-09-19T08:14:07.737002+0000","last_became_active":"2024-09-19T08:13:00.846742+0000","last_became_peered":"2024-09-19T08:13:00.846742+0000","last_unstale":"2024-09-19T08:14:07.737002+0000","last_undegraded":"2024-09-19T08:14:07.737002+0000","last_fullsized":"2024-09-19T08:14:07.737002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:21.785585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.755652+0000","last_change":"2024-09-19T08:14:07.755277+0000","last_active":"2024-09-19T08:14:07.755652+0000","last_peered":"2024-09-19T08:14:07.755652+0000","last_clean":"2024-09-19T08:14:07.755652+0000","last_became_active":"2024-09-19T08:14:07.739773+0000","last_became_peered":"2024-09-19T08:14:07.739773+0000","last_unstale":"2024-09-19T08:14:07.755652+0000","last_undegraded":"2024-09-19T08:14:07.755652+0000","last_fullsized":"2024-09-19T08:14:07.755652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:56.848251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734131+0000","last_change":"2024-09-19T08:13:00.844940+0000","last_active":"2024-09-19T08:14:07.734131+0000","last_peered":"2024-09-19T08:14:07.734131+0000","last_clean":"2024-09-19T08:14:07.734131+0000","last_became_active":"2024-09-19T08:13:00.844454+0000","last_became_peered":"2024-09-19T08:13:00.844454+0000","last_unstale":"2024-09-19T08:14:07.734131+0000","last_undegraded":"2024-09-19T08:14:07.734131+0000","last_fullsized":"2024-09-19T08:14:07.734131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:15.577624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731644+0000","last_change":"2024-09-19T08:12:58.673301+0000","last_active":"2024-09-19T08:14:07.731644+0000","last_peered":"2024-09-19T08:14:07.731644+0000","last_clean":"2024-09-19T08:14:07.731644+0000","last_became_active":"2024-09-19T08:12:58.672981+0000","last_became_peered":"2024-09-19T08:12:58.672981+0000","last_unstale":"2024-09-19T08:14:07.731644+0000","last_undegraded":"2024-09-19T08:14:07.731644+0000","last_fullsized":"2024-09-19T08:14:07.731644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:03.963712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735037+0000","last_change":"2024-09-19T08:13:00.856620+0000","last_active":"2024-09-19T08:14:07.735037+0000","last_peered":"2024-09-19T08:14:07.735037+0000","last_clean":"2024-09-19T08:14:07.735037+0000","last_became_active":"2024-09-19T08:13:00.856310+0000","last_became_peered":"2024-09-19T08:13:00.856310+0000","last_unstale":"2024-09-19T08:14:07.735037+0000","last_undegraded":"2024-09-19T08:14:07.735037+0000","last_fullsized":"2024-09-19T08:14:07.735037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:16.918768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.737741+0000","last_change":"2024-09-19T08:12:58.672282+0000","last_active":"2024-09-19T08:14:07.737741+0000","last_peered":"2024-09-19T08:14:07.737741+0000","last_clean":"2024-09-19T08:14:07.737741+0000","last_became_active":"2024-09-19T08:12:58.671891+0000","last_became_peered":"2024-09-19T08:12:58.671891+0000","last_unstale":"2024-09-19T08:14:07.737741+0000","last_undegraded":"2024-09-19T08:14:07.737741+0000","last_fullsized":"2024-09-19T08:14:07.737741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:41.731571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734550+0000","last_change":"2024-09-19T08:13:00.848107+0000","last_active":"2024-09-19T08:14:07.734550+0000","last_peered":"2024-09-19T08:14:07.734550+0000","last_clean":"2024-09-19T08:14:07.734550+0000","last_became_active":"2024-09-19T08:13:00.847632+0000","last_became_peered":"2024-09-19T08:13:00.847632+0000","last_unstale":"2024-09-19T08:14:07.734550+0000","last_undegraded":"2024-09-19T08:14:07.734550+0000","last_fullsized":"2024-09-19T08:14:07.734550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.491122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732531+0000","last_change":"2024-09-19T08:12:59.179200+0000","last_active":"2024-09-19T08:14:07.732531+0000","last_peered":"2024-09-19T08:14:07.732531+0000","last_clean":"2024-09-19T08:14:07.732531+0000","last_became_active":"2024-09-19T08:12:59.178762+0000","last_became_peered":"2024-09-19T08:12:59.178762+0000","last_unstale":"2024-09-19T08:14:07.732531+0000","last_undegraded":"2024-09-19T08:14:07.732531+0000","last_fullsized":"2024-09-19T08:14:07.732531+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:08.424106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734839+0000","last_change":"2024-09-19T08:13:00.855870+0000","last_active":"2024-09-19T08:14:07.734839+0000","last_peered":"2024-09-19T08:14:07.734839+0000","last_clean":"2024-09-19T08:14:07.734839+0000","last_became_active":"2024-09-19T08:13:00.855538+0000","last_became_peered":"2024-09-19T08:13:00.855538+0000","last_unstale":"2024-09-19T08:14:07.734839+0000","last_undegraded":"2024-09-19T08:14:07.734839+0000","last_fullsized":"2024-09-19T08:14:07.734839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734771+0000","last_change":"2024-09-19T08:11:48.661333+0000","last_active":"2024-09-19T08:14:07.734771+0000","last_peered":"2024-09-19T08:14:07.734771+0000","last_clean":"2024-09-19T08:14:07.734771+0000","last_became_active":"2024-09-19T08:11:48.660873+0000","last_became_peered":"2024-09-19T08:11:48.660873+0000","last_unstale":"2024-09-19T08:14:07.734771+0000","last_undegraded":"2024-09-19T08:14:07.734771+0000","last_fullsized":"2024-09-19T08:14:07.734771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_clean_scrub_stamp":"2024-09-19T08:08:25.372155+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:19.401292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738561+0000","last_change":"2024-09-19T08:12:58.675237+0000","last_active":"2024-09-19T08:14:07.738561+0000","last_peered":"2024-09-19T08:14:07.738561+0000","last_clean":"2024-09-19T08:14:07.738561+0000","last_became_active":"2024-09-19T08:12:58.674910+0000","last_became_peered":"2024-09-19T08:12:58.674910+0000","last_unstale":"2024-09-19T08:14:07.738561+0000","last_undegraded":"2024-09-19T08:14:07.738561+0000","last_fullsized":"2024-09-19T08:14:07.738561+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:47.038328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399776+0000","last_change":"2024-09-19T08:13:00.846754+0000","last_active":"2024-09-19T08:14:08.399776+0000","last_peered":"2024-09-19T08:14:08.399776+0000","last_clean":"2024-09-19T08:14:08.399776+0000","last_became_active":"2024-09-19T08:13:00.846452+0000","last_became_peered":"2024-09-19T08:13:00.846452+0000","last_unstale":"2024-09-19T08:14:08.399776+0000","last_undegraded":"2024-09-19T08:14:08.399776+0000","last_fullsized":"2024-09-19T08:14:08.399776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:22.929689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734639+0000","last_change":"2024-09-19T08:12:58.678958+0000","last_active":"2024-09-19T08:14:07.734639+0000","last_peered":"2024-09-19T08:14:07.734639+0000","last_clean":"2024-09-19T08:14:07.734639+0000","last_became_active":"2024-09-19T08:12:58.678592+0000","last_became_peered":"2024-09-19T08:12:58.678592+0000","last_unstale":"2024-09-19T08:14:07.734639+0000","last_undegraded":"2024-09-19T08:14:07.734639+0000","last_fullsized":"2024-09-19T08:14:07.734639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:58.374606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731205+0000","last_change":"2024-09-19T08:13:00.840973+0000","last_active":"2024-09-19T08:14:07.731205+0000","last_peered":"2024-09-19T08:14:07.731205+0000","last_clean":"2024-09-19T08:14:07.731205+0000","last_became_active":"2024-09-19T08:13:00.840619+0000","last_became_peered":"2024-09-19T08:13:00.840619+0000","last_unstale":"2024-09-19T08:14:07.731205+0000","last_undegraded":"2024-09-19T08:14:07.731205+0000","last_fullsized":"2024-09-19T08:14:07.731205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":85,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.960649+0000","last_change":"2024-09-19T08:12:58.715972+0000","last_active":"2024-09-19T08:14:08.960649+0000","last_peered":"2024-09-19T08:14:08.960649+0000","last_clean":"2024-09-19T08:14:08.960649+0000","last_became_active":"2024-09-19T08:12:58.715630+0000","last_became_peered":"2024-09-19T08:12:58.715630+0000","last_unstale":"2024-09-19T08:14:08.960649+0000","last_undegraded":"2024-09-19T08:14:08.960649+0000","last_fullsized":"2024-09-19T08:14:08.960649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:35.637121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736616+0000","last_change":"2024-09-19T08:13:00.847023+0000","last_active":"2024-09-19T08:14:07.736616+0000","last_peered":"2024-09-19T08:14:07.736616+0000","last_clean":"2024-09-19T08:14:07.736616+0000","last_became_active":"2024-09-19T08:13:00.846387+0000","last_became_peered":"2024-09-19T08:13:00.846387+0000","last_unstale":"2024-09-19T08:14:07.736616+0000","last_undegraded":"2024-09-19T08:14:07.736616+0000","last_fullsized":"2024-09-19T08:14:07.736616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:12.187929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735037+0000","last_change":"2024-09-19T08:12:58.681062+0000","last_active":"2024-09-19T08:14:07.735037+0000","last_peered":"2024-09-19T08:14:07.735037+0000","last_clean":"2024-09-19T08:14:07.735037+0000","last_became_active":"2024-09-19T08:12:58.680524+0000","last_became_peered":"2024-09-19T08:12:58.680524+0000","last_unstale":"2024-09-19T08:14:07.735037+0000","last_undegraded":"2024-09-19T08:14:07.735037+0000","last_fullsized":"2024-09-19T08:14:07.735037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735930+0000","last_change":"2024-09-19T08:13:00.851523+0000","last_active":"2024-09-19T08:14:07.735930+0000","last_peered":"2024-09-19T08:14:07.735930+0000","last_clean":"2024-09-19T08:14:07.735930+0000","last_became_active":"2024-09-19T08:13:00.851068+0000","last_became_peered":"2024-09-19T08:13:00.851068+0000","last_unstale":"2024-09-19T08:14:07.735930+0000","last_undegraded":"2024-09-19T08:14:07.735930+0000","last_fullsized":"2024-09-19T08:14:07.735930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:44.905469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735213+0000","last_change":"2024-09-19T08:12:58.718234+0000","last_active":"2024-09-19T08:14:07.735213+0000","last_peered":"2024-09-19T08:14:07.735213+0000","last_clean":"2024-09-19T08:14:07.735213+0000","last_became_active":"2024-09-19T08:12:58.717840+0000","last_became_peered":"2024-09-19T08:12:58.717840+0000","last_unstale":"2024-09-19T08:14:07.735213+0000","last_undegraded":"2024-09-19T08:14:07.735213+0000","last_fullsized":"2024-09-19T08:14:07.735213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:05.283770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736201+0000","last_change":"2024-09-19T08:13:00.848305+0000","last_active":"2024-09-19T08:14:07.736201+0000","last_peered":"2024-09-19T08:14:07.736201+0000","last_clean":"2024-09-19T08:14:07.736201+0000","last_became_active":"2024-09-19T08:13:00.847981+0000","last_became_peered":"2024-09-19T08:13:00.847981+0000","last_unstale":"2024-09-19T08:14:07.736201+0000","last_undegraded":"2024-09-19T08:14:07.736201+0000","last_fullsized":"2024-09-19T08:14:07.736201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:04.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.739029+0000","last_change":"2024-09-19T08:12:58.715943+0000","last_active":"2024-09-19T08:14:07.739029+0000","last_peered":"2024-09-19T08:14:07.739029+0000","last_clean":"2024-09-19T08:14:07.739029+0000","last_became_active":"2024-09-19T08:12:58.715600+0000","last_became_peered":"2024-09-19T08:12:58.715600+0000","last_unstale":"2024-09-19T08:14:07.739029+0000","last_undegraded":"2024-09-19T08:14:07.739029+0000","last_fullsized":"2024-09-19T08:14:07.739029+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:19.142271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731759+0000","last_change":"2024-09-19T08:13:00.840944+0000","last_active":"2024-09-19T08:14:07.731759+0000","last_peered":"2024-09-19T08:14:07.731759+0000","last_clean":"2024-09-19T08:14:07.731759+0000","last_became_active":"2024-09-19T08:13:00.840608+0000","last_became_peered":"2024-09-19T08:13:00.840608+0000","last_unstale":"2024-09-19T08:14:07.731759+0000","last_undegraded":"2024-09-19T08:14:07.731759+0000","last_fullsized":"2024-09-19T08:14:07.731759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:25.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'11","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735315+0000","last_change":"2024-09-19T08:12:58.672929+0000","last_active":"2024-09-19T08:14:07.735315+0000","last_peered":"2024-09-19T08:14:07.735315+0000","last_clean":"2024-09-19T08:14:07.735315+0000","last_became_active":"2024-09-19T08:12:58.672487+0000","last_became_peered":"2024-09-19T08:12:58.672487+0000","last_unstale":"2024-09-19T08:14:07.735315+0000","last_undegraded":"2024-09-19T08:14:07.735315+0000","last_fullsized":"2024-09-19T08:14:07.735315+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:33.042735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738260+0000","last_change":"2024-09-19T08:14:07.738260+0000","last_active":"2024-09-19T08:14:07.738260+0000","last_peered":"2024-09-19T08:14:07.738260+0000","last_clean":"2024-09-19T08:14:07.738260+0000","last_became_active":"2024-09-19T08:14:07.737915+0000","last_became_peered":"2024-09-19T08:14:07.737915+0000","last_unstale":"2024-09-19T08:14:07.738260+0000","last_undegraded":"2024-09-19T08:14:07.738260+0000","last_fullsized":"2024-09-19T08:14:07.738260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:02.500657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.739005+0000","last_change":"2024-09-19T08:12:59.179510+0000","last_active":"2024-09-19T08:14:07.739005+0000","last_peered":"2024-09-19T08:14:07.739005+0000","last_clean":"2024-09-19T08:14:07.739005+0000","last_became_active":"2024-09-19T08:12:59.179157+0000","last_became_peered":"2024-09-19T08:12:59.179157+0000","last_unstale":"2024-09-19T08:14:07.739005+0000","last_undegraded":"2024-09-19T08:14:07.739005+0000","last_fullsized":"2024-09-19T08:14:07.739005+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:22.981289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733930+0000","last_change":"2024-09-19T08:13:00.841853+0000","last_active":"2024-09-19T08:14:07.733930+0000","last_peered":"2024-09-19T08:14:07.733930+0000","last_clean":"2024-09-19T08:14:07.733930+0000","last_became_active":"2024-09-19T08:13:00.841518+0000","last_became_peered":"2024-09-19T08:13:00.841518+0000","last_unstale":"2024-09-19T08:14:07.733930+0000","last_undegraded":"2024-09-19T08:14:07.733930+0000","last_fullsized":"2024-09-19T08:14:07.733930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:05.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733280+0000","last_change":"2024-09-19T08:12:58.716882+0000","last_active":"2024-09-19T08:14:07.733280+0000","last_peered":"2024-09-19T08:14:07.733280+0000","last_clean":"2024-09-19T08:14:07.733280+0000","last_became_active":"2024-09-19T08:12:58.716545+0000","last_became_peered":"2024-09-19T08:12:58.716545+0000","last_unstale":"2024-09-19T08:14:07.733280+0000","last_undegraded":"2024-09-19T08:14:07.733280+0000","last_fullsized":"2024-09-19T08:14:07.733280+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:14.469109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.739825+0000","last_change":"2024-09-19T08:13:00.850484+0000","last_active":"2024-09-19T08:14:07.739825+0000","last_peered":"2024-09-19T08:14:07.739825+0000","last_clean":"2024-09-19T08:14:07.739825+0000","last_became_active":"2024-09-19T08:13:00.850049+0000","last_became_peered":"2024-09-19T08:13:00.850049+0000","last_unstale":"2024-09-19T08:14:07.739825+0000","last_undegraded":"2024-09-19T08:14:07.739825+0000","last_fullsized":"2024-09-19T08:14:07.739825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:24.424111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399703+0000","last_change":"2024-09-19T08:12:58.669928+0000","last_active":"2024-09-19T08:14:08.399703+0000","last_peered":"2024-09-19T08:14:08.399703+0000","last_clean":"2024-09-19T08:14:08.399703+0000","last_became_active":"2024-09-19T08:12:58.669614+0000","last_became_peered":"2024-09-19T08:12:58.669614+0000","last_unstale":"2024-09-19T08:14:08.399703+0000","last_undegraded":"2024-09-19T08:14:08.399703+0000","last_fullsized":"2024-09-19T08:14:08.399703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:50.924779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734603+0000","last_change":"2024-09-19T08:13:00.853520+0000","last_active":"2024-09-19T08:14:07.734603+0000","last_peered":"2024-09-19T08:14:07.734603+0000","last_clean":"2024-09-19T08:14:07.734603+0000","last_became_active":"2024-09-19T08:13:00.853184+0000","last_became_peered":"2024-09-19T08:13:00.853184+0000","last_unstale":"2024-09-19T08:14:07.734603+0000","last_undegraded":"2024-09-19T08:14:07.734603+0000","last_fullsized":"2024-09-19T08:14:07.734603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:45.160439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732133+0000","last_change":"2024-09-19T08:12:58.674359+0000","last_active":"2024-09-19T08:14:07.732133+0000","last_peered":"2024-09-19T08:14:07.732133+0000","last_clean":"2024-09-19T08:14:07.732133+0000","last_became_active":"2024-09-19T08:12:58.673890+0000","last_became_peered":"2024-09-19T08:12:58.673890+0000","last_unstale":"2024-09-19T08:14:07.732133+0000","last_undegraded":"2024-09-19T08:14:07.732133+0000","last_fullsized":"2024-09-19T08:14:07.732133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:54.939959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399978+0000","last_change":"2024-09-19T08:13:00.851071+0000","last_active":"2024-09-19T08:14:08.399978+0000","last_peered":"2024-09-19T08:14:08.399978+0000","last_clean":"2024-09-19T08:14:08.399978+0000","last_became_active":"2024-09-19T08:13:00.850756+0000","last_became_peered":"2024-09-19T08:13:00.850756+0000","last_unstale":"2024-09-19T08:14:08.399978+0000","last_undegraded":"2024-09-19T08:14:08.399978+0000","last_fullsized":"2024-09-19T08:14:08.399978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:52.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738813+0000","last_change":"2024-09-19T08:12:58.681466+0000","last_active":"2024-09-19T08:14:07.738813+0000","last_peered":"2024-09-19T08:14:07.738813+0000","last_clean":"2024-09-19T08:14:07.738813+0000","last_became_active":"2024-09-19T08:12:58.681113+0000","last_became_peered":"2024-09-19T08:12:58.681113+0000","last_unstale":"2024-09-19T08:14:07.738813+0000","last_undegraded":"2024-09-19T08:14:07.738813+0000","last_fullsized":"2024-09-19T08:14:07.738813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:49.610828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734635+0000","last_change":"2024-09-19T08:13:00.851272+0000","last_active":"2024-09-19T08:14:07.734635+0000","last_peered":"2024-09-19T08:14:07.734635+0000","last_clean":"2024-09-19T08:14:07.734635+0000","last_became_active":"2024-09-19T08:13:00.849098+0000","last_became_peered":"2024-09-19T08:13:00.849098+0000","last_unstale":"2024-09-19T08:14:07.734635+0000","last_undegraded":"2024-09-19T08:14:07.734635+0000","last_fullsized":"2024-09-19T08:14:07.734635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:05.643992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738913+0000","last_change":"2024-09-19T08:14:07.738913+0000","last_active":"2024-09-19T08:14:07.738913+0000","last_peered":"2024-09-19T08:14:07.738913+0000","last_clean":"2024-09-19T08:14:07.738913+0000","last_became_active":"2024-09-19T08:14:07.738560+0000","last_became_peered":"2024-09-19T08:14:07.738560+0000","last_unstale":"2024-09-19T08:14:07.738913+0000","last_undegraded":"2024-09-19T08:14:07.738913+0000","last_fullsized":"2024-09-19T08:14:07.738913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:35.731290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738679+0000","last_change":"2024-09-19T08:12:58.674610+0000","last_active":"2024-09-19T08:14:07.738679+0000","last_peered":"2024-09-19T08:14:07.738679+0000","last_clean":"2024-09-19T08:14:07.738679+0000","last_became_active":"2024-09-19T08:12:58.674247+0000","last_became_peered":"2024-09-19T08:12:58.674247+0000","last_unstale":"2024-09-19T08:14:07.738679+0000","last_undegraded":"2024-09-19T08:14:07.738679+0000","last_fullsized":"2024-09-19T08:14:07.738679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:16.323574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732898+0000","last_change":"2024-09-19T08:13:00.858512+0000","last_active":"2024-09-19T08:14:07.732898+0000","last_peered":"2024-09-19T08:14:07.732898+0000","last_clean":"2024-09-19T08:14:07.732898+0000","last_became_active":"2024-09-19T08:13:00.857897+0000","last_became_peered":"2024-09-19T08:13:00.857897+0000","last_unstale":"2024-09-19T08:14:07.732898+0000","last_undegraded":"2024-09-19T08:14:07.732898+0000","last_fullsized":"2024-09-19T08:14:07.732898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738975+0000","last_change":"2024-09-19T08:12:58.682207+0000","last_active":"2024-09-19T08:14:07.738975+0000","last_peered":"2024-09-19T08:14:07.738975+0000","last_clean":"2024-09-19T08:14:07.738975+0000","last_became_active":"2024-09-19T08:12:58.681842+0000","last_became_peered":"2024-09-19T08:12:58.681842+0000","last_unstale":"2024-09-19T08:14:07.738975+0000","last_undegraded":"2024-09-19T08:14:07.738975+0000","last_fullsized":"2024-09-19T08:14:07.738975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.825858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733650+0000","last_change":"2024-09-19T08:13:00.846748+0000","last_active":"2024-09-19T08:14:07.733650+0000","last_peered":"2024-09-19T08:14:07.733650+0000","last_clean":"2024-09-19T08:14:07.733650+0000","last_became_active":"2024-09-19T08:13:00.846368+0000","last_became_peered":"2024-09-19T08:13:00.846368+0000","last_unstale":"2024-09-19T08:14:07.733650+0000","last_undegraded":"2024-09-19T08:14:07.733650+0000","last_fullsized":"2024-09-19T08:14:07.733650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:59.526512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.400239+0000","last_change":"2024-09-19T08:12:58.716118+0000","last_active":"2024-09-19T08:14:08.400239+0000","last_peered":"2024-09-19T08:14:08.400239+0000","last_clean":"2024-09-19T08:14:08.400239+0000","last_became_active":"2024-09-19T08:12:58.715711+0000","last_became_peered":"2024-09-19T08:12:58.715711+0000","last_unstale":"2024-09-19T08:14:08.400239+0000","last_undegraded":"2024-09-19T08:14:08.400239+0000","last_fullsized":"2024-09-19T08:14:08.400239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:29.439887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.400041+0000","last_change":"2024-09-19T08:13:00.848729+0000","last_active":"2024-09-19T08:14:08.400041+0000","last_peered":"2024-09-19T08:14:08.400041+0000","last_clean":"2024-09-19T08:14:08.400041+0000","last_became_active":"2024-09-19T08:13:00.848470+0000","last_became_peered":"2024-09-19T08:13:00.848470+0000","last_unstale":"2024-09-19T08:14:08.400041+0000","last_undegraded":"2024-09-19T08:14:08.400041+0000","last_fullsized":"2024-09-19T08:14:08.400041+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:30.653101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735443+0000","last_change":"2024-09-19T08:12:58.681654+0000","last_active":"2024-09-19T08:14:07.735443+0000","last_peered":"2024-09-19T08:14:07.735443+0000","last_clean":"2024-09-19T08:14:07.735443+0000","last_became_active":"2024-09-19T08:12:58.681284+0000","last_became_peered":"2024-09-19T08:12:58.681284+0000","last_unstale":"2024-09-19T08:14:07.735443+0000","last_undegraded":"2024-09-19T08:14:07.735443+0000","last_fullsized":"2024-09-19T08:14:07.735443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.429930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65568,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":375,"num_read_kb":253,"num_write":164,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":61306,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":196704,"data_compressed":9611,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251470,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481819,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":445932,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710752,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1069056,"data_stored":1098844,"data_compressed":42203,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874278,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":933888,"data_stored":446034,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103211,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":921600,"data_stored":445820,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332143,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":983040,"data_stored":1038279,"data_compressed":42203,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495668,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":921600,"data_stored":445423,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":995328,"data_stored":1037570,"data_compressed":42203,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":950272,"data_stored":507425,"data_compressed":15422,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":937984,"data_stored":445734,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":446959,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":983040,"data_stored":506388,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":446694,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34755,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34755,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":62209,"data_compressed":9611,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:14:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:15 smithi157 ceph-mon[35143]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1016 B/s rd, 2 op/s; 7.4 KiB/s, 1 objects/s recovering 2024-09-19T08:14:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:15 smithi157 ceph-mon[35143]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:18.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:17 smithi064 ceph-mon[34900]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 6.0 KiB/s, 1 objects/s recovering 2024-09-19T08:14:18.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:17 smithi157 ceph-mon[35143]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 6.0 KiB/s, 1 objects/s recovering 2024-09-19T08:14:18.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:17 smithi022 ceph-mon[30963]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 6.0 KiB/s, 1 objects/s recovering 2024-09-19T08:14:19.118 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:14:19.391 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.939+0000 7ff272326640 1 -- 172.21.15.22:0/3352839724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c10c170 msgr2=0x7ff26c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.939+0000 7ff272326640 1 --2- 172.21.15.22:0/3352839724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c10c170 0x7ff26c10e610 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff264009ca0 tx=0x7ff26402a950 comp rx=0 tx=0).stop 2024-09-19T08:14:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.939+0000 7ff272326640 1 -- 172.21.15.22:0/3352839724 shutdown_connections 2024-09-19T08:14:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.939+0000 7ff272326640 1 --2- 172.21.15.22:0/3352839724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c10c170 0x7ff26c10e610 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.939+0000 7ff272326640 1 --2- 172.21.15.22:0/3352839724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff26c103660 0x7ff26c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.940+0000 7ff272326640 1 --2- 172.21.15.22:0/3352839724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26c102d40 0x7ff26c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.940+0000 7ff272326640 1 -- 172.21.15.22:0/3352839724 >> 172.21.15.22:0/3352839724 conn(0x7ff26c0fc7a0 msgr2=0x7ff26c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.940+0000 7ff272326640 1 -- 172.21.15.22:0/3352839724 shutdown_connections 2024-09-19T08:14:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.940+0000 7ff272326640 1 -- 172.21.15.22:0/3352839724 wait complete. 2024-09-19T08:14:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.940+0000 7ff272326640 1 Processor -- start 2024-09-19T08:14:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.940+0000 7ff272326640 1 -- start start 2024-09-19T08:14:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.940+0000 7ff272326640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26c102d40 0x7ff26c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.940+0000 7ff272326640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff26c103660 0x7ff26c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.940+0000 7ff272326640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c10c170 0x7ff26c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.940+0000 7ff272326640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c111f50 con 0x7ff26c10c170 2024-09-19T08:14:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.940+0000 7ff272326640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c111dd0 con 0x7ff26c102d40 2024-09-19T08:14:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.940+0000 7ff272326640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c1120d0 con 0x7ff26c103660 2024-09-19T08:14:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff2708c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c10c170 0x7ff26c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff263fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff26c103660 0x7ff26c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff2708c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c10c170 0x7ff26c19eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38966/0 (socket says 172.21.15.22:38966) 2024-09-19T08:14:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff2708c3640 1 -- 172.21.15.22:0/3553033856 learned_addr learned my addr 172.21.15.22:0/3553033856 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff26bfff640 1 --2- 172.21.15.22:0/3553033856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26c102d40 0x7ff26c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff2708c3640 1 -- 172.21.15.22:0/3553033856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff26c103660 msgr2=0x7ff26c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff2708c3640 1 --2- 172.21.15.22:0/3553033856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff26c103660 0x7ff26c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff2708c3640 1 -- 172.21.15.22:0/3553033856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26c102d40 msgr2=0x7ff26c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff2708c3640 1 --2- 172.21.15.22:0/3553033856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26c102d40 0x7ff26c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff2708c3640 1 -- 172.21.15.22:0/3553033856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c19f590 con 0x7ff26c10c170 2024-09-19T08:14:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff2708c3640 1 --2- 172.21.15.22:0/3553033856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c10c170 0x7ff26c19eeb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff264004af0 tx=0x7ff26402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff269ffb640 1 -- 172.21.15.22:0/3553033856 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264034040 con 0x7ff26c10c170 2024-09-19T08:14:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff269ffb640 1 -- 172.21.15.22:0/3553033856 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff264030920 con 0x7ff26c10c170 2024-09-19T08:14:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff272326640 1 -- 172.21.15.22:0/3553033856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26c19f820 con 0x7ff26c10c170 2024-09-19T08:14:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff269ffb640 1 -- 172.21.15.22:0/3553033856 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2640047b0 con 0x7ff26c10c170 2024-09-19T08:14:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.941+0000 7ff272326640 1 -- 172.21.15.22:0/3553033856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26c1a7160 con 0x7ff26c10c170 2024-09-19T08:14:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.942+0000 7ff272326640 1 -- 172.21.15.22:0/3553033856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26c103120 con 0x7ff26c10c170 2024-09-19T08:14:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.943+0000 7ff269ffb640 1 -- 172.21.15.22:0/3553033856 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff264037020 con 0x7ff26c10c170 2024-09-19T08:14:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.944+0000 7ff269ffb640 1 --2- 172.21.15.22:0/3553033856 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff2580a3350 0x7ff2580a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.944+0000 7ff26bfff640 1 --2- 172.21.15.22:0/3553033856 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff2580a3350 0x7ff2580a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.944+0000 7ff269ffb640 1 -- 172.21.15.22:0/3553033856 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff2640f06b0 con 0x7ff26c10c170 2024-09-19T08:14:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.945+0000 7ff26bfff640 1 --2- 172.21.15.22:0/3553033856 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff2580a3350 0x7ff2580a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00bbe0 tx=0x7ff25c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:19.948+0000 7ff269ffb640 1 -- 172.21.15.22:0/3553033856 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff2640bd380 con 0x7ff26c10c170 2024-09-19T08:14:20.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:20.119+0000 7ff272326640 1 -- 172.21.15.22:0/3553033856 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff26c0fe2a0 con 0x7ff2580a3350 2024-09-19T08:14:20.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:19 smithi022 ceph-mon[30963]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s; 5.4 KiB/s, 1 objects/s recovering 2024-09-19T08:14:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:20.125+0000 7ff269ffb640 1 -- 172.21.15.22:0/3553033856 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258984 (secure 0 0 0) 0x7ff26c0fe2a0 con 0x7ff2580a3350 2024-09-19T08:14:20.127 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:20.128 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T08:14:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:20.128+0000 7ff272326640 1 -- 172.21.15.22:0/3553033856 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff2580a3350 msgr2=0x7ff2580a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:20.128+0000 7ff272326640 1 --2- 172.21.15.22:0/3553033856 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff2580a3350 0x7ff2580a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00bbe0 tx=0x7ff25c00b390 comp rx=0 tx=0).stop 2024-09-19T08:14:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:20.128+0000 7ff272326640 1 -- 172.21.15.22:0/3553033856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c10c170 msgr2=0x7ff26c19eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:20.128+0000 7ff272326640 1 --2- 172.21.15.22:0/3553033856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c10c170 0x7ff26c19eeb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff264004af0 tx=0x7ff26402f5b0 comp rx=0 tx=0).stop 2024-09-19T08:14:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:20.128+0000 7ff272326640 1 -- 172.21.15.22:0/3553033856 shutdown_connections 2024-09-19T08:14:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:20.128+0000 7ff272326640 1 --2- 172.21.15.22:0/3553033856 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff2580a3350 0x7ff2580a5810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:20.128+0000 7ff272326640 1 --2- 172.21.15.22:0/3553033856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c10c170 0x7ff26c19eeb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:20.128+0000 7ff272326640 1 --2- 172.21.15.22:0/3553033856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff26c103660 0x7ff26c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:20.128+0000 7ff272326640 1 --2- 172.21.15.22:0/3553033856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26c102d40 0x7ff26c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:20.128+0000 7ff272326640 1 -- 172.21.15.22:0/3553033856 >> 172.21.15.22:0/3553033856 conn(0x7ff26c0fc7a0 msgr2=0x7ff26c0fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:20.128+0000 7ff272326640 1 -- 172.21.15.22:0/3553033856 shutdown_connections 2024-09-19T08:14:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:20.128+0000 7ff272326640 1 -- 172.21.15.22:0/3553033856 wait complete. 2024-09-19T08:14:20.177 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:19 smithi064 ceph-mon[34900]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s; 5.4 KiB/s, 1 objects/s recovering 2024-09-19T08:14:20.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:19 smithi157 ceph-mon[35143]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s; 5.4 KiB/s, 1 objects/s recovering 2024-09-19T08:14:20.874 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-19T08:14:18.781612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656170,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":536,"num_read_kb":387,"num_write":347,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1242042,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381360,"kb_used_data":11236,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560464,"statfs":{"total":1151940427776,"available":1151549915136,"internally_reserved":0,"allocated":11505664,"data_stored":7311102,"data_compressed":188519,"data_compressed_allocated":245760,"data_compressed_original":2023424,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":23,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":61306,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.051548"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735251+0000","last_change":"2024-09-19T08:13:00.853947+0000","last_active":"2024-09-19T08:14:07.735251+0000","last_peered":"2024-09-19T08:14:07.735251+0000","last_clean":"2024-09-19T08:14:07.735251+0000","last_became_active":"2024-09-19T08:13:00.852951+0000","last_became_peered":"2024-09-19T08:13:00.852951+0000","last_unstale":"2024-09-19T08:14:07.735251+0000","last_undegraded":"2024-09-19T08:14:07.735251+0000","last_fullsized":"2024-09-19T08:14:07.735251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:25.786828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733811+0000","last_change":"2024-09-19T08:13:00.853389+0000","last_active":"2024-09-19T08:14:07.733811+0000","last_peered":"2024-09-19T08:14:07.733811+0000","last_clean":"2024-09-19T08:14:07.733811+0000","last_became_active":"2024-09-19T08:13:00.853102+0000","last_became_peered":"2024-09-19T08:13:00.853102+0000","last_unstale":"2024-09-19T08:14:07.733811+0000","last_undegraded":"2024-09-19T08:14:07.733811+0000","last_fullsized":"2024-09-19T08:14:07.733811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:42.773917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733742+0000","last_change":"2024-09-19T08:13:00.846544+0000","last_active":"2024-09-19T08:14:07.733742+0000","last_peered":"2024-09-19T08:14:07.733742+0000","last_clean":"2024-09-19T08:14:07.733742+0000","last_became_active":"2024-09-19T08:13:00.846229+0000","last_became_peered":"2024-09-19T08:13:00.846229+0000","last_unstale":"2024-09-19T08:14:07.733742+0000","last_undegraded":"2024-09-19T08:14:07.733742+0000","last_fullsized":"2024-09-19T08:14:07.733742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:43.471684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.739325+0000","last_change":"2024-09-19T08:13:00.848149+0000","last_active":"2024-09-19T08:14:07.739325+0000","last_peered":"2024-09-19T08:14:07.739325+0000","last_clean":"2024-09-19T08:14:07.739325+0000","last_became_active":"2024-09-19T08:13:00.847744+0000","last_became_peered":"2024-09-19T08:13:00.847744+0000","last_unstale":"2024-09-19T08:14:07.739325+0000","last_undegraded":"2024-09-19T08:14:07.739325+0000","last_fullsized":"2024-09-19T08:14:07.739325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:01.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738770+0000","last_change":"2024-09-19T08:13:00.841862+0000","last_active":"2024-09-19T08:14:07.738770+0000","last_peered":"2024-09-19T08:14:07.738770+0000","last_clean":"2024-09-19T08:14:07.738770+0000","last_became_active":"2024-09-19T08:13:00.841331+0000","last_became_peered":"2024-09-19T08:13:00.841331+0000","last_unstale":"2024-09-19T08:14:07.738770+0000","last_undegraded":"2024-09-19T08:14:07.738770+0000","last_fullsized":"2024-09-19T08:14:07.738770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:54.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735492+0000","last_change":"2024-09-19T08:13:00.849626+0000","last_active":"2024-09-19T08:14:07.735492+0000","last_peered":"2024-09-19T08:14:07.735492+0000","last_clean":"2024-09-19T08:14:07.735492+0000","last_became_active":"2024-09-19T08:13:00.849034+0000","last_became_peered":"2024-09-19T08:13:00.849034+0000","last_unstale":"2024-09-19T08:14:07.735492+0000","last_undegraded":"2024-09-19T08:14:07.735492+0000","last_fullsized":"2024-09-19T08:14:07.735492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:39.118978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734312+0000","last_change":"2024-09-19T08:13:00.847522+0000","last_active":"2024-09-19T08:14:07.734312+0000","last_peered":"2024-09-19T08:14:07.734312+0000","last_clean":"2024-09-19T08:14:07.734312+0000","last_became_active":"2024-09-19T08:13:00.847117+0000","last_became_peered":"2024-09-19T08:13:00.847117+0000","last_unstale":"2024-09-19T08:14:07.734312+0000","last_undegraded":"2024-09-19T08:14:07.734312+0000","last_fullsized":"2024-09-19T08:14:07.734312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.836385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399370+0000","last_change":"2024-09-19T08:13:00.849544+0000","last_active":"2024-09-19T08:14:08.399370+0000","last_peered":"2024-09-19T08:14:08.399370+0000","last_clean":"2024-09-19T08:14:08.399370+0000","last_became_active":"2024-09-19T08:13:00.849170+0000","last_became_peered":"2024-09-19T08:13:00.849170+0000","last_unstale":"2024-09-19T08:14:08.399370+0000","last_undegraded":"2024-09-19T08:14:08.399370+0000","last_fullsized":"2024-09-19T08:14:08.399370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:28.371965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733158+0000","last_change":"2024-09-19T08:13:00.856052+0000","last_active":"2024-09-19T08:14:07.733158+0000","last_peered":"2024-09-19T08:14:07.733158+0000","last_clean":"2024-09-19T08:14:07.733158+0000","last_became_active":"2024-09-19T08:13:00.855291+0000","last_became_peered":"2024-09-19T08:13:00.855291+0000","last_unstale":"2024-09-19T08:14:07.733158+0000","last_undegraded":"2024-09-19T08:14:07.733158+0000","last_fullsized":"2024-09-19T08:14:07.733158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:18.581867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735684+0000","last_change":"2024-09-19T08:13:00.850145+0000","last_active":"2024-09-19T08:14:07.735684+0000","last_peered":"2024-09-19T08:14:07.735684+0000","last_clean":"2024-09-19T08:14:07.735684+0000","last_became_active":"2024-09-19T08:13:00.849798+0000","last_became_peered":"2024-09-19T08:13:00.849798+0000","last_unstale":"2024-09-19T08:14:07.735684+0000","last_undegraded":"2024-09-19T08:14:07.735684+0000","last_fullsized":"2024-09-19T08:14:07.735684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:21.400000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734734+0000","last_change":"2024-09-19T08:13:00.859153+0000","last_active":"2024-09-19T08:14:07.734734+0000","last_peered":"2024-09-19T08:14:07.734734+0000","last_clean":"2024-09-19T08:14:07.734734+0000","last_became_active":"2024-09-19T08:13:00.858473+0000","last_became_peered":"2024-09-19T08:13:00.858473+0000","last_unstale":"2024-09-19T08:14:07.734734+0000","last_undegraded":"2024-09-19T08:14:07.734734+0000","last_fullsized":"2024-09-19T08:14:07.734734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:07.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.739385+0000","last_change":"2024-09-19T08:13:00.850797+0000","last_active":"2024-09-19T08:14:07.739385+0000","last_peered":"2024-09-19T08:14:07.739385+0000","last_clean":"2024-09-19T08:14:07.739385+0000","last_became_active":"2024-09-19T08:13:00.850393+0000","last_became_peered":"2024-09-19T08:13:00.850393+0000","last_unstale":"2024-09-19T08:14:07.739385+0000","last_undegraded":"2024-09-19T08:14:07.739385+0000","last_fullsized":"2024-09-19T08:14:07.739385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:59.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731520+0000","last_change":"2024-09-19T08:13:00.850835+0000","last_active":"2024-09-19T08:14:07.731520+0000","last_peered":"2024-09-19T08:14:07.731520+0000","last_clean":"2024-09-19T08:14:07.731520+0000","last_became_active":"2024-09-19T08:13:00.850518+0000","last_became_peered":"2024-09-19T08:13:00.850518+0000","last_unstale":"2024-09-19T08:14:07.731520+0000","last_undegraded":"2024-09-19T08:14:07.731520+0000","last_fullsized":"2024-09-19T08:14:07.731520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.178530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733525+0000","last_change":"2024-09-19T08:13:00.853934+0000","last_active":"2024-09-19T08:14:07.733525+0000","last_peered":"2024-09-19T08:14:07.733525+0000","last_clean":"2024-09-19T08:14:07.733525+0000","last_became_active":"2024-09-19T08:13:00.853604+0000","last_became_peered":"2024-09-19T08:13:00.853604+0000","last_unstale":"2024-09-19T08:14:07.733525+0000","last_undegraded":"2024-09-19T08:14:07.733525+0000","last_fullsized":"2024-09-19T08:14:07.733525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:43.622526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.741370+0000","last_change":"2024-09-19T08:14:07.741370+0000","last_active":"2024-09-19T08:14:07.741370+0000","last_peered":"2024-09-19T08:14:07.741370+0000","last_clean":"2024-09-19T08:14:07.741370+0000","last_became_active":"2024-09-19T08:14:07.741116+0000","last_became_peered":"2024-09-19T08:14:07.741116+0000","last_unstale":"2024-09-19T08:14:07.741370+0000","last_undegraded":"2024-09-19T08:14:07.741370+0000","last_fullsized":"2024-09-19T08:14:07.741370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:45.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732701+0000","last_change":"2024-09-19T08:13:00.850266+0000","last_active":"2024-09-19T08:14:07.732701+0000","last_peered":"2024-09-19T08:14:07.732701+0000","last_clean":"2024-09-19T08:14:07.732701+0000","last_became_active":"2024-09-19T08:13:00.849928+0000","last_became_peered":"2024-09-19T08:13:00.849928+0000","last_unstale":"2024-09-19T08:14:07.732701+0000","last_undegraded":"2024-09-19T08:14:07.732701+0000","last_fullsized":"2024-09-19T08:14:07.732701+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:38.231331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735367+0000","last_change":"2024-09-19T08:13:00.852551+0000","last_active":"2024-09-19T08:14:07.735367+0000","last_peered":"2024-09-19T08:14:07.735367+0000","last_clean":"2024-09-19T08:14:07.735367+0000","last_became_active":"2024-09-19T08:13:00.852225+0000","last_became_peered":"2024-09-19T08:13:00.852225+0000","last_unstale":"2024-09-19T08:14:07.735367+0000","last_undegraded":"2024-09-19T08:14:07.735367+0000","last_fullsized":"2024-09-19T08:14:07.735367+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:33.115108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735168+0000","last_change":"2024-09-19T08:13:00.854033+0000","last_active":"2024-09-19T08:14:07.735168+0000","last_peered":"2024-09-19T08:14:07.735168+0000","last_clean":"2024-09-19T08:14:07.735168+0000","last_became_active":"2024-09-19T08:13:00.853660+0000","last_became_peered":"2024-09-19T08:13:00.853660+0000","last_unstale":"2024-09-19T08:14:07.735168+0000","last_undegraded":"2024-09-19T08:14:07.735168+0000","last_fullsized":"2024-09-19T08:14:07.735168+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:35.602420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732530+0000","last_change":"2024-09-19T08:13:00.850658+0000","last_active":"2024-09-19T08:14:07.732530+0000","last_peered":"2024-09-19T08:14:07.732530+0000","last_clean":"2024-09-19T08:14:07.732530+0000","last_became_active":"2024-09-19T08:13:00.850324+0000","last_became_peered":"2024-09-19T08:13:00.850324+0000","last_unstale":"2024-09-19T08:14:07.732530+0000","last_undegraded":"2024-09-19T08:14:07.732530+0000","last_fullsized":"2024-09-19T08:14:07.732530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:59:49.004605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.741837+0000","last_change":"2024-09-19T08:14:07.741837+0000","last_active":"2024-09-19T08:14:07.741837+0000","last_peered":"2024-09-19T08:14:07.741837+0000","last_clean":"2024-09-19T08:14:07.741837+0000","last_became_active":"2024-09-19T08:14:07.741398+0000","last_became_peered":"2024-09-19T08:14:07.741398+0000","last_unstale":"2024-09-19T08:14:07.741837+0000","last_undegraded":"2024-09-19T08:14:07.741837+0000","last_fullsized":"2024-09-19T08:14:07.741837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:56.716663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731582+0000","last_change":"2024-09-19T08:13:00.855215+0000","last_active":"2024-09-19T08:14:07.731582+0000","last_peered":"2024-09-19T08:14:07.731582+0000","last_clean":"2024-09-19T08:14:07.731582+0000","last_became_active":"2024-09-19T08:13:00.854878+0000","last_became_peered":"2024-09-19T08:13:00.854878+0000","last_unstale":"2024-09-19T08:14:07.731582+0000","last_undegraded":"2024-09-19T08:14:07.731582+0000","last_fullsized":"2024-09-19T08:14:07.731582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:48.216066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733505+0000","last_change":"2024-09-19T08:13:00.856995+0000","last_active":"2024-09-19T08:14:07.733505+0000","last_peered":"2024-09-19T08:14:07.733505+0000","last_clean":"2024-09-19T08:14:07.733505+0000","last_became_active":"2024-09-19T08:13:00.856645+0000","last_became_peered":"2024-09-19T08:13:00.856645+0000","last_unstale":"2024-09-19T08:14:07.733505+0000","last_undegraded":"2024-09-19T08:14:07.733505+0000","last_fullsized":"2024-09-19T08:14:07.733505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:20.699293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399539+0000","last_change":"2024-09-19T08:13:00.853362+0000","last_active":"2024-09-19T08:14:08.399539+0000","last_peered":"2024-09-19T08:14:08.399539+0000","last_clean":"2024-09-19T08:14:08.399539+0000","last_became_active":"2024-09-19T08:13:00.853046+0000","last_became_peered":"2024-09-19T08:13:00.853046+0000","last_unstale":"2024-09-19T08:14:08.399539+0000","last_undegraded":"2024-09-19T08:14:08.399539+0000","last_fullsized":"2024-09-19T08:14:08.399539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736509+0000","last_change":"2024-09-19T08:13:00.845175+0000","last_active":"2024-09-19T08:14:07.736509+0000","last_peered":"2024-09-19T08:14:07.736509+0000","last_clean":"2024-09-19T08:14:07.736509+0000","last_became_active":"2024-09-19T08:13:00.844857+0000","last_became_peered":"2024-09-19T08:13:00.844857+0000","last_unstale":"2024-09-19T08:14:07.736509+0000","last_undegraded":"2024-09-19T08:14:07.736509+0000","last_fullsized":"2024-09-19T08:14:07.736509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:40.818956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399481+0000","last_change":"2024-09-19T08:13:00.847769+0000","last_active":"2024-09-19T08:14:08.399481+0000","last_peered":"2024-09-19T08:14:08.399481+0000","last_clean":"2024-09-19T08:14:08.399481+0000","last_became_active":"2024-09-19T08:13:00.847454+0000","last_became_peered":"2024-09-19T08:13:00.847454+0000","last_unstale":"2024-09-19T08:14:08.399481+0000","last_undegraded":"2024-09-19T08:14:08.399481+0000","last_fullsized":"2024-09-19T08:14:08.399481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.583051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732671+0000","last_change":"2024-09-19T08:13:00.859882+0000","last_active":"2024-09-19T08:14:07.732671+0000","last_peered":"2024-09-19T08:14:07.732671+0000","last_clean":"2024-09-19T08:14:07.732671+0000","last_became_active":"2024-09-19T08:13:00.859491+0000","last_became_peered":"2024-09-19T08:13:00.859491+0000","last_unstale":"2024-09-19T08:14:07.732671+0000","last_undegraded":"2024-09-19T08:14:07.732671+0000","last_fullsized":"2024-09-19T08:14:07.732671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:13.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734271+0000","last_change":"2024-09-19T08:13:00.845584+0000","last_active":"2024-09-19T08:14:07.734271+0000","last_peered":"2024-09-19T08:14:07.734271+0000","last_clean":"2024-09-19T08:14:07.734271+0000","last_became_active":"2024-09-19T08:13:00.845166+0000","last_became_peered":"2024-09-19T08:13:00.845166+0000","last_unstale":"2024-09-19T08:14:07.734271+0000","last_undegraded":"2024-09-19T08:14:07.734271+0000","last_fullsized":"2024-09-19T08:14:07.734271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:02.705793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731667+0000","last_change":"2024-09-19T08:13:00.838069+0000","last_active":"2024-09-19T08:14:07.731667+0000","last_peered":"2024-09-19T08:14:07.731667+0000","last_clean":"2024-09-19T08:14:07.731667+0000","last_became_active":"2024-09-19T08:13:00.837736+0000","last_became_peered":"2024-09-19T08:13:00.837736+0000","last_unstale":"2024-09-19T08:14:07.731667+0000","last_undegraded":"2024-09-19T08:14:07.731667+0000","last_fullsized":"2024-09-19T08:14:07.731667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:01.388290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733793+0000","last_change":"2024-09-19T08:13:00.841507+0000","last_active":"2024-09-19T08:14:07.733793+0000","last_peered":"2024-09-19T08:14:07.733793+0000","last_clean":"2024-09-19T08:14:07.733793+0000","last_became_active":"2024-09-19T08:13:00.841161+0000","last_became_peered":"2024-09-19T08:13:00.841161+0000","last_unstale":"2024-09-19T08:14:07.733793+0000","last_undegraded":"2024-09-19T08:14:07.733793+0000","last_fullsized":"2024-09-19T08:14:07.733793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:41.862290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732148+0000","last_change":"2024-09-19T08:13:00.847657+0000","last_active":"2024-09-19T08:14:07.732148+0000","last_peered":"2024-09-19T08:14:07.732148+0000","last_clean":"2024-09-19T08:14:07.732148+0000","last_became_active":"2024-09-19T08:13:00.846849+0000","last_became_peered":"2024-09-19T08:13:00.846849+0000","last_unstale":"2024-09-19T08:14:07.732148+0000","last_undegraded":"2024-09-19T08:14:07.732148+0000","last_fullsized":"2024-09-19T08:14:07.732148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:35.219913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732183+0000","last_change":"2024-09-19T08:13:00.845971+0000","last_active":"2024-09-19T08:14:07.732183+0000","last_peered":"2024-09-19T08:14:07.732183+0000","last_clean":"2024-09-19T08:14:07.732183+0000","last_became_active":"2024-09-19T08:13:00.845626+0000","last_became_peered":"2024-09-19T08:13:00.845626+0000","last_unstale":"2024-09-19T08:14:07.732183+0000","last_undegraded":"2024-09-19T08:14:07.732183+0000","last_fullsized":"2024-09-19T08:14:07.732183+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:36.945627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738450+0000","last_change":"2024-09-19T08:14:07.738450+0000","last_active":"2024-09-19T08:14:07.738450+0000","last_peered":"2024-09-19T08:14:07.738450+0000","last_clean":"2024-09-19T08:14:07.738450+0000","last_became_active":"2024-09-19T08:14:07.738028+0000","last_became_peered":"2024-09-19T08:14:07.738028+0000","last_unstale":"2024-09-19T08:14:07.738450+0000","last_undegraded":"2024-09-19T08:14:07.738450+0000","last_fullsized":"2024-09-19T08:14:07.738450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:08.659158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738529+0000","last_change":"2024-09-19T08:13:00.848325+0000","last_active":"2024-09-19T08:14:07.738529+0000","last_peered":"2024-09-19T08:14:07.738529+0000","last_clean":"2024-09-19T08:14:07.738529+0000","last_became_active":"2024-09-19T08:13:00.848015+0000","last_became_peered":"2024-09-19T08:13:00.848015+0000","last_unstale":"2024-09-19T08:14:07.738529+0000","last_undegraded":"2024-09-19T08:14:07.738529+0000","last_fullsized":"2024-09-19T08:14:07.738529+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:10.948451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":95,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:13.967954+0000","last_change":"2024-09-19T08:12:58.681134+0000","last_active":"2024-09-19T08:14:13.967954+0000","last_peered":"2024-09-19T08:14:13.967954+0000","last_clean":"2024-09-19T08:14:13.967954+0000","last_became_active":"2024-09-19T08:12:58.680764+0000","last_became_peered":"2024-09-19T08:12:58.680764+0000","last_unstale":"2024-09-19T08:14:13.967954+0000","last_undegraded":"2024-09-19T08:14:13.967954+0000","last_fullsized":"2024-09-19T08:14:13.967954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:24.939263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736894+0000","last_change":"2024-09-19T08:13:00.844319+0000","last_active":"2024-09-19T08:14:07.736894+0000","last_peered":"2024-09-19T08:14:07.736894+0000","last_clean":"2024-09-19T08:14:07.736894+0000","last_became_active":"2024-09-19T08:13:00.843994+0000","last_became_peered":"2024-09-19T08:13:00.843994+0000","last_unstale":"2024-09-19T08:14:07.736894+0000","last_undegraded":"2024-09-19T08:14:07.736894+0000","last_fullsized":"2024-09-19T08:14:07.736894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:46.937466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735608+0000","last_change":"2024-09-19T08:12:58.676541+0000","last_active":"2024-09-19T08:14:07.735608+0000","last_peered":"2024-09-19T08:14:07.735608+0000","last_clean":"2024-09-19T08:14:07.735608+0000","last_became_active":"2024-09-19T08:12:58.675988+0000","last_became_peered":"2024-09-19T08:12:58.675988+0000","last_unstale":"2024-09-19T08:14:07.735608+0000","last_undegraded":"2024-09-19T08:14:07.735608+0000","last_fullsized":"2024-09-19T08:14:07.735608+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:01.434986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731256+0000","last_change":"2024-09-19T08:12:58.677510+0000","last_active":"2024-09-19T08:14:07.731256+0000","last_peered":"2024-09-19T08:14:07.731256+0000","last_clean":"2024-09-19T08:14:07.731256+0000","last_became_active":"2024-09-19T08:12:58.677200+0000","last_became_peered":"2024-09-19T08:12:58.677200+0000","last_unstale":"2024-09-19T08:14:07.731256+0000","last_undegraded":"2024-09-19T08:14:07.731256+0000","last_fullsized":"2024-09-19T08:14:07.731256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:36.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399481+0000","last_change":"2024-09-19T08:13:00.853430+0000","last_active":"2024-09-19T08:14:08.399481+0000","last_peered":"2024-09-19T08:14:08.399481+0000","last_clean":"2024-09-19T08:14:08.399481+0000","last_became_active":"2024-09-19T08:13:00.853096+0000","last_became_peered":"2024-09-19T08:13:00.853096+0000","last_unstale":"2024-09-19T08:14:08.399481+0000","last_undegraded":"2024-09-19T08:14:08.399481+0000","last_fullsized":"2024-09-19T08:14:08.399481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:07.147021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736819+0000","last_change":"2024-09-19T08:12:58.680081+0000","last_active":"2024-09-19T08:14:07.736819+0000","last_peered":"2024-09-19T08:14:07.736819+0000","last_clean":"2024-09-19T08:14:07.736819+0000","last_became_active":"2024-09-19T08:12:58.679704+0000","last_became_peered":"2024-09-19T08:12:58.679704+0000","last_unstale":"2024-09-19T08:14:07.736819+0000","last_undegraded":"2024-09-19T08:14:07.736819+0000","last_fullsized":"2024-09-19T08:14:07.736819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:32.942586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735021+0000","last_change":"2024-09-19T08:13:00.849394+0000","last_active":"2024-09-19T08:14:07.735021+0000","last_peered":"2024-09-19T08:14:07.735021+0000","last_clean":"2024-09-19T08:14:07.735021+0000","last_became_active":"2024-09-19T08:13:00.849051+0000","last_became_peered":"2024-09-19T08:13:00.849051+0000","last_unstale":"2024-09-19T08:14:07.735021+0000","last_undegraded":"2024-09-19T08:14:07.735021+0000","last_fullsized":"2024-09-19T08:14:07.735021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:23.012928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":89,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:13.967259+0000","last_change":"2024-09-19T08:12:58.676121+0000","last_active":"2024-09-19T08:14:13.967259+0000","last_peered":"2024-09-19T08:14:13.967259+0000","last_clean":"2024-09-19T08:14:13.967259+0000","last_became_active":"2024-09-19T08:12:58.675693+0000","last_became_peered":"2024-09-19T08:12:58.675693+0000","last_unstale":"2024-09-19T08:14:13.967259+0000","last_undegraded":"2024-09-19T08:14:13.967259+0000","last_fullsized":"2024-09-19T08:14:13.967259+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:34.882190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733023+0000","last_change":"2024-09-19T08:13:00.858735+0000","last_active":"2024-09-19T08:14:07.733023+0000","last_peered":"2024-09-19T08:14:07.733023+0000","last_clean":"2024-09-19T08:14:07.733023+0000","last_became_active":"2024-09-19T08:13:00.858218+0000","last_became_peered":"2024-09-19T08:13:00.858218+0000","last_unstale":"2024-09-19T08:14:07.733023+0000","last_undegraded":"2024-09-19T08:14:07.733023+0000","last_fullsized":"2024-09-19T08:14:07.733023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:33.356810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.743785+0000","last_change":"2024-09-19T08:14:07.743785+0000","last_active":"2024-09-19T08:14:07.743785+0000","last_peered":"2024-09-19T08:14:07.743785+0000","last_clean":"2024-09-19T08:14:07.743785+0000","last_became_active":"2024-09-19T08:14:07.739531+0000","last_became_peered":"2024-09-19T08:14:07.739531+0000","last_unstale":"2024-09-19T08:14:07.743785+0000","last_undegraded":"2024-09-19T08:14:07.743785+0000","last_fullsized":"2024-09-19T08:14:07.743785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:43.367704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733696+0000","last_change":"2024-09-19T08:13:00.843852+0000","last_active":"2024-09-19T08:14:07.733696+0000","last_peered":"2024-09-19T08:14:07.733696+0000","last_clean":"2024-09-19T08:14:07.733696+0000","last_became_active":"2024-09-19T08:13:00.843497+0000","last_became_peered":"2024-09-19T08:13:00.843497+0000","last_unstale":"2024-09-19T08:14:07.733696+0000","last_undegraded":"2024-09-19T08:14:07.733696+0000","last_fullsized":"2024-09-19T08:14:07.733696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:26.380024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.737669+0000","last_change":"2024-09-19T08:12:58.717225+0000","last_active":"2024-09-19T08:14:07.737669+0000","last_peered":"2024-09-19T08:14:07.737669+0000","last_clean":"2024-09-19T08:14:07.737669+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:14:07.737669+0000","last_undegraded":"2024-09-19T08:14:07.737669+0000","last_fullsized":"2024-09-19T08:14:07.737669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:43.201907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733016+0000","last_change":"2024-09-19T08:13:00.845613+0000","last_active":"2024-09-19T08:14:07.733016+0000","last_peered":"2024-09-19T08:14:07.733016+0000","last_clean":"2024-09-19T08:14:07.733016+0000","last_became_active":"2024-09-19T08:13:00.845260+0000","last_became_peered":"2024-09-19T08:13:00.845260+0000","last_unstale":"2024-09-19T08:14:07.733016+0000","last_undegraded":"2024-09-19T08:14:07.733016+0000","last_fullsized":"2024-09-19T08:14:07.733016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:03:47.033972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'29","reported_seq":204,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:14.753440+0000","last_change":"2024-09-19T08:14:07.749616+0000","last_active":"2024-09-19T08:14:14.753440+0000","last_peered":"2024-09-19T08:14:14.753440+0000","last_clean":"2024-09-19T08:14:14.753440+0000","last_became_active":"2024-09-19T08:14:07.740298+0000","last_became_peered":"2024-09-19T08:14:07.740298+0000","last_unstale":"2024-09-19T08:14:14.753440+0000","last_undegraded":"2024-09-19T08:14:14.753440+0000","last_fullsized":"2024-09-19T08:14:14.753440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:28.796855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61172,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":127,"num_read_kb":122,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61172,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738939+0000","last_change":"2024-09-19T08:13:00.855603+0000","last_active":"2024-09-19T08:14:07.738939+0000","last_peered":"2024-09-19T08:14:07.738939+0000","last_clean":"2024-09-19T08:14:07.738939+0000","last_became_active":"2024-09-19T08:13:00.855241+0000","last_became_peered":"2024-09-19T08:13:00.855241+0000","last_unstale":"2024-09-19T08:14:07.738939+0000","last_undegraded":"2024-09-19T08:14:07.738939+0000","last_fullsized":"2024-09-19T08:14:07.738939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:28.811847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732313+0000","last_change":"2024-09-19T08:12:58.675897+0000","last_active":"2024-09-19T08:14:07.732313+0000","last_peered":"2024-09-19T08:14:07.732313+0000","last_clean":"2024-09-19T08:14:07.732313+0000","last_became_active":"2024-09-19T08:12:58.675525+0000","last_became_peered":"2024-09-19T08:12:58.675525+0000","last_unstale":"2024-09-19T08:14:07.732313+0000","last_undegraded":"2024-09-19T08:14:07.732313+0000","last_fullsized":"2024-09-19T08:14:07.732313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:54.518064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731271+0000","last_change":"2024-09-19T08:13:00.856420+0000","last_active":"2024-09-19T08:14:07.731271+0000","last_peered":"2024-09-19T08:14:07.731271+0000","last_clean":"2024-09-19T08:14:07.731271+0000","last_became_active":"2024-09-19T08:13:00.856100+0000","last_became_peered":"2024-09-19T08:13:00.856100+0000","last_unstale":"2024-09-19T08:14:07.731271+0000","last_undegraded":"2024-09-19T08:14:07.731271+0000","last_fullsized":"2024-09-19T08:14:07.731271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:58.684930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.737378+0000","last_change":"2024-09-19T08:12:58.680778+0000","last_active":"2024-09-19T08:14:07.737378+0000","last_peered":"2024-09-19T08:14:07.737378+0000","last_clean":"2024-09-19T08:14:07.737378+0000","last_became_active":"2024-09-19T08:12:58.680364+0000","last_became_peered":"2024-09-19T08:12:58.680364+0000","last_unstale":"2024-09-19T08:14:07.737378+0000","last_undegraded":"2024-09-19T08:14:07.737378+0000","last_fullsized":"2024-09-19T08:14:07.737378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:10.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734418+0000","last_change":"2024-09-19T08:13:00.853415+0000","last_active":"2024-09-19T08:14:07.734418+0000","last_peered":"2024-09-19T08:14:07.734418+0000","last_clean":"2024-09-19T08:14:07.734418+0000","last_became_active":"2024-09-19T08:13:00.853101+0000","last_became_peered":"2024-09-19T08:13:00.853101+0000","last_unstale":"2024-09-19T08:14:07.734418+0000","last_undegraded":"2024-09-19T08:14:07.734418+0000","last_fullsized":"2024-09-19T08:14:07.734418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:42.375394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":92,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:12.966309+0000","last_change":"2024-09-19T08:12:58.680312+0000","last_active":"2024-09-19T08:14:12.966309+0000","last_peered":"2024-09-19T08:14:12.966309+0000","last_clean":"2024-09-19T08:14:12.966309+0000","last_became_active":"2024-09-19T08:12:58.679870+0000","last_became_peered":"2024-09-19T08:12:58.679870+0000","last_unstale":"2024-09-19T08:14:12.966309+0000","last_undegraded":"2024-09-19T08:14:12.966309+0000","last_fullsized":"2024-09-19T08:14:12.966309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:42.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734408+0000","last_change":"2024-09-19T08:13:00.853920+0000","last_active":"2024-09-19T08:14:07.734408+0000","last_peered":"2024-09-19T08:14:07.734408+0000","last_clean":"2024-09-19T08:14:07.734408+0000","last_became_active":"2024-09-19T08:13:00.853594+0000","last_became_peered":"2024-09-19T08:13:00.853594+0000","last_unstale":"2024-09-19T08:14:07.734408+0000","last_undegraded":"2024-09-19T08:14:07.734408+0000","last_fullsized":"2024-09-19T08:14:07.734408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:31.746133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.752282+0000","last_change":"2024-09-19T08:14:07.751888+0000","last_active":"2024-09-19T08:14:07.752282+0000","last_peered":"2024-09-19T08:14:07.752282+0000","last_clean":"2024-09-19T08:14:07.752282+0000","last_became_active":"2024-09-19T08:14:07.737201+0000","last_became_peered":"2024-09-19T08:14:07.737201+0000","last_unstale":"2024-09-19T08:14:07.752282+0000","last_undegraded":"2024-09-19T08:14:07.752282+0000","last_fullsized":"2024-09-19T08:14:07.752282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:44.095987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.737002+0000","last_change":"2024-09-19T08:13:00.847061+0000","last_active":"2024-09-19T08:14:07.737002+0000","last_peered":"2024-09-19T08:14:07.737002+0000","last_clean":"2024-09-19T08:14:07.737002+0000","last_became_active":"2024-09-19T08:13:00.846742+0000","last_became_peered":"2024-09-19T08:13:00.846742+0000","last_unstale":"2024-09-19T08:14:07.737002+0000","last_undegraded":"2024-09-19T08:14:07.737002+0000","last_fullsized":"2024-09-19T08:14:07.737002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:21.785585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.755652+0000","last_change":"2024-09-19T08:14:07.755277+0000","last_active":"2024-09-19T08:14:07.755652+0000","last_peered":"2024-09-19T08:14:07.755652+0000","last_clean":"2024-09-19T08:14:07.755652+0000","last_became_active":"2024-09-19T08:14:07.739773+0000","last_became_peered":"2024-09-19T08:14:07.739773+0000","last_unstale":"2024-09-19T08:14:07.755652+0000","last_undegraded":"2024-09-19T08:14:07.755652+0000","last_fullsized":"2024-09-19T08:14:07.755652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:56.848251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734131+0000","last_change":"2024-09-19T08:13:00.844940+0000","last_active":"2024-09-19T08:14:07.734131+0000","last_peered":"2024-09-19T08:14:07.734131+0000","last_clean":"2024-09-19T08:14:07.734131+0000","last_became_active":"2024-09-19T08:13:00.844454+0000","last_became_peered":"2024-09-19T08:13:00.844454+0000","last_unstale":"2024-09-19T08:14:07.734131+0000","last_undegraded":"2024-09-19T08:14:07.734131+0000","last_fullsized":"2024-09-19T08:14:07.734131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:15.577624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731644+0000","last_change":"2024-09-19T08:12:58.673301+0000","last_active":"2024-09-19T08:14:07.731644+0000","last_peered":"2024-09-19T08:14:07.731644+0000","last_clean":"2024-09-19T08:14:07.731644+0000","last_became_active":"2024-09-19T08:12:58.672981+0000","last_became_peered":"2024-09-19T08:12:58.672981+0000","last_unstale":"2024-09-19T08:14:07.731644+0000","last_undegraded":"2024-09-19T08:14:07.731644+0000","last_fullsized":"2024-09-19T08:14:07.731644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:03.963712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735037+0000","last_change":"2024-09-19T08:13:00.856620+0000","last_active":"2024-09-19T08:14:07.735037+0000","last_peered":"2024-09-19T08:14:07.735037+0000","last_clean":"2024-09-19T08:14:07.735037+0000","last_became_active":"2024-09-19T08:13:00.856310+0000","last_became_peered":"2024-09-19T08:13:00.856310+0000","last_unstale":"2024-09-19T08:14:07.735037+0000","last_undegraded":"2024-09-19T08:14:07.735037+0000","last_fullsized":"2024-09-19T08:14:07.735037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:16.918768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.737741+0000","last_change":"2024-09-19T08:12:58.672282+0000","last_active":"2024-09-19T08:14:07.737741+0000","last_peered":"2024-09-19T08:14:07.737741+0000","last_clean":"2024-09-19T08:14:07.737741+0000","last_became_active":"2024-09-19T08:12:58.671891+0000","last_became_peered":"2024-09-19T08:12:58.671891+0000","last_unstale":"2024-09-19T08:14:07.737741+0000","last_undegraded":"2024-09-19T08:14:07.737741+0000","last_fullsized":"2024-09-19T08:14:07.737741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:41.731571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734550+0000","last_change":"2024-09-19T08:13:00.848107+0000","last_active":"2024-09-19T08:14:07.734550+0000","last_peered":"2024-09-19T08:14:07.734550+0000","last_clean":"2024-09-19T08:14:07.734550+0000","last_became_active":"2024-09-19T08:13:00.847632+0000","last_became_peered":"2024-09-19T08:13:00.847632+0000","last_unstale":"2024-09-19T08:14:07.734550+0000","last_undegraded":"2024-09-19T08:14:07.734550+0000","last_fullsized":"2024-09-19T08:14:07.734550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.491122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732531+0000","last_change":"2024-09-19T08:12:59.179200+0000","last_active":"2024-09-19T08:14:07.732531+0000","last_peered":"2024-09-19T08:14:07.732531+0000","last_clean":"2024-09-19T08:14:07.732531+0000","last_became_active":"2024-09-19T08:12:59.178762+0000","last_became_peered":"2024-09-19T08:12:59.178762+0000","last_unstale":"2024-09-19T08:14:07.732531+0000","last_undegraded":"2024-09-19T08:14:07.732531+0000","last_fullsized":"2024-09-19T08:14:07.732531+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:08.424106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734839+0000","last_change":"2024-09-19T08:13:00.855870+0000","last_active":"2024-09-19T08:14:07.734839+0000","last_peered":"2024-09-19T08:14:07.734839+0000","last_clean":"2024-09-19T08:14:07.734839+0000","last_became_active":"2024-09-19T08:13:00.855538+0000","last_became_peered":"2024-09-19T08:13:00.855538+0000","last_unstale":"2024-09-19T08:14:07.734839+0000","last_undegraded":"2024-09-19T08:14:07.734839+0000","last_fullsized":"2024-09-19T08:14:07.734839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734771+0000","last_change":"2024-09-19T08:11:48.661333+0000","last_active":"2024-09-19T08:14:07.734771+0000","last_peered":"2024-09-19T08:14:07.734771+0000","last_clean":"2024-09-19T08:14:07.734771+0000","last_became_active":"2024-09-19T08:11:48.660873+0000","last_became_peered":"2024-09-19T08:11:48.660873+0000","last_unstale":"2024-09-19T08:14:07.734771+0000","last_undegraded":"2024-09-19T08:14:07.734771+0000","last_fullsized":"2024-09-19T08:14:07.734771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_clean_scrub_stamp":"2024-09-19T08:08:25.372155+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:19.401292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738561+0000","last_change":"2024-09-19T08:12:58.675237+0000","last_active":"2024-09-19T08:14:07.738561+0000","last_peered":"2024-09-19T08:14:07.738561+0000","last_clean":"2024-09-19T08:14:07.738561+0000","last_became_active":"2024-09-19T08:12:58.674910+0000","last_became_peered":"2024-09-19T08:12:58.674910+0000","last_unstale":"2024-09-19T08:14:07.738561+0000","last_undegraded":"2024-09-19T08:14:07.738561+0000","last_fullsized":"2024-09-19T08:14:07.738561+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:47.038328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399776+0000","last_change":"2024-09-19T08:13:00.846754+0000","last_active":"2024-09-19T08:14:08.399776+0000","last_peered":"2024-09-19T08:14:08.399776+0000","last_clean":"2024-09-19T08:14:08.399776+0000","last_became_active":"2024-09-19T08:13:00.846452+0000","last_became_peered":"2024-09-19T08:13:00.846452+0000","last_unstale":"2024-09-19T08:14:08.399776+0000","last_undegraded":"2024-09-19T08:14:08.399776+0000","last_fullsized":"2024-09-19T08:14:08.399776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:22.929689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734639+0000","last_change":"2024-09-19T08:12:58.678958+0000","last_active":"2024-09-19T08:14:07.734639+0000","last_peered":"2024-09-19T08:14:07.734639+0000","last_clean":"2024-09-19T08:14:07.734639+0000","last_became_active":"2024-09-19T08:12:58.678592+0000","last_became_peered":"2024-09-19T08:12:58.678592+0000","last_unstale":"2024-09-19T08:14:07.734639+0000","last_undegraded":"2024-09-19T08:14:07.734639+0000","last_fullsized":"2024-09-19T08:14:07.734639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:58.374606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731205+0000","last_change":"2024-09-19T08:13:00.840973+0000","last_active":"2024-09-19T08:14:07.731205+0000","last_peered":"2024-09-19T08:14:07.731205+0000","last_clean":"2024-09-19T08:14:07.731205+0000","last_became_active":"2024-09-19T08:13:00.840619+0000","last_became_peered":"2024-09-19T08:13:00.840619+0000","last_unstale":"2024-09-19T08:14:07.731205+0000","last_undegraded":"2024-09-19T08:14:07.731205+0000","last_fullsized":"2024-09-19T08:14:07.731205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":90,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:13.967262+0000","last_change":"2024-09-19T08:12:58.715972+0000","last_active":"2024-09-19T08:14:13.967262+0000","last_peered":"2024-09-19T08:14:13.967262+0000","last_clean":"2024-09-19T08:14:13.967262+0000","last_became_active":"2024-09-19T08:12:58.715630+0000","last_became_peered":"2024-09-19T08:12:58.715630+0000","last_unstale":"2024-09-19T08:14:13.967262+0000","last_undegraded":"2024-09-19T08:14:13.967262+0000","last_fullsized":"2024-09-19T08:14:13.967262+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:35.637121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736616+0000","last_change":"2024-09-19T08:13:00.847023+0000","last_active":"2024-09-19T08:14:07.736616+0000","last_peered":"2024-09-19T08:14:07.736616+0000","last_clean":"2024-09-19T08:14:07.736616+0000","last_became_active":"2024-09-19T08:13:00.846387+0000","last_became_peered":"2024-09-19T08:13:00.846387+0000","last_unstale":"2024-09-19T08:14:07.736616+0000","last_undegraded":"2024-09-19T08:14:07.736616+0000","last_fullsized":"2024-09-19T08:14:07.736616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:12.187929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735037+0000","last_change":"2024-09-19T08:12:58.681062+0000","last_active":"2024-09-19T08:14:07.735037+0000","last_peered":"2024-09-19T08:14:07.735037+0000","last_clean":"2024-09-19T08:14:07.735037+0000","last_became_active":"2024-09-19T08:12:58.680524+0000","last_became_peered":"2024-09-19T08:12:58.680524+0000","last_unstale":"2024-09-19T08:14:07.735037+0000","last_undegraded":"2024-09-19T08:14:07.735037+0000","last_fullsized":"2024-09-19T08:14:07.735037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735930+0000","last_change":"2024-09-19T08:13:00.851523+0000","last_active":"2024-09-19T08:14:07.735930+0000","last_peered":"2024-09-19T08:14:07.735930+0000","last_clean":"2024-09-19T08:14:07.735930+0000","last_became_active":"2024-09-19T08:13:00.851068+0000","last_became_peered":"2024-09-19T08:13:00.851068+0000","last_unstale":"2024-09-19T08:14:07.735930+0000","last_undegraded":"2024-09-19T08:14:07.735930+0000","last_fullsized":"2024-09-19T08:14:07.735930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:44.905469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735213+0000","last_change":"2024-09-19T08:12:58.718234+0000","last_active":"2024-09-19T08:14:07.735213+0000","last_peered":"2024-09-19T08:14:07.735213+0000","last_clean":"2024-09-19T08:14:07.735213+0000","last_became_active":"2024-09-19T08:12:58.717840+0000","last_became_peered":"2024-09-19T08:12:58.717840+0000","last_unstale":"2024-09-19T08:14:07.735213+0000","last_undegraded":"2024-09-19T08:14:07.735213+0000","last_fullsized":"2024-09-19T08:14:07.735213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:05.283770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.736201+0000","last_change":"2024-09-19T08:13:00.848305+0000","last_active":"2024-09-19T08:14:07.736201+0000","last_peered":"2024-09-19T08:14:07.736201+0000","last_clean":"2024-09-19T08:14:07.736201+0000","last_became_active":"2024-09-19T08:13:00.847981+0000","last_became_peered":"2024-09-19T08:13:00.847981+0000","last_unstale":"2024-09-19T08:14:07.736201+0000","last_undegraded":"2024-09-19T08:14:07.736201+0000","last_fullsized":"2024-09-19T08:14:07.736201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:04.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.739029+0000","last_change":"2024-09-19T08:12:58.715943+0000","last_active":"2024-09-19T08:14:07.739029+0000","last_peered":"2024-09-19T08:14:07.739029+0000","last_clean":"2024-09-19T08:14:07.739029+0000","last_became_active":"2024-09-19T08:12:58.715600+0000","last_became_peered":"2024-09-19T08:12:58.715600+0000","last_unstale":"2024-09-19T08:14:07.739029+0000","last_undegraded":"2024-09-19T08:14:07.739029+0000","last_fullsized":"2024-09-19T08:14:07.739029+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:19.142271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.731759+0000","last_change":"2024-09-19T08:13:00.840944+0000","last_active":"2024-09-19T08:14:07.731759+0000","last_peered":"2024-09-19T08:14:07.731759+0000","last_clean":"2024-09-19T08:14:07.731759+0000","last_became_active":"2024-09-19T08:13:00.840608+0000","last_became_peered":"2024-09-19T08:13:00.840608+0000","last_unstale":"2024-09-19T08:14:07.731759+0000","last_undegraded":"2024-09-19T08:14:07.731759+0000","last_fullsized":"2024-09-19T08:14:07.731759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:25.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'11","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735315+0000","last_change":"2024-09-19T08:12:58.672929+0000","last_active":"2024-09-19T08:14:07.735315+0000","last_peered":"2024-09-19T08:14:07.735315+0000","last_clean":"2024-09-19T08:14:07.735315+0000","last_became_active":"2024-09-19T08:12:58.672487+0000","last_became_peered":"2024-09-19T08:12:58.672487+0000","last_unstale":"2024-09-19T08:14:07.735315+0000","last_undegraded":"2024-09-19T08:14:07.735315+0000","last_fullsized":"2024-09-19T08:14:07.735315+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:33.042735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738260+0000","last_change":"2024-09-19T08:14:07.738260+0000","last_active":"2024-09-19T08:14:07.738260+0000","last_peered":"2024-09-19T08:14:07.738260+0000","last_clean":"2024-09-19T08:14:07.738260+0000","last_became_active":"2024-09-19T08:14:07.737915+0000","last_became_peered":"2024-09-19T08:14:07.737915+0000","last_unstale":"2024-09-19T08:14:07.738260+0000","last_undegraded":"2024-09-19T08:14:07.738260+0000","last_fullsized":"2024-09-19T08:14:07.738260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:02.500657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.739005+0000","last_change":"2024-09-19T08:12:59.179510+0000","last_active":"2024-09-19T08:14:07.739005+0000","last_peered":"2024-09-19T08:14:07.739005+0000","last_clean":"2024-09-19T08:14:07.739005+0000","last_became_active":"2024-09-19T08:12:59.179157+0000","last_became_peered":"2024-09-19T08:12:59.179157+0000","last_unstale":"2024-09-19T08:14:07.739005+0000","last_undegraded":"2024-09-19T08:14:07.739005+0000","last_fullsized":"2024-09-19T08:14:07.739005+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:22.981289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733930+0000","last_change":"2024-09-19T08:13:00.841853+0000","last_active":"2024-09-19T08:14:07.733930+0000","last_peered":"2024-09-19T08:14:07.733930+0000","last_clean":"2024-09-19T08:14:07.733930+0000","last_became_active":"2024-09-19T08:13:00.841518+0000","last_became_peered":"2024-09-19T08:13:00.841518+0000","last_unstale":"2024-09-19T08:14:07.733930+0000","last_undegraded":"2024-09-19T08:14:07.733930+0000","last_fullsized":"2024-09-19T08:14:07.733930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:05.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733280+0000","last_change":"2024-09-19T08:12:58.716882+0000","last_active":"2024-09-19T08:14:07.733280+0000","last_peered":"2024-09-19T08:14:07.733280+0000","last_clean":"2024-09-19T08:14:07.733280+0000","last_became_active":"2024-09-19T08:12:58.716545+0000","last_became_peered":"2024-09-19T08:12:58.716545+0000","last_unstale":"2024-09-19T08:14:07.733280+0000","last_undegraded":"2024-09-19T08:14:07.733280+0000","last_fullsized":"2024-09-19T08:14:07.733280+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:14.469109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.739825+0000","last_change":"2024-09-19T08:13:00.850484+0000","last_active":"2024-09-19T08:14:07.739825+0000","last_peered":"2024-09-19T08:14:07.739825+0000","last_clean":"2024-09-19T08:14:07.739825+0000","last_became_active":"2024-09-19T08:13:00.850049+0000","last_became_peered":"2024-09-19T08:13:00.850049+0000","last_unstale":"2024-09-19T08:14:07.739825+0000","last_undegraded":"2024-09-19T08:14:07.739825+0000","last_fullsized":"2024-09-19T08:14:07.739825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:24.424111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399703+0000","last_change":"2024-09-19T08:12:58.669928+0000","last_active":"2024-09-19T08:14:08.399703+0000","last_peered":"2024-09-19T08:14:08.399703+0000","last_clean":"2024-09-19T08:14:08.399703+0000","last_became_active":"2024-09-19T08:12:58.669614+0000","last_became_peered":"2024-09-19T08:12:58.669614+0000","last_unstale":"2024-09-19T08:14:08.399703+0000","last_undegraded":"2024-09-19T08:14:08.399703+0000","last_fullsized":"2024-09-19T08:14:08.399703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:50.924779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734603+0000","last_change":"2024-09-19T08:13:00.853520+0000","last_active":"2024-09-19T08:14:07.734603+0000","last_peered":"2024-09-19T08:14:07.734603+0000","last_clean":"2024-09-19T08:14:07.734603+0000","last_became_active":"2024-09-19T08:13:00.853184+0000","last_became_peered":"2024-09-19T08:13:00.853184+0000","last_unstale":"2024-09-19T08:14:07.734603+0000","last_undegraded":"2024-09-19T08:14:07.734603+0000","last_fullsized":"2024-09-19T08:14:07.734603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:45.160439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732133+0000","last_change":"2024-09-19T08:12:58.674359+0000","last_active":"2024-09-19T08:14:07.732133+0000","last_peered":"2024-09-19T08:14:07.732133+0000","last_clean":"2024-09-19T08:14:07.732133+0000","last_became_active":"2024-09-19T08:12:58.673890+0000","last_became_peered":"2024-09-19T08:12:58.673890+0000","last_unstale":"2024-09-19T08:14:07.732133+0000","last_undegraded":"2024-09-19T08:14:07.732133+0000","last_fullsized":"2024-09-19T08:14:07.732133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:54.939959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.399978+0000","last_change":"2024-09-19T08:13:00.851071+0000","last_active":"2024-09-19T08:14:08.399978+0000","last_peered":"2024-09-19T08:14:08.399978+0000","last_clean":"2024-09-19T08:14:08.399978+0000","last_became_active":"2024-09-19T08:13:00.850756+0000","last_became_peered":"2024-09-19T08:13:00.850756+0000","last_unstale":"2024-09-19T08:14:08.399978+0000","last_undegraded":"2024-09-19T08:14:08.399978+0000","last_fullsized":"2024-09-19T08:14:08.399978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:52.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738813+0000","last_change":"2024-09-19T08:12:58.681466+0000","last_active":"2024-09-19T08:14:07.738813+0000","last_peered":"2024-09-19T08:14:07.738813+0000","last_clean":"2024-09-19T08:14:07.738813+0000","last_became_active":"2024-09-19T08:12:58.681113+0000","last_became_peered":"2024-09-19T08:12:58.681113+0000","last_unstale":"2024-09-19T08:14:07.738813+0000","last_undegraded":"2024-09-19T08:14:07.738813+0000","last_fullsized":"2024-09-19T08:14:07.738813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:49.610828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.734635+0000","last_change":"2024-09-19T08:13:00.851272+0000","last_active":"2024-09-19T08:14:07.734635+0000","last_peered":"2024-09-19T08:14:07.734635+0000","last_clean":"2024-09-19T08:14:07.734635+0000","last_became_active":"2024-09-19T08:13:00.849098+0000","last_became_peered":"2024-09-19T08:13:00.849098+0000","last_unstale":"2024-09-19T08:14:07.734635+0000","last_undegraded":"2024-09-19T08:14:07.734635+0000","last_fullsized":"2024-09-19T08:14:07.734635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:05.643992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738913+0000","last_change":"2024-09-19T08:14:07.738913+0000","last_active":"2024-09-19T08:14:07.738913+0000","last_peered":"2024-09-19T08:14:07.738913+0000","last_clean":"2024-09-19T08:14:07.738913+0000","last_became_active":"2024-09-19T08:14:07.738560+0000","last_became_peered":"2024-09-19T08:14:07.738560+0000","last_unstale":"2024-09-19T08:14:07.738913+0000","last_undegraded":"2024-09-19T08:14:07.738913+0000","last_fullsized":"2024-09-19T08:14:07.738913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:35.731290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738679+0000","last_change":"2024-09-19T08:12:58.674610+0000","last_active":"2024-09-19T08:14:07.738679+0000","last_peered":"2024-09-19T08:14:07.738679+0000","last_clean":"2024-09-19T08:14:07.738679+0000","last_became_active":"2024-09-19T08:12:58.674247+0000","last_became_peered":"2024-09-19T08:12:58.674247+0000","last_unstale":"2024-09-19T08:14:07.738679+0000","last_undegraded":"2024-09-19T08:14:07.738679+0000","last_fullsized":"2024-09-19T08:14:07.738679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:16.323574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.732898+0000","last_change":"2024-09-19T08:13:00.858512+0000","last_active":"2024-09-19T08:14:07.732898+0000","last_peered":"2024-09-19T08:14:07.732898+0000","last_clean":"2024-09-19T08:14:07.732898+0000","last_became_active":"2024-09-19T08:13:00.857897+0000","last_became_peered":"2024-09-19T08:13:00.857897+0000","last_unstale":"2024-09-19T08:14:07.732898+0000","last_undegraded":"2024-09-19T08:14:07.732898+0000","last_fullsized":"2024-09-19T08:14:07.732898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.738975+0000","last_change":"2024-09-19T08:12:58.682207+0000","last_active":"2024-09-19T08:14:07.738975+0000","last_peered":"2024-09-19T08:14:07.738975+0000","last_clean":"2024-09-19T08:14:07.738975+0000","last_became_active":"2024-09-19T08:12:58.681842+0000","last_became_peered":"2024-09-19T08:12:58.681842+0000","last_unstale":"2024-09-19T08:14:07.738975+0000","last_undegraded":"2024-09-19T08:14:07.738975+0000","last_fullsized":"2024-09-19T08:14:07.738975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.825858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.733650+0000","last_change":"2024-09-19T08:13:00.846748+0000","last_active":"2024-09-19T08:14:07.733650+0000","last_peered":"2024-09-19T08:14:07.733650+0000","last_clean":"2024-09-19T08:14:07.733650+0000","last_became_active":"2024-09-19T08:13:00.846368+0000","last_became_peered":"2024-09-19T08:13:00.846368+0000","last_unstale":"2024-09-19T08:14:07.733650+0000","last_undegraded":"2024-09-19T08:14:07.733650+0000","last_fullsized":"2024-09-19T08:14:07.733650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:59.526512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.400239+0000","last_change":"2024-09-19T08:12:58.716118+0000","last_active":"2024-09-19T08:14:08.400239+0000","last_peered":"2024-09-19T08:14:08.400239+0000","last_clean":"2024-09-19T08:14:08.400239+0000","last_became_active":"2024-09-19T08:12:58.715711+0000","last_became_peered":"2024-09-19T08:12:58.715711+0000","last_unstale":"2024-09-19T08:14:08.400239+0000","last_undegraded":"2024-09-19T08:14:08.400239+0000","last_fullsized":"2024-09-19T08:14:08.400239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:29.439887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:08.400041+0000","last_change":"2024-09-19T08:13:00.848729+0000","last_active":"2024-09-19T08:14:08.400041+0000","last_peered":"2024-09-19T08:14:08.400041+0000","last_clean":"2024-09-19T08:14:08.400041+0000","last_became_active":"2024-09-19T08:13:00.848470+0000","last_became_peered":"2024-09-19T08:13:00.848470+0000","last_unstale":"2024-09-19T08:14:08.400041+0000","last_undegraded":"2024-09-19T08:14:08.400041+0000","last_fullsized":"2024-09-19T08:14:08.400041+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:30.653101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T08:14:07.735443+0000","last_change":"2024-09-19T08:12:58.681654+0000","last_active":"2024-09-19T08:14:07.735443+0000","last_peered":"2024-09-19T08:14:07.735443+0000","last_clean":"2024-09-19T08:14:07.735443+0000","last_became_active":"2024-09-19T08:12:58.681284+0000","last_became_peered":"2024-09-19T08:12:58.681284+0000","last_unstale":"2024-09-19T08:14:07.735443+0000","last_undegraded":"2024-09-19T08:14:07.735443+0000","last_fullsized":"2024-09-19T08:14:07.735443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.429930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65568,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":400,"num_read_kb":268,"num_write":164,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":61306,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":196704,"data_compressed":9611,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251470,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481820,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":445932,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710753,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1069056,"data_stored":1098844,"data_compressed":42203,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874279,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":933888,"data_stored":446034,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103212,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":921600,"data_stored":445820,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332144,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":983040,"data_stored":1038279,"data_compressed":42203,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495669,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":921600,"data_stored":445423,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":995328,"data_stored":1037570,"data_compressed":42203,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":950272,"data_stored":507425,"data_compressed":15422,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":937984,"data_stored":445734,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":446959,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":983040,"data_stored":506388,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":446694,"data_compressed":5811,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34755,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34755,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":62209,"data_compressed":9611,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:14:20.878 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T08:14:20.878 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T08:14:20.879 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T08:14:20.879 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T08:14:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:21.160 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:20 smithi064 ceph-mon[34900]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:21.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:20 smithi157 ceph-mon[35143]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:21.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:20 smithi022 ceph-mon[30963]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:14:21.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.595+0000 7ff3d0523640 1 -- 172.21.15.22:0/4164149657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3c810e500 msgr2=0x7ff3c8110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:21.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.595+0000 7ff3d0523640 1 --2- 172.21.15.22:0/4164149657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3c810e500 0x7ff3c8110910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009ca0 tx=0x7ff3c002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:14:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.595+0000 7ff3d0523640 1 -- 172.21.15.22:0/4164149657 shutdown_connections 2024-09-19T08:14:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.595+0000 7ff3d0523640 1 --2- 172.21.15.22:0/4164149657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3c810e500 0x7ff3c8110910 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.595+0000 7ff3d0523640 1 --2- 172.21.15.22:0/4164149657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c8101440 0x7ff3c810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.595+0000 7ff3d0523640 1 --2- 172.21.15.22:0/4164149657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3c8100b20 0x7ff3c8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.595+0000 7ff3d0523640 1 -- 172.21.15.22:0/4164149657 >> 172.21.15.22:0/4164149657 conn(0x7ff3c80fc7d0 msgr2=0x7ff3c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.595+0000 7ff3d0523640 1 -- 172.21.15.22:0/4164149657 shutdown_connections 2024-09-19T08:14:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.595+0000 7ff3d0523640 1 -- 172.21.15.22:0/4164149657 wait complete. 2024-09-19T08:14:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3d0523640 1 Processor -- start 2024-09-19T08:14:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3d0523640 1 -- start start 2024-09-19T08:14:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3d0523640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3c8100b20 0x7ff3c819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3d0523640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3c8101440 0x7ff3c819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3d0523640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c810e500 0x7ff3c819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3d0523640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c8114250 con 0x7ff3c8100b20 2024-09-19T08:14:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3d0523640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c81140d0 con 0x7ff3c8101440 2024-09-19T08:14:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3d0523640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c81143d0 con 0x7ff3c810e500 2024-09-19T08:14:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3ce2bf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3c8100b20 0x7ff3c819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3ce2bf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3c8100b20 0x7ff3c819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38986/0 (socket says 172.21.15.22:38986) 2024-09-19T08:14:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3ce2bf640 1 -- 172.21.15.22:0/4153584615 learned_addr learned my addr 172.21.15.22:0/4153584615 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3cdabe640 1 --2- 172.21.15.22:0/4153584615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3c8101440 0x7ff3c819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3ce2bf640 1 -- 172.21.15.22:0/4153584615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c810e500 msgr2=0x7ff3c819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3ce2bf640 1 --2- 172.21.15.22:0/4153584615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c810e500 0x7ff3c819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3ce2bf640 1 -- 172.21.15.22:0/4153584615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3c8101440 msgr2=0x7ff3c819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.596+0000 7ff3ce2bf640 1 --2- 172.21.15.22:0/4153584615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3c8101440 0x7ff3c819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.597+0000 7ff3ce2bf640 1 -- 172.21.15.22:0/4153584615 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c819f6c0 con 0x7ff3c8100b20 2024-09-19T08:14:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.597+0000 7ff3cdabe640 1 --2- 172.21.15.22:0/4153584615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3c8101440 0x7ff3c819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.597+0000 7ff3ce2bf640 1 --2- 172.21.15.22:0/4153584615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3c8100b20 0x7ff3c819a710 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400d950 tx=0x7ff3c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.597+0000 7ff3bf7fe640 1 -- 172.21.15.22:0/4153584615 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4017030 con 0x7ff3c8100b20 2024-09-19T08:14:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.597+0000 7ff3bf7fe640 1 -- 172.21.15.22:0/4153584615 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c4004020 con 0x7ff3c8100b20 2024-09-19T08:14:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.597+0000 7ff3d0523640 1 -- 172.21.15.22:0/4153584615 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c819f9b0 con 0x7ff3c8100b20 2024-09-19T08:14:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.597+0000 7ff3bf7fe640 1 -- 172.21.15.22:0/4153584615 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c40047a0 con 0x7ff3c8100b20 2024-09-19T08:14:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.597+0000 7ff3d0523640 1 -- 172.21.15.22:0/4153584615 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c81a7290 con 0x7ff3c8100b20 2024-09-19T08:14:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.598+0000 7ff3d0523640 1 -- 172.21.15.22:0/4153584615 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c8100f00 con 0x7ff3c8100b20 2024-09-19T08:14:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.599+0000 7ff3bf7fe640 1 -- 172.21.15.22:0/4153584615 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff3c40041d0 con 0x7ff3c8100b20 2024-09-19T08:14:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.600+0000 7ff3bf7fe640 1 --2- 172.21.15.22:0/4153584615 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff3b40a3350 0x7ff3b40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.600+0000 7ff3cdabe640 1 --2- 172.21.15.22:0/4153584615 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff3b40a3350 0x7ff3b40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.600+0000 7ff3bf7fe640 1 -- 172.21.15.22:0/4153584615 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff3c40cf4d0 con 0x7ff3c8100b20 2024-09-19T08:14:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.600+0000 7ff3cdabe640 1 --2- 172.21.15.22:0/4153584615 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff3b40a3350 0x7ff3b40a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005d90 tx=0x7ff3b8005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.604+0000 7ff3bf7fe640 1 -- 172.21.15.22:0/4153584615 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff3c409c090 con 0x7ff3c8100b20 2024-09-19T08:14:21.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.818+0000 7ff3d0523640 1 -- 172.21.15.22:0/4153584615 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff3c819ba70 con 0x7ff3c8100b20 2024-09-19T08:14:21.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.818+0000 7ff3bf7fe640 1 -- 172.21.15.22:0/4153584615 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff3c409b990 con 0x7ff3c8100b20 2024-09-19T08:14:21.818 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:21.819 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T08:14:21.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.820+0000 7ff3d0523640 1 -- 172.21.15.22:0/4153584615 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff3b40a3350 msgr2=0x7ff3b40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:21.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.820+0000 7ff3d0523640 1 --2- 172.21.15.22:0/4153584615 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff3b40a3350 0x7ff3b40a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005d90 tx=0x7ff3b8005d20 comp rx=0 tx=0).stop 2024-09-19T08:14:21.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.820+0000 7ff3d0523640 1 -- 172.21.15.22:0/4153584615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3c8100b20 msgr2=0x7ff3c819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:21.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.820+0000 7ff3d0523640 1 --2- 172.21.15.22:0/4153584615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3c8100b20 0x7ff3c819a710 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400d950 tx=0x7ff3c400de10 comp rx=0 tx=0).stop 2024-09-19T08:14:21.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.820+0000 7ff3d0523640 1 -- 172.21.15.22:0/4153584615 shutdown_connections 2024-09-19T08:14:21.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.820+0000 7ff3d0523640 1 --2- 172.21.15.22:0/4153584615 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff3b40a3350 0x7ff3b40a5810 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:21.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.820+0000 7ff3d0523640 1 --2- 172.21.15.22:0/4153584615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c810e500 0x7ff3c819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:21.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.820+0000 7ff3d0523640 1 --2- 172.21.15.22:0/4153584615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3c8101440 0x7ff3c819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:21.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.820+0000 7ff3d0523640 1 --2- 172.21.15.22:0/4153584615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3c8100b20 0x7ff3c819a710 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:21.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.820+0000 7ff3d0523640 1 -- 172.21.15.22:0/4153584615 >> 172.21.15.22:0/4153584615 conn(0x7ff3c80fc7d0 msgr2=0x7ff3c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:21.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.820+0000 7ff3d0523640 1 -- 172.21.15.22:0/4153584615 shutdown_connections 2024-09-19T08:14:21.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:21.820+0000 7ff3d0523640 1 -- 172.21.15.22:0/4153584615 wait complete. 2024-09-19T08:14:22.143 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:21 smithi022 ceph-mon[30963]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 5.0 KiB/s, 1 objects/s recovering 2024-09-19T08:14:22.143 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:21 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/4153584615' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T08:14:22.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:21 smithi157 ceph-mon[35143]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 5.0 KiB/s, 1 objects/s recovering 2024-09-19T08:14:22.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:21 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/4153584615' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T08:14:22.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:21 smithi064 ceph-mon[34900]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 5.0 KiB/s, 1 objects/s recovering 2024-09-19T08:14:22.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:21 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/4153584615' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T08:14:22.450 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T08:14:22.450 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:14:22.711 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:23.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.196+0000 7f359ffc2640 1 -- 172.21.15.22:0/3738332292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3598074590 msgr2=0x7f3598074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:23.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.196+0000 7f359ffc2640 1 --2- 172.21.15.22:0/3738332292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3598074590 0x7f3598074970 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3594009900 tx=0x7f359402a480 comp rx=0 tx=0).stop 2024-09-19T08:14:23.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.196+0000 7f359ffc2640 1 -- 172.21.15.22:0/3738332292 shutdown_connections 2024-09-19T08:14:23.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.196+0000 7f359ffc2640 1 --2- 172.21.15.22:0/3738332292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3598112b80 0x7f3598114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:23.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.196+0000 7f359ffc2640 1 --2- 172.21.15.22:0/3738332292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598074eb0 0x7f3598112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:23.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.196+0000 7f359ffc2640 1 --2- 172.21.15.22:0/3738332292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3598074590 0x7f3598074970 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:23.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.196+0000 7f359ffc2640 1 -- 172.21.15.22:0/3738332292 >> 172.21.15.22:0/3738332292 conn(0x7f35980fd8b0 msgr2=0x7f35980ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:23.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.196+0000 7f359ffc2640 1 -- 172.21.15.22:0/3738332292 shutdown_connections 2024-09-19T08:14:23.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.196+0000 7f359ffc2640 1 -- 172.21.15.22:0/3738332292 wait complete. 2024-09-19T08:14:23.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.196+0000 7f359ffc2640 1 Processor -- start 2024-09-19T08:14:23.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.196+0000 7f359ffc2640 1 -- start start 2024-09-19T08:14:23.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359ffc2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3598074590 0x7f35981a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:23.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359ffc2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3598074eb0 0x7f35981a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:23.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359ffc2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598112b80 0x7f359819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:23.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359ffc2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3598118540 con 0x7f3598074590 2024-09-19T08:14:23.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359ffc2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f35981183c0 con 0x7f3598074eb0 2024-09-19T08:14:23.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359ffc2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35981186c0 con 0x7f3598112b80 2024-09-19T08:14:23.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359dd5e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3598074590 0x7f35981a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:23.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359dd5e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3598074590 0x7f35981a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39006/0 (socket says 172.21.15.22:39006) 2024-09-19T08:14:23.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359dd5e640 1 -- 172.21.15.22:0/17549355 learned_addr learned my addr 172.21.15.22:0/17549355 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:23.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359d55d640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3598074eb0 0x7f35981a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:23.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359e55f640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598112b80 0x7f359819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:23.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359dd5e640 1 -- 172.21.15.22:0/17549355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598112b80 msgr2=0x7f359819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:23.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359dd5e640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598112b80 0x7f359819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:23.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359dd5e640 1 -- 172.21.15.22:0/17549355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3598074eb0 msgr2=0x7f35981a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:23.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359dd5e640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3598074eb0 0x7f35981a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:23.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359dd5e640 1 -- 172.21.15.22:0/17549355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359819ef90 con 0x7f3598074590 2024-09-19T08:14:23.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359d55d640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3598074eb0 0x7f35981a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:23.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359e55f640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598112b80 0x7f359819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:14:23.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359dd5e640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3598074590 0x7f35981a46a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3594006010 tx=0x7f35940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:23.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f358effd640 1 -- 172.21.15.22:0/17549355 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359403f030 con 0x7f3598074590 2024-09-19T08:14:23.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f358effd640 1 -- 172.21.15.22:0/17549355 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3594003ce0 con 0x7f3598074590 2024-09-19T08:14:23.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.197+0000 7f359ffc2640 1 -- 172.21.15.22:0/17549355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359819f220 con 0x7f3598074590 2024-09-19T08:14:23.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.198+0000 7f358effd640 1 -- 172.21.15.22:0/17549355 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594035070 con 0x7f3598074590 2024-09-19T08:14:23.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.198+0000 7f359ffc2640 1 -- 172.21.15.22:0/17549355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35981ab570 con 0x7f3598074590 2024-09-19T08:14:23.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.198+0000 7f359ffc2640 1 -- 172.21.15.22:0/17549355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3598074970 con 0x7f3598074590 2024-09-19T08:14:23.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.200+0000 7f358effd640 1 -- 172.21.15.22:0/17549355 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3594042030 con 0x7f3598074590 2024-09-19T08:14:23.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.200+0000 7f358effd640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f35840a32d0 0x7f35840a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:23.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.201+0000 7f359d55d640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f35840a32d0 0x7f35840a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:23.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.201+0000 7f358effd640 1 -- 172.21.15.22:0/17549355 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f35940f04b0 con 0x7f3598074590 2024-09-19T08:14:23.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.201+0000 7f359d55d640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f35840a32d0 0x7f35840a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f35981000d0 tx=0x7f358800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:23.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.204+0000 7f358effd640 1 -- 172.21.15.22:0/17549355 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f35940bd130 con 0x7f3598074590 2024-09-19T08:14:23.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.419+0000 7f359ffc2640 1 -- 172.21.15.22:0/17549355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3598104fc0 con 0x7f3598074590 2024-09-19T08:14:23.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.420+0000 7f358effd640 1 -- 172.21.15.22:0/17549355 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11906 (secure 0 0 0) 0x7f35940bca30 con 0x7f3598074590 2024-09-19T08:14:23.420 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:23.421 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"btime":"2024-09-19T08:13:13:877693+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:13:13.764216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:14:23.421 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-09-19T08:14:23.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.422+0000 7f359ffc2640 1 -- 172.21.15.22:0/17549355 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f35840a32d0 msgr2=0x7f35840a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:23.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.422+0000 7f359ffc2640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f35840a32d0 0x7f35840a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f35981000d0 tx=0x7f358800a5c0 comp rx=0 tx=0).stop 2024-09-19T08:14:23.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.423+0000 7f359ffc2640 1 -- 172.21.15.22:0/17549355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3598074590 msgr2=0x7f35981a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:23.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.423+0000 7f359ffc2640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3598074590 0x7f35981a46a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3594006010 tx=0x7f35940042f0 comp rx=0 tx=0).stop 2024-09-19T08:14:23.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.423+0000 7f359ffc2640 1 -- 172.21.15.22:0/17549355 shutdown_connections 2024-09-19T08:14:23.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.423+0000 7f359ffc2640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f35840a32d0 0x7f35840a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:23.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.423+0000 7f359ffc2640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3598112b80 0x7f359819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:23.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.423+0000 7f359ffc2640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3598074eb0 0x7f35981a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:23.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.423+0000 7f359ffc2640 1 --2- 172.21.15.22:0/17549355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3598074590 0x7f35981a46a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:23.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.423+0000 7f359ffc2640 1 -- 172.21.15.22:0/17549355 >> 172.21.15.22:0/17549355 conn(0x7f35980fd8b0 msgr2=0x7f35980ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:23.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.423+0000 7f359ffc2640 1 -- 172.21.15.22:0/17549355 shutdown_connections 2024-09-19T08:14:23.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:23.423+0000 7f359ffc2640 1 -- 172.21.15.22:0/17549355 wait complete. 2024-09-19T08:14:24.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:14:24.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:23 smithi022 ceph-mon[30963]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 11 B/s, 0 objects/s recovering 2024-09-19T08:14:24.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:23 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/17549355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:24.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:23 smithi157 ceph-mon[35143]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 11 B/s, 0 objects/s recovering 2024-09-19T08:14:24.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:23 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/17549355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:24.334 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:24.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:23 smithi064 ceph-mon[34900]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 11 B/s, 0 objects/s recovering 2024-09-19T08:14:24.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:23 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/17549355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.839+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1428654340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb828106f10 msgr2=0x7fb82810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.839+0000 7fb82fd8a640 1 --2- 172.21.15.22:0/1428654340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb828106f10 0x7fb82810ea90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb820009ca0 tx=0x7fb82002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:14:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.839+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1428654340 shutdown_connections 2024-09-19T08:14:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.839+0000 7fb82fd8a640 1 --2- 172.21.15.22:0/1428654340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb828106f10 0x7fb82810ea90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.839+0000 7fb82fd8a640 1 --2- 172.21.15.22:0/1428654340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb828102790 0x7fb8281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.839+0000 7fb82fd8a640 1 --2- 172.21.15.22:0/1428654340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb828101de0 0x7fb8281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.839+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1428654340 >> 172.21.15.22:0/1428654340 conn(0x7fb8280fd8c0 msgr2=0x7fb8280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.839+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1428654340 shutdown_connections 2024-09-19T08:14:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.839+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1428654340 wait complete. 2024-09-19T08:14:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.840+0000 7fb82fd8a640 1 Processor -- start 2024-09-19T08:14:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.840+0000 7fb82fd8a640 1 -- start start 2024-09-19T08:14:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.840+0000 7fb82fd8a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb828101de0 0x7fb8280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.840+0000 7fb82fd8a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb828102790 0x7fb828074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.840+0000 7fb82fd8a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb828106f10 0x7fb828074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.840+0000 7fb82fd8a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb828111f90 con 0x7fb828102790 2024-09-19T08:14:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.840+0000 7fb82fd8a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb828111e10 con 0x7fb828106f10 2024-09-19T08:14:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.840+0000 7fb82fd8a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb828112110 con 0x7fb828101de0 2024-09-19T08:14:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.840+0000 7fb82d325640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb828102790 0x7fb828074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.840+0000 7fb82d325640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb828102790 0x7fb828074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39030/0 (socket says 172.21.15.22:39030) 2024-09-19T08:14:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.840+0000 7fb82d325640 1 -- 172.21.15.22:0/1319020076 learned_addr learned my addr 172.21.15.22:0/1319020076 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.841+0000 7fb82e327640 1 --2- 172.21.15.22:0/1319020076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb828106f10 0x7fb828074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.841+0000 7fb82db26640 1 --2- 172.21.15.22:0/1319020076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb828101de0 0x7fb8280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.841+0000 7fb82d325640 1 -- 172.21.15.22:0/1319020076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb828101de0 msgr2=0x7fb8280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.841+0000 7fb82d325640 1 --2- 172.21.15.22:0/1319020076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb828101de0 0x7fb8280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.841+0000 7fb82d325640 1 -- 172.21.15.22:0/1319020076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb828106f10 msgr2=0x7fb828074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.841+0000 7fb82d325640 1 --2- 172.21.15.22:0/1319020076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb828106f10 0x7fb828074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.841+0000 7fb82d325640 1 -- 172.21.15.22:0/1319020076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8281a68d0 con 0x7fb828102790 2024-09-19T08:14:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.841+0000 7fb82d325640 1 --2- 172.21.15.22:0/1319020076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb828102790 0x7fb828074a30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb81800b4a0 tx=0x7fb81800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.841+0000 7fb81effd640 1 -- 172.21.15.22:0/1319020076 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818015020 con 0x7fb828102790 2024-09-19T08:14:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.841+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1319020076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8281a6bc0 con 0x7fb828102790 2024-09-19T08:14:24.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.841+0000 7fb81effd640 1 -- 172.21.15.22:0/1319020076 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb818011d10 con 0x7fb828102790 2024-09-19T08:14:24.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.841+0000 7fb81effd640 1 -- 172.21.15.22:0/1319020076 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81800f2e0 con 0x7fb828102790 2024-09-19T08:14:24.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.841+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1319020076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8281a70b0 con 0x7fb828102790 2024-09-19T08:14:24.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.842+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1319020076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8281021c0 con 0x7fb828102790 2024-09-19T08:14:24.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.843+0000 7fb81effd640 1 -- 172.21.15.22:0/1319020076 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb818018020 con 0x7fb828102790 2024-09-19T08:14:24.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.844+0000 7fb81effd640 1 --2- 172.21.15.22:0/1319020076 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb8140a32d0 0x7fb8140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:24.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.844+0000 7fb81effd640 1 -- 172.21.15.22:0/1319020076 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb8180cf230 con 0x7fb828102790 2024-09-19T08:14:24.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.844+0000 7fb82db26640 1 --2- 172.21.15.22:0/1319020076 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb8140a32d0 0x7fb8140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:24.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.845+0000 7fb82db26640 1 --2- 172.21.15.22:0/1319020076 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb8140a32d0 0x7fb8140a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb82400abe0 tx=0x7fb82400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:24.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:24.848+0000 7fb81effd640 1 -- 172.21.15.22:0/1319020076 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb81801f020 con 0x7fb828102790 2024-09-19T08:14:25.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:25.060+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1319020076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8281a72e0 con 0x7fb828102790 2024-09-19T08:14:25.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:25.061+0000 7fb81effd640 1 -- 172.21.15.22:0/1319020076 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11906 (secure 0 0 0) 0x7fb81809bf00 con 0x7fb828102790 2024-09-19T08:14:25.061 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:25.061 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"btime":"2024-09-19T08:13:13:877693+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:13:13.764216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:14:25.061 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-09-19T08:14:25.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:25.063+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1319020076 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb8140a32d0 msgr2=0x7fb8140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:25.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:25.063+0000 7fb82fd8a640 1 --2- 172.21.15.22:0/1319020076 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb8140a32d0 0x7fb8140a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb82400abe0 tx=0x7fb82400a2d0 comp rx=0 tx=0).stop 2024-09-19T08:14:25.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:25.063+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1319020076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb828102790 msgr2=0x7fb828074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:25.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:25.063+0000 7fb82fd8a640 1 --2- 172.21.15.22:0/1319020076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb828102790 0x7fb828074a30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb81800b4a0 tx=0x7fb81800b8a0 comp rx=0 tx=0).stop 2024-09-19T08:14:25.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:25.063+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1319020076 shutdown_connections 2024-09-19T08:14:25.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:25.063+0000 7fb82fd8a640 1 --2- 172.21.15.22:0/1319020076 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fb8140a32d0 0x7fb8140a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:25.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:25.063+0000 7fb82fd8a640 1 --2- 172.21.15.22:0/1319020076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb828106f10 0x7fb828074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:25.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:25.063+0000 7fb82fd8a640 1 --2- 172.21.15.22:0/1319020076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb828102790 0x7fb828074a30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:25.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:25.063+0000 7fb82fd8a640 1 --2- 172.21.15.22:0/1319020076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb828101de0 0x7fb8280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:25.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:25.063+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1319020076 >> 172.21.15.22:0/1319020076 conn(0x7fb8280fd8c0 msgr2=0x7fb8280ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:25.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:25.063+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1319020076 shutdown_connections 2024-09-19T08:14:25.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:25.063+0000 7fb82fd8a640 1 -- 172.21.15.22:0/1319020076 wait complete. 2024-09-19T08:14:25.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:14:26.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:25 smithi022 ceph-mon[30963]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T08:14:26.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:25 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1319020076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:26.072 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:26.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:25 smithi157 ceph-mon[35143]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T08:14:26.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:25 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1319020076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:26.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:25 smithi064 ceph-mon[34900]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T08:14:26.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:25 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1319020076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:26.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.533+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2771332725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e8074590 msgr2=0x7f58e8074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:26.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.533+0000 7f58ef3a5640 1 --2- 172.21.15.22:0/2771332725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e8074590 0x7f58e8074970 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009900 tx=0x7f58e402a480 comp rx=0 tx=0).stop 2024-09-19T08:14:26.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.533+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2771332725 shutdown_connections 2024-09-19T08:14:26.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.533+0000 7f58ef3a5640 1 --2- 172.21.15.22:0/2771332725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e8112b90 0x7f58e8114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:26.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.533+0000 7f58ef3a5640 1 --2- 172.21.15.22:0/2771332725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e8074eb0 0x7f58e8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:26.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.533+0000 7f58ef3a5640 1 --2- 172.21.15.22:0/2771332725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e8074590 0x7f58e8074970 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.533+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2771332725 >> 172.21.15.22:0/2771332725 conn(0x7f58e80fd8c0 msgr2=0x7f58e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.533+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2771332725 shutdown_connections 2024-09-19T08:14:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.533+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2771332725 wait complete. 2024-09-19T08:14:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ef3a5640 1 Processor -- start 2024-09-19T08:14:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ef3a5640 1 -- start start 2024-09-19T08:14:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ef3a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e8074590 0x7f58e81a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ef3a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e8074eb0 0x7f58e81a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ef3a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e8112b90 0x7f58e819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ef3a5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58e81185e0 con 0x7f58e8112b90 2024-09-19T08:14:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ef3a5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f58e8118460 con 0x7f58e8074590 2024-09-19T08:14:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ef3a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f58e8118760 con 0x7f58e8074eb0 2024-09-19T08:14:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ed942640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e8112b90 0x7f58e819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ed942640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e8112b90 0x7f58e819e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39046/0 (socket says 172.21.15.22:39046) 2024-09-19T08:14:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ed942640 1 -- 172.21.15.22:0/2302217947 learned_addr learned my addr 172.21.15.22:0/2302217947 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ed141640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e8074590 0x7f58e81a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ed942640 1 -- 172.21.15.22:0/2302217947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e8074eb0 msgr2=0x7f58e81a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ec940640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e8074eb0 0x7f58e81a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ed942640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e8074eb0 0x7f58e81a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ed942640 1 -- 172.21.15.22:0/2302217947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e8074590 msgr2=0x7f58e81a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ed942640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e8074590 0x7f58e81a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ed942640 1 -- 172.21.15.22:0/2302217947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e819f070 con 0x7f58e8112b90 2024-09-19T08:14:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.534+0000 7f58ec940640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e8074eb0 0x7f58e81a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:14:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.535+0000 7f58ed942640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e8112b90 0x7f58e819e940 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f58e000b9e0 tx=0x7f58e000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.535+0000 7f58de7fc640 1 -- 172.21.15.22:0/2302217947 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0019070 con 0x7f58e8112b90 2024-09-19T08:14:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.535+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2302217947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e819f360 con 0x7f58e8112b90 2024-09-19T08:14:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.535+0000 7f58de7fc640 1 -- 172.21.15.22:0/2302217947 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e0014440 con 0x7f58e8112b90 2024-09-19T08:14:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.535+0000 7f58de7fc640 1 -- 172.21.15.22:0/2302217947 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0012580 con 0x7f58e8112b90 2024-09-19T08:14:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.535+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2302217947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e81ab650 con 0x7f58e8112b90 2024-09-19T08:14:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.536+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2302217947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e8074970 con 0x7f58e8112b90 2024-09-19T08:14:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.537+0000 7f58de7fc640 1 -- 172.21.15.22:0/2302217947 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f58e001c070 con 0x7f58e8112b90 2024-09-19T08:14:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.538+0000 7f58de7fc640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f58d40a32d0 0x7f58d40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.538+0000 7f58ed141640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f58d40a32d0 0x7f58d40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.538+0000 7f58de7fc640 1 -- 172.21.15.22:0/2302217947 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f58e00cf4e0 con 0x7f58e8112b90 2024-09-19T08:14:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.538+0000 7f58ed141640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f58d40a32d0 0x7f58d40a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f58e819fa80 tx=0x7f58e40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.541+0000 7f58de7fc640 1 -- 172.21.15.22:0/2302217947 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f58e009bb60 con 0x7f58e8112b90 2024-09-19T08:14:26.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.720+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2302217947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58e8105030 con 0x7f58e8112b90 2024-09-19T08:14:26.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.721+0000 7f58de7fc640 1 -- 172.21.15.22:0/2302217947 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22724 (secure 0 0 0) 0x7f58e009fe60 con 0x7f58e8112b90 2024-09-19T08:14:26.722 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:26.722 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":85,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","created":"2024-09-19T08:05:19.476952+0000","modified":"2024-09-19T08:14:07.725129+0000","last_up_change":"2024-09-19T08:12:10.339586+0000","last_in_change":"2024-09-19T08:11:47.653576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:08:25.175759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:12:57.608787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:12:59.824327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ed67f2d-9b28-427a-b69f-676e7a2abcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6803","nonce":3442715307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6805","nonce":3442715307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6809","nonce":3442715307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6807","nonce":3442715307}]},"public_addr":"172.21.15.22:6803/3442715307","cluster_addr":"172.21.15.22:6805/3442715307","heartbeat_back_addr":"172.21.15.22:6809/3442715307","heartbeat_front_addr":"172.21.15.22:6807/3442715307","state":["exists","up"]},{"osd":1,"uuid":"bff3e283-4ab1-49d7-86d8-45d361995122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6801","nonce":2379700228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6803","nonce":2379700228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6807","nonce":2379700228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6805","nonce":2379700228}]},"public_addr":"172.21.15.64:6801/2379700228","cluster_addr":"172.21.15.64:6803/2379700228","heartbeat_back_addr":"172.21.15.64:6807/2379700228","heartbeat_front_addr":"172.21.15.64:6805/2379700228","state":["exists","up"]},{"osd":2,"uuid":"81421d08-7dc6-419b-a566-dee1e1e13b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6801","nonce":3363263867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6803","nonce":3363263867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6807","nonce":3363263867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6805","nonce":3363263867}]},"public_addr":"172.21.15.157:6801/3363263867","cluster_addr":"172.21.15.157:6803/3363263867","heartbeat_back_addr":"172.21.15.157:6807/3363263867","heartbeat_front_addr":"172.21.15.157:6805/3363263867","state":["exists","up"]},{"osd":3,"uuid":"34debfd0-3226-4d78-b9e6-e03dbc5190f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6811","nonce":4225229456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6813","nonce":4225229456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6817","nonce":4225229456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6815","nonce":4225229456}]},"public_addr":"172.21.15.22:6811/4225229456","cluster_addr":"172.21.15.22:6813/4225229456","heartbeat_back_addr":"172.21.15.22:6817/4225229456","heartbeat_front_addr":"172.21.15.22:6815/4225229456","state":["exists","up"]},{"osd":4,"uuid":"1293332c-d772-4ca9-bf76-79e0b29451ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6809","nonce":1858090423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6811","nonce":1858090423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6815","nonce":1858090423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6813","nonce":1858090423}]},"public_addr":"172.21.15.64:6809/1858090423","cluster_addr":"172.21.15.64:6811/1858090423","heartbeat_back_addr":"172.21.15.64:6815/1858090423","heartbeat_front_addr":"172.21.15.64:6813/1858090423","state":["exists","up"]},{"osd":5,"uuid":"240a6c42-89a2-41c1-948c-b09914b5722e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6809","nonce":2900701212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2900701212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2900701212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2900701212}]},"public_addr":"172.21.15.157:6809/2900701212","cluster_addr":"172.21.15.157:6811/2900701212","heartbeat_back_addr":"172.21.15.157:6815/2900701212","heartbeat_front_addr":"172.21.15.157:6813/2900701212","state":["exists","up"]},{"osd":6,"uuid":"80199030-109e-41c4-bc1c-731755464770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6819","nonce":4264426242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6821","nonce":4264426242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6825","nonce":4264426242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6823","nonce":4264426242}]},"public_addr":"172.21.15.22:6819/4264426242","cluster_addr":"172.21.15.22:6821/4264426242","heartbeat_back_addr":"172.21.15.22:6825/4264426242","heartbeat_front_addr":"172.21.15.22:6823/4264426242","state":["exists","up"]},{"osd":7,"uuid":"dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6817","nonce":3980224103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6819","nonce":3980224103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6823","nonce":3980224103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6821","nonce":3980224103}]},"public_addr":"172.21.15.64:6817/3980224103","cluster_addr":"172.21.15.64:6819/3980224103","heartbeat_back_addr":"172.21.15.64:6823/3980224103","heartbeat_front_addr":"172.21.15.64:6821/3980224103","state":["exists","up"]},{"osd":8,"uuid":"764b1e6e-48c9-49f9-8124-bc13bcba3a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6817","nonce":973678477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6819","nonce":973678477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6823","nonce":973678477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6821","nonce":973678477}]},"public_addr":"172.21.15.157:6817/973678477","cluster_addr":"172.21.15.157:6819/973678477","heartbeat_back_addr":"172.21.15.157:6823/973678477","heartbeat_front_addr":"172.21.15.157:6821/973678477","state":["exists","up"]},{"osd":9,"uuid":"82914818-df47-4374-9e3e-2f71fc76e78b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6827","nonce":710033990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6829","nonce":710033990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6833","nonce":710033990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6831","nonce":710033990}]},"public_addr":"172.21.15.22:6827/710033990","cluster_addr":"172.21.15.22:6829/710033990","heartbeat_back_addr":"172.21.15.22:6833/710033990","heartbeat_front_addr":"172.21.15.22:6831/710033990","state":["exists","up"]},{"osd":10,"uuid":"36ae177d-c454-4cba-ba21-cbeee0f10dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6825","nonce":373412752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6827","nonce":373412752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6831","nonce":373412752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6829","nonce":373412752}]},"public_addr":"172.21.15.64:6825/373412752","cluster_addr":"172.21.15.64:6827/373412752","heartbeat_back_addr":"172.21.15.64:6831/373412752","heartbeat_front_addr":"172.21.15.64:6829/373412752","state":["exists","up"]},{"osd":11,"uuid":"cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6825","nonce":1918889194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6827","nonce":1918889194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6831","nonce":1918889194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6829","nonce":1918889194}]},"public_addr":"172.21.15.157:6825/1918889194","cluster_addr":"172.21.15.157:6827/1918889194","heartbeat_back_addr":"172.21.15.157:6831/1918889194","heartbeat_front_addr":"172.21.15.157:6829/1918889194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:31.378218+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:56.794167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:20.790267+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:47.039106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:11.483842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:38.151466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:03.553308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:27.865321+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:53.086530+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:18.946661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:43.697243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:12:08.936588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3712426751":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/2133059277":"2024-09-20T08:05:42.354701+0000","172.21.15.22:6801/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2851750481":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/3623845502":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/384027596":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/586163474":"2024-09-20T08:05:42.354701+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:14:26.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.724+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2302217947 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f58d40a32d0 msgr2=0x7f58d40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:26.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.724+0000 7f58ef3a5640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f58d40a32d0 0x7f58d40a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f58e819fa80 tx=0x7f58e40058d0 comp rx=0 tx=0).stop 2024-09-19T08:14:26.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.724+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2302217947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e8112b90 msgr2=0x7f58e819e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:26.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.724+0000 7f58ef3a5640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e8112b90 0x7f58e819e940 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f58e000b9e0 tx=0x7f58e000bea0 comp rx=0 tx=0).stop 2024-09-19T08:14:26.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.724+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2302217947 shutdown_connections 2024-09-19T08:14:26.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.724+0000 7f58ef3a5640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f58d40a32d0 0x7f58d40a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:26.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.724+0000 7f58ef3a5640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58e8112b90 0x7f58e819e940 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:26.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.724+0000 7f58ef3a5640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58e8074eb0 0x7f58e81a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:26.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.724+0000 7f58ef3a5640 1 --2- 172.21.15.22:0/2302217947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e8074590 0x7f58e81a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:26.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.724+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2302217947 >> 172.21.15.22:0/2302217947 conn(0x7f58e80fd8c0 msgr2=0x7f58e80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:26.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.724+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2302217947 shutdown_connections 2024-09-19T08:14:26.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:26.724+0000 7f58ef3a5640 1 -- 172.21.15.22:0/2302217947 wait complete. 2024-09-19T08:14:27.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:26 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2302217947' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:14:27.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:26 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2302217947' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:14:27.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:26 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2302217947' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:14:27.420 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:14:27.683 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:27.963 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:27 smithi022 ceph-mon[30963]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:14:28.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.193+0000 7fe42293a640 1 -- 172.21.15.22:0/1596095179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c102d30 msgr2=0x7fe41c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:28.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.193+0000 7fe42293a640 1 --2- 172.21.15.22:0/1596095179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c102d30 0x7fe41c103110 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009930 tx=0x7fe40c02a420 comp rx=0 tx=0).stop 2024-09-19T08:14:28.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.193+0000 7fe42293a640 1 -- 172.21.15.22:0/1596095179 shutdown_connections 2024-09-19T08:14:28.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.193+0000 7fe42293a640 1 --2- 172.21.15.22:0/1596095179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41c1082a0 0x7fe41c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:28.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.193+0000 7fe42293a640 1 --2- 172.21.15.22:0/1596095179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe41c1036e0 0x7fe41c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:28.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.193+0000 7fe42293a640 1 --2- 172.21.15.22:0/1596095179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c102d30 0x7fe41c103110 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:28.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.193+0000 7fe42293a640 1 -- 172.21.15.22:0/1596095179 >> 172.21.15.22:0/1596095179 conn(0x7fe41c0fc7d0 msgr2=0x7fe41c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:28.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.193+0000 7fe42293a640 1 -- 172.21.15.22:0/1596095179 shutdown_connections 2024-09-19T08:14:28.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.193+0000 7fe42293a640 1 -- 172.21.15.22:0/1596095179 wait complete. 2024-09-19T08:14:28.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.193+0000 7fe42293a640 1 Processor -- start 2024-09-19T08:14:28.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.193+0000 7fe42293a640 1 -- start start 2024-09-19T08:14:28.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe42293a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe41c102d30 0x7fe41c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:28.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe42293a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41c1036e0 0x7fe41c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:28.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe42293a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c1082a0 0x7fe41c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:28.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe42293a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1120a0 con 0x7fe41c1082a0 2024-09-19T08:14:28.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe42293a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c111f20 con 0x7fe41c102d30 2024-09-19T08:14:28.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe42293a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c112220 con 0x7fe41c1036e0 2024-09-19T08:14:28.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe420ed7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c1082a0 0x7fe41c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:28.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe41bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe41c102d30 0x7fe41c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:28.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe420ed7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c1082a0 0x7fe41c19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51140/0 (socket says 172.21.15.22:51140) 2024-09-19T08:14:28.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe420ed7640 1 -- 172.21.15.22:0/2796608079 learned_addr learned my addr 172.21.15.22:0/2796608079 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:28.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe41b7fe640 1 --2- 172.21.15.22:0/2796608079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41c1036e0 0x7fe41c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:28.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe420ed7640 1 -- 172.21.15.22:0/2796608079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41c1036e0 msgr2=0x7fe41c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:28.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe420ed7640 1 --2- 172.21.15.22:0/2796608079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41c1036e0 0x7fe41c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:28.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe420ed7640 1 -- 172.21.15.22:0/2796608079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe41c102d30 msgr2=0x7fe41c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:28.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe420ed7640 1 --2- 172.21.15.22:0/2796608079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe41c102d30 0x7fe41c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:28.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe420ed7640 1 -- 172.21.15.22:0/2796608079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41c19d570 con 0x7fe41c1082a0 2024-09-19T08:14:28.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.194+0000 7fe420ed7640 1 --2- 172.21.15.22:0/2796608079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c1082a0 0x7fe41c19ce90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe41400cad0 tx=0x7fe41400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:28.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.195+0000 7fe4197fa640 1 -- 172.21.15.22:0/2796608079 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41400ae20 con 0x7fe41c1082a0 2024-09-19T08:14:28.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.195+0000 7fe42293a640 1 -- 172.21.15.22:0/2796608079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41c19d860 con 0x7fe41c1082a0 2024-09-19T08:14:28.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.195+0000 7fe4197fa640 1 -- 172.21.15.22:0/2796608079 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe414005020 con 0x7fe41c1082a0 2024-09-19T08:14:28.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.195+0000 7fe4197fa640 1 -- 172.21.15.22:0/2796608079 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41400f6b0 con 0x7fe41c1082a0 2024-09-19T08:14:28.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.195+0000 7fe42293a640 1 -- 172.21.15.22:0/2796608079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41c1a5140 con 0x7fe41c1082a0 2024-09-19T08:14:28.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.196+0000 7fe42293a640 1 -- 172.21.15.22:0/2796608079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe41c06a850 con 0x7fe41c1082a0 2024-09-19T08:14:28.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.197+0000 7fe4197fa640 1 -- 172.21.15.22:0/2796608079 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe4140051d0 con 0x7fe41c1082a0 2024-09-19T08:14:28.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.198+0000 7fe4197fa640 1 --2- 172.21.15.22:0/2796608079 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe4040a32d0 0x7fe4040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:28.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.198+0000 7fe41bfff640 1 --2- 172.21.15.22:0/2796608079 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe4040a32d0 0x7fe4040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:28.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.198+0000 7fe4197fa640 1 -- 172.21.15.22:0/2796608079 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe4140cfdc0 con 0x7fe41c1082a0 2024-09-19T08:14:28.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.198+0000 7fe41bfff640 1 --2- 172.21.15.22:0/2796608079 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe4040a32d0 0x7fe4040a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009900 tx=0x7fe40c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:28.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.206+0000 7fe4197fa640 1 -- 172.21.15.22:0/2796608079 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe414014030 con 0x7fe41c1082a0 2024-09-19T08:14:28.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:27 smithi157 ceph-mon[35143]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:14:28.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:27 smithi064 ceph-mon[34900]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:14:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.421+0000 7fe42293a640 1 -- 172.21.15.22:0/2796608079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe41c10a2b0 con 0x7fe41c1082a0 2024-09-19T08:14:28.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.422+0000 7fe4197fa640 1 -- 172.21.15.22:0/2796608079 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11906 (secure 0 0 0) 0x7fe41409c940 con 0x7fe41c1082a0 2024-09-19T08:14:28.422 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:28.423 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"btime":"2024-09-19T08:13:13:877693+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:13:13.764216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:14:28.423 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-09-19T08:14:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.424+0000 7fe42293a640 1 -- 172.21.15.22:0/2796608079 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe4040a32d0 msgr2=0x7fe4040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.424+0000 7fe42293a640 1 --2- 172.21.15.22:0/2796608079 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe4040a32d0 0x7fe4040a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009900 tx=0x7fe40c0058d0 comp rx=0 tx=0).stop 2024-09-19T08:14:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.424+0000 7fe42293a640 1 -- 172.21.15.22:0/2796608079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c1082a0 msgr2=0x7fe41c19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:28.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.424+0000 7fe42293a640 1 --2- 172.21.15.22:0/2796608079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c1082a0 0x7fe41c19ce90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe41400cad0 tx=0x7fe41400cf90 comp rx=0 tx=0).stop 2024-09-19T08:14:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.424+0000 7fe42293a640 1 -- 172.21.15.22:0/2796608079 shutdown_connections 2024-09-19T08:14:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.424+0000 7fe42293a640 1 --2- 172.21.15.22:0/2796608079 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe4040a32d0 0x7fe4040a5790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.424+0000 7fe42293a640 1 --2- 172.21.15.22:0/2796608079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41c1082a0 0x7fe41c19ce90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.424+0000 7fe42293a640 1 --2- 172.21.15.22:0/2796608079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41c1036e0 0x7fe41c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.424+0000 7fe42293a640 1 --2- 172.21.15.22:0/2796608079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe41c102d30 0x7fe41c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.424+0000 7fe42293a640 1 -- 172.21.15.22:0/2796608079 >> 172.21.15.22:0/2796608079 conn(0x7fe41c0fc7d0 msgr2=0x7fe41c105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.424+0000 7fe42293a640 1 -- 172.21.15.22:0/2796608079 shutdown_connections 2024-09-19T08:14:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:28.424+0000 7fe42293a640 1 -- 172.21.15.22:0/2796608079 wait complete. 2024-09-19T08:14:29.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:28 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2796608079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:29.168 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T08:13:01.503449+0000', 'modified': '2024-09-19T08:13:13.764216+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24481, 'mds_1': 24479, 'mds_2': 24460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6835/3333087560', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 3333087560}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 3333087560}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6835/4158350679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 4158350679}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 4158350679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6837/2142121431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6836', 'nonce': 2142121431}, {'type': 'v1', 'addr': '172.21.15.64:6837', 'nonce': 2142121431}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.157:6837/892624399', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 892624399}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 892624399}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.64:6839/358484749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6838', 'nonce': 358484749}, {'type': 'v1', 'addr': '172.21.15.64:6839', 'nonce': 358484749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.157:6839/3802797238', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 3802797238}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 3802797238}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24481, 'qdb_cluster': [24460, 24479, 24481]} 2024-09-19T08:14:29.168 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T08:14:29.168 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T08:14:29.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:28 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2796608079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:29.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:28 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2796608079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:29.428 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:29.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.887+0000 7f82177b0640 1 -- 172.21.15.22:0/1885493839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8210101420 msgr2=0x7f821010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:29.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.887+0000 7f82177b0640 1 --2- 172.21.15.22:0/1885493839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8210101420 0x7f821010d950 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8208009cd0 tx=0x7f820802a8e0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.887+0000 7f82177b0640 1 -- 172.21.15.22:0/1885493839 shutdown_connections 2024-09-19T08:14:29.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.887+0000 7f82177b0640 1 --2- 172.21.15.22:0/1885493839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8210101420 0x7f821010d950 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.887+0000 7f82177b0640 1 --2- 172.21.15.22:0/1885493839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8210100a80 0x7f8210100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.887+0000 7f82177b0640 1 --2- 172.21.15.22:0/1885493839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8210106a80 0x7f8210106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.887+0000 7f82177b0640 1 -- 172.21.15.22:0/1885493839 >> 172.21.15.22:0/1885493839 conn(0x7f82100fc7a0 msgr2=0x7f82100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:29.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.887+0000 7f82177b0640 1 -- 172.21.15.22:0/1885493839 shutdown_connections 2024-09-19T08:14:29.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.887+0000 7f82177b0640 1 -- 172.21.15.22:0/1885493839 wait complete. 2024-09-19T08:14:29.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.888+0000 7f82177b0640 1 Processor -- start 2024-09-19T08:14:29.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.888+0000 7f82177b0640 1 -- start start 2024-09-19T08:14:29.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f82177b0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8210100a80 0x7f8210197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f82177b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8210101420 0x7f8210197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f82177b0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8210106a80 0x7f821019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f82177b0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8210110e80 con 0x7f8210106a80 2024-09-19T08:14:29.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f82177b0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8210110d00 con 0x7f8210100a80 2024-09-19T08:14:29.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f82177b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8210111000 con 0x7f8210101420 2024-09-19T08:14:29.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f8215d4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8210106a80 0x7f821019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f8215d4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8210106a80 0x7f821019bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51146/0 (socket says 172.21.15.22:51146) 2024-09-19T08:14:29.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f8215d4d640 1 -- 172.21.15.22:0/912912164 learned_addr learned my addr 172.21.15.22:0/912912164 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:29.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f821554c640 1 --2- 172.21.15.22:0/912912164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8210100a80 0x7f8210197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f8214d4b640 1 --2- 172.21.15.22:0/912912164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8210101420 0x7f8210197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f8215d4d640 1 -- 172.21.15.22:0/912912164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8210101420 msgr2=0x7f8210197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:29.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f8215d4d640 1 --2- 172.21.15.22:0/912912164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8210101420 0x7f8210197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f8215d4d640 1 -- 172.21.15.22:0/912912164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8210100a80 msgr2=0x7f8210197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:29.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f8215d4d640 1 --2- 172.21.15.22:0/912912164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8210100a80 0x7f8210197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f8215d4d640 1 -- 172.21.15.22:0/912912164 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821019c2d0 con 0x7f8210106a80 2024-09-19T08:14:29.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f8215d4d640 1 --2- 172.21.15.22:0/912912164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8210106a80 0x7f821019bbf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8208005cc0 tx=0x7f8208003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:29.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f82067fc640 1 -- 172.21.15.22:0/912912164 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208004040 con 0x7f8210106a80 2024-09-19T08:14:29.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f82177b0640 1 -- 172.21.15.22:0/912912164 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821019c560 con 0x7f8210106a80 2024-09-19T08:14:29.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.889+0000 7f82067fc640 1 -- 172.21.15.22:0/912912164 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82080046c0 con 0x7f8210106a80 2024-09-19T08:14:29.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.890+0000 7f82067fc640 1 -- 172.21.15.22:0/912912164 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208046480 con 0x7f8210106a80 2024-09-19T08:14:29.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.890+0000 7f82177b0640 1 -- 172.21.15.22:0/912912164 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82101a3ea0 con 0x7f8210106a80 2024-09-19T08:14:29.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f82067fc640 1 -- 172.21.15.22:0/912912164 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f82080305a0 con 0x7f8210106a80 2024-09-19T08:14:29.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f8219b89740 1 Processor -- start 2024-09-19T08:14:29.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f8219b89740 1 -- start start 2024-09-19T08:14:29.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f8219b89740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa73890 0x55a04aa73c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f8219b89740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 0x55a04aa80880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f8219b89740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa80dc0 0x55a04aa831b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f8219b89740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a04a9da090 con 0x55a04aa73890 2024-09-19T08:14:29.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f8219b89740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a04a981a70 con 0x55a04aa80dc0 2024-09-19T08:14:29.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f8219b89740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a04a9d6050 con 0x55a04aa741b0 2024-09-19T08:14:29.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f8214d4b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa73890 0x55a04aa73c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f8214d4b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa73890 0x55a04aa73c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51160/0 (socket says 172.21.15.22:51160) 2024-09-19T08:14:29.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f8214d4b640 1 -- 172.21.15.22:0/979371399 learned_addr learned my addr 172.21.15.22:0/979371399 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:29.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f8214d4b640 1 --2- 172.21.15.22:0/979371399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa80dc0 0x55a04aa831b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f821554c640 1 --2- 172.21.15.22:0/979371399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 0x55a04aa80880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.893+0000 7f82067fc640 1 --2- 172.21.15.22:0/912912164 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f81fc0a32d0 0x7f81fc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.894+0000 7f8214d4b640 1 -- 172.21.15.22:0/979371399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 msgr2=0x55a04aa80880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:29.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.894+0000 7f8214d4b640 1 --2- 172.21.15.22:0/979371399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 0x55a04aa80880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.894+0000 7f8214d4b640 1 -- 172.21.15.22:0/979371399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa80dc0 msgr2=0x55a04aa831b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:29.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.894+0000 7f8214d4b640 1 --2- 172.21.15.22:0/979371399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa80dc0 0x55a04aa831b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.894+0000 7f8214d4b640 1 -- 172.21.15.22:0/979371399 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04a98e130 con 0x55a04aa73890 2024-09-19T08:14:29.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.894+0000 7f821554c640 1 --2- 172.21.15.22:0/979371399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 0x55a04aa80880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:29.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.894+0000 7f8215d4d640 1 --2- 172.21.15.22:0/912912164 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f81fc0a32d0 0x7f81fc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.894+0000 7f82067fc640 1 -- 172.21.15.22:0/912912164 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f820802e030 con 0x7f8210106a80 2024-09-19T08:14:29.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.894+0000 7f8214d4b640 1 --2- 172.21.15.22:0/979371399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa73890 0x55a04aa73c70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f820000bfa0 tx=0x7f820000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d336ff6c3603de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:29.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.894+0000 7f81fb7fe640 1 -- 172.21.15.22:0/979371399 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200004030 con 0x55a04aa73890 2024-09-19T08:14:29.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.894+0000 7f81fb7fe640 1 -- 172.21.15.22:0/979371399 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82000041d0 con 0x55a04aa73890 2024-09-19T08:14:29.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.894+0000 7f81fb7fe640 1 -- 172.21.15.22:0/979371399 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200010660 con 0x55a04aa73890 2024-09-19T08:14:29.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.895+0000 7f8219b89740 1 -- 172.21.15.22:0/979371399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa73890 msgr2=0x55a04aa73c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:29.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.895+0000 7f8215d4d640 1 --2- 172.21.15.22:0/912912164 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f81fc0a32d0 0x7f81fc0a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f82080bd710 tx=0x7f82080c1040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:29.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.895+0000 7f8219b89740 1 --2- 172.21.15.22:0/979371399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa73890 0x55a04aa73c70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f820000bfa0 tx=0x7f820000b840 comp rx=0 tx=0).stop 2024-09-19T08:14:29.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.895+0000 7f8219b89740 1 -- 172.21.15.22:0/979371399 shutdown_connections 2024-09-19T08:14:29.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.895+0000 7f8219b89740 1 --2- 172.21.15.22:0/979371399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa80dc0 0x55a04aa831b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.895+0000 7f8219b89740 1 --2- 172.21.15.22:0/979371399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 0x55a04aa80880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.895+0000 7f8219b89740 1 --2- 172.21.15.22:0/979371399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa73890 0x55a04aa73c70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.895+0000 7f8219b89740 1 -- 172.21.15.22:0/979371399 >> 172.21.15.22:0/979371399 conn(0x55a04aa6ff60 msgr2=0x55a04aa70340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:29.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.895+0000 7f8219b89740 1 -- 172.21.15.22:0/979371399 shutdown_connections 2024-09-19T08:14:29.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.895+0000 7f8219b89740 1 -- 172.21.15.22:0/979371399 wait complete. 2024-09-19T08:14:29.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.895+0000 7f8219b89740 1 -- start start 2024-09-19T08:14:29.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.895+0000 7f8219b89740 1 Processor -- start 2024-09-19T08:14:29.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8219b89740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa73890 0x55a04aa80650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8219b89740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 0x55a04aa7d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8219b89740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa80dc0 0x55a04aa7dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8219b89740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a04aa83990 con 0x55a04aa80dc0 2024-09-19T08:14:29.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8219b89740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a04a994f20 con 0x55a04aa73890 2024-09-19T08:14:29.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8219b89740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a04aa83b10 con 0x55a04aa741b0 2024-09-19T08:14:29.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8214d4b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa80dc0 0x55a04aa7dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8219b89740 10 client.0 _ll_register_callbacks cb 0x55a04a95e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:14:29.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8214d4b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa80dc0 0x55a04aa7dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51172/0 (socket says 172.21.15.22:51172) 2024-09-19T08:14:29.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8214d4b640 1 -- 172.21.15.22:0/2590309772 learned_addr learned my addr 172.21.15.22:0/2590309772 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:29.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f821554c640 1 --2- 172.21.15.22:0/2590309772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 0x55a04aa7d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8214d4b640 1 --2- 172.21.15.22:0/2590309772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa73890 0x55a04aa80650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8214d4b640 1 -- 172.21.15.22:0/2590309772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 msgr2=0x55a04aa7d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:29.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8214d4b640 1 --2- 172.21.15.22:0/2590309772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 0x55a04aa7d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8214d4b640 1 -- 172.21.15.22:0/2590309772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa73890 msgr2=0x55a04aa80650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:29.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8214d4b640 1 --2- 172.21.15.22:0/2590309772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa73890 0x55a04aa80650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8214d4b640 1 -- 172.21.15.22:0/2590309772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a04a74b2b0 con 0x55a04aa80dc0 2024-09-19T08:14:29.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8214d4b640 1 --2- 172.21.15.22:0/2590309772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa80dc0 0x55a04aa7dee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8200012330 tx=0x7f8200003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef894b897f5afa16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:29.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f81f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:14:29.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f81f8ff9640 1 -- 172.21.15.22:0/2590309772 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200022030 con 0x55a04aa80dc0 2024-09-19T08:14:29.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f81f8ff9640 1 -- 172.21.15.22:0/2590309772 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8200027070 con 0x55a04aa80dc0 2024-09-19T08:14:29.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.896+0000 7f8219b89740 1 -- 172.21.15.22:0/2590309772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82000041d0 con 0x55a04aa80dc0 2024-09-19T08:14:29.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.897+0000 7f81f8ff9640 1 -- 172.21.15.22:0/2590309772 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f82000035f0 con 0x55a04aa80dc0 2024-09-19T08:14:29.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.897+0000 7f8219b89740 10 client.14961 fetch_fsmap learned FSMap version 24 2024-09-19T08:14:29.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.897+0000 7f8219b89740 1 -- 172.21.15.22:0/2590309772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a04aa8f2c0 con 0x55a04aa80dc0 2024-09-19T08:14:29.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.897+0000 7f81f8ff9640 1 -- 172.21.15.22:0/2590309772 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82000041d0 con 0x55a04aa80dc0 2024-09-19T08:14:29.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.897+0000 7f81f8ff9640 1 -- 172.21.15.22:0/2590309772 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f8200010ba0 con 0x55a04aa80dc0 2024-09-19T08:14:29.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.897+0000 7f8219b89740 10 client.14961 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T08:14:29.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.897+0000 7f8219b89740 10 client.14961 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:14:29.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.897+0000 7f8219b89740 20 client.14961 populate_metadata read hostname 'smithi022' 2024-09-19T08:14:29.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.897+0000 7f8219b89740 1 --2- 172.21.15.22:0/2590309772 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a04aa91700 0x55a04aa92b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.897+0000 7f8219b89740 4 client.14961 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:14:29.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.897+0000 7f8219b89740 1 -- 172.21.15.22:0/2590309772 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a04a9e7600 con 0x55a04aa91700 2024-09-19T08:14:29.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.897+0000 7f821554c640 1 --2- 172.21.15.22:0/2590309772 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a04aa91700 0x55a04aa92b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.898+0000 7f821554c640 1 --2- 172.21.15.22:0/2590309772 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a04aa91700 0x55a04aa92b70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a99bf52bd58d1df server_cookie=70400f213a464cd in_seq=0 out_seq=0 2024-09-19T08:14:29.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.898+0000 7f81f8ff9640 10 client.14961 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:14:29.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f81f8ff9640 1 -- 172.21.15.22:0/2590309772 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a04a9e7600 con 0x55a04aa91700 2024-09-19T08:14:29.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f81f8ff9640 10 client.14961 handle_command_reply: tid=0 2024-09-19T08:14:29.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f8219b89740 1 client.14961 shutdown 2024-09-19T08:14:29.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f8219b89740 1 -- 172.21.15.22:0/2590309772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa80dc0 msgr2=0x55a04aa7dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:29.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f8219b89740 1 --2- 172.21.15.22:0/2590309772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa80dc0 0x55a04aa7dee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8200012330 tx=0x7f8200003c80 comp rx=0 tx=0).stop 2024-09-19T08:14:29.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f8219b89740 1 -- 172.21.15.22:0/2590309772 shutdown_connections 2024-09-19T08:14:29.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f8219b89740 1 --2- 172.21.15.22:0/2590309772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa80dc0 0x55a04aa7dee0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f8219b89740 1 --2- 172.21.15.22:0/2590309772 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a04aa91700 0x55a04aa92b70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f8219b89740 1 --2- 172.21.15.22:0/2590309772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 0x55a04aa7d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f8219b89740 1 --2- 172.21.15.22:0/2590309772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa73890 0x55a04aa80650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f8219b89740 1 -- 172.21.15.22:0/2590309772 >> 172.21.15.22:0/2590309772 conn(0x55a04aa6ff60 msgr2=0x55a04aa75cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:29.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f81f8ff9640 0 client.14961 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:14:29.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f8219b89740 1 -- 172.21.15.22:0/2590309772 shutdown_connections 2024-09-19T08:14:29.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f8219b89740 1 -- 172.21.15.22:0/2590309772 wait complete. 2024-09-19T08:14:29.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.899+0000 7f8219b89740 20 client.14961 trim_cache size 0 max 16384 2024-09-19T08:14:29.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.917+0000 7f8219b89740 1 Processor -- start 2024-09-19T08:14:29.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.917+0000 7f8219b89740 1 -- start start 2024-09-19T08:14:29.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.917+0000 7f8219b89740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa73890 0x55a04aa91700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.917+0000 7f8219b89740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa741b0 0x55a04aa7c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.917+0000 7f8219b89740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa80dc0 0x55a04aa9ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.917+0000 7f8219b89740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a04aa90eb0 con 0x55a04aa73890 2024-09-19T08:14:29.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.917+0000 7f8219b89740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a04aa91580 con 0x55a04aa741b0 2024-09-19T08:14:29.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.917+0000 7f8219b89740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a04aa91c50 con 0x55a04aa80dc0 2024-09-19T08:14:29.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.917+0000 7f8214d4b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa73890 0x55a04aa91700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.917+0000 7f8214d4b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa73890 0x55a04aa91700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51188/0 (socket says 172.21.15.22:51188) 2024-09-19T08:14:29.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.917+0000 7f8214d4b640 1 -- 172.21.15.22:0/2793768503 learned_addr learned my addr 172.21.15.22:0/2793768503 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:29.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.918+0000 7f821554c640 1 --2- 172.21.15.22:0/2793768503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa741b0 0x55a04aa7c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.918+0000 7f8214d4b640 1 --2- 172.21.15.22:0/2793768503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa80dc0 0x55a04aa9ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.918+0000 7f8214d4b640 1 -- 172.21.15.22:0/2793768503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa80dc0 msgr2=0x55a04aa9ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:29.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.918+0000 7f8214d4b640 1 --2- 172.21.15.22:0/2793768503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa80dc0 0x55a04aa9ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.918+0000 7f8214d4b640 1 -- 172.21.15.22:0/2793768503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa741b0 msgr2=0x55a04aa7c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:29.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.918+0000 7f8214d4b640 1 --2- 172.21.15.22:0/2793768503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa741b0 0x55a04aa7c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.918+0000 7f8214d4b640 1 -- 172.21.15.22:0/2793768503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04aa809c0 con 0x55a04aa73890 2024-09-19T08:14:29.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.918+0000 7f8214d4b640 1 --2- 172.21.15.22:0/2793768503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa73890 0x55a04aa91700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f820000c010 tx=0x7f820000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8168dbf710e14aaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:29.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.918+0000 7f81faffd640 1 -- 172.21.15.22:0/2793768503 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200022070 con 0x55a04aa73890 2024-09-19T08:14:29.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.918+0000 7f81faffd640 1 -- 172.21.15.22:0/2793768503 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f820000bd60 con 0x55a04aa73890 2024-09-19T08:14:29.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.918+0000 7f81faffd640 1 -- 172.21.15.22:0/2793768503 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82000039f0 con 0x55a04aa73890 2024-09-19T08:14:29.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.919+0000 7f8219b89740 1 -- 172.21.15.22:0/2793768503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa73890 msgr2=0x55a04aa91700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:29.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.919+0000 7f8219b89740 1 --2- 172.21.15.22:0/2793768503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa73890 0x55a04aa91700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f820000c010 tx=0x7f820000bfa0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.919+0000 7f8219b89740 1 -- 172.21.15.22:0/2793768503 shutdown_connections 2024-09-19T08:14:29.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.919+0000 7f8219b89740 1 --2- 172.21.15.22:0/2793768503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa80dc0 0x55a04aa9ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.919+0000 7f8219b89740 1 --2- 172.21.15.22:0/2793768503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa741b0 0x55a04aa7c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.919+0000 7f8219b89740 1 --2- 172.21.15.22:0/2793768503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa73890 0x55a04aa91700 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.919+0000 7f8219b89740 1 -- 172.21.15.22:0/2793768503 >> 172.21.15.22:0/2793768503 conn(0x55a04aa6ff60 msgr2=0x55a04aa6ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:29.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.919+0000 7f8219b89740 1 -- 172.21.15.22:0/2793768503 shutdown_connections 2024-09-19T08:14:29.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.919+0000 7f8219b89740 1 -- 172.21.15.22:0/2793768503 wait complete. 2024-09-19T08:14:29.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.919+0000 7f8219b89740 1 -- start start 2024-09-19T08:14:29.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.919+0000 7f8219b89740 1 Processor -- start 2024-09-19T08:14:29.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8219b89740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa73890 0x55a04aa79c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8219b89740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 0x55a04aa7a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8219b89740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa80dc0 0x55a04aa7a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8219b89740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a04aa929f0 con 0x55a04aa80dc0 2024-09-19T08:14:29.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8219b89740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a04aa92320 con 0x55a04aa73890 2024-09-19T08:14:29.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8219b89740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a04aa930c0 con 0x55a04aa741b0 2024-09-19T08:14:29.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8219b89740 10 client.0 _ll_register_callbacks cb 0x7f8200012480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:14:29.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8214d4b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa80dc0 0x55a04aa7a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8214d4b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa80dc0 0x55a04aa7a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51190/0 (socket says 172.21.15.22:51190) 2024-09-19T08:14:29.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8214d4b640 1 -- 172.21.15.22:0/3919650224 learned_addr learned my addr 172.21.15.22:0/3919650224 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:29.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8214d4b640 1 --2- 172.21.15.22:0/3919650224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa73890 0x55a04aa79c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f821554c640 1 --2- 172.21.15.22:0/3919650224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 0x55a04aa7a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8214d4b640 1 -- 172.21.15.22:0/3919650224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 msgr2=0x55a04aa7a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:29.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8214d4b640 1 --2- 172.21.15.22:0/3919650224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 0x55a04aa7a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8214d4b640 1 -- 172.21.15.22:0/3919650224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa73890 msgr2=0x55a04aa79c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:29.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8214d4b640 1 --2- 172.21.15.22:0/3919650224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa73890 0x55a04aa79c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8214d4b640 1 -- 172.21.15.22:0/3919650224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a04aa99e30 con 0x55a04aa80dc0 2024-09-19T08:14:29.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f8214d4b640 1 --2- 172.21.15.22:0/3919650224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa80dc0 0x55a04aa7a690 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f820000e7c0 tx=0x7f8200020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8460b11bc14450 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:29.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f81f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:14:29.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.920+0000 7f81f8ff9640 1 -- 172.21.15.22:0/3919650224 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200022070 con 0x55a04aa80dc0 2024-09-19T08:14:29.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.921+0000 7f81f8ff9640 1 -- 172.21.15.22:0/3919650224 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f820000ee10 con 0x55a04aa80dc0 2024-09-19T08:14:29.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.921+0000 7f8219b89740 1 -- 172.21.15.22:0/3919650224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f820000bd60 con 0x55a04aa80dc0 2024-09-19T08:14:29.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.921+0000 7f81f8ff9640 1 -- 172.21.15.22:0/3919650224 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8200003790 con 0x55a04aa80dc0 2024-09-19T08:14:29.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.921+0000 7f8219b89740 10 client.14967 fetch_fsmap learned FSMap version 24 2024-09-19T08:14:29.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.921+0000 7f8219b89740 1 -- 172.21.15.22:0/3919650224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a04aa9b0d0 con 0x55a04aa80dc0 2024-09-19T08:14:29.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.921+0000 7f81f8ff9640 1 -- 172.21.15.22:0/3919650224 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f820000bd60 con 0x55a04aa80dc0 2024-09-19T08:14:29.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.921+0000 7f81f8ff9640 1 -- 172.21.15.22:0/3919650224 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f8200010990 con 0x55a04aa80dc0 2024-09-19T08:14:29.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.921+0000 7f8219b89740 10 client.14967 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T08:14:29.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.921+0000 7f8219b89740 10 client.14967 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:14:29.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.921+0000 7f8219b89740 20 client.14967 populate_metadata read hostname 'smithi022' 2024-09-19T08:14:29.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.921+0000 7f8219b89740 1 --2- 172.21.15.22:0/3919650224 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a04aaff9d0 0x55a04aafbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:29.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.921+0000 7f8219b89740 4 client.14967 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:14:29.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.921+0000 7f8219b89740 1 -- 172.21.15.22:0/3919650224 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "status"}) -- 0x55a04a9e1b10 con 0x55a04aaff9d0 2024-09-19T08:14:29.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.921+0000 7f821554c640 1 --2- 172.21.15.22:0/3919650224 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a04aaff9d0 0x55a04aafbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:29.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f821554c640 1 --2- 172.21.15.22:0/3919650224 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a04aaff9d0 0x55a04aafbe10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fab703c21fd77108 server_cookie=d7f84d9cfd813467 in_seq=0 out_seq=0 2024-09-19T08:14:29.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f81f8ff9640 10 client.14967 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:14:29.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f81f8ff9640 1 -- 172.21.15.22:0/3919650224 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a04a9e1b10 con 0x55a04aaff9d0 2024-09-19T08:14:29.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f81f8ff9640 10 client.14967 handle_command_reply: tid=0 2024-09-19T08:14:29.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f8219b89740 1 client.14967 shutdown 2024-09-19T08:14:29.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f8219b89740 1 -- 172.21.15.22:0/3919650224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa80dc0 msgr2=0x55a04aa7a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:29.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f8219b89740 1 --2- 172.21.15.22:0/3919650224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa80dc0 0x55a04aa7a690 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f820000e7c0 tx=0x7f8200020040 comp rx=0 tx=0).stop 2024-09-19T08:14:29.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f8219b89740 1 -- 172.21.15.22:0/3919650224 shutdown_connections 2024-09-19T08:14:29.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f8219b89740 1 --2- 172.21.15.22:0/3919650224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a04aa80dc0 0x55a04aa7a690 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f8219b89740 1 --2- 172.21.15.22:0/3919650224 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a04aaff9d0 0x55a04aafbe10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f8219b89740 1 --2- 172.21.15.22:0/3919650224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a04aa741b0 0x55a04aa7a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f8219b89740 1 --2- 172.21.15.22:0/3919650224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a04aa73890 0x55a04aa79c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f8219b89740 1 -- 172.21.15.22:0/3919650224 >> 172.21.15.22:0/3919650224 conn(0x55a04aa6ff60 msgr2=0x55a04aa7c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:29.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f81f8ff9640 0 client.14967 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:14:29.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f8219b89740 1 -- 172.21.15.22:0/3919650224 shutdown_connections 2024-09-19T08:14:29.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f8219b89740 1 -- 172.21.15.22:0/3919650224 wait complete. 2024-09-19T08:14:29.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.922+0000 7f8219b89740 20 client.14967 trim_cache size 0 max 16384 2024-09-19T08:14:29.931 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:14:29.931 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "c0ad60c6-765d-11ef-bceb-c7b262605968", 2024-09-19T08:14:29.931 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 2, 2024-09-19T08:14:29.931 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24460, 2024-09-19T08:14:29.931 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-19T08:14:29.932 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-19T08:14:29.932 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-19T08:14:29.932 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 79.050606303999999, 2024-09-19T08:14:29.932 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 24, 2024-09-19T08:14:29.932 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 85, 2024-09-19T08:14:29.932 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T08:14:29.932 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 113.159740453 2024-09-19T08:14:29.932 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:14:29.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.923+0000 7f82177b0640 1 -- 172.21.15.22:0/912912164 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f81fc0a32d0 msgr2=0x7f81fc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:29.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.923+0000 7f82177b0640 1 --2- 172.21.15.22:0/912912164 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f81fc0a32d0 0x7f81fc0a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f82080bd710 tx=0x7f82080c1040 comp rx=0 tx=0).stop 2024-09-19T08:14:29.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.923+0000 7f82177b0640 1 -- 172.21.15.22:0/912912164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8210106a80 msgr2=0x7f821019bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:29.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.923+0000 7f82177b0640 1 --2- 172.21.15.22:0/912912164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8210106a80 0x7f821019bbf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8208005cc0 tx=0x7f8208003970 comp rx=0 tx=0).stop 2024-09-19T08:14:29.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.923+0000 7f82177b0640 1 -- 172.21.15.22:0/912912164 shutdown_connections 2024-09-19T08:14:29.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.923+0000 7f82177b0640 1 --2- 172.21.15.22:0/912912164 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f81fc0a32d0 0x7f81fc0a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.923+0000 7f82177b0640 1 --2- 172.21.15.22:0/912912164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8210106a80 0x7f821019bbf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.923+0000 7f82177b0640 1 --2- 172.21.15.22:0/912912164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8210101420 0x7f8210197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.923+0000 7f82177b0640 1 --2- 172.21.15.22:0/912912164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8210100a80 0x7f8210197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:29.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.923+0000 7f82177b0640 1 -- 172.21.15.22:0/912912164 >> 172.21.15.22:0/912912164 conn(0x7f82100fc7a0 msgr2=0x7f82100fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:29.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.923+0000 7f82177b0640 1 -- 172.21.15.22:0/912912164 shutdown_connections 2024-09-19T08:14:29.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:29.923+0000 7f82177b0640 1 -- 172.21.15.22:0/912912164 wait complete. 2024-09-19T08:14:30.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:29 smithi022 ceph-mon[30963]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:30.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:29 smithi157 ceph-mon[35143]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:30.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:29 smithi064 ceph-mon[34900]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:30.687 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T08:14:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:31.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.394+0000 7fe23388a640 1 -- 172.21.15.22:0/1454057368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe22c10c170 msgr2=0x7fe22c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:31.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.394+0000 7fe23388a640 1 --2- 172.21.15.22:0/1454057368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe22c10c170 0x7fe22c10e610 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe224009ca0 tx=0x7fe22402a950 comp rx=0 tx=0).stop 2024-09-19T08:14:31.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.394+0000 7fe23388a640 1 -- 172.21.15.22:0/1454057368 shutdown_connections 2024-09-19T08:14:31.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.394+0000 7fe23388a640 1 --2- 172.21.15.22:0/1454057368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe22c10c170 0x7fe22c10e610 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.394+0000 7fe23388a640 1 --2- 172.21.15.22:0/1454057368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe22c103660 0x7fe22c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.394+0000 7fe23388a640 1 --2- 172.21.15.22:0/1454057368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c102d40 0x7fe22c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.394+0000 7fe23388a640 1 -- 172.21.15.22:0/1454057368 >> 172.21.15.22:0/1454057368 conn(0x7fe22c0fc7a0 msgr2=0x7fe22c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:31.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.394+0000 7fe23388a640 1 -- 172.21.15.22:0/1454057368 shutdown_connections 2024-09-19T08:14:31.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.394+0000 7fe23388a640 1 -- 172.21.15.22:0/1454057368 wait complete. 2024-09-19T08:14:31.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.394+0000 7fe23388a640 1 Processor -- start 2024-09-19T08:14:31.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.394+0000 7fe23388a640 1 -- start start 2024-09-19T08:14:31.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe23388a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe22c102d40 0x7fe22c1a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe23388a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c103660 0x7fe22c1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe23388a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe22c10c170 0x7fe22c10ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe23388a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c111f30 con 0x7fe22c102d40 2024-09-19T08:14:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe23388a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c111db0 con 0x7fe22c10c170 2024-09-19T08:14:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe23388a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1120b0 con 0x7fe22c103660 2024-09-19T08:14:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe231626640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe22c102d40 0x7fe22c1a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe231626640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe22c102d40 0x7fe22c1a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51210/0 (socket says 172.21.15.22:51210) 2024-09-19T08:14:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe231626640 1 -- 172.21.15.22:0/1655465149 learned_addr learned my addr 172.21.15.22:0/1655465149 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe231e27640 1 --2- 172.21.15.22:0/1655465149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe22c10c170 0x7fe22c10ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe230e25640 1 --2- 172.21.15.22:0/1655465149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c103660 0x7fe22c1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe231626640 1 -- 172.21.15.22:0/1655465149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c103660 msgr2=0x7fe22c1a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe231626640 1 --2- 172.21.15.22:0/1655465149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c103660 0x7fe22c1a4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe231626640 1 -- 172.21.15.22:0/1655465149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe22c10c170 msgr2=0x7fe22c10ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe231626640 1 --2- 172.21.15.22:0/1655465149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe22c10c170 0x7fe22c10ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe231626640 1 -- 172.21.15.22:0/1655465149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c10b540 con 0x7fe22c102d40 2024-09-19T08:14:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.395+0000 7fe231626640 1 --2- 172.21.15.22:0/1655465149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe22c102d40 0x7fe22c1a4700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c920 tx=0x7fe22800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.396+0000 7fe2227fc640 1 -- 172.21.15.22:0/1655465149 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228007c70 con 0x7fe22c102d40 2024-09-19T08:14:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.396+0000 7fe2227fc640 1 -- 172.21.15.22:0/1655465149 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe228007e10 con 0x7fe22c102d40 2024-09-19T08:14:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.396+0000 7fe2227fc640 1 -- 172.21.15.22:0/1655465149 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228018890 con 0x7fe22c102d40 2024-09-19T08:14:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.396+0000 7fe23388a640 1 -- 172.21.15.22:0/1655465149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22c10b830 con 0x7fe22c102d40 2024-09-19T08:14:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.396+0000 7fe23388a640 1 -- 172.21.15.22:0/1655465149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22c1ab680 con 0x7fe22c102d40 2024-09-19T08:14:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.398+0000 7fe2227fc640 1 -- 172.21.15.22:0/1655465149 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe228018a70 con 0x7fe22c102d40 2024-09-19T08:14:31.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe235c63740 1 Processor -- start 2024-09-19T08:14:31.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe235c63740 1 -- start start 2024-09-19T08:14:31.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe235c63740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c058192db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe235c63740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe235c63740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 0x55c05819ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe235c63740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c0580f9090 con 0x55c0581932f0 2024-09-19T08:14:31.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe235c63740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c0580a0a70 con 0x55c0581929d0 2024-09-19T08:14:31.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe235c63740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c0580f5050 con 0x55c05819dbb0 2024-09-19T08:14:31.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe2227fc640 1 --2- 172.21.15.22:0/1655465149 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe2180a3350 0x7fe2180a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe231626640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe231626640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51214/0 (socket says 172.21.15.22:51214) 2024-09-19T08:14:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe231626640 1 -- 172.21.15.22:0/863711684 learned_addr learned my addr 172.21.15.22:0/863711684 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe231e27640 1 --2- 172.21.15.22:0/1655465149 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe2180a3350 0x7fe2180a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe231e27640 1 --2- 172.21.15.22:0/863711684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c058192db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe230e25640 1 --2- 172.21.15.22:0/863711684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 0x55c05819ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.399+0000 7fe2227fc640 1 -- 172.21.15.22:0/1655465149 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe2280cfbd0 con 0x7fe22c102d40 2024-09-19T08:14:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.400+0000 7fe231626640 1 -- 172.21.15.22:0/863711684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 msgr2=0x55c05819ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.400+0000 7fe231626640 1 --2- 172.21.15.22:0/863711684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 0x55c05819ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.400+0000 7fe231626640 1 -- 172.21.15.22:0/863711684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 msgr2=0x55c058192db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.400+0000 7fe231626640 1 --2- 172.21.15.22:0/863711684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c058192db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.400+0000 7fe231626640 1 -- 172.21.15.22:0/863711684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0580ad130 con 0x55c0581932f0 2024-09-19T08:14:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.400+0000 7fe230e25640 1 --2- 172.21.15.22:0/863711684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 0x55c05819ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.400+0000 7fe231e27640 1 --2- 172.21.15.22:0/863711684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c058192db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.400+0000 7fe231626640 1 --2- 172.21.15.22:0/863711684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819d670 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c8f0 tx=0x7fe22809bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f963dcd0335629 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.400+0000 7fe231e27640 1 --2- 172.21.15.22:0/1655465149 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe2180a3350 0x7fe2180a5810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe224005e30 tx=0x7fe22402fe10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.400+0000 7fe2177fe640 1 -- 172.21.15.22:0/863711684 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22809c050 con 0x55c0581932f0 2024-09-19T08:14:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.400+0000 7fe2177fe640 1 -- 172.21.15.22:0/863711684 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2280a0400 con 0x55c0581932f0 2024-09-19T08:14:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.400+0000 7fe2177fe640 1 -- 172.21.15.22:0/863711684 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2280986c0 con 0x55c0581932f0 2024-09-19T08:14:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.401+0000 7fe235c63740 1 -- 172.21.15.22:0/863711684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 msgr2=0x55c05819d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.401+0000 7fe235c63740 1 --2- 172.21.15.22:0/863711684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819d670 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c8f0 tx=0x7fe22809bf50 comp rx=0 tx=0).stop 2024-09-19T08:14:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.401+0000 7fe235c63740 1 -- 172.21.15.22:0/863711684 shutdown_connections 2024-09-19T08:14:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.401+0000 7fe235c63740 1 --2- 172.21.15.22:0/863711684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 0x55c05819ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.401+0000 7fe235c63740 1 --2- 172.21.15.22:0/863711684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819d670 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.401+0000 7fe235c63740 1 --2- 172.21.15.22:0/863711684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c058192db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.401+0000 7fe235c63740 1 -- 172.21.15.22:0/863711684 >> 172.21.15.22:0/863711684 conn(0x55c05818ef60 msgr2=0x55c05818f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.401+0000 7fe235c63740 1 -- 172.21.15.22:0/863711684 shutdown_connections 2024-09-19T08:14:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.401+0000 7fe235c63740 1 -- 172.21.15.22:0/863711684 wait complete. 2024-09-19T08:14:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.401+0000 7fe235c63740 1 -- start start 2024-09-19T08:14:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.401+0000 7fe235c63740 1 Processor -- start 2024-09-19T08:14:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe235c63740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0581929d0 0x55c05819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe235c63740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe235c63740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05819dbb0 0x55c05819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe235c63740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c0581a0780 con 0x55c0581932f0 2024-09-19T08:14:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe235c63740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c0580b3f20 con 0x55c05819dbb0 2024-09-19T08:14:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe235c63740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c0581a0900 con 0x55c0581929d0 2024-09-19T08:14:31.437 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:14:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "c0ad60c6-765d-11ef-bceb-c7b262605968", 2024-09-19T08:14:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 1, 2024-09-19T08:14:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24479, 2024-09-19T08:14:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-19T08:14:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-19T08:14:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-19T08:14:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 81.565979589999998, 2024-09-19T08:14:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 24, 2024-09-19T08:14:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 85, 2024-09-19T08:14:31.438 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T08:14:31.438 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 100.78103578699999 2024-09-19T08:14:31.438 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:14:31.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe235c63740 10 client.0 _ll_register_callbacks cb 0x55c05807d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:14:31.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe230e25640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe230e25640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51230/0 (socket says 172.21.15.22:51230) 2024-09-19T08:14:31.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe230e25640 1 -- 172.21.15.22:0/3209217001 learned_addr learned my addr 172.21.15.22:0/3209217001 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:31.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe231626640 1 --2- 172.21.15.22:0/3209217001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0581929d0 0x55c05819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe231e27640 1 --2- 172.21.15.22:0/3209217001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05819dbb0 0x55c05819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe230e25640 1 -- 172.21.15.22:0/3209217001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0581929d0 msgr2=0x55c05819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:31.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe230e25640 1 --2- 172.21.15.22:0/3209217001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0581929d0 0x55c05819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe230e25640 1 -- 172.21.15.22:0/3209217001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05819dbb0 msgr2=0x55c05819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:31.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe230e25640 1 --2- 172.21.15.22:0/3209217001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05819dbb0 0x55c05819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe230e25640 1 -- 172.21.15.22:0/3209217001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c057e6a2b0 con 0x55c0581932f0 2024-09-19T08:14:31.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe230e25640 1 --2- 172.21.15.22:0/3209217001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819a820 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe21c010420 tx=0x7fe21c010820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4406bd251f0614 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:31.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe214ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:14:31.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.402+0000 7fe214ff9640 1 -- 172.21.15.22:0/3209217001 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c016070 con 0x55c0581932f0 2024-09-19T08:14:31.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.403+0000 7fe214ff9640 1 -- 172.21.15.22:0/3209217001 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe21c012920 con 0x55c0581932f0 2024-09-19T08:14:31.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.403+0000 7fe235c63740 1 -- 172.21.15.22:0/3209217001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2280a0400 con 0x55c0581932f0 2024-09-19T08:14:31.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.403+0000 7fe214ff9640 1 -- 172.21.15.22:0/3209217001 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe21c012b20 con 0x55c0581932f0 2024-09-19T08:14:31.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.403+0000 7fe235c63740 10 client.14979 fetch_fsmap learned FSMap version 24 2024-09-19T08:14:31.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.403+0000 7fe235c63740 1 -- 172.21.15.22:0/3209217001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c0581ac1c0 con 0x55c0581932f0 2024-09-19T08:14:31.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.403+0000 7fe214ff9640 1 -- 172.21.15.22:0/3209217001 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2280a0400 con 0x55c0581932f0 2024-09-19T08:14:31.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.403+0000 7fe214ff9640 1 -- 172.21.15.22:0/3209217001 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe21c00f040 con 0x55c0581932f0 2024-09-19T08:14:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.403+0000 7fe235c63740 10 client.14979 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T08:14:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.403+0000 7fe235c63740 10 client.14979 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T08:14:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.403+0000 7fe235c63740 20 client.14979 populate_metadata read hostname 'smithi022' 2024-09-19T08:14:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.403+0000 7fe235c63740 1 --2- 172.21.15.22:0/3209217001 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55c0581ae810 0x55c0581afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.403+0000 7fe235c63740 4 client.14979 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:14:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.403+0000 7fe235c63740 1 -- 172.21.15.22:0/3209217001 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c058106600 con 0x55c0581ae810 2024-09-19T08:14:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.403+0000 7fe231626640 1 --2- 172.21.15.22:0/3209217001 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55c0581ae810 0x55c0581afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.404+0000 7fe231626640 1 --2- 172.21.15.22:0/3209217001 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55c0581ae810 0x55c0581afc80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad5573cc54f7dd9a server_cookie=806e7d38aa066b1d in_seq=0 out_seq=0 2024-09-19T08:14:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.404+0000 7fe214ff9640 10 client.14979 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:14:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe214ff9640 1 -- 172.21.15.22:0/3209217001 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c058106600 con 0x55c0581ae810 2024-09-19T08:14:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe214ff9640 10 client.14979 handle_command_reply: tid=0 2024-09-19T08:14:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe235c63740 1 client.14979 shutdown 2024-09-19T08:14:31.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe235c63740 1 -- 172.21.15.22:0/3209217001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 msgr2=0x55c05819a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:31.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe235c63740 1 --2- 172.21.15.22:0/3209217001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819a820 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe21c010420 tx=0x7fe21c010820 comp rx=0 tx=0).stop 2024-09-19T08:14:31.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe235c63740 1 -- 172.21.15.22:0/3209217001 shutdown_connections 2024-09-19T08:14:31.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe235c63740 1 --2- 172.21.15.22:0/3209217001 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55c0581ae810 0x55c0581afc80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe235c63740 1 --2- 172.21.15.22:0/3209217001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05819dbb0 0x55c05819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe235c63740 1 --2- 172.21.15.22:0/3209217001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819a820 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe235c63740 1 --2- 172.21.15.22:0/3209217001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0581929d0 0x55c05819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe235c63740 1 -- 172.21.15.22:0/3209217001 >> 172.21.15.22:0/3209217001 conn(0x55c05818ef60 msgr2=0x55c058194c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:31.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe214ff9640 0 client.14979 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:14:31.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe235c63740 1 -- 172.21.15.22:0/3209217001 shutdown_connections 2024-09-19T08:14:31.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe235c63740 1 -- 172.21.15.22:0/3209217001 wait complete. 2024-09-19T08:14:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.405+0000 7fe235c63740 20 client.14979 trim_cache size 0 max 16384 2024-09-19T08:14:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe235c63740 1 Processor -- start 2024-09-19T08:14:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe235c63740 1 -- start start 2024-09-19T08:14:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe235c63740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c0581ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe235c63740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe235c63740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 0x55c0581b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe235c63740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c0581adfc0 con 0x55c0581932f0 2024-09-19T08:14:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe235c63740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c0581ae690 con 0x55c0581929d0 2024-09-19T08:14:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe235c63740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c0581aed60 con 0x55c05819dbb0 2024-09-19T08:14:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe230e25640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe230e25640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51236/0 (socket says 172.21.15.22:51236) 2024-09-19T08:14:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe230e25640 1 -- 172.21.15.22:0/3226432076 learned_addr learned my addr 172.21.15.22:0/3226432076 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe231626640 1 --2- 172.21.15.22:0/3226432076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c0581ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe230e25640 1 -- 172.21.15.22:0/3226432076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 msgr2=0x55c0581b7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe230e25640 1 --2- 172.21.15.22:0/3226432076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 0x55c0581b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe230e25640 1 -- 172.21.15.22:0/3226432076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 msgr2=0x55c0581ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe230e25640 1 --2- 172.21.15.22:0/3226432076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c0581ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe230e25640 1 -- 172.21.15.22:0/3226432076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0581927e0 con 0x55c0581932f0 2024-09-19T08:14:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.423+0000 7fe231626640 1 --2- 172.21.15.22:0/3226432076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c0581ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:14:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.424+0000 7fe230e25640 1 --2- 172.21.15.22:0/3226432076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819c680 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe21c01cfd0 tx=0x7fe21c013cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6056a5802fbe0210 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.424+0000 7fe216ffd640 1 -- 172.21.15.22:0/3226432076 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c033070 con 0x55c0581932f0 2024-09-19T08:14:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.424+0000 7fe216ffd640 1 -- 172.21.15.22:0/3226432076 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe21c005520 con 0x55c0581932f0 2024-09-19T08:14:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.424+0000 7fe216ffd640 1 -- 172.21.15.22:0/3226432076 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c016050 con 0x55c0581932f0 2024-09-19T08:14:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.424+0000 7fe235c63740 1 -- 172.21.15.22:0/3226432076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 msgr2=0x55c05819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.424+0000 7fe235c63740 1 --2- 172.21.15.22:0/3226432076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819c680 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe21c01cfd0 tx=0x7fe21c013cd0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.425+0000 7fe235c63740 1 -- 172.21.15.22:0/3226432076 shutdown_connections 2024-09-19T08:14:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.425+0000 7fe235c63740 1 --2- 172.21.15.22:0/3226432076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 0x55c0581b7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.425+0000 7fe235c63740 1 --2- 172.21.15.22:0/3226432076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c05819c680 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.425+0000 7fe235c63740 1 --2- 172.21.15.22:0/3226432076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c0581ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.425+0000 7fe235c63740 1 -- 172.21.15.22:0/3226432076 >> 172.21.15.22:0/3226432076 conn(0x55c05818ef60 msgr2=0x55c05819bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.425+0000 7fe235c63740 1 -- 172.21.15.22:0/3226432076 shutdown_connections 2024-09-19T08:14:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.425+0000 7fe235c63740 1 -- 172.21.15.22:0/3226432076 wait complete. 2024-09-19T08:14:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.425+0000 7fe235c63740 1 -- start start 2024-09-19T08:14:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.425+0000 7fe235c63740 1 Processor -- start 2024-09-19T08:14:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.425+0000 7fe235c63740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c0581b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.425+0000 7fe235c63740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c0581b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe235c63740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 0x55c058216f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe235c63740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c0581afb00 con 0x55c0581932f0 2024-09-19T08:14:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe235c63740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c0581af430 con 0x55c0581929d0 2024-09-19T08:14:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe235c63740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c0581b01d0 con 0x55c05819dbb0 2024-09-19T08:14:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe235c63740 10 client.0 _ll_register_callbacks cb 0x7fe22c06b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:14:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe230e25640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c0581b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe230e25640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c0581b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51238/0 (socket says 172.21.15.22:51238) 2024-09-19T08:14:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe230e25640 1 -- 172.21.15.22:0/2752059043 learned_addr learned my addr 172.21.15.22:0/2752059043 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe231626640 1 --2- 172.21.15.22:0/2752059043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c0581b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe231e27640 1 --2- 172.21.15.22:0/2752059043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 0x55c058216f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe230e25640 1 -- 172.21.15.22:0/2752059043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 msgr2=0x55c058216f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe230e25640 1 --2- 172.21.15.22:0/2752059043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 0x55c058216f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe230e25640 1 -- 172.21.15.22:0/2752059043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 msgr2=0x55c0581b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe230e25640 1 --2- 172.21.15.22:0/2752059043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c0581b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe230e25640 1 -- 172.21.15.22:0/2752059043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0582174d0 con 0x55c0581932f0 2024-09-19T08:14:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe231626640 1 --2- 172.21.15.22:0/2752059043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c0581b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:14:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe230e25640 1 --2- 172.21.15.22:0/2752059043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c0581b3950 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe21c010860 tx=0x7fe21c010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c98bdc2cd7600a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe214ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:14:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe214ff9640 1 -- 172.21.15.22:0/2752059043 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c03c060 con 0x55c0581932f0 2024-09-19T08:14:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe214ff9640 1 -- 172.21.15.22:0/2752059043 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe21c0115f0 con 0x55c0581932f0 2024-09-19T08:14:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe235c63740 1 -- 172.21.15.22:0/2752059043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe21c005520 con 0x55c0581932f0 2024-09-19T08:14:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.426+0000 7fe214ff9640 1 -- 172.21.15.22:0/2752059043 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe21c033070 con 0x55c0581932f0 2024-09-19T08:14:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.427+0000 7fe235c63740 10 client.14985 fetch_fsmap learned FSMap version 24 2024-09-19T08:14:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.427+0000 7fe235c63740 1 -- 172.21.15.22:0/2752059043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c0582186e0 con 0x55c0581932f0 2024-09-19T08:14:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.427+0000 7fe214ff9640 1 -- 172.21.15.22:0/2752059043 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe21c005520 con 0x55c0581932f0 2024-09-19T08:14:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.427+0000 7fe214ff9640 1 -- 172.21.15.22:0/2752059043 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe21c041070 con 0x55c0581932f0 2024-09-19T08:14:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.427+0000 7fe235c63740 10 client.14985 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T08:14:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.427+0000 7fe235c63740 10 client.14985 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T08:14:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.427+0000 7fe235c63740 20 client.14985 populate_metadata read hostname 'smithi022' 2024-09-19T08:14:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.427+0000 7fe235c63740 1 --2- 172.21.15.22:0/2752059043 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55c05821ce30 0x55c05821bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.427+0000 7fe235c63740 4 client.14985 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:14:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.427+0000 7fe235c63740 1 -- 172.21.15.22:0/2752059043 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "status"}) -- 0x55c058100b10 con 0x55c05821ce30 2024-09-19T08:14:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.427+0000 7fe231626640 1 --2- 172.21.15.22:0/2752059043 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55c05821ce30 0x55c05821bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe214ff9640 10 client.14985 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:14:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe231626640 1 --2- 172.21.15.22:0/2752059043 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55c05821ce30 0x55c05821bb40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8470141aa2b1b28d server_cookie=c0bba128b9b7a564 in_seq=0 out_seq=0 2024-09-19T08:14:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe214ff9640 1 -- 172.21.15.22:0/2752059043 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c058100b10 con 0x55c05821ce30 2024-09-19T08:14:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe214ff9640 10 client.14985 handle_command_reply: tid=0 2024-09-19T08:14:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe235c63740 1 client.14985 shutdown 2024-09-19T08:14:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe235c63740 1 -- 172.21.15.22:0/2752059043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 msgr2=0x55c0581b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe235c63740 1 --2- 172.21.15.22:0/2752059043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c0581b3950 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe21c010860 tx=0x7fe21c010d20 comp rx=0 tx=0).stop 2024-09-19T08:14:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe235c63740 1 -- 172.21.15.22:0/2752059043 shutdown_connections 2024-09-19T08:14:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe235c63740 1 --2- 172.21.15.22:0/2752059043 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55c05821ce30 0x55c05821bb40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe235c63740 1 --2- 172.21.15.22:0/2752059043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c05819dbb0 0x55c058216f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe235c63740 1 --2- 172.21.15.22:0/2752059043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0581932f0 0x55c0581b3950 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe235c63740 1 --2- 172.21.15.22:0/2752059043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0581929d0 0x55c0581b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe235c63740 1 -- 172.21.15.22:0/2752059043 >> 172.21.15.22:0/2752059043 conn(0x55c05818ef60 msgr2=0x55c058199800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe214ff9640 0 client.14985 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:14:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe235c63740 1 -- 172.21.15.22:0/2752059043 shutdown_connections 2024-09-19T08:14:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe235c63740 1 -- 172.21.15.22:0/2752059043 wait complete. 2024-09-19T08:14:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.428+0000 7fe235c63740 20 client.14985 trim_cache size 0 max 16384 2024-09-19T08:14:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.429+0000 7fe23388a640 1 -- 172.21.15.22:0/1655465149 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe2180a3350 msgr2=0x7fe2180a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.429+0000 7fe23388a640 1 --2- 172.21.15.22:0/1655465149 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe2180a3350 0x7fe2180a5810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe224005e30 tx=0x7fe22402fe10 comp rx=0 tx=0).stop 2024-09-19T08:14:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.429+0000 7fe23388a640 1 -- 172.21.15.22:0/1655465149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe22c102d40 msgr2=0x7fe22c1a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.429+0000 7fe23388a640 1 --2- 172.21.15.22:0/1655465149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe22c102d40 0x7fe22c1a4700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c920 tx=0x7fe22800cde0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.429+0000 7fe23388a640 1 -- 172.21.15.22:0/1655465149 shutdown_connections 2024-09-19T08:14:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.429+0000 7fe23388a640 1 --2- 172.21.15.22:0/1655465149 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fe2180a3350 0x7fe2180a5810 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.429+0000 7fe23388a640 1 --2- 172.21.15.22:0/1655465149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe22c10c170 0x7fe22c10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.429+0000 7fe23388a640 1 --2- 172.21.15.22:0/1655465149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe22c103660 0x7fe22c1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.429+0000 7fe23388a640 1 --2- 172.21.15.22:0/1655465149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe22c102d40 0x7fe22c1a4700 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.429+0000 7fe23388a640 1 -- 172.21.15.22:0/1655465149 >> 172.21.15.22:0/1655465149 conn(0x7fe22c0fc7a0 msgr2=0x7fe22c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.429+0000 7fe23388a640 1 -- 172.21.15.22:0/1655465149 shutdown_connections 2024-09-19T08:14:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:31.429+0000 7fe23388a640 1 -- 172.21.15.22:0/1655465149 wait complete. 2024-09-19T08:14:32.109 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:31 smithi022 ceph-mon[30963]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:14:32.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T08:14:32.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:31 smithi157 ceph-mon[35143]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:14:32.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:31 smithi064 ceph-mon[34900]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:14:32.405 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:32.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.864+0000 7f378b27d640 1 -- 172.21.15.22:0/2989089737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3784102dd0 msgr2=0x7f37841031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:32.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.864+0000 7f378b27d640 1 --2- 172.21.15.22:0/2989089737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3784102dd0 0x7f37841031b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f377c009930 tx=0x7f377c02a440 comp rx=0 tx=0).stop 2024-09-19T08:14:32.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.864+0000 7f378b27d640 1 -- 172.21.15.22:0/2989089737 shutdown_connections 2024-09-19T08:14:32.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.864+0000 7f378b27d640 1 --2- 172.21.15.22:0/2989089737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3784108300 0x7f378410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.864+0000 7f378b27d640 1 --2- 172.21.15.22:0/2989089737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3784103780 0x7f3784107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.864+0000 7f378b27d640 1 --2- 172.21.15.22:0/2989089737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3784102dd0 0x7f37841031b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.864+0000 7f378b27d640 1 -- 172.21.15.22:0/2989089737 >> 172.21.15.22:0/2989089737 conn(0x7f37840fc7d0 msgr2=0x7f37840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:32.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.865+0000 7f378b27d640 1 -- 172.21.15.22:0/2989089737 shutdown_connections 2024-09-19T08:14:32.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.865+0000 7f378b27d640 1 -- 172.21.15.22:0/2989089737 wait complete. 2024-09-19T08:14:32.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.865+0000 7f378b27d640 1 Processor -- start 2024-09-19T08:14:32.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.865+0000 7f378b27d640 1 -- start start 2024-09-19T08:14:32.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f378b27d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3784102dd0 0x7f3784198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f378b27d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3784103780 0x7f3784198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f378b27d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3784108300 0x7f378419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f378b27d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f37841120f0 con 0x7f3784103780 2024-09-19T08:14:32.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f378b27d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3784111f70 con 0x7f3784108300 2024-09-19T08:14:32.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f378b27d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3784112270 con 0x7f3784102dd0 2024-09-19T08:14:32.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f3788818640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3784103780 0x7f3784198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f3788818640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3784103780 0x7f3784198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51254/0 (socket says 172.21.15.22:51254) 2024-09-19T08:14:32.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f3788818640 1 -- 172.21.15.22:0/3187066720 learned_addr learned my addr 172.21.15.22:0/3187066720 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:32.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f378981a640 1 --2- 172.21.15.22:0/3187066720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3784108300 0x7f378419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f3789019640 1 --2- 172.21.15.22:0/3187066720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3784102dd0 0x7f3784198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f3788818640 1 -- 172.21.15.22:0/3187066720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3784102dd0 msgr2=0x7f3784198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:32.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f3788818640 1 --2- 172.21.15.22:0/3187066720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3784102dd0 0x7f3784198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f3788818640 1 -- 172.21.15.22:0/3187066720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3784108300 msgr2=0x7f378419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:32.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f3788818640 1 --2- 172.21.15.22:0/3187066720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3784108300 0x7f378419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f3788818640 1 -- 172.21.15.22:0/3187066720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378419d540 con 0x7f3784103780 2024-09-19T08:14:32.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f3788818640 1 --2- 172.21.15.22:0/3187066720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3784103780 0x7f3784198ad0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3778006fd0 tx=0x7f377800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:32.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f37827fc640 1 -- 172.21.15.22:0/3187066720 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778011830 con 0x7f3784103780 2024-09-19T08:14:32.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f37827fc640 1 -- 172.21.15.22:0/3187066720 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3778016070 con 0x7f3784103780 2024-09-19T08:14:32.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f378b27d640 1 -- 172.21.15.22:0/3187066720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378419d830 con 0x7f3784103780 2024-09-19T08:14:32.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f37827fc640 1 -- 172.21.15.22:0/3187066720 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377800f2e0 con 0x7f3784103780 2024-09-19T08:14:32.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.866+0000 7f378b27d640 1 -- 172.21.15.22:0/3187066720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37841a5110 con 0x7f3784103780 2024-09-19T08:14:32.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.869+0000 7f37827fc640 1 -- 172.21.15.22:0/3187066720 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f377801b460 con 0x7f3784103780 2024-09-19T08:14:32.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.869+0000 7f378d656740 1 Processor -- start 2024-09-19T08:14:32.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.869+0000 7f378d656740 1 -- start start 2024-09-19T08:14:32.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.869+0000 7f378d656740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 0x5560ff5cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.869+0000 7f378d656740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f378d656740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5d9bb0 0x5560ff5dbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f378d656740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5560ff535090 con 0x5560ff5cf2f0 2024-09-19T08:14:32.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f378d656740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5560ff4dca70 con 0x5560ff5d9bb0 2024-09-19T08:14:32.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f378d656740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5560ff531050 con 0x5560ff5ce9d0 2024-09-19T08:14:32.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f3789019640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f3789019640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51264/0 (socket says 172.21.15.22:51264) 2024-09-19T08:14:32.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f3789019640 1 -- 172.21.15.22:0/1435890852 learned_addr learned my addr 172.21.15.22:0/1435890852 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:32.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f378981a640 1 --2- 172.21.15.22:0/1435890852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 0x5560ff5cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f3788818640 1 --2- 172.21.15.22:0/1435890852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5d9bb0 0x5560ff5dbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f3789019640 1 -- 172.21.15.22:0/1435890852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 msgr2=0x5560ff5cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:32.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f3789019640 1 --2- 172.21.15.22:0/1435890852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 0x5560ff5cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f3789019640 1 -- 172.21.15.22:0/1435890852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5d9bb0 msgr2=0x5560ff5dbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:32.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f3789019640 1 --2- 172.21.15.22:0/1435890852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5d9bb0 0x5560ff5dbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f3789019640 1 -- 172.21.15.22:0/1435890852 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560ff4e9130 con 0x5560ff5cf2f0 2024-09-19T08:14:32.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f37827fc640 1 --2- 172.21.15.22:0/3187066720 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f376c0a3350 0x7f376c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f378981a640 1 --2- 172.21.15.22:0/3187066720 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f376c0a3350 0x7f376c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f3789019640 1 --2- 172.21.15.22:0/1435890852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d9670 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f377c005940 tx=0x7f377c032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cee14aa5165ea2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:32.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.870+0000 7f37827fc640 1 -- 172.21.15.22:0/3187066720 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f37780cf0b0 con 0x7f3784103780 2024-09-19T08:14:32.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.871+0000 7f378981a640 1 --2- 172.21.15.22:0/3187066720 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f376c0a3350 0x7f376c0a5810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3774007ed0 tx=0x7f3774007680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:32.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.871+0000 7f376b7fe640 1 -- 172.21.15.22:0/1435890852 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c03f070 con 0x5560ff5cf2f0 2024-09-19T08:14:32.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.871+0000 7f376b7fe640 1 -- 172.21.15.22:0/1435890852 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377c0317b0 con 0x5560ff5cf2f0 2024-09-19T08:14:32.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.871+0000 7f376b7fe640 1 -- 172.21.15.22:0/1435890852 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c035070 con 0x5560ff5cf2f0 2024-09-19T08:14:32.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.872+0000 7f378d656740 1 -- 172.21.15.22:0/1435890852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 msgr2=0x5560ff5d9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:32.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.872+0000 7f378d656740 1 --2- 172.21.15.22:0/1435890852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d9670 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f377c005940 tx=0x7f377c032cb0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.872+0000 7f378d656740 1 -- 172.21.15.22:0/1435890852 shutdown_connections 2024-09-19T08:14:32.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.872+0000 7f378d656740 1 --2- 172.21.15.22:0/1435890852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5d9bb0 0x5560ff5dbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.872+0000 7f378d656740 1 --2- 172.21.15.22:0/1435890852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d9670 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.872+0000 7f378d656740 1 --2- 172.21.15.22:0/1435890852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 0x5560ff5cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.872+0000 7f378d656740 1 -- 172.21.15.22:0/1435890852 >> 172.21.15.22:0/1435890852 conn(0x5560ff5caf60 msgr2=0x5560ff5cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:32.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.872+0000 7f378d656740 1 -- 172.21.15.22:0/1435890852 shutdown_connections 2024-09-19T08:14:32.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.872+0000 7f378d656740 1 -- 172.21.15.22:0/1435890852 wait complete. 2024-09-19T08:14:32.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.872+0000 7f378d656740 1 -- start start 2024-09-19T08:14:32.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.872+0000 7f378d656740 1 Processor -- start 2024-09-19T08:14:32.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f378d656740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 0x5560ff5d9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f378d656740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f378d656740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5d9bb0 0x5560ff5d6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f378d656740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5560ff5dc780 con 0x5560ff5cf2f0 2024-09-19T08:14:32.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f378d656740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5560ff4eff20 con 0x5560ff5d9bb0 2024-09-19T08:14:32.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f378d656740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5560ff5dc900 con 0x5560ff5ce9d0 2024-09-19T08:14:32.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f378d656740 10 client.0 _ll_register_callbacks cb 0x5560ff4b9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:14:32.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f3788818640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f3788818640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51278/0 (socket says 172.21.15.22:51278) 2024-09-19T08:14:32.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f3788818640 1 -- 172.21.15.22:0/2280440540 learned_addr learned my addr 172.21.15.22:0/2280440540 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:32.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f378981a640 1 --2- 172.21.15.22:0/2280440540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5d9bb0 0x5560ff5d6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f3789019640 1 --2- 172.21.15.22:0/2280440540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 0x5560ff5d9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f3788818640 1 -- 172.21.15.22:0/2280440540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 msgr2=0x5560ff5d9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:32.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f3788818640 1 --2- 172.21.15.22:0/2280440540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 0x5560ff5d9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f3788818640 1 -- 172.21.15.22:0/2280440540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5d9bb0 msgr2=0x5560ff5d6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:32.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f3788818640 1 --2- 172.21.15.22:0/2280440540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5d9bb0 0x5560ff5d6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f3788818640 1 -- 172.21.15.22:0/2280440540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560ff2a62b0 con 0x5560ff5cf2f0 2024-09-19T08:14:32.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f3789019640 1 --2- 172.21.15.22:0/2280440540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 0x5560ff5d9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:32.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f3788818640 1 --2- 172.21.15.22:0/2280440540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d67c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3778009fd0 tx=0x7f3778010fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92bd86e36708ab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:32.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f3768ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:14:32.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.873+0000 7f3768ff9640 1 -- 172.21.15.22:0/2280440540 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377809b1c0 con 0x5560ff5cf2f0 2024-09-19T08:14:32.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.874+0000 7f3768ff9640 1 -- 172.21.15.22:0/2280440540 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f37780cccb0 con 0x5560ff5cf2f0 2024-09-19T08:14:32.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.874+0000 7f378d656740 1 -- 172.21.15.22:0/2280440540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377c0317b0 con 0x5560ff5cf2f0 2024-09-19T08:14:32.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.874+0000 7f3768ff9640 1 -- 172.21.15.22:0/2280440540 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f377809afe0 con 0x5560ff5cf2f0 2024-09-19T08:14:32.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.874+0000 7f378d656740 10 client.14997 fetch_fsmap learned FSMap version 24 2024-09-19T08:14:32.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.874+0000 7f378d656740 1 -- 172.21.15.22:0/2280440540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5560ff5e80d0 con 0x5560ff5cf2f0 2024-09-19T08:14:32.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.874+0000 7f3768ff9640 1 -- 172.21.15.22:0/2280440540 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377c0317b0 con 0x5560ff5cf2f0 2024-09-19T08:14:32.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.874+0000 7f3768ff9640 1 -- 172.21.15.22:0/2280440540 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f3778097cf0 con 0x5560ff5cf2f0 2024-09-19T08:14:32.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.874+0000 7f378d656740 10 client.14997 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T08:14:32.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.874+0000 7f378d656740 10 client.14997 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T08:14:32.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.874+0000 7f378d656740 20 client.14997 populate_metadata read hostname 'smithi022' 2024-09-19T08:14:32.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.874+0000 7f378d656740 1 --2- 172.21.15.22:0/2280440540 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5560ff5ea710 0x5560ff5ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.874+0000 7f378d656740 4 client.14997 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:14:32.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.874+0000 7f378d656740 1 -- 172.21.15.22:0/2280440540 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560ff542600 con 0x5560ff5ea710 2024-09-19T08:14:32.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.874+0000 7f3789019640 1 --2- 172.21.15.22:0/2280440540 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5560ff5ea710 0x5560ff5ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.875+0000 7f3789019640 1 --2- 172.21.15.22:0/2280440540 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5560ff5ea710 0x5560ff5ebb80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43e47ed98cf9928 server_cookie=cd11f9b479fcee84 in_seq=0 out_seq=0 2024-09-19T08:14:32.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.875+0000 7f3768ff9640 10 client.14997 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:14:32.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f3768ff9640 1 -- 172.21.15.22:0/2280440540 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560ff542600 con 0x5560ff5ea710 2024-09-19T08:14:32.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f3768ff9640 10 client.14997 handle_command_reply: tid=0 2024-09-19T08:14:32.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f378d656740 1 client.14997 shutdown 2024-09-19T08:14:32.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f378d656740 1 -- 172.21.15.22:0/2280440540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 msgr2=0x5560ff5d67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:32.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f378d656740 1 --2- 172.21.15.22:0/2280440540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d67c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3778009fd0 tx=0x7f3778010fe0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f378d656740 1 -- 172.21.15.22:0/2280440540 shutdown_connections 2024-09-19T08:14:32.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f378d656740 1 --2- 172.21.15.22:0/2280440540 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5560ff5ea710 0x5560ff5ebb80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f378d656740 1 --2- 172.21.15.22:0/2280440540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5d9bb0 0x5560ff5d6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f378d656740 1 --2- 172.21.15.22:0/2280440540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d67c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f378d656740 1 --2- 172.21.15.22:0/2280440540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 0x5560ff5d9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f3768ff9640 0 client.14997 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:14:32.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f378d656740 1 -- 172.21.15.22:0/2280440540 >> 172.21.15.22:0/2280440540 conn(0x5560ff5caf60 msgr2=0x5560ff5d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:32.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f378d656740 1 -- 172.21.15.22:0/2280440540 shutdown_connections 2024-09-19T08:14:32.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f378d656740 1 -- 172.21.15.22:0/2280440540 wait complete. 2024-09-19T08:14:32.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.876+0000 7f378d656740 20 client.14997 trim_cache size 0 max 16384 2024-09-19T08:14:32.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.894+0000 7f378d656740 1 Processor -- start 2024-09-19T08:14:32.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.894+0000 7f378d656740 1 -- start start 2024-09-19T08:14:32.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f378d656740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5ce9d0 0x5560ff5ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f378d656740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f378d656740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5d9bb0 0x5560ff5d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f378d656740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5560ff5e9ec0 con 0x5560ff5ce9d0 2024-09-19T08:14:32.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f378d656740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5560ff5ea590 con 0x5560ff5cf2f0 2024-09-19T08:14:32.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f378d656740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5560ff5eac60 con 0x5560ff5d9bb0 2024-09-19T08:14:32.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f3789019640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5ce9d0 0x5560ff5ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f3789019640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5ce9d0 0x5560ff5ebb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51290/0 (socket says 172.21.15.22:51290) 2024-09-19T08:14:32.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f3789019640 1 -- 172.21.15.22:0/1372748276 learned_addr learned my addr 172.21.15.22:0/1372748276 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:32.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f3788818640 1 --2- 172.21.15.22:0/1372748276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f3789019640 1 -- 172.21.15.22:0/1372748276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5d9bb0 msgr2=0x5560ff5d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:32.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f3789019640 1 --2- 172.21.15.22:0/1372748276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5d9bb0 0x5560ff5d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f3789019640 1 -- 172.21.15.22:0/1372748276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5cf2f0 msgr2=0x5560ff5d6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:32.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f3789019640 1 --2- 172.21.15.22:0/1372748276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f3789019640 1 -- 172.21.15.22:0/1372748276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560ff5d9090 con 0x5560ff5ce9d0 2024-09-19T08:14:32.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f3788818640 1 --2- 172.21.15.22:0/1372748276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:14:32.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f3789019640 1 --2- 172.21.15.22:0/1372748276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5ce9d0 0x5560ff5ebb80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f377c02a960 tx=0x7f377c02a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26ab7d16eecbf62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:32.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f376affd640 1 -- 172.21.15.22:0/1372748276 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c035050 con 0x5560ff5ce9d0 2024-09-19T08:14:32.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f376affd640 1 -- 172.21.15.22:0/1372748276 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377c052070 con 0x5560ff5ce9d0 2024-09-19T08:14:32.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.895+0000 7f376affd640 1 -- 172.21.15.22:0/1372748276 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c03f040 con 0x5560ff5ce9d0 2024-09-19T08:14:32.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.896+0000 7f378d656740 1 -- 172.21.15.22:0/1372748276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5ce9d0 msgr2=0x5560ff5ebb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:32.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.896+0000 7f378d656740 1 --2- 172.21.15.22:0/1372748276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5ce9d0 0x5560ff5ebb80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f377c02a960 tx=0x7f377c02a440 comp rx=0 tx=0).stop 2024-09-19T08:14:32.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.896+0000 7f378d656740 1 -- 172.21.15.22:0/1372748276 shutdown_connections 2024-09-19T08:14:32.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.896+0000 7f378d656740 1 --2- 172.21.15.22:0/1372748276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5d9bb0 0x5560ff5d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.896+0000 7f378d656740 1 --2- 172.21.15.22:0/1372748276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5cf2f0 0x5560ff5d6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.896+0000 7f378d656740 1 --2- 172.21.15.22:0/1372748276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5ce9d0 0x5560ff5ebb80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.896+0000 7f378d656740 1 -- 172.21.15.22:0/1372748276 >> 172.21.15.22:0/1372748276 conn(0x5560ff5caf60 msgr2=0x5560ff5cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:32.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.896+0000 7f378d656740 1 -- 172.21.15.22:0/1372748276 shutdown_connections 2024-09-19T08:14:32.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.896+0000 7f378d656740 1 -- 172.21.15.22:0/1372748276 wait complete. 2024-09-19T08:14:32.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.896+0000 7f378d656740 1 -- start start 2024-09-19T08:14:32.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f378d656740 1 Processor -- start 2024-09-19T08:14:32.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f378d656740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 0x5560ff5eef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f378d656740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f378d656740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5d9bb0 0x5560ff652e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f378d656740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5560ff5eba00 con 0x5560ff5cf2f0 2024-09-19T08:14:32.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f378d656740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5560ff5eb330 con 0x5560ff5d9bb0 2024-09-19T08:14:32.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f378d656740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5560ff5ec0d0 con 0x5560ff5ce9d0 2024-09-19T08:14:32.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f3788818640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f378d656740 10 client.0 _ll_register_callbacks cb 0x7f37780d28d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:14:32.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f3788818640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5ef4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51296/0 (socket says 172.21.15.22:51296) 2024-09-19T08:14:32.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f3788818640 1 -- 172.21.15.22:0/1768674364 learned_addr learned my addr 172.21.15.22:0/1768674364 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:32.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f3788818640 1 -- 172.21.15.22:0/1768674364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 msgr2=0x5560ff5eef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:32.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f3788818640 1 --2- 172.21.15.22:0/1768674364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 0x5560ff5eef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f3788818640 1 -- 172.21.15.22:0/1768674364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5d9bb0 msgr2=0x5560ff652e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:32.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f3788818640 1 --2- 172.21.15.22:0/1768674364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5d9bb0 0x5560ff652e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.897+0000 7f3788818640 1 -- 172.21.15.22:0/1768674364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560ff653350 con 0x5560ff5cf2f0 2024-09-19T08:14:32.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f3788818640 1 --2- 172.21.15.22:0/1768674364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5ef4b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f377809b610 tx=0x7f3778099b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d5adce7db8a60e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:32.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f3768ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:14:32.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f3768ff9640 1 -- 172.21.15.22:0/1768674364 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377809b1c0 con 0x5560ff5cf2f0 2024-09-19T08:14:32.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f3768ff9640 1 -- 172.21.15.22:0/1768674364 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f37780afda0 con 0x5560ff5cf2f0 2024-09-19T08:14:32.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f378d656740 1 -- 172.21.15.22:0/1768674364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377c052070 con 0x5560ff5cf2f0 2024-09-19T08:14:32.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f3768ff9640 1 -- 172.21.15.22:0/1768674364 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3778021250 con 0x5560ff5cf2f0 2024-09-19T08:14:32.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f378d656740 10 client.15003 fetch_fsmap learned FSMap version 24 2024-09-19T08:14:32.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f378d656740 1 -- 172.21.15.22:0/1768674364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5560ff654460 con 0x5560ff5cf2f0 2024-09-19T08:14:32.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f3768ff9640 1 -- 172.21.15.22:0/1768674364 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377c052070 con 0x5560ff5cf2f0 2024-09-19T08:14:32.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f3768ff9640 1 -- 172.21.15.22:0/1768674364 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f37780ccab0 con 0x5560ff5cf2f0 2024-09-19T08:14:32.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f378d656740 10 client.15003 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T08:14:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f378d656740 10 client.15003 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T08:14:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f378d656740 20 client.15003 populate_metadata read hostname 'smithi022' 2024-09-19T08:14:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f378d656740 1 --2- 172.21.15.22:0/1768674364 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5560ff659100 0x5560ff657e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f378d656740 4 client.15003 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:14:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.898+0000 7f378d656740 1 -- 172.21.15.22:0/1768674364 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "status"}) -- 0x5560ff53cb10 con 0x5560ff659100 2024-09-19T08:14:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f3789019640 1 --2- 172.21.15.22:0/1768674364 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5560ff659100 0x5560ff657e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f3789019640 1 --2- 172.21.15.22:0/1768674364 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5560ff659100 0x5560ff657e10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48397670c8391eb3 server_cookie=56939026096c7c63 in_seq=0 out_seq=0 2024-09-19T08:14:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f3768ff9640 10 client.15003 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:14:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f3768ff9640 1 -- 172.21.15.22:0/1768674364 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5560ff53cb10 con 0x5560ff659100 2024-09-19T08:14:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f3768ff9640 10 client.15003 handle_command_reply: tid=0 2024-09-19T08:14:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f378d656740 1 client.15003 shutdown 2024-09-19T08:14:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f378d656740 1 -- 172.21.15.22:0/1768674364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 msgr2=0x5560ff5ef4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f378d656740 1 --2- 172.21.15.22:0/1768674364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5ef4b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f377809b610 tx=0x7f3778099b90 comp rx=0 tx=0).stop 2024-09-19T08:14:32.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f378d656740 1 -- 172.21.15.22:0/1768674364 shutdown_connections 2024-09-19T08:14:32.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f378d656740 1 --2- 172.21.15.22:0/1768674364 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5560ff659100 0x5560ff657e10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f378d656740 1 --2- 172.21.15.22:0/1768674364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5560ff5d9bb0 0x5560ff652e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f378d656740 1 --2- 172.21.15.22:0/1768674364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560ff5cf2f0 0x5560ff5ef4b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f378d656740 1 --2- 172.21.15.22:0/1768674364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ff5ce9d0 0x5560ff5eef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f378d656740 1 -- 172.21.15.22:0/1768674364 >> 172.21.15.22:0/1768674364 conn(0x5560ff5caf60 msgr2=0x5560ff5c97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:32.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.899+0000 7f3768ff9640 0 client.15003 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:14:32.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378d656740 1 -- 172.21.15.22:0/1768674364 shutdown_connections 2024-09-19T08:14:32.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378d656740 1 -- 172.21.15.22:0/1768674364 wait complete. 2024-09-19T08:14:32.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378d656740 20 client.15003 trim_cache size 0 max 16384 2024-09-19T08:14:32.912 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:14:32.913 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "c0ad60c6-765d-11ef-bceb-c7b262605968", 2024-09-19T08:14:32.913 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-19T08:14:32.913 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24481, 2024-09-19T08:14:32.913 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-19T08:14:32.913 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-19T08:14:32.913 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-19T08:14:32.913 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 90.639507530000003, 2024-09-19T08:14:32.913 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 24, 2024-09-19T08:14:32.913 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 85, 2024-09-19T08:14:32.913 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T08:14:32.913 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 111.04002551799999 2024-09-19T08:14:32.913 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:14:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378b27d640 1 -- 172.21.15.22:0/3187066720 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f376c0a3350 msgr2=0x7f376c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378b27d640 1 --2- 172.21.15.22:0/3187066720 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f376c0a3350 0x7f376c0a5810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3774007ed0 tx=0x7f3774007680 comp rx=0 tx=0).stop 2024-09-19T08:14:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378b27d640 1 -- 172.21.15.22:0/3187066720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3784103780 msgr2=0x7f3784198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378b27d640 1 --2- 172.21.15.22:0/3187066720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3784103780 0x7f3784198ad0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3778006fd0 tx=0x7f377800b8e0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378b27d640 1 -- 172.21.15.22:0/3187066720 shutdown_connections 2024-09-19T08:14:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378b27d640 1 --2- 172.21.15.22:0/3187066720 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f376c0a3350 0x7f376c0a5810 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378b27d640 1 --2- 172.21.15.22:0/3187066720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3784108300 0x7f378419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378b27d640 1 --2- 172.21.15.22:0/3187066720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3784103780 0x7f3784198ad0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378b27d640 1 --2- 172.21.15.22:0/3187066720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3784102dd0 0x7f3784198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378b27d640 1 -- 172.21.15.22:0/3187066720 >> 172.21.15.22:0/3187066720 conn(0x7f37840fc7d0 msgr2=0x7f3784105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378b27d640 1 -- 172.21.15.22:0/3187066720 shutdown_connections 2024-09-19T08:14:32.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:32.900+0000 7f378b27d640 1 -- 172.21.15.22:0/3187066720 wait complete. 2024-09-19T08:14:33.684 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T08:14:33.685 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T08:14:33.698 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-09-19T08:14:33.698 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T08:14:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:34.206 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:33 smithi022 ceph-mon[30963]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:34.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:33 smithi157 ceph-mon[35143]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:34.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:33 smithi064 ceph-mon[34900]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:34.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.414+0000 7f8124fc9640 1 -- 172.21.15.22:0/6396732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81201037c0 msgr2=0x7f812010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:34.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.414+0000 7f8124fc9640 1 --2- 172.21.15.22:0/6396732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81201037c0 0x7f812010a050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8118009ca0 tx=0x7f811802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:14:34.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.414+0000 7f8124fc9640 1 -- 172.21.15.22:0/6396732 shutdown_connections 2024-09-19T08:14:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.414+0000 7f8124fc9640 1 --2- 172.21.15.22:0/6396732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81201037c0 0x7f812010a050 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.414+0000 7f8124fc9640 1 --2- 172.21.15.22:0/6396732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8120102e00 0x7f8120103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.414+0000 7f8124fc9640 1 --2- 172.21.15.22:0/6396732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8120101c00 0x7f8120102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.414+0000 7f8124fc9640 1 -- 172.21.15.22:0/6396732 >> 172.21.15.22:0/6396732 conn(0x7f81200fd390 msgr2=0x7f81200ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.414+0000 7f8124fc9640 1 -- 172.21.15.22:0/6396732 shutdown_connections 2024-09-19T08:14:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.414+0000 7f8124fc9640 1 -- 172.21.15.22:0/6396732 wait complete. 2024-09-19T08:14:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.415+0000 7f8124fc9640 1 Processor -- start 2024-09-19T08:14:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.415+0000 7f8124fc9640 1 -- start start 2024-09-19T08:14:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.415+0000 7f8124fc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8120101c00 0x7f8120075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.415+0000 7f8124fc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8120102e00 0x7f81200745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.415+0000 7f8124fc9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81201037c0 0x7f8120074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.415+0000 7f8124fc9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f812010d8e0 con 0x7f81201037c0 2024-09-19T08:14:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.415+0000 7f8124fc9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f812010d760 con 0x7f8120102e00 2024-09-19T08:14:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.415+0000 7f8124fc9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f812010da60 con 0x7f8120101c00 2024-09-19T08:14:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.415+0000 7f811ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81201037c0 0x7f8120074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.415+0000 7f811ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81201037c0 0x7f8120074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51300/0 (socket says 172.21.15.22:51300) 2024-09-19T08:14:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.415+0000 7f811ed9d640 1 -- 172.21.15.22:0/1977802576 learned_addr learned my addr 172.21.15.22:0/1977802576 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.416+0000 7f811e59c640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8120101c00 0x7f8120075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.416+0000 7f811dd9b640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8120102e00 0x7f81200745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.416+0000 7f811ed9d640 1 -- 172.21.15.22:0/1977802576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8120101c00 msgr2=0x7f8120075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.416+0000 7f811ed9d640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8120101c00 0x7f8120075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.416+0000 7f811ed9d640 1 -- 172.21.15.22:0/1977802576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8120102e00 msgr2=0x7f81200745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.416+0000 7f811ed9d640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8120102e00 0x7f81200745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.416+0000 7f811ed9d640 1 -- 172.21.15.22:0/1977802576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81200753e0 con 0x7f81201037c0 2024-09-19T08:14:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.416+0000 7f811e59c640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8120101c00 0x7f8120075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:14:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.416+0000 7f811ed9d640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81201037c0 0x7f8120074b20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f81180098d0 tx=0x7f811802f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.416+0000 7f810f7fe640 1 -- 172.21.15.22:0/1977802576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811802fc30 con 0x7f81201037c0 2024-09-19T08:14:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.416+0000 7f8124fc9640 1 -- 172.21.15.22:0/1977802576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81201a2030 con 0x7f81201037c0 2024-09-19T08:14:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.416+0000 7f810f7fe640 1 -- 172.21.15.22:0/1977802576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8118003e50 con 0x7f81201037c0 2024-09-19T08:14:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.416+0000 7f810f7fe640 1 -- 172.21.15.22:0/1977802576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81180305c0 con 0x7f81201037c0 2024-09-19T08:14:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.416+0000 7f8124fc9640 1 -- 172.21.15.22:0/1977802576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81201a25c0 con 0x7f81201037c0 2024-09-19T08:14:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.417+0000 7f8124fc9640 1 -- 172.21.15.22:0/1977802576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8120109000 con 0x7f81201037c0 2024-09-19T08:14:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.418+0000 7f810f7fe640 1 -- 172.21.15.22:0/1977802576 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8118004b40 con 0x7f81201037c0 2024-09-19T08:14:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.419+0000 7f810f7fe640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f81080a32d0 0x7f81080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.419+0000 7f811e59c640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f81080a32d0 0x7f81080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.419+0000 7f810f7fe640 1 -- 172.21.15.22:0/1977802576 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f81180f0f60 con 0x7f81201037c0 2024-09-19T08:14:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.420+0000 7f811e59c640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f81080a32d0 0x7f81080a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8120102c60 tx=0x7f8110005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.422+0000 7f810f7fe640 1 -- 172.21.15.22:0/1977802576 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f81180bdc30 con 0x7f81201037c0 2024-09-19T08:14:34.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.614+0000 7f8124fc9640 1 -- 172.21.15.22:0/1977802576 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f8120062630 con 0x7f81080a32d0 2024-09-19T08:14:34.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.615+0000 7f810f7fe640 1 -- 172.21.15.22:0/1977802576 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f8120062630 con 0x7f81080a32d0 2024-09-19T08:14:34.615 INFO:teuthology.orchestra.run.smithi022.stdout:Backend: cephadm 2024-09-19T08:14:34.615 INFO:teuthology.orchestra.run.smithi022.stdout:Available: Yes 2024-09-19T08:14:34.615 INFO:teuthology.orchestra.run.smithi022.stdout:Paused: No 2024-09-19T08:14:34.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.617+0000 7f8124fc9640 1 -- 172.21.15.22:0/1977802576 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f81080a32d0 msgr2=0x7f81080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:34.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.617+0000 7f8124fc9640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f81080a32d0 0x7f81080a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8120102c60 tx=0x7f8110005d30 comp rx=0 tx=0).stop 2024-09-19T08:14:34.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.617+0000 7f8124fc9640 1 -- 172.21.15.22:0/1977802576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81201037c0 msgr2=0x7f8120074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:34.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.617+0000 7f8124fc9640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81201037c0 0x7f8120074b20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f81180098d0 tx=0x7f811802f840 comp rx=0 tx=0).stop 2024-09-19T08:14:34.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.617+0000 7f8124fc9640 1 -- 172.21.15.22:0/1977802576 shutdown_connections 2024-09-19T08:14:34.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.617+0000 7f8124fc9640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f81080a32d0 0x7f81080a5790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:34.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.617+0000 7f8124fc9640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81201037c0 0x7f8120074b20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:34.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.617+0000 7f8124fc9640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8120102e00 0x7f81200745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:34.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.617+0000 7f8124fc9640 1 --2- 172.21.15.22:0/1977802576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8120101c00 0x7f8120075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:34.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.617+0000 7f8124fc9640 1 -- 172.21.15.22:0/1977802576 >> 172.21.15.22:0/1977802576 conn(0x7f81200fd390 msgr2=0x7f81201061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:34.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.617+0000 7f8124fc9640 1 -- 172.21.15.22:0/1977802576 shutdown_connections 2024-09-19T08:14:34.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:34.617+0000 7f8124fc9640 1 -- 172.21.15.22:0/1977802576 wait complete. 2024-09-19T08:14:35.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:34 smithi157 ceph-mon[35143]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:35.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:34 smithi022 ceph-mon[30963]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:35.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:34 smithi064 ceph-mon[34900]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:35.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T08:14:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:36.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:35 smithi022 ceph-mon[30963]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T08:14:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.099+0000 7f6cc1644640 1 -- 172.21.15.22:0/3250027783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc101d10 msgr2=0x7f6cbc102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.099+0000 7f6cc1644640 1 --2- 172.21.15.22:0/3250027783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc101d10 0x7f6cbc102190 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009900 tx=0x7f6ca402a410 comp rx=0 tx=0).stop 2024-09-19T08:14:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.099+0000 7f6cc1644640 1 -- 172.21.15.22:0/3250027783 shutdown_connections 2024-09-19T08:14:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.099+0000 7f6cc1644640 1 --2- 172.21.15.22:0/3250027783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cbc1026d0 0x7f6cbc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.099+0000 7f6cc1644640 1 --2- 172.21.15.22:0/3250027783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc101d10 0x7f6cbc102190 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.099+0000 7f6cc1644640 1 --2- 172.21.15.22:0/3250027783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cbc100b10 0x7f6cbc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.099+0000 7f6cc1644640 1 -- 172.21.15.22:0/3250027783 >> 172.21.15.22:0/3250027783 conn(0x7f6cbc0fc2a0 msgr2=0x7f6cbc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.099+0000 7f6cc1644640 1 -- 172.21.15.22:0/3250027783 shutdown_connections 2024-09-19T08:14:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.099+0000 7f6cc1644640 1 -- 172.21.15.22:0/3250027783 wait complete. 2024-09-19T08:14:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.099+0000 7f6cc1644640 1 Processor -- start 2024-09-19T08:14:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.099+0000 7f6cc1644640 1 -- start start 2024-09-19T08:14:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cc1644640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc100b10 0x7f6cbc10b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cc1644640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cbc101d10 0x7f6cbc109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cc1644640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cbc1026d0 0x7f6cbc10a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cc1644640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc10d8c0 con 0x7f6cbc100b10 2024-09-19T08:14:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cc1644640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc10d740 con 0x7f6cbc1026d0 2024-09-19T08:14:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cc1644640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc10da40 con 0x7f6cbc101d10 2024-09-19T08:14:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cbaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc100b10 0x7f6cbc10b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cbaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc100b10 0x7f6cbc10b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51304/0 (socket says 172.21.15.22:51304) 2024-09-19T08:14:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cbaffd640 1 -- 172.21.15.22:0/3972218913 learned_addr learned my addr 172.21.15.22:0/3972218913 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cbb7fe640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cbc1026d0 0x7f6cbc10a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cb3fff640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cbc101d10 0x7f6cbc109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cbaffd640 1 -- 172.21.15.22:0/3972218913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cbc101d10 msgr2=0x7f6cbc109ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cbaffd640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cbc101d10 0x7f6cbc109ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cbaffd640 1 -- 172.21.15.22:0/3972218913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cbc1026d0 msgr2=0x7f6cbc10a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cbaffd640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cbc1026d0 0x7f6cbc10a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cbaffd640 1 -- 172.21.15.22:0/3972218913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc10a8e0 con 0x7f6cbc100b10 2024-09-19T08:14:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cbb7fe640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cbc1026d0 0x7f6cbc10a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cbaffd640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc100b10 0x7f6cbc10b490 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00d950 tx=0x7f6cac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cb8ff9640 1 -- 172.21.15.22:0/3972218913 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac017020 con 0x7f6cbc100b10 2024-09-19T08:14:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.100+0000 7f6cc1644640 1 -- 172.21.15.22:0/3972218913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cbc1a1fe0 con 0x7f6cbc100b10 2024-09-19T08:14:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.101+0000 7f6cb8ff9640 1 -- 172.21.15.22:0/3972218913 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cac009c30 con 0x7f6cbc100b10 2024-09-19T08:14:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.101+0000 7f6cb8ff9640 1 -- 172.21.15.22:0/3972218913 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00f6b0 con 0x7f6cbc100b10 2024-09-19T08:14:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.101+0000 7f6cc1644640 1 -- 172.21.15.22:0/3972218913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cbc1a2430 con 0x7f6cbc100b10 2024-09-19T08:14:36.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.102+0000 7f6cc1644640 1 -- 172.21.15.22:0/3972218913 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cbc107e40 con 0x7f6cbc100b10 2024-09-19T08:14:36.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.103+0000 7f6cb8ff9640 1 -- 172.21.15.22:0/3972218913 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6cac01a020 con 0x7f6cbc100b10 2024-09-19T08:14:36.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.104+0000 7f6cb8ff9640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6ca80a32d0 0x7f6ca80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:36.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.104+0000 7f6cb3fff640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6ca80a32d0 0x7f6ca80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:36.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.104+0000 7f6cb8ff9640 1 -- 172.21.15.22:0/3972218913 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6cac0cfb90 con 0x7f6cbc100b10 2024-09-19T08:14:36.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.104+0000 7f6cb3fff640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6ca80a32d0 0x7f6ca80a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4000c00 tx=0x7f6ca40315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:36.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.109+0000 7f6cb8ff9640 1 -- 172.21.15.22:0/3972218913 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6cac021020 con 0x7f6cbc100b10 2024-09-19T08:14:36.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:35 smithi157 ceph-mon[35143]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T08:14:36.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.292+0000 7f6cc1644640 1 -- 172.21.15.22:0/3972218913 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6cbc062630 con 0x7f6ca80a32d0 2024-09-19T08:14:36.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.306+0000 7f6cb8ff9640 1 -- 172.21.15.22:0/3972218913 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f6cbc062630 con 0x7f6ca80a32d0 2024-09-19T08:14:36.306 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T08:14:36.306 INFO:teuthology.orchestra.run.smithi022.stdout:mds.a smithi022 running (108s) 26s ago 108s 16.5M - 19.1.1-361-g2f599a46 d2baa855c067 df7590e175c9 2024-09-19T08:14:36.306 INFO:teuthology.orchestra.run.smithi022.stdout:mds.b smithi064 running (106s) 38s ago 105s 20.0M - 19.1.1-361-g2f599a46 d2baa855c067 702394fa3774 2024-09-19T08:14:36.306 INFO:teuthology.orchestra.run.smithi022.stdout:mds.c smithi157 running (119s) 38s ago 119s 19.5M - 19.1.1-361-g2f599a46 d2baa855c067 876264be95c9 2024-09-19T08:14:36.306 INFO:teuthology.orchestra.run.smithi022.stdout:mds.d smithi022 running (2m) 26s ago 2m 15.9M - 19.1.1-361-g2f599a46 d2baa855c067 9aa1f6ec73dc 2024-09-19T08:14:36.306 INFO:teuthology.orchestra.run.smithi022.stdout:mds.e smithi064 running (2m) 38s ago 2m 16.0M - 19.1.1-361-g2f599a46 d2baa855c067 7906b0b24f38 2024-09-19T08:14:36.306 INFO:teuthology.orchestra.run.smithi022.stdout:mds.f smithi157 running (2m) 38s ago 2m 16.9M - 19.1.1-361-g2f599a46 d2baa855c067 b84a29081160 2024-09-19T08:14:36.306 INFO:teuthology.orchestra.run.smithi022.stdout:mds.g smithi022 running (112s) 26s ago 111s 14.0M - 19.1.1-361-g2f599a46 d2baa855c067 5a88e318c867 2024-09-19T08:14:36.307 INFO:teuthology.orchestra.run.smithi022.stdout:mds.h smithi064 running (117s) 38s ago 116s 20.0M - 19.1.1-361-g2f599a46 d2baa855c067 af25c49e9d74 2024-09-19T08:14:36.307 INFO:teuthology.orchestra.run.smithi022.stdout:mds.i smithi157 running (114s) 38s ago 114s 26.2M - 19.1.1-361-g2f599a46 d2baa855c067 b42f9d647701 2024-09-19T08:14:36.307 INFO:teuthology.orchestra.run.smithi022.stdout:mds.j smithi022 running (2m) 26s ago 2m 14.3M - 19.1.1-361-g2f599a46 d2baa855c067 d180067f1c3f 2024-09-19T08:14:36.307 INFO:teuthology.orchestra.run.smithi022.stdout:mds.k smithi064 running (2m) 38s ago 2m 16.0M - 19.1.1-361-g2f599a46 d2baa855c067 32cd86275c8f 2024-09-19T08:14:36.307 INFO:teuthology.orchestra.run.smithi022.stdout:mds.l smithi157 running (2m) 38s ago 2m 16.7M - 19.1.1-361-g2f599a46 d2baa855c067 d09a90274cda 2024-09-19T08:14:36.307 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.x smithi022 *:9283,8765 running (9m) 26s ago 9m 509M - 19.1.1-361-g2f599a46 d2baa855c067 e278914c5998 2024-09-19T08:14:36.307 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.y smithi064 *:8443,8765 running (7m) 38s ago 7m 443M - 19.1.1-361-g2f599a46 d2baa855c067 faa40f033561 2024-09-19T08:14:36.307 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.z smithi157 *:8443,8765 running (7m) 38s ago 7m 441M - 19.1.1-361-g2f599a46 d2baa855c067 f6f15a448e8b 2024-09-19T08:14:36.307 INFO:teuthology.orchestra.run.smithi022.stdout:mon.a smithi022 running (9m) 26s ago 9m 51.2M 2048M 19.1.1-361-g2f599a46 d2baa855c067 3ded52544efc 2024-09-19T08:14:36.307 INFO:teuthology.orchestra.run.smithi022.stdout:mon.b smithi064 running (7m) 38s ago 7m 41.9M 2048M 19.1.1-361-g2f599a46 d2baa855c067 a7fa335b0bdd 2024-09-19T08:14:36.307 INFO:teuthology.orchestra.run.smithi022.stdout:mon.c smithi157 running (7m) 38s ago 7m 39.1M 2048M 19.1.1-361-g2f599a46 d2baa855c067 9ec3c64fd242 2024-09-19T08:14:36.307 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (7m) 26s ago 7m 43.5M 4273M 19.1.1-361-g2f599a46 d2baa855c067 a36e989e6a8b 2024-09-19T08:14:36.307 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi064 running (6m) 38s ago 6m 45.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 e95db63a18ad 2024-09-19T08:14:36.307 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi157 running (6m) 38s ago 6m 61.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 8022bd2f4d65 2024-09-19T08:14:36.308 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi022 running (5m) 26s ago 5m 56.3M 4273M 19.1.1-361-g2f599a46 d2baa855c067 e962e50610c1 2024-09-19T08:14:36.308 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi064 running (5m) 38s ago 5m 41.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 5850ed38c0ee 2024-09-19T08:14:36.308 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi157 running (5m) 38s ago 5m 55.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 44655cee5d6f 2024-09-19T08:14:36.308 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 smithi022 running (4m) 26s ago 4m 53.2M 4273M 19.1.1-361-g2f599a46 d2baa855c067 4ca125d64fd0 2024-09-19T08:14:36.308 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 smithi064 running (4m) 38s ago 4m 44.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 949b1a177285 2024-09-19T08:14:36.308 INFO:teuthology.orchestra.run.smithi022.stdout:osd.8 smithi157 running (3m) 38s ago 3m 54.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 1150a2ee5060 2024-09-19T08:14:36.308 INFO:teuthology.orchestra.run.smithi022.stdout:osd.9 smithi022 running (3m) 26s ago 3m 41.5M 4273M 19.1.1-361-g2f599a46 d2baa855c067 62df57551882 2024-09-19T08:14:36.308 INFO:teuthology.orchestra.run.smithi022.stdout:osd.10 smithi064 running (2m) 38s ago 2m 56.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 7bd01999cc7c 2024-09-19T08:14:36.308 INFO:teuthology.orchestra.run.smithi022.stdout:osd.11 smithi157 running (2m) 38s ago 2m 41.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 706a931c51e7 2024-09-19T08:14:36.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.308+0000 7f6cc1644640 1 -- 172.21.15.22:0/3972218913 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6ca80a32d0 msgr2=0x7f6ca80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:36.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.308+0000 7f6cc1644640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6ca80a32d0 0x7f6ca80a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4000c00 tx=0x7f6ca40315d0 comp rx=0 tx=0).stop 2024-09-19T08:14:36.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.308+0000 7f6cc1644640 1 -- 172.21.15.22:0/3972218913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc100b10 msgr2=0x7f6cbc10b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:36.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.308+0000 7f6cc1644640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc100b10 0x7f6cbc10b490 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00d950 tx=0x7f6cac00de10 comp rx=0 tx=0).stop 2024-09-19T08:14:36.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.308+0000 7f6cc1644640 1 -- 172.21.15.22:0/3972218913 shutdown_connections 2024-09-19T08:14:36.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.308+0000 7f6cc1644640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6ca80a32d0 0x7f6ca80a5790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:36.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.308+0000 7f6cc1644640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cbc1026d0 0x7f6cbc10a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:36.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.308+0000 7f6cc1644640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cbc101d10 0x7f6cbc109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:36.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.308+0000 7f6cc1644640 1 --2- 172.21.15.22:0/3972218913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc100b10 0x7f6cbc10b490 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:36.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.308+0000 7f6cc1644640 1 -- 172.21.15.22:0/3972218913 >> 172.21.15.22:0/3972218913 conn(0x7f6cbc0fc2a0 msgr2=0x7f6cbc105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:36.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.308+0000 7f6cc1644640 1 -- 172.21.15.22:0/3972218913 shutdown_connections 2024-09-19T08:14:36.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:36.308+0000 7f6cc1644640 1 -- 172.21.15.22:0/3972218913 wait complete. 2024-09-19T08:14:36.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:35 smithi064 ceph-mon[34900]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T08:14:36.949 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T08:14:37.205 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:37.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:36 smithi157 ceph-mon[35143]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:37.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:36 smithi022 ceph-mon[30963]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:37.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:36 smithi064 ceph-mon[34900]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:37.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.690+0000 7f316a2f7640 1 -- 172.21.15.22:0/2458761326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31641026c0 msgr2=0x7f3164104ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:37.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.690+0000 7f316a2f7640 1 --2- 172.21.15.22:0/2458761326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31641026c0 0x7f3164104ab0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3154009900 tx=0x7f315402a410 comp rx=0 tx=0).stop 2024-09-19T08:14:37.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.690+0000 7f316a2f7640 1 -- 172.21.15.22:0/2458761326 shutdown_connections 2024-09-19T08:14:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.690+0000 7f316a2f7640 1 --2- 172.21.15.22:0/2458761326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3164107920 0x7f3164109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.690+0000 7f316a2f7640 1 --2- 172.21.15.22:0/2458761326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3164104ff0 0x7f31641073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.690+0000 7f316a2f7640 1 --2- 172.21.15.22:0/2458761326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31641026c0 0x7f3164104ab0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.690+0000 7f316a2f7640 1 -- 172.21.15.22:0/2458761326 >> 172.21.15.22:0/2458761326 conn(0x7f31640fc280 msgr2=0x7f31640fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.691+0000 7f316a2f7640 1 -- 172.21.15.22:0/2458761326 shutdown_connections 2024-09-19T08:14:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.691+0000 7f316a2f7640 1 -- 172.21.15.22:0/2458761326 wait complete. 2024-09-19T08:14:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.691+0000 7f316a2f7640 1 Processor -- start 2024-09-19T08:14:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.691+0000 7f316a2f7640 1 -- start start 2024-09-19T08:14:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.691+0000 7f316a2f7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31641026c0 0x7f3164197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.691+0000 7f316a2f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3164104ff0 0x7f31641984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.691+0000 7f316a2f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3164107920 0x7f316419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.691+0000 7f316a2f7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f316410d8e0 con 0x7f3164107920 2024-09-19T08:14:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.691+0000 7f316a2f7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f316410d760 con 0x7f3164104ff0 2024-09-19T08:14:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.691+0000 7f316a2f7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f316410da60 con 0x7f31641026c0 2024-09-19T08:14:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f3168894640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3164107920 0x7f316419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f3168894640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3164107920 0x7f316419f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43734/0 (socket says 172.21.15.22:43734) 2024-09-19T08:14:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f3168894640 1 -- 172.21.15.22:0/2962521002 learned_addr learned my addr 172.21.15.22:0/2962521002 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f31637fe640 1 --2- 172.21.15.22:0/2962521002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3164104ff0 0x7f31641984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f3163fff640 1 --2- 172.21.15.22:0/2962521002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31641026c0 0x7f3164197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f3168894640 1 -- 172.21.15.22:0/2962521002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31641026c0 msgr2=0x7f3164197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f3168894640 1 --2- 172.21.15.22:0/2962521002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31641026c0 0x7f3164197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f3168894640 1 -- 172.21.15.22:0/2962521002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3164104ff0 msgr2=0x7f31641984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f3168894640 1 --2- 172.21.15.22:0/2962521002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3164104ff0 0x7f31641984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f3168894640 1 -- 172.21.15.22:0/2962521002 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316419fb90 con 0x7f3164107920 2024-09-19T08:14:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f3168894640 1 --2- 172.21.15.22:0/2962521002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3164107920 0x7f316419f520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f315c00b970 tx=0x7f315c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f31617fa640 1 -- 172.21.15.22:0/2962521002 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c019070 con 0x7f3164107920 2024-09-19T08:14:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f31617fa640 1 -- 172.21.15.22:0/2962521002 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f315c00cb40 con 0x7f3164107920 2024-09-19T08:14:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f316a2f7640 1 -- 172.21.15.22:0/2962521002 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316419fe80 con 0x7f3164107920 2024-09-19T08:14:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f31617fa640 1 -- 172.21.15.22:0/2962521002 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c007650 con 0x7f3164107920 2024-09-19T08:14:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.692+0000 7f316a2f7640 1 -- 172.21.15.22:0/2962521002 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31641a03b0 con 0x7f3164107920 2024-09-19T08:14:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.693+0000 7f316a2f7640 1 -- 172.21.15.22:0/2962521002 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31640fdd70 con 0x7f3164107920 2024-09-19T08:14:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.694+0000 7f31617fa640 1 -- 172.21.15.22:0/2962521002 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f315c01c070 con 0x7f3164107920 2024-09-19T08:14:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.695+0000 7f31617fa640 1 --2- 172.21.15.22:0/2962521002 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f314c0a3270 0x7f314c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.695+0000 7f3163fff640 1 --2- 172.21.15.22:0/2962521002 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f314c0a3270 0x7f314c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.695+0000 7f31617fa640 1 -- 172.21.15.22:0/2962521002 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f315c0cee30 con 0x7f3164107920 2024-09-19T08:14:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.696+0000 7f3163fff640 1 --2- 172.21.15.22:0/2962521002 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f314c0a3270 0x7f314c0a5730 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f31540314e0 tx=0x7f3154000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.700+0000 7f31617fa640 1 -- 172.21.15.22:0/2962521002 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f315c09b500 con 0x7f3164107920 2024-09-19T08:14:37.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.883+0000 7f316a2f7640 1 -- 172.21.15.22:0/2962521002 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f3164062630 con 0x7f314c0a3270 2024-09-19T08:14:37.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.889+0000 7f31617fa640 1 -- 172.21.15.22:0/2962521002 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f3164062630 con 0x7f314c0a3270 2024-09-19T08:14:37.889 INFO:teuthology.orchestra.run.smithi022.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T08:14:37.889 INFO:teuthology.orchestra.run.smithi022.stdout:mds.all 12/12 40s ago 2m smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-09-19T08:14:37.889 INFO:teuthology.orchestra.run.smithi022.stdout:mgr 3/3 40s ago 7m smithi022=x;smithi064=y;smithi157=z;count:3 2024-09-19T08:14:37.889 INFO:teuthology.orchestra.run.smithi022.stdout:mon 3/3 40s ago 8m smithi022:172.21.15.22=a;smithi064:172.21.15.64=b;smithi157:172.21.15.157=c;count:3 2024-09-19T08:14:37.889 INFO:teuthology.orchestra.run.smithi022.stdout:osd 12 40s ago - 2024-09-19T08:14:37.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.891+0000 7f316a2f7640 1 -- 172.21.15.22:0/2962521002 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f314c0a3270 msgr2=0x7f314c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:37.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.891+0000 7f316a2f7640 1 --2- 172.21.15.22:0/2962521002 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f314c0a3270 0x7f314c0a5730 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f31540314e0 tx=0x7f3154000bc0 comp rx=0 tx=0).stop 2024-09-19T08:14:37.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.891+0000 7f316a2f7640 1 -- 172.21.15.22:0/2962521002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3164107920 msgr2=0x7f316419f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:37.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.891+0000 7f316a2f7640 1 --2- 172.21.15.22:0/2962521002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3164107920 0x7f316419f520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f315c00b970 tx=0x7f315c00be30 comp rx=0 tx=0).stop 2024-09-19T08:14:37.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.891+0000 7f316a2f7640 1 -- 172.21.15.22:0/2962521002 shutdown_connections 2024-09-19T08:14:37.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.891+0000 7f316a2f7640 1 --2- 172.21.15.22:0/2962521002 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f314c0a3270 0x7f314c0a5730 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:37.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.891+0000 7f316a2f7640 1 --2- 172.21.15.22:0/2962521002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3164107920 0x7f316419f520 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:37.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.891+0000 7f316a2f7640 1 --2- 172.21.15.22:0/2962521002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3164104ff0 0x7f31641984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:37.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.891+0000 7f316a2f7640 1 --2- 172.21.15.22:0/2962521002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31641026c0 0x7f3164197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:37.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.891+0000 7f316a2f7640 1 -- 172.21.15.22:0/2962521002 >> 172.21.15.22:0/2962521002 conn(0x7f31640fc280 msgr2=0x7f3164102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:37.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.891+0000 7f316a2f7640 1 -- 172.21.15.22:0/2962521002 shutdown_connections 2024-09-19T08:14:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:37.891+0000 7f316a2f7640 1 -- 172.21.15.22:0/2962521002 wait complete. 2024-09-19T08:14:38.198 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:37 smithi022 ceph-mon[30963]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:14:38.198 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:37 smithi022 ceph-mon[30963]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T08:14:38.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:37 smithi157 ceph-mon[35143]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:14:38.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:37 smithi157 ceph-mon[35143]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T08:14:38.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:37 smithi064 ceph-mon[34900]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:14:38.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:37 smithi064 ceph-mon[34900]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T08:14:38.813 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T08:14:39.069 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:39.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:38 smithi022 ceph-mon[30963]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:39.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:38 smithi157 ceph-mon[35143]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:39.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:38 smithi064 ceph-mon[34900]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:39.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.487+0000 7f37b176d640 1 -- 172.21.15.22:0/2353476951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37ac102e00 msgr2=0x7f37ac103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:39.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.487+0000 7f37b176d640 1 --2- 172.21.15.22:0/2353476951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37ac102e00 0x7f37ac103280 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3794009900 tx=0x7f379402a410 comp rx=0 tx=0).stop 2024-09-19T08:14:39.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.487+0000 7f37b176d640 1 -- 172.21.15.22:0/2353476951 shutdown_connections 2024-09-19T08:14:39.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.487+0000 7f37b176d640 1 --2- 172.21.15.22:0/2353476951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac1037c0 0x7f37ac10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:39.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.487+0000 7f37b176d640 1 --2- 172.21.15.22:0/2353476951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37ac102e00 0x7f37ac103280 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:39.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.487+0000 7f37b176d640 1 --2- 172.21.15.22:0/2353476951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37ac101c00 0x7f37ac102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:39.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.487+0000 7f37b176d640 1 -- 172.21.15.22:0/2353476951 >> 172.21.15.22:0/2353476951 conn(0x7f37ac0fd390 msgr2=0x7f37ac0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:39.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.487+0000 7f37b176d640 1 -- 172.21.15.22:0/2353476951 shutdown_connections 2024-09-19T08:14:39.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.487+0000 7f37b176d640 1 -- 172.21.15.22:0/2353476951 wait complete. 2024-09-19T08:14:39.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.488+0000 7f37b176d640 1 Processor -- start 2024-09-19T08:14:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.488+0000 7f37b176d640 1 -- start start 2024-09-19T08:14:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.488+0000 7f37b176d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac101c00 0x7f37ac197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.488+0000 7f37b176d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37ac102e00 0x7f37ac1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.488+0000 7f37b176d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37ac1037c0 0x7f37ac19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.488+0000 7f37b176d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac10d8b0 con 0x7f37ac1037c0 2024-09-19T08:14:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.488+0000 7f37b176d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac10d730 con 0x7f37ac101c00 2024-09-19T08:14:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.488+0000 7f37b176d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac10da30 con 0x7f37ac102e00 2024-09-19T08:14:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.488+0000 7f37ab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37ac1037c0 0x7f37ac19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.488+0000 7f37ab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37ac1037c0 0x7f37ac19f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43758/0 (socket says 172.21.15.22:43758) 2024-09-19T08:14:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.488+0000 7f37ab7fe640 1 -- 172.21.15.22:0/3796996947 learned_addr learned my addr 172.21.15.22:0/3796996947 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.488+0000 7f37aaffd640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac101c00 0x7f37ac197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.488+0000 7f37a3fff640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37ac102e00 0x7f37ac1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.489+0000 7f37ab7fe640 1 -- 172.21.15.22:0/3796996947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37ac102e00 msgr2=0x7f37ac1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.489+0000 7f37ab7fe640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37ac102e00 0x7f37ac1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.489+0000 7f37ab7fe640 1 -- 172.21.15.22:0/3796996947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac101c00 msgr2=0x7f37ac197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.489+0000 7f37ab7fe640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac101c00 0x7f37ac197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.489+0000 7f37ab7fe640 1 -- 172.21.15.22:0/3796996947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac19fab0 con 0x7f37ac1037c0 2024-09-19T08:14:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.489+0000 7f37aaffd640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac101c00 0x7f37ac197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:14:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.489+0000 7f37ab7fe640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37ac1037c0 0x7f37ac19f440 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f37a4007f80 tx=0x7f37a400a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.489+0000 7f37a8ff9640 1 -- 172.21.15.22:0/3796996947 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a401d070 con 0x7f37ac1037c0 2024-09-19T08:14:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.489+0000 7f37a8ff9640 1 -- 172.21.15.22:0/3796996947 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a4016a00 con 0x7f37ac1037c0 2024-09-19T08:14:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.489+0000 7f37a8ff9640 1 -- 172.21.15.22:0/3796996947 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4018b50 con 0x7f37ac1037c0 2024-09-19T08:14:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.489+0000 7f37b176d640 1 -- 172.21.15.22:0/3796996947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ac074480 con 0x7f37ac1037c0 2024-09-19T08:14:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.489+0000 7f37b176d640 1 -- 172.21.15.22:0/3796996947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ac074940 con 0x7f37ac1037c0 2024-09-19T08:14:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.490+0000 7f37b176d640 1 -- 172.21.15.22:0/3796996947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ac108f30 con 0x7f37ac1037c0 2024-09-19T08:14:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.491+0000 7f37a8ff9640 1 -- 172.21.15.22:0/3796996947 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f37a4020070 con 0x7f37ac1037c0 2024-09-19T08:14:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.492+0000 7f37a8ff9640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f37980a3270 0x7f37980a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:39.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.492+0000 7f37aaffd640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f37980a3270 0x7f37980a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:39.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.492+0000 7f37a8ff9640 1 -- 172.21.15.22:0/3796996947 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f37a40d36e0 con 0x7f37ac1037c0 2024-09-19T08:14:39.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.493+0000 7f37aaffd640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f37980a3270 0x7f37980a5730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f37ac102c60 tx=0x7f379c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.496+0000 7f37a8ff9640 1 -- 172.21.15.22:0/3796996947 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f37a409fd70 con 0x7f37ac1037c0 2024-09-19T08:14:39.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.681+0000 7f37b176d640 1 -- 172.21.15.22:0/3796996947 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f37ac062630 con 0x7f37980a3270 2024-09-19T08:14:39.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.683+0000 7f37a8ff9640 1 -- 172.21.15.22:0/3796996947 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f37ac062630 con 0x7f37980a3270 2024-09-19T08:14:39.683 INFO:teuthology.orchestra.run.smithi022.stdout:HOST ADDR LABELS STATUS 2024-09-19T08:14:39.683 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 172.21.15.22 2024-09-19T08:14:39.683 INFO:teuthology.orchestra.run.smithi022.stdout:smithi064 172.21.15.64 2024-09-19T08:14:39.683 INFO:teuthology.orchestra.run.smithi022.stdout:smithi157 172.21.15.157 2024-09-19T08:14:39.683 INFO:teuthology.orchestra.run.smithi022.stdout:3 hosts in cluster 2024-09-19T08:14:39.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.685+0000 7f37b176d640 1 -- 172.21.15.22:0/3796996947 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f37980a3270 msgr2=0x7f37980a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:39.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.685+0000 7f37b176d640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f37980a3270 0x7f37980a5730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f37ac102c60 tx=0x7f379c00a2d0 comp rx=0 tx=0).stop 2024-09-19T08:14:39.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.685+0000 7f37b176d640 1 -- 172.21.15.22:0/3796996947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37ac1037c0 msgr2=0x7f37ac19f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:39.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.685+0000 7f37b176d640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37ac1037c0 0x7f37ac19f440 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f37a4007f80 tx=0x7f37a400a6e0 comp rx=0 tx=0).stop 2024-09-19T08:14:39.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.685+0000 7f37b176d640 1 -- 172.21.15.22:0/3796996947 shutdown_connections 2024-09-19T08:14:39.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.685+0000 7f37b176d640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f37980a3270 0x7f37980a5730 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:39.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.685+0000 7f37b176d640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37ac1037c0 0x7f37ac19f440 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:39.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.685+0000 7f37b176d640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37ac102e00 0x7f37ac1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:39.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.685+0000 7f37b176d640 1 --2- 172.21.15.22:0/3796996947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac101c00 0x7f37ac197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:39.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.685+0000 7f37b176d640 1 -- 172.21.15.22:0/3796996947 >> 172.21.15.22:0/3796996947 conn(0x7f37ac0fd390 msgr2=0x7f37ac106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:39.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.685+0000 7f37b176d640 1 -- 172.21.15.22:0/3796996947 shutdown_connections 2024-09-19T08:14:39.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:39.685+0000 7f37b176d640 1 -- 172.21.15.22:0/3796996947 wait complete. 2024-09-19T08:14:40.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:39 smithi022 ceph-mon[30963]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:40.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:39 smithi022 ceph-mon[30963]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:40.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:39 smithi157 ceph-mon[35143]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:40.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:39 smithi157 ceph-mon[35143]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:40.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:39 smithi064 ceph-mon[34900]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:14:40.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:39 smithi064 ceph-mon[34900]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:40.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T08:14:40.732 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:41.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.223+0000 7fdd5097f640 1 -- 172.21.15.22:0/64175405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd4c1037c0 msgr2=0x7fdd4c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:41.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.223+0000 7fdd5097f640 1 --2- 172.21.15.22:0/64175405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd4c1037c0 0x7fdd4c10a050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009ca0 tx=0x7fdd4402a8c0 comp rx=0 tx=0).stop 2024-09-19T08:14:41.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.223+0000 7fdd5097f640 1 -- 172.21.15.22:0/64175405 shutdown_connections 2024-09-19T08:14:41.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.223+0000 7fdd5097f640 1 --2- 172.21.15.22:0/64175405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd4c1037c0 0x7fdd4c10a050 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:41.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.223+0000 7fdd5097f640 1 --2- 172.21.15.22:0/64175405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd4c102e00 0x7fdd4c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:41.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.223+0000 7fdd5097f640 1 --2- 172.21.15.22:0/64175405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd4c101c00 0x7fdd4c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:41.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.223+0000 7fdd5097f640 1 -- 172.21.15.22:0/64175405 >> 172.21.15.22:0/64175405 conn(0x7fdd4c0fd390 msgr2=0x7fdd4c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:41.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.224+0000 7fdd5097f640 1 -- 172.21.15.22:0/64175405 shutdown_connections 2024-09-19T08:14:41.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.224+0000 7fdd5097f640 1 -- 172.21.15.22:0/64175405 wait complete. 2024-09-19T08:14:41.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.224+0000 7fdd5097f640 1 Processor -- start 2024-09-19T08:14:41.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.224+0000 7fdd5097f640 1 -- start start 2024-09-19T08:14:41.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd5097f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd4c101c00 0x7fdd4c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:41.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd5097f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd4c102e00 0x7fdd4c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:41.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd5097f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd4c1037c0 0x7fdd4c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:41.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd5097f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c10d880 con 0x7fdd4c101c00 2024-09-19T08:14:41.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd5097f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c10d700 con 0x7fdd4c1037c0 2024-09-19T08:14:41.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd5097f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c10da00 con 0x7fdd4c102e00 2024-09-19T08:14:41.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd4a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd4c101c00 0x7fdd4c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:41.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd4a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd4c101c00 0x7fdd4c075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43778/0 (socket says 172.21.15.22:43778) 2024-09-19T08:14:41.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd4a59c640 1 -- 172.21.15.22:0/2278074497 learned_addr learned my addr 172.21.15.22:0/2278074497 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:41.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd49d9b640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd4c102e00 0x7fdd4c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:41.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd4ad9d640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd4c1037c0 0x7fdd4c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:41.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd4a59c640 1 -- 172.21.15.22:0/2278074497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd4c102e00 msgr2=0x7fdd4c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:41.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd4a59c640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd4c102e00 0x7fdd4c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:41.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd4a59c640 1 -- 172.21.15.22:0/2278074497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd4c1037c0 msgr2=0x7fdd4c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:41.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd4a59c640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd4c1037c0 0x7fdd4c074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:41.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd4a59c640 1 -- 172.21.15.22:0/2278074497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4c0753e0 con 0x7fdd4c101c00 2024-09-19T08:14:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd4ad9d640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd4c1037c0 0x7fdd4c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd4a59c640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd4c101c00 0x7fdd4c075f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00d950 tx=0x7fdd3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd3b7fe640 1 -- 172.21.15.22:0/2278074497 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c017020 con 0x7fdd4c101c00 2024-09-19T08:14:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd5097f640 1 -- 172.21.15.22:0/2278074497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd4c1a2060 con 0x7fdd4c101c00 2024-09-19T08:14:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd3b7fe640 1 -- 172.21.15.22:0/2278074497 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd3c009c30 con 0x7fdd4c101c00 2024-09-19T08:14:41.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.225+0000 7fdd3b7fe640 1 -- 172.21.15.22:0/2278074497 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c00f720 con 0x7fdd4c101c00 2024-09-19T08:14:41.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.226+0000 7fdd5097f640 1 -- 172.21.15.22:0/2278074497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd4c1a2570 con 0x7fdd4c101c00 2024-09-19T08:14:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.226+0000 7fdd5097f640 1 -- 172.21.15.22:0/2278074497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd4c108f50 con 0x7fdd4c101c00 2024-09-19T08:14:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.228+0000 7fdd3b7fe640 1 -- 172.21.15.22:0/2278074497 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fdd3c01a020 con 0x7fdd4c101c00 2024-09-19T08:14:41.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.228+0000 7fdd3b7fe640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdd340a3240 0x7fdd340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:41.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.229+0000 7fdd49d9b640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdd340a3240 0x7fdd340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:41.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.229+0000 7fdd3b7fe640 1 -- 172.21.15.22:0/2278074497 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdd3c0cfc20 con 0x7fdd4c101c00 2024-09-19T08:14:41.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.229+0000 7fdd49d9b640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdd340a3240 0x7fdd340a5700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c075d10 tx=0x7fdd40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:41.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.232+0000 7fdd3b7fe640 1 -- 172.21.15.22:0/2278074497 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdd3c021020 con 0x7fdd4c101c00 2024-09-19T08:14:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.413+0000 7fdd5097f640 1 -- 172.21.15.22:0/2278074497 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fdd4c062630 con 0x7fdd340a3240 2024-09-19T08:14:41.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.415+0000 7fdd3b7fe640 1 -- 172.21.15.22:0/2278074497 <== mgr.14150 v2:172.21.15.22:6800/1866276413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fdd4c062630 con 0x7fdd340a3240 2024-09-19T08:14:41.415 INFO:teuthology.orchestra.run.smithi022.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T08:14:41.415 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402EC480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-19T08:14:41.415 INFO:teuthology.orchestra.run.smithi022.stdout:smithi064 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT534400C2400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T08:14:41.415 INFO:teuthology.orchestra.run.smithi022.stdout:smithi157 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002V480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-19T08:14:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.417+0000 7fdd5097f640 1 -- 172.21.15.22:0/2278074497 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdd340a3240 msgr2=0x7fdd340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.417+0000 7fdd5097f640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdd340a3240 0x7fdd340a5700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c075d10 tx=0x7fdd40005f90 comp rx=0 tx=0).stop 2024-09-19T08:14:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.417+0000 7fdd5097f640 1 -- 172.21.15.22:0/2278074497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd4c101c00 msgr2=0x7fdd4c075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.417+0000 7fdd5097f640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd4c101c00 0x7fdd4c075f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00d950 tx=0x7fdd3c00de10 comp rx=0 tx=0).stop 2024-09-19T08:14:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.417+0000 7fdd5097f640 1 -- 172.21.15.22:0/2278074497 shutdown_connections 2024-09-19T08:14:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.417+0000 7fdd5097f640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdd340a3240 0x7fdd340a5700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.417+0000 7fdd5097f640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd4c1037c0 0x7fdd4c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.417+0000 7fdd5097f640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd4c102e00 0x7fdd4c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.417+0000 7fdd5097f640 1 --2- 172.21.15.22:0/2278074497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd4c101c00 0x7fdd4c075f90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:41.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.417+0000 7fdd5097f640 1 -- 172.21.15.22:0/2278074497 >> 172.21.15.22:0/2278074497 conn(0x7fdd4c0fd390 msgr2=0x7fdd4c106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:41.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.417+0000 7fdd5097f640 1 -- 172.21.15.22:0/2278074497 shutdown_connections 2024-09-19T08:14:41.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:41.417+0000 7fdd5097f640 1 -- 172.21.15.22:0/2278074497 wait complete. 2024-09-19T08:14:42.121 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T08:14:42.130 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-09-19T08:14:42.131 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T08:14:42.154 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:41 smithi022 ceph-mon[30963]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T08:14:42.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:41 smithi022 ceph-mon[30963]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:42.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:41 smithi157 ceph-mon[35143]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T08:14:42.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:41 smithi157 ceph-mon[35143]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:42.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:41 smithi064 ceph-mon[34900]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T08:14:42.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:41 smithi064 ceph-mon[34900]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:14:42.391 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:42.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.863+0000 7f3e8814f640 1 -- 172.21.15.22:0/2359723602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80101c00 msgr2=0x7f3e80102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:42.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.863+0000 7f3e8814f640 1 --2- 172.21.15.22:0/2359723602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80101c00 0x7f3e80102000 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009900 tx=0x7f3e7c02a410 comp rx=0 tx=0).stop 2024-09-19T08:14:42.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.863+0000 7f3e8814f640 1 -- 172.21.15.22:0/2359723602 shutdown_connections 2024-09-19T08:14:42.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.863+0000 7f3e8814f640 1 --2- 172.21.15.22:0/2359723602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e801037c0 0x7f3e8010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:42.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.863+0000 7f3e8814f640 1 --2- 172.21.15.22:0/2359723602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80102e00 0x7f3e80103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:42.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.863+0000 7f3e8814f640 1 --2- 172.21.15.22:0/2359723602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80101c00 0x7f3e80102000 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:42.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.863+0000 7f3e8814f640 1 -- 172.21.15.22:0/2359723602 >> 172.21.15.22:0/2359723602 conn(0x7f3e800fd390 msgr2=0x7f3e800ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:42.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.863+0000 7f3e8814f640 1 -- 172.21.15.22:0/2359723602 shutdown_connections 2024-09-19T08:14:42.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.863+0000 7f3e8814f640 1 -- 172.21.15.22:0/2359723602 wait complete. 2024-09-19T08:14:42.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.864+0000 7f3e8814f640 1 Processor -- start 2024-09-19T08:14:42.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.864+0000 7f3e8814f640 1 -- start start 2024-09-19T08:14:42.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.864+0000 7f3e8814f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80101c00 0x7f3e80197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:42.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.864+0000 7f3e8814f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80102e00 0x7f3e80198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:42.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.864+0000 7f3e8814f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e801037c0 0x7f3e8019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:42.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.864+0000 7f3e8814f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8010d890 con 0x7f3e80102e00 2024-09-19T08:14:42.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.864+0000 7f3e8814f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8010d710 con 0x7f3e80101c00 2024-09-19T08:14:42.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.864+0000 7f3e8814f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8010da10 con 0x7f3e801037c0 2024-09-19T08:14:42.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e856ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80102e00 0x7f3e80198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e856ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80102e00 0x7f3e80198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43788/0 (socket says 172.21.15.22:43788) 2024-09-19T08:14:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e856ea640 1 -- 172.21.15.22:0/2199841734 learned_addr learned my addr 172.21.15.22:0/2199841734 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e866ec640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e801037c0 0x7f3e8019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e85eeb640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80101c00 0x7f3e80197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e856ea640 1 -- 172.21.15.22:0/2199841734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e801037c0 msgr2=0x7f3e8019f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e856ea640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e801037c0 0x7f3e8019f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e856ea640 1 -- 172.21.15.22:0/2199841734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80101c00 msgr2=0x7f3e80197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e856ea640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80101c00 0x7f3e80197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e856ea640 1 -- 172.21.15.22:0/2199841734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8019fb40 con 0x7f3e80102e00 2024-09-19T08:14:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e866ec640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e801037c0 0x7f3e8019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e85eeb640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80101c00 0x7f3e80197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e856ea640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80102e00 0x7f3e80198450 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000c890 tx=0x7f3e7000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e76ffd640 1 -- 172.21.15.22:0/2199841734 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70004220 con 0x7f3e80102e00 2024-09-19T08:14:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e76ffd640 1 -- 172.21.15.22:0/2199841734 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e700043c0 con 0x7f3e80102e00 2024-09-19T08:14:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e76ffd640 1 -- 172.21.15.22:0/2199841734 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e700049e0 con 0x7f3e80102e00 2024-09-19T08:14:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e8814f640 1 -- 172.21.15.22:0/2199841734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e8019fe30 con 0x7f3e80102e00 2024-09-19T08:14:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.865+0000 7f3e8814f640 1 -- 172.21.15.22:0/2199841734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e801a0330 con 0x7f3e80102e00 2024-09-19T08:14:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.866+0000 7f3e8814f640 1 -- 172.21.15.22:0/2199841734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e80109020 con 0x7f3e80102e00 2024-09-19T08:14:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.867+0000 7f3e76ffd640 1 -- 172.21.15.22:0/2199841734 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3e700079b0 con 0x7f3e80102e00 2024-09-19T08:14:42.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.868+0000 7f3e76ffd640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f3e6c0a32f0 0x7f3e6c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:42.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.868+0000 7f3e85eeb640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f3e6c0a32f0 0x7f3e6c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:42.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.868+0000 7f3e76ffd640 1 -- 172.21.15.22:0/2199841734 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3e700cf990 con 0x7f3e80102e00 2024-09-19T08:14:42.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.869+0000 7f3e85eeb640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f3e6c0a32f0 0x7f3e6c0a57b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c006010 tx=0x7f3e7c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:42.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:42.873+0000 7f3e76ffd640 1 -- 172.21.15.22:0/2199841734 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3e70014030 con 0x7f3e80102e00 2024-09-19T08:14:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:43.083+0000 7f3e8814f640 1 -- 172.21.15.22:0/2199841734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3e8002d7c0 con 0x7f3e80102e00 2024-09-19T08:14:43.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:43.084+0000 7f3e76ffd640 1 -- 172.21.15.22:0/2199841734 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3083 (secure 0 0 0) 0x7f3e7009c6b0 con 0x7f3e80102e00 2024-09-19T08:14:43.084 INFO:teuthology.orchestra.run.smithi022.stdout:e24 2024-09-19T08:14:43.084 INFO:teuthology.orchestra.run.smithi022.stdout:btime 2024-09-19T08:13:13:877693+0000 2024-09-19T08:14:43.085 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T08:14:43.085 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T08:14:43.085 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-09-19T08:14:43.085 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:43.085 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-09-19T08:14:43.085 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-09-19T08:14:43.085 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 24 2024-09-19T08:14:43.085 INFO:teuthology.orchestra.run.smithi022.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T08:14:43.085 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-09-19T08:13:01.503449+0000 2024-09-19T08:14:43.085 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-09-19T08:13:13.764216+0000 2024-09-19T08:14:43.085 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-09-19T08:14:43.085 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-09-19T08:14:43.085 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 300 2024-09-19T08:14:43.086 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-09-19T08:14:43.086 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-09-19T08:14:43.086 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-09-19T08:14:43.086 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-09-19T08:14:43.086 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-09-19T08:14:43.086 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 0 2024-09-19T08:14:43.086 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T08:14:43.086 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 3 2024-09-19T08:14:43.086 INFO:teuthology.orchestra.run.smithi022.stdout:in 0,1,2 2024-09-19T08:14:43.086 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=24481,1=24479,2=24460} 2024-09-19T08:14:43.086 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-09-19T08:14:43.086 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-09-19T08:14:43.087 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-09-19T08:14:43.087 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-09-19T08:14:43.087 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-09-19T08:14:43.087 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data disabled 2024-09-19T08:14:43.087 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-09-19T08:14:43.087 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-09-19T08:14:43.087 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 0 2024-09-19T08:14:43.087 INFO:teuthology.orchestra.run.smithi022.stdout:qdb_cluster leader: 24481 members: 24460,24479,24481 2024-09-19T08:14:43.087 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.i{0:24481} state up:active seq 7 addr [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:14:43.087 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.h{0:24455} state up:standby-replay seq 1 addr [v2:172.21.15.64:6836/2142121431,v1:172.21.15.64:6837/2142121431] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:14:43.087 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b{1:24479} state up:active seq 6 addr [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:14:43.087 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.f{1:24454} state up:standby-replay seq 1 addr [v2:172.21.15.157:6834/4158350679,v1:172.21.15.157:6835/4158350679] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:14:43.087 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c{2:24460} state up:active seq 10 addr [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:14:43.088 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.e{2:24428} state up:standby-replay seq 1 addr [v2:172.21.15.64:6834/3333087560,v1:172.21.15.64:6835/3333087560] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:14:43.088 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:43.088 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:43.088 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-09-19T08:14:43.088 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:43.088 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.j{-1:14511} state up:standby seq 1 addr [v2:172.21.15.22:6834/3714062414,v1:172.21.15.22:6835/3714062414] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:14:43.088 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d{-1:14523} state up:standby seq 1 addr [v2:172.21.15.22:6836/3668291316,v1:172.21.15.22:6837/3668291316] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:14:43.088 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.g{-1:14565} state up:standby seq 1 addr [v2:172.21.15.22:6838/3762277518,v1:172.21.15.22:6839/3762277518] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:14:43.088 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a{-1:14577} state up:standby seq 1 addr [v2:172.21.15.22:6840/1594373985,v1:172.21.15.22:6841/1594373985] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:14:43.088 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.k{-1:24404} state up:standby seq 1 addr [v2:172.21.15.64:6832/138036781,v1:172.21.15.64:6833/138036781] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:14:43.088 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.l{-1:24427} state up:standby seq 1 addr [v2:172.21.15.157:6832/2040635685,v1:172.21.15.157:6833/2040635685] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:14:43.089 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-09-19T08:14:43.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:43.086+0000 7f3e8814f640 1 -- 172.21.15.22:0/2199841734 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f3e6c0a32f0 msgr2=0x7f3e6c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:43.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:43.086+0000 7f3e8814f640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f3e6c0a32f0 0x7f3e6c0a57b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c006010 tx=0x7f3e7c000bc0 comp rx=0 tx=0).stop 2024-09-19T08:14:43.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:43.086+0000 7f3e8814f640 1 -- 172.21.15.22:0/2199841734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80102e00 msgr2=0x7f3e80198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:43.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:43.086+0000 7f3e8814f640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80102e00 0x7f3e80198450 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000c890 tx=0x7f3e7000cd50 comp rx=0 tx=0).stop 2024-09-19T08:14:43.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:43.086+0000 7f3e8814f640 1 -- 172.21.15.22:0/2199841734 shutdown_connections 2024-09-19T08:14:43.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:43.086+0000 7f3e8814f640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f3e6c0a32f0 0x7f3e6c0a57b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:43.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:43.086+0000 7f3e8814f640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e801037c0 0x7f3e8019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:43.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:43.086+0000 7f3e8814f640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80102e00 0x7f3e80198450 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:43.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:43.086+0000 7f3e8814f640 1 --2- 172.21.15.22:0/2199841734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80101c00 0x7f3e80197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:43.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:43.086+0000 7f3e8814f640 1 -- 172.21.15.22:0/2199841734 >> 172.21.15.22:0/2199841734 conn(0x7f3e800fd390 msgr2=0x7f3e801061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:43.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:43.086+0000 7f3e8814f640 1 -- 172.21.15.22:0/2199841734 shutdown_connections 2024-09-19T08:14:43.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:43.086+0000 7f3e8814f640 1 -- 172.21.15.22:0/2199841734 wait complete. 2024-09-19T08:14:43.738 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T08:14:44.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:43 smithi022 ceph-mon[30963]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T08:14:44.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:43 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2199841734' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T08:14:44.036 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:44.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:43 smithi157 ceph-mon[35143]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T08:14:44.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:43 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2199841734' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T08:14:44.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:43 smithi064 ceph-mon[34900]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T08:14:44.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:43 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2199841734' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T08:14:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.537+0000 7f99b4e11640 1 -- 172.21.15.22:0/509484329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b0101cd0 msgr2=0x7f99b0102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.537+0000 7f99b4e11640 1 --2- 172.21.15.22:0/509484329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b0101cd0 0x7f99b0102150 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9998009900 tx=0x7f999802a410 comp rx=0 tx=0).stop 2024-09-19T08:14:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.538+0000 7f99b4e11640 1 -- 172.21.15.22:0/509484329 shutdown_connections 2024-09-19T08:14:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.538+0000 7f99b4e11640 1 --2- 172.21.15.22:0/509484329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b0102690 0x7f99b0108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.538+0000 7f99b4e11640 1 --2- 172.21.15.22:0/509484329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b0101cd0 0x7f99b0102150 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.538+0000 7f99b4e11640 1 --2- 172.21.15.22:0/509484329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0100ad0 0x7f99b0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.538+0000 7f99b4e11640 1 -- 172.21.15.22:0/509484329 >> 172.21.15.22:0/509484329 conn(0x7f99b00fc280 msgr2=0x7f99b00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.538+0000 7f99b4e11640 1 -- 172.21.15.22:0/509484329 shutdown_connections 2024-09-19T08:14:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.538+0000 7f99b4e11640 1 -- 172.21.15.22:0/509484329 wait complete. 2024-09-19T08:14:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.538+0000 7f99b4e11640 1 Processor -- start 2024-09-19T08:14:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.538+0000 7f99b4e11640 1 -- start start 2024-09-19T08:14:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99b4e11640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b0100ad0 0x7f99b006aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99b4e11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0101cd0 0x7f99b019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99b4e11640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b0102690 0x7f99b019f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99b4e11640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99b010c7b0 con 0x7f99b0100ad0 2024-09-19T08:14:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99b4e11640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99b010c630 con 0x7f99b0101cd0 2024-09-19T08:14:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99b4e11640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99b010c930 con 0x7f99b0102690 2024-09-19T08:14:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99ae59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b0100ad0 0x7f99b006aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99ae59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b0100ad0 0x7f99b006aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43802/0 (socket says 172.21.15.22:43802) 2024-09-19T08:14:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99ae59c640 1 -- 172.21.15.22:0/3307444034 learned_addr learned my addr 172.21.15.22:0/3307444034 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99add9b640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0101cd0 0x7f99b019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99aed9d640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b0102690 0x7f99b019f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99ae59c640 1 -- 172.21.15.22:0/3307444034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b0102690 msgr2=0x7f99b019f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99ae59c640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b0102690 0x7f99b019f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99ae59c640 1 -- 172.21.15.22:0/3307444034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0101cd0 msgr2=0x7f99b019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99ae59c640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0101cd0 0x7f99b019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:44.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99ae59c640 1 -- 172.21.15.22:0/3307444034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b019fbe0 con 0x7f99b0100ad0 2024-09-19T08:14:44.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.539+0000 7f99aed9d640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b0102690 0x7f99b019f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:44.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.540+0000 7f99ae59c640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b0100ad0 0x7f99b006aad0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f99a000d770 tx=0x7f99a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:44.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.540+0000 7f99a77fe640 1 -- 172.21.15.22:0/3307444034 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0017070 con 0x7f99b0100ad0 2024-09-19T08:14:44.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.540+0000 7f99a77fe640 1 -- 172.21.15.22:0/3307444034 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a0009c30 con 0x7f99b0100ad0 2024-09-19T08:14:44.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.540+0000 7f99b4e11640 1 -- 172.21.15.22:0/3307444034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b019fe70 con 0x7f99b0100ad0 2024-09-19T08:14:44.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.540+0000 7f99a77fe640 1 -- 172.21.15.22:0/3307444034 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0003dd0 con 0x7f99b0100ad0 2024-09-19T08:14:44.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.540+0000 7f99b4e11640 1 -- 172.21.15.22:0/3307444034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b01a0340 con 0x7f99b0100ad0 2024-09-19T08:14:44.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.541+0000 7f99b4e11640 1 -- 172.21.15.22:0/3307444034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b0107ed0 con 0x7f99b0100ad0 2024-09-19T08:14:44.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.542+0000 7f99a77fe640 1 -- 172.21.15.22:0/3307444034 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f99a0003f70 con 0x7f99b0100ad0 2024-09-19T08:14:44.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.543+0000 7f99a77fe640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f999c0a3240 0x7f999c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:44.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.543+0000 7f99add9b640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f999c0a3240 0x7f999c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:44.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.543+0000 7f99a77fe640 1 -- 172.21.15.22:0/3307444034 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f99a00cf220 con 0x7f99b0100ad0 2024-09-19T08:14:44.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.544+0000 7f99add9b640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f999c0a3240 0x7f999c0a5700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f99980314e0 tx=0x7f9998000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:44.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.547+0000 7f99a77fe640 1 -- 172.21.15.22:0/3307444034 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f99a009bf40 con 0x7f99b0100ad0 2024-09-19T08:14:44.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.724+0000 7f99b4e11640 1 -- 172.21.15.22:0/3307444034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f99b0062ef0 con 0x7f99b0100ad0 2024-09-19T08:14:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.725+0000 7f99a77fe640 1 -- 172.21.15.22:0/3307444034 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) ==== 56+0+5294 (secure 0 0 0) 0x7f99a0009de0 con 0x7f99b0100ad0 2024-09-19T08:14:44.726 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 85 2024-09-19T08:14:44.726 INFO:teuthology.orchestra.run.smithi022.stdout:fsid c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:14:44.726 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-09-19T08:05:19.476952+0000 2024-09-19T08:14:44.726 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-09-19T08:14:07.725129+0000 2024-09-19T08:14:44.726 INFO:teuthology.orchestra.run.smithi022.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T08:14:44.726 INFO:teuthology.orchestra.run.smithi022.stdout:crush_version 26 2024-09-19T08:14:44.726 INFO:teuthology.orchestra.run.smithi022.stdout:full_ratio 0.95 2024-09-19T08:14:44.726 INFO:teuthology.orchestra.run.smithi022.stdout:backfillfull_ratio 0.9 2024-09-19T08:14:44.727 INFO:teuthology.orchestra.run.smithi022.stdout:nearfull_ratio 0.85 2024-09-19T08:14:44.727 INFO:teuthology.orchestra.run.smithi022.stdout:require_min_compat_client luminous 2024-09-19T08:14:44.727 INFO:teuthology.orchestra.run.smithi022.stdout:min_compat_client luminous 2024-09-19T08:14:44.727 INFO:teuthology.orchestra.run.smithi022.stdout:require_osd_release squid 2024-09-19T08:14:44.727 INFO:teuthology.orchestra.run.smithi022.stdout:stretch_mode_enabled false 2024-09-19T08:14:44.727 INFO:teuthology.orchestra.run.smithi022.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T08:14:44.727 INFO:teuthology.orchestra.run.smithi022.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-19T08:14:44.727 INFO:teuthology.orchestra.run.smithi022.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-19T08:14:44.727 INFO:teuthology.orchestra.run.smithi022.stdout:max_osd 12 2024-09-19T08:14:44.727 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6802/3442715307,v1:172.21.15.22:6803/3442715307] [v2:172.21.15.22:6804/3442715307,v1:172.21.15.22:6805/3442715307] exists,up 2ed67f2d-9b28-427a-b69f-676e7a2abcae 2024-09-19T08:14:44.727 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6800/2379700228,v1:172.21.15.64:6801/2379700228] [v2:172.21.15.64:6802/2379700228,v1:172.21.15.64:6803/2379700228] exists,up bff3e283-4ab1-49d7-86d8-45d361995122 2024-09-19T08:14:44.727 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6800/3363263867,v1:172.21.15.157:6801/3363263867] [v2:172.21.15.157:6802/3363263867,v1:172.21.15.157:6803/3363263867] exists,up 81421d08-7dc6-419b-a566-dee1e1e13b59 2024-09-19T08:14:44.728 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6810/4225229456,v1:172.21.15.22:6811/4225229456] [v2:172.21.15.22:6812/4225229456,v1:172.21.15.22:6813/4225229456] exists,up 34debfd0-3226-4d78-b9e6-e03dbc5190f0 2024-09-19T08:14:44.728 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 up in weight 1 up_from 33 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6808/1858090423,v1:172.21.15.64:6809/1858090423] [v2:172.21.15.64:6810/1858090423,v1:172.21.15.64:6811/1858090423] exists,up 1293332c-d772-4ca9-bf76-79e0b29451ee 2024-09-19T08:14:44.728 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 up in weight 1 up_from 38 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6808/2900701212,v1:172.21.15.157:6809/2900701212] [v2:172.21.15.157:6810/2900701212,v1:172.21.15.157:6811/2900701212] exists,up 240a6c42-89a2-41c1-948c-b09914b5722e 2024-09-19T08:14:44.728 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 up in weight 1 up_from 46 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6818/4264426242,v1:172.21.15.22:6819/4264426242] [v2:172.21.15.22:6820/4264426242,v1:172.21.15.22:6821/4264426242] exists,up 80199030-109e-41c4-bc1c-731755464770 2024-09-19T08:14:44.728 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 up in weight 1 up_from 51 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6816/3980224103,v1:172.21.15.64:6817/3980224103] [v2:172.21.15.64:6818/3980224103,v1:172.21.15.64:6819/3980224103] exists,up dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb 2024-09-19T08:14:44.728 INFO:teuthology.orchestra.run.smithi022.stdout:osd.8 up in weight 1 up_from 58 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6816/973678477,v1:172.21.15.157:6817/973678477] [v2:172.21.15.157:6818/973678477,v1:172.21.15.157:6819/973678477] exists,up 764b1e6e-48c9-49f9-8124-bc13bcba3a08 2024-09-19T08:14:44.728 INFO:teuthology.orchestra.run.smithi022.stdout:osd.9 up in weight 1 up_from 65 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6826/710033990,v1:172.21.15.22:6827/710033990] [v2:172.21.15.22:6828/710033990,v1:172.21.15.22:6829/710033990] exists,up 82914818-df47-4374-9e3e-2f71fc76e78b 2024-09-19T08:14:44.728 INFO:teuthology.orchestra.run.smithi022.stdout:osd.10 up in weight 1 up_from 70 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6824/373412752,v1:172.21.15.64:6825/373412752] [v2:172.21.15.64:6826/373412752,v1:172.21.15.64:6827/373412752] exists,up 36ae177d-c454-4cba-ba21-cbeee0f10dbb 2024-09-19T08:14:44.728 INFO:teuthology.orchestra.run.smithi022.stdout:osd.11 up in weight 1 up_from 77 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6824/1918889194,v1:172.21.15.157:6825/1918889194] [v2:172.21.15.157:6826/1918889194,v1:172.21.15.157:6827/1918889194] exists,up cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf 2024-09-19T08:14:44.728 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.f [8,11] 2024-09-19T08:14:44.728 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.10 [8,11] 2024-09-19T08:14:44.729 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.14 [2,4] 2024-09-19T08:14:44.729 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.16 [2,4] 2024-09-19T08:14:44.729 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.7 [3,9] 2024-09-19T08:14:44.729 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.1d [4,6] 2024-09-19T08:14:44.729 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.20 [5,9] 2024-09-19T08:14:44.729 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.2c [4,9] 2024-09-19T08:14:44.729 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.31 [4,8] 2024-09-19T08:14:44.729 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/3712426751 expires 2024-09-20T08:06:06.676475+0000 2024-09-19T08:14:44.729 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6800/4065414102 expires 2024-09-20T08:06:06.676475+0000 2024-09-19T08:14:44.729 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/2133059277 expires 2024-09-20T08:05:42.354701+0000 2024-09-19T08:14:44.729 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6801/4065414102 expires 2024-09-20T08:06:06.676475+0000 2024-09-19T08:14:44.729 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6801/2990808963 expires 2024-09-20T08:05:42.354701+0000 2024-09-19T08:14:44.729 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/2851750481 expires 2024-09-20T08:06:06.676475+0000 2024-09-19T08:14:44.730 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/3623845502 expires 2024-09-20T08:05:42.354701+0000 2024-09-19T08:14:44.730 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/384027596 expires 2024-09-20T08:06:06.676475+0000 2024-09-19T08:14:44.730 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6800/2990808963 expires 2024-09-20T08:05:42.354701+0000 2024-09-19T08:14:44.730 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/586163474 expires 2024-09-20T08:05:42.354701+0000 2024-09-19T08:14:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.729+0000 7f99b4e11640 1 -- 172.21.15.22:0/3307444034 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f999c0a3240 msgr2=0x7f999c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.729+0000 7f99b4e11640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f999c0a3240 0x7f999c0a5700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f99980314e0 tx=0x7f9998000bc0 comp rx=0 tx=0).stop 2024-09-19T08:14:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.729+0000 7f99b4e11640 1 -- 172.21.15.22:0/3307444034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b0100ad0 msgr2=0x7f99b006aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.729+0000 7f99b4e11640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b0100ad0 0x7f99b006aad0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f99a000d770 tx=0x7f99a000dc30 comp rx=0 tx=0).stop 2024-09-19T08:14:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.729+0000 7f99b4e11640 1 -- 172.21.15.22:0/3307444034 shutdown_connections 2024-09-19T08:14:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.729+0000 7f99b4e11640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f999c0a3240 0x7f999c0a5700 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.729+0000 7f99b4e11640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b0102690 0x7f99b019f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.729+0000 7f99b4e11640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0101cd0 0x7f99b019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.729+0000 7f99b4e11640 1 --2- 172.21.15.22:0/3307444034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b0100ad0 0x7f99b006aad0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.729+0000 7f99b4e11640 1 -- 172.21.15.22:0/3307444034 >> 172.21.15.22:0/3307444034 conn(0x7f99b00fc280 msgr2=0x7f99b01050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.729+0000 7f99b4e11640 1 -- 172.21.15.22:0/3307444034 shutdown_connections 2024-09-19T08:14:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:44.729+0000 7f99b4e11640 1 -- 172.21.15.22:0/3307444034 wait complete. 2024-09-19T08:14:45.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:44 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3307444034' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T08:14:45.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:44 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3307444034' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T08:14:45.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:44 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3307444034' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T08:14:45.389 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T08:14:45.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:14:45.654 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:46.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:45 smithi022 ceph-mon[30963]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T08:14:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.155+0000 7f7af2b5d640 1 -- 172.21.15.22:0/2388472255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec074eb0 msgr2=0x7f7aec112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.155+0000 7f7af2b5d640 1 --2- 172.21.15.22:0/2388472255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec074eb0 0x7f7aec112650 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009900 tx=0x7f7ad802a480 comp rx=0 tx=0).stop 2024-09-19T08:14:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.155+0000 7f7af2b5d640 1 -- 172.21.15.22:0/2388472255 shutdown_connections 2024-09-19T08:14:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.155+0000 7f7af2b5d640 1 --2- 172.21.15.22:0/2388472255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec112b90 0x7f7aec114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.155+0000 7f7af2b5d640 1 --2- 172.21.15.22:0/2388472255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec074eb0 0x7f7aec112650 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.155+0000 7f7af2b5d640 1 --2- 172.21.15.22:0/2388472255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aec074590 0x7f7aec074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.155+0000 7f7af2b5d640 1 -- 172.21.15.22:0/2388472255 >> 172.21.15.22:0/2388472255 conn(0x7f7aec0fd8c0 msgr2=0x7f7aec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.155+0000 7f7af2b5d640 1 -- 172.21.15.22:0/2388472255 shutdown_connections 2024-09-19T08:14:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.155+0000 7f7af2b5d640 1 -- 172.21.15.22:0/2388472255 wait complete. 2024-09-19T08:14:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.156+0000 7f7af2b5d640 1 Processor -- start 2024-09-19T08:14:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.156+0000 7f7af2b5d640 1 -- start start 2024-09-19T08:14:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.156+0000 7f7af2b5d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec074590 0x7f7aec1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.156+0000 7f7af2b5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aec074eb0 0x7f7aec1a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.156+0000 7f7af2b5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec112b90 0x7f7aec19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.156+0000 7f7af2b5d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec118540 con 0x7f7aec112b90 2024-09-19T08:14:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.156+0000 7f7af2b5d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1183c0 con 0x7f7aec074eb0 2024-09-19T08:14:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.156+0000 7f7af2b5d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1186c0 con 0x7f7aec074590 2024-09-19T08:14:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7af10fa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec112b90 0x7f7aec19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7af10fa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec112b90 0x7f7aec19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43824/0 (socket says 172.21.15.22:43824) 2024-09-19T08:14:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7af10fa640 1 -- 172.21.15.22:0/1130342192 learned_addr learned my addr 172.21.15.22:0/1130342192 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7aebfff640 1 --2- 172.21.15.22:0/1130342192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aec074eb0 0x7f7aec1a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7af08f9640 1 --2- 172.21.15.22:0/1130342192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec074590 0x7f7aec1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7af10fa640 1 -- 172.21.15.22:0/1130342192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec074590 msgr2=0x7f7aec1a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7af10fa640 1 --2- 172.21.15.22:0/1130342192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec074590 0x7f7aec1a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7af10fa640 1 -- 172.21.15.22:0/1130342192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aec074eb0 msgr2=0x7f7aec1a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7af10fa640 1 --2- 172.21.15.22:0/1130342192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aec074eb0 0x7f7aec1a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7af10fa640 1 -- 172.21.15.22:0/1130342192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec19efc0 con 0x7f7aec112b90 2024-09-19T08:14:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7af10fa640 1 --2- 172.21.15.22:0/1130342192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec112b90 0x7f7aec19e890 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400ccb0 tx=0x7f7ae400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7ae9ffb640 1 -- 172.21.15.22:0/1130342192 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4003970 con 0x7f7aec112b90 2024-09-19T08:14:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7ae9ffb640 1 -- 172.21.15.22:0/1130342192 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae4004030 con 0x7f7aec112b90 2024-09-19T08:14:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7af2b5d640 1 -- 172.21.15.22:0/1130342192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aec19f2b0 con 0x7f7aec112b90 2024-09-19T08:14:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7ae9ffb640 1 -- 172.21.15.22:0/1130342192 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae400f720 con 0x7f7aec112b90 2024-09-19T08:14:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.157+0000 7f7af2b5d640 1 -- 172.21.15.22:0/1130342192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aec1ab5a0 con 0x7f7aec112b90 2024-09-19T08:14:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.158+0000 7f7af2b5d640 1 -- 172.21.15.22:0/1130342192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aec074970 con 0x7f7aec112b90 2024-09-19T08:14:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.160+0000 7f7ae9ffb640 1 -- 172.21.15.22:0/1130342192 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7ae40041e0 con 0x7f7aec112b90 2024-09-19T08:14:46.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.160+0000 7f7ae9ffb640 1 --2- 172.21.15.22:0/1130342192 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7adc0a3240 0x7f7adc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:46.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.160+0000 7f7af08f9640 1 --2- 172.21.15.22:0/1130342192 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7adc0a3240 0x7f7adc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:46.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.160+0000 7f7ae9ffb640 1 -- 172.21.15.22:0/1130342192 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7ae40cf9d0 con 0x7f7aec112b90 2024-09-19T08:14:46.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.161+0000 7f7af08f9640 1 --2- 172.21.15.22:0/1130342192 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7adc0a3240 0x7f7adc0a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7aec19f9d0 tx=0x7f7ae0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:46.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.165+0000 7f7ae9ffb640 1 -- 172.21.15.22:0/1130342192 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7ae4014070 con 0x7f7aec112b90 2024-09-19T08:14:46.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:45 smithi157 ceph-mon[35143]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T08:14:46.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:45 smithi064 ceph-mon[34900]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T08:14:46.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.377+0000 7f7af2b5d640 1 -- 172.21.15.22:0/1130342192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7aec02d7c0 con 0x7f7aec112b90 2024-09-19T08:14:46.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.378+0000 7f7ae9ffb640 1 -- 172.21.15.22:0/1130342192 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11906 (secure 0 0 0) 0x7f7ae409c6b0 con 0x7f7aec112b90 2024-09-19T08:14:46.378 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:46.379 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"btime":"2024-09-19T08:13:13:877693+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:13:13.764216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:14:46.379 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-09-19T08:14:46.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.381+0000 7f7af2b5d640 1 -- 172.21.15.22:0/1130342192 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7adc0a3240 msgr2=0x7f7adc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:46.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.381+0000 7f7af2b5d640 1 --2- 172.21.15.22:0/1130342192 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7adc0a3240 0x7f7adc0a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7aec19f9d0 tx=0x7f7ae0005f90 comp rx=0 tx=0).stop 2024-09-19T08:14:46.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.381+0000 7f7af2b5d640 1 -- 172.21.15.22:0/1130342192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec112b90 msgr2=0x7f7aec19e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:46.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.381+0000 7f7af2b5d640 1 --2- 172.21.15.22:0/1130342192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec112b90 0x7f7aec19e890 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400ccb0 tx=0x7f7ae400a620 comp rx=0 tx=0).stop 2024-09-19T08:14:46.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.381+0000 7f7af2b5d640 1 -- 172.21.15.22:0/1130342192 shutdown_connections 2024-09-19T08:14:46.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.381+0000 7f7af2b5d640 1 --2- 172.21.15.22:0/1130342192 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7adc0a3240 0x7f7adc0a5700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:46.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.381+0000 7f7af2b5d640 1 --2- 172.21.15.22:0/1130342192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec112b90 0x7f7aec19e890 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:46.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.381+0000 7f7af2b5d640 1 --2- 172.21.15.22:0/1130342192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aec074eb0 0x7f7aec1a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:46.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.381+0000 7f7af2b5d640 1 --2- 172.21.15.22:0/1130342192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aec074590 0x7f7aec1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:46.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.381+0000 7f7af2b5d640 1 -- 172.21.15.22:0/1130342192 >> 172.21.15.22:0/1130342192 conn(0x7f7aec0fd8c0 msgr2=0x7f7aec0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:46.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.381+0000 7f7af2b5d640 1 -- 172.21.15.22:0/1130342192 shutdown_connections 2024-09-19T08:14:46.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:46.381+0000 7f7af2b5d640 1 -- 172.21.15.22:0/1130342192 wait complete. 2024-09-19T08:14:47.079 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:46 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1130342192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:47.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:14:47.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:46 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1130342192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:47.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:46 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1130342192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:47.374 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:47.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.871+0000 7ff222a7b640 1 -- 172.21.15.22:0/1230601193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21c100a60 msgr2=0x7ff21c100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:47.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.871+0000 7ff222a7b640 1 --2- 172.21.15.22:0/1230601193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21c100a60 0x7ff21c100ec0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff208009930 tx=0x7ff20802a420 comp rx=0 tx=0).stop 2024-09-19T08:14:47.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.871+0000 7ff222a7b640 1 -- 172.21.15.22:0/1230601193 shutdown_connections 2024-09-19T08:14:47.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.871+0000 7ff222a7b640 1 --2- 172.21.15.22:0/1230601193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff21c101400 0x7ff21c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:47.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.871+0000 7ff222a7b640 1 --2- 172.21.15.22:0/1230601193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21c100a60 0x7ff21c100ec0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:47.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.871+0000 7ff222a7b640 1 --2- 172.21.15.22:0/1230601193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff21c106a60 0x7ff21c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:47.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.871+0000 7ff222a7b640 1 -- 172.21.15.22:0/1230601193 >> 172.21.15.22:0/1230601193 conn(0x7ff21c0fc780 msgr2=0x7ff21c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:47.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.871+0000 7ff222a7b640 1 -- 172.21.15.22:0/1230601193 shutdown_connections 2024-09-19T08:14:47.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.871+0000 7ff222a7b640 1 -- 172.21.15.22:0/1230601193 wait complete. 2024-09-19T08:14:47.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.871+0000 7ff222a7b640 1 Processor -- start 2024-09-19T08:14:47.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.871+0000 7ff222a7b640 1 -- start start 2024-09-19T08:14:47.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff222a7b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21c100a60 0x7ff21c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:47.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff222a7b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff21c101400 0x7ff21c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:47.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff222a7b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff21c106a60 0x7ff21c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:47.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff222a7b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c110e80 con 0x7ff21c100a60 2024-09-19T08:14:47.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff222a7b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c110d00 con 0x7ff21c101400 2024-09-19T08:14:47.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff222a7b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c111000 con 0x7ff21c106a60 2024-09-19T08:14:47.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff220817640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21c100a60 0x7ff21c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:47.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff220817640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21c100a60 0x7ff21c1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34586/0 (socket says 172.21.15.22:34586) 2024-09-19T08:14:47.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff220817640 1 -- 172.21.15.22:0/3476129045 learned_addr learned my addr 172.21.15.22:0/3476129045 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:47.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff21bfff640 1 --2- 172.21.15.22:0/3476129045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff21c101400 0x7ff21c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:47.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff220817640 1 -- 172.21.15.22:0/3476129045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff21c106a60 msgr2=0x7ff21c19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:47.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff220817640 1 --2- 172.21.15.22:0/3476129045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff21c106a60 0x7ff21c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:47.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff220817640 1 -- 172.21.15.22:0/3476129045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff21c101400 msgr2=0x7ff21c198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:47.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff220817640 1 --2- 172.21.15.22:0/3476129045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff21c101400 0x7ff21c198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:47.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff220817640 1 -- 172.21.15.22:0/3476129045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c19d490 con 0x7ff21c100a60 2024-09-19T08:14:47.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff21bfff640 1 --2- 172.21.15.22:0/3476129045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff21c101400 0x7ff21c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:47.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff220817640 1 --2- 172.21.15.22:0/3476129045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21c100a60 0x7ff21c1984e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff21000c890 tx=0x7ff21000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:47.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff219ffb640 1 -- 172.21.15.22:0/3476129045 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210004220 con 0x7ff21c100a60 2024-09-19T08:14:47.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff222a7b640 1 -- 172.21.15.22:0/3476129045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21c19d780 con 0x7ff21c100a60 2024-09-19T08:14:47.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff219ffb640 1 -- 172.21.15.22:0/3476129045 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2100043c0 con 0x7ff21c100a60 2024-09-19T08:14:47.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.872+0000 7ff219ffb640 1 -- 172.21.15.22:0/3476129045 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210004a00 con 0x7ff21c100a60 2024-09-19T08:14:47.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.873+0000 7ff222a7b640 1 -- 172.21.15.22:0/3476129045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21c1a5060 con 0x7ff21c100a60 2024-09-19T08:14:47.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.874+0000 7ff222a7b640 1 -- 172.21.15.22:0/3476129045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff21c100ec0 con 0x7ff21c100a60 2024-09-19T08:14:47.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.875+0000 7ff219ffb640 1 -- 172.21.15.22:0/3476129045 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff2100079b0 con 0x7ff21c100a60 2024-09-19T08:14:47.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.875+0000 7ff219ffb640 1 --2- 172.21.15.22:0/3476129045 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff20c0a3240 0x7ff20c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:47.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.876+0000 7ff21bfff640 1 --2- 172.21.15.22:0/3476129045 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff20c0a3240 0x7ff20c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:47.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.876+0000 7ff219ffb640 1 -- 172.21.15.22:0/3476129045 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff2100cf700 con 0x7ff21c100a60 2024-09-19T08:14:47.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.876+0000 7ff21bfff640 1 --2- 172.21.15.22:0/3476129045 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff20c0a3240 0x7ff20c0a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff208000c00 tx=0x7ff208005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:47.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:47.880+0000 7ff219ffb640 1 -- 172.21.15.22:0/3476129045 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff210014030 con 0x7ff21c100a60 2024-09-19T08:14:48.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:48.094+0000 7ff222a7b640 1 -- 172.21.15.22:0/3476129045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff21c02d7c0 con 0x7ff21c100a60 2024-09-19T08:14:48.095 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:47 smithi022 ceph-mon[30963]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T08:14:48.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:48.095+0000 7ff219ffb640 1 -- 172.21.15.22:0/3476129045 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11906 (secure 0 0 0) 0x7ff21009c3e0 con 0x7ff21c100a60 2024-09-19T08:14:48.095 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:48.096 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"btime":"2024-09-19T08:13:13:877693+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:13:13.764216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:14:48.096 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-09-19T08:14:48.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:48.098+0000 7ff222a7b640 1 -- 172.21.15.22:0/3476129045 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff20c0a3240 msgr2=0x7ff20c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:48.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:48.098+0000 7ff222a7b640 1 --2- 172.21.15.22:0/3476129045 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff20c0a3240 0x7ff20c0a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff208000c00 tx=0x7ff208005a20 comp rx=0 tx=0).stop 2024-09-19T08:14:48.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:48.098+0000 7ff222a7b640 1 -- 172.21.15.22:0/3476129045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21c100a60 msgr2=0x7ff21c1984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:48.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:48.098+0000 7ff222a7b640 1 --2- 172.21.15.22:0/3476129045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21c100a60 0x7ff21c1984e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff21000c890 tx=0x7ff21000cd50 comp rx=0 tx=0).stop 2024-09-19T08:14:48.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:48.098+0000 7ff222a7b640 1 -- 172.21.15.22:0/3476129045 shutdown_connections 2024-09-19T08:14:48.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:48.098+0000 7ff222a7b640 1 --2- 172.21.15.22:0/3476129045 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7ff20c0a3240 0x7ff20c0a5700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:48.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:48.098+0000 7ff222a7b640 1 --2- 172.21.15.22:0/3476129045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff21c106a60 0x7ff21c19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:48.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:48.098+0000 7ff222a7b640 1 --2- 172.21.15.22:0/3476129045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff21c101400 0x7ff21c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:48.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:48.098+0000 7ff222a7b640 1 --2- 172.21.15.22:0/3476129045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff21c100a60 0x7ff21c1984e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:48.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:48.098+0000 7ff222a7b640 1 -- 172.21.15.22:0/3476129045 >> 172.21.15.22:0/3476129045 conn(0x7ff21c0fc780 msgr2=0x7ff21c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:48.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:48.099+0000 7ff222a7b640 1 -- 172.21.15.22:0/3476129045 shutdown_connections 2024-09-19T08:14:48.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:48.099+0000 7ff222a7b640 1 -- 172.21.15.22:0/3476129045 wait complete. 2024-09-19T08:14:48.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:47 smithi157 ceph-mon[35143]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T08:14:48.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:47 smithi064 ceph-mon[34900]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T08:14:48.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:14:49.182 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:49.238 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:48 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3476129045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:49.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:48 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3476129045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:49.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:48 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3476129045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:14:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.682+0000 7f4e1b05d640 1 -- 172.21.15.22:0/2001017057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e14101df0 msgr2=0x7f4e141021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.682+0000 7f4e1b05d640 1 --2- 172.21.15.22:0/2001017057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e14101df0 0x7f4e141021d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009900 tx=0x7f4e0402a410 comp rx=0 tx=0).stop 2024-09-19T08:14:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.682+0000 7f4e1b05d640 1 -- 172.21.15.22:0/2001017057 shutdown_connections 2024-09-19T08:14:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.682+0000 7f4e1b05d640 1 --2- 172.21.15.22:0/2001017057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e14106f20 0x7f4e1410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.682+0000 7f4e1b05d640 1 --2- 172.21.15.22:0/2001017057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e141027a0 0x7f4e141067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.682+0000 7f4e1b05d640 1 --2- 172.21.15.22:0/2001017057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e14101df0 0x7f4e141021d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.682+0000 7f4e1b05d640 1 -- 172.21.15.22:0/2001017057 >> 172.21.15.22:0/2001017057 conn(0x7f4e140fd8b0 msgr2=0x7f4e140ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.682+0000 7f4e1b05d640 1 -- 172.21.15.22:0/2001017057 shutdown_connections 2024-09-19T08:14:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.682+0000 7f4e1b05d640 1 -- 172.21.15.22:0/2001017057 wait complete. 2024-09-19T08:14:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.683+0000 7f4e1b05d640 1 Processor -- start 2024-09-19T08:14:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.683+0000 7f4e1b05d640 1 -- start start 2024-09-19T08:14:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.683+0000 7f4e1b05d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e14101df0 0x7f4e140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.683+0000 7f4e1b05d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e141027a0 0x7f4e14074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.683+0000 7f4e1b05d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e14106f20 0x7f4e14074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.683+0000 7f4e1b05d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4e14111fa0 con 0x7f4e141027a0 2024-09-19T08:14:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.683+0000 7f4e1b05d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4e14111e20 con 0x7f4e14106f20 2024-09-19T08:14:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.683+0000 7f4e1b05d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e14112120 con 0x7f4e14101df0 2024-09-19T08:14:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.683+0000 7f4e13fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e141027a0 0x7f4e14074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.683+0000 7f4e13fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e141027a0 0x7f4e14074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34600/0 (socket says 172.21.15.22:34600) 2024-09-19T08:14:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.683+0000 7f4e13fff640 1 -- 172.21.15.22:0/1060357971 learned_addr learned my addr 172.21.15.22:0/1060357971 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.683+0000 7f4e195fa640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e14106f20 0x7f4e14074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.683+0000 7f4e18df9640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e14101df0 0x7f4e140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:49.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.684+0000 7f4e13fff640 1 -- 172.21.15.22:0/1060357971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e14101df0 msgr2=0x7f4e140744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:49.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.684+0000 7f4e13fff640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e14101df0 0x7f4e140744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:49.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.684+0000 7f4e13fff640 1 -- 172.21.15.22:0/1060357971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e14106f20 msgr2=0x7f4e14074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:49.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.684+0000 7f4e13fff640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e14106f20 0x7f4e14074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:49.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.684+0000 7f4e13fff640 1 -- 172.21.15.22:0/1060357971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e141a6990 con 0x7f4e141027a0 2024-09-19T08:14:49.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.684+0000 7f4e18df9640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e14101df0 0x7f4e140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:49.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.684+0000 7f4e13fff640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e141027a0 0x7f4e14074a30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00b4a0 tx=0x7f4dfc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:49.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.684+0000 7f4e11ffb640 1 -- 172.21.15.22:0/1060357971 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc014070 con 0x7f4e141027a0 2024-09-19T08:14:49.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.684+0000 7f4e1b05d640 1 -- 172.21.15.22:0/1060357971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e141a6c80 con 0x7f4e141027a0 2024-09-19T08:14:49.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.684+0000 7f4e11ffb640 1 -- 172.21.15.22:0/1060357971 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dfc010af0 con 0x7f4e141027a0 2024-09-19T08:14:49.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.684+0000 7f4e11ffb640 1 -- 172.21.15.22:0/1060357971 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc017d10 con 0x7f4e141027a0 2024-09-19T08:14:49.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.684+0000 7f4e1b05d640 1 -- 172.21.15.22:0/1060357971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e141a71c0 con 0x7f4e141027a0 2024-09-19T08:14:49.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.685+0000 7f4e1b05d640 1 -- 172.21.15.22:0/1060357971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e141021d0 con 0x7f4e141027a0 2024-09-19T08:14:49.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.686+0000 7f4e11ffb640 1 -- 172.21.15.22:0/1060357971 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4dfc017070 con 0x7f4e141027a0 2024-09-19T08:14:49.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.687+0000 7f4e11ffb640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4e000a3240 0x7f4e000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:49.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.687+0000 7f4e18df9640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4e000a3240 0x7f4e000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:49.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.687+0000 7f4e11ffb640 1 -- 172.21.15.22:0/1060357971 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4dfc0ced20 con 0x7f4e141027a0 2024-09-19T08:14:49.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.687+0000 7f4e18df9640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4e000a3240 0x7f4e000a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4e04005ab0 tx=0x7f4e04005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:49.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.691+0000 7f4e11ffb640 1 -- 172.21.15.22:0/1060357971 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4dfc09ba40 con 0x7f4e141027a0 2024-09-19T08:14:49.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.866+0000 7f4e1b05d640 1 -- 172.21.15.22:0/1060357971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e1410a100 con 0x7f4e141027a0 2024-09-19T08:14:49.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.868+0000 7f4e11ffb640 1 -- 172.21.15.22:0/1060357971 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22724 (secure 0 0 0) 0x7f4dfc09b340 con 0x7f4e141027a0 2024-09-19T08:14:49.868 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:14:49.868 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":85,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","created":"2024-09-19T08:05:19.476952+0000","modified":"2024-09-19T08:14:07.725129+0000","last_up_change":"2024-09-19T08:12:10.339586+0000","last_in_change":"2024-09-19T08:11:47.653576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:08:25.175759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:12:57.608787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:12:59.824327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ed67f2d-9b28-427a-b69f-676e7a2abcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6803","nonce":3442715307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6805","nonce":3442715307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6809","nonce":3442715307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6807","nonce":3442715307}]},"public_addr":"172.21.15.22:6803/3442715307","cluster_addr":"172.21.15.22:6805/3442715307","heartbeat_back_addr":"172.21.15.22:6809/3442715307","heartbeat_front_addr":"172.21.15.22:6807/3442715307","state":["exists","up"]},{"osd":1,"uuid":"bff3e283-4ab1-49d7-86d8-45d361995122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6801","nonce":2379700228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6803","nonce":2379700228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6807","nonce":2379700228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6805","nonce":2379700228}]},"public_addr":"172.21.15.64:6801/2379700228","cluster_addr":"172.21.15.64:6803/2379700228","heartbeat_back_addr":"172.21.15.64:6807/2379700228","heartbeat_front_addr":"172.21.15.64:6805/2379700228","state":["exists","up"]},{"osd":2,"uuid":"81421d08-7dc6-419b-a566-dee1e1e13b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6801","nonce":3363263867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6803","nonce":3363263867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6807","nonce":3363263867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6805","nonce":3363263867}]},"public_addr":"172.21.15.157:6801/3363263867","cluster_addr":"172.21.15.157:6803/3363263867","heartbeat_back_addr":"172.21.15.157:6807/3363263867","heartbeat_front_addr":"172.21.15.157:6805/3363263867","state":["exists","up"]},{"osd":3,"uuid":"34debfd0-3226-4d78-b9e6-e03dbc5190f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6811","nonce":4225229456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6813","nonce":4225229456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6817","nonce":4225229456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6815","nonce":4225229456}]},"public_addr":"172.21.15.22:6811/4225229456","cluster_addr":"172.21.15.22:6813/4225229456","heartbeat_back_addr":"172.21.15.22:6817/4225229456","heartbeat_front_addr":"172.21.15.22:6815/4225229456","state":["exists","up"]},{"osd":4,"uuid":"1293332c-d772-4ca9-bf76-79e0b29451ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6809","nonce":1858090423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6811","nonce":1858090423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6815","nonce":1858090423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6813","nonce":1858090423}]},"public_addr":"172.21.15.64:6809/1858090423","cluster_addr":"172.21.15.64:6811/1858090423","heartbeat_back_addr":"172.21.15.64:6815/1858090423","heartbeat_front_addr":"172.21.15.64:6813/1858090423","state":["exists","up"]},{"osd":5,"uuid":"240a6c42-89a2-41c1-948c-b09914b5722e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6809","nonce":2900701212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2900701212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2900701212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2900701212}]},"public_addr":"172.21.15.157:6809/2900701212","cluster_addr":"172.21.15.157:6811/2900701212","heartbeat_back_addr":"172.21.15.157:6815/2900701212","heartbeat_front_addr":"172.21.15.157:6813/2900701212","state":["exists","up"]},{"osd":6,"uuid":"80199030-109e-41c4-bc1c-731755464770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6819","nonce":4264426242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6821","nonce":4264426242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6825","nonce":4264426242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6823","nonce":4264426242}]},"public_addr":"172.21.15.22:6819/4264426242","cluster_addr":"172.21.15.22:6821/4264426242","heartbeat_back_addr":"172.21.15.22:6825/4264426242","heartbeat_front_addr":"172.21.15.22:6823/4264426242","state":["exists","up"]},{"osd":7,"uuid":"dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6817","nonce":3980224103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6819","nonce":3980224103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6823","nonce":3980224103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6821","nonce":3980224103}]},"public_addr":"172.21.15.64:6817/3980224103","cluster_addr":"172.21.15.64:6819/3980224103","heartbeat_back_addr":"172.21.15.64:6823/3980224103","heartbeat_front_addr":"172.21.15.64:6821/3980224103","state":["exists","up"]},{"osd":8,"uuid":"764b1e6e-48c9-49f9-8124-bc13bcba3a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6817","nonce":973678477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6819","nonce":973678477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6823","nonce":973678477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6821","nonce":973678477}]},"public_addr":"172.21.15.157:6817/973678477","cluster_addr":"172.21.15.157:6819/973678477","heartbeat_back_addr":"172.21.15.157:6823/973678477","heartbeat_front_addr":"172.21.15.157:6821/973678477","state":["exists","up"]},{"osd":9,"uuid":"82914818-df47-4374-9e3e-2f71fc76e78b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6827","nonce":710033990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6829","nonce":710033990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6833","nonce":710033990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6831","nonce":710033990}]},"public_addr":"172.21.15.22:6827/710033990","cluster_addr":"172.21.15.22:6829/710033990","heartbeat_back_addr":"172.21.15.22:6833/710033990","heartbeat_front_addr":"172.21.15.22:6831/710033990","state":["exists","up"]},{"osd":10,"uuid":"36ae177d-c454-4cba-ba21-cbeee0f10dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6825","nonce":373412752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6827","nonce":373412752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6831","nonce":373412752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6829","nonce":373412752}]},"public_addr":"172.21.15.64:6825/373412752","cluster_addr":"172.21.15.64:6827/373412752","heartbeat_back_addr":"172.21.15.64:6831/373412752","heartbeat_front_addr":"172.21.15.64:6829/373412752","state":["exists","up"]},{"osd":11,"uuid":"cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6825","nonce":1918889194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6827","nonce":1918889194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6831","nonce":1918889194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6829","nonce":1918889194}]},"public_addr":"172.21.15.157:6825/1918889194","cluster_addr":"172.21.15.157:6827/1918889194","heartbeat_back_addr":"172.21.15.157:6831/1918889194","heartbeat_front_addr":"172.21.15.157:6829/1918889194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:31.378218+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:56.794167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:20.790267+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:47.039106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:11.483842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:38.151466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:03.553308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:27.865321+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:53.086530+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:18.946661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:43.697243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:12:08.936588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3712426751":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/2133059277":"2024-09-20T08:05:42.354701+0000","172.21.15.22:6801/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2851750481":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/3623845502":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/384027596":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/586163474":"2024-09-20T08:05:42.354701+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:14:49.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.870+0000 7f4e1b05d640 1 -- 172.21.15.22:0/1060357971 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4e000a3240 msgr2=0x7f4e000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:49.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.870+0000 7f4e1b05d640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4e000a3240 0x7f4e000a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4e04005ab0 tx=0x7f4e04005a20 comp rx=0 tx=0).stop 2024-09-19T08:14:49.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.870+0000 7f4e1b05d640 1 -- 172.21.15.22:0/1060357971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e141027a0 msgr2=0x7f4e14074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:49.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.870+0000 7f4e1b05d640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e141027a0 0x7f4e14074a30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00b4a0 tx=0x7f4dfc00b960 comp rx=0 tx=0).stop 2024-09-19T08:14:49.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.870+0000 7f4e1b05d640 1 -- 172.21.15.22:0/1060357971 shutdown_connections 2024-09-19T08:14:49.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.870+0000 7f4e1b05d640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f4e000a3240 0x7f4e000a5700 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:49.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.870+0000 7f4e1b05d640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e14106f20 0x7f4e14074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:49.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.870+0000 7f4e1b05d640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e141027a0 0x7f4e14074a30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:49.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.870+0000 7f4e1b05d640 1 --2- 172.21.15.22:0/1060357971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e14101df0 0x7f4e140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:49.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.870+0000 7f4e1b05d640 1 -- 172.21.15.22:0/1060357971 >> 172.21.15.22:0/1060357971 conn(0x7f4e140fd8b0 msgr2=0x7f4e140fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:49.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.870+0000 7f4e1b05d640 1 -- 172.21.15.22:0/1060357971 shutdown_connections 2024-09-19T08:14:49.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:49.870+0000 7f4e1b05d640 1 -- 172.21.15.22:0/1060357971 wait complete. 2024-09-19T08:14:50.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:49 smithi022 ceph-mon[30963]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T08:14:50.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:49 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1060357971' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:14:50.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:49 smithi157 ceph-mon[35143]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T08:14:50.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:49 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1060357971' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:14:50.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:49 smithi064 ceph-mon[34900]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T08:14:50.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:49 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1060357971' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:14:50.706 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T08:13:01.503449+0000', 'modified': '2024-09-19T08:13:13.764216+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24481, 'mds_1': 24479, 'mds_2': 24460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6835/3333087560', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 3333087560}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 3333087560}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6835/4158350679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 4158350679}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 4158350679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6837/2142121431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6836', 'nonce': 2142121431}, {'type': 'v1', 'addr': '172.21.15.64:6837', 'nonce': 2142121431}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.157:6837/892624399', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 892624399}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 892624399}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.64:6839/358484749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6838', 'nonce': 358484749}, {'type': 'v1', 'addr': '172.21.15.64:6839', 'nonce': 358484749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.157:6839/3802797238', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 3802797238}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 3802797238}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24481, 'qdb_cluster': [24460, 24479, 24481]} 2024-09-19T08:14:50.706 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T08:14:50.707 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T08:14:50.967 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.419+0000 7f6cec015640 1 -- 172.21.15.22:0/3012724201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ce4101df0 msgr2=0x7f6ce41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.419+0000 7f6cec015640 1 --2- 172.21.15.22:0/3012724201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ce4101df0 0x7f6ce41021d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009ca0 tx=0x7f6cd402a8c0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.419+0000 7f6cec015640 1 -- 172.21.15.22:0/3012724201 shutdown_connections 2024-09-19T08:14:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.419+0000 7f6cec015640 1 --2- 172.21.15.22:0/3012724201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ce4106f20 0x7f6ce410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.419+0000 7f6cec015640 1 --2- 172.21.15.22:0/3012724201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce41027a0 0x7f6ce41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.419+0000 7f6cec015640 1 --2- 172.21.15.22:0/3012724201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ce4101df0 0x7f6ce41021d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.419+0000 7f6cec015640 1 -- 172.21.15.22:0/3012724201 >> 172.21.15.22:0/3012724201 conn(0x7f6ce40fd8b0 msgr2=0x7f6ce40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.419+0000 7f6cec015640 1 -- 172.21.15.22:0/3012724201 shutdown_connections 2024-09-19T08:14:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.419+0000 7f6cec015640 1 -- 172.21.15.22:0/3012724201 wait complete. 2024-09-19T08:14:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cec015640 1 Processor -- start 2024-09-19T08:14:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cec015640 1 -- start start 2024-09-19T08:14:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cec015640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ce4101df0 0x7f6ce4198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cec015640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce41027a0 0x7f6ce4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cec015640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ce4106f20 0x7f6ce419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cec015640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce4111fa0 con 0x7f6ce4106f20 2024-09-19T08:14:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cec015640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce4111e20 con 0x7f6ce4101df0 2024-09-19T08:14:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cec015640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce4112120 con 0x7f6ce41027a0 2024-09-19T08:14:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cea5b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ce4106f20 0x7f6ce419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cea5b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ce4106f20 0x7f6ce419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34618/0 (socket says 172.21.15.22:34618) 2024-09-19T08:14:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cea5b2640 1 -- 172.21.15.22:0/2823002718 learned_addr learned my addr 172.21.15.22:0/2823002718 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6ce9db1640 1 --2- 172.21.15.22:0/2823002718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ce4101df0 0x7f6ce4198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6ce95b0640 1 --2- 172.21.15.22:0/2823002718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce41027a0 0x7f6ce4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cea5b2640 1 -- 172.21.15.22:0/2823002718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce41027a0 msgr2=0x7f6ce4198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cea5b2640 1 --2- 172.21.15.22:0/2823002718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce41027a0 0x7f6ce4198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cea5b2640 1 -- 172.21.15.22:0/2823002718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ce4101df0 msgr2=0x7f6ce4198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cea5b2640 1 --2- 172.21.15.22:0/2823002718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ce4101df0 0x7f6ce4198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.420+0000 7f6cea5b2640 1 -- 172.21.15.22:0/2823002718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce419d400 con 0x7f6ce4106f20 2024-09-19T08:14:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.421+0000 7f6cea5b2640 1 --2- 172.21.15.22:0/2823002718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ce4106f20 0x7f6ce419cd20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c970 tx=0x7f6cdc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.421+0000 7f6ce2ffd640 1 -- 172.21.15.22:0/2823002718 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc007bb0 con 0x7f6ce4106f20 2024-09-19T08:14:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.421+0000 7f6ce2ffd640 1 -- 172.21.15.22:0/2823002718 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cdc007d50 con 0x7f6ce4106f20 2024-09-19T08:14:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.421+0000 7f6cec015640 1 -- 172.21.15.22:0/2823002718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce419d6f0 con 0x7f6ce4106f20 2024-09-19T08:14:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.421+0000 7f6ce2ffd640 1 -- 172.21.15.22:0/2823002718 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc018800 con 0x7f6ce4106f20 2024-09-19T08:14:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.421+0000 7f6cec015640 1 -- 172.21.15.22:0/2823002718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce41a4fd0 con 0x7f6ce4106f20 2024-09-19T08:14:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.423+0000 7f6ce2ffd640 1 -- 172.21.15.22:0/2823002718 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f6cdc00fa50 con 0x7f6ce4106f20 2024-09-19T08:14:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.423+0000 7f6cee3ee740 1 Processor -- start 2024-09-19T08:14:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6cee3ee740 1 -- start start 2024-09-19T08:14:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6cee3ee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41e09d0 0x564fa41e0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6cee3ee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa41eb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6cee3ee740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41ebbb0 0x564fa41edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6cee3ee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564fa4147090 con 0x564fa41e12f0 2024-09-19T08:14:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6cee3ee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564fa40eea70 con 0x564fa41e09d0 2024-09-19T08:14:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6cee3ee740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564fa4143050 con 0x564fa41ebbb0 2024-09-19T08:14:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6ce9db1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa41eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6ce2ffd640 1 --2- 172.21.15.22:0/2823002718 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6ccc0a32c0 0x7f6ccc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6ce9db1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa41eb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34632/0 (socket says 172.21.15.22:34632) 2024-09-19T08:14:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6ce9db1640 1 -- 172.21.15.22:0/687016238 learned_addr learned my addr 172.21.15.22:0/687016238 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6cea5b2640 1 --2- 172.21.15.22:0/2823002718 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6ccc0a32c0 0x7f6ccc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6ce95b0640 1 --2- 172.21.15.22:0/687016238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41e09d0 0x564fa41e0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6ce95b0640 1 --2- 172.21.15.22:0/687016238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41ebbb0 0x564fa41edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6ce2ffd640 1 -- 172.21.15.22:0/2823002718 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6cdc0cf910 con 0x7f6ce4106f20 2024-09-19T08:14:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6ce9db1640 1 -- 172.21.15.22:0/687016238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41ebbb0 msgr2=0x564fa41edfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.424+0000 7f6ce9db1640 1 --2- 172.21.15.22:0/687016238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41ebbb0 0x564fa41edfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.425+0000 7f6ce9db1640 1 -- 172.21.15.22:0/687016238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41e09d0 msgr2=0x564fa41e0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.425+0000 7f6ce9db1640 1 --2- 172.21.15.22:0/687016238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41e09d0 0x564fa41e0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.425+0000 7f6ce9db1640 1 -- 172.21.15.22:0/687016238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa40fb130 con 0x564fa41e12f0 2024-09-19T08:14:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.425+0000 7f6ce9db1640 1 --2- 172.21.15.22:0/687016238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa41eb670 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009ca0 tx=0x7f6cd402f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fa91d7f5106807 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.425+0000 7f6cea5b2640 1 --2- 172.21.15.22:0/2823002718 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6ccc0a32c0 0x7f6ccc0a5780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc098350 tx=0x7f6cdc0c6cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.425+0000 7f6ccbfff640 1 -- 172.21.15.22:0/687016238 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4049070 con 0x564fa41e12f0 2024-09-19T08:14:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.425+0000 7f6ccbfff640 1 -- 172.21.15.22:0/687016238 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd4036040 con 0x564fa41e12f0 2024-09-19T08:14:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.425+0000 7f6ccbfff640 1 -- 172.21.15.22:0/687016238 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd402fd00 con 0x564fa41e12f0 2024-09-19T08:14:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.426+0000 7f6cee3ee740 1 -- 172.21.15.22:0/687016238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 msgr2=0x564fa41eb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.426+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/687016238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa41eb670 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009ca0 tx=0x7f6cd402f450 comp rx=0 tx=0).stop 2024-09-19T08:14:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.426+0000 7f6cee3ee740 1 -- 172.21.15.22:0/687016238 shutdown_connections 2024-09-19T08:14:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.426+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/687016238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41ebbb0 0x564fa41edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.426+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/687016238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa41eb670 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.426+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/687016238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41e09d0 0x564fa41e0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.426+0000 7f6cee3ee740 1 -- 172.21.15.22:0/687016238 >> 172.21.15.22:0/687016238 conn(0x564fa41dcf60 msgr2=0x564fa41dd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.426+0000 7f6cee3ee740 1 -- 172.21.15.22:0/687016238 shutdown_connections 2024-09-19T08:14:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.426+0000 7f6cee3ee740 1 -- 172.21.15.22:0/687016238 wait complete. 2024-09-19T08:14:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.426+0000 7f6cee3ee740 1 -- start start 2024-09-19T08:14:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.426+0000 7f6cee3ee740 1 Processor -- start 2024-09-19T08:14:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6cee3ee740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41e09d0 0x564fa41eb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6cee3ee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa41e87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6cee3ee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41ebbb0 0x564fa41e8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6cee3ee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564fa41ee600 con 0x564fa41e12f0 2024-09-19T08:14:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6cee3ee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564fa4101f20 con 0x564fa41ebbb0 2024-09-19T08:14:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6cee3ee740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564fa41ee780 con 0x564fa41e09d0 2024-09-19T08:14:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6cee3ee740 10 client.0 _ll_register_callbacks cb 0x564fa40cb030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:14:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6ce9db1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa41e87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6ce9db1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa41e87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34636/0 (socket says 172.21.15.22:34636) 2024-09-19T08:14:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6ce9db1640 1 -- 172.21.15.22:0/1193036878 learned_addr learned my addr 172.21.15.22:0/1193036878 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6ce9db1640 1 -- 172.21.15.22:0/1193036878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41e09d0 msgr2=0x564fa41eb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6ce9db1640 1 --2- 172.21.15.22:0/1193036878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41e09d0 0x564fa41eb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6ce9db1640 1 -- 172.21.15.22:0/1193036878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41ebbb0 msgr2=0x564fa41e8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6ce9db1640 1 --2- 172.21.15.22:0/1193036878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41ebbb0 0x564fa41e8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6ce9db1640 1 -- 172.21.15.22:0/1193036878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa3eb82b0 con 0x564fa41e12f0 2024-09-19T08:14:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6ce9db1640 1 --2- 172.21.15.22:0/1193036878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa41e87c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4005c50 tx=0x7f6cd402afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6344649cb17ac03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6cc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:14:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6cc97fa640 1 -- 172.21.15.22:0/1193036878 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4054060 con 0x564fa41e12f0 2024-09-19T08:14:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6cc97fa640 1 -- 172.21.15.22:0/1193036878 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6cd4031450 con 0x564fa41e12f0 2024-09-19T08:14:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.427+0000 7f6cee3ee740 1 -- 172.21.15.22:0/1193036878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cd4036040 con 0x564fa41e12f0 2024-09-19T08:14:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.428+0000 7f6cc97fa640 1 -- 172.21.15.22:0/1193036878 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6cd4049070 con 0x564fa41e12f0 2024-09-19T08:14:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.428+0000 7f6cee3ee740 10 client.15075 fetch_fsmap learned FSMap version 24 2024-09-19T08:14:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.428+0000 7f6cee3ee740 1 -- 172.21.15.22:0/1193036878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564fa41ee600 con 0x564fa41e12f0 2024-09-19T08:14:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.428+0000 7f6cc97fa640 1 -- 172.21.15.22:0/1193036878 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd4036040 con 0x564fa41e12f0 2024-09-19T08:14:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.428+0000 7f6cc97fa640 1 -- 172.21.15.22:0/1193036878 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6cd4059070 con 0x564fa41e12f0 2024-09-19T08:14:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.428+0000 7f6cee3ee740 10 client.15075 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T08:14:51.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.428+0000 7f6cee3ee740 10 client.15075 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:14:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.428+0000 7f6cee3ee740 20 client.15075 populate_metadata read hostname 'smithi022' 2024-09-19T08:14:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.428+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/1193036878 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x564fa41fc290 0x564fa41fd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.428+0000 7f6cee3ee740 4 client.15075 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:14:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.428+0000 7f6cee3ee740 1 -- 172.21.15.22:0/1193036878 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fa4154600 con 0x564fa41fc290 2024-09-19T08:14:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.429+0000 7f6ce95b0640 1 --2- 172.21.15.22:0/1193036878 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x564fa41fc290 0x564fa41fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.429+0000 7f6ce95b0640 1 --2- 172.21.15.22:0/1193036878 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x564fa41fc290 0x564fa41fd700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=421bc90f53ec15be server_cookie=d0579d988efbfdc7 in_seq=0 out_seq=0 2024-09-19T08:14:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.429+0000 7f6cc97fa640 10 client.15075 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:14:51.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cc97fa640 1 -- 172.21.15.22:0/1193036878 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fa4154600 con 0x564fa41fc290 2024-09-19T08:14:51.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cc97fa640 10 client.15075 handle_command_reply: tid=0 2024-09-19T08:14:51.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cee3ee740 1 client.15075 shutdown 2024-09-19T08:14:51.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cee3ee740 1 -- 172.21.15.22:0/1193036878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 msgr2=0x564fa41e87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:51.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/1193036878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa41e87c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4005c50 tx=0x7f6cd402afc0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cee3ee740 1 -- 172.21.15.22:0/1193036878 shutdown_connections 2024-09-19T08:14:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/1193036878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41ebbb0 0x564fa41e8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/1193036878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa41e87c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/1193036878 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x564fa41fc290 0x564fa41fd700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/1193036878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41e09d0 0x564fa41eb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cee3ee740 1 -- 172.21.15.22:0/1193036878 >> 172.21.15.22:0/1193036878 conn(0x564fa41dcf60 msgr2=0x564fa41e2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cc97fa640 0 client.15075 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:14:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cee3ee740 1 -- 172.21.15.22:0/1193036878 shutdown_connections 2024-09-19T08:14:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cee3ee740 1 -- 172.21.15.22:0/1193036878 wait complete. 2024-09-19T08:14:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.430+0000 7f6cee3ee740 20 client.15075 trim_cache size 0 max 16384 2024-09-19T08:14:51.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6cee3ee740 1 Processor -- start 2024-09-19T08:14:51.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6cee3ee740 1 -- start start 2024-09-19T08:14:51.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6cee3ee740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41e09d0 0x564fa41fd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6cee3ee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41e12f0 0x564fa41dd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6cee3ee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41ebbb0 0x564fa41dbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6cee3ee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564fa41fba40 con 0x564fa41ebbb0 2024-09-19T08:14:51.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6cee3ee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564fa41fc110 con 0x564fa41e12f0 2024-09-19T08:14:51.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6cee3ee740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564fa41fc7e0 con 0x564fa41e09d0 2024-09-19T08:14:51.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6ce95b0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41ebbb0 0x564fa41dbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6ce95b0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41ebbb0 0x564fa41dbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34644/0 (socket says 172.21.15.22:34644) 2024-09-19T08:14:51.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6ce95b0640 1 -- 172.21.15.22:0/124800187 learned_addr learned my addr 172.21.15.22:0/124800187 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:51.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6ce9db1640 1 --2- 172.21.15.22:0/124800187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41e12f0 0x564fa41dd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6ce95b0640 1 -- 172.21.15.22:0/124800187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41e09d0 msgr2=0x564fa41fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:51.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6ce95b0640 1 --2- 172.21.15.22:0/124800187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41e09d0 0x564fa41fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.448+0000 7f6ce95b0640 1 -- 172.21.15.22:0/124800187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41e12f0 msgr2=0x564fa41dd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:51.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.449+0000 7f6ce95b0640 1 --2- 172.21.15.22:0/124800187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41e12f0 0x564fa41dd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.449+0000 7f6ce95b0640 1 -- 172.21.15.22:0/124800187 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa41e9800 con 0x564fa41ebbb0 2024-09-19T08:14:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.449+0000 7f6ce95b0640 1 --2- 172.21.15.22:0/124800187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41ebbb0 0x564fa41dbef0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x564fa41ebb80 tx=0x7f6cd8019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b90dd40791ce16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.449+0000 7f6ccb7fe640 1 -- 172.21.15.22:0/124800187 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8026070 con 0x564fa41ebbb0 2024-09-19T08:14:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.449+0000 7f6ccb7fe640 1 -- 172.21.15.22:0/124800187 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd8019070 con 0x564fa41ebbb0 2024-09-19T08:14:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.449+0000 7f6ccb7fe640 1 -- 172.21.15.22:0/124800187 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd800f040 con 0x564fa41ebbb0 2024-09-19T08:14:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.449+0000 7f6cee3ee740 1 -- 172.21.15.22:0/124800187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41ebbb0 msgr2=0x564fa41dbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.449+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/124800187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41ebbb0 0x564fa41dbef0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x564fa41ebb80 tx=0x7f6cd8019760 comp rx=0 tx=0).stop 2024-09-19T08:14:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.450+0000 7f6cee3ee740 1 -- 172.21.15.22:0/124800187 shutdown_connections 2024-09-19T08:14:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.450+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/124800187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41ebbb0 0x564fa41dbef0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.450+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/124800187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41e12f0 0x564fa41dd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.450+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/124800187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41e09d0 0x564fa41fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.450+0000 7f6cee3ee740 1 -- 172.21.15.22:0/124800187 >> 172.21.15.22:0/124800187 conn(0x564fa41dcf60 msgr2=0x564fa41db7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.450+0000 7f6cee3ee740 1 -- 172.21.15.22:0/124800187 shutdown_connections 2024-09-19T08:14:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.450+0000 7f6cee3ee740 1 -- 172.21.15.22:0/124800187 wait complete. 2024-09-19T08:14:51.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.450+0000 7f6cee3ee740 1 -- start start 2024-09-19T08:14:51.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.450+0000 7f6cee3ee740 1 Processor -- start 2024-09-19T08:14:51.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.450+0000 7f6cee3ee740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41e09d0 0x564fa4206110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.450+0000 7f6cee3ee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa4201440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.450+0000 7f6cee3ee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41ebbb0 0x564fa4201980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.450+0000 7f6cee3ee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564fa41fd580 con 0x564fa41e12f0 2024-09-19T08:14:51.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6cee3ee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564fa41fceb0 con 0x564fa41ebbb0 2024-09-19T08:14:51.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6cee3ee740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564fa41fdc50 con 0x564fa41e09d0 2024-09-19T08:14:51.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6cee3ee740 10 client.0 _ll_register_callbacks cb 0x7f6cd4047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:14:51.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6ce9db1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa4201440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6ce9db1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa4201440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34656/0 (socket says 172.21.15.22:34656) 2024-09-19T08:14:51.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6ce9db1640 1 -- 172.21.15.22:0/3763529688 learned_addr learned my addr 172.21.15.22:0/3763529688 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:51.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6ce95b0640 1 --2- 172.21.15.22:0/3763529688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41ebbb0 0x564fa4201980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6ce95b0640 1 --2- 172.21.15.22:0/3763529688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41e09d0 0x564fa4206110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6ce9db1640 1 -- 172.21.15.22:0/3763529688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41e09d0 msgr2=0x564fa4206110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:51.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6ce9db1640 1 --2- 172.21.15.22:0/3763529688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41e09d0 0x564fa4206110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6ce9db1640 1 -- 172.21.15.22:0/3763529688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41ebbb0 msgr2=0x564fa4201980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:51.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6ce9db1640 1 --2- 172.21.15.22:0/3763529688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41ebbb0 0x564fa4201980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6ce9db1640 1 -- 172.21.15.22:0/3763529688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa4202160 con 0x564fa41e12f0 2024-09-19T08:14:51.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6ce95b0640 1 --2- 172.21.15.22:0/3763529688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41e09d0 0x564fa4206110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:14:51.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6ce9db1640 1 --2- 172.21.15.22:0/3763529688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa4201440 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4030f40 tx=0x7f6cd4003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f7f6ff1b1551c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:51.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6cc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:14:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6cc97fa640 1 -- 172.21.15.22:0/3763529688 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4054040 con 0x564fa41e12f0 2024-09-19T08:14:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6cc97fa640 1 -- 172.21.15.22:0/3763529688 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6cd4031860 con 0x564fa41e12f0 2024-09-19T08:14:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6cee3ee740 1 -- 172.21.15.22:0/3763529688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cd8019070 con 0x564fa41e12f0 2024-09-19T08:14:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6cc97fa640 1 -- 172.21.15.22:0/3763529688 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6cd405f070 con 0x564fa41e12f0 2024-09-19T08:14:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6cee3ee740 10 client.15081 fetch_fsmap learned FSMap version 24 2024-09-19T08:14:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.451+0000 7f6cee3ee740 1 -- 172.21.15.22:0/3763529688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564fa42669b0 con 0x564fa41e12f0 2024-09-19T08:14:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.452+0000 7f6cc97fa640 1 -- 172.21.15.22:0/3763529688 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd8019070 con 0x564fa41e12f0 2024-09-19T08:14:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.452+0000 7f6cc97fa640 1 -- 172.21.15.22:0/3763529688 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6cd4059040 con 0x564fa41e12f0 2024-09-19T08:14:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.452+0000 7f6cee3ee740 10 client.15081 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T08:14:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.452+0000 7f6cee3ee740 10 client.15081 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:14:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.452+0000 7f6cee3ee740 20 client.15081 populate_metadata read hostname 'smithi022' 2024-09-19T08:14:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.452+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/3763529688 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x564fa426b4d0 0x564fa426a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.452+0000 7f6cee3ee740 4 client.15081 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:14:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.452+0000 7f6cee3ee740 1 -- 172.21.15.22:0/3763529688 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "status"}) -- 0x564fa414eb10 con 0x564fa426b4d0 2024-09-19T08:14:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.452+0000 7f6ce95b0640 1 --2- 172.21.15.22:0/3763529688 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x564fa426b4d0 0x564fa426a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6ce95b0640 1 --2- 172.21.15.22:0/3763529688 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x564fa426b4d0 0x564fa426a1e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21b1c8a5b7115633 server_cookie=d12d994c4ca3dcab in_seq=0 out_seq=0 2024-09-19T08:14:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cc97fa640 10 client.15081 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:14:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cc97fa640 1 -- 172.21.15.22:0/3763529688 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564fa414eb10 con 0x564fa426b4d0 2024-09-19T08:14:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cc97fa640 10 client.15081 handle_command_reply: tid=0 2024-09-19T08:14:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cee3ee740 1 client.15081 shutdown 2024-09-19T08:14:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cee3ee740 1 -- 172.21.15.22:0/3763529688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 msgr2=0x564fa4201440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/3763529688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa4201440 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4030f40 tx=0x7f6cd4003e10 comp rx=0 tx=0).stop 2024-09-19T08:14:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cee3ee740 1 -- 172.21.15.22:0/3763529688 shutdown_connections 2024-09-19T08:14:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/3763529688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fa41ebbb0 0x564fa4201980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/3763529688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa41e12f0 0x564fa4201440 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/3763529688 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x564fa426b4d0 0x564fa426a1e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cee3ee740 1 --2- 172.21.15.22:0/3763529688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564fa41e09d0 0x564fa4206110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cc97fa640 0 client.15081 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:14:51.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cee3ee740 1 -- 172.21.15.22:0/3763529688 >> 172.21.15.22:0/3763529688 conn(0x564fa41dcf60 msgr2=0x564fa41dc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:51.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cee3ee740 1 -- 172.21.15.22:0/3763529688 shutdown_connections 2024-09-19T08:14:51.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cee3ee740 1 -- 172.21.15.22:0/3763529688 wait complete. 2024-09-19T08:14:51.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.453+0000 7f6cee3ee740 20 client.15081 trim_cache size 0 max 16384 2024-09-19T08:14:51.459 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:14:51.459 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "c0ad60c6-765d-11ef-bceb-c7b262605968", 2024-09-19T08:14:51.459 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 2, 2024-09-19T08:14:51.460 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24460, 2024-09-19T08:14:51.460 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-19T08:14:51.460 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-19T08:14:51.460 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-19T08:14:51.460 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 100.58151436999999, 2024-09-19T08:14:51.460 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 24, 2024-09-19T08:14:51.460 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 85, 2024-09-19T08:14:51.460 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T08:14:51.460 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 134.690649054 2024-09-19T08:14:51.460 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:14:51.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.454+0000 7f6cec015640 1 -- 172.21.15.22:0/2823002718 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6ccc0a32c0 msgr2=0x7f6ccc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:51.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.454+0000 7f6cec015640 1 --2- 172.21.15.22:0/2823002718 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6ccc0a32c0 0x7f6ccc0a5780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc098350 tx=0x7f6cdc0c6cc0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.454+0000 7f6cec015640 1 -- 172.21.15.22:0/2823002718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ce4106f20 msgr2=0x7f6ce419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:51.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.454+0000 7f6cec015640 1 --2- 172.21.15.22:0/2823002718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ce4106f20 0x7f6ce419cd20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c970 tx=0x7f6cdc00ce30 comp rx=0 tx=0).stop 2024-09-19T08:14:51.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.454+0000 7f6cec015640 1 -- 172.21.15.22:0/2823002718 shutdown_connections 2024-09-19T08:14:51.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.454+0000 7f6cec015640 1 --2- 172.21.15.22:0/2823002718 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f6ccc0a32c0 0x7f6ccc0a5780 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.454+0000 7f6cec015640 1 --2- 172.21.15.22:0/2823002718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ce4106f20 0x7f6ce419cd20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.454+0000 7f6cec015640 1 --2- 172.21.15.22:0/2823002718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce41027a0 0x7f6ce4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.454+0000 7f6cec015640 1 --2- 172.21.15.22:0/2823002718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ce4101df0 0x7f6ce4198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:51.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.454+0000 7f6cec015640 1 -- 172.21.15.22:0/2823002718 >> 172.21.15.22:0/2823002718 conn(0x7f6ce40fd8b0 msgr2=0x7f6ce40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:51.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.454+0000 7f6cec015640 1 -- 172.21.15.22:0/2823002718 shutdown_connections 2024-09-19T08:14:51.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:51.454+0000 7f6cec015640 1 -- 172.21.15.22:0/2823002718 wait complete. 2024-09-19T08:14:52.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:51 smithi157 ceph-mon[35143]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T08:14:52.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:51 smithi022 ceph-mon[30963]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T08:14:52.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T08:14:52.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:51 smithi064 ceph-mon[34900]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T08:14:52.585 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:53.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.046+0000 7fc45862e640 1 -- 172.21.15.22:0/1960325423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc450100e20 msgr2=0x7fc450101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:53.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.046+0000 7fc45862e640 1 --2- 172.21.15.22:0/1960325423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc450100e20 0x7fc450101200 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009900 tx=0x7fc44c02a480 comp rx=0 tx=0).stop 2024-09-19T08:14:53.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.047+0000 7fc45862e640 1 -- 172.21.15.22:0/1960325423 shutdown_connections 2024-09-19T08:14:53.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.047+0000 7fc45862e640 1 --2- 172.21.15.22:0/1960325423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc450102ef0 0x7fc45010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.047+0000 7fc45862e640 1 --2- 172.21.15.22:0/1960325423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc450102570 0x7fc4501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.047+0000 7fc45862e640 1 --2- 172.21.15.22:0/1960325423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc450100e20 0x7fc450101200 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.047+0000 7fc45862e640 1 -- 172.21.15.22:0/1960325423 >> 172.21.15.22:0/1960325423 conn(0x7fc4500fc7d0 msgr2=0x7fc4500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:53.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.047+0000 7fc45862e640 1 -- 172.21.15.22:0/1960325423 shutdown_connections 2024-09-19T08:14:53.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.047+0000 7fc45862e640 1 -- 172.21.15.22:0/1960325423 wait complete. 2024-09-19T08:14:53.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.047+0000 7fc45862e640 1 Processor -- start 2024-09-19T08:14:53.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.047+0000 7fc45862e640 1 -- start start 2024-09-19T08:14:53.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.047+0000 7fc45862e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc450100e20 0x7fc4501985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.047+0000 7fc45862e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc450102570 0x7fc450198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc45862e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc450102ef0 0x7fc45019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc45862e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc4501120a0 con 0x7fc450102ef0 2024-09-19T08:14:53.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc45862e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc450111f20 con 0x7fc450100e20 2024-09-19T08:14:53.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc45862e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc450112220 con 0x7fc450102570 2024-09-19T08:14:53.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc456bcb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc450102ef0 0x7fc45019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc456bcb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc450102ef0 0x7fc45019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34670/0 (socket says 172.21.15.22:34670) 2024-09-19T08:14:53.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc456bcb640 1 -- 172.21.15.22:0/1114436576 learned_addr learned my addr 172.21.15.22:0/1114436576 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:53.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc455bc9640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc450102570 0x7fc450198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc4563ca640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc450100e20 0x7fc4501985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc456bcb640 1 -- 172.21.15.22:0/1114436576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc450102570 msgr2=0x7fc450198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:53.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc456bcb640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc450102570 0x7fc450198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc456bcb640 1 -- 172.21.15.22:0/1114436576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc450100e20 msgr2=0x7fc4501985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:53.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc456bcb640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc450100e20 0x7fc4501985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc456bcb640 1 -- 172.21.15.22:0/1114436576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45019d570 con 0x7fc450102ef0 2024-09-19T08:14:53.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc4563ca640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc450100e20 0x7fc4501985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:53.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc456bcb640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc450102ef0 0x7fc45019ce90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc44800eaa0 tx=0x7fc44800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:53.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc4477fe640 1 -- 172.21.15.22:0/1114436576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448004d20 con 0x7fc450102ef0 2024-09-19T08:14:53.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc4477fe640 1 -- 172.21.15.22:0/1114436576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc448005020 con 0x7fc450102ef0 2024-09-19T08:14:53.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc45862e640 1 -- 172.21.15.22:0/1114436576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45019d860 con 0x7fc450102ef0 2024-09-19T08:14:53.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.048+0000 7fc4477fe640 1 -- 172.21.15.22:0/1114436576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4480116d0 con 0x7fc450102ef0 2024-09-19T08:14:53.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.049+0000 7fc45862e640 1 -- 172.21.15.22:0/1114436576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4501a5140 con 0x7fc450102ef0 2024-09-19T08:14:53.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.051+0000 7fc4477fe640 1 -- 172.21.15.22:0/1114436576 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fc4480051d0 con 0x7fc450102ef0 2024-09-19T08:14:53.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.051+0000 7fc45aa07740 1 Processor -- start 2024-09-19T08:14:53.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.051+0000 7fc45aa07740 1 -- start start 2024-09-19T08:14:53.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.051+0000 7fc45aa07740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f59d0 0x558ad42f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.051+0000 7fc45aa07740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f62f0 0x558ad4300670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.051+0000 7fc45aa07740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 0x558ad4302fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.051+0000 7fc45aa07740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558ad425c090 con 0x558ad42f62f0 2024-09-19T08:14:53.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.051+0000 7fc45aa07740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558ad4203a70 con 0x558ad42f59d0 2024-09-19T08:14:53.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.051+0000 7fc45aa07740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558ad4258050 con 0x558ad4300bb0 2024-09-19T08:14:53.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc4563ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f62f0 0x558ad4300670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc4563ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f62f0 0x558ad4300670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34674/0 (socket says 172.21.15.22:34674) 2024-09-19T08:14:53.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc4563ca640 1 -- 172.21.15.22:0/3360623179 learned_addr learned my addr 172.21.15.22:0/3360623179 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:53.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc455bc9640 1 --2- 172.21.15.22:0/3360623179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 0x558ad4302fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc455bc9640 1 --2- 172.21.15.22:0/3360623179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f59d0 0x558ad42f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc4563ca640 1 -- 172.21.15.22:0/3360623179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 msgr2=0x558ad4302fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:53.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc4563ca640 1 --2- 172.21.15.22:0/3360623179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 0x558ad4302fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc4563ca640 1 -- 172.21.15.22:0/3360623179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f59d0 msgr2=0x558ad42f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc4563ca640 1 --2- 172.21.15.22:0/3360623179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f59d0 0x558ad42f5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc4563ca640 1 -- 172.21.15.22:0/3360623179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ad4210130 con 0x558ad42f62f0 2024-09-19T08:14:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc455bc9640 1 --2- 172.21.15.22:0/3360623179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 0x558ad4302fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc4477fe640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc43c0a32c0 0x7fc43c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc4563ca640 1 --2- 172.21.15.22:0/3360623179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f62f0 0x558ad4300670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0314e0 tx=0x7fc44c004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d8662d0edcfc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:53.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc455bc9640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc43c0a32c0 0x7fc43c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc4477fe640 1 -- 172.21.15.22:0/1114436576 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc4480cfb20 con 0x7fc450102ef0 2024-09-19T08:14:53.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc444ff9640 1 -- 172.21.15.22:0/3360623179 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c035030 con 0x558ad42f62f0 2024-09-19T08:14:53.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc444ff9640 1 -- 172.21.15.22:0/3360623179 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44c005520 con 0x558ad42f62f0 2024-09-19T08:14:53.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.052+0000 7fc444ff9640 1 -- 172.21.15.22:0/3360623179 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c003850 con 0x558ad42f62f0 2024-09-19T08:14:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.053+0000 7fc45aa07740 1 -- 172.21.15.22:0/3360623179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f62f0 msgr2=0x558ad4300670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.053+0000 7fc455bc9640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc43c0a32c0 0x7fc43c0a5780 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x558ad42f32e0 tx=0x7fc440008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.053+0000 7fc45aa07740 1 --2- 172.21.15.22:0/3360623179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f62f0 0x558ad4300670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0314e0 tx=0x7fc44c004200 comp rx=0 tx=0).stop 2024-09-19T08:14:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.053+0000 7fc45aa07740 1 -- 172.21.15.22:0/3360623179 shutdown_connections 2024-09-19T08:14:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.053+0000 7fc45aa07740 1 --2- 172.21.15.22:0/3360623179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 0x558ad4302fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.053+0000 7fc45aa07740 1 --2- 172.21.15.22:0/3360623179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f62f0 0x558ad4300670 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.053+0000 7fc45aa07740 1 --2- 172.21.15.22:0/3360623179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f59d0 0x558ad42f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.053+0000 7fc45aa07740 1 -- 172.21.15.22:0/3360623179 >> 172.21.15.22:0/3360623179 conn(0x558ad42f1f60 msgr2=0x558ad42f2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:53.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.053+0000 7fc45aa07740 1 -- 172.21.15.22:0/3360623179 shutdown_connections 2024-09-19T08:14:53.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.053+0000 7fc45aa07740 1 -- 172.21.15.22:0/3360623179 wait complete. 2024-09-19T08:14:53.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc45aa07740 1 -- start start 2024-09-19T08:14:53.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc45aa07740 1 Processor -- start 2024-09-19T08:14:53.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc45aa07740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f59d0 0x558ad43004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc45aa07740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f62f0 0x558ad42fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc45aa07740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 0x558ad42fdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc45aa07740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558ad4303780 con 0x558ad42f59d0 2024-09-19T08:14:53.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc45aa07740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558ad4216f20 con 0x558ad42f62f0 2024-09-19T08:14:53.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc45aa07740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558ad4303900 con 0x558ad4300bb0 2024-09-19T08:14:53.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc45aa07740 10 client.0 _ll_register_callbacks cb 0x558ad41e0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:14:53.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc4563ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f59d0 0x558ad43004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc4563ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f59d0 0x558ad43004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34686/0 (socket says 172.21.15.22:34686) 2024-09-19T08:14:53.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc4563ca640 1 -- 172.21.15.22:0/4127400301 learned_addr learned my addr 172.21.15.22:0/4127400301 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:53.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc455bc9640 1 --2- 172.21.15.22:0/4127400301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f62f0 0x558ad42fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc456bcb640 1 --2- 172.21.15.22:0/4127400301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 0x558ad42fdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc4563ca640 1 -- 172.21.15.22:0/4127400301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 msgr2=0x558ad42fdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:53.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc4563ca640 1 --2- 172.21.15.22:0/4127400301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 0x558ad42fdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc4563ca640 1 -- 172.21.15.22:0/4127400301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f62f0 msgr2=0x558ad42fd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:53.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc4563ca640 1 --2- 172.21.15.22:0/4127400301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f62f0 0x558ad42fd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc4563ca640 1 -- 172.21.15.22:0/4127400301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ad3fcd2b0 con 0x558ad42f59d0 2024-09-19T08:14:53.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.054+0000 7fc456bcb640 1 --2- 172.21.15.22:0/4127400301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 0x558ad42fdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:14:53.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.055+0000 7fc4563ca640 1 --2- 172.21.15.22:0/4127400301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f59d0 0x558ad43004a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc44c02a4c0 tx=0x7fc44c004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d301550e2cec28f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:53.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.055+0000 7fc439ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:14:53.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.055+0000 7fc439ffb640 1 -- 172.21.15.22:0/4127400301 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c035070 con 0x558ad42f59d0 2024-09-19T08:14:53.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.055+0000 7fc439ffb640 1 -- 172.21.15.22:0/4127400301 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc44c0050f0 con 0x558ad42f59d0 2024-09-19T08:14:53.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.055+0000 7fc45aa07740 1 -- 172.21.15.22:0/4127400301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc44c005520 con 0x558ad42f59d0 2024-09-19T08:14:53.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.055+0000 7fc439ffb640 1 -- 172.21.15.22:0/4127400301 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc44c0052f0 con 0x558ad42f59d0 2024-09-19T08:14:53.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.055+0000 7fc45aa07740 10 client.15093 fetch_fsmap learned FSMap version 24 2024-09-19T08:14:53.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.055+0000 7fc45aa07740 1 -- 172.21.15.22:0/4127400301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558ad430f0f0 con 0x558ad42f59d0 2024-09-19T08:14:53.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.055+0000 7fc439ffb640 1 -- 172.21.15.22:0/4127400301 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44c005520 con 0x558ad42f59d0 2024-09-19T08:14:53.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.055+0000 7fc439ffb640 1 -- 172.21.15.22:0/4127400301 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc44c0318a0 con 0x558ad42f59d0 2024-09-19T08:14:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.055+0000 7fc45aa07740 10 client.15093 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T08:14:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.055+0000 7fc45aa07740 10 client.15093 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T08:14:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.055+0000 7fc45aa07740 20 client.15093 populate_metadata read hostname 'smithi022' 2024-09-19T08:14:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.056+0000 7fc45aa07740 1 --2- 172.21.15.22:0/4127400301 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x558ad4311200 0x558ad4312670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.056+0000 7fc45aa07740 4 client.15093 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:14:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.056+0000 7fc45aa07740 1 -- 172.21.15.22:0/4127400301 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ad4269600 con 0x558ad4311200 2024-09-19T08:14:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.056+0000 7fc455bc9640 1 --2- 172.21.15.22:0/4127400301 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x558ad4311200 0x558ad4312670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.056+0000 7fc439ffb640 10 client.15093 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:14:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.056+0000 7fc455bc9640 1 --2- 172.21.15.22:0/4127400301 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x558ad4311200 0x558ad4312670 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eadd144d054f084e server_cookie=73c275c6f87c9f82 in_seq=0 out_seq=0 2024-09-19T08:14:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc439ffb640 1 -- 172.21.15.22:0/4127400301 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ad4269600 con 0x558ad4311200 2024-09-19T08:14:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc439ffb640 10 client.15093 handle_command_reply: tid=0 2024-09-19T08:14:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc45aa07740 1 client.15093 shutdown 2024-09-19T08:14:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc45aa07740 1 -- 172.21.15.22:0/4127400301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f59d0 msgr2=0x558ad43004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc45aa07740 1 --2- 172.21.15.22:0/4127400301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f59d0 0x558ad43004a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc44c02a4c0 tx=0x7fc44c004b00 comp rx=0 tx=0).stop 2024-09-19T08:14:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc45aa07740 1 -- 172.21.15.22:0/4127400301 shutdown_connections 2024-09-19T08:14:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc45aa07740 1 --2- 172.21.15.22:0/4127400301 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x558ad4311200 0x558ad4312670 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc45aa07740 1 --2- 172.21.15.22:0/4127400301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 0x558ad42fdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc45aa07740 1 --2- 172.21.15.22:0/4127400301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f62f0 0x558ad42fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc45aa07740 1 --2- 172.21.15.22:0/4127400301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f59d0 0x558ad43004a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc45aa07740 1 -- 172.21.15.22:0/4127400301 >> 172.21.15.22:0/4127400301 conn(0x558ad42f1f60 msgr2=0x558ad42f7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc439ffb640 0 client.15093 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:14:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc45aa07740 1 -- 172.21.15.22:0/4127400301 shutdown_connections 2024-09-19T08:14:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc45aa07740 1 -- 172.21.15.22:0/4127400301 wait complete. 2024-09-19T08:14:53.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.057+0000 7fc45aa07740 20 client.15093 trim_cache size 0 max 16384 2024-09-19T08:14:53.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.075+0000 7fc45aa07740 1 Processor -- start 2024-09-19T08:14:53.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.075+0000 7fc45aa07740 1 -- start start 2024-09-19T08:14:53.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc45aa07740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f59d0 0x558ad43118d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc45aa07740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad42f62f0 0x558ad42ff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc45aa07740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad4300bb0 0x558ad42faab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc45aa07740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558ad43109b0 con 0x558ad42f59d0 2024-09-19T08:14:53.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc45aa07740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558ad4311080 con 0x558ad4300bb0 2024-09-19T08:14:53.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc45aa07740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558ad4311750 con 0x558ad42f62f0 2024-09-19T08:14:53.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc4563ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f59d0 0x558ad43118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc4563ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f59d0 0x558ad43118d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34688/0 (socket says 172.21.15.22:34688) 2024-09-19T08:14:53.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc4563ca640 1 -- 172.21.15.22:0/2571581803 learned_addr learned my addr 172.21.15.22:0/2571581803 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:53.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc455bc9640 1 --2- 172.21.15.22:0/2571581803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad42f62f0 0x558ad42ff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc4563ca640 1 -- 172.21.15.22:0/2571581803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad42f62f0 msgr2=0x558ad42ff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:53.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc456bcb640 1 --2- 172.21.15.22:0/2571581803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad4300bb0 0x558ad42faab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc4563ca640 1 --2- 172.21.15.22:0/2571581803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad42f62f0 0x558ad42ff650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc4563ca640 1 -- 172.21.15.22:0/2571581803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad4300bb0 msgr2=0x558ad42faab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:53.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc4563ca640 1 --2- 172.21.15.22:0/2571581803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad4300bb0 0x558ad42faab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc4563ca640 1 -- 172.21.15.22:0/2571581803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ad4300810 con 0x558ad42f59d0 2024-09-19T08:14:53.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc455bc9640 1 --2- 172.21.15.22:0/2571581803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad42f62f0 0x558ad42ff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:14:53.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc4563ca640 1 --2- 172.21.15.22:0/2571581803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f59d0 0x558ad43118d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc44c006010 tx=0x7fc44c03b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc223c26e6b02671 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:53.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc43bfff640 1 -- 172.21.15.22:0/2571581803 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c035070 con 0x558ad42f59d0 2024-09-19T08:14:53.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc43bfff640 1 -- 172.21.15.22:0/2571581803 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44c0050f0 con 0x558ad42f59d0 2024-09-19T08:14:53.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.076+0000 7fc43bfff640 1 -- 172.21.15.22:0/2571581803 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c003990 con 0x558ad42f59d0 2024-09-19T08:14:53.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.077+0000 7fc45aa07740 1 -- 172.21.15.22:0/2571581803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f59d0 msgr2=0x558ad43118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:53.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.077+0000 7fc45aa07740 1 --2- 172.21.15.22:0/2571581803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f59d0 0x558ad43118d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc44c006010 tx=0x7fc44c03b430 comp rx=0 tx=0).stop 2024-09-19T08:14:53.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.077+0000 7fc45aa07740 1 -- 172.21.15.22:0/2571581803 shutdown_connections 2024-09-19T08:14:53.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.077+0000 7fc45aa07740 1 --2- 172.21.15.22:0/2571581803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad4300bb0 0x558ad42faab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.077+0000 7fc45aa07740 1 --2- 172.21.15.22:0/2571581803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad42f62f0 0x558ad42ff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.077+0000 7fc45aa07740 1 --2- 172.21.15.22:0/2571581803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f59d0 0x558ad43118d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.077+0000 7fc45aa07740 1 -- 172.21.15.22:0/2571581803 >> 172.21.15.22:0/2571581803 conn(0x558ad42f1f60 msgr2=0x558ad42fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.077+0000 7fc45aa07740 1 -- 172.21.15.22:0/2571581803 shutdown_connections 2024-09-19T08:14:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.077+0000 7fc45aa07740 1 -- 172.21.15.22:0/2571581803 wait complete. 2024-09-19T08:14:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.077+0000 7fc45aa07740 1 -- start start 2024-09-19T08:14:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc45aa07740 1 Processor -- start 2024-09-19T08:14:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc45aa07740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f59d0 0x558ad4315e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc45aa07740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f62f0 0x558ad4316340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc45aa07740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 0x558ad4379900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc45aa07740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558ad43124f0 con 0x558ad42f62f0 2024-09-19T08:14:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc45aa07740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558ad4311e20 con 0x558ad42f59d0 2024-09-19T08:14:53.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc45aa07740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558ad4312bc0 con 0x558ad4300bb0 2024-09-19T08:14:53.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc45aa07740 10 client.0 _ll_register_callbacks cb 0x7fc44c0040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:14:53.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc455bc9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f62f0 0x558ad4316340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc455bc9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f62f0 0x558ad4316340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34702/0 (socket says 172.21.15.22:34702) 2024-09-19T08:14:53.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc455bc9640 1 -- 172.21.15.22:0/3794843605 learned_addr learned my addr 172.21.15.22:0/3794843605 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:53.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc4563ca640 1 --2- 172.21.15.22:0/3794843605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f59d0 0x558ad4315e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc456bcb640 1 --2- 172.21.15.22:0/3794843605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 0x558ad4379900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc455bc9640 1 -- 172.21.15.22:0/3794843605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 msgr2=0x558ad4379900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:53.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc455bc9640 1 --2- 172.21.15.22:0/3794843605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 0x558ad4379900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc455bc9640 1 -- 172.21.15.22:0/3794843605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f59d0 msgr2=0x558ad4315e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:53.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc455bc9640 1 --2- 172.21.15.22:0/3794843605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f59d0 0x558ad4315e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc455bc9640 1 -- 172.21.15.22:0/3794843605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ad4379f40 con 0x558ad42f62f0 2024-09-19T08:14:53.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc455bc9640 1 --2- 172.21.15.22:0/3794843605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f62f0 0x558ad4316340 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x558ad42fee30 tx=0x7fc440009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14e5bd919a9fee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:53.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.078+0000 7fc439ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:14:53.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.079+0000 7fc439ffb640 1 -- 172.21.15.22:0/3794843605 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4400246c0 con 0x558ad42f62f0 2024-09-19T08:14:53.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.079+0000 7fc439ffb640 1 -- 172.21.15.22:0/3794843605 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc44000f040 con 0x558ad42f62f0 2024-09-19T08:14:53.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.079+0000 7fc45aa07740 1 -- 172.21.15.22:0/3794843605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc44c0050f0 con 0x558ad42f62f0 2024-09-19T08:14:53.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.079+0000 7fc439ffb640 1 -- 172.21.15.22:0/3794843605 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc440024860 con 0x558ad42f62f0 2024-09-19T08:14:53.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.079+0000 7fc45aa07740 10 client.15099 fetch_fsmap learned FSMap version 24 2024-09-19T08:14:53.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.079+0000 7fc45aa07740 1 -- 172.21.15.22:0/3794843605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558ad437b1e0 con 0x558ad42f62f0 2024-09-19T08:14:53.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.079+0000 7fc439ffb640 1 -- 172.21.15.22:0/3794843605 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44c0050f0 con 0x558ad42f62f0 2024-09-19T08:14:53.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.079+0000 7fc439ffb640 1 -- 172.21.15.22:0/3794843605 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc440035a30 con 0x558ad42f62f0 2024-09-19T08:14:53.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.079+0000 7fc45aa07740 10 client.15099 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T08:14:53.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.079+0000 7fc45aa07740 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T08:14:53.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.079+0000 7fc45aa07740 20 client.15099 populate_metadata read hostname 'smithi022' 2024-09-19T08:14:53.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.079+0000 7fc45aa07740 1 --2- 172.21.15.22:0/3794843605 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x558ad437fe10 0x558ad437eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:53.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.079+0000 7fc45aa07740 4 client.15099 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:14:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.079+0000 7fc45aa07740 1 -- 172.21.15.22:0/3794843605 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "status"}) -- 0x558ad4263b10 con 0x558ad437fe10 2024-09-19T08:14:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.080+0000 7fc4563ca640 1 --2- 172.21.15.22:0/3794843605 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x558ad437fe10 0x558ad437eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.080+0000 7fc4563ca640 1 --2- 172.21.15.22:0/3794843605 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x558ad437fe10 0x558ad437eb20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=682f2fe9c32a091b server_cookie=e17bae74780982f4 in_seq=0 out_seq=0 2024-09-19T08:14:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.080+0000 7fc439ffb640 10 client.15099 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:14:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.080+0000 7fc439ffb640 1 -- 172.21.15.22:0/3794843605 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558ad4263b10 con 0x558ad437fe10 2024-09-19T08:14:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.080+0000 7fc439ffb640 10 client.15099 handle_command_reply: tid=0 2024-09-19T08:14:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.080+0000 7fc45aa07740 1 client.15099 shutdown 2024-09-19T08:14:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.080+0000 7fc45aa07740 1 -- 172.21.15.22:0/3794843605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f62f0 msgr2=0x558ad4316340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.080+0000 7fc45aa07740 1 --2- 172.21.15.22:0/3794843605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f62f0 0x558ad4316340 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x558ad42fee30 tx=0x7fc440009110 comp rx=0 tx=0).stop 2024-09-19T08:14:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.080+0000 7fc45aa07740 1 -- 172.21.15.22:0/3794843605 shutdown_connections 2024-09-19T08:14:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45aa07740 1 --2- 172.21.15.22:0/3794843605 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x558ad437fe10 0x558ad437eb20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45aa07740 1 --2- 172.21.15.22:0/3794843605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558ad4300bb0 0x558ad4379900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45aa07740 1 --2- 172.21.15.22:0/3794843605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ad42f62f0 0x558ad4316340 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45aa07740 1 --2- 172.21.15.22:0/3794843605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ad42f59d0 0x558ad4315e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc439ffb640 0 client.15099 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:14:53.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45aa07740 1 -- 172.21.15.22:0/3794843605 >> 172.21.15.22:0/3794843605 conn(0x558ad42f1f60 msgr2=0x558ad42f1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:53.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45aa07740 1 -- 172.21.15.22:0/3794843605 shutdown_connections 2024-09-19T08:14:53.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45aa07740 1 -- 172.21.15.22:0/3794843605 wait complete. 2024-09-19T08:14:53.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45aa07740 20 client.15099 trim_cache size 0 max 16384 2024-09-19T08:14:53.095 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:14:53.095 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "c0ad60c6-765d-11ef-bceb-c7b262605968", 2024-09-19T08:14:53.096 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 1, 2024-09-19T08:14:53.096 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24479, 2024-09-19T08:14:53.096 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-19T08:14:53.096 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-19T08:14:53.096 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-19T08:14:53.096 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 103.218482197, 2024-09-19T08:14:53.096 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 24, 2024-09-19T08:14:53.096 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 85, 2024-09-19T08:14:53.096 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T08:14:53.096 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 122.43353813900001 2024-09-19T08:14:53.096 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:14:53.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45862e640 1 -- 172.21.15.22:0/1114436576 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc43c0a32c0 msgr2=0x7fc43c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:53.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45862e640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc43c0a32c0 0x7fc43c0a5780 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x558ad42f32e0 tx=0x7fc440008040 comp rx=0 tx=0).stop 2024-09-19T08:14:53.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45862e640 1 -- 172.21.15.22:0/1114436576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc450102ef0 msgr2=0x7fc45019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:53.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45862e640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc450102ef0 0x7fc45019ce90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc44800eaa0 tx=0x7fc44800eea0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45862e640 1 -- 172.21.15.22:0/1114436576 shutdown_connections 2024-09-19T08:14:53.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45862e640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fc43c0a32c0 0x7fc43c0a5780 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45862e640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc450102ef0 0x7fc45019ce90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45862e640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc450102570 0x7fc450198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45862e640 1 --2- 172.21.15.22:0/1114436576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc450100e20 0x7fc4501985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:53.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45862e640 1 -- 172.21.15.22:0/1114436576 >> 172.21.15.22:0/1114436576 conn(0x7fc4500fc7d0 msgr2=0x7fc4500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:53.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45862e640 1 -- 172.21.15.22:0/1114436576 shutdown_connections 2024-09-19T08:14:53.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:53.081+0000 7fc45862e640 1 -- 172.21.15.22:0/1114436576 wait complete. 2024-09-19T08:14:53.800 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T08:14:54.062 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:54.183 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:53 smithi022 ceph-mon[30963]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:14:54.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:53 smithi157 ceph-mon[35143]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:14:54.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:53 smithi064 ceph-mon[34900]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:14:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.530+0000 7f2bc62f2640 1 -- 172.21.15.22:0/2207606955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bc010c170 msgr2=0x7f2bc010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.530+0000 7f2bc62f2640 1 --2- 172.21.15.22:0/2207606955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bc010c170 0x7f2bc010e610 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009ca0 tx=0x7f2bb802a950 comp rx=0 tx=0).stop 2024-09-19T08:14:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.530+0000 7f2bc62f2640 1 -- 172.21.15.22:0/2207606955 shutdown_connections 2024-09-19T08:14:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.530+0000 7f2bc62f2640 1 --2- 172.21.15.22:0/2207606955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bc010c170 0x7f2bc010e610 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.530+0000 7f2bc62f2640 1 --2- 172.21.15.22:0/2207606955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc0103660 0x7f2bc010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.530+0000 7f2bc62f2640 1 --2- 172.21.15.22:0/2207606955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bc0102d40 0x7f2bc0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.530+0000 7f2bc62f2640 1 -- 172.21.15.22:0/2207606955 >> 172.21.15.22:0/2207606955 conn(0x7f2bc00fc7a0 msgr2=0x7f2bc00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.530+0000 7f2bc62f2640 1 -- 172.21.15.22:0/2207606955 shutdown_connections 2024-09-19T08:14:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.530+0000 7f2bc62f2640 1 -- 172.21.15.22:0/2207606955 wait complete. 2024-09-19T08:14:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.531+0000 7f2bc62f2640 1 Processor -- start 2024-09-19T08:14:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.531+0000 7f2bc62f2640 1 -- start start 2024-09-19T08:14:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.531+0000 7f2bc62f2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bc0102d40 0x7f2bc01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.531+0000 7f2bc62f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc0103660 0x7f2bc0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.531+0000 7f2bc62f2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bc010c170 0x7f2bc0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.531+0000 7f2bc62f2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0111fb0 con 0x7f2bc010c170 2024-09-19T08:14:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.531+0000 7f2bc62f2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0111e30 con 0x7f2bc0102d40 2024-09-19T08:14:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.531+0000 7f2bc62f2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0112130 con 0x7f2bc0103660 2024-09-19T08:14:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bc488f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bc010c170 0x7f2bc0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bc488f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bc010c170 0x7f2bc0101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34722/0 (socket says 172.21.15.22:34722) 2024-09-19T08:14:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bc488f640 1 -- 172.21.15.22:0/2381428934 learned_addr learned my addr 172.21.15.22:0/2381428934 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bbffff640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bc0102d40 0x7f2bc01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc0103660 0x7f2bc0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bc488f640 1 -- 172.21.15.22:0/2381428934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc0103660 msgr2=0x7f2bc0100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bc488f640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc0103660 0x7f2bc0100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bc488f640 1 -- 172.21.15.22:0/2381428934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bc0102d40 msgr2=0x7f2bc01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bc488f640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bc0102d40 0x7f2bc01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bc488f640 1 -- 172.21.15.22:0/2381428934 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc01a6930 con 0x7f2bc010c170 2024-09-19T08:14:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc0103660 0x7f2bc0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bc488f640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bc010c170 0x7f2bc0101420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2bb802ae70 tx=0x7f2bb802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bbd7fa640 1 -- 172.21.15.22:0/2381428934 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8034040 con 0x7f2bc010c170 2024-09-19T08:14:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bc62f2640 1 -- 172.21.15.22:0/2381428934 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc01a6bc0 con 0x7f2bc010c170 2024-09-19T08:14:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bbd7fa640 1 -- 172.21.15.22:0/2381428934 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb8030920 con 0x7f2bc010c170 2024-09-19T08:14:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bbd7fa640 1 -- 172.21.15.22:0/2381428934 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8045480 con 0x7f2bc010c170 2024-09-19T08:14:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.532+0000 7f2bc62f2640 1 -- 172.21.15.22:0/2381428934 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc01a70f0 con 0x7f2bc010c170 2024-09-19T08:14:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.535+0000 7f2bbd7fa640 1 -- 172.21.15.22:0/2381428934 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f2bb8037020 con 0x7f2bc010c170 2024-09-19T08:14:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.535+0000 7f2bc86cb740 1 Processor -- start 2024-09-19T08:14:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bc86cb740 1 -- start start 2024-09-19T08:14:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bc86cb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f29d0 0x5580e14f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bc86cb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f32f0 0x5580e14fd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bc86cb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14fdbb0 0x5580e14fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bc86cb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5580e1459090 con 0x5580e14f29d0 2024-09-19T08:14:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bc86cb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5580e1400a70 con 0x5580e14f32f0 2024-09-19T08:14:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bc86cb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5580e1455050 con 0x5580e14fdbb0 2024-09-19T08:14:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bbf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f29d0 0x5580e14f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bbf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f29d0 0x5580e14f2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34736/0 (socket says 172.21.15.22:34736) 2024-09-19T08:14:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bbf7fe640 1 -- 172.21.15.22:0/2767825134 learned_addr learned my addr 172.21.15.22:0/2767825134 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bbd7fa640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2ba80a3240 0x7f2ba80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/2767825134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14fdbb0 0x5580e14fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bbffff640 1 --2- 172.21.15.22:0/2767825134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f32f0 0x5580e14fd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bbf7fe640 1 -- 172.21.15.22:0/2767825134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14fdbb0 msgr2=0x5580e14fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:54.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/2767825134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14fdbb0 0x5580e14fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bbf7fe640 1 -- 172.21.15.22:0/2767825134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f32f0 msgr2=0x5580e14fd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:54.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/2767825134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f32f0 0x5580e14fd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bc488f640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2ba80a3240 0x7f2ba80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bbf7fe640 1 -- 172.21.15.22:0/2767825134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580e140d130 con 0x5580e14f29d0 2024-09-19T08:14:54.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bbffff640 1 --2- 172.21.15.22:0/2767825134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f32f0 0x5580e14fd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:14:54.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.536+0000 7f2bbd7fa640 1 -- 172.21.15.22:0/2381428934 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2bb80f0810 con 0x7f2bc010c170 2024-09-19T08:14:54.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.537+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/2767825134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f29d0 0x5580e14f2db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4007e80 tx=0x7f2bb4007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892e51a55b2d60f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:54.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.537+0000 7f2ba67fc640 1 -- 172.21.15.22:0/2767825134 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb401c070 con 0x5580e14f29d0 2024-09-19T08:14:54.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.537+0000 7f2ba67fc640 1 -- 172.21.15.22:0/2767825134 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb4010040 con 0x5580e14f29d0 2024-09-19T08:14:54.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.537+0000 7f2ba67fc640 1 -- 172.21.15.22:0/2767825134 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb40147d0 con 0x5580e14f29d0 2024-09-19T08:14:54.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.537+0000 7f2bc86cb740 1 -- 172.21.15.22:0/2767825134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f29d0 msgr2=0x5580e14f2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:54.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.537+0000 7f2bc488f640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2ba80a3240 0x7f2ba80a5700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80bcb30 tx=0x7f2bb808fcb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:54.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.537+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/2767825134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f29d0 0x5580e14f2db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4007e80 tx=0x7f2bb4007410 comp rx=0 tx=0).stop 2024-09-19T08:14:54.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.537+0000 7f2bc86cb740 1 -- 172.21.15.22:0/2767825134 shutdown_connections 2024-09-19T08:14:54.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.537+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/2767825134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14fdbb0 0x5580e14fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.537+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/2767825134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f32f0 0x5580e14fd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.537+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/2767825134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f29d0 0x5580e14f2db0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.537+0000 7f2bc86cb740 1 -- 172.21.15.22:0/2767825134 >> 172.21.15.22:0/2767825134 conn(0x5580e14eef60 msgr2=0x5580e14ef340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:54.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.538+0000 7f2bc86cb740 1 -- 172.21.15.22:0/2767825134 shutdown_connections 2024-09-19T08:14:54.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.538+0000 7f2bc86cb740 1 -- 172.21.15.22:0/2767825134 wait complete. 2024-09-19T08:14:54.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.538+0000 7f2bc86cb740 1 -- start start 2024-09-19T08:14:54.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.538+0000 7f2bc86cb740 1 Processor -- start 2024-09-19T08:14:54.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.538+0000 7f2bc86cb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f29d0 0x5580e14fd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.538+0000 7f2bc86cb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f32f0 0x5580e14fa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.538+0000 7f2bc86cb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14fdbb0 0x5580e14fad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.538+0000 7f2bc86cb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5580e1500780 con 0x5580e14f32f0 2024-09-19T08:14:54.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.538+0000 7f2bc86cb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5580e1413f20 con 0x5580e14f29d0 2024-09-19T08:14:54.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.538+0000 7f2bc86cb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5580e1500900 con 0x5580e14fdbb0 2024-09-19T08:14:54.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.538+0000 7f2bc86cb740 10 client.0 _ll_register_callbacks cb 0x5580e13dd030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:14:54.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.538+0000 7f2bbffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f32f0 0x5580e14fa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.538+0000 7f2bbffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f32f0 0x5580e14fa7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34746/0 (socket says 172.21.15.22:34746) 2024-09-19T08:14:54.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.538+0000 7f2bbffff640 1 -- 172.21.15.22:0/1908608421 learned_addr learned my addr 172.21.15.22:0/1908608421 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:54.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/1908608421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f29d0 0x5580e14fd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/1908608421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14fdbb0 0x5580e14fad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2bbffff640 1 -- 172.21.15.22:0/1908608421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14fdbb0 msgr2=0x5580e14fad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:54.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2bbffff640 1 --2- 172.21.15.22:0/1908608421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14fdbb0 0x5580e14fad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2bbffff640 1 -- 172.21.15.22:0/1908608421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f29d0 msgr2=0x5580e14fd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:54.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2bbffff640 1 --2- 172.21.15.22:0/1908608421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f29d0 0x5580e14fd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2bbffff640 1 -- 172.21.15.22:0/1908608421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580e11ca2b0 con 0x5580e14f32f0 2024-09-19T08:14:54.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/1908608421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14fdbb0 0x5580e14fad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:14:54.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2bbffff640 1 --2- 172.21.15.22:0/1908608421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f32f0 0x5580e14fa7c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000b980 tx=0x7f2bb000be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4e8e38ad7f65d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:54.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2b9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:14:54.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/1908608421 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0005220 con 0x5580e14f32f0 2024-09-19T08:14:54.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/1908608421 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2bb00053c0 con 0x5580e14f32f0 2024-09-19T08:14:54.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2bc86cb740 1 -- 172.21.15.22:0/1908608421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bb4010040 con 0x5580e14f32f0 2024-09-19T08:14:54.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/1908608421 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2bb0003e40 con 0x5580e14f32f0 2024-09-19T08:14:54.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2bc86cb740 10 client.15111 fetch_fsmap learned FSMap version 24 2024-09-19T08:14:54.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2bc86cb740 1 -- 172.21.15.22:0/1908608421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5580e150c0d0 con 0x5580e14f32f0 2024-09-19T08:14:54.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.539+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/1908608421 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb4010040 con 0x5580e14f32f0 2024-09-19T08:14:54.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.540+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/1908608421 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2bb0012850 con 0x5580e14f32f0 2024-09-19T08:14:54.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.540+0000 7f2bc86cb740 10 client.15111 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T08:14:54.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.540+0000 7f2bc86cb740 10 client.15111 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T08:14:54.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.540+0000 7f2bc86cb740 20 client.15111 populate_metadata read hostname 'smithi022' 2024-09-19T08:14:54.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.540+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/1908608421 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5580e150e6e0 0x5580e150fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.540+0000 7f2bc86cb740 4 client.15111 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:14:54.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.540+0000 7f2bc86cb740 1 -- 172.21.15.22:0/1908608421 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580e1466600 con 0x5580e150e6e0 2024-09-19T08:14:54.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.540+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/1908608421 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5580e150e6e0 0x5580e150fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.540+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/1908608421 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5580e150e6e0 0x5580e150fb50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3b5c31f3566ef42 server_cookie=bc670c4f311b5ad in_seq=0 out_seq=0 2024-09-19T08:14:54.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.540+0000 7f2b9f7fe640 10 client.15111 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:14:54.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.541+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/1908608421 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580e1466600 con 0x5580e150e6e0 2024-09-19T08:14:54.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.541+0000 7f2b9f7fe640 10 client.15111 handle_command_reply: tid=0 2024-09-19T08:14:54.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.541+0000 7f2bc86cb740 1 client.15111 shutdown 2024-09-19T08:14:54.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.541+0000 7f2bc86cb740 1 -- 172.21.15.22:0/1908608421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f32f0 msgr2=0x5580e14fa7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:54.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.541+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/1908608421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f32f0 0x5580e14fa7c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000b980 tx=0x7f2bb000be40 comp rx=0 tx=0).stop 2024-09-19T08:14:54.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.541+0000 7f2bc86cb740 1 -- 172.21.15.22:0/1908608421 shutdown_connections 2024-09-19T08:14:54.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.541+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/1908608421 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5580e150e6e0 0x5580e150fb50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.541+0000 7f2b9f7fe640 0 client.15111 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:14:54.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.541+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/1908608421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14fdbb0 0x5580e14fad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.541+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/1908608421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f32f0 0x5580e14fa7c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.541+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/1908608421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f29d0 0x5580e14fd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.541+0000 7f2bc86cb740 1 -- 172.21.15.22:0/1908608421 >> 172.21.15.22:0/1908608421 conn(0x5580e14eef60 msgr2=0x5580e14f4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:54.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.542+0000 7f2bc86cb740 1 -- 172.21.15.22:0/1908608421 shutdown_connections 2024-09-19T08:14:54.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.542+0000 7f2bc86cb740 1 -- 172.21.15.22:0/1908608421 wait complete. 2024-09-19T08:14:54.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.542+0000 7f2bc86cb740 20 client.15111 trim_cache size 0 max 16384 2024-09-19T08:14:54.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.559+0000 7f2bc86cb740 1 Processor -- start 2024-09-19T08:14:54.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.559+0000 7f2bc86cb740 1 -- start start 2024-09-19T08:14:54.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.559+0000 7f2bc86cb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f29d0 0x5580e14f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.559+0000 7f2bc86cb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14f32f0 0x5580e1517e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.559+0000 7f2bc86cb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14fdbb0 0x5580e1518370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.559+0000 7f2bc86cb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5580e150de90 con 0x5580e14f29d0 2024-09-19T08:14:54.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.559+0000 7f2bc86cb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5580e150e560 con 0x5580e14fdbb0 2024-09-19T08:14:54.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.559+0000 7f2bc86cb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5580e150ec30 con 0x5580e14f32f0 2024-09-19T08:14:54.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.560+0000 7f2bbf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f29d0 0x5580e14f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.560+0000 7f2bbf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f29d0 0x5580e14f4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34750/0 (socket says 172.21.15.22:34750) 2024-09-19T08:14:54.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.560+0000 7f2bbf7fe640 1 -- 172.21.15.22:0/1250265157 learned_addr learned my addr 172.21.15.22:0/1250265157 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:54.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.560+0000 7f2bbffff640 1 --2- 172.21.15.22:0/1250265157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14f32f0 0x5580e1517e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.560+0000 7f2bbf7fe640 1 -- 172.21.15.22:0/1250265157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14f32f0 msgr2=0x5580e1517e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:54.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.560+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/1250265157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14f32f0 0x5580e1517e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.560+0000 7f2bbf7fe640 1 -- 172.21.15.22:0/1250265157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14fdbb0 msgr2=0x5580e1518370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:54.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.560+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/1250265157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14fdbb0 0x5580e1518370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.560+0000 7f2bbf7fe640 1 -- 172.21.15.22:0/1250265157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580e14f8d10 con 0x5580e14f29d0 2024-09-19T08:14:54.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.560+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/1250265157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f29d0 0x5580e14f4b30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4017fa0 tx=0x7f2bb4007cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed63a28e9affae54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:54.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.560+0000 7f2ba5ffb640 1 -- 172.21.15.22:0/1250265157 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb401c040 con 0x5580e14f29d0 2024-09-19T08:14:54.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.560+0000 7f2ba5ffb640 1 -- 172.21.15.22:0/1250265157 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb402d050 con 0x5580e14f29d0 2024-09-19T08:14:54.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.560+0000 7f2ba5ffb640 1 -- 172.21.15.22:0/1250265157 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4014b70 con 0x5580e14f29d0 2024-09-19T08:14:54.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.561+0000 7f2bc86cb740 1 -- 172.21.15.22:0/1250265157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f29d0 msgr2=0x5580e14f4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:54.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.561+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/1250265157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f29d0 0x5580e14f4b30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4017fa0 tx=0x7f2bb4007cd0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.561+0000 7f2bc86cb740 1 -- 172.21.15.22:0/1250265157 shutdown_connections 2024-09-19T08:14:54.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.561+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/1250265157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14fdbb0 0x5580e1518370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.561+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/1250265157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14f32f0 0x5580e1517e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.561+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/1250265157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14f29d0 0x5580e14f4b30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.561+0000 7f2bc86cb740 1 -- 172.21.15.22:0/1250265157 >> 172.21.15.22:0/1250265157 conn(0x5580e14eef60 msgr2=0x5580e14f4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:54.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.561+0000 7f2bc86cb740 1 -- 172.21.15.22:0/1250265157 shutdown_connections 2024-09-19T08:14:54.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.561+0000 7f2bc86cb740 1 -- 172.21.15.22:0/1250265157 wait complete. 2024-09-19T08:14:54.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.561+0000 7f2bc86cb740 1 -- start start 2024-09-19T08:14:54.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bc86cb740 1 Processor -- start 2024-09-19T08:14:54.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bc86cb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14f29d0 0x5580e1518480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bc86cb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f32f0 0x5580e1515f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bc86cb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14fdbb0 0x5580e1516490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bc86cb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5580e150f9d0 con 0x5580e14fdbb0 2024-09-19T08:14:54.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bc86cb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5580e150f300 con 0x5580e14f32f0 2024-09-19T08:14:54.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bc86cb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5580e15100a0 con 0x5580e14f29d0 2024-09-19T08:14:54.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bc86cb740 10 client.0 _ll_register_callbacks cb 0x7f2bb00040c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:14:54.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bbf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14fdbb0 0x5580e1516490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bbf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14fdbb0 0x5580e1516490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34758/0 (socket says 172.21.15.22:34758) 2024-09-19T08:14:54.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bbf7fe640 1 -- 172.21.15.22:0/3162277614 learned_addr learned my addr 172.21.15.22:0/3162277614 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:54.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bbffff640 1 --2- 172.21.15.22:0/3162277614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f32f0 0x5580e1515f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bbf7fe640 1 -- 172.21.15.22:0/3162277614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14f29d0 msgr2=0x5580e1518480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:54.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/3162277614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14f29d0 0x5580e1518480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bbf7fe640 1 -- 172.21.15.22:0/3162277614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f32f0 msgr2=0x5580e1515f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:14:54.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/3162277614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f32f0 0x5580e1515f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bbf7fe640 1 -- 172.21.15.22:0/3162277614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580e1516cc0 con 0x5580e14fdbb0 2024-09-19T08:14:54.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2bbf7fe640 1 --2- 172.21.15.22:0/3162277614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14fdbb0 0x5580e1516490 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4007670 tx=0x7f2bb4007160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bb8d9679ae4837 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:54.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.562+0000 7f2b9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:14:54.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.563+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/3162277614 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb401c040 con 0x5580e14fdbb0 2024-09-19T08:14:54.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.563+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/3162277614 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2bb4005bc0 con 0x5580e14fdbb0 2024-09-19T08:14:54.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.563+0000 7f2bc86cb740 1 -- 172.21.15.22:0/3162277614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bb402d050 con 0x5580e14fdbb0 2024-09-19T08:14:54.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.563+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/3162277614 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2bb4014e10 con 0x5580e14fdbb0 2024-09-19T08:14:54.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.563+0000 7f2bc86cb740 10 client.15117 fetch_fsmap learned FSMap version 24 2024-09-19T08:14:54.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.563+0000 7f2bc86cb740 1 -- 172.21.15.22:0/3162277614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5580e1581600 con 0x5580e14fdbb0 2024-09-19T08:14:54.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.563+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/3162277614 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb402d050 con 0x5580e14fdbb0 2024-09-19T08:14:54.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.563+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/3162277614 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2bb4022040 con 0x5580e14fdbb0 2024-09-19T08:14:54.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.563+0000 7f2bc86cb740 10 client.15117 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T08:14:54.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.563+0000 7f2bc86cb740 10 client.15117 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T08:14:54.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.563+0000 7f2bc86cb740 20 client.15117 populate_metadata read hostname 'smithi022' 2024-09-19T08:14:54.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.563+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/3162277614 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5580e150e6e0 0x5580e1584810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:54.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.563+0000 7f2bc86cb740 4 client.15117 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:14:54.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.563+0000 7f2bc86cb740 1 -- 172.21.15.22:0/3162277614 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "status"}) -- 0x5580e1460b10 con 0x5580e150e6e0 2024-09-19T08:14:54.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.564+0000 7f2bbffff640 1 --2- 172.21.15.22:0/3162277614 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5580e150e6e0 0x5580e1584810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:54.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.564+0000 7f2bbffff640 1 --2- 172.21.15.22:0/3162277614 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5580e150e6e0 0x5580e1584810 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9afb9c4c0db87c7c server_cookie=c50e839363899c1c in_seq=0 out_seq=0 2024-09-19T08:14:54.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.564+0000 7f2b9f7fe640 10 client.15117 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:14:54.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.564+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/3162277614 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5580e1460b10 con 0x5580e150e6e0 2024-09-19T08:14:54.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.564+0000 7f2b9f7fe640 10 client.15117 handle_command_reply: tid=0 2024-09-19T08:14:54.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.564+0000 7f2bc86cb740 1 client.15117 shutdown 2024-09-19T08:14:54.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc86cb740 1 -- 172.21.15.22:0/3162277614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14fdbb0 msgr2=0x5580e1516490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:54.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/3162277614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14fdbb0 0x5580e1516490 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4007670 tx=0x7f2bb4007160 comp rx=0 tx=0).stop 2024-09-19T08:14:54.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc86cb740 1 -- 172.21.15.22:0/3162277614 shutdown_connections 2024-09-19T08:14:54.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/3162277614 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5580e150e6e0 0x5580e1584810 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/3162277614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5580e14fdbb0 0x5580e1516490 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/3162277614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580e14f32f0 0x5580e1515f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc86cb740 1 --2- 172.21.15.22:0/3162277614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580e14f29d0 0x5580e1518480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc86cb740 1 -- 172.21.15.22:0/3162277614 >> 172.21.15.22:0/3162277614 conn(0x5580e14eef60 msgr2=0x5580e14f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:54.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2b9f7fe640 0 client.15117 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:14:54.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc86cb740 1 -- 172.21.15.22:0/3162277614 shutdown_connections 2024-09-19T08:14:54.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc86cb740 1 -- 172.21.15.22:0/3162277614 wait complete. 2024-09-19T08:14:54.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc86cb740 20 client.15117 trim_cache size 0 max 16384 2024-09-19T08:14:54.578 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:14:54.578 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "c0ad60c6-765d-11ef-bceb-c7b262605968", 2024-09-19T08:14:54.578 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-19T08:14:54.578 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24481, 2024-09-19T08:14:54.579 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-19T08:14:54.579 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-19T08:14:54.579 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-19T08:14:54.579 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 112.30472431600001, 2024-09-19T08:14:54.579 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 24, 2024-09-19T08:14:54.579 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 85, 2024-09-19T08:14:54.579 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T08:14:54.579 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 132.70524113100001 2024-09-19T08:14:54.579 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:14:54.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc62f2640 1 -- 172.21.15.22:0/2381428934 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2ba80a3240 msgr2=0x7f2ba80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:54.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc62f2640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2ba80a3240 0x7f2ba80a5700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80bcb30 tx=0x7f2bb808fcb0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc62f2640 1 -- 172.21.15.22:0/2381428934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bc010c170 msgr2=0x7f2bc0101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:54.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc62f2640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bc010c170 0x7f2bc0101420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2bb802ae70 tx=0x7f2bb802f5b0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc62f2640 1 -- 172.21.15.22:0/2381428934 shutdown_connections 2024-09-19T08:14:54.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc62f2640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f2ba80a3240 0x7f2ba80a5700 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc62f2640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2bc010c170 0x7f2bc0101420 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc62f2640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc0103660 0x7f2bc0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc62f2640 1 --2- 172.21.15.22:0/2381428934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bc0102d40 0x7f2bc01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:54.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.565+0000 7f2bc62f2640 1 -- 172.21.15.22:0/2381428934 >> 172.21.15.22:0/2381428934 conn(0x7f2bc00fc7a0 msgr2=0x7f2bc00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:54.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.566+0000 7f2bc62f2640 1 -- 172.21.15.22:0/2381428934 shutdown_connections 2024-09-19T08:14:54.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:54.566+0000 7f2bc62f2640 1 -- 172.21.15.22:0/2381428934 wait complete. 2024-09-19T08:14:55.371 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T08:14:55.382 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T08:14:55.382 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T08:14:55.382 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T08:14:55.382 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-19T08:14:55.383 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T08:14:55.383 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-09-19T08:14:55.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-19T08:14:55.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.496+0000 7fbcb6b03640 1 -- 172.21.15.22:0/1808585264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcb0101c30 msgr2=0x7fbcb01020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:55.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.496+0000 7fbcb6b03640 1 --2- 172.21.15.22:0/1808585264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcb0101c30 0x7fbcb01020b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009900 tx=0x7fbca402a410 comp rx=0 tx=0).stop 2024-09-19T08:14:55.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.496+0000 7fbcb6b03640 1 -- 172.21.15.22:0/1808585264 shutdown_connections 2024-09-19T08:14:55.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.496+0000 7fbcb6b03640 1 --2- 172.21.15.22:0/1808585264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcb01025f0 0x7fbcb0108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:55.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.496+0000 7fbcb6b03640 1 --2- 172.21.15.22:0/1808585264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcb0101c30 0x7fbcb01020b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:55.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.496+0000 7fbcb6b03640 1 --2- 172.21.15.22:0/1808585264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb0100a40 0x7fbcb0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:55.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.496+0000 7fbcb6b03640 1 -- 172.21.15.22:0/1808585264 >> 172.21.15.22:0/1808585264 conn(0x7fbcb00fc1e0 msgr2=0x7fbcb00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:55.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.496+0000 7fbcb6b03640 1 -- 172.21.15.22:0/1808585264 shutdown_connections 2024-09-19T08:14:55.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.496+0000 7fbcb6b03640 1 -- 172.21.15.22:0/1808585264 wait complete. 2024-09-19T08:14:55.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.496+0000 7fbcb6b03640 1 Processor -- start 2024-09-19T08:14:55.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb6b03640 1 -- start start 2024-09-19T08:14:55.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb6b03640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb0100a40 0x7fbcb006a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:55.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb6b03640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcb0101c30 0x7fbcb019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:55.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb6b03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcb01025f0 0x7fbcb019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:55.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb6b03640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb010c710 con 0x7fbcb01025f0 2024-09-19T08:14:55.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb6b03640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb010c590 con 0x7fbcb0100a40 2024-09-19T08:14:55.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb6b03640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb010c890 con 0x7fbcb0101c30 2024-09-19T08:14:55.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb50a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcb01025f0 0x7fbcb019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:55.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb50a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcb01025f0 0x7fbcb019f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34772/0 (socket says 172.21.15.22:34772) 2024-09-19T08:14:55.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb50a0640 1 -- 172.21.15.22:0/3887483745 learned_addr learned my addr 172.21.15.22:0/3887483745 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:55.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb489f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb0100a40 0x7fbcb006a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:55.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcaffff640 1 --2- 172.21.15.22:0/3887483745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcb0101c30 0x7fbcb019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:55.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb50a0640 1 -- 172.21.15.22:0/3887483745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcb0101c30 msgr2=0x7fbcb019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:55.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb50a0640 1 --2- 172.21.15.22:0/3887483745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcb0101c30 0x7fbcb019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:55.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb50a0640 1 -- 172.21.15.22:0/3887483745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb0100a40 msgr2=0x7fbcb006a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:55.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb50a0640 1 --2- 172.21.15.22:0/3887483745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb0100a40 0x7fbcb006a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:55.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb50a0640 1 -- 172.21.15.22:0/3887483745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb019f950 con 0x7fbcb01025f0 2024-09-19T08:14:55.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.497+0000 7fbcb50a0640 1 --2- 172.21.15.22:0/3887483745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcb01025f0 0x7fbcb019f090 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbca800eaa0 tx=0x7fbca800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:55.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.498+0000 7fbcadffb640 1 -- 172.21.15.22:0/3887483745 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca800cdb0 con 0x7fbcb01025f0 2024-09-19T08:14:55.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.498+0000 7fbcadffb640 1 -- 172.21.15.22:0/3887483745 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbca8005020 con 0x7fbcb01025f0 2024-09-19T08:14:55.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.498+0000 7fbcb6b03640 1 -- 172.21.15.22:0/3887483745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb019fc40 con 0x7fbcb01025f0 2024-09-19T08:14:55.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.498+0000 7fbcadffb640 1 -- 172.21.15.22:0/3887483745 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca80116b0 con 0x7fbcb01025f0 2024-09-19T08:14:55.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.498+0000 7fbcb6b03640 1 -- 172.21.15.22:0/3887483745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb01a01b0 con 0x7fbcb01025f0 2024-09-19T08:14:55.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.499+0000 7fbcb6b03640 1 -- 172.21.15.22:0/3887483745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb0107d00 con 0x7fbcb01025f0 2024-09-19T08:14:55.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.500+0000 7fbcadffb640 1 -- 172.21.15.22:0/3887483745 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fbca8005560 con 0x7fbcb01025f0 2024-09-19T08:14:55.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.500+0000 7fbcadffb640 1 --2- 172.21.15.22:0/3887483745 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fbc980a3370 0x7fbc980a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:55.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.501+0000 7fbcb489f640 1 --2- 172.21.15.22:0/3887483745 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fbc980a3370 0x7fbc980a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:55.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.501+0000 7fbcadffb640 1 -- 172.21.15.22:0/3887483745 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbca80cfa50 con 0x7fbcb01025f0 2024-09-19T08:14:55.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.501+0000 7fbcb489f640 1 --2- 172.21.15.22:0/3887483745 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fbc980a3370 0x7fbc980a5830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0101a90 tx=0x7fbca0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:55.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.505+0000 7fbcadffb640 1 -- 172.21.15.22:0/3887483745 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbca8016030 con 0x7fbcb01025f0 2024-09-19T08:14:55.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.731+0000 7fbcb6b03640 1 -- 172.21.15.22:0/3887483745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fbcb0062df0 con 0x7fbcb01025f0 2024-09-19T08:14:55.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.929+0000 7fbcadffb640 1 -- 172.21.15.22:0/3887483745 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fbca809c0e0 con 0x7fbcb01025f0 2024-09-19T08:14:55.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.932+0000 7fbcb6b03640 1 -- 172.21.15.22:0/3887483745 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fbc980a3370 msgr2=0x7fbc980a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:55.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.932+0000 7fbcb6b03640 1 --2- 172.21.15.22:0/3887483745 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fbc980a3370 0x7fbc980a5830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0101a90 tx=0x7fbca0005f90 comp rx=0 tx=0).stop 2024-09-19T08:14:55.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.932+0000 7fbcb6b03640 1 -- 172.21.15.22:0/3887483745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcb01025f0 msgr2=0x7fbcb019f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:55.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.932+0000 7fbcb6b03640 1 --2- 172.21.15.22:0/3887483745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcb01025f0 0x7fbcb019f090 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbca800eaa0 tx=0x7fbca800ef60 comp rx=0 tx=0).stop 2024-09-19T08:14:55.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.932+0000 7fbcb6b03640 1 -- 172.21.15.22:0/3887483745 shutdown_connections 2024-09-19T08:14:55.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.932+0000 7fbcb6b03640 1 --2- 172.21.15.22:0/3887483745 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fbc980a3370 0x7fbc980a5830 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:55.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.932+0000 7fbcb6b03640 1 --2- 172.21.15.22:0/3887483745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcb01025f0 0x7fbcb019f090 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:55.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.932+0000 7fbcb6b03640 1 --2- 172.21.15.22:0/3887483745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcb0101c30 0x7fbcb019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:55.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.932+0000 7fbcb6b03640 1 --2- 172.21.15.22:0/3887483745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb0100a40 0x7fbcb006a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:55.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.932+0000 7fbcb6b03640 1 -- 172.21.15.22:0/3887483745 >> 172.21.15.22:0/3887483745 conn(0x7fbcb00fc1e0 msgr2=0x7fbcb0104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:55.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.932+0000 7fbcb6b03640 1 -- 172.21.15.22:0/3887483745 shutdown_connections 2024-09-19T08:14:55.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:55.932+0000 7fbcb6b03640 1 -- 172.21.15.22:0/3887483745 wait complete. 2024-09-19T08:14:55.948 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T08:14:55.974 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T08:14:55.974 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T08:14:55.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:14:55.974 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-19T08:14:55.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:14:55.992 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-09-19T08:14:56.062 INFO:teuthology.orchestra.run.smithi022.stderr:Cannot find device "ceph-brx" 2024-09-19T08:14:56.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:14:56.064 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-19T08:14:56.064 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T08:14:56.064 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:14:56.064 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T08:14:56.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:55 smithi157 ceph-mon[35143]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T08:14:56.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:55 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3887483745' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T08:14:56.253 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:55 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: ignoring --setuser ceph since I am not root 2024-09-19T08:14:56.253 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:55 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: ignoring --setgroup ceph since I am not root 2024-09-19T08:14:56.254 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:55 smithi157 ceph-mgr[36944]: -- 172.21.15.157:0/4265793712 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a57ea50e0 con 0x561a57e82800 2024-09-19T08:14:56.254 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:56.084+0000 7fb259011040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:14:56.254 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:56.140+0000 7fb259011040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:14:56.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:55 smithi022 ceph-mon[30963]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T08:14:56.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:55 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3887483745' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T08:14:56.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:55 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: ignoring --setuser ceph since I am not root 2024-09-19T08:14:56.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:55 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: ignoring --setgroup ceph since I am not root 2024-09-19T08:14:56.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:56 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:56.088+0000 7f98e71df040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:14:56.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:56 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:56.149+0000 7f98e71df040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:14:56.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:55 smithi064 ceph-mon[34900]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T08:14:56.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:55 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3887483745' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T08:14:56.299 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:55 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: ignoring --setuser ceph since I am not root 2024-09-19T08:14:56.299 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:55 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: ignoring --setgroup ceph since I am not root 2024-09-19T08:14:56.299 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:55 smithi064 ceph-mgr[36269]: -- 172.21.15.64:0/1547291331 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55957019b0e0 con 0x559570179000 2024-09-19T08:14:56.299 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:56.087+0000 7f0c6ea2c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:14:56.299 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:56.146+0000 7f0c6ea2c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:14:56.299 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:56.298+0000 7f0c6ea2c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:14:56.379 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:14:56.544 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:56 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:56.310+0000 7f98e71df040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:14:56.544 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:56 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:56.392+0000 7f98e71df040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:14:56.586 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:56.367+0000 7f0c6ea2c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:14:56.587 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:56.520+0000 7f0c6ea2c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:14:56.642 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:56.287+0000 7fb259011040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:14:56.642 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:56.353+0000 7fb259011040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:14:56.642 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:56.499+0000 7fb259011040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:14:56.642 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:56.641+0000 7fb259011040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:14:56.809 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:56 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:56.555+0000 7f98e71df040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:14:56.810 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:56 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:56.712+0000 7f98e71df040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:14:56.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.893+0000 7f1fa918a640 1 -- 172.21.15.22:0/296715608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fa4103730 msgr2=0x7f1fa410bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:56.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.893+0000 7f1fa918a640 1 --2- 172.21.15.22:0/296715608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fa4103730 0x7f1fa410bc20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009900 tx=0x7f1f9802a480 comp rx=0 tx=0).stop 2024-09-19T08:14:56.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.894+0000 7f1fa918a640 1 -- 172.21.15.22:0/296715608 shutdown_connections 2024-09-19T08:14:56.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.894+0000 7f1fa918a640 1 --2- 172.21.15.22:0/296715608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa410c1b0 0x7f1fa410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:56.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.894+0000 7f1fa918a640 1 --2- 172.21.15.22:0/296715608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fa4103730 0x7f1fa410bc20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:56.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.894+0000 7f1fa918a640 1 --2- 172.21.15.22:0/296715608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa4102e10 0x7f1fa41031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:56.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.894+0000 7f1fa918a640 1 -- 172.21.15.22:0/296715608 >> 172.21.15.22:0/296715608 conn(0x7f1fa40fc7d0 msgr2=0x7f1fa40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:56.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.894+0000 7f1fa918a640 1 -- 172.21.15.22:0/296715608 shutdown_connections 2024-09-19T08:14:56.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.894+0000 7f1fa918a640 1 -- 172.21.15.22:0/296715608 wait complete. 2024-09-19T08:14:56.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.894+0000 7f1fa918a640 1 Processor -- start 2024-09-19T08:14:56.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa918a640 1 -- start start 2024-09-19T08:14:56.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa918a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa4102e10 0x7f1fa419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:56.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa918a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fa4103730 0x7f1fa419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:56.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa918a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa410c1b0 0x7f1fa419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:56.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa918a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa4111f80 con 0x7f1fa4103730 2024-09-19T08:14:56.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa918a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa4111e00 con 0x7f1fa4102e10 2024-09-19T08:14:56.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa918a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa4112100 con 0x7f1fa410c1b0 2024-09-19T08:14:56.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa4102e10 0x7f1fa419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:56.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fa4103730 0x7f1fa419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:56.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa4102e10 0x7f1fa419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:47300/0 (socket says 172.21.15.22:47300) 2024-09-19T08:14:56.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa2d9d640 1 -- 172.21.15.22:0/3381489425 learned_addr learned my addr 172.21.15.22:0/3381489425 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:56.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa359e640 1 --2- 172.21.15.22:0/3381489425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa410c1b0 0x7f1fa419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:56.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa2d9d640 1 -- 172.21.15.22:0/3381489425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa410c1b0 msgr2=0x7f1fa419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:56.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa2d9d640 1 --2- 172.21.15.22:0/3381489425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa410c1b0 0x7f1fa419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:56.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa2d9d640 1 -- 172.21.15.22:0/3381489425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fa4103730 msgr2=0x7f1fa419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:56.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa2d9d640 1 --2- 172.21.15.22:0/3381489425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fa4103730 0x7f1fa419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:56.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa2d9d640 1 -- 172.21.15.22:0/3381489425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa419f5d0 con 0x7f1fa4102e10 2024-09-19T08:14:56.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.895+0000 7f1fa359e640 1 --2- 172.21.15.22:0/3381489425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa410c1b0 0x7f1fa419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:14:56.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.896+0000 7f1fa2d9d640 1 --2- 172.21.15.22:0/3381489425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa4102e10 0x7f1fa419a620 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400ea60 tx=0x7f1f9400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:56.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.896+0000 7f1f8bfff640 1 -- 172.21.15.22:0/3381489425 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400cde0 con 0x7f1fa4102e10 2024-09-19T08:14:56.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.896+0000 7f1fa918a640 1 -- 172.21.15.22:0/3381489425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa419f8c0 con 0x7f1fa4102e10 2024-09-19T08:14:56.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.896+0000 7f1f8bfff640 1 -- 172.21.15.22:0/3381489425 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f940035f0 con 0x7f1fa4102e10 2024-09-19T08:14:56.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.896+0000 7f1f8bfff640 1 -- 172.21.15.22:0/3381489425 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94010670 con 0x7f1fa4102e10 2024-09-19T08:14:56.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.896+0000 7f1fa918a640 1 -- 172.21.15.22:0/3381489425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa41a71a0 con 0x7f1fa4102e10 2024-09-19T08:14:56.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.897+0000 7f1fa918a640 1 -- 172.21.15.22:0/3381489425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa41031f0 con 0x7f1fa4102e10 2024-09-19T08:14:56.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.898+0000 7f1f8bfff640 1 -- 172.21.15.22:0/3381489425 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7f1f940037a0 con 0x7f1fa4102e10 2024-09-19T08:14:56.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.899+0000 7f1f8bfff640 1 --2- 172.21.15.22:0/3381489425 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f1f8c0a3290 0x7f1f8c0a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:56.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.899+0000 7f1fa259c640 1 -- 172.21.15.22:0/3381489425 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f1f8c0a3290 msgr2=0x7f1f8c0a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1866276413 2024-09-19T08:14:56.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.899+0000 7f1f8bfff640 1 -- 172.21.15.22:0/3381489425 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1f940cfc20 con 0x7f1fa4102e10 2024-09-19T08:14:56.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.899+0000 7f1fa259c640 1 --2- 172.21.15.22:0/3381489425 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f1f8c0a3290 0x7f1f8c0a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:14:56.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:56.903+0000 7f1f8bfff640 1 -- 172.21.15.22:0/3381489425 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1f9409c940 con 0x7f1fa4102e10 2024-09-19T08:14:56.930 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:56.666+0000 7f0c6ea2c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:14:57.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.100+0000 7f1fa259c640 1 -- 172.21.15.22:0/3381489425 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f1f8c0a3290 msgr2=0x7f1f8c0a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1866276413 2024-09-19T08:14:57.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.100+0000 7f1fa259c640 1 --2- 172.21.15.22:0/3381489425 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f1f8c0a3290 0x7f1f8c0a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:14:57.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:56 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3887483745' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T08:14:57.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:56 smithi022 ceph-mon[30963]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T08:14:57.102 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:57 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:57.056+0000 7f98e71df040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:14:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.139+0000 7f1fa918a640 1 -- 172.21.15.22:0/3381489425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1fa41a73d0 con 0x7f1fa4102e10 2024-09-19T08:14:57.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.140+0000 7f1f8bfff640 1 -- 172.21.15.22:0/3381489425 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f1f9409c240 con 0x7f1fa4102e10 2024-09-19T08:14:57.141 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:14:57.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.142+0000 7f1fa918a640 1 -- 172.21.15.22:0/3381489425 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f1f8c0a3290 msgr2=0x7f1f8c0a5750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:14:57.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.142+0000 7f1fa918a640 1 --2- 172.21.15.22:0/3381489425 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f1f8c0a3290 0x7f1f8c0a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.142+0000 7f1fa918a640 1 -- 172.21.15.22:0/3381489425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa4102e10 msgr2=0x7f1fa419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.142+0000 7f1fa918a640 1 --2- 172.21.15.22:0/3381489425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa4102e10 0x7f1fa419a620 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400ea60 tx=0x7f1f9400ef20 comp rx=0 tx=0).stop 2024-09-19T08:14:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.142+0000 7f1fa918a640 1 -- 172.21.15.22:0/3381489425 shutdown_connections 2024-09-19T08:14:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.142+0000 7f1fa918a640 1 --2- 172.21.15.22:0/3381489425 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f1f8c0a3290 0x7f1f8c0a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.142+0000 7f1fa918a640 1 --2- 172.21.15.22:0/3381489425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa410c1b0 0x7f1fa419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.142+0000 7f1fa918a640 1 --2- 172.21.15.22:0/3381489425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fa4103730 0x7f1fa419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.142+0000 7f1fa918a640 1 --2- 172.21.15.22:0/3381489425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fa4102e10 0x7f1fa419a620 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.142+0000 7f1fa918a640 1 -- 172.21.15.22:0/3381489425 >> 172.21.15.22:0/3381489425 conn(0x7f1fa40fc7d0 msgr2=0x7f1fa4101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.142+0000 7f1fa918a640 1 -- 172.21.15.22:0/3381489425 shutdown_connections 2024-09-19T08:14:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:57.142+0000 7f1fa918a640 1 -- 172.21.15.22:0/3381489425 wait complete. 2024-09-19T08:14:57.199 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:56 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3887483745' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T08:14:57.199 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:56 smithi157 ceph-mon[35143]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T08:14:57.199 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:56.956+0000 7fb259011040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:14:57.200 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:57 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:57.016+0000 7fb259011040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:14:57.200 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:57 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:57.198+0000 7fb259011040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:14:57.248 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:56 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3887483745' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T08:14:57.248 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:56 smithi064 ceph-mon[34900]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T08:14:57.248 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:56.992+0000 7f0c6ea2c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:14:57.248 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:57 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:57.055+0000 7f0c6ea2c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:14:57.506 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:57 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:57.130+0000 7f98e71df040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:14:57.507 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:57 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:57.330+0000 7f98e71df040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:14:57.904 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:14:57.904 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:14:57.904 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-19T08:14:57.904 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T08:14:57.904 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:14:57.904 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:14:57.904 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:14:57.904 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:14:57.904 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:14:57.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:14:57.905 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-19T08:14:57.923 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:14:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:14:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:14:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:14:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-19T08:14:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:14:57.924 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T08:14:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-19T08:14:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-19T08:14:57.924 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T08:14:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-19T08:14:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-19T08:14:57.924 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:14:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:14:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-19T08:14:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T08:14:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42155sec preferred_lft 42155sec 2024-09-19T08:14:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-19T08:14:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:14:57.925 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T08:14:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:14:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-19T08:14:57.926 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T08:14:57.927 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:14:57.927 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-19T08:14:57.927 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2024-09-19T08:14:57.927 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2024-09-19T08:14:57.927 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2024-09-19T08:14:57.927 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T08:14:57.927 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-19T08:14:58.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:14:58.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:14:57 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3381489425' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T08:14:58.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:14:58.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:14:58.139 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:14:58.165 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-19T08:14:58.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:14:58.167 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-19T08:14:58.231 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:14:57 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3381489425' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T08:14:58.232 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:57 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:57.248+0000 7f0c6ea2c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:14:58.232 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:58 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:58.161+0000 7f0c6ea2c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:14:58.232 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:58 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:58.231+0000 7f0c6ea2c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:14:58.238 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-19T08:14:58.238 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:14:58.238 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T08:14:58.238 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T08:14:58.238 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:14:58.239 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:14:58.239 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-19T08:14:58.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T08:14:58.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T08:14:58.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T08:14:58.239 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-19T08:14:58.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:14:57 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3381489425' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T08:14:58.253 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:58 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:58.075+0000 7fb259011040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:14:58.253 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:58 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:58.141+0000 7fb259011040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:14:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:14:58.340 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:58 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:58.286+0000 7f98e71df040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:14:58.429 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:58 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:58.360+0000 7f98e71df040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:14:58.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:14:58.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:14:58.528 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-19T08:14:58.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:14:58.545 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2024-09-19T08:14:58.602 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:14:58.602 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-19T08:14:58.602 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:14:58.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T08:14:58.603 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-19T08:14:58.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:14:58.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:14:58.727 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T08:14:58.727 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:14:58.727 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-19T08:14:58.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T08:14:58.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T08:14:58.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T08:14:58.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T08:14:58.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T08:14:58.728 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-19T08:14:58.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:14:58.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:14:58.862 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:14:58.862 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-19T08:14:58.862 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-19T08:14:58.862 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T08:14:58.862 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-19T08:14:58.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:14:58.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:14:58.943 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:14:58.943 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T08:14:58.959 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:14:58.959 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:14:58.959 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:14:59.016 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-19T08:14:59.016 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-19T08:14:59.016 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/c12f3609-5f1b-47c1-8ca8-c1ddc3b712c0 2024-09-19T08:14:59.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:14:59.016 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/c12f3609-5f1b-47c1-8ca8-c1ddc3b712c0 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-19T08:14:59.111 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:58 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:58.851+0000 7fb259011040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:14:59.111 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:58 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:58.966+0000 7fb259011040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:14:59.119 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-19T08:14:59.119 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-19T08:14:59.242 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:58 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:58.972+0000 7f0c6ea2c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:14:59.242 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:59.093+0000 7f0c6ea2c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:14:59.272 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-19T08:14:59.272 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-19T08:14:59.272 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.64:3300,172.21.15.157:3300" 2024-09-19T08:14:59.272 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with new device syntax: 0@c0ad60c6-765d-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/c12f3609-5f1b-47c1-8ca8-c1ddc3b712c0 2024-09-19T08:14:59.272 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.22:3300/172.21.15.64:3300/172.21.15.157:3300" will pass to kernel 2024-09-19T08:14:59.273 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-19T08:14:59.273 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-19T08:14:59.273 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-19T08:14:59.273 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-19T08:14:59.273 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-19T08:14:59.275 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T08:14:59.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:14:59.275 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:14:59.306 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T08:14:59.391 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:59.110+0000 7fb259011040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:14:59.469 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:59.171+0000 7fb259011040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:14:59.469 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:59.322+0000 7fb259011040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:14:59.469 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:59.390+0000 7fb259011040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:14:59.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T08:14:59.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:14:59.471 INFO:teuthology.orchestra.run.smithi022.stdout:client15150 2024-09-19T08:14:59.471 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:14:59.471 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:14:59.472 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:59 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:59.143+0000 7f98e71df040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:14:59.472 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:59 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:59.268+0000 7f98e71df040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:14:59.490 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T08:14:59.533 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:59.242+0000 7f0c6ea2c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:14:59.533 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:59.307+0000 7f0c6ea2c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:14:59.533 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:59.462+0000 7f0c6ea2c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:14:59.533 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:59.533+0000 7f0c6ea2c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:14:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T08:14:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:14:59.588 INFO:teuthology.orchestra.run.smithi022.stdout:client15150 2024-09-19T08:14:59.588 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T08:14:59.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T08:14:59.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:14:59.647 INFO:teuthology.orchestra.run.smithi022.stdout:client15150 2024-09-19T08:14:59.647 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968.client15150/status') 2024-09-19T08:14:59.673 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:59.450+0000 7fb259011040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:14:59.673 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:59.608+0000 7fb259011040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:14:59.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:14:59.706 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:59 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:59.422+0000 7f98e71df040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:14:59.706 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:59 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:59.487+0000 7f98e71df040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:14:59.706 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:59 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:59.647+0000 7f98e71df040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:14:59.708 INFO:teuthology.orchestra.run.smithi022.stdout:instance: client.15150 (3)192.168.0.1:0/1342503512 2024-09-19T08:14:59.708 INFO:teuthology.orchestra.run.smithi022.stdout:blocklisted: false 2024-09-19T08:14:59.708 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records in 2024-09-19T08:14:59.708 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records out 2024-09-19T08:14:59.708 INFO:teuthology.orchestra.run.smithi022.stderr:70 bytes copied, 2.3436e-05 s, 3.0 MB/s 2024-09-19T08:14:59.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:14:59.716 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T08:14:59.725 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T08:14:59.725 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-09-19T08:14:59.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-19T08:14:59.828 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:59.595+0000 7f0c6ea2c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:14:59.828 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:59.760+0000 7f0c6ea2c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:14:59.828 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:59.828+0000 7f0c6ea2c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:14:59.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.831+0000 7fdbeb749640 1 -- 172.21.15.22:0/3800465656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe4100a40 msgr2=0x7fdbe4100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:59.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.831+0000 7fdbeb749640 1 --2- 172.21.15.22:0/3800465656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe4100a40 0x7fdbe4100e40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009900 tx=0x7fdbe002a410 comp rx=0 tx=0).stop 2024-09-19T08:14:59.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.831+0000 7fdbeb749640 1 -- 172.21.15.22:0/3800465656 shutdown_connections 2024-09-19T08:14:59.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.831+0000 7fdbeb749640 1 --2- 172.21.15.22:0/3800465656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe41025f0 0x7fdbe4108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:59.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.831+0000 7fdbeb749640 1 --2- 172.21.15.22:0/3800465656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe4101c30 0x7fdbe41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:59.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.831+0000 7fdbeb749640 1 --2- 172.21.15.22:0/3800465656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe4100a40 0x7fdbe4100e40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:59.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.831+0000 7fdbeb749640 1 -- 172.21.15.22:0/3800465656 >> 172.21.15.22:0/3800465656 conn(0x7fdbe40fc1e0 msgr2=0x7fdbe40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:14:59.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.831+0000 7fdbeb749640 1 -- 172.21.15.22:0/3800465656 shutdown_connections 2024-09-19T08:14:59.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.831+0000 7fdbeb749640 1 -- 172.21.15.22:0/3800465656 wait complete. 2024-09-19T08:14:59.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.832+0000 7fdbeb749640 1 Processor -- start 2024-09-19T08:14:59.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.832+0000 7fdbeb749640 1 -- start start 2024-09-19T08:14:59.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.832+0000 7fdbeb749640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe4100a40 0x7fdbe406a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:59.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.832+0000 7fdbeb749640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe4101c30 0x7fdbe419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:59.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.832+0000 7fdbeb749640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe41025f0 0x7fdbe419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:59.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.832+0000 7fdbeb749640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe410c6a0 con 0x7fdbe4100a40 2024-09-19T08:14:59.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.832+0000 7fdbeb749640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe410c520 con 0x7fdbe41025f0 2024-09-19T08:14:59.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.832+0000 7fdbeb749640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe410c820 con 0x7fdbe4101c30 2024-09-19T08:14:59.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.832+0000 7fdbe94e5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe4100a40 0x7fdbe406a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:59.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.832+0000 7fdbe94e5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe4100a40 0x7fdbe406a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42188/0 (socket says 172.21.15.22:42188) 2024-09-19T08:14:59.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.832+0000 7fdbe94e5640 1 -- 172.21.15.22:0/389510102 learned_addr learned my addr 172.21.15.22:0/389510102 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:14:59.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.832+0000 7fdbe8ce4640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe4101c30 0x7fdbe419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:59.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.832+0000 7fdbe9ce6640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe41025f0 0x7fdbe419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:14:59.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.833+0000 7fdbe94e5640 1 -- 172.21.15.22:0/389510102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe4101c30 msgr2=0x7fdbe419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:59.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.833+0000 7fdbe94e5640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe4101c30 0x7fdbe419cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:59.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.833+0000 7fdbe94e5640 1 -- 172.21.15.22:0/389510102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe41025f0 msgr2=0x7fdbe419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:14:59.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.833+0000 7fdbe94e5640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe41025f0 0x7fdbe419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:14:59.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.833+0000 7fdbe94e5640 1 -- 172.21.15.22:0/389510102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe419fa40 con 0x7fdbe4100a40 2024-09-19T08:14:59.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.833+0000 7fdbe8ce4640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe4101c30 0x7fdbe419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:14:59.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.833+0000 7fdbe9ce6640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe41025f0 0x7fdbe419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:14:59.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.833+0000 7fdbe94e5640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe4100a40 0x7fdbe406a410 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0031610 tx=0x7fdbe0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:14:59.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.833+0000 7fdbda7fc640 1 -- 172.21.15.22:0/389510102 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe003f030 con 0x7fdbe4100a40 2024-09-19T08:14:59.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.833+0000 7fdbda7fc640 1 -- 172.21.15.22:0/389510102 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe0003ad0 con 0x7fdbe4100a40 2024-09-19T08:14:59.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.833+0000 7fdbda7fc640 1 -- 172.21.15.22:0/389510102 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0035070 con 0x7fdbe4100a40 2024-09-19T08:14:59.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.833+0000 7fdbeb749640 1 -- 172.21.15.22:0/389510102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe419fc70 con 0x7fdbe4100a40 2024-09-19T08:14:59.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.833+0000 7fdbeb749640 1 -- 172.21.15.22:0/389510102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe41a01b0 con 0x7fdbe4100a40 2024-09-19T08:14:59.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.834+0000 7fdbeb749640 1 -- 172.21.15.22:0/389510102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe4107d60 con 0x7fdbe4100a40 2024-09-19T08:14:59.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.836+0000 7fdbda7fc640 1 -- 172.21.15.22:0/389510102 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7fdbe0042030 con 0x7fdbe4100a40 2024-09-19T08:14:59.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.837+0000 7fdbda7fc640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdbd00a33c0 0x7fdbd00a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:14:59.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.837+0000 7fdbe8ce4640 1 -- 172.21.15.22:0/389510102 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdbd00a33c0 msgr2=0x7fdbd00a5880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1866276413 2024-09-19T08:14:59.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.838+0000 7fdbda7fc640 1 -- 172.21.15.22:0/389510102 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdbe00f0130 con 0x7fdbe4100a40 2024-09-19T08:14:59.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.838+0000 7fdbe8ce4640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdbd00a33c0 0x7fdbd00a5880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:14:59.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:14:59.840+0000 7fdbda7fc640 1 -- 172.21.15.22:0/389510102 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdbe00bc800 con 0x7fdbe4100a40 2024-09-19T08:14:59.942 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:59.672+0000 7fb259011040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:14:59.943 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:59.739+0000 7fb259011040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:14:59.943 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:59.821+0000 7fb259011040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:14:59.943 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:59.882+0000 7fb259011040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:14:59.943 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:14:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:14:59.941+0000 7fb259011040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:14:59.960 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:59 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:59.722+0000 7f98e71df040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:14:59.960 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:59 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:59.786+0000 7f98e71df040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:14:59.960 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:14:59 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:14:59.958+0000 7f98e71df040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:15:00.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.039+0000 7fdbe8ce4640 1 -- 172.21.15.22:0/389510102 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdbd00a33c0 msgr2=0x7fdbd00a5880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1866276413 2024-09-19T08:15:00.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.039+0000 7fdbe8ce4640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdbd00a33c0 0x7fdbd00a5880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:15:00.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.057+0000 7fdbeb749640 1 -- 172.21.15.22:0/389510102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fdbe402da10 con 0x7fdbe4100a40 2024-09-19T08:15:00.086 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:59.897+0000 7f0c6ea2c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:15:00.087 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:14:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:14:59.983+0000 7f0c6ea2c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:15:00.087 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:15:00 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:15:00.045+0000 7f0c6ea2c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:15:00.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.109+0000 7fdbda7fc640 1 -- 172.21.15.22:0/389510102 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7fdbe00c0b00 con 0x7fdbe4100a40 2024-09-19T08:15:00.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.111+0000 7fdbeb749640 1 -- 172.21.15.22:0/389510102 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdbd00a33c0 msgr2=0x7fdbd00a5880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:15:00.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.111+0000 7fdbeb749640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdbd00a33c0 0x7fdbd00a5880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:00.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.111+0000 7fdbeb749640 1 -- 172.21.15.22:0/389510102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe4100a40 msgr2=0x7fdbe406a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:00.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.111+0000 7fdbeb749640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe4100a40 0x7fdbe406a410 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0031610 tx=0x7fdbe0004270 comp rx=0 tx=0).stop 2024-09-19T08:15:00.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.111+0000 7fdbeb749640 1 -- 172.21.15.22:0/389510102 shutdown_connections 2024-09-19T08:15:00.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.111+0000 7fdbeb749640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7fdbd00a33c0 0x7fdbd00a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:00.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.111+0000 7fdbeb749640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbe41025f0 0x7fdbe419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:00.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.111+0000 7fdbeb749640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbe4101c30 0x7fdbe419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:00.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.111+0000 7fdbeb749640 1 --2- 172.21.15.22:0/389510102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe4100a40 0x7fdbe406a410 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:00.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.111+0000 7fdbeb749640 1 -- 172.21.15.22:0/389510102 >> 172.21.15.22:0/389510102 conn(0x7fdbe40fc1e0 msgr2=0x7fdbe4104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:00.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.111+0000 7fdbeb749640 1 -- 172.21.15.22:0/389510102 shutdown_connections 2024-09-19T08:15:00.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.111+0000 7fdbeb749640 1 -- 172.21.15.22:0/389510102 wait complete. 2024-09-19T08:15:00.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-19T08:15:00.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.234+0000 7f7d59d2c640 1 -- 172.21.15.22:0/1564827211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d54104ef0 msgr2=0x7f7d541072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:00.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.234+0000 7f7d59d2c640 1 --2- 172.21.15.22:0/1564827211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d54104ef0 0x7f7d541072e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009900 tx=0x7f7d4802a410 comp rx=0 tx=0).stop 2024-09-19T08:15:00.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.234+0000 7f7d59d2c640 1 -- 172.21.15.22:0/1564827211 shutdown_connections 2024-09-19T08:15:00.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.234+0000 7f7d59d2c640 1 --2- 172.21.15.22:0/1564827211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d54107820 0x7f7d54109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:00.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.234+0000 7f7d59d2c640 1 --2- 172.21.15.22:0/1564827211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d54104ef0 0x7f7d541072e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:00.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.234+0000 7f7d59d2c640 1 --2- 172.21.15.22:0/1564827211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d541025c0 0x7f7d541049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:00.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.234+0000 7f7d59d2c640 1 -- 172.21.15.22:0/1564827211 >> 172.21.15.22:0/1564827211 conn(0x7f7d540fc1e0 msgr2=0x7f7d540fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:00.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.234+0000 7f7d59d2c640 1 -- 172.21.15.22:0/1564827211 shutdown_connections 2024-09-19T08:15:00.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.234+0000 7f7d59d2c640 1 -- 172.21.15.22:0/1564827211 wait complete. 2024-09-19T08:15:00.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.235+0000 7f7d59d2c640 1 Processor -- start 2024-09-19T08:15:00.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.235+0000 7f7d59d2c640 1 -- start start 2024-09-19T08:15:00.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.235+0000 7f7d59d2c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d541025c0 0x7f7d54197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:00.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.235+0000 7f7d59d2c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d54104ef0 0x7f7d541983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:00.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.235+0000 7f7d59d2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d54107820 0x7f7d5419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:00.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.235+0000 7f7d59d2c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5410d7f0 con 0x7f7d54104ef0 2024-09-19T08:15:00.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.235+0000 7f7d59d2c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5410d670 con 0x7f7d54107820 2024-09-19T08:15:00.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.235+0000 7f7d59d2c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5410d970 con 0x7f7d541025c0 2024-09-19T08:15:00.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d52ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d54104ef0 0x7f7d541983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:00.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d52ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d54104ef0 0x7f7d541983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42202/0 (socket says 172.21.15.22:42202) 2024-09-19T08:15:00.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d537fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d541025c0 0x7f7d54197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:00.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d52ffd640 1 -- 172.21.15.22:0/75712111 learned_addr learned my addr 172.21.15.22:0/75712111 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:00.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d53fff640 1 --2- 172.21.15.22:0/75712111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d54107820 0x7f7d5419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:00.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d52ffd640 1 -- 172.21.15.22:0/75712111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d541025c0 msgr2=0x7f7d54197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:00.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d52ffd640 1 --2- 172.21.15.22:0/75712111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d541025c0 0x7f7d54197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:00.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d52ffd640 1 -- 172.21.15.22:0/75712111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d54107820 msgr2=0x7f7d5419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:00.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d52ffd640 1 --2- 172.21.15.22:0/75712111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d54107820 0x7f7d5419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:00.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d52ffd640 1 -- 172.21.15.22:0/75712111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5419fa90 con 0x7f7d54104ef0 2024-09-19T08:15:00.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d52ffd640 1 --2- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d54104ef0 0x7f7d541983b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7d48031610 tx=0x7f7d48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:00.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d50ff9640 1 -- 172.21.15.22:0/75712111 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4803f030 con 0x7f7d54104ef0 2024-09-19T08:15:00.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d59d2c640 1 -- 172.21.15.22:0/75712111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5419fd20 con 0x7f7d54104ef0 2024-09-19T08:15:00.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d50ff9640 1 -- 172.21.15.22:0/75712111 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d48003ad0 con 0x7f7d54104ef0 2024-09-19T08:15:00.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d50ff9640 1 -- 172.21.15.22:0/75712111 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48035070 con 0x7f7d54104ef0 2024-09-19T08:15:00.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.236+0000 7f7d59d2c640 1 -- 172.21.15.22:0/75712111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d541a01c0 con 0x7f7d54104ef0 2024-09-19T08:15:00.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.237+0000 7f7d59d2c640 1 -- 172.21.15.22:0/75712111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d540fdcd0 con 0x7f7d54104ef0 2024-09-19T08:15:00.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.239+0000 7f7d50ff9640 1 -- 172.21.15.22:0/75712111 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7f7d48042030 con 0x7f7d54104ef0 2024-09-19T08:15:00.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.240+0000 7f7d50ff9640 1 --2- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7d3c0a3340 0x7f7d3c0a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:00.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.240+0000 7f7d537fe640 1 -- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7d3c0a3340 msgr2=0x7f7d3c0a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1866276413 2024-09-19T08:15:00.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.240+0000 7f7d50ff9640 1 -- 172.21.15.22:0/75712111 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7d480f0200 con 0x7f7d54104ef0 2024-09-19T08:15:00.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.240+0000 7f7d537fe640 1 --2- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7d3c0a3340 0x7f7d3c0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:15:00.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.243+0000 7f7d50ff9640 1 -- 172.21.15.22:0/75712111 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7d480bc890 con 0x7f7d54104ef0 2024-09-19T08:15:00.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:00 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/389510102' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T08:15:00.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:00 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/389510102' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T08:15:00.256 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:00.028+0000 7f98e71df040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:15:00.256 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:00.098+0000 7f98e71df040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:15:00.256 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:00.189+0000 7f98e71df040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:15:00.256 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:00.255+0000 7f98e71df040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:15:00.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.441+0000 7f7d537fe640 1 -- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7d3c0a3340 msgr2=0x7f7d3c0a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1866276413 2024-09-19T08:15:00.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.441+0000 7f7d537fe640 1 --2- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7d3c0a3340 0x7f7d3c0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:15:00.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.446+0000 7f7d59d2c640 1 -- 172.21.15.22:0/75712111 --> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f7d54062790 con 0x7f7d3c0a3340 2024-09-19T08:15:00.506 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:00.319+0000 7f98e71df040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:15:00.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:00 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/389510102' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T08:15:00.587 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:15:00 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:15:00.106+0000 7f0c6ea2c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:15:00.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.842+0000 7f7d537fe640 1 -- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7d3c0a3340 msgr2=0x7f7d3c0a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1866276413 2024-09-19T08:15:00.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:00.842+0000 7f7d537fe640 1 --2- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7d3c0a3340 0x7f7d3c0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T08:15:01.003 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:15:00 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z[36940]: 2024-09-19T08:15:00.731+0000 7fb259011040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:15:01.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:01.127+0000 7f7d50ff9640 1 -- 172.21.15.22:0/75712111 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 20) ==== 138372+0+0 (secure 0 0 0) 0x7f7d4803b460 con 0x7f7d54104ef0 2024-09-19T08:15:01.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:01.175+0000 7f7d50ff9640 1 -- 172.21.15.22:0/75712111 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 21) ==== 138040+0+0 (secure 0 0 0) 0x7f7d480edd40 con 0x7f7d54104ef0 2024-09-19T08:15:01.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:01.175+0000 7f7d50ff9640 1 -- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7d3c0a3340 msgr2=0x7f7d3c0a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:15:01.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:01.175+0000 7f7d50ff9640 1 --2- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:6800/1866276413,v1:172.21.15.22:6801/1866276413] conn(0x7f7d3c0a3340 0x7f7d3c0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:01.336 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:15:00 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y[36265]: 2024-09-19T08:15:00.935+0000 7f0c6ea2c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:15:01.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:01 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/389510102' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T08:15:01.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:01 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:15:01.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:01 smithi064 ceph-mon[34900]: Standby manager daemon z restarted 2024-09-19T08:15:01.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:01 smithi064 ceph-mon[34900]: Standby manager daemon z started 2024-09-19T08:15:01.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:01 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.157:0/1946674196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:15:01.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:01 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.157:0/1946674196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:15:01.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:01 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.157:0/1946674196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:15:01.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:01 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.157:0/1946674196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:15:01.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:01 smithi064 ceph-mon[34900]: Standby manager daemon y restarted 2024-09-19T08:15:01.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:01 smithi064 ceph-mon[34900]: Standby manager daemon y started 2024-09-19T08:15:01.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:01 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.64:0/239148705' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:15:01.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:01 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.64:0/239148705' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:15:01.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:01 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.64:0/239148705' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:15:01.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:01 smithi064 ceph-mon[34900]: from='mgr.? 172.21.15.64:0/239148705' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:15:01.400 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:01.162+0000 7f98e71df040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:15:01.400 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:01.399+0000 7f9852333640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:01.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/389510102' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T08:15:01.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:15:01.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-mon[30963]: Standby manager daemon z restarted 2024-09-19T08:15:01.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-mon[30963]: Standby manager daemon z started 2024-09-19T08:15:01.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.157:0/1946674196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:15:01.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.157:0/1946674196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:15:01.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.157:0/1946674196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:15:01.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.157:0/1946674196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:15:01.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-mon[30963]: Standby manager daemon y restarted 2024-09-19T08:15:01.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-mon[30963]: Standby manager daemon y started 2024-09-19T08:15:01.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.64:0/239148705' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:15:01.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.64:0/239148705' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:15:01.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.64:0/239148705' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:15:01.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-mon[30963]: from='mgr.? 172.21.15.64:0/239148705' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:15:01.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:01 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/389510102' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T08:15:01.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:01 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:15:01.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:01 smithi157 ceph-mon[35143]: Standby manager daemon z restarted 2024-09-19T08:15:01.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:01 smithi157 ceph-mon[35143]: Standby manager daemon z started 2024-09-19T08:15:01.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:01 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.157:0/1946674196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:15:01.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:01 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.157:0/1946674196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:15:01.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:01 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.157:0/1946674196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:15:01.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:01 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.157:0/1946674196' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:15:01.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:01 smithi157 ceph-mon[35143]: Standby manager daemon y restarted 2024-09-19T08:15:01.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:01 smithi157 ceph-mon[35143]: Standby manager daemon y started 2024-09-19T08:15:01.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:01 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.64:0/239148705' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:15:01.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:01 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.64:0/239148705' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:15:01.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:01 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.64:0/239148705' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:15:01.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:01 smithi157 ceph-mon[35143]: from='mgr.? 172.21.15.64:0/239148705' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:15:01.760 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:01.399+0000 7f9852333640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:01.760 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:01.399+0000 7f9852333640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:01.760 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:01.399+0000 7f9852333640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:01.760 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:01.399+0000 7f9852333640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:01.760 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:01.399+0000 7f984f32d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:01.760 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:01.399+0000 7f984f32d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:01.760 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:01.399+0000 7f984f32d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:01.760 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:01.399+0000 7f984f32d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:01.760 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:01.399+0000 7f984f32d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:02.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.181+0000 7f7d50ff9640 1 -- 172.21.15.22:0/75712111 <== mon.0 v2:172.21.15.22:3300/0 9 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f7d48003c80 con 0x7f7d54104ef0 2024-09-19T08:15:02.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.182+0000 7f7d50ff9640 1 --2- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7d3c035280 0x7f7d3c035660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:02.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.182+0000 7f7d50ff9640 1 -- 172.21.15.22:0/75712111 --> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f7d480ba730 con 0x7f7d3c035280 2024-09-19T08:15:02.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.185+0000 7f7d537fe640 1 --2- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7d3c035280 0x7f7d3c035660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:02.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.188+0000 7f7d537fe640 1 --2- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7d3c035280 0x7f7d3c035660 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7d440097d0 tx=0x7f7d44005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:02.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.204+0000 7f7d50ff9640 1 -- 172.21.15.22:0/75712111 <== mgr.15141 v2:172.21.15.22:6800/497251935 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f7d480ba730 con 0x7f7d3c035280 2024-09-19T08:15:02.205 INFO:teuthology.orchestra.run.smithi022.stdout:{} 2024-09-19T08:15:02.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.206+0000 7f7d59d2c640 1 -- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7d3c035280 msgr2=0x7f7d3c035660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:02.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.206+0000 7f7d59d2c640 1 --2- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7d3c035280 0x7f7d3c035660 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7d440097d0 tx=0x7f7d44005f90 comp rx=0 tx=0).stop 2024-09-19T08:15:02.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.206+0000 7f7d59d2c640 1 -- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d54104ef0 msgr2=0x7f7d541983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:02.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.206+0000 7f7d59d2c640 1 --2- 172.21.15.22:0/75712111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d54104ef0 0x7f7d541983b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7d48031610 tx=0x7f7d48004270 comp rx=0 tx=0).stop 2024-09-19T08:15:02.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.206+0000 7f7d53fff640 1 -- 172.21.15.22:0/75712111 reap_dead start 2024-09-19T08:15:02.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.206+0000 7f7d59d2c640 1 -- 172.21.15.22:0/75712111 shutdown_connections 2024-09-19T08:15:02.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.206+0000 7f7d59d2c640 1 -- 172.21.15.22:0/75712111 >> 172.21.15.22:0/75712111 conn(0x7f7d540fc1e0 msgr2=0x7f7d54102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:02.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.206+0000 7f7d59d2c640 1 -- 172.21.15.22:0/75712111 shutdown_connections 2024-09-19T08:15:02.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.206+0000 7f7d59d2c640 1 -- 172.21.15.22:0/75712111 wait complete. 2024-09-19T08:15:02.218 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T08:15:02.229 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-19T08:15:02.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:15:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.376+0000 7fa318c0a640 1 -- 172.21.15.22:0/3085578881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa314100a10 msgr2=0x7fa314100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.376+0000 7fa318c0a640 1 --2- 172.21.15.22:0/3085578881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa314100a10 0x7fa314100e10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa304009900 tx=0x7fa30402a3f0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.376+0000 7fa318c0a640 1 -- 172.21.15.22:0/3085578881 shutdown_connections 2024-09-19T08:15:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.376+0000 7fa318c0a640 1 --2- 172.21.15.22:0/3085578881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3141025c0 0x7fa314108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.376+0000 7fa318c0a640 1 --2- 172.21.15.22:0/3085578881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa314101c00 0x7fa314102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.376+0000 7fa318c0a640 1 --2- 172.21.15.22:0/3085578881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa314100a10 0x7fa314100e10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.376+0000 7fa318c0a640 1 -- 172.21.15.22:0/3085578881 >> 172.21.15.22:0/3085578881 conn(0x7fa3140fc170 msgr2=0x7fa3140fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.376+0000 7fa318c0a640 1 -- 172.21.15.22:0/3085578881 shutdown_connections 2024-09-19T08:15:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.376+0000 7fa318c0a640 1 -- 172.21.15.22:0/3085578881 wait complete. 2024-09-19T08:15:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.377+0000 7fa318c0a640 1 Processor -- start 2024-09-19T08:15:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.377+0000 7fa318c0a640 1 -- start start 2024-09-19T08:15:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.377+0000 7fa318c0a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa314100a10 0x7fa314197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.377+0000 7fa318c0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa314101c00 0x7fa3141983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.377+0000 7fa318c0a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3141025c0 0x7fa31419f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.377+0000 7fa318c0a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa31410c690 con 0x7fa314101c00 2024-09-19T08:15:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.377+0000 7fa318c0a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa31410c510 con 0x7fa3141025c0 2024-09-19T08:15:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.377+0000 7fa318c0a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa31410c810 con 0x7fa314100a10 2024-09-19T08:15:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa311d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa314101c00 0x7fa3141983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa312d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3141025c0 0x7fa31419f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa311d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa314101c00 0x7fa3141983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42378/0 (socket says 172.21.15.22:42378) 2024-09-19T08:15:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa312d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3141025c0 0x7fa31419f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:60850/0 (socket says 172.21.15.22:60850) 2024-09-19T08:15:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa31259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa314100a10 0x7fa314197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa311d9b640 1 -- 172.21.15.22:0/3154505793 learned_addr learned my addr 172.21.15.22:0/3154505793 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa311d9b640 1 -- 172.21.15.22:0/3154505793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa314100a10 msgr2=0x7fa314197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa311d9b640 1 --2- 172.21.15.22:0/3154505793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa314100a10 0x7fa314197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa311d9b640 1 -- 172.21.15.22:0/3154505793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3141025c0 msgr2=0x7fa31419f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa311d9b640 1 --2- 172.21.15.22:0/3154505793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3141025c0 0x7fa31419f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa311d9b640 1 -- 172.21.15.22:0/3154505793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31419fab0 con 0x7fa314101c00 2024-09-19T08:15:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa312d9d640 1 --2- 172.21.15.22:0/3154505793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3141025c0 0x7fa31419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa311d9b640 1 --2- 172.21.15.22:0/3154505793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa314101c00 0x7fa3141983d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa30800e9f0 tx=0x7fa30800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa3037fe640 1 -- 172.21.15.22:0/3154505793 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30800cd80 con 0x7fa314101c00 2024-09-19T08:15:02.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa3037fe640 1 -- 172.21.15.22:0/3154505793 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa308010910 con 0x7fa314101c00 2024-09-19T08:15:02.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa318c0a640 1 -- 172.21.15.22:0/3154505793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31419fda0 con 0x7fa314101c00 2024-09-19T08:15:02.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa3037fe640 1 -- 172.21.15.22:0/3154505793 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308018990 con 0x7fa314101c00 2024-09-19T08:15:02.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.378+0000 7fa318c0a640 1 -- 172.21.15.22:0/3154505793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3141a0260 con 0x7fa314101c00 2024-09-19T08:15:02.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.380+0000 7fa318c0a640 1 -- 172.21.15.22:0/3154505793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3141a0490 con 0x7fa314101c00 2024-09-19T08:15:02.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.381+0000 7fa3037fe640 1 -- 172.21.15.22:0/3154505793 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fa308010ac0 con 0x7fa314101c00 2024-09-19T08:15:02.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.381+0000 7fa3037fe640 1 --2- 172.21.15.22:0/3154505793 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa2fc0a3050 0x7fa2fc0a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:02.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.382+0000 7fa31259c640 1 --2- 172.21.15.22:0/3154505793 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa2fc0a3050 0x7fa2fc0a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:02.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.382+0000 7fa3037fe640 1 -- 172.21.15.22:0/3154505793 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa3080cf980 con 0x7fa314101c00 2024-09-19T08:15:02.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.382+0000 7fa31259c640 1 --2- 172.21.15.22:0/3154505793 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa2fc0a3050 0x7fa2fc0a5510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa304005ab0 tx=0x7fa304000bc0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:02.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.386+0000 7fa3037fe640 1 -- 172.21.15.22:0/3154505793 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa30809c1f0 con 0x7fa314101c00 2024-09-19T08:15:02.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-19T08:15:02.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: Active manager daemon x restarted 2024-09-19T08:15:02.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: Activating manager daemon x 2024-09-19T08:15:02.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T08:15:02.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: mgrmap e21: x(active, starting, since 0.0127344s), standbys: z, y 2024-09-19T08:15:02.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:15:02.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:15:02.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:15:02.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T08:15:02.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T08:15:02.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:15:02.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:15:02.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:15:02.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:15:02.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:15:02.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:15:02.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:15:02.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:15:02.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:15:02.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:15:02.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:15:02.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:15:02.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:15:02.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:15:02.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:15:02.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:15:02.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:15:02.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:15:02.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:15:02.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:15:02.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:15:02.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:15:02.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:15:02.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:15:02.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:15:02.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:15:02.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:15:02.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:15:02.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: Manager daemon x is now available 2024-09-19T08:15:02.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:02.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:15:02.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:15:02.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:15:02.428 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:02 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:15:02.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-19T08:15:02.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: Active manager daemon x restarted 2024-09-19T08:15:02.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: Activating manager daemon x 2024-09-19T08:15:02.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T08:15:02.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: mgrmap e21: x(active, starting, since 0.0127344s), standbys: z, y 2024-09-19T08:15:02.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:15:02.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:15:02.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:15:02.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T08:15:02.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T08:15:02.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:15:02.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:15:02.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:15:02.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:15:02.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:15:02.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:15:02.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:15:02.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:15:02.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:15:02.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:15:02.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:15:02.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:15:02.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:15:02.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:15:02.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:15:02.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:15:02.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:15:02.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:15:02.489 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:02.194+0000 7f984c327640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:02.489 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:02.194+0000 7f984c327640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:02.489 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:02.194+0000 7f984c327640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:02.490 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:02.194+0000 7f984c327640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:02.490 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:02.194+0000 7f984c327640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:15:02.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:15:02.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:15:02.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:15:02.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:15:02.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:15:02.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:15:02.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:15:02.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:15:02.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:15:02.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:15:02.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: Manager daemon x is now available 2024-09-19T08:15:02.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:02.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:15:02.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:15:02.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:15:02.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:02 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:15:02.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.586+0000 7fa318c0a640 1 -- 172.21.15.22:0/3154505793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3141a0ca0 con 0x7fa314101c00 2024-09-19T08:15:02.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-19T08:15:02.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: Active manager daemon x restarted 2024-09-19T08:15:02.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: Activating manager daemon x 2024-09-19T08:15:02.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T08:15:02.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: mgrmap e21: x(active, starting, since 0.0127344s), standbys: z, y 2024-09-19T08:15:02.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:15:02.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:15:02.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:15:02.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T08:15:02.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T08:15:02.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:15:02.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:15:02.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:15:02.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:15:02.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:15:02.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:15:02.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:15:02.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:15:02.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:15:02.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:15:02.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:15:02.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:15:02.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:15:02.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:15:02.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:15:02.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:15:02.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:15:02.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:15:02.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:15:02.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:15:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:15:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:15:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:15:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:15:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:15:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:15:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:15:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:15:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: Manager daemon x is now available 2024-09-19T08:15:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:15:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:15:02.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:15:02.591 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:02 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:15:02.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.587+0000 7fa3037fe640 1 -- 172.21.15.22:0/3154505793 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+23100 (secure 0 0 0) 0x7fa3080a04f0 con 0x7fa314101c00 2024-09-19T08:15:02.592 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:02.592 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":86,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","created":"2024-09-19T08:05:19.476952+0000","modified":"2024-09-19T08:15:01.165514+0000","last_up_change":"2024-09-19T08:12:10.339586+0000","last_in_change":"2024-09-19T08:11:47.653576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:08:25.175759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:12:57.608787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:12:59.824327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ed67f2d-9b28-427a-b69f-676e7a2abcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6803","nonce":3442715307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6805","nonce":3442715307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6809","nonce":3442715307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6807","nonce":3442715307}]},"public_addr":"172.21.15.22:6803/3442715307","cluster_addr":"172.21.15.22:6805/3442715307","heartbeat_back_addr":"172.21.15.22:6809/3442715307","heartbeat_front_addr":"172.21.15.22:6807/3442715307","state":["exists","up"]},{"osd":1,"uuid":"bff3e283-4ab1-49d7-86d8-45d361995122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6801","nonce":2379700228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6803","nonce":2379700228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6807","nonce":2379700228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6805","nonce":2379700228}]},"public_addr":"172.21.15.64:6801/2379700228","cluster_addr":"172.21.15.64:6803/2379700228","heartbeat_back_addr":"172.21.15.64:6807/2379700228","heartbeat_front_addr":"172.21.15.64:6805/2379700228","state":["exists","up"]},{"osd":2,"uuid":"81421d08-7dc6-419b-a566-dee1e1e13b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6801","nonce":3363263867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6803","nonce":3363263867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6807","nonce":3363263867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6805","nonce":3363263867}]},"public_addr":"172.21.15.157:6801/3363263867","cluster_addr":"172.21.15.157:6803/3363263867","heartbeat_back_addr":"172.21.15.157:6807/3363263867","heartbeat_front_addr":"172.21.15.157:6805/3363263867","state":["exists","up"]},{"osd":3,"uuid":"34debfd0-3226-4d78-b9e6-e03dbc5190f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6811","nonce":4225229456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6813","nonce":4225229456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6817","nonce":4225229456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6815","nonce":4225229456}]},"public_addr":"172.21.15.22:6811/4225229456","cluster_addr":"172.21.15.22:6813/4225229456","heartbeat_back_addr":"172.21.15.22:6817/4225229456","heartbeat_front_addr":"172.21.15.22:6815/4225229456","state":["exists","up"]},{"osd":4,"uuid":"1293332c-d772-4ca9-bf76-79e0b29451ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6809","nonce":1858090423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6811","nonce":1858090423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6815","nonce":1858090423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6813","nonce":1858090423}]},"public_addr":"172.21.15.64:6809/1858090423","cluster_addr":"172.21.15.64:6811/1858090423","heartbeat_back_addr":"172.21.15.64:6815/1858090423","heartbeat_front_addr":"172.21.15.64:6813/1858090423","state":["exists","up"]},{"osd":5,"uuid":"240a6c42-89a2-41c1-948c-b09914b5722e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6809","nonce":2900701212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2900701212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2900701212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2900701212}]},"public_addr":"172.21.15.157:6809/2900701212","cluster_addr":"172.21.15.157:6811/2900701212","heartbeat_back_addr":"172.21.15.157:6815/2900701212","heartbeat_front_addr":"172.21.15.157:6813/2900701212","state":["exists","up"]},{"osd":6,"uuid":"80199030-109e-41c4-bc1c-731755464770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6819","nonce":4264426242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6821","nonce":4264426242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6825","nonce":4264426242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6823","nonce":4264426242}]},"public_addr":"172.21.15.22:6819/4264426242","cluster_addr":"172.21.15.22:6821/4264426242","heartbeat_back_addr":"172.21.15.22:6825/4264426242","heartbeat_front_addr":"172.21.15.22:6823/4264426242","state":["exists","up"]},{"osd":7,"uuid":"dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6817","nonce":3980224103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6819","nonce":3980224103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6823","nonce":3980224103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6821","nonce":3980224103}]},"public_addr":"172.21.15.64:6817/3980224103","cluster_addr":"172.21.15.64:6819/3980224103","heartbeat_back_addr":"172.21.15.64:6823/3980224103","heartbeat_front_addr":"172.21.15.64:6821/3980224103","state":["exists","up"]},{"osd":8,"uuid":"764b1e6e-48c9-49f9-8124-bc13bcba3a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6817","nonce":973678477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6819","nonce":973678477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6823","nonce":973678477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6821","nonce":973678477}]},"public_addr":"172.21.15.157:6817/973678477","cluster_addr":"172.21.15.157:6819/973678477","heartbeat_back_addr":"172.21.15.157:6823/973678477","heartbeat_front_addr":"172.21.15.157:6821/973678477","state":["exists","up"]},{"osd":9,"uuid":"82914818-df47-4374-9e3e-2f71fc76e78b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6827","nonce":710033990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6829","nonce":710033990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6833","nonce":710033990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6831","nonce":710033990}]},"public_addr":"172.21.15.22:6827/710033990","cluster_addr":"172.21.15.22:6829/710033990","heartbeat_back_addr":"172.21.15.22:6833/710033990","heartbeat_front_addr":"172.21.15.22:6831/710033990","state":["exists","up"]},{"osd":10,"uuid":"36ae177d-c454-4cba-ba21-cbeee0f10dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6825","nonce":373412752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6827","nonce":373412752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6831","nonce":373412752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6829","nonce":373412752}]},"public_addr":"172.21.15.64:6825/373412752","cluster_addr":"172.21.15.64:6827/373412752","heartbeat_back_addr":"172.21.15.64:6831/373412752","heartbeat_front_addr":"172.21.15.64:6829/373412752","state":["exists","up"]},{"osd":11,"uuid":"cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6825","nonce":1918889194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6827","nonce":1918889194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6831","nonce":1918889194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6829","nonce":1918889194}]},"public_addr":"172.21.15.157:6825/1918889194","cluster_addr":"172.21.15.157:6827/1918889194","heartbeat_back_addr":"172.21.15.157:6831/1918889194","heartbeat_front_addr":"172.21.15.157:6829/1918889194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:31.378218+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:56.794167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:20.790267+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:47.039106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:11.483842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:38.151466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:03.553308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:27.865321+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:53.086530+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:18.946661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:43.697243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:12:08.936588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/422860013":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/2837500116":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/1845420423":"2024-09-20T08:15:01.165466+0000","172.21.15.22:6800/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/3623845502":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2851750481":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2133059277":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/384027596":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/586163474":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/3712426751":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/1866276413":"2024-09-20T08:15:01.165466+0000","172.21.15.22:6801/1866276413":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/155709052":"2024-09-20T08:15:01.165466+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:15:02.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.589+0000 7fa318c0a640 1 -- 172.21.15.22:0/3154505793 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa2fc0a3050 msgr2=0x7fa2fc0a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:02.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.589+0000 7fa318c0a640 1 --2- 172.21.15.22:0/3154505793 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa2fc0a3050 0x7fa2fc0a5510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa304005ab0 tx=0x7fa304000bc0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.589+0000 7fa318c0a640 1 -- 172.21.15.22:0/3154505793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa314101c00 msgr2=0x7fa3141983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:02.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.589+0000 7fa318c0a640 1 --2- 172.21.15.22:0/3154505793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa314101c00 0x7fa3141983d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa30800e9f0 tx=0x7fa30800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.589+0000 7fa318c0a640 1 -- 172.21.15.22:0/3154505793 shutdown_connections 2024-09-19T08:15:02.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.589+0000 7fa318c0a640 1 --2- 172.21.15.22:0/3154505793 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa2fc0a3050 0x7fa2fc0a5510 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.589+0000 7fa318c0a640 1 --2- 172.21.15.22:0/3154505793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3141025c0 0x7fa31419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.589+0000 7fa318c0a640 1 --2- 172.21.15.22:0/3154505793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa314101c00 0x7fa3141983d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.589+0000 7fa318c0a640 1 --2- 172.21.15.22:0/3154505793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa314100a10 0x7fa314197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.589+0000 7fa318c0a640 1 -- 172.21.15.22:0/3154505793 >> 172.21.15.22:0/3154505793 conn(0x7fa3140fc170 msgr2=0x7fa3140693f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:02.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.589+0000 7fa318c0a640 1 -- 172.21.15.22:0/3154505793 shutdown_connections 2024-09-19T08:15:02.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.590+0000 7fa318c0a640 1 -- 172.21.15.22:0/3154505793 wait complete. 2024-09-19T08:15:02.603 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:08:25.175759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T08:12:57.608787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T08:12:59.824327+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:15:02.604 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:15:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.710+0000 7f4304e77640 1 -- 172.21.15.22:0/2348215458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4300102580 msgr2=0x7f4300104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.710+0000 7f4304e77640 1 --2- 172.21.15.22:0/2348215458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4300102580 0x7f4300104970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009900 tx=0x7f42f002a410 comp rx=0 tx=0).stop 2024-09-19T08:15:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.711+0000 7f4304e77640 1 -- 172.21.15.22:0/2348215458 shutdown_connections 2024-09-19T08:15:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.711+0000 7f4304e77640 1 --2- 172.21.15.22:0/2348215458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43001077e0 0x7f4300109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.711+0000 7f4304e77640 1 --2- 172.21.15.22:0/2348215458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4300104eb0 0x7f43001072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.711+0000 7f4304e77640 1 --2- 172.21.15.22:0/2348215458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4300102580 0x7f4300104970 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.711+0000 7f4304e77640 1 -- 172.21.15.22:0/2348215458 >> 172.21.15.22:0/2348215458 conn(0x7f43000fc140 msgr2=0x7f43000fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:02.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.711+0000 7f4304e77640 1 -- 172.21.15.22:0/2348215458 shutdown_connections 2024-09-19T08:15:02.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.711+0000 7f4304e77640 1 -- 172.21.15.22:0/2348215458 wait complete. 2024-09-19T08:15:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f4304e77640 1 Processor -- start 2024-09-19T08:15:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f4304e77640 1 -- start start 2024-09-19T08:15:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f4304e77640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4300102580 0x7f430019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f4304e77640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4300104eb0 0x7f430019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f4304e77640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43001077e0 0x7f43001a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f4304e77640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f430010d7e0 con 0x7f4300102580 2024-09-19T08:15:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f4304e77640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f430010d660 con 0x7f4300104eb0 2024-09-19T08:15:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f4304e77640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f430010d960 con 0x7f43001077e0 2024-09-19T08:15:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f42fe59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4300102580 0x7f430019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f42fe59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4300102580 0x7f430019a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42394/0 (socket says 172.21.15.22:42394) 2024-09-19T08:15:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f42fe59c640 1 -- 172.21.15.22:0/3084238049 learned_addr learned my addr 172.21.15.22:0/3084238049 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f42fdd9b640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4300104eb0 0x7f430019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f42fed9d640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43001077e0 0x7f43001a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f42fe59c640 1 -- 172.21.15.22:0/3084238049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43001077e0 msgr2=0x7f43001a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f42fe59c640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43001077e0 0x7f43001a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f42fe59c640 1 -- 172.21.15.22:0/3084238049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4300104eb0 msgr2=0x7f430019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f42fe59c640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4300104eb0 0x7f430019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f42fe59c640 1 -- 172.21.15.22:0/3084238049 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43001a1d00 con 0x7f4300102580 2024-09-19T08:15:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f42fdd9b640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4300104eb0 0x7f430019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:15:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f42fed9d640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43001077e0 0x7f43001a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:15:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.712+0000 7f42fe59c640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4300102580 0x7f430019a0e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009230 tx=0x7f42f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.713+0000 7f42ef7fe640 1 -- 172.21.15.22:0/3084238049 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f003f030 con 0x7f4300102580 2024-09-19T08:15:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.713+0000 7f42ef7fe640 1 -- 172.21.15.22:0/3084238049 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f0003ce0 con 0x7f4300102580 2024-09-19T08:15:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.713+0000 7f4304e77640 1 -- 172.21.15.22:0/3084238049 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43001a1f90 con 0x7f4300102580 2024-09-19T08:15:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.713+0000 7f42ef7fe640 1 -- 172.21.15.22:0/3084238049 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0035070 con 0x7f4300102580 2024-09-19T08:15:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.713+0000 7f4304e77640 1 -- 172.21.15.22:0/3084238049 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43001a23a0 con 0x7f4300102580 2024-09-19T08:15:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.714+0000 7f4304e77640 1 -- 172.21.15.22:0/3084238049 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4300109320 con 0x7f4300102580 2024-09-19T08:15:02.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.715+0000 7f42ef7fe640 1 -- 172.21.15.22:0/3084238049 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f42f0042030 con 0x7f4300102580 2024-09-19T08:15:02.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.715+0000 7f42ef7fe640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f42e80a3050 0x7f42e80a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:02.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.716+0000 7f42fdd9b640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f42e80a3050 0x7f42e80a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:02.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.716+0000 7f42ef7fe640 1 -- 172.21.15.22:0/3084238049 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f42f00efc30 con 0x7f4300102580 2024-09-19T08:15:02.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.716+0000 7f42fdd9b640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f42e80a3050 0x7f42e80a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f430019b5f0 tx=0x7f42f4005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.720+0000 7f42ef7fe640 1 -- 172.21.15.22:0/3084238049 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f42f00bcaf0 con 0x7f4300102580 2024-09-19T08:15:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.932+0000 7f4304e77640 1 -- 172.21.15.22:0/3084238049 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4300062e90 con 0x7f4300102580 2024-09-19T08:15:02.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.932+0000 7f42ef7fe640 1 -- 172.21.15.22:0/3084238049 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f42f00bc3f0 con 0x7f4300102580 2024-09-19T08:15:02.933 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-09-19T08:15:02.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.934+0000 7f4304e77640 1 -- 172.21.15.22:0/3084238049 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f42e80a3050 msgr2=0x7f42e80a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:02.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.934+0000 7f4304e77640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f42e80a3050 0x7f42e80a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f430019b5f0 tx=0x7f42f4005f90 comp rx=0 tx=0).stop 2024-09-19T08:15:02.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.934+0000 7f4304e77640 1 -- 172.21.15.22:0/3084238049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4300102580 msgr2=0x7f430019a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:02.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.934+0000 7f4304e77640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4300102580 0x7f430019a0e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009230 tx=0x7f42f00042f0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.934+0000 7f4304e77640 1 -- 172.21.15.22:0/3084238049 shutdown_connections 2024-09-19T08:15:02.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.934+0000 7f4304e77640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f42e80a3050 0x7f42e80a5510 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.934+0000 7f4304e77640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43001077e0 0x7f43001a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.935+0000 7f4304e77640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4300104eb0 0x7f430019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.935+0000 7f4304e77640 1 --2- 172.21.15.22:0/3084238049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4300102580 0x7f430019a0e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:02.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.935+0000 7f4304e77640 1 -- 172.21.15.22:0/3084238049 >> 172.21.15.22:0/3084238049 conn(0x7f43000fc140 msgr2=0x7f4300101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:02.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.935+0000 7f4304e77640 1 -- 172.21.15.22:0/3084238049 shutdown_connections 2024-09-19T08:15:02.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:02.935+0000 7f4304e77640 1 -- 172.21.15.22:0/3084238049 wait complete. 2024-09-19T08:15:02.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T08:15:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.063+0000 7f2b74103640 1 -- 172.21.15.22:0/515422620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b6c104eb0 msgr2=0x7f2b6c1072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.063+0000 7f2b74103640 1 --2- 172.21.15.22:0/515422620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b6c104eb0 0x7f2b6c1072a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009900 tx=0x7f2b5c02a3f0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.063+0000 7f2b74103640 1 -- 172.21.15.22:0/515422620 shutdown_connections 2024-09-19T08:15:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.063+0000 7f2b74103640 1 --2- 172.21.15.22:0/515422620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b6c1077e0 0x7f2b6c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.063+0000 7f2b74103640 1 --2- 172.21.15.22:0/515422620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b6c104eb0 0x7f2b6c1072a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.063+0000 7f2b74103640 1 --2- 172.21.15.22:0/515422620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6c102580 0x7f2b6c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.063+0000 7f2b74103640 1 -- 172.21.15.22:0/515422620 >> 172.21.15.22:0/515422620 conn(0x7f2b6c0fc140 msgr2=0x7f2b6c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.063+0000 7f2b74103640 1 -- 172.21.15.22:0/515422620 shutdown_connections 2024-09-19T08:15:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.063+0000 7f2b74103640 1 -- 172.21.15.22:0/515422620 wait complete. 2024-09-19T08:15:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b74103640 1 Processor -- start 2024-09-19T08:15:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b74103640 1 -- start start 2024-09-19T08:15:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b74103640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6c102580 0x7f2b6c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b74103640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b6c104eb0 0x7f2b6c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b74103640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b6c1077e0 0x7f2b6c19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b74103640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c10d7b0 con 0x7f2b6c104eb0 2024-09-19T08:15:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b74103640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c10d630 con 0x7f2b6c1077e0 2024-09-19T08:15:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b74103640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c10d930 con 0x7f2b6c102580 2024-09-19T08:15:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b7169e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b6c104eb0 0x7f2b6c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b71e9f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6c102580 0x7f2b6c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b7169e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b6c104eb0 0x7f2b6c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42400/0 (socket says 172.21.15.22:42400) 2024-09-19T08:15:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b7169e640 1 -- 172.21.15.22:0/2210769789 learned_addr learned my addr 172.21.15.22:0/2210769789 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b726a0640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b6c1077e0 0x7f2b6c19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b71e9f640 1 -- 172.21.15.22:0/2210769789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b6c1077e0 msgr2=0x7f2b6c19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b71e9f640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b6c1077e0 0x7f2b6c19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b71e9f640 1 -- 172.21.15.22:0/2210769789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b6c104eb0 msgr2=0x7f2b6c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b71e9f640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b6c104eb0 0x7f2b6c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b71e9f640 1 -- 172.21.15.22:0/2210769789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c19fb40 con 0x7f2b6c102580 2024-09-19T08:15:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b726a0640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b6c1077e0 0x7f2b6c19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.064+0000 7f2b7169e640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b6c104eb0 0x7f2b6c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.065+0000 7f2b71e9f640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6c102580 0x7f2b6c197f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800e9f0 tx=0x7f2b6800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.065+0000 7f2b62ffd640 1 -- 172.21.15.22:0/2210769789 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6800ccc0 con 0x7f2b6c102580 2024-09-19T08:15:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.065+0000 7f2b62ffd640 1 -- 172.21.15.22:0/2210769789 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b68005020 con 0x7f2b6c102580 2024-09-19T08:15:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.065+0000 7f2b74103640 1 -- 172.21.15.22:0/2210769789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6c19fe30 con 0x7f2b6c102580 2024-09-19T08:15:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.065+0000 7f2b62ffd640 1 -- 172.21.15.22:0/2210769789 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b680049e0 con 0x7f2b6c102580 2024-09-19T08:15:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.065+0000 7f2b74103640 1 -- 172.21.15.22:0/2210769789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6c1a0240 con 0x7f2b6c102580 2024-09-19T08:15:03.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.066+0000 7f2b74103640 1 -- 172.21.15.22:0/2210769789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b6c0fdc30 con 0x7f2b6c102580 2024-09-19T08:15:03.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.067+0000 7f2b62ffd640 1 -- 172.21.15.22:0/2210769789 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f2b68005560 con 0x7f2b6c102580 2024-09-19T08:15:03.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.068+0000 7f2b62ffd640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2b580a3050 0x7f2b580a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:03.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.068+0000 7f2b62ffd640 1 -- 172.21.15.22:0/2210769789 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2b680cf950 con 0x7f2b6c102580 2024-09-19T08:15:03.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.068+0000 7f2b7169e640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2b580a3050 0x7f2b580a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:03.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.068+0000 7f2b7169e640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2b580a3050 0x7f2b580a5510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c000c00 tx=0x7f2b5c0046c0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.071+0000 7f2b62ffd640 1 -- 172.21.15.22:0/2210769789 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2b680051d0 con 0x7f2b6c102580 2024-09-19T08:15:03.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.188+0000 7f2b62ffd640 1 -- 172.21.15.22:0/2210769789 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mgrmap(e 23) ==== 138526+0+0 (secure 0 0 0) 0x7f2b68098780 con 0x7f2b6c102580 2024-09-19T08:15:03.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-mon[30963]: mgrmap e22: x(active, since 1.01836s), standbys: z, y 2024-09-19T08:15:03.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3154505793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:15:03.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-mon[30963]: [19/Sep/2024:08:15:02] ENGINE Bus STARTING 2024-09-19T08:15:03.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3084238049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:15:03.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-mon[30963]: [19/Sep/2024:08:15:02] ENGINE Serving on http://172.21.15.22:8765 2024-09-19T08:15:03.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:03.106+0000 7f98309f6640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:08:15:03] ENGINE Error in HTTPServer.serve 2024-09-19T08:15:03.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: Traceback (most recent call last): 2024-09-19T08:15:03.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:15:03.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: self._connections.run(self.expiration_interval) 2024-09-19T08:15:03.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:15:03.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: self._run(expiration_interval) 2024-09-19T08:15:03.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:15:03.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:15:03.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:15:03.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:15:03.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:15:03.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: s = self.context.wrap_socket( 2024-09-19T08:15:03.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:15:03.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: return self.sslsocket_class._create( 2024-09-19T08:15:03.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:15:03.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: self.do_handshake() 2024-09-19T08:15:03.258 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:15:03.259 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: self._sslobj.do_handshake() 2024-09-19T08:15:03.259 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:15:03.259 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:15:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:15:03.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.285+0000 7f2b74103640 1 -- 172.21.15.22:0/2210769789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2b6c02da10 con 0x7f2b6c102580 2024-09-19T08:15:03.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.286+0000 7f2b62ffd640 1 -- 172.21.15.22:0/2210769789 <== mon.1 v2:172.21.15.157:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7f2b6809c810 con 0x7f2b6c102580 2024-09-19T08:15:03.287 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-09-19T08:15:03.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.288+0000 7f2b74103640 1 -- 172.21.15.22:0/2210769789 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2b580a3050 msgr2=0x7f2b580a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:03.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.288+0000 7f2b74103640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2b580a3050 0x7f2b580a5510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c000c00 tx=0x7f2b5c0046c0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.288+0000 7f2b74103640 1 -- 172.21.15.22:0/2210769789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6c102580 msgr2=0x7f2b6c197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.288+0000 7f2b74103640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6c102580 0x7f2b6c197f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800e9f0 tx=0x7f2b6800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.288+0000 7f2b74103640 1 -- 172.21.15.22:0/2210769789 shutdown_connections 2024-09-19T08:15:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.288+0000 7f2b74103640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2b580a3050 0x7f2b580a5510 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.288+0000 7f2b74103640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b6c1077e0 0x7f2b6c19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.288+0000 7f2b74103640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b6c104eb0 0x7f2b6c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.288+0000 7f2b74103640 1 --2- 172.21.15.22:0/2210769789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6c102580 0x7f2b6c197f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.288+0000 7f2b74103640 1 -- 172.21.15.22:0/2210769789 >> 172.21.15.22:0/2210769789 conn(0x7f2b6c0fc140 msgr2=0x7f2b6c102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.288+0000 7f2b74103640 1 -- 172.21.15.22:0/2210769789 shutdown_connections 2024-09-19T08:15:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.288+0000 7f2b74103640 1 -- 172.21.15.22:0/2210769789 wait complete. 2024-09-19T08:15:03.303 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T08:15:03.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.414+0000 7f239618d640 1 -- 172.21.15.22:0/3777358390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2390107860 msgr2=0x7f2390109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.414+0000 7f239618d640 1 --2- 172.21.15.22:0/3777358390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2390107860 0x7f2390109c50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2388009be0 tx=0x7f238802a6c0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.414+0000 7f239618d640 1 -- 172.21.15.22:0/3777358390 shutdown_connections 2024-09-19T08:15:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.414+0000 7f239618d640 1 --2- 172.21.15.22:0/3777358390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2390107860 0x7f2390109c50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.414+0000 7f239618d640 1 --2- 172.21.15.22:0/3777358390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390104f30 0x7f2390107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.414+0000 7f239618d640 1 --2- 172.21.15.22:0/3777358390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2390102600 0x7f23901049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.414+0000 7f239618d640 1 -- 172.21.15.22:0/3777358390 >> 172.21.15.22:0/3777358390 conn(0x7f23900fc140 msgr2=0x7f23900fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.414+0000 7f239618d640 1 -- 172.21.15.22:0/3777358390 shutdown_connections 2024-09-19T08:15:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.414+0000 7f239618d640 1 -- 172.21.15.22:0/3777358390 wait complete. 2024-09-19T08:15:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.415+0000 7f239618d640 1 Processor -- start 2024-09-19T08:15:03.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.415+0000 7f239618d640 1 -- start start 2024-09-19T08:15:03.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.415+0000 7f239618d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102600 0x7f2390197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:03.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.415+0000 7f239618d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2390104f30 0x7f23901983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:03.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.415+0000 7f239618d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2390107860 0x7f239019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:03.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.415+0000 7f239618d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f239010d750 con 0x7f2390104f30 2024-09-19T08:15:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.415+0000 7f239618d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f239010d5d0 con 0x7f2390102600 2024-09-19T08:15:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.415+0000 7f239618d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f239010d8d0 con 0x7f2390107860 2024-09-19T08:15:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.415+0000 7f238effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2390104f30 0x7f23901983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.415+0000 7f238effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2390104f30 0x7f23901983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42450/0 (socket says 172.21.15.22:42450) 2024-09-19T08:15:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.415+0000 7f238effd640 1 -- 172.21.15.22:0/70490559 learned_addr learned my addr 172.21.15.22:0/70490559 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.415+0000 7f238f7fe640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102600 0x7f2390197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.415+0000 7f238ffff640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2390107860 0x7f239019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.416+0000 7f238effd640 1 -- 172.21.15.22:0/70490559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2390107860 msgr2=0x7f239019f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.416+0000 7f238effd640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2390107860 0x7f239019f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.416+0000 7f238effd640 1 -- 172.21.15.22:0/70490559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102600 msgr2=0x7f2390197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.416+0000 7f238effd640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102600 0x7f2390197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.416+0000 7f238effd640 1 -- 172.21.15.22:0/70490559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239019fa80 con 0x7f2390104f30 2024-09-19T08:15:03.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.416+0000 7f238f7fe640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102600 0x7f2390197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:03.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.416+0000 7f238effd640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2390104f30 0x7f23901983a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f238400e9a0 tx=0x7f238400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:03.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.416+0000 7f238cff9640 1 -- 172.21.15.22:0/70490559 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238400cd80 con 0x7f2390104f30 2024-09-19T08:15:03.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.416+0000 7f238cff9640 1 -- 172.21.15.22:0/70490559 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23840035f0 con 0x7f2390104f30 2024-09-19T08:15:03.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.416+0000 7f238cff9640 1 -- 172.21.15.22:0/70490559 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384010670 con 0x7f2390104f30 2024-09-19T08:15:03.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.416+0000 7f239618d640 1 -- 172.21.15.22:0/70490559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239019fd70 con 0x7f2390104f30 2024-09-19T08:15:03.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.416+0000 7f239618d640 1 -- 172.21.15.22:0/70490559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23901a0260 con 0x7f2390104f30 2024-09-19T08:15:03.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.417+0000 7f239618d640 1 -- 172.21.15.22:0/70490559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23900fdc30 con 0x7f2390104f30 2024-09-19T08:15:03.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.419+0000 7f238cff9640 1 -- 172.21.15.22:0/70490559 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 23) ==== 138526+0+0 (secure 0 0 0) 0x7f23840037a0 con 0x7f2390104f30 2024-09-19T08:15:03.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.419+0000 7f238cff9640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f23780a3450 0x7f23780a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:03.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.419+0000 7f238f7fe640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f23780a3450 0x7f23780a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:03.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.419+0000 7f238cff9640 1 -- 172.21.15.22:0/70490559 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f23840d01d0 con 0x7f2390104f30 2024-09-19T08:15:03.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.420+0000 7f238f7fe640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f23780a3450 0x7f23780a5910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2380009fc0 tx=0x7f2380005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:03.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.427+0000 7f238cff9640 1 -- 172.21.15.22:0/70490559 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f238409cae0 con 0x7f2390104f30 2024-09-19T08:15:03.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:03 smithi157 ceph-mon[35143]: mgrmap e22: x(active, since 1.01836s), standbys: z, y 2024-09-19T08:15:03.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:03 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3154505793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:15:03.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:03 smithi157 ceph-mon[35143]: [19/Sep/2024:08:15:02] ENGINE Bus STARTING 2024-09-19T08:15:03.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:03 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3084238049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:15:03.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:03 smithi157 ceph-mon[35143]: [19/Sep/2024:08:15:02] ENGINE Serving on http://172.21.15.22:8765 2024-09-19T08:15:03.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:03 smithi064 ceph-mon[34900]: mgrmap e22: x(active, since 1.01836s), standbys: z, y 2024-09-19T08:15:03.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:03 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3154505793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:15:03.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:03 smithi064 ceph-mon[34900]: [19/Sep/2024:08:15:02] ENGINE Bus STARTING 2024-09-19T08:15:03.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:03 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3084238049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:15:03.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:03 smithi064 ceph-mon[34900]: [19/Sep/2024:08:15:02] ENGINE Serving on http://172.21.15.22:8765 2024-09-19T08:15:03.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.604+0000 7f239618d640 1 -- 172.21.15.22:0/70490559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f239002da10 con 0x7f2390104f30 2024-09-19T08:15:03.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.604+0000 7f238cff9640 1 -- 172.21.15.22:0/70490559 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7f238409c3e0 con 0x7f2390104f30 2024-09-19T08:15:03.605 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-09-19T08:15:03.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.606+0000 7f239618d640 1 -- 172.21.15.22:0/70490559 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f23780a3450 msgr2=0x7f23780a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:03.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.606+0000 7f239618d640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f23780a3450 0x7f23780a5910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2380009fc0 tx=0x7f2380005f90 comp rx=0 tx=0).stop 2024-09-19T08:15:03.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.606+0000 7f239618d640 1 -- 172.21.15.22:0/70490559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2390104f30 msgr2=0x7f23901983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:03.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.606+0000 7f239618d640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2390104f30 0x7f23901983a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f238400e9a0 tx=0x7f238400ee60 comp rx=0 tx=0).stop 2024-09-19T08:15:03.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.606+0000 7f239618d640 1 -- 172.21.15.22:0/70490559 shutdown_connections 2024-09-19T08:15:03.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.606+0000 7f239618d640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f23780a3450 0x7f23780a5910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.606+0000 7f239618d640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2390107860 0x7f239019f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.606+0000 7f239618d640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2390104f30 0x7f23901983a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.606+0000 7f239618d640 1 --2- 172.21.15.22:0/70490559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102600 0x7f2390197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:03.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.606+0000 7f239618d640 1 -- 172.21.15.22:0/70490559 >> 172.21.15.22:0/70490559 conn(0x7f23900fc140 msgr2=0x7f2390102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:03.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.606+0000 7f239618d640 1 -- 172.21.15.22:0/70490559 shutdown_connections 2024-09-19T08:15:03.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:03.606+0000 7f239618d640 1 -- 172.21.15.22:0/70490559 wait complete. 2024-09-19T08:15:03.621 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-19T08:15:03.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:15:03.885 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:04.328 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: [19/Sep/2024:08:15:03] ENGINE Serving on https://172.21.15.22:7150 2024-09-19T08:15:04.329 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: [19/Sep/2024:08:15:03] ENGINE Bus STARTED 2024-09-19T08:15:04.329 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: [19/Sep/2024:08:15:03] ENGINE Error in HTTPServer.serve 2024-09-19T08:15:04.329 INFO:journalctl@ceph.mon.a.smithi022.stdout: Traceback (most recent call last): 2024-09-19T08:15:04.329 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:15:04.329 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._connections.run(self.expiration_interval) 2024-09-19T08:15:04.329 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:15:04.329 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._run(expiration_interval) 2024-09-19T08:15:04.329 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:15:04.329 INFO:journalctl@ceph.mon.a.smithi022.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:15:04.329 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:15:04.329 INFO:journalctl@ceph.mon.a.smithi022.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout: s = self.context.wrap_socket( 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout: return self.sslsocket_class._create( 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout: self.do_handshake() 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._sslobj.do_handshake() 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2210769789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T08:15:04.330 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/70490559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T08:15:04.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:04 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.425+0000 7fc2ac207640 1 -- 172.21.15.22:0/1408394036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a410c2f0 msgr2=0x7fc2a410e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:04.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.425+0000 7fc2ac207640 1 --2- 172.21.15.22:0/1408394036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a410c2f0 0x7fc2a410e790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009ca0 tx=0x7fc29c02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:15:04.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.425+0000 7fc2ac207640 1 -- 172.21.15.22:0/1408394036 shutdown_connections 2024-09-19T08:15:04.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.425+0000 7fc2ac207640 1 --2- 172.21.15.22:0/1408394036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a410c2f0 0x7fc2a410e790 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:04.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.425+0000 7fc2ac207640 1 --2- 172.21.15.22:0/1408394036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2a41037a0 0x7fc2a410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.425+0000 7fc2ac207640 1 --2- 172.21.15.22:0/1408394036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a4102df0 0x7fc2a41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.425+0000 7fc2ac207640 1 -- 172.21.15.22:0/1408394036 >> 172.21.15.22:0/1408394036 conn(0x7fc2a40fc7d0 msgr2=0x7fc2a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.425+0000 7fc2ac207640 1 -- 172.21.15.22:0/1408394036 shutdown_connections 2024-09-19T08:15:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.425+0000 7fc2ac207640 1 -- 172.21.15.22:0/1408394036 wait complete. 2024-09-19T08:15:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.426+0000 7fc2ac207640 1 Processor -- start 2024-09-19T08:15:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.426+0000 7fc2ac207640 1 -- start start 2024-09-19T08:15:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.426+0000 7fc2ac207640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2a4102df0 0x7fc2a4100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.426+0000 7fc2ac207640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a41037a0 0x7fc2a4100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.426+0000 7fc2ac207640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a410c2f0 0x7fc2a41014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.427+0000 7fc2a97a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a41037a0 0x7fc2a4100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.427+0000 7fc2aa7a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a410c2f0 0x7fc2a41014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.427+0000 7fc2a97a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a41037a0 0x7fc2a4100f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42694/0 (socket says 172.21.15.22:42694) 2024-09-19T08:15:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.427+0000 7fc2a97a2640 1 -- 172.21.15.22:0/32777901 learned_addr learned my addr 172.21.15.22:0/32777901 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.427+0000 7fc2a9fa3640 1 --2- 172.21.15.22:0/32777901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2a4102df0 0x7fc2a4100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.427+0000 7fc2ac207640 1 -- 172.21.15.22:0/32777901 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a41120f0 con 0x7fc2a410c2f0 2024-09-19T08:15:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.427+0000 7fc2ac207640 1 -- 172.21.15.22:0/32777901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a4111f70 con 0x7fc2a4102df0 2024-09-19T08:15:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.427+0000 7fc2ac207640 1 -- 172.21.15.22:0/32777901 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a4112270 con 0x7fc2a41037a0 2024-09-19T08:15:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.427+0000 7fc2a97a2640 1 -- 172.21.15.22:0/32777901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2a4102df0 msgr2=0x7fc2a4100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.427+0000 7fc2a97a2640 1 --2- 172.21.15.22:0/32777901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2a4102df0 0x7fc2a4100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.427+0000 7fc2a97a2640 1 -- 172.21.15.22:0/32777901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a410c2f0 msgr2=0x7fc2a41014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.427+0000 7fc2a97a2640 1 --2- 172.21.15.22:0/32777901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a410c2f0 0x7fc2a41014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.427+0000 7fc2a97a2640 1 -- 172.21.15.22:0/32777901 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a41a6a50 con 0x7fc2a41037a0 2024-09-19T08:15:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.427+0000 7fc2a97a2640 1 --2- 172.21.15.22:0/32777901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a41037a0 0x7fc2a4100f90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc29800e9f0 tx=0x7fc29800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.428+0000 7fc296ffd640 1 -- 172.21.15.22:0/32777901 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29800ccc0 con 0x7fc2a41037a0 2024-09-19T08:15:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.428+0000 7fc2ac207640 1 -- 172.21.15.22:0/32777901 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a41a6d40 con 0x7fc2a41037a0 2024-09-19T08:15:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.428+0000 7fc296ffd640 1 -- 172.21.15.22:0/32777901 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc298010910 con 0x7fc2a41037a0 2024-09-19T08:15:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.428+0000 7fc296ffd640 1 -- 172.21.15.22:0/32777901 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298018920 con 0x7fc2a41037a0 2024-09-19T08:15:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.428+0000 7fc2ac207640 1 -- 172.21.15.22:0/32777901 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a41a7220 con 0x7fc2a41037a0 2024-09-19T08:15:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.429+0000 7fc2ac207640 1 -- 172.21.15.22:0/32777901 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a41031d0 con 0x7fc2a41037a0 2024-09-19T08:15:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.430+0000 7fc296ffd640 1 -- 172.21.15.22:0/32777901 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 23) ==== 138526+0+0 (secure 0 0 0) 0x7fc298010430 con 0x7fc2a41037a0 2024-09-19T08:15:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.431+0000 7fc296ffd640 1 --2- 172.21.15.22:0/32777901 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc2900a3440 0x7fc2900a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.431+0000 7fc2a9fa3640 1 --2- 172.21.15.22:0/32777901 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc2900a3440 0x7fc2900a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.431+0000 7fc296ffd640 1 -- 172.21.15.22:0/32777901 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc2980d0130 con 0x7fc2a41037a0 2024-09-19T08:15:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.432+0000 7fc2a9fa3640 1 --2- 172.21.15.22:0/32777901 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc2900a3440 0x7fc2900a5900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009bd0 tx=0x7fc2a0005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.435+0000 7fc296ffd640 1 -- 172.21.15.22:0/32777901 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc29809c300 con 0x7fc2a41037a0 2024-09-19T08:15:04.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: [19/Sep/2024:08:15:03] ENGINE Serving on https://172.21.15.22:7150 2024-09-19T08:15:04.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: [19/Sep/2024:08:15:03] ENGINE Bus STARTED 2024-09-19T08:15:04.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: [19/Sep/2024:08:15:03] ENGINE Error in HTTPServer.serve 2024-09-19T08:15:04.503 INFO:journalctl@ceph.mon.c.smithi157.stdout: Traceback (most recent call last): 2024-09-19T08:15:04.503 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:15:04.503 INFO:journalctl@ceph.mon.c.smithi157.stdout: self._connections.run(self.expiration_interval) 2024-09-19T08:15:04.504 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:15:04.504 INFO:journalctl@ceph.mon.c.smithi157.stdout: self._run(expiration_interval) 2024-09-19T08:15:04.504 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:15:04.504 INFO:journalctl@ceph.mon.c.smithi157.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:15:04.504 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:15:04.504 INFO:journalctl@ceph.mon.c.smithi157.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:15:04.504 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:15:04.504 INFO:journalctl@ceph.mon.c.smithi157.stdout: s = self.context.wrap_socket( 2024-09-19T08:15:04.504 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:15:04.504 INFO:journalctl@ceph.mon.c.smithi157.stdout: return self.sslsocket_class._create( 2024-09-19T08:15:04.504 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:15:04.504 INFO:journalctl@ceph.mon.c.smithi157.stdout: self.do_handshake() 2024-09-19T08:15:04.504 INFO:journalctl@ceph.mon.c.smithi157.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:15:04.505 INFO:journalctl@ceph.mon.c.smithi157.stdout: self._sslobj.do_handshake() 2024-09-19T08:15:04.505 INFO:journalctl@ceph.mon.c.smithi157.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:15:04.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:15:04.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T08:15:04.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2210769789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T08:15:04.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/70490559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T08:15:04.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.505 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:04 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: [19/Sep/2024:08:15:03] ENGINE Serving on https://172.21.15.22:7150 2024-09-19T08:15:04.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: [19/Sep/2024:08:15:03] ENGINE Bus STARTED 2024-09-19T08:15:04.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: [19/Sep/2024:08:15:03] ENGINE Error in HTTPServer.serve 2024-09-19T08:15:04.587 INFO:journalctl@ceph.mon.b.smithi064.stdout: Traceback (most recent call last): 2024-09-19T08:15:04.587 INFO:journalctl@ceph.mon.b.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:15:04.587 INFO:journalctl@ceph.mon.b.smithi064.stdout: self._connections.run(self.expiration_interval) 2024-09-19T08:15:04.587 INFO:journalctl@ceph.mon.b.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:15:04.588 INFO:journalctl@ceph.mon.b.smithi064.stdout: self._run(expiration_interval) 2024-09-19T08:15:04.588 INFO:journalctl@ceph.mon.b.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:15:04.588 INFO:journalctl@ceph.mon.b.smithi064.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:15:04.588 INFO:journalctl@ceph.mon.b.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:15:04.588 INFO:journalctl@ceph.mon.b.smithi064.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:15:04.588 INFO:journalctl@ceph.mon.b.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:15:04.588 INFO:journalctl@ceph.mon.b.smithi064.stdout: s = self.context.wrap_socket( 2024-09-19T08:15:04.588 INFO:journalctl@ceph.mon.b.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:15:04.589 INFO:journalctl@ceph.mon.b.smithi064.stdout: return self.sslsocket_class._create( 2024-09-19T08:15:04.589 INFO:journalctl@ceph.mon.b.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:15:04.589 INFO:journalctl@ceph.mon.b.smithi064.stdout: self.do_handshake() 2024-09-19T08:15:04.589 INFO:journalctl@ceph.mon.b.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:15:04.589 INFO:journalctl@ceph.mon.b.smithi064.stdout: self._sslobj.do_handshake() 2024-09-19T08:15:04.589 INFO:journalctl@ceph.mon.b.smithi064.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:15:04.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:15:04.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T08:15:04.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2210769789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T08:15:04.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/70490559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T08:15:04.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:04.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.591 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.591 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.591 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.591 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.591 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.591 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.591 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.591 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.591 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.591 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.591 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:04 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:04.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.675+0000 7fc2ac207640 1 -- 172.21.15.22:0/32777901 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2a41072c0 con 0x7fc2a41037a0 2024-09-19T08:15:04.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.676+0000 7fc296ffd640 1 -- 172.21.15.22:0/32777901 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11906 (secure 0 0 0) 0x7fc2980a0600 con 0x7fc2a41037a0 2024-09-19T08:15:04.678 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:04.679 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"btime":"2024-09-19T08:15:00:109390+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:15:00.059208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:15:04.679 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-09-19T08:15:04.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.679+0000 7fc2ac207640 1 -- 172.21.15.22:0/32777901 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc2900a3440 msgr2=0x7fc2900a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:04.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.679+0000 7fc2ac207640 1 --2- 172.21.15.22:0/32777901 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc2900a3440 0x7fc2900a5900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009bd0 tx=0x7fc2a0005f90 comp rx=0 tx=0).stop 2024-09-19T08:15:04.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.679+0000 7fc2ac207640 1 -- 172.21.15.22:0/32777901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a41037a0 msgr2=0x7fc2a4100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:04.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.679+0000 7fc2ac207640 1 --2- 172.21.15.22:0/32777901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a41037a0 0x7fc2a4100f90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc29800e9f0 tx=0x7fc29800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:15:04.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.679+0000 7fc2ac207640 1 -- 172.21.15.22:0/32777901 shutdown_connections 2024-09-19T08:15:04.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.679+0000 7fc2ac207640 1 --2- 172.21.15.22:0/32777901 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc2900a3440 0x7fc2900a5900 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:04.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.679+0000 7fc2ac207640 1 --2- 172.21.15.22:0/32777901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a410c2f0 0x7fc2a41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:04.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.679+0000 7fc2ac207640 1 --2- 172.21.15.22:0/32777901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2a41037a0 0x7fc2a4100f90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:04.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.679+0000 7fc2ac207640 1 --2- 172.21.15.22:0/32777901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2a4102df0 0x7fc2a4100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:04.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.679+0000 7fc2ac207640 1 -- 172.21.15.22:0/32777901 >> 172.21.15.22:0/32777901 conn(0x7fc2a40fc7d0 msgr2=0x7fc2a40fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:04.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.679+0000 7fc2ac207640 1 -- 172.21.15.22:0/32777901 shutdown_connections 2024-09-19T08:15:04.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:04.679+0000 7fc2ac207640 1 -- 172.21.15.22:0/32777901 wait complete. 2024-09-19T08:15:05.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T08:15:05.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Unable to set osd_memory_target on smithi157 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T08:15:05.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-19T08:15:05.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Unable to set osd_memory_target on smithi064 to 184598220: error parsing value: Value '184598220' is below minimum 939524096 2024-09-19T08:15:05.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi022 to 177.0M 2024-09-19T08:15:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Unable to set osd_memory_target on smithi022 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T08:15:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Updating smithi022:/etc/ceph/ceph.conf 2024-09-19T08:15:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-19T08:15:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-19T08:15:05.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Updating smithi064:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:15:05.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Updating smithi157:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:15:05.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Updating smithi022:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:15:05.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/32777901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:05.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:15:05.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:15:05.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:15:05.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Updating smithi064:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:15:05.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Updating smithi157:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:15:05.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: Updating smithi022:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:15:05.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:05 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T08:15:05.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Unable to set osd_memory_target on smithi157 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T08:15:05.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-19T08:15:05.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Unable to set osd_memory_target on smithi064 to 184598220: error parsing value: Value '184598220' is below minimum 939524096 2024-09-19T08:15:05.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi022 to 177.0M 2024-09-19T08:15:05.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Unable to set osd_memory_target on smithi022 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T08:15:05.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:05.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:05.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Updating smithi022:/etc/ceph/ceph.conf 2024-09-19T08:15:05.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-19T08:15:05.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-19T08:15:05.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Updating smithi064:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:15:05.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Updating smithi157:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:15:05.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Updating smithi022:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:15:05.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/32777901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:05.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:15:05.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:15:05.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:15:05.508 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Updating smithi064:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:15:05.509 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Updating smithi157:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:15:05.509 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: Updating smithi022:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:15:05.509 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.509 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.509 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.509 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:05 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.529 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-19T08:15:05.529 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-19T08:15:05.529 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:15:05.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T08:15:05.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Unable to set osd_memory_target on smithi157 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T08:15:05.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-19T08:15:05.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Unable to set osd_memory_target on smithi064 to 184598220: error parsing value: Value '184598220' is below minimum 939524096 2024-09-19T08:15:05.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:15:05.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi022 to 177.0M 2024-09-19T08:15:05.588 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Unable to set osd_memory_target on smithi022 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T08:15:05.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:05.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:05.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Updating smithi022:/etc/ceph/ceph.conf 2024-09-19T08:15:05.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-19T08:15:05.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-19T08:15:05.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Updating smithi064:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:15:05.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Updating smithi157:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:15:05.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Updating smithi022:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:15:05.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/32777901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:05.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:15:05.589 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:15:05.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:15:05.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Updating smithi064:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:15:05.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Updating smithi157:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:15:05.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: Updating smithi022:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:15:05.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.590 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:05 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:05.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.633+0000 7f90c2552640 1 -- 172.21.15.22:0/3780712266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90bc102cf0 msgr2=0x7f90bc1030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:05.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.633+0000 7f90c2552640 1 --2- 172.21.15.22:0/3780712266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90bc102cf0 0x7f90bc1030d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009900 tx=0x7f90ac02a480 comp rx=0 tx=0).stop 2024-09-19T08:15:05.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.633+0000 7f90c2552640 1 -- 172.21.15.22:0/3780712266 shutdown_connections 2024-09-19T08:15:05.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.633+0000 7f90c2552640 1 --2- 172.21.15.22:0/3780712266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90bc10c0c0 0x7f90bc10e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.633+0000 7f90c2552640 1 --2- 172.21.15.22:0/3780712266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90bc103610 0x7f90bc10baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.633+0000 7f90c2552640 1 --2- 172.21.15.22:0/3780712266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90bc102cf0 0x7f90bc1030d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.633+0000 7f90c2552640 1 -- 172.21.15.22:0/3780712266 >> 172.21.15.22:0/3780712266 conn(0x7f90bc077a10 msgr2=0x7f90bc0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:05.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.633+0000 7f90c2552640 1 -- 172.21.15.22:0/3780712266 shutdown_connections 2024-09-19T08:15:05.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.633+0000 7f90c2552640 1 -- 172.21.15.22:0/3780712266 wait complete. 2024-09-19T08:15:05.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.633+0000 7f90c2552640 1 Processor -- start 2024-09-19T08:15:05.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.633+0000 7f90c2552640 1 -- start start 2024-09-19T08:15:05.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c2552640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90bc102cf0 0x7f90bc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:05.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c2552640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90bc103610 0x7f90bc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:05.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c2552640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90bc10c0c0 0x7f90bc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:05.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c2552640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc111ee0 con 0x7f90bc10c0c0 2024-09-19T08:15:05.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c2552640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc111d60 con 0x7f90bc102cf0 2024-09-19T08:15:05.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c2552640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc112060 con 0x7f90bc103610 2024-09-19T08:15:05.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c0aef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90bc10c0c0 0x7f90bc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:05.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c0aef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90bc10c0c0 0x7f90bc19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42480/0 (socket says 172.21.15.22:42480) 2024-09-19T08:15:05.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c0aef640 1 -- 172.21.15.22:0/3984060795 learned_addr learned my addr 172.21.15.22:0/3984060795 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:05.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90bbfff640 1 --2- 172.21.15.22:0/3984060795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90bc102cf0 0x7f90bc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:05.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90bb7fe640 1 --2- 172.21.15.22:0/3984060795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90bc103610 0x7f90bc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:05.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c0aef640 1 -- 172.21.15.22:0/3984060795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90bc103610 msgr2=0x7f90bc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:05.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c0aef640 1 --2- 172.21.15.22:0/3984060795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90bc103610 0x7f90bc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c0aef640 1 -- 172.21.15.22:0/3984060795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90bc102cf0 msgr2=0x7f90bc1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:05.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c0aef640 1 --2- 172.21.15.22:0/3984060795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90bc102cf0 0x7f90bc1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c0aef640 1 -- 172.21.15.22:0/3984060795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc19d3c0 con 0x7f90bc10c0c0 2024-09-19T08:15:05.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c0aef640 1 --2- 172.21.15.22:0/3984060795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90bc10c0c0 0x7f90bc19cc90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f90b400cad0 tx=0x7f90b400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:05.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90b97fa640 1 -- 172.21.15.22:0/3984060795 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b400ae20 con 0x7f90bc10c0c0 2024-09-19T08:15:05.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.634+0000 7f90c2552640 1 -- 172.21.15.22:0/3984060795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90bc19d6b0 con 0x7f90bc10c0c0 2024-09-19T08:15:05.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.635+0000 7f90b97fa640 1 -- 172.21.15.22:0/3984060795 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b4005020 con 0x7f90bc10c0c0 2024-09-19T08:15:05.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.635+0000 7f90b97fa640 1 -- 172.21.15.22:0/3984060795 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b400f6f0 con 0x7f90bc10c0c0 2024-09-19T08:15:05.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.635+0000 7f90c2552640 1 -- 172.21.15.22:0/3984060795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90bc1a4f40 con 0x7f90bc10c0c0 2024-09-19T08:15:05.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.636+0000 7f90c2552640 1 -- 172.21.15.22:0/3984060795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90bc1030d0 con 0x7f90bc10c0c0 2024-09-19T08:15:05.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.637+0000 7f90b97fa640 1 -- 172.21.15.22:0/3984060795 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f90b40051d0 con 0x7f90bc10c0c0 2024-09-19T08:15:05.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.638+0000 7f90b97fa640 1 --2- 172.21.15.22:0/3984060795 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f90a40a34e0 0x7f90a40a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:05.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.638+0000 7f90bbfff640 1 --2- 172.21.15.22:0/3984060795 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f90a40a34e0 0x7f90a40a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:05.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.638+0000 7f90b97fa640 1 -- 172.21.15.22:0/3984060795 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f90b40cff00 con 0x7f90bc10c0c0 2024-09-19T08:15:05.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.638+0000 7f90bbfff640 1 --2- 172.21.15.22:0/3984060795 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f90a40a34e0 0x7f90a40a59a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f90ac000c00 tx=0x7f90ac005df0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:05.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.642+0000 7f90b97fa640 1 -- 172.21.15.22:0/3984060795 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f90b4014030 con 0x7f90bc10c0c0 2024-09-19T08:15:05.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.818+0000 7f90c2552640 1 -- 172.21.15.22:0/3984060795 --> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90bc062630 con 0x7f90a40a34e0 2024-09-19T08:15:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.826+0000 7f90b97fa640 1 -- 172.21.15.22:0/3984060795 <== mgr.15141 v2:172.21.15.22:6800/497251935 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258026 (secure 0 0 0) 0x7f90bc062630 con 0x7f90a40a34e0 2024-09-19T08:15:05.830 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T08:15:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.829+0000 7f90c2552640 1 -- 172.21.15.22:0/3984060795 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f90a40a34e0 msgr2=0x7f90a40a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.829+0000 7f90c2552640 1 --2- 172.21.15.22:0/3984060795 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f90a40a34e0 0x7f90a40a59a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f90ac000c00 tx=0x7f90ac005df0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.829+0000 7f90c2552640 1 -- 172.21.15.22:0/3984060795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90bc10c0c0 msgr2=0x7f90bc19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.829+0000 7f90c2552640 1 --2- 172.21.15.22:0/3984060795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90bc10c0c0 0x7f90bc19cc90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f90b400cad0 tx=0x7f90b400cf90 comp rx=0 tx=0).stop 2024-09-19T08:15:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.829+0000 7f90c2552640 1 -- 172.21.15.22:0/3984060795 shutdown_connections 2024-09-19T08:15:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.829+0000 7f90c2552640 1 --2- 172.21.15.22:0/3984060795 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f90a40a34e0 0x7f90a40a59a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.829+0000 7f90c2552640 1 --2- 172.21.15.22:0/3984060795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90bc10c0c0 0x7f90bc19cc90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.829+0000 7f90c2552640 1 --2- 172.21.15.22:0/3984060795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90bc103610 0x7f90bc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.829+0000 7f90c2552640 1 --2- 172.21.15.22:0/3984060795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90bc102cf0 0x7f90bc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.829+0000 7f90c2552640 1 -- 172.21.15.22:0/3984060795 >> 172.21.15.22:0/3984060795 conn(0x7f90bc077a10 msgr2=0x7f90bc0fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.829+0000 7f90c2552640 1 -- 172.21.15.22:0/3984060795 shutdown_connections 2024-09-19T08:15:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.829+0000 7f90c2552640 1 -- 172.21.15.22:0/3984060795 wait complete. 2024-09-19T08:15:05.841 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T08:15:05.185903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188412,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":920,"num_read_kb":620,"num_write":480,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1242042,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381604,"kb_used_data":11520,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560220,"statfs":{"total":1151940427776,"available":1151549665280,"internally_reserved":0,"allocated":11796480,"data_stored":9004314,"data_compressed":299297,"data_compressed_allocated":393216,"data_compressed_original":3670016,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.993097"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309110+0000","last_change":"2024-09-19T08:13:00.853947+0000","last_active":"2024-09-19T08:15:01.309110+0000","last_peered":"2024-09-19T08:15:01.309110+0000","last_clean":"2024-09-19T08:15:01.309110+0000","last_became_active":"2024-09-19T08:13:00.852951+0000","last_became_peered":"2024-09-19T08:13:00.852951+0000","last_unstale":"2024-09-19T08:15:01.309110+0000","last_undegraded":"2024-09-19T08:15:01.309110+0000","last_fullsized":"2024-09-19T08:15:01.309110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:25.786828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308563+0000","last_change":"2024-09-19T08:13:00.853389+0000","last_active":"2024-09-19T08:15:01.308563+0000","last_peered":"2024-09-19T08:15:01.308563+0000","last_clean":"2024-09-19T08:15:01.308563+0000","last_became_active":"2024-09-19T08:13:00.853102+0000","last_became_peered":"2024-09-19T08:13:00.853102+0000","last_unstale":"2024-09-19T08:15:01.308563+0000","last_undegraded":"2024-09-19T08:15:01.308563+0000","last_fullsized":"2024-09-19T08:15:01.308563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:42.773917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308818+0000","last_change":"2024-09-19T08:13:00.846544+0000","last_active":"2024-09-19T08:15:01.308818+0000","last_peered":"2024-09-19T08:15:01.308818+0000","last_clean":"2024-09-19T08:15:01.308818+0000","last_became_active":"2024-09-19T08:13:00.846229+0000","last_became_peered":"2024-09-19T08:13:00.846229+0000","last_unstale":"2024-09-19T08:15:01.308818+0000","last_undegraded":"2024-09-19T08:15:01.308818+0000","last_fullsized":"2024-09-19T08:15:01.308818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:43.471684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172883+0000","last_change":"2024-09-19T08:13:00.848149+0000","last_active":"2024-09-19T08:15:01.172883+0000","last_peered":"2024-09-19T08:15:01.172883+0000","last_clean":"2024-09-19T08:15:01.172883+0000","last_became_active":"2024-09-19T08:13:00.847744+0000","last_became_peered":"2024-09-19T08:13:00.847744+0000","last_unstale":"2024-09-19T08:15:01.172883+0000","last_undegraded":"2024-09-19T08:15:01.172883+0000","last_fullsized":"2024-09-19T08:15:01.172883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:01.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263000+0000","last_change":"2024-09-19T08:13:00.841862+0000","last_active":"2024-09-19T08:15:01.263000+0000","last_peered":"2024-09-19T08:15:01.263000+0000","last_clean":"2024-09-19T08:15:01.263000+0000","last_became_active":"2024-09-19T08:13:00.841331+0000","last_became_peered":"2024-09-19T08:13:00.841331+0000","last_unstale":"2024-09-19T08:15:01.263000+0000","last_undegraded":"2024-09-19T08:15:01.263000+0000","last_fullsized":"2024-09-19T08:15:01.263000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:54.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271477+0000","last_change":"2024-09-19T08:13:00.849626+0000","last_active":"2024-09-19T08:15:01.271477+0000","last_peered":"2024-09-19T08:15:01.271477+0000","last_clean":"2024-09-19T08:15:01.271477+0000","last_became_active":"2024-09-19T08:13:00.849034+0000","last_became_peered":"2024-09-19T08:13:00.849034+0000","last_unstale":"2024-09-19T08:15:01.271477+0000","last_undegraded":"2024-09-19T08:15:01.271477+0000","last_fullsized":"2024-09-19T08:15:01.271477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:39.118978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308024+0000","last_change":"2024-09-19T08:13:00.847522+0000","last_active":"2024-09-19T08:15:01.308024+0000","last_peered":"2024-09-19T08:15:01.308024+0000","last_clean":"2024-09-19T08:15:01.308024+0000","last_became_active":"2024-09-19T08:13:00.847117+0000","last_became_peered":"2024-09-19T08:13:00.847117+0000","last_unstale":"2024-09-19T08:15:01.308024+0000","last_undegraded":"2024-09-19T08:15:01.308024+0000","last_fullsized":"2024-09-19T08:15:01.308024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.836385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170733+0000","last_change":"2024-09-19T08:13:00.849544+0000","last_active":"2024-09-19T08:15:01.170733+0000","last_peered":"2024-09-19T08:15:01.170733+0000","last_clean":"2024-09-19T08:15:01.170733+0000","last_became_active":"2024-09-19T08:13:00.849170+0000","last_became_peered":"2024-09-19T08:13:00.849170+0000","last_unstale":"2024-09-19T08:15:01.170733+0000","last_undegraded":"2024-09-19T08:15:01.170733+0000","last_fullsized":"2024-09-19T08:15:01.170733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:28.371965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223727+0000","last_change":"2024-09-19T08:13:00.856052+0000","last_active":"2024-09-19T08:15:01.223727+0000","last_peered":"2024-09-19T08:15:01.223727+0000","last_clean":"2024-09-19T08:15:01.223727+0000","last_became_active":"2024-09-19T08:13:00.855291+0000","last_became_peered":"2024-09-19T08:13:00.855291+0000","last_unstale":"2024-09-19T08:15:01.223727+0000","last_undegraded":"2024-09-19T08:15:01.223727+0000","last_fullsized":"2024-09-19T08:15:01.223727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:18.581867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271887+0000","last_change":"2024-09-19T08:13:00.850145+0000","last_active":"2024-09-19T08:15:01.271887+0000","last_peered":"2024-09-19T08:15:01.271887+0000","last_clean":"2024-09-19T08:15:01.271887+0000","last_became_active":"2024-09-19T08:13:00.849798+0000","last_became_peered":"2024-09-19T08:13:00.849798+0000","last_unstale":"2024-09-19T08:15:01.271887+0000","last_undegraded":"2024-09-19T08:15:01.271887+0000","last_fullsized":"2024-09-19T08:15:01.271887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:21.400000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308275+0000","last_change":"2024-09-19T08:13:00.859153+0000","last_active":"2024-09-19T08:15:01.308275+0000","last_peered":"2024-09-19T08:15:01.308275+0000","last_clean":"2024-09-19T08:15:01.308275+0000","last_became_active":"2024-09-19T08:13:00.858473+0000","last_became_peered":"2024-09-19T08:13:00.858473+0000","last_unstale":"2024-09-19T08:15:01.308275+0000","last_undegraded":"2024-09-19T08:15:01.308275+0000","last_fullsized":"2024-09-19T08:15:01.308275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:07.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172922+0000","last_change":"2024-09-19T08:13:00.850797+0000","last_active":"2024-09-19T08:15:01.172922+0000","last_peered":"2024-09-19T08:15:01.172922+0000","last_clean":"2024-09-19T08:15:01.172922+0000","last_became_active":"2024-09-19T08:13:00.850393+0000","last_became_peered":"2024-09-19T08:13:00.850393+0000","last_unstale":"2024-09-19T08:15:01.172922+0000","last_undegraded":"2024-09-19T08:15:01.172922+0000","last_fullsized":"2024-09-19T08:15:01.172922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:59.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310082+0000","last_change":"2024-09-19T08:13:00.850835+0000","last_active":"2024-09-19T08:15:01.310082+0000","last_peered":"2024-09-19T08:15:01.310082+0000","last_clean":"2024-09-19T08:15:01.310082+0000","last_became_active":"2024-09-19T08:13:00.850518+0000","last_became_peered":"2024-09-19T08:13:00.850518+0000","last_unstale":"2024-09-19T08:15:01.310082+0000","last_undegraded":"2024-09-19T08:15:01.310082+0000","last_fullsized":"2024-09-19T08:15:01.310082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.178530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308450+0000","last_change":"2024-09-19T08:13:00.853934+0000","last_active":"2024-09-19T08:15:01.308450+0000","last_peered":"2024-09-19T08:15:01.308450+0000","last_clean":"2024-09-19T08:15:01.308450+0000","last_became_active":"2024-09-19T08:13:00.853604+0000","last_became_peered":"2024-09-19T08:13:00.853604+0000","last_unstale":"2024-09-19T08:15:01.308450+0000","last_undegraded":"2024-09-19T08:15:01.308450+0000","last_fullsized":"2024-09-19T08:15:01.308450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:43.622526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308262+0000","last_change":"2024-09-19T08:14:07.741370+0000","last_active":"2024-09-19T08:15:01.308262+0000","last_peered":"2024-09-19T08:15:01.308262+0000","last_clean":"2024-09-19T08:15:01.308262+0000","last_became_active":"2024-09-19T08:14:07.741116+0000","last_became_peered":"2024-09-19T08:14:07.741116+0000","last_unstale":"2024-09-19T08:15:01.308262+0000","last_undegraded":"2024-09-19T08:15:01.308262+0000","last_fullsized":"2024-09-19T08:15:01.308262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:45.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223320+0000","last_change":"2024-09-19T08:13:00.850266+0000","last_active":"2024-09-19T08:15:01.223320+0000","last_peered":"2024-09-19T08:15:01.223320+0000","last_clean":"2024-09-19T08:15:01.223320+0000","last_became_active":"2024-09-19T08:13:00.849928+0000","last_became_peered":"2024-09-19T08:13:00.849928+0000","last_unstale":"2024-09-19T08:15:01.223320+0000","last_undegraded":"2024-09-19T08:15:01.223320+0000","last_fullsized":"2024-09-19T08:15:01.223320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:38.231331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309177+0000","last_change":"2024-09-19T08:13:00.852551+0000","last_active":"2024-09-19T08:15:01.309177+0000","last_peered":"2024-09-19T08:15:01.309177+0000","last_clean":"2024-09-19T08:15:01.309177+0000","last_became_active":"2024-09-19T08:13:00.852225+0000","last_became_peered":"2024-09-19T08:13:00.852225+0000","last_unstale":"2024-09-19T08:15:01.309177+0000","last_undegraded":"2024-09-19T08:15:01.309177+0000","last_fullsized":"2024-09-19T08:15:01.309177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:33.115108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.261006+0000","last_change":"2024-09-19T08:13:00.854033+0000","last_active":"2024-09-19T08:15:01.261006+0000","last_peered":"2024-09-19T08:15:01.261006+0000","last_clean":"2024-09-19T08:15:01.261006+0000","last_became_active":"2024-09-19T08:13:00.853660+0000","last_became_peered":"2024-09-19T08:13:00.853660+0000","last_unstale":"2024-09-19T08:15:01.261006+0000","last_undegraded":"2024-09-19T08:15:01.261006+0000","last_fullsized":"2024-09-19T08:15:01.261006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:35.602420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308765+0000","last_change":"2024-09-19T08:13:00.850658+0000","last_active":"2024-09-19T08:15:01.308765+0000","last_peered":"2024-09-19T08:15:01.308765+0000","last_clean":"2024-09-19T08:15:01.308765+0000","last_became_active":"2024-09-19T08:13:00.850324+0000","last_became_peered":"2024-09-19T08:13:00.850324+0000","last_unstale":"2024-09-19T08:15:01.308765+0000","last_undegraded":"2024-09-19T08:15:01.308765+0000","last_fullsized":"2024-09-19T08:15:01.308765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:59:49.004605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308718+0000","last_change":"2024-09-19T08:14:07.741837+0000","last_active":"2024-09-19T08:15:01.308718+0000","last_peered":"2024-09-19T08:15:01.308718+0000","last_clean":"2024-09-19T08:15:01.308718+0000","last_became_active":"2024-09-19T08:14:07.741398+0000","last_became_peered":"2024-09-19T08:14:07.741398+0000","last_unstale":"2024-09-19T08:15:01.308718+0000","last_undegraded":"2024-09-19T08:15:01.308718+0000","last_fullsized":"2024-09-19T08:15:01.308718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:56.716663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310236+0000","last_change":"2024-09-19T08:13:00.855215+0000","last_active":"2024-09-19T08:15:01.310236+0000","last_peered":"2024-09-19T08:15:01.310236+0000","last_clean":"2024-09-19T08:15:01.310236+0000","last_became_active":"2024-09-19T08:13:00.854878+0000","last_became_peered":"2024-09-19T08:13:00.854878+0000","last_unstale":"2024-09-19T08:15:01.310236+0000","last_undegraded":"2024-09-19T08:15:01.310236+0000","last_fullsized":"2024-09-19T08:15:01.310236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:48.216066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223444+0000","last_change":"2024-09-19T08:13:00.856995+0000","last_active":"2024-09-19T08:15:01.223444+0000","last_peered":"2024-09-19T08:15:01.223444+0000","last_clean":"2024-09-19T08:15:01.223444+0000","last_became_active":"2024-09-19T08:13:00.856645+0000","last_became_peered":"2024-09-19T08:13:00.856645+0000","last_unstale":"2024-09-19T08:15:01.223444+0000","last_undegraded":"2024-09-19T08:15:01.223444+0000","last_fullsized":"2024-09-19T08:15:01.223444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:20.699293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170753+0000","last_change":"2024-09-19T08:13:00.853362+0000","last_active":"2024-09-19T08:15:01.170753+0000","last_peered":"2024-09-19T08:15:01.170753+0000","last_clean":"2024-09-19T08:15:01.170753+0000","last_became_active":"2024-09-19T08:13:00.853046+0000","last_became_peered":"2024-09-19T08:13:00.853046+0000","last_unstale":"2024-09-19T08:15:01.170753+0000","last_undegraded":"2024-09-19T08:15:01.170753+0000","last_fullsized":"2024-09-19T08:15:01.170753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172184+0000","last_change":"2024-09-19T08:13:00.845175+0000","last_active":"2024-09-19T08:15:01.172184+0000","last_peered":"2024-09-19T08:15:01.172184+0000","last_clean":"2024-09-19T08:15:01.172184+0000","last_became_active":"2024-09-19T08:13:00.844857+0000","last_became_peered":"2024-09-19T08:13:00.844857+0000","last_unstale":"2024-09-19T08:15:01.172184+0000","last_undegraded":"2024-09-19T08:15:01.172184+0000","last_fullsized":"2024-09-19T08:15:01.172184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:40.818956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171025+0000","last_change":"2024-09-19T08:13:00.847769+0000","last_active":"2024-09-19T08:15:01.171025+0000","last_peered":"2024-09-19T08:15:01.171025+0000","last_clean":"2024-09-19T08:15:01.171025+0000","last_became_active":"2024-09-19T08:13:00.847454+0000","last_became_peered":"2024-09-19T08:13:00.847454+0000","last_unstale":"2024-09-19T08:15:01.171025+0000","last_undegraded":"2024-09-19T08:15:01.171025+0000","last_fullsized":"2024-09-19T08:15:01.171025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.583051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308835+0000","last_change":"2024-09-19T08:13:00.859882+0000","last_active":"2024-09-19T08:15:01.308835+0000","last_peered":"2024-09-19T08:15:01.308835+0000","last_clean":"2024-09-19T08:15:01.308835+0000","last_became_active":"2024-09-19T08:13:00.859491+0000","last_became_peered":"2024-09-19T08:13:00.859491+0000","last_unstale":"2024-09-19T08:15:01.308835+0000","last_undegraded":"2024-09-19T08:15:01.308835+0000","last_fullsized":"2024-09-19T08:15:01.308835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:13.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308625+0000","last_change":"2024-09-19T08:13:00.845584+0000","last_active":"2024-09-19T08:15:01.308625+0000","last_peered":"2024-09-19T08:15:01.308625+0000","last_clean":"2024-09-19T08:15:01.308625+0000","last_became_active":"2024-09-19T08:13:00.845166+0000","last_became_peered":"2024-09-19T08:13:00.845166+0000","last_unstale":"2024-09-19T08:15:01.308625+0000","last_undegraded":"2024-09-19T08:15:01.308625+0000","last_fullsized":"2024-09-19T08:15:01.308625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:02.705793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310490+0000","last_change":"2024-09-19T08:13:00.838069+0000","last_active":"2024-09-19T08:15:01.310490+0000","last_peered":"2024-09-19T08:15:01.310490+0000","last_clean":"2024-09-19T08:15:01.310490+0000","last_became_active":"2024-09-19T08:13:00.837736+0000","last_became_peered":"2024-09-19T08:13:00.837736+0000","last_unstale":"2024-09-19T08:15:01.310490+0000","last_undegraded":"2024-09-19T08:15:01.310490+0000","last_fullsized":"2024-09-19T08:15:01.310490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:01.388290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308563+0000","last_change":"2024-09-19T08:13:00.841507+0000","last_active":"2024-09-19T08:15:01.308563+0000","last_peered":"2024-09-19T08:15:01.308563+0000","last_clean":"2024-09-19T08:15:01.308563+0000","last_became_active":"2024-09-19T08:13:00.841161+0000","last_became_peered":"2024-09-19T08:13:00.841161+0000","last_unstale":"2024-09-19T08:15:01.308563+0000","last_undegraded":"2024-09-19T08:15:01.308563+0000","last_fullsized":"2024-09-19T08:15:01.308563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:41.862290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308471+0000","last_change":"2024-09-19T08:13:00.847657+0000","last_active":"2024-09-19T08:15:01.308471+0000","last_peered":"2024-09-19T08:15:01.308471+0000","last_clean":"2024-09-19T08:15:01.308471+0000","last_became_active":"2024-09-19T08:13:00.846849+0000","last_became_peered":"2024-09-19T08:13:00.846849+0000","last_unstale":"2024-09-19T08:15:01.308471+0000","last_undegraded":"2024-09-19T08:15:01.308471+0000","last_fullsized":"2024-09-19T08:15:01.308471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:35.219913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309005+0000","last_change":"2024-09-19T08:13:00.845971+0000","last_active":"2024-09-19T08:15:01.309005+0000","last_peered":"2024-09-19T08:15:01.309005+0000","last_clean":"2024-09-19T08:15:01.309005+0000","last_became_active":"2024-09-19T08:13:00.845626+0000","last_became_peered":"2024-09-19T08:13:00.845626+0000","last_unstale":"2024-09-19T08:15:01.309005+0000","last_undegraded":"2024-09-19T08:15:01.309005+0000","last_fullsized":"2024-09-19T08:15:01.309005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:36.945627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308386+0000","last_change":"2024-09-19T08:14:07.738450+0000","last_active":"2024-09-19T08:15:01.308386+0000","last_peered":"2024-09-19T08:15:01.308386+0000","last_clean":"2024-09-19T08:15:01.308386+0000","last_became_active":"2024-09-19T08:14:07.738028+0000","last_became_peered":"2024-09-19T08:14:07.738028+0000","last_unstale":"2024-09-19T08:15:01.308386+0000","last_undegraded":"2024-09-19T08:15:01.308386+0000","last_fullsized":"2024-09-19T08:15:01.308386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:08.659158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262954+0000","last_change":"2024-09-19T08:13:00.848325+0000","last_active":"2024-09-19T08:15:01.262954+0000","last_peered":"2024-09-19T08:15:01.262954+0000","last_clean":"2024-09-19T08:15:01.262954+0000","last_became_active":"2024-09-19T08:13:00.848015+0000","last_became_peered":"2024-09-19T08:13:00.848015+0000","last_unstale":"2024-09-19T08:15:01.262954+0000","last_undegraded":"2024-09-19T08:15:01.262954+0000","last_fullsized":"2024-09-19T08:15:01.262954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:10.948451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:02.038549+0000","last_change":"2024-09-19T08:12:58.681134+0000","last_active":"2024-09-19T08:15:02.038549+0000","last_peered":"2024-09-19T08:15:02.038549+0000","last_clean":"2024-09-19T08:15:02.038549+0000","last_became_active":"2024-09-19T08:12:58.680764+0000","last_became_peered":"2024-09-19T08:12:58.680764+0000","last_unstale":"2024-09-19T08:15:02.038549+0000","last_undegraded":"2024-09-19T08:15:02.038549+0000","last_fullsized":"2024-09-19T08:15:02.038549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:24.939263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172533+0000","last_change":"2024-09-19T08:13:00.844319+0000","last_active":"2024-09-19T08:15:01.172533+0000","last_peered":"2024-09-19T08:15:01.172533+0000","last_clean":"2024-09-19T08:15:01.172533+0000","last_became_active":"2024-09-19T08:13:00.843994+0000","last_became_peered":"2024-09-19T08:13:00.843994+0000","last_unstale":"2024-09-19T08:15:01.172533+0000","last_undegraded":"2024-09-19T08:15:01.172533+0000","last_fullsized":"2024-09-19T08:15:01.172533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:46.937466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271532+0000","last_change":"2024-09-19T08:12:58.676541+0000","last_active":"2024-09-19T08:15:01.271532+0000","last_peered":"2024-09-19T08:15:01.271532+0000","last_clean":"2024-09-19T08:15:01.271532+0000","last_became_active":"2024-09-19T08:12:58.675988+0000","last_became_peered":"2024-09-19T08:12:58.675988+0000","last_unstale":"2024-09-19T08:15:01.271532+0000","last_undegraded":"2024-09-19T08:15:01.271532+0000","last_fullsized":"2024-09-19T08:15:01.271532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:01.434986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309788+0000","last_change":"2024-09-19T08:12:58.677510+0000","last_active":"2024-09-19T08:15:01.309788+0000","last_peered":"2024-09-19T08:15:01.309788+0000","last_clean":"2024-09-19T08:15:01.309788+0000","last_became_active":"2024-09-19T08:12:58.677200+0000","last_became_peered":"2024-09-19T08:12:58.677200+0000","last_unstale":"2024-09-19T08:15:01.309788+0000","last_undegraded":"2024-09-19T08:15:01.309788+0000","last_fullsized":"2024-09-19T08:15:01.309788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:36.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170825+0000","last_change":"2024-09-19T08:13:00.853430+0000","last_active":"2024-09-19T08:15:01.170825+0000","last_peered":"2024-09-19T08:15:01.170825+0000","last_clean":"2024-09-19T08:15:01.170825+0000","last_became_active":"2024-09-19T08:13:00.853096+0000","last_became_peered":"2024-09-19T08:13:00.853096+0000","last_unstale":"2024-09-19T08:15:01.170825+0000","last_undegraded":"2024-09-19T08:15:01.170825+0000","last_fullsized":"2024-09-19T08:15:01.170825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:07.147021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172384+0000","last_change":"2024-09-19T08:12:58.680081+0000","last_active":"2024-09-19T08:15:01.172384+0000","last_peered":"2024-09-19T08:15:01.172384+0000","last_clean":"2024-09-19T08:15:01.172384+0000","last_became_active":"2024-09-19T08:12:58.679704+0000","last_became_peered":"2024-09-19T08:12:58.679704+0000","last_unstale":"2024-09-19T08:15:01.172384+0000","last_undegraded":"2024-09-19T08:15:01.172384+0000","last_fullsized":"2024-09-19T08:15:01.172384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:32.942586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.260317+0000","last_change":"2024-09-19T08:13:00.849394+0000","last_active":"2024-09-19T08:15:01.260317+0000","last_peered":"2024-09-19T08:15:01.260317+0000","last_clean":"2024-09-19T08:15:01.260317+0000","last_became_active":"2024-09-19T08:13:00.849051+0000","last_became_peered":"2024-09-19T08:13:00.849051+0000","last_unstale":"2024-09-19T08:15:01.260317+0000","last_undegraded":"2024-09-19T08:15:01.260317+0000","last_fullsized":"2024-09-19T08:15:01.260317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:23.012928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'5","reported_seq":140,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.418714+0000","last_change":"2024-09-19T08:12:58.676121+0000","last_active":"2024-09-19T08:15:01.418714+0000","last_peered":"2024-09-19T08:15:01.418714+0000","last_clean":"2024-09-19T08:15:01.418714+0000","last_became_active":"2024-09-19T08:12:58.675693+0000","last_became_peered":"2024-09-19T08:12:58.675693+0000","last_unstale":"2024-09-19T08:15:01.418714+0000","last_undegraded":"2024-09-19T08:15:01.418714+0000","last_fullsized":"2024-09-19T08:15:01.418714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:34.882190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223583+0000","last_change":"2024-09-19T08:13:00.858735+0000","last_active":"2024-09-19T08:15:01.223583+0000","last_peered":"2024-09-19T08:15:01.223583+0000","last_clean":"2024-09-19T08:15:01.223583+0000","last_became_active":"2024-09-19T08:13:00.858218+0000","last_became_peered":"2024-09-19T08:13:00.858218+0000","last_unstale":"2024-09-19T08:15:01.223583+0000","last_undegraded":"2024-09-19T08:15:01.223583+0000","last_fullsized":"2024-09-19T08:15:01.223583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:33.356810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308176+0000","last_change":"2024-09-19T08:14:07.743785+0000","last_active":"2024-09-19T08:15:01.308176+0000","last_peered":"2024-09-19T08:15:01.308176+0000","last_clean":"2024-09-19T08:15:01.308176+0000","last_became_active":"2024-09-19T08:14:07.739531+0000","last_became_peered":"2024-09-19T08:14:07.739531+0000","last_unstale":"2024-09-19T08:15:01.308176+0000","last_undegraded":"2024-09-19T08:15:01.308176+0000","last_fullsized":"2024-09-19T08:15:01.308176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:43.367704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308976+0000","last_change":"2024-09-19T08:13:00.843852+0000","last_active":"2024-09-19T08:15:01.308976+0000","last_peered":"2024-09-19T08:15:01.308976+0000","last_clean":"2024-09-19T08:15:01.308976+0000","last_became_active":"2024-09-19T08:13:00.843497+0000","last_became_peered":"2024-09-19T08:13:00.843497+0000","last_unstale":"2024-09-19T08:15:01.308976+0000","last_undegraded":"2024-09-19T08:15:01.308976+0000","last_fullsized":"2024-09-19T08:15:01.308976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:26.380024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172698+0000","last_change":"2024-09-19T08:12:58.717225+0000","last_active":"2024-09-19T08:15:01.172698+0000","last_peered":"2024-09-19T08:15:01.172698+0000","last_clean":"2024-09-19T08:15:01.172698+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:15:01.172698+0000","last_undegraded":"2024-09-19T08:15:01.172698+0000","last_fullsized":"2024-09-19T08:15:01.172698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:43.201907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223446+0000","last_change":"2024-09-19T08:13:00.845613+0000","last_active":"2024-09-19T08:15:01.223446+0000","last_peered":"2024-09-19T08:15:01.223446+0000","last_clean":"2024-09-19T08:15:01.223446+0000","last_became_active":"2024-09-19T08:13:00.845260+0000","last_became_peered":"2024-09-19T08:13:00.845260+0000","last_unstale":"2024-09-19T08:15:01.223446+0000","last_undegraded":"2024-09-19T08:15:01.223446+0000","last_fullsized":"2024-09-19T08:15:01.223446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:03:47.033972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'36","reported_seq":311,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.820230+0000","last_change":"2024-09-19T08:14:07.749616+0000","last_active":"2024-09-19T08:15:01.820230+0000","last_peered":"2024-09-19T08:15:01.820230+0000","last_clean":"2024-09-19T08:15:01.820230+0000","last_became_active":"2024-09-19T08:14:07.740298+0000","last_became_peered":"2024-09-19T08:14:07.740298+0000","last_unstale":"2024-09-19T08:15:01.820230+0000","last_undegraded":"2024-09-19T08:15:01.820230+0000","last_fullsized":"2024-09-19T08:15:01.820230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:28.796855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66829,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":225,"num_read_kb":177,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61172,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263216+0000","last_change":"2024-09-19T08:13:00.855603+0000","last_active":"2024-09-19T08:15:01.263216+0000","last_peered":"2024-09-19T08:15:01.263216+0000","last_clean":"2024-09-19T08:15:01.263216+0000","last_became_active":"2024-09-19T08:13:00.855241+0000","last_became_peered":"2024-09-19T08:13:00.855241+0000","last_unstale":"2024-09-19T08:15:01.263216+0000","last_undegraded":"2024-09-19T08:15:01.263216+0000","last_fullsized":"2024-09-19T08:15:01.263216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:28.811847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308653+0000","last_change":"2024-09-19T08:12:58.675897+0000","last_active":"2024-09-19T08:15:01.308653+0000","last_peered":"2024-09-19T08:15:01.308653+0000","last_clean":"2024-09-19T08:15:01.308653+0000","last_became_active":"2024-09-19T08:12:58.675525+0000","last_became_peered":"2024-09-19T08:12:58.675525+0000","last_unstale":"2024-09-19T08:15:01.308653+0000","last_undegraded":"2024-09-19T08:15:01.308653+0000","last_fullsized":"2024-09-19T08:15:01.308653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:54.518064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309636+0000","last_change":"2024-09-19T08:13:00.856420+0000","last_active":"2024-09-19T08:15:01.309636+0000","last_peered":"2024-09-19T08:15:01.309636+0000","last_clean":"2024-09-19T08:15:01.309636+0000","last_became_active":"2024-09-19T08:13:00.856100+0000","last_became_peered":"2024-09-19T08:13:00.856100+0000","last_unstale":"2024-09-19T08:15:01.309636+0000","last_undegraded":"2024-09-19T08:15:01.309636+0000","last_fullsized":"2024-09-19T08:15:01.309636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:58.684930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172292+0000","last_change":"2024-09-19T08:12:58.680778+0000","last_active":"2024-09-19T08:15:01.172292+0000","last_peered":"2024-09-19T08:15:01.172292+0000","last_clean":"2024-09-19T08:15:01.172292+0000","last_became_active":"2024-09-19T08:12:58.680364+0000","last_became_peered":"2024-09-19T08:12:58.680364+0000","last_unstale":"2024-09-19T08:15:01.172292+0000","last_undegraded":"2024-09-19T08:15:01.172292+0000","last_fullsized":"2024-09-19T08:15:01.172292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:10.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308288+0000","last_change":"2024-09-19T08:13:00.853415+0000","last_active":"2024-09-19T08:15:01.308288+0000","last_peered":"2024-09-19T08:15:01.308288+0000","last_clean":"2024-09-19T08:15:01.308288+0000","last_became_active":"2024-09-19T08:13:00.853101+0000","last_became_peered":"2024-09-19T08:13:00.853101+0000","last_unstale":"2024-09-19T08:15:01.308288+0000","last_undegraded":"2024-09-19T08:15:01.308288+0000","last_fullsized":"2024-09-19T08:15:01.308288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:42.375394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":149,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:02.037848+0000","last_change":"2024-09-19T08:12:58.680312+0000","last_active":"2024-09-19T08:15:02.037848+0000","last_peered":"2024-09-19T08:15:02.037848+0000","last_clean":"2024-09-19T08:15:02.037848+0000","last_became_active":"2024-09-19T08:12:58.679870+0000","last_became_peered":"2024-09-19T08:12:58.679870+0000","last_unstale":"2024-09-19T08:15:02.037848+0000","last_undegraded":"2024-09-19T08:15:02.037848+0000","last_fullsized":"2024-09-19T08:15:02.037848+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:42.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308034+0000","last_change":"2024-09-19T08:13:00.853920+0000","last_active":"2024-09-19T08:15:01.308034+0000","last_peered":"2024-09-19T08:15:01.308034+0000","last_clean":"2024-09-19T08:15:01.308034+0000","last_became_active":"2024-09-19T08:13:00.853594+0000","last_became_peered":"2024-09-19T08:13:00.853594+0000","last_unstale":"2024-09-19T08:15:01.308034+0000","last_undegraded":"2024-09-19T08:15:01.308034+0000","last_fullsized":"2024-09-19T08:15:01.308034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:31.746133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308308+0000","last_change":"2024-09-19T08:14:07.751888+0000","last_active":"2024-09-19T08:15:01.308308+0000","last_peered":"2024-09-19T08:15:01.308308+0000","last_clean":"2024-09-19T08:15:01.308308+0000","last_became_active":"2024-09-19T08:14:07.737201+0000","last_became_peered":"2024-09-19T08:14:07.737201+0000","last_unstale":"2024-09-19T08:15:01.308308+0000","last_undegraded":"2024-09-19T08:15:01.308308+0000","last_fullsized":"2024-09-19T08:15:01.308308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:44.095987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172245+0000","last_change":"2024-09-19T08:13:00.847061+0000","last_active":"2024-09-19T08:15:01.172245+0000","last_peered":"2024-09-19T08:15:01.172245+0000","last_clean":"2024-09-19T08:15:01.172245+0000","last_became_active":"2024-09-19T08:13:00.846742+0000","last_became_peered":"2024-09-19T08:13:00.846742+0000","last_unstale":"2024-09-19T08:15:01.172245+0000","last_undegraded":"2024-09-19T08:15:01.172245+0000","last_fullsized":"2024-09-19T08:15:01.172245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:21.785585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309033+0000","last_change":"2024-09-19T08:14:07.755277+0000","last_active":"2024-09-19T08:15:01.309033+0000","last_peered":"2024-09-19T08:15:01.309033+0000","last_clean":"2024-09-19T08:15:01.309033+0000","last_became_active":"2024-09-19T08:14:07.739773+0000","last_became_peered":"2024-09-19T08:14:07.739773+0000","last_unstale":"2024-09-19T08:15:01.309033+0000","last_undegraded":"2024-09-19T08:15:01.309033+0000","last_fullsized":"2024-09-19T08:15:01.309033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:56.848251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308865+0000","last_change":"2024-09-19T08:13:00.844940+0000","last_active":"2024-09-19T08:15:01.308865+0000","last_peered":"2024-09-19T08:15:01.308865+0000","last_clean":"2024-09-19T08:15:01.308865+0000","last_became_active":"2024-09-19T08:13:00.844454+0000","last_became_peered":"2024-09-19T08:13:00.844454+0000","last_unstale":"2024-09-19T08:15:01.308865+0000","last_undegraded":"2024-09-19T08:15:01.308865+0000","last_fullsized":"2024-09-19T08:15:01.308865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:15.577624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309967+0000","last_change":"2024-09-19T08:12:58.673301+0000","last_active":"2024-09-19T08:15:01.309967+0000","last_peered":"2024-09-19T08:15:01.309967+0000","last_clean":"2024-09-19T08:15:01.309967+0000","last_became_active":"2024-09-19T08:12:58.672981+0000","last_became_peered":"2024-09-19T08:12:58.672981+0000","last_unstale":"2024-09-19T08:15:01.309967+0000","last_undegraded":"2024-09-19T08:15:01.309967+0000","last_fullsized":"2024-09-19T08:15:01.309967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:03.963712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308223+0000","last_change":"2024-09-19T08:13:00.856620+0000","last_active":"2024-09-19T08:15:01.308223+0000","last_peered":"2024-09-19T08:15:01.308223+0000","last_clean":"2024-09-19T08:15:01.308223+0000","last_became_active":"2024-09-19T08:13:00.856310+0000","last_became_peered":"2024-09-19T08:13:00.856310+0000","last_unstale":"2024-09-19T08:15:01.308223+0000","last_undegraded":"2024-09-19T08:15:01.308223+0000","last_fullsized":"2024-09-19T08:15:01.308223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:16.918768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'39","reported_seq":71,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.371288+0000","last_change":"2024-09-19T08:12:58.672282+0000","last_active":"2024-09-19T08:15:01.371288+0000","last_peered":"2024-09-19T08:15:01.371288+0000","last_clean":"2024-09-19T08:15:01.371288+0000","last_became_active":"2024-09-19T08:12:58.671891+0000","last_became_peered":"2024-09-19T08:12:58.671891+0000","last_unstale":"2024-09-19T08:15:01.371288+0000","last_undegraded":"2024-09-19T08:15:01.371288+0000","last_fullsized":"2024-09-19T08:15:01.371288+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:41.731571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308128+0000","last_change":"2024-09-19T08:13:00.848107+0000","last_active":"2024-09-19T08:15:01.308128+0000","last_peered":"2024-09-19T08:15:01.308128+0000","last_clean":"2024-09-19T08:15:01.308128+0000","last_became_active":"2024-09-19T08:13:00.847632+0000","last_became_peered":"2024-09-19T08:13:00.847632+0000","last_unstale":"2024-09-19T08:15:01.308128+0000","last_undegraded":"2024-09-19T08:15:01.308128+0000","last_fullsized":"2024-09-19T08:15:01.308128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.491122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308832+0000","last_change":"2024-09-19T08:12:59.179200+0000","last_active":"2024-09-19T08:15:01.308832+0000","last_peered":"2024-09-19T08:15:01.308832+0000","last_clean":"2024-09-19T08:15:01.308832+0000","last_became_active":"2024-09-19T08:12:59.178762+0000","last_became_peered":"2024-09-19T08:12:59.178762+0000","last_unstale":"2024-09-19T08:15:01.308832+0000","last_undegraded":"2024-09-19T08:15:01.308832+0000","last_fullsized":"2024-09-19T08:15:01.308832+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:08.424106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308358+0000","last_change":"2024-09-19T08:13:00.855870+0000","last_active":"2024-09-19T08:15:01.308358+0000","last_peered":"2024-09-19T08:15:01.308358+0000","last_clean":"2024-09-19T08:15:01.308358+0000","last_became_active":"2024-09-19T08:13:00.855538+0000","last_became_peered":"2024-09-19T08:13:00.855538+0000","last_unstale":"2024-09-19T08:15:01.308358+0000","last_undegraded":"2024-09-19T08:15:01.308358+0000","last_fullsized":"2024-09-19T08:15:01.308358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308350+0000","last_change":"2024-09-19T08:11:48.661333+0000","last_active":"2024-09-19T08:15:01.308350+0000","last_peered":"2024-09-19T08:15:01.308350+0000","last_clean":"2024-09-19T08:15:01.308350+0000","last_became_active":"2024-09-19T08:11:48.660873+0000","last_became_peered":"2024-09-19T08:11:48.660873+0000","last_unstale":"2024-09-19T08:15:01.308350+0000","last_undegraded":"2024-09-19T08:15:01.308350+0000","last_fullsized":"2024-09-19T08:15:01.308350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_clean_scrub_stamp":"2024-09-19T08:08:25.372155+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:19.401292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172985+0000","last_change":"2024-09-19T08:12:58.675237+0000","last_active":"2024-09-19T08:15:01.172985+0000","last_peered":"2024-09-19T08:15:01.172985+0000","last_clean":"2024-09-19T08:15:01.172985+0000","last_became_active":"2024-09-19T08:12:58.674910+0000","last_became_peered":"2024-09-19T08:12:58.674910+0000","last_unstale":"2024-09-19T08:15:01.172985+0000","last_undegraded":"2024-09-19T08:15:01.172985+0000","last_fullsized":"2024-09-19T08:15:01.172985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:47.038328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171023+0000","last_change":"2024-09-19T08:13:00.846754+0000","last_active":"2024-09-19T08:15:01.171023+0000","last_peered":"2024-09-19T08:15:01.171023+0000","last_clean":"2024-09-19T08:15:01.171023+0000","last_became_active":"2024-09-19T08:13:00.846452+0000","last_became_peered":"2024-09-19T08:13:00.846452+0000","last_unstale":"2024-09-19T08:15:01.171023+0000","last_undegraded":"2024-09-19T08:15:01.171023+0000","last_fullsized":"2024-09-19T08:15:01.171023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:22.929689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308239+0000","last_change":"2024-09-19T08:12:58.678958+0000","last_active":"2024-09-19T08:15:01.308239+0000","last_peered":"2024-09-19T08:15:01.308239+0000","last_clean":"2024-09-19T08:15:01.308239+0000","last_became_active":"2024-09-19T08:12:58.678592+0000","last_became_peered":"2024-09-19T08:12:58.678592+0000","last_unstale":"2024-09-19T08:15:01.308239+0000","last_undegraded":"2024-09-19T08:15:01.308239+0000","last_fullsized":"2024-09-19T08:15:01.308239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:58.374606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309700+0000","last_change":"2024-09-19T08:13:00.840973+0000","last_active":"2024-09-19T08:15:01.309700+0000","last_peered":"2024-09-19T08:15:01.309700+0000","last_clean":"2024-09-19T08:15:01.309700+0000","last_became_active":"2024-09-19T08:13:00.840619+0000","last_became_peered":"2024-09-19T08:13:00.840619+0000","last_unstale":"2024-09-19T08:15:01.309700+0000","last_undegraded":"2024-09-19T08:15:01.309700+0000","last_fullsized":"2024-09-19T08:15:01.309700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:02.037417+0000","last_change":"2024-09-19T08:12:58.715972+0000","last_active":"2024-09-19T08:15:02.037417+0000","last_peered":"2024-09-19T08:15:02.037417+0000","last_clean":"2024-09-19T08:15:02.037417+0000","last_became_active":"2024-09-19T08:12:58.715630+0000","last_became_peered":"2024-09-19T08:12:58.715630+0000","last_unstale":"2024-09-19T08:15:02.037417+0000","last_undegraded":"2024-09-19T08:15:02.037417+0000","last_fullsized":"2024-09-19T08:15:02.037417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:35.637121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172244+0000","last_change":"2024-09-19T08:13:00.847023+0000","last_active":"2024-09-19T08:15:01.172244+0000","last_peered":"2024-09-19T08:15:01.172244+0000","last_clean":"2024-09-19T08:15:01.172244+0000","last_became_active":"2024-09-19T08:13:00.846387+0000","last_became_peered":"2024-09-19T08:13:00.846387+0000","last_unstale":"2024-09-19T08:15:01.172244+0000","last_undegraded":"2024-09-19T08:15:01.172244+0000","last_fullsized":"2024-09-19T08:15:01.172244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:12.187929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308283+0000","last_change":"2024-09-19T08:12:58.681062+0000","last_active":"2024-09-19T08:15:01.308283+0000","last_peered":"2024-09-19T08:15:01.308283+0000","last_clean":"2024-09-19T08:15:01.308283+0000","last_became_active":"2024-09-19T08:12:58.680524+0000","last_became_peered":"2024-09-19T08:12:58.680524+0000","last_unstale":"2024-09-19T08:15:01.308283+0000","last_undegraded":"2024-09-19T08:15:01.308283+0000","last_fullsized":"2024-09-19T08:15:01.308283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271612+0000","last_change":"2024-09-19T08:13:00.851523+0000","last_active":"2024-09-19T08:15:01.271612+0000","last_peered":"2024-09-19T08:15:01.271612+0000","last_clean":"2024-09-19T08:15:01.271612+0000","last_became_active":"2024-09-19T08:13:00.851068+0000","last_became_peered":"2024-09-19T08:13:00.851068+0000","last_unstale":"2024-09-19T08:15:01.271612+0000","last_undegraded":"2024-09-19T08:15:01.271612+0000","last_fullsized":"2024-09-19T08:15:01.271612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:44.905469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.260876+0000","last_change":"2024-09-19T08:12:58.718234+0000","last_active":"2024-09-19T08:15:01.260876+0000","last_peered":"2024-09-19T08:15:01.260876+0000","last_clean":"2024-09-19T08:15:01.260876+0000","last_became_active":"2024-09-19T08:12:58.717840+0000","last_became_peered":"2024-09-19T08:12:58.717840+0000","last_unstale":"2024-09-19T08:15:01.260876+0000","last_undegraded":"2024-09-19T08:15:01.260876+0000","last_fullsized":"2024-09-19T08:15:01.260876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:05.283770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271611+0000","last_change":"2024-09-19T08:13:00.848305+0000","last_active":"2024-09-19T08:15:01.271611+0000","last_peered":"2024-09-19T08:15:01.271611+0000","last_clean":"2024-09-19T08:15:01.271611+0000","last_became_active":"2024-09-19T08:13:00.847981+0000","last_became_peered":"2024-09-19T08:13:00.847981+0000","last_unstale":"2024-09-19T08:15:01.271611+0000","last_undegraded":"2024-09-19T08:15:01.271611+0000","last_fullsized":"2024-09-19T08:15:01.271611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:04.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263326+0000","last_change":"2024-09-19T08:12:58.715943+0000","last_active":"2024-09-19T08:15:01.263326+0000","last_peered":"2024-09-19T08:15:01.263326+0000","last_clean":"2024-09-19T08:15:01.263326+0000","last_became_active":"2024-09-19T08:12:58.715600+0000","last_became_peered":"2024-09-19T08:12:58.715600+0000","last_unstale":"2024-09-19T08:15:01.263326+0000","last_undegraded":"2024-09-19T08:15:01.263326+0000","last_fullsized":"2024-09-19T08:15:01.263326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:19.142271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310614+0000","last_change":"2024-09-19T08:13:00.840944+0000","last_active":"2024-09-19T08:15:01.310614+0000","last_peered":"2024-09-19T08:15:01.310614+0000","last_clean":"2024-09-19T08:15:01.310614+0000","last_became_active":"2024-09-19T08:13:00.840608+0000","last_became_peered":"2024-09-19T08:13:00.840608+0000","last_unstale":"2024-09-19T08:15:01.310614+0000","last_undegraded":"2024-09-19T08:15:01.310614+0000","last_fullsized":"2024-09-19T08:15:01.310614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:25.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'47","reported_seq":94,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.374602+0000","last_change":"2024-09-19T08:12:58.672929+0000","last_active":"2024-09-19T08:15:01.374602+0000","last_peered":"2024-09-19T08:15:01.374602+0000","last_clean":"2024-09-19T08:15:01.374602+0000","last_became_active":"2024-09-19T08:12:58.672487+0000","last_became_peered":"2024-09-19T08:12:58.672487+0000","last_unstale":"2024-09-19T08:15:01.374602+0000","last_undegraded":"2024-09-19T08:15:01.374602+0000","last_fullsized":"2024-09-19T08:15:01.374602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:33.042735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.261959+0000","last_change":"2024-09-19T08:14:07.738260+0000","last_active":"2024-09-19T08:15:01.261959+0000","last_peered":"2024-09-19T08:15:01.261959+0000","last_clean":"2024-09-19T08:15:01.261959+0000","last_became_active":"2024-09-19T08:14:07.737915+0000","last_became_peered":"2024-09-19T08:14:07.737915+0000","last_unstale":"2024-09-19T08:15:01.261959+0000","last_undegraded":"2024-09-19T08:15:01.261959+0000","last_fullsized":"2024-09-19T08:15:01.261959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:02.500657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.173256+0000","last_change":"2024-09-19T08:12:59.179510+0000","last_active":"2024-09-19T08:15:01.173256+0000","last_peered":"2024-09-19T08:15:01.173256+0000","last_clean":"2024-09-19T08:15:01.173256+0000","last_became_active":"2024-09-19T08:12:59.179157+0000","last_became_peered":"2024-09-19T08:12:59.179157+0000","last_unstale":"2024-09-19T08:15:01.173256+0000","last_undegraded":"2024-09-19T08:15:01.173256+0000","last_fullsized":"2024-09-19T08:15:01.173256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:22.981289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308618+0000","last_change":"2024-09-19T08:13:00.841853+0000","last_active":"2024-09-19T08:15:01.308618+0000","last_peered":"2024-09-19T08:15:01.308618+0000","last_clean":"2024-09-19T08:15:01.308618+0000","last_became_active":"2024-09-19T08:13:00.841518+0000","last_became_peered":"2024-09-19T08:13:00.841518+0000","last_unstale":"2024-09-19T08:15:01.308618+0000","last_undegraded":"2024-09-19T08:15:01.308618+0000","last_fullsized":"2024-09-19T08:15:01.308618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:05.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223399+0000","last_change":"2024-09-19T08:12:58.716882+0000","last_active":"2024-09-19T08:15:01.223399+0000","last_peered":"2024-09-19T08:15:01.223399+0000","last_clean":"2024-09-19T08:15:01.223399+0000","last_became_active":"2024-09-19T08:12:58.716545+0000","last_became_peered":"2024-09-19T08:12:58.716545+0000","last_unstale":"2024-09-19T08:15:01.223399+0000","last_undegraded":"2024-09-19T08:15:01.223399+0000","last_fullsized":"2024-09-19T08:15:01.223399+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:14.469109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262936+0000","last_change":"2024-09-19T08:13:00.850484+0000","last_active":"2024-09-19T08:15:01.262936+0000","last_peered":"2024-09-19T08:15:01.262936+0000","last_clean":"2024-09-19T08:15:01.262936+0000","last_became_active":"2024-09-19T08:13:00.850049+0000","last_became_peered":"2024-09-19T08:13:00.850049+0000","last_unstale":"2024-09-19T08:15:01.262936+0000","last_undegraded":"2024-09-19T08:15:01.262936+0000","last_fullsized":"2024-09-19T08:15:01.262936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:24.424111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170936+0000","last_change":"2024-09-19T08:12:58.669928+0000","last_active":"2024-09-19T08:15:01.170936+0000","last_peered":"2024-09-19T08:15:01.170936+0000","last_clean":"2024-09-19T08:15:01.170936+0000","last_became_active":"2024-09-19T08:12:58.669614+0000","last_became_peered":"2024-09-19T08:12:58.669614+0000","last_unstale":"2024-09-19T08:15:01.170936+0000","last_undegraded":"2024-09-19T08:15:01.170936+0000","last_fullsized":"2024-09-19T08:15:01.170936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:50.924779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308456+0000","last_change":"2024-09-19T08:13:00.853520+0000","last_active":"2024-09-19T08:15:01.308456+0000","last_peered":"2024-09-19T08:15:01.308456+0000","last_clean":"2024-09-19T08:15:01.308456+0000","last_became_active":"2024-09-19T08:13:00.853184+0000","last_became_peered":"2024-09-19T08:13:00.853184+0000","last_unstale":"2024-09-19T08:15:01.308456+0000","last_undegraded":"2024-09-19T08:15:01.308456+0000","last_fullsized":"2024-09-19T08:15:01.308456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:45.160439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308612+0000","last_change":"2024-09-19T08:12:58.674359+0000","last_active":"2024-09-19T08:15:01.308612+0000","last_peered":"2024-09-19T08:15:01.308612+0000","last_clean":"2024-09-19T08:15:01.308612+0000","last_became_active":"2024-09-19T08:12:58.673890+0000","last_became_peered":"2024-09-19T08:12:58.673890+0000","last_unstale":"2024-09-19T08:15:01.308612+0000","last_undegraded":"2024-09-19T08:15:01.308612+0000","last_fullsized":"2024-09-19T08:15:01.308612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:54.939959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171020+0000","last_change":"2024-09-19T08:13:00.851071+0000","last_active":"2024-09-19T08:15:01.171020+0000","last_peered":"2024-09-19T08:15:01.171020+0000","last_clean":"2024-09-19T08:15:01.171020+0000","last_became_active":"2024-09-19T08:13:00.850756+0000","last_became_peered":"2024-09-19T08:13:00.850756+0000","last_unstale":"2024-09-19T08:15:01.171020+0000","last_undegraded":"2024-09-19T08:15:01.171020+0000","last_fullsized":"2024-09-19T08:15:01.171020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:52.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263021+0000","last_change":"2024-09-19T08:12:58.681466+0000","last_active":"2024-09-19T08:15:01.263021+0000","last_peered":"2024-09-19T08:15:01.263021+0000","last_clean":"2024-09-19T08:15:01.263021+0000","last_became_active":"2024-09-19T08:12:58.681113+0000","last_became_peered":"2024-09-19T08:12:58.681113+0000","last_unstale":"2024-09-19T08:15:01.263021+0000","last_undegraded":"2024-09-19T08:15:01.263021+0000","last_fullsized":"2024-09-19T08:15:01.263021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:49.610828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308386+0000","last_change":"2024-09-19T08:13:00.851272+0000","last_active":"2024-09-19T08:15:01.308386+0000","last_peered":"2024-09-19T08:15:01.308386+0000","last_clean":"2024-09-19T08:15:01.308386+0000","last_became_active":"2024-09-19T08:13:00.849098+0000","last_became_peered":"2024-09-19T08:13:00.849098+0000","last_unstale":"2024-09-19T08:15:01.308386+0000","last_undegraded":"2024-09-19T08:15:01.308386+0000","last_fullsized":"2024-09-19T08:15:01.308386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:05.643992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172513+0000","last_change":"2024-09-19T08:14:07.738913+0000","last_active":"2024-09-19T08:15:01.172513+0000","last_peered":"2024-09-19T08:15:01.172513+0000","last_clean":"2024-09-19T08:15:01.172513+0000","last_became_active":"2024-09-19T08:14:07.738560+0000","last_became_peered":"2024-09-19T08:14:07.738560+0000","last_unstale":"2024-09-19T08:15:01.172513+0000","last_undegraded":"2024-09-19T08:15:01.172513+0000","last_fullsized":"2024-09-19T08:15:01.172513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:35.731290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263192+0000","last_change":"2024-09-19T08:12:58.674610+0000","last_active":"2024-09-19T08:15:01.263192+0000","last_peered":"2024-09-19T08:15:01.263192+0000","last_clean":"2024-09-19T08:15:01.263192+0000","last_became_active":"2024-09-19T08:12:58.674247+0000","last_became_peered":"2024-09-19T08:12:58.674247+0000","last_unstale":"2024-09-19T08:15:01.263192+0000","last_undegraded":"2024-09-19T08:15:01.263192+0000","last_fullsized":"2024-09-19T08:15:01.263192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:16.323574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223395+0000","last_change":"2024-09-19T08:13:00.858512+0000","last_active":"2024-09-19T08:15:01.223395+0000","last_peered":"2024-09-19T08:15:01.223395+0000","last_clean":"2024-09-19T08:15:01.223395+0000","last_became_active":"2024-09-19T08:13:00.857897+0000","last_became_peered":"2024-09-19T08:13:00.857897+0000","last_unstale":"2024-09-19T08:15:01.223395+0000","last_undegraded":"2024-09-19T08:15:01.223395+0000","last_fullsized":"2024-09-19T08:15:01.223395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263238+0000","last_change":"2024-09-19T08:12:58.682207+0000","last_active":"2024-09-19T08:15:01.263238+0000","last_peered":"2024-09-19T08:15:01.263238+0000","last_clean":"2024-09-19T08:15:01.263238+0000","last_became_active":"2024-09-19T08:12:58.681842+0000","last_became_peered":"2024-09-19T08:12:58.681842+0000","last_unstale":"2024-09-19T08:15:01.263238+0000","last_undegraded":"2024-09-19T08:15:01.263238+0000","last_fullsized":"2024-09-19T08:15:01.263238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.825858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308840+0000","last_change":"2024-09-19T08:13:00.846748+0000","last_active":"2024-09-19T08:15:01.308840+0000","last_peered":"2024-09-19T08:15:01.308840+0000","last_clean":"2024-09-19T08:15:01.308840+0000","last_became_active":"2024-09-19T08:13:00.846368+0000","last_became_peered":"2024-09-19T08:13:00.846368+0000","last_unstale":"2024-09-19T08:15:01.308840+0000","last_undegraded":"2024-09-19T08:15:01.308840+0000","last_fullsized":"2024-09-19T08:15:01.308840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:59.526512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171334+0000","last_change":"2024-09-19T08:12:58.716118+0000","last_active":"2024-09-19T08:15:01.171334+0000","last_peered":"2024-09-19T08:15:01.171334+0000","last_clean":"2024-09-19T08:15:01.171334+0000","last_became_active":"2024-09-19T08:12:58.715711+0000","last_became_peered":"2024-09-19T08:12:58.715711+0000","last_unstale":"2024-09-19T08:15:01.171334+0000","last_undegraded":"2024-09-19T08:15:01.171334+0000","last_fullsized":"2024-09-19T08:15:01.171334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:29.439887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171172+0000","last_change":"2024-09-19T08:13:00.848729+0000","last_active":"2024-09-19T08:15:01.171172+0000","last_peered":"2024-09-19T08:15:01.171172+0000","last_clean":"2024-09-19T08:15:01.171172+0000","last_became_active":"2024-09-19T08:13:00.848470+0000","last_became_peered":"2024-09-19T08:13:00.848470+0000","last_unstale":"2024-09-19T08:15:01.171172+0000","last_undegraded":"2024-09-19T08:15:01.171172+0000","last_fullsized":"2024-09-19T08:15:01.171172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:30.653101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262232+0000","last_change":"2024-09-19T08:12:58.681654+0000","last_active":"2024-09-19T08:15:01.262232+0000","last_peered":"2024-09-19T08:15:01.262232+0000","last_clean":"2024-09-19T08:15:01.262232+0000","last_became_active":"2024-09-19T08:12:58.681284+0000","last_became_peered":"2024-09-19T08:12:58.681284+0000","last_unstale":"2024-09-19T08:15:01.262232+0000","last_undegraded":"2024-09-19T08:15:01.262232+0000","last_fullsized":"2024-09-19T08:15:01.262232+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.429930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597810,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":784,"num_read_kb":501,"num_write":297,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":61306,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":1793430,"data_compressed":85517,"data_compressed_allocated":110592,"data_compressed_original":1605632,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":109176,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481828,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":454888,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710762,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":1246292,"data_compressed":51265,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":848206,"data_compressed":27863,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103220,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":958464,"data_stored":847992,"data_compressed":27863,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":1049004,"data_compressed":45109,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":933888,"data_stored":454379,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":1178660,"data_compressed":51265,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":517387,"data_compressed":18328,"data_compressed_allocated":24576,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":979506,"data_compressed":34019,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":950272,"data_stored":456449,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":515901,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":455650,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":69101,"data_compressed":9611,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134003,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":393820,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201635,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:15:05.845 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:15:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.983+0000 7f7b1dc23640 1 -- 172.21.15.22:0/242273818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b18103610 msgr2=0x7f7b1810baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.983+0000 7f7b1dc23640 1 --2- 172.21.15.22:0/242273818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b18103610 0x7f7b1810baf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009900 tx=0x7f7b0002a480 comp rx=0 tx=0).stop 2024-09-19T08:15:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.984+0000 7f7b1dc23640 1 -- 172.21.15.22:0/242273818 shutdown_connections 2024-09-19T08:15:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.984+0000 7f7b1dc23640 1 --2- 172.21.15.22:0/242273818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b1810c0c0 0x7f7b1810e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.984+0000 7f7b1dc23640 1 --2- 172.21.15.22:0/242273818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b18103610 0x7f7b1810baf0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.984+0000 7f7b1dc23640 1 --2- 172.21.15.22:0/242273818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b18102cf0 0x7f7b181030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.984+0000 7f7b1dc23640 1 -- 172.21.15.22:0/242273818 >> 172.21.15.22:0/242273818 conn(0x7f7b18077a10 msgr2=0x7f7b180fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.984+0000 7f7b1dc23640 1 -- 172.21.15.22:0/242273818 shutdown_connections 2024-09-19T08:15:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.984+0000 7f7b1dc23640 1 -- 172.21.15.22:0/242273818 wait complete. 2024-09-19T08:15:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.984+0000 7f7b1dc23640 1 Processor -- start 2024-09-19T08:15:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.984+0000 7f7b1dc23640 1 -- start start 2024-09-19T08:15:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b1dc23640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b18102cf0 0x7f7b18198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b1dc23640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b18103610 0x7f7b18198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b1dc23640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b1810c0c0 0x7f7b1819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b1dc23640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7b18111e90 con 0x7f7b18103610 2024-09-19T08:15:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b1dc23640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7b18111d10 con 0x7f7b1810c0c0 2024-09-19T08:15:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b1dc23640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7b18112010 con 0x7f7b18102cf0 2024-09-19T08:15:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b16ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b18103610 0x7f7b18198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b16ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b18103610 0x7f7b18198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42484/0 (socket says 172.21.15.22:42484) 2024-09-19T08:15:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b16ffd640 1 -- 172.21.15.22:0/3336557196 learned_addr learned my addr 172.21.15.22:0/3336557196 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b177fe640 1 --2- 172.21.15.22:0/3336557196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b18102cf0 0x7f7b18198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b17fff640 1 --2- 172.21.15.22:0/3336557196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b1810c0c0 0x7f7b1819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b16ffd640 1 -- 172.21.15.22:0/3336557196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b18102cf0 msgr2=0x7f7b18198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b16ffd640 1 --2- 172.21.15.22:0/3336557196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b18102cf0 0x7f7b18198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b16ffd640 1 -- 172.21.15.22:0/3336557196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b1810c0c0 msgr2=0x7f7b1819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b16ffd640 1 --2- 172.21.15.22:0/3336557196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b1810c0c0 0x7f7b1819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b16ffd640 1 -- 172.21.15.22:0/3336557196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1819d430 con 0x7f7b18103610 2024-09-19T08:15:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b16ffd640 1 --2- 172.21.15.22:0/3336557196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b18103610 0x7f7b18198980 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7b00006010 tx=0x7f7b000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b14ff9640 1 -- 172.21.15.22:0/3336557196 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0003f030 con 0x7f7b18103610 2024-09-19T08:15:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b1dc23640 1 -- 172.21.15.22:0/3336557196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1819d6c0 con 0x7f7b18103610 2024-09-19T08:15:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b14ff9640 1 -- 172.21.15.22:0/3336557196 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b00003ce0 con 0x7f7b18103610 2024-09-19T08:15:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b14ff9640 1 -- 172.21.15.22:0/3336557196 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00035070 con 0x7f7b18103610 2024-09-19T08:15:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.985+0000 7f7b1dc23640 1 -- 172.21.15.22:0/3336557196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b181a4fb0 con 0x7f7b18103610 2024-09-19T08:15:05.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.987+0000 7f7b1dc23640 1 -- 172.21.15.22:0/3336557196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b181030d0 con 0x7f7b18103610 2024-09-19T08:15:05.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.988+0000 7f7b14ff9640 1 -- 172.21.15.22:0/3336557196 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f7b00042030 con 0x7f7b18103610 2024-09-19T08:15:05.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.989+0000 7f7b14ff9640 1 --2- 172.21.15.22:0/3336557196 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7b040a34e0 0x7f7b040a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.989+0000 7f7b177fe640 1 --2- 172.21.15.22:0/3336557196 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7b040a34e0 0x7f7b040a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.989+0000 7f7b14ff9640 1 -- 172.21.15.22:0/3336557196 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7b000f0610 con 0x7f7b18103610 2024-09-19T08:15:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.989+0000 7f7b177fe640 1 --2- 172.21.15.22:0/3336557196 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7b040a34e0 0x7f7b040a59a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b08005fd0 tx=0x7f7b0800b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:05.992+0000 7f7b14ff9640 1 -- 172.21.15.22:0/3336557196 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7b000bc890 con 0x7f7b18103610 2024-09-19T08:15:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.166+0000 7f7b1dc23640 1 -- 172.21.15.22:0/3336557196 --> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b18062630 con 0x7f7b040a34e0 2024-09-19T08:15:06.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.172+0000 7f7b14ff9640 1 -- 172.21.15.22:0/3336557196 <== mgr.15141 v2:172.21.15.22:6800/497251935 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258026 (secure 0 0 0) 0x7f7b18062630 con 0x7f7b040a34e0 2024-09-19T08:15:06.178 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:06.180 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T08:15:06.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.175+0000 7f7b1dc23640 1 -- 172.21.15.22:0/3336557196 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7b040a34e0 msgr2=0x7f7b040a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:06.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.175+0000 7f7b1dc23640 1 --2- 172.21.15.22:0/3336557196 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7b040a34e0 0x7f7b040a59a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b08005fd0 tx=0x7f7b0800b390 comp rx=0 tx=0).stop 2024-09-19T08:15:06.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.175+0000 7f7b1dc23640 1 -- 172.21.15.22:0/3336557196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b18103610 msgr2=0x7f7b18198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:06.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.175+0000 7f7b1dc23640 1 --2- 172.21.15.22:0/3336557196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b18103610 0x7f7b18198980 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7b00006010 tx=0x7f7b000042f0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.175+0000 7f7b1dc23640 1 -- 172.21.15.22:0/3336557196 shutdown_connections 2024-09-19T08:15:06.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.175+0000 7f7b1dc23640 1 --2- 172.21.15.22:0/3336557196 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7b040a34e0 0x7f7b040a59a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.175+0000 7f7b1dc23640 1 --2- 172.21.15.22:0/3336557196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b1810c0c0 0x7f7b1819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.175+0000 7f7b1dc23640 1 --2- 172.21.15.22:0/3336557196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b18103610 0x7f7b18198980 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.175+0000 7f7b1dc23640 1 --2- 172.21.15.22:0/3336557196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b18102cf0 0x7f7b18198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.175+0000 7f7b1dc23640 1 -- 172.21.15.22:0/3336557196 >> 172.21.15.22:0/3336557196 conn(0x7f7b18077a10 msgr2=0x7f7b180fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:06.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.176+0000 7f7b1dc23640 1 -- 172.21.15.22:0/3336557196 shutdown_connections 2024-09-19T08:15:06.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.176+0000 7f7b1dc23640 1 -- 172.21.15.22:0/3336557196 wait complete. 2024-09-19T08:15:06.188 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T08:15:05.185903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188412,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":920,"num_read_kb":620,"num_write":480,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1242042,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381604,"kb_used_data":11520,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560220,"statfs":{"total":1151940427776,"available":1151549665280,"internally_reserved":0,"allocated":11796480,"data_stored":9004314,"data_compressed":299297,"data_compressed_allocated":393216,"data_compressed_original":3670016,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.993097"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309110+0000","last_change":"2024-09-19T08:13:00.853947+0000","last_active":"2024-09-19T08:15:01.309110+0000","last_peered":"2024-09-19T08:15:01.309110+0000","last_clean":"2024-09-19T08:15:01.309110+0000","last_became_active":"2024-09-19T08:13:00.852951+0000","last_became_peered":"2024-09-19T08:13:00.852951+0000","last_unstale":"2024-09-19T08:15:01.309110+0000","last_undegraded":"2024-09-19T08:15:01.309110+0000","last_fullsized":"2024-09-19T08:15:01.309110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:25.786828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308563+0000","last_change":"2024-09-19T08:13:00.853389+0000","last_active":"2024-09-19T08:15:01.308563+0000","last_peered":"2024-09-19T08:15:01.308563+0000","last_clean":"2024-09-19T08:15:01.308563+0000","last_became_active":"2024-09-19T08:13:00.853102+0000","last_became_peered":"2024-09-19T08:13:00.853102+0000","last_unstale":"2024-09-19T08:15:01.308563+0000","last_undegraded":"2024-09-19T08:15:01.308563+0000","last_fullsized":"2024-09-19T08:15:01.308563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:42.773917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308818+0000","last_change":"2024-09-19T08:13:00.846544+0000","last_active":"2024-09-19T08:15:01.308818+0000","last_peered":"2024-09-19T08:15:01.308818+0000","last_clean":"2024-09-19T08:15:01.308818+0000","last_became_active":"2024-09-19T08:13:00.846229+0000","last_became_peered":"2024-09-19T08:13:00.846229+0000","last_unstale":"2024-09-19T08:15:01.308818+0000","last_undegraded":"2024-09-19T08:15:01.308818+0000","last_fullsized":"2024-09-19T08:15:01.308818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:43.471684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172883+0000","last_change":"2024-09-19T08:13:00.848149+0000","last_active":"2024-09-19T08:15:01.172883+0000","last_peered":"2024-09-19T08:15:01.172883+0000","last_clean":"2024-09-19T08:15:01.172883+0000","last_became_active":"2024-09-19T08:13:00.847744+0000","last_became_peered":"2024-09-19T08:13:00.847744+0000","last_unstale":"2024-09-19T08:15:01.172883+0000","last_undegraded":"2024-09-19T08:15:01.172883+0000","last_fullsized":"2024-09-19T08:15:01.172883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:01.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263000+0000","last_change":"2024-09-19T08:13:00.841862+0000","last_active":"2024-09-19T08:15:01.263000+0000","last_peered":"2024-09-19T08:15:01.263000+0000","last_clean":"2024-09-19T08:15:01.263000+0000","last_became_active":"2024-09-19T08:13:00.841331+0000","last_became_peered":"2024-09-19T08:13:00.841331+0000","last_unstale":"2024-09-19T08:15:01.263000+0000","last_undegraded":"2024-09-19T08:15:01.263000+0000","last_fullsized":"2024-09-19T08:15:01.263000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:54.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271477+0000","last_change":"2024-09-19T08:13:00.849626+0000","last_active":"2024-09-19T08:15:01.271477+0000","last_peered":"2024-09-19T08:15:01.271477+0000","last_clean":"2024-09-19T08:15:01.271477+0000","last_became_active":"2024-09-19T08:13:00.849034+0000","last_became_peered":"2024-09-19T08:13:00.849034+0000","last_unstale":"2024-09-19T08:15:01.271477+0000","last_undegraded":"2024-09-19T08:15:01.271477+0000","last_fullsized":"2024-09-19T08:15:01.271477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:39.118978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308024+0000","last_change":"2024-09-19T08:13:00.847522+0000","last_active":"2024-09-19T08:15:01.308024+0000","last_peered":"2024-09-19T08:15:01.308024+0000","last_clean":"2024-09-19T08:15:01.308024+0000","last_became_active":"2024-09-19T08:13:00.847117+0000","last_became_peered":"2024-09-19T08:13:00.847117+0000","last_unstale":"2024-09-19T08:15:01.308024+0000","last_undegraded":"2024-09-19T08:15:01.308024+0000","last_fullsized":"2024-09-19T08:15:01.308024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.836385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170733+0000","last_change":"2024-09-19T08:13:00.849544+0000","last_active":"2024-09-19T08:15:01.170733+0000","last_peered":"2024-09-19T08:15:01.170733+0000","last_clean":"2024-09-19T08:15:01.170733+0000","last_became_active":"2024-09-19T08:13:00.849170+0000","last_became_peered":"2024-09-19T08:13:00.849170+0000","last_unstale":"2024-09-19T08:15:01.170733+0000","last_undegraded":"2024-09-19T08:15:01.170733+0000","last_fullsized":"2024-09-19T08:15:01.170733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:28.371965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223727+0000","last_change":"2024-09-19T08:13:00.856052+0000","last_active":"2024-09-19T08:15:01.223727+0000","last_peered":"2024-09-19T08:15:01.223727+0000","last_clean":"2024-09-19T08:15:01.223727+0000","last_became_active":"2024-09-19T08:13:00.855291+0000","last_became_peered":"2024-09-19T08:13:00.855291+0000","last_unstale":"2024-09-19T08:15:01.223727+0000","last_undegraded":"2024-09-19T08:15:01.223727+0000","last_fullsized":"2024-09-19T08:15:01.223727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:18.581867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271887+0000","last_change":"2024-09-19T08:13:00.850145+0000","last_active":"2024-09-19T08:15:01.271887+0000","last_peered":"2024-09-19T08:15:01.271887+0000","last_clean":"2024-09-19T08:15:01.271887+0000","last_became_active":"2024-09-19T08:13:00.849798+0000","last_became_peered":"2024-09-19T08:13:00.849798+0000","last_unstale":"2024-09-19T08:15:01.271887+0000","last_undegraded":"2024-09-19T08:15:01.271887+0000","last_fullsized":"2024-09-19T08:15:01.271887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:21.400000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308275+0000","last_change":"2024-09-19T08:13:00.859153+0000","last_active":"2024-09-19T08:15:01.308275+0000","last_peered":"2024-09-19T08:15:01.308275+0000","last_clean":"2024-09-19T08:15:01.308275+0000","last_became_active":"2024-09-19T08:13:00.858473+0000","last_became_peered":"2024-09-19T08:13:00.858473+0000","last_unstale":"2024-09-19T08:15:01.308275+0000","last_undegraded":"2024-09-19T08:15:01.308275+0000","last_fullsized":"2024-09-19T08:15:01.308275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:07.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172922+0000","last_change":"2024-09-19T08:13:00.850797+0000","last_active":"2024-09-19T08:15:01.172922+0000","last_peered":"2024-09-19T08:15:01.172922+0000","last_clean":"2024-09-19T08:15:01.172922+0000","last_became_active":"2024-09-19T08:13:00.850393+0000","last_became_peered":"2024-09-19T08:13:00.850393+0000","last_unstale":"2024-09-19T08:15:01.172922+0000","last_undegraded":"2024-09-19T08:15:01.172922+0000","last_fullsized":"2024-09-19T08:15:01.172922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:59.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310082+0000","last_change":"2024-09-19T08:13:00.850835+0000","last_active":"2024-09-19T08:15:01.310082+0000","last_peered":"2024-09-19T08:15:01.310082+0000","last_clean":"2024-09-19T08:15:01.310082+0000","last_became_active":"2024-09-19T08:13:00.850518+0000","last_became_peered":"2024-09-19T08:13:00.850518+0000","last_unstale":"2024-09-19T08:15:01.310082+0000","last_undegraded":"2024-09-19T08:15:01.310082+0000","last_fullsized":"2024-09-19T08:15:01.310082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.178530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308450+0000","last_change":"2024-09-19T08:13:00.853934+0000","last_active":"2024-09-19T08:15:01.308450+0000","last_peered":"2024-09-19T08:15:01.308450+0000","last_clean":"2024-09-19T08:15:01.308450+0000","last_became_active":"2024-09-19T08:13:00.853604+0000","last_became_peered":"2024-09-19T08:13:00.853604+0000","last_unstale":"2024-09-19T08:15:01.308450+0000","last_undegraded":"2024-09-19T08:15:01.308450+0000","last_fullsized":"2024-09-19T08:15:01.308450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:43.622526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308262+0000","last_change":"2024-09-19T08:14:07.741370+0000","last_active":"2024-09-19T08:15:01.308262+0000","last_peered":"2024-09-19T08:15:01.308262+0000","last_clean":"2024-09-19T08:15:01.308262+0000","last_became_active":"2024-09-19T08:14:07.741116+0000","last_became_peered":"2024-09-19T08:14:07.741116+0000","last_unstale":"2024-09-19T08:15:01.308262+0000","last_undegraded":"2024-09-19T08:15:01.308262+0000","last_fullsized":"2024-09-19T08:15:01.308262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:45.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223320+0000","last_change":"2024-09-19T08:13:00.850266+0000","last_active":"2024-09-19T08:15:01.223320+0000","last_peered":"2024-09-19T08:15:01.223320+0000","last_clean":"2024-09-19T08:15:01.223320+0000","last_became_active":"2024-09-19T08:13:00.849928+0000","last_became_peered":"2024-09-19T08:13:00.849928+0000","last_unstale":"2024-09-19T08:15:01.223320+0000","last_undegraded":"2024-09-19T08:15:01.223320+0000","last_fullsized":"2024-09-19T08:15:01.223320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:38.231331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309177+0000","last_change":"2024-09-19T08:13:00.852551+0000","last_active":"2024-09-19T08:15:01.309177+0000","last_peered":"2024-09-19T08:15:01.309177+0000","last_clean":"2024-09-19T08:15:01.309177+0000","last_became_active":"2024-09-19T08:13:00.852225+0000","last_became_peered":"2024-09-19T08:13:00.852225+0000","last_unstale":"2024-09-19T08:15:01.309177+0000","last_undegraded":"2024-09-19T08:15:01.309177+0000","last_fullsized":"2024-09-19T08:15:01.309177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:33.115108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.261006+0000","last_change":"2024-09-19T08:13:00.854033+0000","last_active":"2024-09-19T08:15:01.261006+0000","last_peered":"2024-09-19T08:15:01.261006+0000","last_clean":"2024-09-19T08:15:01.261006+0000","last_became_active":"2024-09-19T08:13:00.853660+0000","last_became_peered":"2024-09-19T08:13:00.853660+0000","last_unstale":"2024-09-19T08:15:01.261006+0000","last_undegraded":"2024-09-19T08:15:01.261006+0000","last_fullsized":"2024-09-19T08:15:01.261006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:35.602420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308765+0000","last_change":"2024-09-19T08:13:00.850658+0000","last_active":"2024-09-19T08:15:01.308765+0000","last_peered":"2024-09-19T08:15:01.308765+0000","last_clean":"2024-09-19T08:15:01.308765+0000","last_became_active":"2024-09-19T08:13:00.850324+0000","last_became_peered":"2024-09-19T08:13:00.850324+0000","last_unstale":"2024-09-19T08:15:01.308765+0000","last_undegraded":"2024-09-19T08:15:01.308765+0000","last_fullsized":"2024-09-19T08:15:01.308765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:59:49.004605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308718+0000","last_change":"2024-09-19T08:14:07.741837+0000","last_active":"2024-09-19T08:15:01.308718+0000","last_peered":"2024-09-19T08:15:01.308718+0000","last_clean":"2024-09-19T08:15:01.308718+0000","last_became_active":"2024-09-19T08:14:07.741398+0000","last_became_peered":"2024-09-19T08:14:07.741398+0000","last_unstale":"2024-09-19T08:15:01.308718+0000","last_undegraded":"2024-09-19T08:15:01.308718+0000","last_fullsized":"2024-09-19T08:15:01.308718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:56.716663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310236+0000","last_change":"2024-09-19T08:13:00.855215+0000","last_active":"2024-09-19T08:15:01.310236+0000","last_peered":"2024-09-19T08:15:01.310236+0000","last_clean":"2024-09-19T08:15:01.310236+0000","last_became_active":"2024-09-19T08:13:00.854878+0000","last_became_peered":"2024-09-19T08:13:00.854878+0000","last_unstale":"2024-09-19T08:15:01.310236+0000","last_undegraded":"2024-09-19T08:15:01.310236+0000","last_fullsized":"2024-09-19T08:15:01.310236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:48.216066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223444+0000","last_change":"2024-09-19T08:13:00.856995+0000","last_active":"2024-09-19T08:15:01.223444+0000","last_peered":"2024-09-19T08:15:01.223444+0000","last_clean":"2024-09-19T08:15:01.223444+0000","last_became_active":"2024-09-19T08:13:00.856645+0000","last_became_peered":"2024-09-19T08:13:00.856645+0000","last_unstale":"2024-09-19T08:15:01.223444+0000","last_undegraded":"2024-09-19T08:15:01.223444+0000","last_fullsized":"2024-09-19T08:15:01.223444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:20.699293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170753+0000","last_change":"2024-09-19T08:13:00.853362+0000","last_active":"2024-09-19T08:15:01.170753+0000","last_peered":"2024-09-19T08:15:01.170753+0000","last_clean":"2024-09-19T08:15:01.170753+0000","last_became_active":"2024-09-19T08:13:00.853046+0000","last_became_peered":"2024-09-19T08:13:00.853046+0000","last_unstale":"2024-09-19T08:15:01.170753+0000","last_undegraded":"2024-09-19T08:15:01.170753+0000","last_fullsized":"2024-09-19T08:15:01.170753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172184+0000","last_change":"2024-09-19T08:13:00.845175+0000","last_active":"2024-09-19T08:15:01.172184+0000","last_peered":"2024-09-19T08:15:01.172184+0000","last_clean":"2024-09-19T08:15:01.172184+0000","last_became_active":"2024-09-19T08:13:00.844857+0000","last_became_peered":"2024-09-19T08:13:00.844857+0000","last_unstale":"2024-09-19T08:15:01.172184+0000","last_undegraded":"2024-09-19T08:15:01.172184+0000","last_fullsized":"2024-09-19T08:15:01.172184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:40.818956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171025+0000","last_change":"2024-09-19T08:13:00.847769+0000","last_active":"2024-09-19T08:15:01.171025+0000","last_peered":"2024-09-19T08:15:01.171025+0000","last_clean":"2024-09-19T08:15:01.171025+0000","last_became_active":"2024-09-19T08:13:00.847454+0000","last_became_peered":"2024-09-19T08:13:00.847454+0000","last_unstale":"2024-09-19T08:15:01.171025+0000","last_undegraded":"2024-09-19T08:15:01.171025+0000","last_fullsized":"2024-09-19T08:15:01.171025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.583051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308835+0000","last_change":"2024-09-19T08:13:00.859882+0000","last_active":"2024-09-19T08:15:01.308835+0000","last_peered":"2024-09-19T08:15:01.308835+0000","last_clean":"2024-09-19T08:15:01.308835+0000","last_became_active":"2024-09-19T08:13:00.859491+0000","last_became_peered":"2024-09-19T08:13:00.859491+0000","last_unstale":"2024-09-19T08:15:01.308835+0000","last_undegraded":"2024-09-19T08:15:01.308835+0000","last_fullsized":"2024-09-19T08:15:01.308835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:13.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308625+0000","last_change":"2024-09-19T08:13:00.845584+0000","last_active":"2024-09-19T08:15:01.308625+0000","last_peered":"2024-09-19T08:15:01.308625+0000","last_clean":"2024-09-19T08:15:01.308625+0000","last_became_active":"2024-09-19T08:13:00.845166+0000","last_became_peered":"2024-09-19T08:13:00.845166+0000","last_unstale":"2024-09-19T08:15:01.308625+0000","last_undegraded":"2024-09-19T08:15:01.308625+0000","last_fullsized":"2024-09-19T08:15:01.308625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:02.705793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310490+0000","last_change":"2024-09-19T08:13:00.838069+0000","last_active":"2024-09-19T08:15:01.310490+0000","last_peered":"2024-09-19T08:15:01.310490+0000","last_clean":"2024-09-19T08:15:01.310490+0000","last_became_active":"2024-09-19T08:13:00.837736+0000","last_became_peered":"2024-09-19T08:13:00.837736+0000","last_unstale":"2024-09-19T08:15:01.310490+0000","last_undegraded":"2024-09-19T08:15:01.310490+0000","last_fullsized":"2024-09-19T08:15:01.310490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:01.388290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308563+0000","last_change":"2024-09-19T08:13:00.841507+0000","last_active":"2024-09-19T08:15:01.308563+0000","last_peered":"2024-09-19T08:15:01.308563+0000","last_clean":"2024-09-19T08:15:01.308563+0000","last_became_active":"2024-09-19T08:13:00.841161+0000","last_became_peered":"2024-09-19T08:13:00.841161+0000","last_unstale":"2024-09-19T08:15:01.308563+0000","last_undegraded":"2024-09-19T08:15:01.308563+0000","last_fullsized":"2024-09-19T08:15:01.308563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:41.862290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308471+0000","last_change":"2024-09-19T08:13:00.847657+0000","last_active":"2024-09-19T08:15:01.308471+0000","last_peered":"2024-09-19T08:15:01.308471+0000","last_clean":"2024-09-19T08:15:01.308471+0000","last_became_active":"2024-09-19T08:13:00.846849+0000","last_became_peered":"2024-09-19T08:13:00.846849+0000","last_unstale":"2024-09-19T08:15:01.308471+0000","last_undegraded":"2024-09-19T08:15:01.308471+0000","last_fullsized":"2024-09-19T08:15:01.308471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:35.219913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309005+0000","last_change":"2024-09-19T08:13:00.845971+0000","last_active":"2024-09-19T08:15:01.309005+0000","last_peered":"2024-09-19T08:15:01.309005+0000","last_clean":"2024-09-19T08:15:01.309005+0000","last_became_active":"2024-09-19T08:13:00.845626+0000","last_became_peered":"2024-09-19T08:13:00.845626+0000","last_unstale":"2024-09-19T08:15:01.309005+0000","last_undegraded":"2024-09-19T08:15:01.309005+0000","last_fullsized":"2024-09-19T08:15:01.309005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:36.945627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308386+0000","last_change":"2024-09-19T08:14:07.738450+0000","last_active":"2024-09-19T08:15:01.308386+0000","last_peered":"2024-09-19T08:15:01.308386+0000","last_clean":"2024-09-19T08:15:01.308386+0000","last_became_active":"2024-09-19T08:14:07.738028+0000","last_became_peered":"2024-09-19T08:14:07.738028+0000","last_unstale":"2024-09-19T08:15:01.308386+0000","last_undegraded":"2024-09-19T08:15:01.308386+0000","last_fullsized":"2024-09-19T08:15:01.308386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:08.659158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262954+0000","last_change":"2024-09-19T08:13:00.848325+0000","last_active":"2024-09-19T08:15:01.262954+0000","last_peered":"2024-09-19T08:15:01.262954+0000","last_clean":"2024-09-19T08:15:01.262954+0000","last_became_active":"2024-09-19T08:13:00.848015+0000","last_became_peered":"2024-09-19T08:13:00.848015+0000","last_unstale":"2024-09-19T08:15:01.262954+0000","last_undegraded":"2024-09-19T08:15:01.262954+0000","last_fullsized":"2024-09-19T08:15:01.262954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:10.948451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:02.038549+0000","last_change":"2024-09-19T08:12:58.681134+0000","last_active":"2024-09-19T08:15:02.038549+0000","last_peered":"2024-09-19T08:15:02.038549+0000","last_clean":"2024-09-19T08:15:02.038549+0000","last_became_active":"2024-09-19T08:12:58.680764+0000","last_became_peered":"2024-09-19T08:12:58.680764+0000","last_unstale":"2024-09-19T08:15:02.038549+0000","last_undegraded":"2024-09-19T08:15:02.038549+0000","last_fullsized":"2024-09-19T08:15:02.038549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:24.939263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172533+0000","last_change":"2024-09-19T08:13:00.844319+0000","last_active":"2024-09-19T08:15:01.172533+0000","last_peered":"2024-09-19T08:15:01.172533+0000","last_clean":"2024-09-19T08:15:01.172533+0000","last_became_active":"2024-09-19T08:13:00.843994+0000","last_became_peered":"2024-09-19T08:13:00.843994+0000","last_unstale":"2024-09-19T08:15:01.172533+0000","last_undegraded":"2024-09-19T08:15:01.172533+0000","last_fullsized":"2024-09-19T08:15:01.172533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:46.937466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271532+0000","last_change":"2024-09-19T08:12:58.676541+0000","last_active":"2024-09-19T08:15:01.271532+0000","last_peered":"2024-09-19T08:15:01.271532+0000","last_clean":"2024-09-19T08:15:01.271532+0000","last_became_active":"2024-09-19T08:12:58.675988+0000","last_became_peered":"2024-09-19T08:12:58.675988+0000","last_unstale":"2024-09-19T08:15:01.271532+0000","last_undegraded":"2024-09-19T08:15:01.271532+0000","last_fullsized":"2024-09-19T08:15:01.271532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:01.434986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309788+0000","last_change":"2024-09-19T08:12:58.677510+0000","last_active":"2024-09-19T08:15:01.309788+0000","last_peered":"2024-09-19T08:15:01.309788+0000","last_clean":"2024-09-19T08:15:01.309788+0000","last_became_active":"2024-09-19T08:12:58.677200+0000","last_became_peered":"2024-09-19T08:12:58.677200+0000","last_unstale":"2024-09-19T08:15:01.309788+0000","last_undegraded":"2024-09-19T08:15:01.309788+0000","last_fullsized":"2024-09-19T08:15:01.309788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:36.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170825+0000","last_change":"2024-09-19T08:13:00.853430+0000","last_active":"2024-09-19T08:15:01.170825+0000","last_peered":"2024-09-19T08:15:01.170825+0000","last_clean":"2024-09-19T08:15:01.170825+0000","last_became_active":"2024-09-19T08:13:00.853096+0000","last_became_peered":"2024-09-19T08:13:00.853096+0000","last_unstale":"2024-09-19T08:15:01.170825+0000","last_undegraded":"2024-09-19T08:15:01.170825+0000","last_fullsized":"2024-09-19T08:15:01.170825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:07.147021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172384+0000","last_change":"2024-09-19T08:12:58.680081+0000","last_active":"2024-09-19T08:15:01.172384+0000","last_peered":"2024-09-19T08:15:01.172384+0000","last_clean":"2024-09-19T08:15:01.172384+0000","last_became_active":"2024-09-19T08:12:58.679704+0000","last_became_peered":"2024-09-19T08:12:58.679704+0000","last_unstale":"2024-09-19T08:15:01.172384+0000","last_undegraded":"2024-09-19T08:15:01.172384+0000","last_fullsized":"2024-09-19T08:15:01.172384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:32.942586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.260317+0000","last_change":"2024-09-19T08:13:00.849394+0000","last_active":"2024-09-19T08:15:01.260317+0000","last_peered":"2024-09-19T08:15:01.260317+0000","last_clean":"2024-09-19T08:15:01.260317+0000","last_became_active":"2024-09-19T08:13:00.849051+0000","last_became_peered":"2024-09-19T08:13:00.849051+0000","last_unstale":"2024-09-19T08:15:01.260317+0000","last_undegraded":"2024-09-19T08:15:01.260317+0000","last_fullsized":"2024-09-19T08:15:01.260317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:23.012928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'5","reported_seq":140,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.418714+0000","last_change":"2024-09-19T08:12:58.676121+0000","last_active":"2024-09-19T08:15:01.418714+0000","last_peered":"2024-09-19T08:15:01.418714+0000","last_clean":"2024-09-19T08:15:01.418714+0000","last_became_active":"2024-09-19T08:12:58.675693+0000","last_became_peered":"2024-09-19T08:12:58.675693+0000","last_unstale":"2024-09-19T08:15:01.418714+0000","last_undegraded":"2024-09-19T08:15:01.418714+0000","last_fullsized":"2024-09-19T08:15:01.418714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:34.882190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223583+0000","last_change":"2024-09-19T08:13:00.858735+0000","last_active":"2024-09-19T08:15:01.223583+0000","last_peered":"2024-09-19T08:15:01.223583+0000","last_clean":"2024-09-19T08:15:01.223583+0000","last_became_active":"2024-09-19T08:13:00.858218+0000","last_became_peered":"2024-09-19T08:13:00.858218+0000","last_unstale":"2024-09-19T08:15:01.223583+0000","last_undegraded":"2024-09-19T08:15:01.223583+0000","last_fullsized":"2024-09-19T08:15:01.223583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:33.356810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308176+0000","last_change":"2024-09-19T08:14:07.743785+0000","last_active":"2024-09-19T08:15:01.308176+0000","last_peered":"2024-09-19T08:15:01.308176+0000","last_clean":"2024-09-19T08:15:01.308176+0000","last_became_active":"2024-09-19T08:14:07.739531+0000","last_became_peered":"2024-09-19T08:14:07.739531+0000","last_unstale":"2024-09-19T08:15:01.308176+0000","last_undegraded":"2024-09-19T08:15:01.308176+0000","last_fullsized":"2024-09-19T08:15:01.308176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:43.367704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308976+0000","last_change":"2024-09-19T08:13:00.843852+0000","last_active":"2024-09-19T08:15:01.308976+0000","last_peered":"2024-09-19T08:15:01.308976+0000","last_clean":"2024-09-19T08:15:01.308976+0000","last_became_active":"2024-09-19T08:13:00.843497+0000","last_became_peered":"2024-09-19T08:13:00.843497+0000","last_unstale":"2024-09-19T08:15:01.308976+0000","last_undegraded":"2024-09-19T08:15:01.308976+0000","last_fullsized":"2024-09-19T08:15:01.308976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:26.380024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172698+0000","last_change":"2024-09-19T08:12:58.717225+0000","last_active":"2024-09-19T08:15:01.172698+0000","last_peered":"2024-09-19T08:15:01.172698+0000","last_clean":"2024-09-19T08:15:01.172698+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:15:01.172698+0000","last_undegraded":"2024-09-19T08:15:01.172698+0000","last_fullsized":"2024-09-19T08:15:01.172698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:43.201907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223446+0000","last_change":"2024-09-19T08:13:00.845613+0000","last_active":"2024-09-19T08:15:01.223446+0000","last_peered":"2024-09-19T08:15:01.223446+0000","last_clean":"2024-09-19T08:15:01.223446+0000","last_became_active":"2024-09-19T08:13:00.845260+0000","last_became_peered":"2024-09-19T08:13:00.845260+0000","last_unstale":"2024-09-19T08:15:01.223446+0000","last_undegraded":"2024-09-19T08:15:01.223446+0000","last_fullsized":"2024-09-19T08:15:01.223446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:03:47.033972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'36","reported_seq":311,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.820230+0000","last_change":"2024-09-19T08:14:07.749616+0000","last_active":"2024-09-19T08:15:01.820230+0000","last_peered":"2024-09-19T08:15:01.820230+0000","last_clean":"2024-09-19T08:15:01.820230+0000","last_became_active":"2024-09-19T08:14:07.740298+0000","last_became_peered":"2024-09-19T08:14:07.740298+0000","last_unstale":"2024-09-19T08:15:01.820230+0000","last_undegraded":"2024-09-19T08:15:01.820230+0000","last_fullsized":"2024-09-19T08:15:01.820230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:28.796855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66829,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":225,"num_read_kb":177,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61172,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263216+0000","last_change":"2024-09-19T08:13:00.855603+0000","last_active":"2024-09-19T08:15:01.263216+0000","last_peered":"2024-09-19T08:15:01.263216+0000","last_clean":"2024-09-19T08:15:01.263216+0000","last_became_active":"2024-09-19T08:13:00.855241+0000","last_became_peered":"2024-09-19T08:13:00.855241+0000","last_unstale":"2024-09-19T08:15:01.263216+0000","last_undegraded":"2024-09-19T08:15:01.263216+0000","last_fullsized":"2024-09-19T08:15:01.263216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:28.811847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308653+0000","last_change":"2024-09-19T08:12:58.675897+0000","last_active":"2024-09-19T08:15:01.308653+0000","last_peered":"2024-09-19T08:15:01.308653+0000","last_clean":"2024-09-19T08:15:01.308653+0000","last_became_active":"2024-09-19T08:12:58.675525+0000","last_became_peered":"2024-09-19T08:12:58.675525+0000","last_unstale":"2024-09-19T08:15:01.308653+0000","last_undegraded":"2024-09-19T08:15:01.308653+0000","last_fullsized":"2024-09-19T08:15:01.308653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:54.518064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309636+0000","last_change":"2024-09-19T08:13:00.856420+0000","last_active":"2024-09-19T08:15:01.309636+0000","last_peered":"2024-09-19T08:15:01.309636+0000","last_clean":"2024-09-19T08:15:01.309636+0000","last_became_active":"2024-09-19T08:13:00.856100+0000","last_became_peered":"2024-09-19T08:13:00.856100+0000","last_unstale":"2024-09-19T08:15:01.309636+0000","last_undegraded":"2024-09-19T08:15:01.309636+0000","last_fullsized":"2024-09-19T08:15:01.309636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:58.684930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172292+0000","last_change":"2024-09-19T08:12:58.680778+0000","last_active":"2024-09-19T08:15:01.172292+0000","last_peered":"2024-09-19T08:15:01.172292+0000","last_clean":"2024-09-19T08:15:01.172292+0000","last_became_active":"2024-09-19T08:12:58.680364+0000","last_became_peered":"2024-09-19T08:12:58.680364+0000","last_unstale":"2024-09-19T08:15:01.172292+0000","last_undegraded":"2024-09-19T08:15:01.172292+0000","last_fullsized":"2024-09-19T08:15:01.172292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:10.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308288+0000","last_change":"2024-09-19T08:13:00.853415+0000","last_active":"2024-09-19T08:15:01.308288+0000","last_peered":"2024-09-19T08:15:01.308288+0000","last_clean":"2024-09-19T08:15:01.308288+0000","last_became_active":"2024-09-19T08:13:00.853101+0000","last_became_peered":"2024-09-19T08:13:00.853101+0000","last_unstale":"2024-09-19T08:15:01.308288+0000","last_undegraded":"2024-09-19T08:15:01.308288+0000","last_fullsized":"2024-09-19T08:15:01.308288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:42.375394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":149,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:02.037848+0000","last_change":"2024-09-19T08:12:58.680312+0000","last_active":"2024-09-19T08:15:02.037848+0000","last_peered":"2024-09-19T08:15:02.037848+0000","last_clean":"2024-09-19T08:15:02.037848+0000","last_became_active":"2024-09-19T08:12:58.679870+0000","last_became_peered":"2024-09-19T08:12:58.679870+0000","last_unstale":"2024-09-19T08:15:02.037848+0000","last_undegraded":"2024-09-19T08:15:02.037848+0000","last_fullsized":"2024-09-19T08:15:02.037848+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:42.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308034+0000","last_change":"2024-09-19T08:13:00.853920+0000","last_active":"2024-09-19T08:15:01.308034+0000","last_peered":"2024-09-19T08:15:01.308034+0000","last_clean":"2024-09-19T08:15:01.308034+0000","last_became_active":"2024-09-19T08:13:00.853594+0000","last_became_peered":"2024-09-19T08:13:00.853594+0000","last_unstale":"2024-09-19T08:15:01.308034+0000","last_undegraded":"2024-09-19T08:15:01.308034+0000","last_fullsized":"2024-09-19T08:15:01.308034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:31.746133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308308+0000","last_change":"2024-09-19T08:14:07.751888+0000","last_active":"2024-09-19T08:15:01.308308+0000","last_peered":"2024-09-19T08:15:01.308308+0000","last_clean":"2024-09-19T08:15:01.308308+0000","last_became_active":"2024-09-19T08:14:07.737201+0000","last_became_peered":"2024-09-19T08:14:07.737201+0000","last_unstale":"2024-09-19T08:15:01.308308+0000","last_undegraded":"2024-09-19T08:15:01.308308+0000","last_fullsized":"2024-09-19T08:15:01.308308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:44.095987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172245+0000","last_change":"2024-09-19T08:13:00.847061+0000","last_active":"2024-09-19T08:15:01.172245+0000","last_peered":"2024-09-19T08:15:01.172245+0000","last_clean":"2024-09-19T08:15:01.172245+0000","last_became_active":"2024-09-19T08:13:00.846742+0000","last_became_peered":"2024-09-19T08:13:00.846742+0000","last_unstale":"2024-09-19T08:15:01.172245+0000","last_undegraded":"2024-09-19T08:15:01.172245+0000","last_fullsized":"2024-09-19T08:15:01.172245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:21.785585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309033+0000","last_change":"2024-09-19T08:14:07.755277+0000","last_active":"2024-09-19T08:15:01.309033+0000","last_peered":"2024-09-19T08:15:01.309033+0000","last_clean":"2024-09-19T08:15:01.309033+0000","last_became_active":"2024-09-19T08:14:07.739773+0000","last_became_peered":"2024-09-19T08:14:07.739773+0000","last_unstale":"2024-09-19T08:15:01.309033+0000","last_undegraded":"2024-09-19T08:15:01.309033+0000","last_fullsized":"2024-09-19T08:15:01.309033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:56.848251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308865+0000","last_change":"2024-09-19T08:13:00.844940+0000","last_active":"2024-09-19T08:15:01.308865+0000","last_peered":"2024-09-19T08:15:01.308865+0000","last_clean":"2024-09-19T08:15:01.308865+0000","last_became_active":"2024-09-19T08:13:00.844454+0000","last_became_peered":"2024-09-19T08:13:00.844454+0000","last_unstale":"2024-09-19T08:15:01.308865+0000","last_undegraded":"2024-09-19T08:15:01.308865+0000","last_fullsized":"2024-09-19T08:15:01.308865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:15.577624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309967+0000","last_change":"2024-09-19T08:12:58.673301+0000","last_active":"2024-09-19T08:15:01.309967+0000","last_peered":"2024-09-19T08:15:01.309967+0000","last_clean":"2024-09-19T08:15:01.309967+0000","last_became_active":"2024-09-19T08:12:58.672981+0000","last_became_peered":"2024-09-19T08:12:58.672981+0000","last_unstale":"2024-09-19T08:15:01.309967+0000","last_undegraded":"2024-09-19T08:15:01.309967+0000","last_fullsized":"2024-09-19T08:15:01.309967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:03.963712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308223+0000","last_change":"2024-09-19T08:13:00.856620+0000","last_active":"2024-09-19T08:15:01.308223+0000","last_peered":"2024-09-19T08:15:01.308223+0000","last_clean":"2024-09-19T08:15:01.308223+0000","last_became_active":"2024-09-19T08:13:00.856310+0000","last_became_peered":"2024-09-19T08:13:00.856310+0000","last_unstale":"2024-09-19T08:15:01.308223+0000","last_undegraded":"2024-09-19T08:15:01.308223+0000","last_fullsized":"2024-09-19T08:15:01.308223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:16.918768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'39","reported_seq":71,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.371288+0000","last_change":"2024-09-19T08:12:58.672282+0000","last_active":"2024-09-19T08:15:01.371288+0000","last_peered":"2024-09-19T08:15:01.371288+0000","last_clean":"2024-09-19T08:15:01.371288+0000","last_became_active":"2024-09-19T08:12:58.671891+0000","last_became_peered":"2024-09-19T08:12:58.671891+0000","last_unstale":"2024-09-19T08:15:01.371288+0000","last_undegraded":"2024-09-19T08:15:01.371288+0000","last_fullsized":"2024-09-19T08:15:01.371288+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:41.731571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308128+0000","last_change":"2024-09-19T08:13:00.848107+0000","last_active":"2024-09-19T08:15:01.308128+0000","last_peered":"2024-09-19T08:15:01.308128+0000","last_clean":"2024-09-19T08:15:01.308128+0000","last_became_active":"2024-09-19T08:13:00.847632+0000","last_became_peered":"2024-09-19T08:13:00.847632+0000","last_unstale":"2024-09-19T08:15:01.308128+0000","last_undegraded":"2024-09-19T08:15:01.308128+0000","last_fullsized":"2024-09-19T08:15:01.308128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.491122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308832+0000","last_change":"2024-09-19T08:12:59.179200+0000","last_active":"2024-09-19T08:15:01.308832+0000","last_peered":"2024-09-19T08:15:01.308832+0000","last_clean":"2024-09-19T08:15:01.308832+0000","last_became_active":"2024-09-19T08:12:59.178762+0000","last_became_peered":"2024-09-19T08:12:59.178762+0000","last_unstale":"2024-09-19T08:15:01.308832+0000","last_undegraded":"2024-09-19T08:15:01.308832+0000","last_fullsized":"2024-09-19T08:15:01.308832+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:08.424106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308358+0000","last_change":"2024-09-19T08:13:00.855870+0000","last_active":"2024-09-19T08:15:01.308358+0000","last_peered":"2024-09-19T08:15:01.308358+0000","last_clean":"2024-09-19T08:15:01.308358+0000","last_became_active":"2024-09-19T08:13:00.855538+0000","last_became_peered":"2024-09-19T08:13:00.855538+0000","last_unstale":"2024-09-19T08:15:01.308358+0000","last_undegraded":"2024-09-19T08:15:01.308358+0000","last_fullsized":"2024-09-19T08:15:01.308358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308350+0000","last_change":"2024-09-19T08:11:48.661333+0000","last_active":"2024-09-19T08:15:01.308350+0000","last_peered":"2024-09-19T08:15:01.308350+0000","last_clean":"2024-09-19T08:15:01.308350+0000","last_became_active":"2024-09-19T08:11:48.660873+0000","last_became_peered":"2024-09-19T08:11:48.660873+0000","last_unstale":"2024-09-19T08:15:01.308350+0000","last_undegraded":"2024-09-19T08:15:01.308350+0000","last_fullsized":"2024-09-19T08:15:01.308350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_clean_scrub_stamp":"2024-09-19T08:08:25.372155+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:19.401292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172985+0000","last_change":"2024-09-19T08:12:58.675237+0000","last_active":"2024-09-19T08:15:01.172985+0000","last_peered":"2024-09-19T08:15:01.172985+0000","last_clean":"2024-09-19T08:15:01.172985+0000","last_became_active":"2024-09-19T08:12:58.674910+0000","last_became_peered":"2024-09-19T08:12:58.674910+0000","last_unstale":"2024-09-19T08:15:01.172985+0000","last_undegraded":"2024-09-19T08:15:01.172985+0000","last_fullsized":"2024-09-19T08:15:01.172985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:47.038328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171023+0000","last_change":"2024-09-19T08:13:00.846754+0000","last_active":"2024-09-19T08:15:01.171023+0000","last_peered":"2024-09-19T08:15:01.171023+0000","last_clean":"2024-09-19T08:15:01.171023+0000","last_became_active":"2024-09-19T08:13:00.846452+0000","last_became_peered":"2024-09-19T08:13:00.846452+0000","last_unstale":"2024-09-19T08:15:01.171023+0000","last_undegraded":"2024-09-19T08:15:01.171023+0000","last_fullsized":"2024-09-19T08:15:01.171023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:22.929689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308239+0000","last_change":"2024-09-19T08:12:58.678958+0000","last_active":"2024-09-19T08:15:01.308239+0000","last_peered":"2024-09-19T08:15:01.308239+0000","last_clean":"2024-09-19T08:15:01.308239+0000","last_became_active":"2024-09-19T08:12:58.678592+0000","last_became_peered":"2024-09-19T08:12:58.678592+0000","last_unstale":"2024-09-19T08:15:01.308239+0000","last_undegraded":"2024-09-19T08:15:01.308239+0000","last_fullsized":"2024-09-19T08:15:01.308239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:58.374606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309700+0000","last_change":"2024-09-19T08:13:00.840973+0000","last_active":"2024-09-19T08:15:01.309700+0000","last_peered":"2024-09-19T08:15:01.309700+0000","last_clean":"2024-09-19T08:15:01.309700+0000","last_became_active":"2024-09-19T08:13:00.840619+0000","last_became_peered":"2024-09-19T08:13:00.840619+0000","last_unstale":"2024-09-19T08:15:01.309700+0000","last_undegraded":"2024-09-19T08:15:01.309700+0000","last_fullsized":"2024-09-19T08:15:01.309700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:02.037417+0000","last_change":"2024-09-19T08:12:58.715972+0000","last_active":"2024-09-19T08:15:02.037417+0000","last_peered":"2024-09-19T08:15:02.037417+0000","last_clean":"2024-09-19T08:15:02.037417+0000","last_became_active":"2024-09-19T08:12:58.715630+0000","last_became_peered":"2024-09-19T08:12:58.715630+0000","last_unstale":"2024-09-19T08:15:02.037417+0000","last_undegraded":"2024-09-19T08:15:02.037417+0000","last_fullsized":"2024-09-19T08:15:02.037417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:35.637121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172244+0000","last_change":"2024-09-19T08:13:00.847023+0000","last_active":"2024-09-19T08:15:01.172244+0000","last_peered":"2024-09-19T08:15:01.172244+0000","last_clean":"2024-09-19T08:15:01.172244+0000","last_became_active":"2024-09-19T08:13:00.846387+0000","last_became_peered":"2024-09-19T08:13:00.846387+0000","last_unstale":"2024-09-19T08:15:01.172244+0000","last_undegraded":"2024-09-19T08:15:01.172244+0000","last_fullsized":"2024-09-19T08:15:01.172244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:12.187929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308283+0000","last_change":"2024-09-19T08:12:58.681062+0000","last_active":"2024-09-19T08:15:01.308283+0000","last_peered":"2024-09-19T08:15:01.308283+0000","last_clean":"2024-09-19T08:15:01.308283+0000","last_became_active":"2024-09-19T08:12:58.680524+0000","last_became_peered":"2024-09-19T08:12:58.680524+0000","last_unstale":"2024-09-19T08:15:01.308283+0000","last_undegraded":"2024-09-19T08:15:01.308283+0000","last_fullsized":"2024-09-19T08:15:01.308283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271612+0000","last_change":"2024-09-19T08:13:00.851523+0000","last_active":"2024-09-19T08:15:01.271612+0000","last_peered":"2024-09-19T08:15:01.271612+0000","last_clean":"2024-09-19T08:15:01.271612+0000","last_became_active":"2024-09-19T08:13:00.851068+0000","last_became_peered":"2024-09-19T08:13:00.851068+0000","last_unstale":"2024-09-19T08:15:01.271612+0000","last_undegraded":"2024-09-19T08:15:01.271612+0000","last_fullsized":"2024-09-19T08:15:01.271612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:44.905469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.260876+0000","last_change":"2024-09-19T08:12:58.718234+0000","last_active":"2024-09-19T08:15:01.260876+0000","last_peered":"2024-09-19T08:15:01.260876+0000","last_clean":"2024-09-19T08:15:01.260876+0000","last_became_active":"2024-09-19T08:12:58.717840+0000","last_became_peered":"2024-09-19T08:12:58.717840+0000","last_unstale":"2024-09-19T08:15:01.260876+0000","last_undegraded":"2024-09-19T08:15:01.260876+0000","last_fullsized":"2024-09-19T08:15:01.260876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:05.283770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271611+0000","last_change":"2024-09-19T08:13:00.848305+0000","last_active":"2024-09-19T08:15:01.271611+0000","last_peered":"2024-09-19T08:15:01.271611+0000","last_clean":"2024-09-19T08:15:01.271611+0000","last_became_active":"2024-09-19T08:13:00.847981+0000","last_became_peered":"2024-09-19T08:13:00.847981+0000","last_unstale":"2024-09-19T08:15:01.271611+0000","last_undegraded":"2024-09-19T08:15:01.271611+0000","last_fullsized":"2024-09-19T08:15:01.271611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:04.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263326+0000","last_change":"2024-09-19T08:12:58.715943+0000","last_active":"2024-09-19T08:15:01.263326+0000","last_peered":"2024-09-19T08:15:01.263326+0000","last_clean":"2024-09-19T08:15:01.263326+0000","last_became_active":"2024-09-19T08:12:58.715600+0000","last_became_peered":"2024-09-19T08:12:58.715600+0000","last_unstale":"2024-09-19T08:15:01.263326+0000","last_undegraded":"2024-09-19T08:15:01.263326+0000","last_fullsized":"2024-09-19T08:15:01.263326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:19.142271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310614+0000","last_change":"2024-09-19T08:13:00.840944+0000","last_active":"2024-09-19T08:15:01.310614+0000","last_peered":"2024-09-19T08:15:01.310614+0000","last_clean":"2024-09-19T08:15:01.310614+0000","last_became_active":"2024-09-19T08:13:00.840608+0000","last_became_peered":"2024-09-19T08:13:00.840608+0000","last_unstale":"2024-09-19T08:15:01.310614+0000","last_undegraded":"2024-09-19T08:15:01.310614+0000","last_fullsized":"2024-09-19T08:15:01.310614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:25.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'47","reported_seq":94,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.374602+0000","last_change":"2024-09-19T08:12:58.672929+0000","last_active":"2024-09-19T08:15:01.374602+0000","last_peered":"2024-09-19T08:15:01.374602+0000","last_clean":"2024-09-19T08:15:01.374602+0000","last_became_active":"2024-09-19T08:12:58.672487+0000","last_became_peered":"2024-09-19T08:12:58.672487+0000","last_unstale":"2024-09-19T08:15:01.374602+0000","last_undegraded":"2024-09-19T08:15:01.374602+0000","last_fullsized":"2024-09-19T08:15:01.374602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:33.042735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.261959+0000","last_change":"2024-09-19T08:14:07.738260+0000","last_active":"2024-09-19T08:15:01.261959+0000","last_peered":"2024-09-19T08:15:01.261959+0000","last_clean":"2024-09-19T08:15:01.261959+0000","last_became_active":"2024-09-19T08:14:07.737915+0000","last_became_peered":"2024-09-19T08:14:07.737915+0000","last_unstale":"2024-09-19T08:15:01.261959+0000","last_undegraded":"2024-09-19T08:15:01.261959+0000","last_fullsized":"2024-09-19T08:15:01.261959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:02.500657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.173256+0000","last_change":"2024-09-19T08:12:59.179510+0000","last_active":"2024-09-19T08:15:01.173256+0000","last_peered":"2024-09-19T08:15:01.173256+0000","last_clean":"2024-09-19T08:15:01.173256+0000","last_became_active":"2024-09-19T08:12:59.179157+0000","last_became_peered":"2024-09-19T08:12:59.179157+0000","last_unstale":"2024-09-19T08:15:01.173256+0000","last_undegraded":"2024-09-19T08:15:01.173256+0000","last_fullsized":"2024-09-19T08:15:01.173256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:22.981289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308618+0000","last_change":"2024-09-19T08:13:00.841853+0000","last_active":"2024-09-19T08:15:01.308618+0000","last_peered":"2024-09-19T08:15:01.308618+0000","last_clean":"2024-09-19T08:15:01.308618+0000","last_became_active":"2024-09-19T08:13:00.841518+0000","last_became_peered":"2024-09-19T08:13:00.841518+0000","last_unstale":"2024-09-19T08:15:01.308618+0000","last_undegraded":"2024-09-19T08:15:01.308618+0000","last_fullsized":"2024-09-19T08:15:01.308618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:05.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223399+0000","last_change":"2024-09-19T08:12:58.716882+0000","last_active":"2024-09-19T08:15:01.223399+0000","last_peered":"2024-09-19T08:15:01.223399+0000","last_clean":"2024-09-19T08:15:01.223399+0000","last_became_active":"2024-09-19T08:12:58.716545+0000","last_became_peered":"2024-09-19T08:12:58.716545+0000","last_unstale":"2024-09-19T08:15:01.223399+0000","last_undegraded":"2024-09-19T08:15:01.223399+0000","last_fullsized":"2024-09-19T08:15:01.223399+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:14.469109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262936+0000","last_change":"2024-09-19T08:13:00.850484+0000","last_active":"2024-09-19T08:15:01.262936+0000","last_peered":"2024-09-19T08:15:01.262936+0000","last_clean":"2024-09-19T08:15:01.262936+0000","last_became_active":"2024-09-19T08:13:00.850049+0000","last_became_peered":"2024-09-19T08:13:00.850049+0000","last_unstale":"2024-09-19T08:15:01.262936+0000","last_undegraded":"2024-09-19T08:15:01.262936+0000","last_fullsized":"2024-09-19T08:15:01.262936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:24.424111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170936+0000","last_change":"2024-09-19T08:12:58.669928+0000","last_active":"2024-09-19T08:15:01.170936+0000","last_peered":"2024-09-19T08:15:01.170936+0000","last_clean":"2024-09-19T08:15:01.170936+0000","last_became_active":"2024-09-19T08:12:58.669614+0000","last_became_peered":"2024-09-19T08:12:58.669614+0000","last_unstale":"2024-09-19T08:15:01.170936+0000","last_undegraded":"2024-09-19T08:15:01.170936+0000","last_fullsized":"2024-09-19T08:15:01.170936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:50.924779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308456+0000","last_change":"2024-09-19T08:13:00.853520+0000","last_active":"2024-09-19T08:15:01.308456+0000","last_peered":"2024-09-19T08:15:01.308456+0000","last_clean":"2024-09-19T08:15:01.308456+0000","last_became_active":"2024-09-19T08:13:00.853184+0000","last_became_peered":"2024-09-19T08:13:00.853184+0000","last_unstale":"2024-09-19T08:15:01.308456+0000","last_undegraded":"2024-09-19T08:15:01.308456+0000","last_fullsized":"2024-09-19T08:15:01.308456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:45.160439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308612+0000","last_change":"2024-09-19T08:12:58.674359+0000","last_active":"2024-09-19T08:15:01.308612+0000","last_peered":"2024-09-19T08:15:01.308612+0000","last_clean":"2024-09-19T08:15:01.308612+0000","last_became_active":"2024-09-19T08:12:58.673890+0000","last_became_peered":"2024-09-19T08:12:58.673890+0000","last_unstale":"2024-09-19T08:15:01.308612+0000","last_undegraded":"2024-09-19T08:15:01.308612+0000","last_fullsized":"2024-09-19T08:15:01.308612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:54.939959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171020+0000","last_change":"2024-09-19T08:13:00.851071+0000","last_active":"2024-09-19T08:15:01.171020+0000","last_peered":"2024-09-19T08:15:01.171020+0000","last_clean":"2024-09-19T08:15:01.171020+0000","last_became_active":"2024-09-19T08:13:00.850756+0000","last_became_peered":"2024-09-19T08:13:00.850756+0000","last_unstale":"2024-09-19T08:15:01.171020+0000","last_undegraded":"2024-09-19T08:15:01.171020+0000","last_fullsized":"2024-09-19T08:15:01.171020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:52.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263021+0000","last_change":"2024-09-19T08:12:58.681466+0000","last_active":"2024-09-19T08:15:01.263021+0000","last_peered":"2024-09-19T08:15:01.263021+0000","last_clean":"2024-09-19T08:15:01.263021+0000","last_became_active":"2024-09-19T08:12:58.681113+0000","last_became_peered":"2024-09-19T08:12:58.681113+0000","last_unstale":"2024-09-19T08:15:01.263021+0000","last_undegraded":"2024-09-19T08:15:01.263021+0000","last_fullsized":"2024-09-19T08:15:01.263021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:49.610828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308386+0000","last_change":"2024-09-19T08:13:00.851272+0000","last_active":"2024-09-19T08:15:01.308386+0000","last_peered":"2024-09-19T08:15:01.308386+0000","last_clean":"2024-09-19T08:15:01.308386+0000","last_became_active":"2024-09-19T08:13:00.849098+0000","last_became_peered":"2024-09-19T08:13:00.849098+0000","last_unstale":"2024-09-19T08:15:01.308386+0000","last_undegraded":"2024-09-19T08:15:01.308386+0000","last_fullsized":"2024-09-19T08:15:01.308386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:05.643992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172513+0000","last_change":"2024-09-19T08:14:07.738913+0000","last_active":"2024-09-19T08:15:01.172513+0000","last_peered":"2024-09-19T08:15:01.172513+0000","last_clean":"2024-09-19T08:15:01.172513+0000","last_became_active":"2024-09-19T08:14:07.738560+0000","last_became_peered":"2024-09-19T08:14:07.738560+0000","last_unstale":"2024-09-19T08:15:01.172513+0000","last_undegraded":"2024-09-19T08:15:01.172513+0000","last_fullsized":"2024-09-19T08:15:01.172513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:35.731290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263192+0000","last_change":"2024-09-19T08:12:58.674610+0000","last_active":"2024-09-19T08:15:01.263192+0000","last_peered":"2024-09-19T08:15:01.263192+0000","last_clean":"2024-09-19T08:15:01.263192+0000","last_became_active":"2024-09-19T08:12:58.674247+0000","last_became_peered":"2024-09-19T08:12:58.674247+0000","last_unstale":"2024-09-19T08:15:01.263192+0000","last_undegraded":"2024-09-19T08:15:01.263192+0000","last_fullsized":"2024-09-19T08:15:01.263192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:16.323574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223395+0000","last_change":"2024-09-19T08:13:00.858512+0000","last_active":"2024-09-19T08:15:01.223395+0000","last_peered":"2024-09-19T08:15:01.223395+0000","last_clean":"2024-09-19T08:15:01.223395+0000","last_became_active":"2024-09-19T08:13:00.857897+0000","last_became_peered":"2024-09-19T08:13:00.857897+0000","last_unstale":"2024-09-19T08:15:01.223395+0000","last_undegraded":"2024-09-19T08:15:01.223395+0000","last_fullsized":"2024-09-19T08:15:01.223395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263238+0000","last_change":"2024-09-19T08:12:58.682207+0000","last_active":"2024-09-19T08:15:01.263238+0000","last_peered":"2024-09-19T08:15:01.263238+0000","last_clean":"2024-09-19T08:15:01.263238+0000","last_became_active":"2024-09-19T08:12:58.681842+0000","last_became_peered":"2024-09-19T08:12:58.681842+0000","last_unstale":"2024-09-19T08:15:01.263238+0000","last_undegraded":"2024-09-19T08:15:01.263238+0000","last_fullsized":"2024-09-19T08:15:01.263238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.825858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308840+0000","last_change":"2024-09-19T08:13:00.846748+0000","last_active":"2024-09-19T08:15:01.308840+0000","last_peered":"2024-09-19T08:15:01.308840+0000","last_clean":"2024-09-19T08:15:01.308840+0000","last_became_active":"2024-09-19T08:13:00.846368+0000","last_became_peered":"2024-09-19T08:13:00.846368+0000","last_unstale":"2024-09-19T08:15:01.308840+0000","last_undegraded":"2024-09-19T08:15:01.308840+0000","last_fullsized":"2024-09-19T08:15:01.308840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:59.526512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171334+0000","last_change":"2024-09-19T08:12:58.716118+0000","last_active":"2024-09-19T08:15:01.171334+0000","last_peered":"2024-09-19T08:15:01.171334+0000","last_clean":"2024-09-19T08:15:01.171334+0000","last_became_active":"2024-09-19T08:12:58.715711+0000","last_became_peered":"2024-09-19T08:12:58.715711+0000","last_unstale":"2024-09-19T08:15:01.171334+0000","last_undegraded":"2024-09-19T08:15:01.171334+0000","last_fullsized":"2024-09-19T08:15:01.171334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:29.439887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171172+0000","last_change":"2024-09-19T08:13:00.848729+0000","last_active":"2024-09-19T08:15:01.171172+0000","last_peered":"2024-09-19T08:15:01.171172+0000","last_clean":"2024-09-19T08:15:01.171172+0000","last_became_active":"2024-09-19T08:13:00.848470+0000","last_became_peered":"2024-09-19T08:13:00.848470+0000","last_unstale":"2024-09-19T08:15:01.171172+0000","last_undegraded":"2024-09-19T08:15:01.171172+0000","last_fullsized":"2024-09-19T08:15:01.171172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:30.653101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262232+0000","last_change":"2024-09-19T08:12:58.681654+0000","last_active":"2024-09-19T08:15:01.262232+0000","last_peered":"2024-09-19T08:15:01.262232+0000","last_clean":"2024-09-19T08:15:01.262232+0000","last_became_active":"2024-09-19T08:12:58.681284+0000","last_became_peered":"2024-09-19T08:12:58.681284+0000","last_unstale":"2024-09-19T08:15:01.262232+0000","last_undegraded":"2024-09-19T08:15:01.262232+0000","last_fullsized":"2024-09-19T08:15:01.262232+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.429930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597810,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":784,"num_read_kb":501,"num_write":297,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":61306,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":1793430,"data_compressed":85517,"data_compressed_allocated":110592,"data_compressed_original":1605632,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":109176,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481828,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":454888,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710762,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":1246292,"data_compressed":51265,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":848206,"data_compressed":27863,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103220,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":958464,"data_stored":847992,"data_compressed":27863,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":1049004,"data_compressed":45109,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":933888,"data_stored":454379,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":1178660,"data_compressed":51265,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":517387,"data_compressed":18328,"data_compressed_allocated":24576,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":979506,"data_compressed":34019,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":950272,"data_stored":456449,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":515901,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":455650,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":69101,"data_compressed":9611,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134003,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":393820,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201635,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:15:06.195 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-19T08:15:06.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:15:06.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.330+0000 7f5ce372c640 1 -- 172.21.15.22:0/2907025108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cdc101ab0 msgr2=0x7f5cdc101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:06.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.330+0000 7f5ce372c640 1 --2- 172.21.15.22:0/2907025108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cdc101ab0 0x7f5cdc101f10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009930 tx=0x7f5ccc02a420 comp rx=0 tx=0).stop 2024-09-19T08:15:06.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.330+0000 7f5ce372c640 1 -- 172.21.15.22:0/2907025108 shutdown_connections 2024-09-19T08:15:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.330+0000 7f5ce372c640 1 --2- 172.21.15.22:0/2907025108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc102450 0x7f5cdc10e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.330+0000 7f5ce372c640 1 --2- 172.21.15.22:0/2907025108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cdc101ab0 0x7f5cdc101f10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.330+0000 7f5ce372c640 1 --2- 172.21.15.22:0/2907025108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cdc107ac0 0x7f5cdc107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.330+0000 7f5ce372c640 1 -- 172.21.15.22:0/2907025108 >> 172.21.15.22:0/2907025108 conn(0x7f5cdc0fd7d0 msgr2=0x7f5cdc0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.330+0000 7f5ce372c640 1 -- 172.21.15.22:0/2907025108 shutdown_connections 2024-09-19T08:15:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.330+0000 7f5ce372c640 1 -- 172.21.15.22:0/2907025108 wait complete. 2024-09-19T08:15:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.331+0000 7f5ce372c640 1 Processor -- start 2024-09-19T08:15:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.331+0000 7f5ce372c640 1 -- start start 2024-09-19T08:15:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.331+0000 7f5ce372c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cdc101ab0 0x7f5cdc198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.331+0000 7f5ce372c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc102450 0x7f5cdc1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.331+0000 7f5ce372c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cdc107ac0 0x7f5cdc19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.331+0000 7f5ce372c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc111f00 con 0x7f5cdc107ac0 2024-09-19T08:15:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.331+0000 7f5ce372c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc111d80 con 0x7f5cdc101ab0 2024-09-19T08:15:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.331+0000 7f5ce372c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc112080 con 0x7f5cdc102450 2024-09-19T08:15:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.331+0000 7f5ce1cc9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cdc107ac0 0x7f5cdc19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.331+0000 7f5ce1cc9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cdc107ac0 0x7f5cdc19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42498/0 (socket says 172.21.15.22:42498) 2024-09-19T08:15:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.331+0000 7f5ce1cc9640 1 -- 172.21.15.22:0/1821454022 learned_addr learned my addr 172.21.15.22:0/1821454022 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.331+0000 7f5ce14c8640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cdc101ab0 0x7f5cdc198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.331+0000 7f5ce0cc7640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc102450 0x7f5cdc1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.332+0000 7f5ce1cc9640 1 -- 172.21.15.22:0/1821454022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc102450 msgr2=0x7f5cdc1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.332+0000 7f5ce1cc9640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc102450 0x7f5cdc1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.332+0000 7f5ce1cc9640 1 -- 172.21.15.22:0/1821454022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cdc101ab0 msgr2=0x7f5cdc198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.332+0000 7f5ce1cc9640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cdc101ab0 0x7f5cdc198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.332+0000 7f5ce1cc9640 1 -- 172.21.15.22:0/1821454022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cdc19d380 con 0x7f5cdc107ac0 2024-09-19T08:15:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.332+0000 7f5ce14c8640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cdc101ab0 0x7f5cdc198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.332+0000 7f5ce1cc9640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cdc107ac0 0x7f5cdc19cc50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400eaa0 tx=0x7f5cd400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.332+0000 7f5cd27fc640 1 -- 172.21.15.22:0/1821454022 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd400cdb0 con 0x7f5cdc107ac0 2024-09-19T08:15:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.332+0000 7f5ce372c640 1 -- 172.21.15.22:0/1821454022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cdc19d670 con 0x7f5cdc107ac0 2024-09-19T08:15:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.332+0000 7f5cd27fc640 1 -- 172.21.15.22:0/1821454022 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd4005020 con 0x7f5cdc107ac0 2024-09-19T08:15:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.332+0000 7f5cd27fc640 1 -- 172.21.15.22:0/1821454022 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd40116d0 con 0x7f5cdc107ac0 2024-09-19T08:15:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.332+0000 7f5ce372c640 1 -- 172.21.15.22:0/1821454022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cdc1a4f00 con 0x7f5cdc107ac0 2024-09-19T08:15:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.333+0000 7f5ce372c640 1 -- 172.21.15.22:0/1821454022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cdc101f10 con 0x7f5cdc107ac0 2024-09-19T08:15:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.334+0000 7f5cd27fc640 1 -- 172.21.15.22:0/1821454022 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f5cd40054d0 con 0x7f5cdc107ac0 2024-09-19T08:15:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.335+0000 7f5cd27fc640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5cc80a34e0 0x7f5cc80a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.335+0000 7f5cd27fc640 1 -- 172.21.15.22:0/1821454022 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5cd40d0660 con 0x7f5cdc107ac0 2024-09-19T08:15:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.335+0000 7f5ce14c8640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5cc80a34e0 0x7f5cc80a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.335+0000 7f5ce14c8640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5cc80a34e0 0x7f5cc80a59a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8005e10 tx=0x7f5cd8005da0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:06.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.338+0000 7f5cd27fc640 1 -- 172.21.15.22:0/1821454022 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5cd4016030 con 0x7f5cdc107ac0 2024-09-19T08:15:06.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:06 smithi157 ceph-mon[35143]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:15:06.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:06 smithi157 ceph-mon[35143]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T08:15:06.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:06 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:06.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:06 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:06.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:06 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:06.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:06 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:06.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:06 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:06.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:06 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:06.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:06 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:06.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:06 smithi157 ceph-mon[35143]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:06.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:06 smithi022 ceph-mon[30963]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:15:06.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:06 smithi022 ceph-mon[30963]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T08:15:06.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:06 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:06.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:06 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:06.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:06 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:06.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:06 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:06.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:06 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:06.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:06 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:06.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:06 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:06.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:06 smithi022 ceph-mon[30963]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:06.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.512+0000 7f5ce372c640 1 -- 172.21.15.22:0/1821454022 --> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5cdc0fe650 con 0x7f5cc80a34e0 2024-09-19T08:15:06.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.519+0000 7f5cd27fc640 1 -- 172.21.15.22:0/1821454022 <== mgr.15141 v2:172.21.15.22:6800/497251935 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258026 (secure 0 0 0) 0x7f5cdc0fe650 con 0x7f5cc80a34e0 2024-09-19T08:15:06.522 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:06.523 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T08:15:06.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.522+0000 7f5ce372c640 1 -- 172.21.15.22:0/1821454022 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5cc80a34e0 msgr2=0x7f5cc80a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:06.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.522+0000 7f5ce372c640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5cc80a34e0 0x7f5cc80a59a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8005e10 tx=0x7f5cd8005da0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.522+0000 7f5ce372c640 1 -- 172.21.15.22:0/1821454022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cdc107ac0 msgr2=0x7f5cdc19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:06.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.522+0000 7f5ce372c640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cdc107ac0 0x7f5cdc19cc50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400eaa0 tx=0x7f5cd400ef60 comp rx=0 tx=0).stop 2024-09-19T08:15:06.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.522+0000 7f5ce372c640 1 -- 172.21.15.22:0/1821454022 shutdown_connections 2024-09-19T08:15:06.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.522+0000 7f5ce372c640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5cc80a34e0 0x7f5cc80a59a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.522+0000 7f5ce372c640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cdc107ac0 0x7f5cdc19cc50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.522+0000 7f5ce372c640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc102450 0x7f5cdc1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.522+0000 7f5ce372c640 1 --2- 172.21.15.22:0/1821454022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cdc101ab0 0x7f5cdc198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:06.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.522+0000 7f5ce372c640 1 -- 172.21.15.22:0/1821454022 >> 172.21.15.22:0/1821454022 conn(0x7f5cdc0fd7d0 msgr2=0x7f5cdc0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:06.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.522+0000 7f5ce372c640 1 -- 172.21.15.22:0/1821454022 shutdown_connections 2024-09-19T08:15:06.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:06.522+0000 7f5ce372c640 1 -- 172.21.15.22:0/1821454022 wait complete. 2024-09-19T08:15:06.534 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T08:15:05.185903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188412,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":920,"num_read_kb":620,"num_write":480,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1242042,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381604,"kb_used_data":11520,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560220,"statfs":{"total":1151940427776,"available":1151549665280,"internally_reserved":0,"allocated":11796480,"data_stored":9004314,"data_compressed":299297,"data_compressed_allocated":393216,"data_compressed_original":3670016,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.993097"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309110+0000","last_change":"2024-09-19T08:13:00.853947+0000","last_active":"2024-09-19T08:15:01.309110+0000","last_peered":"2024-09-19T08:15:01.309110+0000","last_clean":"2024-09-19T08:15:01.309110+0000","last_became_active":"2024-09-19T08:13:00.852951+0000","last_became_peered":"2024-09-19T08:13:00.852951+0000","last_unstale":"2024-09-19T08:15:01.309110+0000","last_undegraded":"2024-09-19T08:15:01.309110+0000","last_fullsized":"2024-09-19T08:15:01.309110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:25.786828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308563+0000","last_change":"2024-09-19T08:13:00.853389+0000","last_active":"2024-09-19T08:15:01.308563+0000","last_peered":"2024-09-19T08:15:01.308563+0000","last_clean":"2024-09-19T08:15:01.308563+0000","last_became_active":"2024-09-19T08:13:00.853102+0000","last_became_peered":"2024-09-19T08:13:00.853102+0000","last_unstale":"2024-09-19T08:15:01.308563+0000","last_undegraded":"2024-09-19T08:15:01.308563+0000","last_fullsized":"2024-09-19T08:15:01.308563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:42.773917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308818+0000","last_change":"2024-09-19T08:13:00.846544+0000","last_active":"2024-09-19T08:15:01.308818+0000","last_peered":"2024-09-19T08:15:01.308818+0000","last_clean":"2024-09-19T08:15:01.308818+0000","last_became_active":"2024-09-19T08:13:00.846229+0000","last_became_peered":"2024-09-19T08:13:00.846229+0000","last_unstale":"2024-09-19T08:15:01.308818+0000","last_undegraded":"2024-09-19T08:15:01.308818+0000","last_fullsized":"2024-09-19T08:15:01.308818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:43.471684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172883+0000","last_change":"2024-09-19T08:13:00.848149+0000","last_active":"2024-09-19T08:15:01.172883+0000","last_peered":"2024-09-19T08:15:01.172883+0000","last_clean":"2024-09-19T08:15:01.172883+0000","last_became_active":"2024-09-19T08:13:00.847744+0000","last_became_peered":"2024-09-19T08:13:00.847744+0000","last_unstale":"2024-09-19T08:15:01.172883+0000","last_undegraded":"2024-09-19T08:15:01.172883+0000","last_fullsized":"2024-09-19T08:15:01.172883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:01.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263000+0000","last_change":"2024-09-19T08:13:00.841862+0000","last_active":"2024-09-19T08:15:01.263000+0000","last_peered":"2024-09-19T08:15:01.263000+0000","last_clean":"2024-09-19T08:15:01.263000+0000","last_became_active":"2024-09-19T08:13:00.841331+0000","last_became_peered":"2024-09-19T08:13:00.841331+0000","last_unstale":"2024-09-19T08:15:01.263000+0000","last_undegraded":"2024-09-19T08:15:01.263000+0000","last_fullsized":"2024-09-19T08:15:01.263000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:54.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271477+0000","last_change":"2024-09-19T08:13:00.849626+0000","last_active":"2024-09-19T08:15:01.271477+0000","last_peered":"2024-09-19T08:15:01.271477+0000","last_clean":"2024-09-19T08:15:01.271477+0000","last_became_active":"2024-09-19T08:13:00.849034+0000","last_became_peered":"2024-09-19T08:13:00.849034+0000","last_unstale":"2024-09-19T08:15:01.271477+0000","last_undegraded":"2024-09-19T08:15:01.271477+0000","last_fullsized":"2024-09-19T08:15:01.271477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:39.118978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308024+0000","last_change":"2024-09-19T08:13:00.847522+0000","last_active":"2024-09-19T08:15:01.308024+0000","last_peered":"2024-09-19T08:15:01.308024+0000","last_clean":"2024-09-19T08:15:01.308024+0000","last_became_active":"2024-09-19T08:13:00.847117+0000","last_became_peered":"2024-09-19T08:13:00.847117+0000","last_unstale":"2024-09-19T08:15:01.308024+0000","last_undegraded":"2024-09-19T08:15:01.308024+0000","last_fullsized":"2024-09-19T08:15:01.308024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.836385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170733+0000","last_change":"2024-09-19T08:13:00.849544+0000","last_active":"2024-09-19T08:15:01.170733+0000","last_peered":"2024-09-19T08:15:01.170733+0000","last_clean":"2024-09-19T08:15:01.170733+0000","last_became_active":"2024-09-19T08:13:00.849170+0000","last_became_peered":"2024-09-19T08:13:00.849170+0000","last_unstale":"2024-09-19T08:15:01.170733+0000","last_undegraded":"2024-09-19T08:15:01.170733+0000","last_fullsized":"2024-09-19T08:15:01.170733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:28.371965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223727+0000","last_change":"2024-09-19T08:13:00.856052+0000","last_active":"2024-09-19T08:15:01.223727+0000","last_peered":"2024-09-19T08:15:01.223727+0000","last_clean":"2024-09-19T08:15:01.223727+0000","last_became_active":"2024-09-19T08:13:00.855291+0000","last_became_peered":"2024-09-19T08:13:00.855291+0000","last_unstale":"2024-09-19T08:15:01.223727+0000","last_undegraded":"2024-09-19T08:15:01.223727+0000","last_fullsized":"2024-09-19T08:15:01.223727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:18.581867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271887+0000","last_change":"2024-09-19T08:13:00.850145+0000","last_active":"2024-09-19T08:15:01.271887+0000","last_peered":"2024-09-19T08:15:01.271887+0000","last_clean":"2024-09-19T08:15:01.271887+0000","last_became_active":"2024-09-19T08:13:00.849798+0000","last_became_peered":"2024-09-19T08:13:00.849798+0000","last_unstale":"2024-09-19T08:15:01.271887+0000","last_undegraded":"2024-09-19T08:15:01.271887+0000","last_fullsized":"2024-09-19T08:15:01.271887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:21.400000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308275+0000","last_change":"2024-09-19T08:13:00.859153+0000","last_active":"2024-09-19T08:15:01.308275+0000","last_peered":"2024-09-19T08:15:01.308275+0000","last_clean":"2024-09-19T08:15:01.308275+0000","last_became_active":"2024-09-19T08:13:00.858473+0000","last_became_peered":"2024-09-19T08:13:00.858473+0000","last_unstale":"2024-09-19T08:15:01.308275+0000","last_undegraded":"2024-09-19T08:15:01.308275+0000","last_fullsized":"2024-09-19T08:15:01.308275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:07.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172922+0000","last_change":"2024-09-19T08:13:00.850797+0000","last_active":"2024-09-19T08:15:01.172922+0000","last_peered":"2024-09-19T08:15:01.172922+0000","last_clean":"2024-09-19T08:15:01.172922+0000","last_became_active":"2024-09-19T08:13:00.850393+0000","last_became_peered":"2024-09-19T08:13:00.850393+0000","last_unstale":"2024-09-19T08:15:01.172922+0000","last_undegraded":"2024-09-19T08:15:01.172922+0000","last_fullsized":"2024-09-19T08:15:01.172922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:59.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310082+0000","last_change":"2024-09-19T08:13:00.850835+0000","last_active":"2024-09-19T08:15:01.310082+0000","last_peered":"2024-09-19T08:15:01.310082+0000","last_clean":"2024-09-19T08:15:01.310082+0000","last_became_active":"2024-09-19T08:13:00.850518+0000","last_became_peered":"2024-09-19T08:13:00.850518+0000","last_unstale":"2024-09-19T08:15:01.310082+0000","last_undegraded":"2024-09-19T08:15:01.310082+0000","last_fullsized":"2024-09-19T08:15:01.310082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.178530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308450+0000","last_change":"2024-09-19T08:13:00.853934+0000","last_active":"2024-09-19T08:15:01.308450+0000","last_peered":"2024-09-19T08:15:01.308450+0000","last_clean":"2024-09-19T08:15:01.308450+0000","last_became_active":"2024-09-19T08:13:00.853604+0000","last_became_peered":"2024-09-19T08:13:00.853604+0000","last_unstale":"2024-09-19T08:15:01.308450+0000","last_undegraded":"2024-09-19T08:15:01.308450+0000","last_fullsized":"2024-09-19T08:15:01.308450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:43.622526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308262+0000","last_change":"2024-09-19T08:14:07.741370+0000","last_active":"2024-09-19T08:15:01.308262+0000","last_peered":"2024-09-19T08:15:01.308262+0000","last_clean":"2024-09-19T08:15:01.308262+0000","last_became_active":"2024-09-19T08:14:07.741116+0000","last_became_peered":"2024-09-19T08:14:07.741116+0000","last_unstale":"2024-09-19T08:15:01.308262+0000","last_undegraded":"2024-09-19T08:15:01.308262+0000","last_fullsized":"2024-09-19T08:15:01.308262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:45.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223320+0000","last_change":"2024-09-19T08:13:00.850266+0000","last_active":"2024-09-19T08:15:01.223320+0000","last_peered":"2024-09-19T08:15:01.223320+0000","last_clean":"2024-09-19T08:15:01.223320+0000","last_became_active":"2024-09-19T08:13:00.849928+0000","last_became_peered":"2024-09-19T08:13:00.849928+0000","last_unstale":"2024-09-19T08:15:01.223320+0000","last_undegraded":"2024-09-19T08:15:01.223320+0000","last_fullsized":"2024-09-19T08:15:01.223320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:38.231331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309177+0000","last_change":"2024-09-19T08:13:00.852551+0000","last_active":"2024-09-19T08:15:01.309177+0000","last_peered":"2024-09-19T08:15:01.309177+0000","last_clean":"2024-09-19T08:15:01.309177+0000","last_became_active":"2024-09-19T08:13:00.852225+0000","last_became_peered":"2024-09-19T08:13:00.852225+0000","last_unstale":"2024-09-19T08:15:01.309177+0000","last_undegraded":"2024-09-19T08:15:01.309177+0000","last_fullsized":"2024-09-19T08:15:01.309177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:33.115108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.261006+0000","last_change":"2024-09-19T08:13:00.854033+0000","last_active":"2024-09-19T08:15:01.261006+0000","last_peered":"2024-09-19T08:15:01.261006+0000","last_clean":"2024-09-19T08:15:01.261006+0000","last_became_active":"2024-09-19T08:13:00.853660+0000","last_became_peered":"2024-09-19T08:13:00.853660+0000","last_unstale":"2024-09-19T08:15:01.261006+0000","last_undegraded":"2024-09-19T08:15:01.261006+0000","last_fullsized":"2024-09-19T08:15:01.261006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:35.602420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308765+0000","last_change":"2024-09-19T08:13:00.850658+0000","last_active":"2024-09-19T08:15:01.308765+0000","last_peered":"2024-09-19T08:15:01.308765+0000","last_clean":"2024-09-19T08:15:01.308765+0000","last_became_active":"2024-09-19T08:13:00.850324+0000","last_became_peered":"2024-09-19T08:13:00.850324+0000","last_unstale":"2024-09-19T08:15:01.308765+0000","last_undegraded":"2024-09-19T08:15:01.308765+0000","last_fullsized":"2024-09-19T08:15:01.308765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:59:49.004605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308718+0000","last_change":"2024-09-19T08:14:07.741837+0000","last_active":"2024-09-19T08:15:01.308718+0000","last_peered":"2024-09-19T08:15:01.308718+0000","last_clean":"2024-09-19T08:15:01.308718+0000","last_became_active":"2024-09-19T08:14:07.741398+0000","last_became_peered":"2024-09-19T08:14:07.741398+0000","last_unstale":"2024-09-19T08:15:01.308718+0000","last_undegraded":"2024-09-19T08:15:01.308718+0000","last_fullsized":"2024-09-19T08:15:01.308718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:56.716663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310236+0000","last_change":"2024-09-19T08:13:00.855215+0000","last_active":"2024-09-19T08:15:01.310236+0000","last_peered":"2024-09-19T08:15:01.310236+0000","last_clean":"2024-09-19T08:15:01.310236+0000","last_became_active":"2024-09-19T08:13:00.854878+0000","last_became_peered":"2024-09-19T08:13:00.854878+0000","last_unstale":"2024-09-19T08:15:01.310236+0000","last_undegraded":"2024-09-19T08:15:01.310236+0000","last_fullsized":"2024-09-19T08:15:01.310236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:48.216066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223444+0000","last_change":"2024-09-19T08:13:00.856995+0000","last_active":"2024-09-19T08:15:01.223444+0000","last_peered":"2024-09-19T08:15:01.223444+0000","last_clean":"2024-09-19T08:15:01.223444+0000","last_became_active":"2024-09-19T08:13:00.856645+0000","last_became_peered":"2024-09-19T08:13:00.856645+0000","last_unstale":"2024-09-19T08:15:01.223444+0000","last_undegraded":"2024-09-19T08:15:01.223444+0000","last_fullsized":"2024-09-19T08:15:01.223444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:20.699293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170753+0000","last_change":"2024-09-19T08:13:00.853362+0000","last_active":"2024-09-19T08:15:01.170753+0000","last_peered":"2024-09-19T08:15:01.170753+0000","last_clean":"2024-09-19T08:15:01.170753+0000","last_became_active":"2024-09-19T08:13:00.853046+0000","last_became_peered":"2024-09-19T08:13:00.853046+0000","last_unstale":"2024-09-19T08:15:01.170753+0000","last_undegraded":"2024-09-19T08:15:01.170753+0000","last_fullsized":"2024-09-19T08:15:01.170753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172184+0000","last_change":"2024-09-19T08:13:00.845175+0000","last_active":"2024-09-19T08:15:01.172184+0000","last_peered":"2024-09-19T08:15:01.172184+0000","last_clean":"2024-09-19T08:15:01.172184+0000","last_became_active":"2024-09-19T08:13:00.844857+0000","last_became_peered":"2024-09-19T08:13:00.844857+0000","last_unstale":"2024-09-19T08:15:01.172184+0000","last_undegraded":"2024-09-19T08:15:01.172184+0000","last_fullsized":"2024-09-19T08:15:01.172184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:40.818956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171025+0000","last_change":"2024-09-19T08:13:00.847769+0000","last_active":"2024-09-19T08:15:01.171025+0000","last_peered":"2024-09-19T08:15:01.171025+0000","last_clean":"2024-09-19T08:15:01.171025+0000","last_became_active":"2024-09-19T08:13:00.847454+0000","last_became_peered":"2024-09-19T08:13:00.847454+0000","last_unstale":"2024-09-19T08:15:01.171025+0000","last_undegraded":"2024-09-19T08:15:01.171025+0000","last_fullsized":"2024-09-19T08:15:01.171025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.583051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308835+0000","last_change":"2024-09-19T08:13:00.859882+0000","last_active":"2024-09-19T08:15:01.308835+0000","last_peered":"2024-09-19T08:15:01.308835+0000","last_clean":"2024-09-19T08:15:01.308835+0000","last_became_active":"2024-09-19T08:13:00.859491+0000","last_became_peered":"2024-09-19T08:13:00.859491+0000","last_unstale":"2024-09-19T08:15:01.308835+0000","last_undegraded":"2024-09-19T08:15:01.308835+0000","last_fullsized":"2024-09-19T08:15:01.308835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:13.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308625+0000","last_change":"2024-09-19T08:13:00.845584+0000","last_active":"2024-09-19T08:15:01.308625+0000","last_peered":"2024-09-19T08:15:01.308625+0000","last_clean":"2024-09-19T08:15:01.308625+0000","last_became_active":"2024-09-19T08:13:00.845166+0000","last_became_peered":"2024-09-19T08:13:00.845166+0000","last_unstale":"2024-09-19T08:15:01.308625+0000","last_undegraded":"2024-09-19T08:15:01.308625+0000","last_fullsized":"2024-09-19T08:15:01.308625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:02.705793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310490+0000","last_change":"2024-09-19T08:13:00.838069+0000","last_active":"2024-09-19T08:15:01.310490+0000","last_peered":"2024-09-19T08:15:01.310490+0000","last_clean":"2024-09-19T08:15:01.310490+0000","last_became_active":"2024-09-19T08:13:00.837736+0000","last_became_peered":"2024-09-19T08:13:00.837736+0000","last_unstale":"2024-09-19T08:15:01.310490+0000","last_undegraded":"2024-09-19T08:15:01.310490+0000","last_fullsized":"2024-09-19T08:15:01.310490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:01.388290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308563+0000","last_change":"2024-09-19T08:13:00.841507+0000","last_active":"2024-09-19T08:15:01.308563+0000","last_peered":"2024-09-19T08:15:01.308563+0000","last_clean":"2024-09-19T08:15:01.308563+0000","last_became_active":"2024-09-19T08:13:00.841161+0000","last_became_peered":"2024-09-19T08:13:00.841161+0000","last_unstale":"2024-09-19T08:15:01.308563+0000","last_undegraded":"2024-09-19T08:15:01.308563+0000","last_fullsized":"2024-09-19T08:15:01.308563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:41.862290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308471+0000","last_change":"2024-09-19T08:13:00.847657+0000","last_active":"2024-09-19T08:15:01.308471+0000","last_peered":"2024-09-19T08:15:01.308471+0000","last_clean":"2024-09-19T08:15:01.308471+0000","last_became_active":"2024-09-19T08:13:00.846849+0000","last_became_peered":"2024-09-19T08:13:00.846849+0000","last_unstale":"2024-09-19T08:15:01.308471+0000","last_undegraded":"2024-09-19T08:15:01.308471+0000","last_fullsized":"2024-09-19T08:15:01.308471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:35.219913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309005+0000","last_change":"2024-09-19T08:13:00.845971+0000","last_active":"2024-09-19T08:15:01.309005+0000","last_peered":"2024-09-19T08:15:01.309005+0000","last_clean":"2024-09-19T08:15:01.309005+0000","last_became_active":"2024-09-19T08:13:00.845626+0000","last_became_peered":"2024-09-19T08:13:00.845626+0000","last_unstale":"2024-09-19T08:15:01.309005+0000","last_undegraded":"2024-09-19T08:15:01.309005+0000","last_fullsized":"2024-09-19T08:15:01.309005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:36.945627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308386+0000","last_change":"2024-09-19T08:14:07.738450+0000","last_active":"2024-09-19T08:15:01.308386+0000","last_peered":"2024-09-19T08:15:01.308386+0000","last_clean":"2024-09-19T08:15:01.308386+0000","last_became_active":"2024-09-19T08:14:07.738028+0000","last_became_peered":"2024-09-19T08:14:07.738028+0000","last_unstale":"2024-09-19T08:15:01.308386+0000","last_undegraded":"2024-09-19T08:15:01.308386+0000","last_fullsized":"2024-09-19T08:15:01.308386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:08.659158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262954+0000","last_change":"2024-09-19T08:13:00.848325+0000","last_active":"2024-09-19T08:15:01.262954+0000","last_peered":"2024-09-19T08:15:01.262954+0000","last_clean":"2024-09-19T08:15:01.262954+0000","last_became_active":"2024-09-19T08:13:00.848015+0000","last_became_peered":"2024-09-19T08:13:00.848015+0000","last_unstale":"2024-09-19T08:15:01.262954+0000","last_undegraded":"2024-09-19T08:15:01.262954+0000","last_fullsized":"2024-09-19T08:15:01.262954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:10.948451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:02.038549+0000","last_change":"2024-09-19T08:12:58.681134+0000","last_active":"2024-09-19T08:15:02.038549+0000","last_peered":"2024-09-19T08:15:02.038549+0000","last_clean":"2024-09-19T08:15:02.038549+0000","last_became_active":"2024-09-19T08:12:58.680764+0000","last_became_peered":"2024-09-19T08:12:58.680764+0000","last_unstale":"2024-09-19T08:15:02.038549+0000","last_undegraded":"2024-09-19T08:15:02.038549+0000","last_fullsized":"2024-09-19T08:15:02.038549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:24.939263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172533+0000","last_change":"2024-09-19T08:13:00.844319+0000","last_active":"2024-09-19T08:15:01.172533+0000","last_peered":"2024-09-19T08:15:01.172533+0000","last_clean":"2024-09-19T08:15:01.172533+0000","last_became_active":"2024-09-19T08:13:00.843994+0000","last_became_peered":"2024-09-19T08:13:00.843994+0000","last_unstale":"2024-09-19T08:15:01.172533+0000","last_undegraded":"2024-09-19T08:15:01.172533+0000","last_fullsized":"2024-09-19T08:15:01.172533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:46.937466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271532+0000","last_change":"2024-09-19T08:12:58.676541+0000","last_active":"2024-09-19T08:15:01.271532+0000","last_peered":"2024-09-19T08:15:01.271532+0000","last_clean":"2024-09-19T08:15:01.271532+0000","last_became_active":"2024-09-19T08:12:58.675988+0000","last_became_peered":"2024-09-19T08:12:58.675988+0000","last_unstale":"2024-09-19T08:15:01.271532+0000","last_undegraded":"2024-09-19T08:15:01.271532+0000","last_fullsized":"2024-09-19T08:15:01.271532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:01.434986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309788+0000","last_change":"2024-09-19T08:12:58.677510+0000","last_active":"2024-09-19T08:15:01.309788+0000","last_peered":"2024-09-19T08:15:01.309788+0000","last_clean":"2024-09-19T08:15:01.309788+0000","last_became_active":"2024-09-19T08:12:58.677200+0000","last_became_peered":"2024-09-19T08:12:58.677200+0000","last_unstale":"2024-09-19T08:15:01.309788+0000","last_undegraded":"2024-09-19T08:15:01.309788+0000","last_fullsized":"2024-09-19T08:15:01.309788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:36.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170825+0000","last_change":"2024-09-19T08:13:00.853430+0000","last_active":"2024-09-19T08:15:01.170825+0000","last_peered":"2024-09-19T08:15:01.170825+0000","last_clean":"2024-09-19T08:15:01.170825+0000","last_became_active":"2024-09-19T08:13:00.853096+0000","last_became_peered":"2024-09-19T08:13:00.853096+0000","last_unstale":"2024-09-19T08:15:01.170825+0000","last_undegraded":"2024-09-19T08:15:01.170825+0000","last_fullsized":"2024-09-19T08:15:01.170825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:07.147021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172384+0000","last_change":"2024-09-19T08:12:58.680081+0000","last_active":"2024-09-19T08:15:01.172384+0000","last_peered":"2024-09-19T08:15:01.172384+0000","last_clean":"2024-09-19T08:15:01.172384+0000","last_became_active":"2024-09-19T08:12:58.679704+0000","last_became_peered":"2024-09-19T08:12:58.679704+0000","last_unstale":"2024-09-19T08:15:01.172384+0000","last_undegraded":"2024-09-19T08:15:01.172384+0000","last_fullsized":"2024-09-19T08:15:01.172384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:32.942586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.260317+0000","last_change":"2024-09-19T08:13:00.849394+0000","last_active":"2024-09-19T08:15:01.260317+0000","last_peered":"2024-09-19T08:15:01.260317+0000","last_clean":"2024-09-19T08:15:01.260317+0000","last_became_active":"2024-09-19T08:13:00.849051+0000","last_became_peered":"2024-09-19T08:13:00.849051+0000","last_unstale":"2024-09-19T08:15:01.260317+0000","last_undegraded":"2024-09-19T08:15:01.260317+0000","last_fullsized":"2024-09-19T08:15:01.260317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:23.012928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'5","reported_seq":140,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.418714+0000","last_change":"2024-09-19T08:12:58.676121+0000","last_active":"2024-09-19T08:15:01.418714+0000","last_peered":"2024-09-19T08:15:01.418714+0000","last_clean":"2024-09-19T08:15:01.418714+0000","last_became_active":"2024-09-19T08:12:58.675693+0000","last_became_peered":"2024-09-19T08:12:58.675693+0000","last_unstale":"2024-09-19T08:15:01.418714+0000","last_undegraded":"2024-09-19T08:15:01.418714+0000","last_fullsized":"2024-09-19T08:15:01.418714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:34.882190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223583+0000","last_change":"2024-09-19T08:13:00.858735+0000","last_active":"2024-09-19T08:15:01.223583+0000","last_peered":"2024-09-19T08:15:01.223583+0000","last_clean":"2024-09-19T08:15:01.223583+0000","last_became_active":"2024-09-19T08:13:00.858218+0000","last_became_peered":"2024-09-19T08:13:00.858218+0000","last_unstale":"2024-09-19T08:15:01.223583+0000","last_undegraded":"2024-09-19T08:15:01.223583+0000","last_fullsized":"2024-09-19T08:15:01.223583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:33.356810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308176+0000","last_change":"2024-09-19T08:14:07.743785+0000","last_active":"2024-09-19T08:15:01.308176+0000","last_peered":"2024-09-19T08:15:01.308176+0000","last_clean":"2024-09-19T08:15:01.308176+0000","last_became_active":"2024-09-19T08:14:07.739531+0000","last_became_peered":"2024-09-19T08:14:07.739531+0000","last_unstale":"2024-09-19T08:15:01.308176+0000","last_undegraded":"2024-09-19T08:15:01.308176+0000","last_fullsized":"2024-09-19T08:15:01.308176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:43.367704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308976+0000","last_change":"2024-09-19T08:13:00.843852+0000","last_active":"2024-09-19T08:15:01.308976+0000","last_peered":"2024-09-19T08:15:01.308976+0000","last_clean":"2024-09-19T08:15:01.308976+0000","last_became_active":"2024-09-19T08:13:00.843497+0000","last_became_peered":"2024-09-19T08:13:00.843497+0000","last_unstale":"2024-09-19T08:15:01.308976+0000","last_undegraded":"2024-09-19T08:15:01.308976+0000","last_fullsized":"2024-09-19T08:15:01.308976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:26.380024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172698+0000","last_change":"2024-09-19T08:12:58.717225+0000","last_active":"2024-09-19T08:15:01.172698+0000","last_peered":"2024-09-19T08:15:01.172698+0000","last_clean":"2024-09-19T08:15:01.172698+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:15:01.172698+0000","last_undegraded":"2024-09-19T08:15:01.172698+0000","last_fullsized":"2024-09-19T08:15:01.172698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:43.201907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223446+0000","last_change":"2024-09-19T08:13:00.845613+0000","last_active":"2024-09-19T08:15:01.223446+0000","last_peered":"2024-09-19T08:15:01.223446+0000","last_clean":"2024-09-19T08:15:01.223446+0000","last_became_active":"2024-09-19T08:13:00.845260+0000","last_became_peered":"2024-09-19T08:13:00.845260+0000","last_unstale":"2024-09-19T08:15:01.223446+0000","last_undegraded":"2024-09-19T08:15:01.223446+0000","last_fullsized":"2024-09-19T08:15:01.223446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:03:47.033972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'36","reported_seq":311,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.820230+0000","last_change":"2024-09-19T08:14:07.749616+0000","last_active":"2024-09-19T08:15:01.820230+0000","last_peered":"2024-09-19T08:15:01.820230+0000","last_clean":"2024-09-19T08:15:01.820230+0000","last_became_active":"2024-09-19T08:14:07.740298+0000","last_became_peered":"2024-09-19T08:14:07.740298+0000","last_unstale":"2024-09-19T08:15:01.820230+0000","last_undegraded":"2024-09-19T08:15:01.820230+0000","last_fullsized":"2024-09-19T08:15:01.820230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:28.796855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66829,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":225,"num_read_kb":177,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61172,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263216+0000","last_change":"2024-09-19T08:13:00.855603+0000","last_active":"2024-09-19T08:15:01.263216+0000","last_peered":"2024-09-19T08:15:01.263216+0000","last_clean":"2024-09-19T08:15:01.263216+0000","last_became_active":"2024-09-19T08:13:00.855241+0000","last_became_peered":"2024-09-19T08:13:00.855241+0000","last_unstale":"2024-09-19T08:15:01.263216+0000","last_undegraded":"2024-09-19T08:15:01.263216+0000","last_fullsized":"2024-09-19T08:15:01.263216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:28.811847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308653+0000","last_change":"2024-09-19T08:12:58.675897+0000","last_active":"2024-09-19T08:15:01.308653+0000","last_peered":"2024-09-19T08:15:01.308653+0000","last_clean":"2024-09-19T08:15:01.308653+0000","last_became_active":"2024-09-19T08:12:58.675525+0000","last_became_peered":"2024-09-19T08:12:58.675525+0000","last_unstale":"2024-09-19T08:15:01.308653+0000","last_undegraded":"2024-09-19T08:15:01.308653+0000","last_fullsized":"2024-09-19T08:15:01.308653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:54.518064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309636+0000","last_change":"2024-09-19T08:13:00.856420+0000","last_active":"2024-09-19T08:15:01.309636+0000","last_peered":"2024-09-19T08:15:01.309636+0000","last_clean":"2024-09-19T08:15:01.309636+0000","last_became_active":"2024-09-19T08:13:00.856100+0000","last_became_peered":"2024-09-19T08:13:00.856100+0000","last_unstale":"2024-09-19T08:15:01.309636+0000","last_undegraded":"2024-09-19T08:15:01.309636+0000","last_fullsized":"2024-09-19T08:15:01.309636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:58.684930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172292+0000","last_change":"2024-09-19T08:12:58.680778+0000","last_active":"2024-09-19T08:15:01.172292+0000","last_peered":"2024-09-19T08:15:01.172292+0000","last_clean":"2024-09-19T08:15:01.172292+0000","last_became_active":"2024-09-19T08:12:58.680364+0000","last_became_peered":"2024-09-19T08:12:58.680364+0000","last_unstale":"2024-09-19T08:15:01.172292+0000","last_undegraded":"2024-09-19T08:15:01.172292+0000","last_fullsized":"2024-09-19T08:15:01.172292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:10.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308288+0000","last_change":"2024-09-19T08:13:00.853415+0000","last_active":"2024-09-19T08:15:01.308288+0000","last_peered":"2024-09-19T08:15:01.308288+0000","last_clean":"2024-09-19T08:15:01.308288+0000","last_became_active":"2024-09-19T08:13:00.853101+0000","last_became_peered":"2024-09-19T08:13:00.853101+0000","last_unstale":"2024-09-19T08:15:01.308288+0000","last_undegraded":"2024-09-19T08:15:01.308288+0000","last_fullsized":"2024-09-19T08:15:01.308288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:42.375394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":149,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:02.037848+0000","last_change":"2024-09-19T08:12:58.680312+0000","last_active":"2024-09-19T08:15:02.037848+0000","last_peered":"2024-09-19T08:15:02.037848+0000","last_clean":"2024-09-19T08:15:02.037848+0000","last_became_active":"2024-09-19T08:12:58.679870+0000","last_became_peered":"2024-09-19T08:12:58.679870+0000","last_unstale":"2024-09-19T08:15:02.037848+0000","last_undegraded":"2024-09-19T08:15:02.037848+0000","last_fullsized":"2024-09-19T08:15:02.037848+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:42.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308034+0000","last_change":"2024-09-19T08:13:00.853920+0000","last_active":"2024-09-19T08:15:01.308034+0000","last_peered":"2024-09-19T08:15:01.308034+0000","last_clean":"2024-09-19T08:15:01.308034+0000","last_became_active":"2024-09-19T08:13:00.853594+0000","last_became_peered":"2024-09-19T08:13:00.853594+0000","last_unstale":"2024-09-19T08:15:01.308034+0000","last_undegraded":"2024-09-19T08:15:01.308034+0000","last_fullsized":"2024-09-19T08:15:01.308034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:31.746133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308308+0000","last_change":"2024-09-19T08:14:07.751888+0000","last_active":"2024-09-19T08:15:01.308308+0000","last_peered":"2024-09-19T08:15:01.308308+0000","last_clean":"2024-09-19T08:15:01.308308+0000","last_became_active":"2024-09-19T08:14:07.737201+0000","last_became_peered":"2024-09-19T08:14:07.737201+0000","last_unstale":"2024-09-19T08:15:01.308308+0000","last_undegraded":"2024-09-19T08:15:01.308308+0000","last_fullsized":"2024-09-19T08:15:01.308308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:44.095987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172245+0000","last_change":"2024-09-19T08:13:00.847061+0000","last_active":"2024-09-19T08:15:01.172245+0000","last_peered":"2024-09-19T08:15:01.172245+0000","last_clean":"2024-09-19T08:15:01.172245+0000","last_became_active":"2024-09-19T08:13:00.846742+0000","last_became_peered":"2024-09-19T08:13:00.846742+0000","last_unstale":"2024-09-19T08:15:01.172245+0000","last_undegraded":"2024-09-19T08:15:01.172245+0000","last_fullsized":"2024-09-19T08:15:01.172245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:21.785585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309033+0000","last_change":"2024-09-19T08:14:07.755277+0000","last_active":"2024-09-19T08:15:01.309033+0000","last_peered":"2024-09-19T08:15:01.309033+0000","last_clean":"2024-09-19T08:15:01.309033+0000","last_became_active":"2024-09-19T08:14:07.739773+0000","last_became_peered":"2024-09-19T08:14:07.739773+0000","last_unstale":"2024-09-19T08:15:01.309033+0000","last_undegraded":"2024-09-19T08:15:01.309033+0000","last_fullsized":"2024-09-19T08:15:01.309033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:56.848251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308865+0000","last_change":"2024-09-19T08:13:00.844940+0000","last_active":"2024-09-19T08:15:01.308865+0000","last_peered":"2024-09-19T08:15:01.308865+0000","last_clean":"2024-09-19T08:15:01.308865+0000","last_became_active":"2024-09-19T08:13:00.844454+0000","last_became_peered":"2024-09-19T08:13:00.844454+0000","last_unstale":"2024-09-19T08:15:01.308865+0000","last_undegraded":"2024-09-19T08:15:01.308865+0000","last_fullsized":"2024-09-19T08:15:01.308865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:15.577624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309967+0000","last_change":"2024-09-19T08:12:58.673301+0000","last_active":"2024-09-19T08:15:01.309967+0000","last_peered":"2024-09-19T08:15:01.309967+0000","last_clean":"2024-09-19T08:15:01.309967+0000","last_became_active":"2024-09-19T08:12:58.672981+0000","last_became_peered":"2024-09-19T08:12:58.672981+0000","last_unstale":"2024-09-19T08:15:01.309967+0000","last_undegraded":"2024-09-19T08:15:01.309967+0000","last_fullsized":"2024-09-19T08:15:01.309967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:03.963712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308223+0000","last_change":"2024-09-19T08:13:00.856620+0000","last_active":"2024-09-19T08:15:01.308223+0000","last_peered":"2024-09-19T08:15:01.308223+0000","last_clean":"2024-09-19T08:15:01.308223+0000","last_became_active":"2024-09-19T08:13:00.856310+0000","last_became_peered":"2024-09-19T08:13:00.856310+0000","last_unstale":"2024-09-19T08:15:01.308223+0000","last_undegraded":"2024-09-19T08:15:01.308223+0000","last_fullsized":"2024-09-19T08:15:01.308223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:16.918768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'39","reported_seq":71,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.371288+0000","last_change":"2024-09-19T08:12:58.672282+0000","last_active":"2024-09-19T08:15:01.371288+0000","last_peered":"2024-09-19T08:15:01.371288+0000","last_clean":"2024-09-19T08:15:01.371288+0000","last_became_active":"2024-09-19T08:12:58.671891+0000","last_became_peered":"2024-09-19T08:12:58.671891+0000","last_unstale":"2024-09-19T08:15:01.371288+0000","last_undegraded":"2024-09-19T08:15:01.371288+0000","last_fullsized":"2024-09-19T08:15:01.371288+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:41.731571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308128+0000","last_change":"2024-09-19T08:13:00.848107+0000","last_active":"2024-09-19T08:15:01.308128+0000","last_peered":"2024-09-19T08:15:01.308128+0000","last_clean":"2024-09-19T08:15:01.308128+0000","last_became_active":"2024-09-19T08:13:00.847632+0000","last_became_peered":"2024-09-19T08:13:00.847632+0000","last_unstale":"2024-09-19T08:15:01.308128+0000","last_undegraded":"2024-09-19T08:15:01.308128+0000","last_fullsized":"2024-09-19T08:15:01.308128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.491122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308832+0000","last_change":"2024-09-19T08:12:59.179200+0000","last_active":"2024-09-19T08:15:01.308832+0000","last_peered":"2024-09-19T08:15:01.308832+0000","last_clean":"2024-09-19T08:15:01.308832+0000","last_became_active":"2024-09-19T08:12:59.178762+0000","last_became_peered":"2024-09-19T08:12:59.178762+0000","last_unstale":"2024-09-19T08:15:01.308832+0000","last_undegraded":"2024-09-19T08:15:01.308832+0000","last_fullsized":"2024-09-19T08:15:01.308832+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:08.424106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308358+0000","last_change":"2024-09-19T08:13:00.855870+0000","last_active":"2024-09-19T08:15:01.308358+0000","last_peered":"2024-09-19T08:15:01.308358+0000","last_clean":"2024-09-19T08:15:01.308358+0000","last_became_active":"2024-09-19T08:13:00.855538+0000","last_became_peered":"2024-09-19T08:13:00.855538+0000","last_unstale":"2024-09-19T08:15:01.308358+0000","last_undegraded":"2024-09-19T08:15:01.308358+0000","last_fullsized":"2024-09-19T08:15:01.308358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308350+0000","last_change":"2024-09-19T08:11:48.661333+0000","last_active":"2024-09-19T08:15:01.308350+0000","last_peered":"2024-09-19T08:15:01.308350+0000","last_clean":"2024-09-19T08:15:01.308350+0000","last_became_active":"2024-09-19T08:11:48.660873+0000","last_became_peered":"2024-09-19T08:11:48.660873+0000","last_unstale":"2024-09-19T08:15:01.308350+0000","last_undegraded":"2024-09-19T08:15:01.308350+0000","last_fullsized":"2024-09-19T08:15:01.308350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_clean_scrub_stamp":"2024-09-19T08:08:25.372155+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:19.401292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172985+0000","last_change":"2024-09-19T08:12:58.675237+0000","last_active":"2024-09-19T08:15:01.172985+0000","last_peered":"2024-09-19T08:15:01.172985+0000","last_clean":"2024-09-19T08:15:01.172985+0000","last_became_active":"2024-09-19T08:12:58.674910+0000","last_became_peered":"2024-09-19T08:12:58.674910+0000","last_unstale":"2024-09-19T08:15:01.172985+0000","last_undegraded":"2024-09-19T08:15:01.172985+0000","last_fullsized":"2024-09-19T08:15:01.172985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:47.038328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171023+0000","last_change":"2024-09-19T08:13:00.846754+0000","last_active":"2024-09-19T08:15:01.171023+0000","last_peered":"2024-09-19T08:15:01.171023+0000","last_clean":"2024-09-19T08:15:01.171023+0000","last_became_active":"2024-09-19T08:13:00.846452+0000","last_became_peered":"2024-09-19T08:13:00.846452+0000","last_unstale":"2024-09-19T08:15:01.171023+0000","last_undegraded":"2024-09-19T08:15:01.171023+0000","last_fullsized":"2024-09-19T08:15:01.171023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:22.929689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308239+0000","last_change":"2024-09-19T08:12:58.678958+0000","last_active":"2024-09-19T08:15:01.308239+0000","last_peered":"2024-09-19T08:15:01.308239+0000","last_clean":"2024-09-19T08:15:01.308239+0000","last_became_active":"2024-09-19T08:12:58.678592+0000","last_became_peered":"2024-09-19T08:12:58.678592+0000","last_unstale":"2024-09-19T08:15:01.308239+0000","last_undegraded":"2024-09-19T08:15:01.308239+0000","last_fullsized":"2024-09-19T08:15:01.308239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:58.374606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309700+0000","last_change":"2024-09-19T08:13:00.840973+0000","last_active":"2024-09-19T08:15:01.309700+0000","last_peered":"2024-09-19T08:15:01.309700+0000","last_clean":"2024-09-19T08:15:01.309700+0000","last_became_active":"2024-09-19T08:13:00.840619+0000","last_became_peered":"2024-09-19T08:13:00.840619+0000","last_unstale":"2024-09-19T08:15:01.309700+0000","last_undegraded":"2024-09-19T08:15:01.309700+0000","last_fullsized":"2024-09-19T08:15:01.309700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:02.037417+0000","last_change":"2024-09-19T08:12:58.715972+0000","last_active":"2024-09-19T08:15:02.037417+0000","last_peered":"2024-09-19T08:15:02.037417+0000","last_clean":"2024-09-19T08:15:02.037417+0000","last_became_active":"2024-09-19T08:12:58.715630+0000","last_became_peered":"2024-09-19T08:12:58.715630+0000","last_unstale":"2024-09-19T08:15:02.037417+0000","last_undegraded":"2024-09-19T08:15:02.037417+0000","last_fullsized":"2024-09-19T08:15:02.037417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:35.637121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172244+0000","last_change":"2024-09-19T08:13:00.847023+0000","last_active":"2024-09-19T08:15:01.172244+0000","last_peered":"2024-09-19T08:15:01.172244+0000","last_clean":"2024-09-19T08:15:01.172244+0000","last_became_active":"2024-09-19T08:13:00.846387+0000","last_became_peered":"2024-09-19T08:13:00.846387+0000","last_unstale":"2024-09-19T08:15:01.172244+0000","last_undegraded":"2024-09-19T08:15:01.172244+0000","last_fullsized":"2024-09-19T08:15:01.172244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:12.187929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308283+0000","last_change":"2024-09-19T08:12:58.681062+0000","last_active":"2024-09-19T08:15:01.308283+0000","last_peered":"2024-09-19T08:15:01.308283+0000","last_clean":"2024-09-19T08:15:01.308283+0000","last_became_active":"2024-09-19T08:12:58.680524+0000","last_became_peered":"2024-09-19T08:12:58.680524+0000","last_unstale":"2024-09-19T08:15:01.308283+0000","last_undegraded":"2024-09-19T08:15:01.308283+0000","last_fullsized":"2024-09-19T08:15:01.308283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271612+0000","last_change":"2024-09-19T08:13:00.851523+0000","last_active":"2024-09-19T08:15:01.271612+0000","last_peered":"2024-09-19T08:15:01.271612+0000","last_clean":"2024-09-19T08:15:01.271612+0000","last_became_active":"2024-09-19T08:13:00.851068+0000","last_became_peered":"2024-09-19T08:13:00.851068+0000","last_unstale":"2024-09-19T08:15:01.271612+0000","last_undegraded":"2024-09-19T08:15:01.271612+0000","last_fullsized":"2024-09-19T08:15:01.271612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:44.905469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.260876+0000","last_change":"2024-09-19T08:12:58.718234+0000","last_active":"2024-09-19T08:15:01.260876+0000","last_peered":"2024-09-19T08:15:01.260876+0000","last_clean":"2024-09-19T08:15:01.260876+0000","last_became_active":"2024-09-19T08:12:58.717840+0000","last_became_peered":"2024-09-19T08:12:58.717840+0000","last_unstale":"2024-09-19T08:15:01.260876+0000","last_undegraded":"2024-09-19T08:15:01.260876+0000","last_fullsized":"2024-09-19T08:15:01.260876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:05.283770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271611+0000","last_change":"2024-09-19T08:13:00.848305+0000","last_active":"2024-09-19T08:15:01.271611+0000","last_peered":"2024-09-19T08:15:01.271611+0000","last_clean":"2024-09-19T08:15:01.271611+0000","last_became_active":"2024-09-19T08:13:00.847981+0000","last_became_peered":"2024-09-19T08:13:00.847981+0000","last_unstale":"2024-09-19T08:15:01.271611+0000","last_undegraded":"2024-09-19T08:15:01.271611+0000","last_fullsized":"2024-09-19T08:15:01.271611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:04.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263326+0000","last_change":"2024-09-19T08:12:58.715943+0000","last_active":"2024-09-19T08:15:01.263326+0000","last_peered":"2024-09-19T08:15:01.263326+0000","last_clean":"2024-09-19T08:15:01.263326+0000","last_became_active":"2024-09-19T08:12:58.715600+0000","last_became_peered":"2024-09-19T08:12:58.715600+0000","last_unstale":"2024-09-19T08:15:01.263326+0000","last_undegraded":"2024-09-19T08:15:01.263326+0000","last_fullsized":"2024-09-19T08:15:01.263326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:19.142271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310614+0000","last_change":"2024-09-19T08:13:00.840944+0000","last_active":"2024-09-19T08:15:01.310614+0000","last_peered":"2024-09-19T08:15:01.310614+0000","last_clean":"2024-09-19T08:15:01.310614+0000","last_became_active":"2024-09-19T08:13:00.840608+0000","last_became_peered":"2024-09-19T08:13:00.840608+0000","last_unstale":"2024-09-19T08:15:01.310614+0000","last_undegraded":"2024-09-19T08:15:01.310614+0000","last_fullsized":"2024-09-19T08:15:01.310614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:25.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'47","reported_seq":94,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.374602+0000","last_change":"2024-09-19T08:12:58.672929+0000","last_active":"2024-09-19T08:15:01.374602+0000","last_peered":"2024-09-19T08:15:01.374602+0000","last_clean":"2024-09-19T08:15:01.374602+0000","last_became_active":"2024-09-19T08:12:58.672487+0000","last_became_peered":"2024-09-19T08:12:58.672487+0000","last_unstale":"2024-09-19T08:15:01.374602+0000","last_undegraded":"2024-09-19T08:15:01.374602+0000","last_fullsized":"2024-09-19T08:15:01.374602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:33.042735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.261959+0000","last_change":"2024-09-19T08:14:07.738260+0000","last_active":"2024-09-19T08:15:01.261959+0000","last_peered":"2024-09-19T08:15:01.261959+0000","last_clean":"2024-09-19T08:15:01.261959+0000","last_became_active":"2024-09-19T08:14:07.737915+0000","last_became_peered":"2024-09-19T08:14:07.737915+0000","last_unstale":"2024-09-19T08:15:01.261959+0000","last_undegraded":"2024-09-19T08:15:01.261959+0000","last_fullsized":"2024-09-19T08:15:01.261959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:02.500657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.173256+0000","last_change":"2024-09-19T08:12:59.179510+0000","last_active":"2024-09-19T08:15:01.173256+0000","last_peered":"2024-09-19T08:15:01.173256+0000","last_clean":"2024-09-19T08:15:01.173256+0000","last_became_active":"2024-09-19T08:12:59.179157+0000","last_became_peered":"2024-09-19T08:12:59.179157+0000","last_unstale":"2024-09-19T08:15:01.173256+0000","last_undegraded":"2024-09-19T08:15:01.173256+0000","last_fullsized":"2024-09-19T08:15:01.173256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:22.981289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308618+0000","last_change":"2024-09-19T08:13:00.841853+0000","last_active":"2024-09-19T08:15:01.308618+0000","last_peered":"2024-09-19T08:15:01.308618+0000","last_clean":"2024-09-19T08:15:01.308618+0000","last_became_active":"2024-09-19T08:13:00.841518+0000","last_became_peered":"2024-09-19T08:13:00.841518+0000","last_unstale":"2024-09-19T08:15:01.308618+0000","last_undegraded":"2024-09-19T08:15:01.308618+0000","last_fullsized":"2024-09-19T08:15:01.308618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:05.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223399+0000","last_change":"2024-09-19T08:12:58.716882+0000","last_active":"2024-09-19T08:15:01.223399+0000","last_peered":"2024-09-19T08:15:01.223399+0000","last_clean":"2024-09-19T08:15:01.223399+0000","last_became_active":"2024-09-19T08:12:58.716545+0000","last_became_peered":"2024-09-19T08:12:58.716545+0000","last_unstale":"2024-09-19T08:15:01.223399+0000","last_undegraded":"2024-09-19T08:15:01.223399+0000","last_fullsized":"2024-09-19T08:15:01.223399+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:14.469109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262936+0000","last_change":"2024-09-19T08:13:00.850484+0000","last_active":"2024-09-19T08:15:01.262936+0000","last_peered":"2024-09-19T08:15:01.262936+0000","last_clean":"2024-09-19T08:15:01.262936+0000","last_became_active":"2024-09-19T08:13:00.850049+0000","last_became_peered":"2024-09-19T08:13:00.850049+0000","last_unstale":"2024-09-19T08:15:01.262936+0000","last_undegraded":"2024-09-19T08:15:01.262936+0000","last_fullsized":"2024-09-19T08:15:01.262936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:24.424111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170936+0000","last_change":"2024-09-19T08:12:58.669928+0000","last_active":"2024-09-19T08:15:01.170936+0000","last_peered":"2024-09-19T08:15:01.170936+0000","last_clean":"2024-09-19T08:15:01.170936+0000","last_became_active":"2024-09-19T08:12:58.669614+0000","last_became_peered":"2024-09-19T08:12:58.669614+0000","last_unstale":"2024-09-19T08:15:01.170936+0000","last_undegraded":"2024-09-19T08:15:01.170936+0000","last_fullsized":"2024-09-19T08:15:01.170936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:50.924779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308456+0000","last_change":"2024-09-19T08:13:00.853520+0000","last_active":"2024-09-19T08:15:01.308456+0000","last_peered":"2024-09-19T08:15:01.308456+0000","last_clean":"2024-09-19T08:15:01.308456+0000","last_became_active":"2024-09-19T08:13:00.853184+0000","last_became_peered":"2024-09-19T08:13:00.853184+0000","last_unstale":"2024-09-19T08:15:01.308456+0000","last_undegraded":"2024-09-19T08:15:01.308456+0000","last_fullsized":"2024-09-19T08:15:01.308456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:45.160439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308612+0000","last_change":"2024-09-19T08:12:58.674359+0000","last_active":"2024-09-19T08:15:01.308612+0000","last_peered":"2024-09-19T08:15:01.308612+0000","last_clean":"2024-09-19T08:15:01.308612+0000","last_became_active":"2024-09-19T08:12:58.673890+0000","last_became_peered":"2024-09-19T08:12:58.673890+0000","last_unstale":"2024-09-19T08:15:01.308612+0000","last_undegraded":"2024-09-19T08:15:01.308612+0000","last_fullsized":"2024-09-19T08:15:01.308612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:54.939959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171020+0000","last_change":"2024-09-19T08:13:00.851071+0000","last_active":"2024-09-19T08:15:01.171020+0000","last_peered":"2024-09-19T08:15:01.171020+0000","last_clean":"2024-09-19T08:15:01.171020+0000","last_became_active":"2024-09-19T08:13:00.850756+0000","last_became_peered":"2024-09-19T08:13:00.850756+0000","last_unstale":"2024-09-19T08:15:01.171020+0000","last_undegraded":"2024-09-19T08:15:01.171020+0000","last_fullsized":"2024-09-19T08:15:01.171020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:52.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263021+0000","last_change":"2024-09-19T08:12:58.681466+0000","last_active":"2024-09-19T08:15:01.263021+0000","last_peered":"2024-09-19T08:15:01.263021+0000","last_clean":"2024-09-19T08:15:01.263021+0000","last_became_active":"2024-09-19T08:12:58.681113+0000","last_became_peered":"2024-09-19T08:12:58.681113+0000","last_unstale":"2024-09-19T08:15:01.263021+0000","last_undegraded":"2024-09-19T08:15:01.263021+0000","last_fullsized":"2024-09-19T08:15:01.263021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:49.610828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308386+0000","last_change":"2024-09-19T08:13:00.851272+0000","last_active":"2024-09-19T08:15:01.308386+0000","last_peered":"2024-09-19T08:15:01.308386+0000","last_clean":"2024-09-19T08:15:01.308386+0000","last_became_active":"2024-09-19T08:13:00.849098+0000","last_became_peered":"2024-09-19T08:13:00.849098+0000","last_unstale":"2024-09-19T08:15:01.308386+0000","last_undegraded":"2024-09-19T08:15:01.308386+0000","last_fullsized":"2024-09-19T08:15:01.308386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:05.643992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172513+0000","last_change":"2024-09-19T08:14:07.738913+0000","last_active":"2024-09-19T08:15:01.172513+0000","last_peered":"2024-09-19T08:15:01.172513+0000","last_clean":"2024-09-19T08:15:01.172513+0000","last_became_active":"2024-09-19T08:14:07.738560+0000","last_became_peered":"2024-09-19T08:14:07.738560+0000","last_unstale":"2024-09-19T08:15:01.172513+0000","last_undegraded":"2024-09-19T08:15:01.172513+0000","last_fullsized":"2024-09-19T08:15:01.172513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:35.731290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263192+0000","last_change":"2024-09-19T08:12:58.674610+0000","last_active":"2024-09-19T08:15:01.263192+0000","last_peered":"2024-09-19T08:15:01.263192+0000","last_clean":"2024-09-19T08:15:01.263192+0000","last_became_active":"2024-09-19T08:12:58.674247+0000","last_became_peered":"2024-09-19T08:12:58.674247+0000","last_unstale":"2024-09-19T08:15:01.263192+0000","last_undegraded":"2024-09-19T08:15:01.263192+0000","last_fullsized":"2024-09-19T08:15:01.263192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:16.323574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223395+0000","last_change":"2024-09-19T08:13:00.858512+0000","last_active":"2024-09-19T08:15:01.223395+0000","last_peered":"2024-09-19T08:15:01.223395+0000","last_clean":"2024-09-19T08:15:01.223395+0000","last_became_active":"2024-09-19T08:13:00.857897+0000","last_became_peered":"2024-09-19T08:13:00.857897+0000","last_unstale":"2024-09-19T08:15:01.223395+0000","last_undegraded":"2024-09-19T08:15:01.223395+0000","last_fullsized":"2024-09-19T08:15:01.223395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263238+0000","last_change":"2024-09-19T08:12:58.682207+0000","last_active":"2024-09-19T08:15:01.263238+0000","last_peered":"2024-09-19T08:15:01.263238+0000","last_clean":"2024-09-19T08:15:01.263238+0000","last_became_active":"2024-09-19T08:12:58.681842+0000","last_became_peered":"2024-09-19T08:12:58.681842+0000","last_unstale":"2024-09-19T08:15:01.263238+0000","last_undegraded":"2024-09-19T08:15:01.263238+0000","last_fullsized":"2024-09-19T08:15:01.263238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.825858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308840+0000","last_change":"2024-09-19T08:13:00.846748+0000","last_active":"2024-09-19T08:15:01.308840+0000","last_peered":"2024-09-19T08:15:01.308840+0000","last_clean":"2024-09-19T08:15:01.308840+0000","last_became_active":"2024-09-19T08:13:00.846368+0000","last_became_peered":"2024-09-19T08:13:00.846368+0000","last_unstale":"2024-09-19T08:15:01.308840+0000","last_undegraded":"2024-09-19T08:15:01.308840+0000","last_fullsized":"2024-09-19T08:15:01.308840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:59.526512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171334+0000","last_change":"2024-09-19T08:12:58.716118+0000","last_active":"2024-09-19T08:15:01.171334+0000","last_peered":"2024-09-19T08:15:01.171334+0000","last_clean":"2024-09-19T08:15:01.171334+0000","last_became_active":"2024-09-19T08:12:58.715711+0000","last_became_peered":"2024-09-19T08:12:58.715711+0000","last_unstale":"2024-09-19T08:15:01.171334+0000","last_undegraded":"2024-09-19T08:15:01.171334+0000","last_fullsized":"2024-09-19T08:15:01.171334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:29.439887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171172+0000","last_change":"2024-09-19T08:13:00.848729+0000","last_active":"2024-09-19T08:15:01.171172+0000","last_peered":"2024-09-19T08:15:01.171172+0000","last_clean":"2024-09-19T08:15:01.171172+0000","last_became_active":"2024-09-19T08:13:00.848470+0000","last_became_peered":"2024-09-19T08:13:00.848470+0000","last_unstale":"2024-09-19T08:15:01.171172+0000","last_undegraded":"2024-09-19T08:15:01.171172+0000","last_fullsized":"2024-09-19T08:15:01.171172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:30.653101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262232+0000","last_change":"2024-09-19T08:12:58.681654+0000","last_active":"2024-09-19T08:15:01.262232+0000","last_peered":"2024-09-19T08:15:01.262232+0000","last_clean":"2024-09-19T08:15:01.262232+0000","last_became_active":"2024-09-19T08:12:58.681284+0000","last_became_peered":"2024-09-19T08:12:58.681284+0000","last_unstale":"2024-09-19T08:15:01.262232+0000","last_undegraded":"2024-09-19T08:15:01.262232+0000","last_fullsized":"2024-09-19T08:15:01.262232+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.429930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597810,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":784,"num_read_kb":501,"num_write":297,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":61306,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":1793430,"data_compressed":85517,"data_compressed_allocated":110592,"data_compressed_original":1605632,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":109176,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481828,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":454888,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710762,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":1246292,"data_compressed":51265,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":848206,"data_compressed":27863,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103220,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":958464,"data_stored":847992,"data_compressed":27863,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":1049004,"data_compressed":45109,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":933888,"data_stored":454379,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":1178660,"data_compressed":51265,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":517387,"data_compressed":18328,"data_compressed_allocated":24576,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":979506,"data_compressed":34019,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":950272,"data_stored":456449,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":515901,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":455650,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":69101,"data_compressed":9611,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134003,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":393820,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201635,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:15:06.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:15:06.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:06 smithi064 ceph-mon[34900]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:15:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:06 smithi064 ceph-mon[34900]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T08:15:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:06 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:06 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:06 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:06 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:06 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:06 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:06 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:06.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:06 smithi064 ceph-mon[34900]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.311+0000 7fb5ae102640 1 -- 172.21.15.22:0/1177599103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5a8102790 msgr2=0x7fb5a81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.311+0000 7fb5ae102640 1 --2- 172.21.15.22:0/1177599103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5a8102790 0x7fb5a81067e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009900 tx=0x7fb59c02a410 comp rx=0 tx=0).stop 2024-09-19T08:15:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.311+0000 7fb5ae102640 1 -- 172.21.15.22:0/1177599103 shutdown_connections 2024-09-19T08:15:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.311+0000 7fb5ae102640 1 --2- 172.21.15.22:0/1177599103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a8106f10 0x7fb5a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.311+0000 7fb5ae102640 1 --2- 172.21.15.22:0/1177599103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5a8102790 0x7fb5a81067e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.311+0000 7fb5ae102640 1 --2- 172.21.15.22:0/1177599103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5a8101de0 0x7fb5a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.311+0000 7fb5ae102640 1 -- 172.21.15.22:0/1177599103 >> 172.21.15.22:0/1177599103 conn(0x7fb5a80fd8c0 msgr2=0x7fb5a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.311+0000 7fb5ae102640 1 -- 172.21.15.22:0/1177599103 shutdown_connections 2024-09-19T08:15:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.311+0000 7fb5ae102640 1 -- 172.21.15.22:0/1177599103 wait complete. 2024-09-19T08:15:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.312+0000 7fb5ae102640 1 Processor -- start 2024-09-19T08:15:07.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.312+0000 7fb5ae102640 1 -- start start 2024-09-19T08:15:07.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.312+0000 7fb5ae102640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5a8101de0 0x7fb5a8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:07.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.312+0000 7fb5ae102640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a8102790 0x7fb5a8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:07.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5ae102640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5a8106f10 0x7fb5a819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:07.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5ae102640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a8111f70 con 0x7fb5a8106f10 2024-09-19T08:15:07.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5ae102640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a8111df0 con 0x7fb5a8102790 2024-09-19T08:15:07.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5ae102640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a81120f0 con 0x7fb5a8101de0 2024-09-19T08:15:07.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5a7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5a8106f10 0x7fb5a819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:07.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5a7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5a8106f10 0x7fb5a819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42524/0 (socket says 172.21.15.22:42524) 2024-09-19T08:15:07.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5a7fff640 1 -- 172.21.15.22:0/332833883 learned_addr learned my addr 172.21.15.22:0/332833883 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:07.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5a77fe640 1 --2- 172.21.15.22:0/332833883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5a8101de0 0x7fb5a8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:07.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5a6ffd640 1 --2- 172.21.15.22:0/332833883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a8102790 0x7fb5a8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5a7fff640 1 -- 172.21.15.22:0/332833883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5a8101de0 msgr2=0x7fb5a8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5a7fff640 1 --2- 172.21.15.22:0/332833883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5a8101de0 0x7fb5a8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5a7fff640 1 -- 172.21.15.22:0/332833883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a8102790 msgr2=0x7fb5a8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5a7fff640 1 --2- 172.21.15.22:0/332833883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a8102790 0x7fb5a8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5a7fff640 1 -- 172.21.15.22:0/332833883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a819d3e0 con 0x7fb5a8106f10 2024-09-19T08:15:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5a7fff640 1 --2- 172.21.15.22:0/332833883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5a8106f10 0x7fb5a819cd00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000eaa0 tx=0x7fb5a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5a4ff9640 1 -- 172.21.15.22:0/332833883 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a000cdb0 con 0x7fb5a8106f10 2024-09-19T08:15:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5a4ff9640 1 -- 172.21.15.22:0/332833883 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a0005020 con 0x7fb5a8106f10 2024-09-19T08:15:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5ae102640 1 -- 172.21.15.22:0/332833883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a819d6d0 con 0x7fb5a8106f10 2024-09-19T08:15:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5a4ff9640 1 -- 172.21.15.22:0/332833883 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a00116b0 con 0x7fb5a8106f10 2024-09-19T08:15:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.313+0000 7fb5ae102640 1 -- 172.21.15.22:0/332833883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a81a4fb0 con 0x7fb5a8106f10 2024-09-19T08:15:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.314+0000 7fb5ae102640 1 -- 172.21.15.22:0/332833883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a81021c0 con 0x7fb5a8106f10 2024-09-19T08:15:07.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.316+0000 7fb5a4ff9640 1 -- 172.21.15.22:0/332833883 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fb5a0005560 con 0x7fb5a8106f10 2024-09-19T08:15:07.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.316+0000 7fb5a4ff9640 1 --2- 172.21.15.22:0/332833883 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb5900a34d0 0x7fb5900a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:07.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.316+0000 7fb5a77fe640 1 --2- 172.21.15.22:0/332833883 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb5900a34d0 0x7fb5900a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:07.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.316+0000 7fb5a4ff9640 1 -- 172.21.15.22:0/332833883 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb5a00d01b0 con 0x7fb5a8106f10 2024-09-19T08:15:07.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.317+0000 7fb5a77fe640 1 --2- 172.21.15.22:0/332833883 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb5900a34d0 0x7fb5900a5990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb5980058c0 tx=0x7fb59800a250 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:07.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.320+0000 7fb5a4ff9640 1 -- 172.21.15.22:0/332833883 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb5a0016030 con 0x7fb5a8106f10 2024-09-19T08:15:07.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:07 smithi157 ceph-mon[35143]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:07.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:07 smithi157 ceph-mon[35143]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:07 smithi022 ceph-mon[30963]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:07 smithi022 ceph-mon[30963]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.535+0000 7fb5ae102640 1 -- 172.21.15.22:0/332833883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5a810a150 con 0x7fb5a8106f10 2024-09-19T08:15:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.536+0000 7fb5a4ff9640 1 -- 172.21.15.22:0/332833883 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11906 (secure 0 0 0) 0x7fb5a009c950 con 0x7fb5a8106f10 2024-09-19T08:15:07.537 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:07.537 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"btime":"2024-09-19T08:15:00:109390+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:15:00.059208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:15:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-09-19T08:15:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.538+0000 7fb5ae102640 1 -- 172.21.15.22:0/332833883 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb5900a34d0 msgr2=0x7fb5900a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.538+0000 7fb5ae102640 1 --2- 172.21.15.22:0/332833883 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb5900a34d0 0x7fb5900a5990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb5980058c0 tx=0x7fb59800a250 comp rx=0 tx=0).stop 2024-09-19T08:15:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.538+0000 7fb5ae102640 1 -- 172.21.15.22:0/332833883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5a8106f10 msgr2=0x7fb5a819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.538+0000 7fb5ae102640 1 --2- 172.21.15.22:0/332833883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5a8106f10 0x7fb5a819cd00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000eaa0 tx=0x7fb5a000ef60 comp rx=0 tx=0).stop 2024-09-19T08:15:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.538+0000 7fb5ae102640 1 -- 172.21.15.22:0/332833883 shutdown_connections 2024-09-19T08:15:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.538+0000 7fb5ae102640 1 --2- 172.21.15.22:0/332833883 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb5900a34d0 0x7fb5900a5990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.538+0000 7fb5ae102640 1 --2- 172.21.15.22:0/332833883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5a8106f10 0x7fb5a819cd00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.538+0000 7fb5ae102640 1 --2- 172.21.15.22:0/332833883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a8102790 0x7fb5a8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.538+0000 7fb5ae102640 1 --2- 172.21.15.22:0/332833883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5a8101de0 0x7fb5a8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.538+0000 7fb5ae102640 1 -- 172.21.15.22:0/332833883 >> 172.21.15.22:0/332833883 conn(0x7fb5a80fd8c0 msgr2=0x7fb5a80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.538+0000 7fb5ae102640 1 -- 172.21.15.22:0/332833883 shutdown_connections 2024-09-19T08:15:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:07.538+0000 7fb5ae102640 1 -- 172.21.15.22:0/332833883 wait complete. 2024-09-19T08:15:07.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:07 smithi064 ceph-mon[34900]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:07.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:07 smithi064 ceph-mon[34900]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:08.390 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:08 smithi022 ceph-mon[30963]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 205 B/s wr, 1 op/s 2024-09-19T08:15:08.390 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:08 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/332833883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:08.426 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:15:08.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:08 smithi157 ceph-mon[35143]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 205 B/s wr, 1 op/s 2024-09-19T08:15:08.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:08 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/332833883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:08.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:08 smithi064 ceph-mon[34900]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 205 B/s wr, 1 op/s 2024-09-19T08:15:08.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:08 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/332833883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:08.690 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:09.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.186+0000 7f3e84a28640 1 -- 172.21.15.22:0/1336956078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80102570 msgr2=0x7f3e801029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.186+0000 7f3e84a28640 1 --2- 172.21.15.22:0/1336956078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80102570 0x7f3e801029b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3e74009900 tx=0x7f3e7402a480 comp rx=0 tx=0).stop 2024-09-19T08:15:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.186+0000 7f3e84a28640 1 -- 172.21.15.22:0/1336956078 shutdown_connections 2024-09-19T08:15:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.186+0000 7f3e84a28640 1 --2- 172.21.15.22:0/1336956078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e80102ef0 0x7f3e8010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.186+0000 7f3e84a28640 1 --2- 172.21.15.22:0/1336956078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80102570 0x7f3e801029b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.186+0000 7f3e84a28640 1 --2- 172.21.15.22:0/1336956078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80100e20 0x7f3e80101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.186+0000 7f3e84a28640 1 -- 172.21.15.22:0/1336956078 >> 172.21.15.22:0/1336956078 conn(0x7f3e800fc7d0 msgr2=0x7f3e800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.186+0000 7f3e84a28640 1 -- 172.21.15.22:0/1336956078 shutdown_connections 2024-09-19T08:15:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.186+0000 7f3e84a28640 1 -- 172.21.15.22:0/1336956078 wait complete. 2024-09-19T08:15:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.187+0000 7f3e84a28640 1 Processor -- start 2024-09-19T08:15:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.187+0000 7f3e84a28640 1 -- start start 2024-09-19T08:15:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.187+0000 7f3e84a28640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80100e20 0x7f3e800fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.187+0000 7f3e84a28640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80102570 0x7f3e800fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.187+0000 7f3e84a28640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e80102ef0 0x7f3e800fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.187+0000 7f3e84a28640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e80110f80 con 0x7f3e80100e20 2024-09-19T08:15:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.187+0000 7f3e84a28640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3e80110e00 con 0x7f3e80102570 2024-09-19T08:15:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.187+0000 7f3e84a28640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3e80111100 con 0x7f3e80102ef0 2024-09-19T08:15:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.187+0000 7f3e7e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80100e20 0x7f3e800fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.187+0000 7f3e7e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80100e20 0x7f3e800fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42682/0 (socket says 172.21.15.22:42682) 2024-09-19T08:15:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e7e59c640 1 -- 172.21.15.22:0/3364220680 learned_addr learned my addr 172.21.15.22:0/3364220680 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e7ed9d640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e80102ef0 0x7f3e800fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e7dd9b640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80102570 0x7f3e800fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e7e59c640 1 -- 172.21.15.22:0/3364220680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e80102ef0 msgr2=0x7f3e800fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e7e59c640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e80102ef0 0x7f3e800fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e7e59c640 1 -- 172.21.15.22:0/3364220680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80102570 msgr2=0x7f3e800fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e7e59c640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80102570 0x7f3e800fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e7e59c640 1 -- 172.21.15.22:0/3364220680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e801a6990 con 0x7f3e80100e20 2024-09-19T08:15:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e7dd9b640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80102570 0x7f3e800fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e7e59c640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80100e20 0x7f3e800fcc80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000d950 tx=0x7f3e7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e6f7fe640 1 -- 172.21.15.22:0/3364220680 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70017070 con 0x7f3e80100e20 2024-09-19T08:15:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e84a28640 1 -- 172.21.15.22:0/3364220680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e801a6c80 con 0x7f3e80100e20 2024-09-19T08:15:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e6f7fe640 1 -- 172.21.15.22:0/3364220680 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e70009c30 con 0x7f3e80100e20 2024-09-19T08:15:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e6f7fe640 1 -- 172.21.15.22:0/3364220680 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7000f6b0 con 0x7f3e80100e20 2024-09-19T08:15:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.188+0000 7f3e84a28640 1 -- 172.21.15.22:0/3364220680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e801a71c0 con 0x7f3e80100e20 2024-09-19T08:15:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.189+0000 7f3e84a28640 1 -- 172.21.15.22:0/3364220680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e80101200 con 0x7f3e80100e20 2024-09-19T08:15:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.191+0000 7f3e6f7fe640 1 -- 172.21.15.22:0/3364220680 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f3e700109d0 con 0x7f3e80100e20 2024-09-19T08:15:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.192+0000 7f3e6f7fe640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3e680a3550 0x7f3e680a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.193+0000 7f3e7dd9b640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3e680a3550 0x7f3e680a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.193+0000 7f3e6f7fe640 1 -- 172.21.15.22:0/3364220680 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3e700cfbe0 con 0x7f3e80100e20 2024-09-19T08:15:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.193+0000 7f3e7dd9b640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3e680a3550 0x7f3e680a5a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e800fe670 tx=0x7f3e74005a20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.195+0000 7f3e6f7fe640 1 -- 172.21.15.22:0/3364220680 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3e70009de0 con 0x7f3e80100e20 2024-09-19T08:15:09.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.373+0000 7f3e84a28640 1 -- 172.21.15.22:0/3364220680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e8010a0d0 con 0x7f3e80100e20 2024-09-19T08:15:09.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.375+0000 7f3e6f7fe640 1 -- 172.21.15.22:0/3364220680 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+23100 (secure 0 0 0) 0x7f3e7009c3a0 con 0x7f3e80100e20 2024-09-19T08:15:09.376 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:09.376 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":86,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","created":"2024-09-19T08:05:19.476952+0000","modified":"2024-09-19T08:15:01.165514+0000","last_up_change":"2024-09-19T08:12:10.339586+0000","last_in_change":"2024-09-19T08:11:47.653576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:08:25.175759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:12:57.608787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:12:59.824327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ed67f2d-9b28-427a-b69f-676e7a2abcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6803","nonce":3442715307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6805","nonce":3442715307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6809","nonce":3442715307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6807","nonce":3442715307}]},"public_addr":"172.21.15.22:6803/3442715307","cluster_addr":"172.21.15.22:6805/3442715307","heartbeat_back_addr":"172.21.15.22:6809/3442715307","heartbeat_front_addr":"172.21.15.22:6807/3442715307","state":["exists","up"]},{"osd":1,"uuid":"bff3e283-4ab1-49d7-86d8-45d361995122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6801","nonce":2379700228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6803","nonce":2379700228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6807","nonce":2379700228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6805","nonce":2379700228}]},"public_addr":"172.21.15.64:6801/2379700228","cluster_addr":"172.21.15.64:6803/2379700228","heartbeat_back_addr":"172.21.15.64:6807/2379700228","heartbeat_front_addr":"172.21.15.64:6805/2379700228","state":["exists","up"]},{"osd":2,"uuid":"81421d08-7dc6-419b-a566-dee1e1e13b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6801","nonce":3363263867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6803","nonce":3363263867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6807","nonce":3363263867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6805","nonce":3363263867}]},"public_addr":"172.21.15.157:6801/3363263867","cluster_addr":"172.21.15.157:6803/3363263867","heartbeat_back_addr":"172.21.15.157:6807/3363263867","heartbeat_front_addr":"172.21.15.157:6805/3363263867","state":["exists","up"]},{"osd":3,"uuid":"34debfd0-3226-4d78-b9e6-e03dbc5190f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6811","nonce":4225229456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6813","nonce":4225229456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6817","nonce":4225229456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6815","nonce":4225229456}]},"public_addr":"172.21.15.22:6811/4225229456","cluster_addr":"172.21.15.22:6813/4225229456","heartbeat_back_addr":"172.21.15.22:6817/4225229456","heartbeat_front_addr":"172.21.15.22:6815/4225229456","state":["exists","up"]},{"osd":4,"uuid":"1293332c-d772-4ca9-bf76-79e0b29451ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6809","nonce":1858090423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6811","nonce":1858090423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6815","nonce":1858090423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6813","nonce":1858090423}]},"public_addr":"172.21.15.64:6809/1858090423","cluster_addr":"172.21.15.64:6811/1858090423","heartbeat_back_addr":"172.21.15.64:6815/1858090423","heartbeat_front_addr":"172.21.15.64:6813/1858090423","state":["exists","up"]},{"osd":5,"uuid":"240a6c42-89a2-41c1-948c-b09914b5722e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6809","nonce":2900701212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2900701212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2900701212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2900701212}]},"public_addr":"172.21.15.157:6809/2900701212","cluster_addr":"172.21.15.157:6811/2900701212","heartbeat_back_addr":"172.21.15.157:6815/2900701212","heartbeat_front_addr":"172.21.15.157:6813/2900701212","state":["exists","up"]},{"osd":6,"uuid":"80199030-109e-41c4-bc1c-731755464770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6819","nonce":4264426242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6821","nonce":4264426242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6825","nonce":4264426242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6823","nonce":4264426242}]},"public_addr":"172.21.15.22:6819/4264426242","cluster_addr":"172.21.15.22:6821/4264426242","heartbeat_back_addr":"172.21.15.22:6825/4264426242","heartbeat_front_addr":"172.21.15.22:6823/4264426242","state":["exists","up"]},{"osd":7,"uuid":"dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6817","nonce":3980224103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6819","nonce":3980224103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6823","nonce":3980224103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6821","nonce":3980224103}]},"public_addr":"172.21.15.64:6817/3980224103","cluster_addr":"172.21.15.64:6819/3980224103","heartbeat_back_addr":"172.21.15.64:6823/3980224103","heartbeat_front_addr":"172.21.15.64:6821/3980224103","state":["exists","up"]},{"osd":8,"uuid":"764b1e6e-48c9-49f9-8124-bc13bcba3a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6817","nonce":973678477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6819","nonce":973678477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6823","nonce":973678477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6821","nonce":973678477}]},"public_addr":"172.21.15.157:6817/973678477","cluster_addr":"172.21.15.157:6819/973678477","heartbeat_back_addr":"172.21.15.157:6823/973678477","heartbeat_front_addr":"172.21.15.157:6821/973678477","state":["exists","up"]},{"osd":9,"uuid":"82914818-df47-4374-9e3e-2f71fc76e78b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6827","nonce":710033990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6829","nonce":710033990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6833","nonce":710033990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6831","nonce":710033990}]},"public_addr":"172.21.15.22:6827/710033990","cluster_addr":"172.21.15.22:6829/710033990","heartbeat_back_addr":"172.21.15.22:6833/710033990","heartbeat_front_addr":"172.21.15.22:6831/710033990","state":["exists","up"]},{"osd":10,"uuid":"36ae177d-c454-4cba-ba21-cbeee0f10dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6825","nonce":373412752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6827","nonce":373412752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6831","nonce":373412752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6829","nonce":373412752}]},"public_addr":"172.21.15.64:6825/373412752","cluster_addr":"172.21.15.64:6827/373412752","heartbeat_back_addr":"172.21.15.64:6831/373412752","heartbeat_front_addr":"172.21.15.64:6829/373412752","state":["exists","up"]},{"osd":11,"uuid":"cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6825","nonce":1918889194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6827","nonce":1918889194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6831","nonce":1918889194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6829","nonce":1918889194}]},"public_addr":"172.21.15.157:6825/1918889194","cluster_addr":"172.21.15.157:6827/1918889194","heartbeat_back_addr":"172.21.15.157:6831/1918889194","heartbeat_front_addr":"172.21.15.157:6829/1918889194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:31.378218+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:56.794167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:20.790267+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:47.039106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:11.483842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:38.151466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:03.553308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:27.865321+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:53.086530+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:18.946661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:43.697243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:12:08.936588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/422860013":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/2837500116":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/1845420423":"2024-09-20T08:15:01.165466+0000","172.21.15.22:6800/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/3623845502":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2851750481":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2133059277":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/384027596":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/586163474":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/3712426751":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/1866276413":"2024-09-20T08:15:01.165466+0000","172.21.15.22:6801/1866276413":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/155709052":"2024-09-20T08:15:01.165466+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:15:09.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.377+0000 7f3e84a28640 1 -- 172.21.15.22:0/3364220680 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3e680a3550 msgr2=0x7f3e680a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:09.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.377+0000 7f3e84a28640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3e680a3550 0x7f3e680a5a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e800fe670 tx=0x7f3e74005a20 comp rx=0 tx=0).stop 2024-09-19T08:15:09.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.377+0000 7f3e84a28640 1 -- 172.21.15.22:0/3364220680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80100e20 msgr2=0x7f3e800fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:09.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.377+0000 7f3e84a28640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80100e20 0x7f3e800fcc80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000d950 tx=0x7f3e7000de10 comp rx=0 tx=0).stop 2024-09-19T08:15:09.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.377+0000 7f3e84a28640 1 -- 172.21.15.22:0/3364220680 shutdown_connections 2024-09-19T08:15:09.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.377+0000 7f3e84a28640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3e680a3550 0x7f3e680a5a10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:09.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.377+0000 7f3e84a28640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e80102ef0 0x7f3e800fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.377+0000 7f3e84a28640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e80102570 0x7f3e800fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.377+0000 7f3e84a28640 1 --2- 172.21.15.22:0/3364220680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e80100e20 0x7f3e800fcc80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.377+0000 7f3e84a28640 1 -- 172.21.15.22:0/3364220680 >> 172.21.15.22:0/3364220680 conn(0x7f3e800fc7d0 msgr2=0x7f3e8006a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.377+0000 7f3e84a28640 1 -- 172.21.15.22:0/3364220680 shutdown_connections 2024-09-19T08:15:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:09.377+0000 7f3e84a28640 1 -- 172.21.15.22:0/3364220680 wait complete. 2024-09-19T08:15:10.111 DEBUG:tasks.fwd_scrub:Yielding 2024-09-19T08:15:10.111 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T08:15:10.120 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T08:15:10.120 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-09-19T08:15:10.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-19T08:15:10.121 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-19T08:15:10.121 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:15:10.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:15:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.229+0000 7f6dde352640 1 -- 172.21.15.22:0/2550202061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd8102d20 msgr2=0x7f6dd8103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.229+0000 7f6dde352640 1 --2- 172.21.15.22:0/2550202061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd8102d20 0x7f6dd8103100 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009900 tx=0x7f6dc802a480 comp rx=0 tx=0).stop 2024-09-19T08:15:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.229+0000 7f6dde352640 1 -- 172.21.15.22:0/2550202061 shutdown_connections 2024-09-19T08:15:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.229+0000 7f6dde352640 1 --2- 172.21.15.22:0/2550202061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dd810c0f0 0x7f6dd810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.229+0000 7f6dde352640 1 --2- 172.21.15.22:0/2550202061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd8103640 0x7f6dd810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.229+0000 7f6dde352640 1 --2- 172.21.15.22:0/2550202061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd8102d20 0x7f6dd8103100 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.229+0000 7f6dde352640 1 -- 172.21.15.22:0/2550202061 >> 172.21.15.22:0/2550202061 conn(0x7f6dd8077990 msgr2=0x7f6dd80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.230+0000 7f6dde352640 1 -- 172.21.15.22:0/2550202061 shutdown_connections 2024-09-19T08:15:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.230+0000 7f6dde352640 1 -- 172.21.15.22:0/2550202061 wait complete. 2024-09-19T08:15:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.230+0000 7f6dde352640 1 Processor -- start 2024-09-19T08:15:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.230+0000 7f6dde352640 1 -- start start 2024-09-19T08:15:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dde352640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd8102d20 0x7f6dd8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dde352640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dd8103640 0x7f6dd8198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dde352640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd810c0f0 0x7f6dd819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dde352640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd8111f10 con 0x7f6dd8102d20 2024-09-19T08:15:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dde352640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd8111d90 con 0x7f6dd810c0f0 2024-09-19T08:15:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dde352640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd8112090 con 0x7f6dd8103640 2024-09-19T08:15:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dd7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd8102d20 0x7f6dd8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dd7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd8102d20 0x7f6dd8198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42706/0 (socket says 172.21.15.22:42706) 2024-09-19T08:15:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dd7fff640 1 -- 172.21.15.22:0/3549112373 learned_addr learned my addr 172.21.15.22:0/3549112373 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6ddc8ef640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd810c0f0 0x7f6dd819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dd77fe640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dd8103640 0x7f6dd8198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dd7fff640 1 -- 172.21.15.22:0/3549112373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dd8103640 msgr2=0x7f6dd8198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dd7fff640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dd8103640 0x7f6dd8198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dd7fff640 1 -- 172.21.15.22:0/3549112373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd810c0f0 msgr2=0x7f6dd819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dd7fff640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd810c0f0 0x7f6dd819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dd7fff640 1 -- 172.21.15.22:0/3549112373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd819d2f0 con 0x7f6dd8102d20 2024-09-19T08:15:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6ddc8ef640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd810c0f0 0x7f6dd819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.231+0000 7f6dd7fff640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd8102d20 0x7f6dd8198300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8004770 tx=0x7f6dc8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:10.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.236+0000 7f6dd57fa640 1 -- 172.21.15.22:0/3549112373 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc803f070 con 0x7f6dd8102d20 2024-09-19T08:15:10.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.236+0000 7f6dde352640 1 -- 172.21.15.22:0/3549112373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd819d580 con 0x7f6dd8102d20 2024-09-19T08:15:10.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.236+0000 7f6dd57fa640 1 -- 172.21.15.22:0/3549112373 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dc8031ca0 con 0x7f6dd8102d20 2024-09-19T08:15:10.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.236+0000 7f6dd57fa640 1 -- 172.21.15.22:0/3549112373 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8035030 con 0x7f6dd8102d20 2024-09-19T08:15:10.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.236+0000 7f6dde352640 1 -- 172.21.15.22:0/3549112373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd81a4e70 con 0x7f6dd8102d20 2024-09-19T08:15:10.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.238+0000 7f6dde352640 1 -- 172.21.15.22:0/3549112373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd8103100 con 0x7f6dd8102d20 2024-09-19T08:15:10.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.239+0000 7f6dd57fa640 1 -- 172.21.15.22:0/3549112373 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f6dc8042070 con 0x7f6dd8102d20 2024-09-19T08:15:10.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.239+0000 7f6dd57fa640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6dc00a3580 0x7f6dc00a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.239+0000 7f6dd57fa640 1 -- 172.21.15.22:0/3549112373 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6dc80f0570 con 0x7f6dd8102d20 2024-09-19T08:15:10.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.239+0000 7f6dd77fe640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6dc00a3580 0x7f6dc00a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.240+0000 7f6dd77fe640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6dc00a3580 0x7f6dc00a5a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80ffe00 tx=0x7f6dcc005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:10.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.248+0000 7f6dd57fa640 1 -- 172.21.15.22:0/3549112373 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6dc80bc790 con 0x7f6dd8102d20 2024-09-19T08:15:10.410 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:10.435 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:10 smithi022 ceph-mon[30963]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1 KiB/s wr, 22 op/s 2024-09-19T08:15:10.435 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:10 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3364220680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:15:10.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.444+0000 7f6dde352640 1 -- 172.21.15.22:0/3549112373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f6dd81a5040 con 0x7f6dd8102d20 2024-09-19T08:15:10.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.448+0000 7f6dd57fa640 1 -- 172.21.15.22:0/3549112373 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7f6dc80c0a90 con 0x7f6dd8102d20 2024-09-19T08:15:10.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.450+0000 7f6dde352640 1 -- 172.21.15.22:0/3549112373 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6dc00a3580 msgr2=0x7f6dc00a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.450+0000 7f6dde352640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6dc00a3580 0x7f6dc00a5a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80ffe00 tx=0x7f6dcc005f90 comp rx=0 tx=0).stop 2024-09-19T08:15:10.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.450+0000 7f6dde352640 1 -- 172.21.15.22:0/3549112373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd8102d20 msgr2=0x7f6dd8198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.450+0000 7f6dde352640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd8102d20 0x7f6dd8198300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8004770 tx=0x7f6dc8003c10 comp rx=0 tx=0).stop 2024-09-19T08:15:10.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.450+0000 7f6dde352640 1 -- 172.21.15.22:0/3549112373 shutdown_connections 2024-09-19T08:15:10.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.450+0000 7f6dde352640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6dc00a3580 0x7f6dc00a5a40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.450+0000 7f6dde352640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dd810c0f0 0x7f6dd819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.450+0000 7f6dde352640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dd8103640 0x7f6dd8198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.450+0000 7f6dde352640 1 --2- 172.21.15.22:0/3549112373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd8102d20 0x7f6dd8198300 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.450+0000 7f6dde352640 1 -- 172.21.15.22:0/3549112373 >> 172.21.15.22:0/3549112373 conn(0x7f6dd8077990 msgr2=0x7f6dd80fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:10.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.450+0000 7f6dde352640 1 -- 172.21.15.22:0/3549112373 shutdown_connections 2024-09-19T08:15:10.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.450+0000 7f6dde352640 1 -- 172.21.15.22:0/3549112373 wait complete. 2024-09-19T08:15:10.463 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-19T08:15:10.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:10 smithi157 ceph-mon[35143]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1 KiB/s wr, 22 op/s 2024-09-19T08:15:10.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:10 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3364220680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:15:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.572+0000 7f6b4796d640 1 -- 172.21.15.22:0/2241804091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b40102cc0 msgr2=0x7f6b401030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.572+0000 7f6b4796d640 1 --2- 172.21.15.22:0/2241804091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b40102cc0 0x7f6b401030a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009900 tx=0x7f6b3c02a480 comp rx=0 tx=0).stop 2024-09-19T08:15:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.572+0000 7f6b4796d640 1 -- 172.21.15.22:0/2241804091 shutdown_connections 2024-09-19T08:15:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.572+0000 7f6b4796d640 1 --2- 172.21.15.22:0/2241804091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b4010c110 0x7f6b4010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.572+0000 7f6b4796d640 1 --2- 172.21.15.22:0/2241804091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b401035e0 0x7f6b4010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.572+0000 7f6b4796d640 1 --2- 172.21.15.22:0/2241804091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b40102cc0 0x7f6b401030a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.572+0000 7f6b4796d640 1 -- 172.21.15.22:0/2241804091 >> 172.21.15.22:0/2241804091 conn(0x7f6b400fc6b0 msgr2=0x7f6b400fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.573+0000 7f6b4796d640 1 -- 172.21.15.22:0/2241804091 shutdown_connections 2024-09-19T08:15:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.573+0000 7f6b4796d640 1 -- 172.21.15.22:0/2241804091 wait complete. 2024-09-19T08:15:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.573+0000 7f6b4796d640 1 Processor -- start 2024-09-19T08:15:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.573+0000 7f6b4796d640 1 -- start start 2024-09-19T08:15:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.573+0000 7f6b4796d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b40102cc0 0x7f6b40198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.573+0000 7f6b4796d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b401035e0 0x7f6b40198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.573+0000 7f6b4796d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b4010c110 0x7f6b4019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.573+0000 7f6b4796d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6b40111ef0 con 0x7f6b40102cc0 2024-09-19T08:15:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.573+0000 7f6b4796d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6b40111d70 con 0x7f6b4010c110 2024-09-19T08:15:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.573+0000 7f6b4796d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6b40112070 con 0x7f6b401035e0 2024-09-19T08:15:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.573+0000 7f6b45709640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b40102cc0 0x7f6b40198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.573+0000 7f6b45709640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b40102cc0 0x7f6b40198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42722/0 (socket says 172.21.15.22:42722) 2024-09-19T08:15:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.573+0000 7f6b45709640 1 -- 172.21.15.22:0/4224074455 learned_addr learned my addr 172.21.15.22:0/4224074455 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.574+0000 7f6b44f08640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b401035e0 0x7f6b40198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.574+0000 7f6b45f0a640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b4010c110 0x7f6b4019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.574+0000 7f6b45709640 1 -- 172.21.15.22:0/4224074455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b401035e0 msgr2=0x7f6b40198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.574+0000 7f6b45709640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b401035e0 0x7f6b40198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.574+0000 7f6b45709640 1 -- 172.21.15.22:0/4224074455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b4010c110 msgr2=0x7f6b4019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.574+0000 7f6b45709640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b4010c110 0x7f6b4019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.574+0000 7f6b45709640 1 -- 172.21.15.22:0/4224074455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4019d410 con 0x7f6b40102cc0 2024-09-19T08:15:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.574+0000 7f6b45f0a640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b4010c110 0x7f6b4019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.574+0000 7f6b45709640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b40102cc0 0x7f6b40198420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c006010 tx=0x7f6b3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.574+0000 7f6b367fc640 1 -- 172.21.15.22:0/4224074455 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c03f030 con 0x7f6b40102cc0 2024-09-19T08:15:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.574+0000 7f6b367fc640 1 -- 172.21.15.22:0/4224074455 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b3c003ce0 con 0x7f6b40102cc0 2024-09-19T08:15:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.574+0000 7f6b4796d640 1 -- 172.21.15.22:0/4224074455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4019d6a0 con 0x7f6b40102cc0 2024-09-19T08:15:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.574+0000 7f6b367fc640 1 -- 172.21.15.22:0/4224074455 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c035070 con 0x7f6b40102cc0 2024-09-19T08:15:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.574+0000 7f6b4796d640 1 -- 172.21.15.22:0/4224074455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b401a4f90 con 0x7f6b40102cc0 2024-09-19T08:15:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.575+0000 7f6b4796d640 1 -- 172.21.15.22:0/4224074455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b401030a0 con 0x7f6b40102cc0 2024-09-19T08:15:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.576+0000 7f6b367fc640 1 -- 172.21.15.22:0/4224074455 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f6b3c042030 con 0x7f6b40102cc0 2024-09-19T08:15:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.577+0000 7f6b367fc640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6b2c0a3600 0x7f6b2c0a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.577+0000 7f6b367fc640 1 -- 172.21.15.22:0/4224074455 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6b3c0f03b0 con 0x7f6b40102cc0 2024-09-19T08:15:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.577+0000 7f6b44f08640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6b2c0a3600 0x7f6b2c0a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.577+0000 7f6b44f08640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6b2c0a3600 0x7f6b2c0a5ac0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6b4010d4c0 tx=0x7f6b3000b2e0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:10.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.581+0000 7f6b367fc640 1 -- 172.21.15.22:0/4224074455 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6b3c0bcce0 con 0x7f6b40102cc0 2024-09-19T08:15:10.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:10 smithi064 ceph-mon[34900]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1 KiB/s wr, 22 op/s 2024-09-19T08:15:10.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:10 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3364220680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:15:10.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.765+0000 7f6b4796d640 1 -- 172.21.15.22:0/4224074455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f6b40107180 con 0x7f6b40102cc0 2024-09-19T08:15:10.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.769+0000 7f6b367fc640 1 -- 172.21.15.22:0/4224074455 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f6b3c0bc5e0 con 0x7f6b40102cc0 2024-09-19T08:15:10.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.771+0000 7f6b4796d640 1 -- 172.21.15.22:0/4224074455 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6b2c0a3600 msgr2=0x7f6b2c0a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.771+0000 7f6b4796d640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6b2c0a3600 0x7f6b2c0a5ac0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6b4010d4c0 tx=0x7f6b3000b2e0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.771+0000 7f6b4796d640 1 -- 172.21.15.22:0/4224074455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b40102cc0 msgr2=0x7f6b40198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.771+0000 7f6b4796d640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b40102cc0 0x7f6b40198420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c006010 tx=0x7f6b3c0042f0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.771+0000 7f6b4796d640 1 -- 172.21.15.22:0/4224074455 shutdown_connections 2024-09-19T08:15:10.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.771+0000 7f6b4796d640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6b2c0a3600 0x7f6b2c0a5ac0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.771+0000 7f6b4796d640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b4010c110 0x7f6b4019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.771+0000 7f6b4796d640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b401035e0 0x7f6b40198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.771+0000 7f6b4796d640 1 --2- 172.21.15.22:0/4224074455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b40102cc0 0x7f6b40198420 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.772+0000 7f6b4796d640 1 -- 172.21.15.22:0/4224074455 >> 172.21.15.22:0/4224074455 conn(0x7f6b400fc6b0 msgr2=0x7f6b400fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:10.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.772+0000 7f6b4796d640 1 -- 172.21.15.22:0/4224074455 shutdown_connections 2024-09-19T08:15:10.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.772+0000 7f6b4796d640 1 -- 172.21.15.22:0/4224074455 wait complete. 2024-09-19T08:15:10.784 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-19T08:15:10.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.900+0000 7fa3c7da1640 1 -- 172.21.15.22:0/1188289682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3c0101ad0 msgr2=0x7fa3c0101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.900+0000 7fa3c7da1640 1 --2- 172.21.15.22:0/1188289682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3c0101ad0 0x7fa3c0101f30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009930 tx=0x7fa3b002a420 comp rx=0 tx=0).stop 2024-09-19T08:15:10.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.900+0000 7fa3c7da1640 1 -- 172.21.15.22:0/1188289682 shutdown_connections 2024-09-19T08:15:10.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.900+0000 7fa3c7da1640 1 --2- 172.21.15.22:0/1188289682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c0102470 0x7fa3c010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.900+0000 7fa3c7da1640 1 --2- 172.21.15.22:0/1188289682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3c0101ad0 0x7fa3c0101f30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.900+0000 7fa3c7da1640 1 --2- 172.21.15.22:0/1188289682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c0107b70 0x7fa3c0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.900+0000 7fa3c7da1640 1 -- 172.21.15.22:0/1188289682 >> 172.21.15.22:0/1188289682 conn(0x7fa3c00fd7d0 msgr2=0x7fa3c00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:10.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.900+0000 7fa3c7da1640 1 -- 172.21.15.22:0/1188289682 shutdown_connections 2024-09-19T08:15:10.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.900+0000 7fa3c7da1640 1 -- 172.21.15.22:0/1188289682 wait complete. 2024-09-19T08:15:10.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c7da1640 1 Processor -- start 2024-09-19T08:15:10.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c7da1640 1 -- start start 2024-09-19T08:15:10.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c7da1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3c0101ad0 0x7fa3c019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c7da1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c0102470 0x7fa3c019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c7da1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c0107b70 0x7fa3c019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c7da1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0111f20 con 0x7fa3c0101ad0 2024-09-19T08:15:10.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c7da1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0111da0 con 0x7fa3c0107b70 2024-09-19T08:15:10.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c7da1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c01120a0 con 0x7fa3c0102470 2024-09-19T08:15:10.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c5b3d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3c0101ad0 0x7fa3c019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c5b3d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3c0101ad0 0x7fa3c019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42748/0 (socket says 172.21.15.22:42748) 2024-09-19T08:15:10.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c5b3d640 1 -- 172.21.15.22:0/3258662420 learned_addr learned my addr 172.21.15.22:0/3258662420 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:10.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c533c640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c0102470 0x7fa3c019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c633e640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c0107b70 0x7fa3c019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c5b3d640 1 -- 172.21.15.22:0/3258662420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c0102470 msgr2=0x7fa3c019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c5b3d640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c0102470 0x7fa3c019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c5b3d640 1 -- 172.21.15.22:0/3258662420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c0107b70 msgr2=0x7fa3c019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c5b3d640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c0107b70 0x7fa3c019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c5b3d640 1 -- 172.21.15.22:0/3258662420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c019f530 con 0x7fa3c0101ad0 2024-09-19T08:15:10.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.901+0000 7fa3c633e640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c0107b70 0x7fa3c019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:10.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.902+0000 7fa3c5b3d640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3c0101ad0 0x7fa3c019a540 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00d950 tx=0x7fa3bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:10.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.902+0000 7fa3b6ffd640 1 -- 172.21.15.22:0/3258662420 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc017030 con 0x7fa3c0101ad0 2024-09-19T08:15:10.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.902+0000 7fa3b6ffd640 1 -- 172.21.15.22:0/3258662420 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3bc004020 con 0x7fa3c0101ad0 2024-09-19T08:15:10.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.902+0000 7fa3c7da1640 1 -- 172.21.15.22:0/3258662420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c019f820 con 0x7fa3c0101ad0 2024-09-19T08:15:10.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.902+0000 7fa3b6ffd640 1 -- 172.21.15.22:0/3258662420 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc0047a0 con 0x7fa3c0101ad0 2024-09-19T08:15:10.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.902+0000 7fa3c7da1640 1 -- 172.21.15.22:0/3258662420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c01a70b0 con 0x7fa3c0101ad0 2024-09-19T08:15:10.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.903+0000 7fa3c7da1640 1 -- 172.21.15.22:0/3258662420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c0101f30 con 0x7fa3c0101ad0 2024-09-19T08:15:10.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.905+0000 7fa3b6ffd640 1 -- 172.21.15.22:0/3258662420 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fa3bc0041d0 con 0x7fa3c0101ad0 2024-09-19T08:15:10.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.906+0000 7fa3b6ffd640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3ac0a3600 0x7fa3ac0a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.906+0000 7fa3c533c640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3ac0a3600 0x7fa3ac0a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.906+0000 7fa3b6ffd640 1 -- 172.21.15.22:0/3258662420 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa3bc0cff00 con 0x7fa3c0101ad0 2024-09-19T08:15:10.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.907+0000 7fa3c533c640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3ac0a3600 0x7fa3ac0a5ac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009900 tx=0x7fa3b00058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:10.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.911+0000 7fa3b6ffd640 1 -- 172.21.15.22:0/3258662420 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa3bc09c720 con 0x7fa3c0101ad0 2024-09-19T08:15:10.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.923+0000 7f8585023640 1 -- 172.21.15.22:0/3785466947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f858010c180 msgr2=0x7f858010e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.923+0000 7f8585023640 1 --2- 172.21.15.22:0/3785466947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f858010c180 0x7f858010e620 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8578009ca0 tx=0x7f857802a950 comp rx=0 tx=0).stop 2024-09-19T08:15:10.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.923+0000 7f8585023640 1 -- 172.21.15.22:0/3785466947 shutdown_connections 2024-09-19T08:15:10.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.923+0000 7f8585023640 1 --2- 172.21.15.22:0/3785466947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f858010c180 0x7f858010e620 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.923+0000 7f8585023640 1 --2- 172.21.15.22:0/3785466947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8580103670 0x7f858010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.923+0000 7f8585023640 1 --2- 172.21.15.22:0/3785466947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8580102d50 0x7f8580103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.923+0000 7f8585023640 1 -- 172.21.15.22:0/3785466947 >> 172.21.15.22:0/3785466947 conn(0x7f85800fc7d0 msgr2=0x7f85800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:10.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.923+0000 7f8585023640 1 -- 172.21.15.22:0/3785466947 shutdown_connections 2024-09-19T08:15:10.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.923+0000 7f8585023640 1 -- 172.21.15.22:0/3785466947 wait complete. 2024-09-19T08:15:10.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.924+0000 7f8585023640 1 Processor -- start 2024-09-19T08:15:10.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.924+0000 7f8585023640 1 -- start start 2024-09-19T08:15:10.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f8585023640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8580102d50 0x7f85801009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f8585023640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8580103670 0x7f8580100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f8585023640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858010c180 0x7f8580101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f8585023640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8580111f60 con 0x7f8580102d50 2024-09-19T08:15:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f8585023640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8580111de0 con 0x7f8580103670 2024-09-19T08:15:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f8585023640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85801120e0 con 0x7f858010c180 2024-09-19T08:15:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f857ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8580102d50 0x7f85801009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f857ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8580102d50 0x7f85801009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42764/0 (socket says 172.21.15.22:42764) 2024-09-19T08:15:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f857ed9d640 1 -- 172.21.15.22:0/1744838919 learned_addr learned my addr 172.21.15.22:0/1744838919 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f857659c640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8580103670 0x7f8580100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f857f59e640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858010c180 0x7f8580101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f857ed9d640 1 -- 172.21.15.22:0/1744838919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858010c180 msgr2=0x7f8580101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f857ed9d640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858010c180 0x7f8580101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f857ed9d640 1 -- 172.21.15.22:0/1744838919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8580103670 msgr2=0x7f8580100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f857ed9d640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8580103670 0x7f8580100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f857ed9d640 1 -- 172.21.15.22:0/1744838919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85801a68c0 con 0x7f8580102d50 2024-09-19T08:15:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f857659c640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8580103670 0x7f8580100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f857ed9d640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8580102d50 0x7f85801009b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f857000e9f0 tx=0x7f857000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.925+0000 7f857cd99640 1 -- 172.21.15.22:0/1744838919 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857000cd80 con 0x7f8580102d50 2024-09-19T08:15:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.926+0000 7f857cd99640 1 -- 172.21.15.22:0/1744838919 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8570010910 con 0x7f8580102d50 2024-09-19T08:15:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.926+0000 7f857cd99640 1 -- 172.21.15.22:0/1744838919 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570018990 con 0x7f8580102d50 2024-09-19T08:15:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.926+0000 7f8585023640 1 -- 172.21.15.22:0/1744838919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85801a6bb0 con 0x7f8580102d50 2024-09-19T08:15:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.926+0000 7f8585023640 1 -- 172.21.15.22:0/1744838919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85801a70f0 con 0x7f8580102d50 2024-09-19T08:15:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.928+0000 7f857cd99640 1 -- 172.21.15.22:0/1744838919 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f8570010ac0 con 0x7f8580102d50 2024-09-19T08:15:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f85873fc740 1 Processor -- start 2024-09-19T08:15:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f85873fc740 1 -- start start 2024-09-19T08:15:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f85873fc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437861b60 0x55f437861f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f85873fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437862480 0x55f43786c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f85873fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f43786f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f857cd99640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f856c0a34d0 0x7f856c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f85873fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f4377c7280 con 0x55f43786cd40 2024-09-19T08:15:10.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f85873fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f43776eb10 con 0x55f437862480 2024-09-19T08:15:10.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f85873fc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f4377c3240 con 0x55f437861b60 2024-09-19T08:15:10.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f857659c640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f856c0a34d0 0x7f856c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f857f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f43786f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f857cd99640 1 -- 172.21.15.22:0/1744838919 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f85700d02b0 con 0x7f8580102d50 2024-09-19T08:15:10.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f857f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f43786f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42774/0 (socket says 172.21.15.22:42774) 2024-09-19T08:15:10.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f857f59e640 1 -- 172.21.15.22:0/2245809686 learned_addr learned my addr 172.21.15.22:0/2245809686 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:10.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f857ed9d640 1 --2- 172.21.15.22:0/2245809686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437862480 0x55f43786c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.929+0000 7f857f59e640 1 --2- 172.21.15.22:0/2245809686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437861b60 0x55f437861f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.930+0000 7f857659c640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f856c0a34d0 0x7f856c0a5990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8580102440 tx=0x7f8568005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:10.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.930+0000 7f857f59e640 1 -- 172.21.15.22:0/2245809686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437861b60 msgr2=0x55f437861f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:10.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.930+0000 7f857f59e640 1 --2- 172.21.15.22:0/2245809686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437861b60 0x55f437861f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.930+0000 7f857f59e640 1 -- 172.21.15.22:0/2245809686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437862480 msgr2=0x55f43786c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:10.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.930+0000 7f857f59e640 1 --2- 172.21.15.22:0/2245809686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437862480 0x55f43786c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.930+0000 7f857f59e640 1 -- 172.21.15.22:0/2245809686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f43777b1f0 con 0x55f43786cd40 2024-09-19T08:15:10.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.930+0000 7f857f59e640 1 --2- 172.21.15.22:0/2245809686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f43786f130 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8578009ca0 tx=0x7f8578004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4291a565c72030 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:10.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.930+0000 7f857559a640 1 -- 172.21.15.22:0/2245809686 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578043070 con 0x55f43786cd40 2024-09-19T08:15:10.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.930+0000 7f857559a640 1 -- 172.21.15.22:0/2245809686 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8578041030 con 0x55f43786cd40 2024-09-19T08:15:10.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.930+0000 7f857559a640 1 -- 172.21.15.22:0/2245809686 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578034040 con 0x55f43786cd40 2024-09-19T08:15:10.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.931+0000 7f85873fc740 1 -- 172.21.15.22:0/2245809686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 msgr2=0x55f43786f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.931+0000 7f85873fc740 1 --2- 172.21.15.22:0/2245809686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f43786f130 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8578009ca0 tx=0x7f8578004a60 comp rx=0 tx=0).stop 2024-09-19T08:15:10.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.931+0000 7f85873fc740 1 -- 172.21.15.22:0/2245809686 shutdown_connections 2024-09-19T08:15:10.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.931+0000 7f85873fc740 1 --2- 172.21.15.22:0/2245809686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f43786f130 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.931+0000 7f85873fc740 1 --2- 172.21.15.22:0/2245809686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437862480 0x55f43786c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.931+0000 7f85873fc740 1 --2- 172.21.15.22:0/2245809686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437861b60 0x55f437861f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.931+0000 7f85873fc740 1 -- 172.21.15.22:0/2245809686 >> 172.21.15.22:0/2245809686 conn(0x55f43785e0f0 msgr2=0x55f43785e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:10.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.931+0000 7f85873fc740 1 -- 172.21.15.22:0/2245809686 shutdown_connections 2024-09-19T08:15:10.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.931+0000 7f85873fc740 1 -- 172.21.15.22:0/2245809686 wait complete. 2024-09-19T08:15:10.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.932+0000 7f85873fc740 1 -- start start 2024-09-19T08:15:10.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.932+0000 7f85873fc740 1 Processor -- start 2024-09-19T08:15:10.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.932+0000 7f85873fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437861b60 0x55f437869bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.932+0000 7f85873fc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437862480 0x55f43786a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.932+0000 7f85873fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f43786a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.932+0000 7f85873fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f43786f910 con 0x55f43786cd40 2024-09-19T08:15:10.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.932+0000 7f85873fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f437781f10 con 0x55f437861b60 2024-09-19T08:15:10.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.932+0000 7f85873fc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f43786fa90 con 0x55f437862480 2024-09-19T08:15:10.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.932+0000 7f85873fc740 10 client.0 _ll_register_callbacks cb 0x55f43774aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.932+0000 7f857f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f43786a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.932+0000 7f857f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f43786a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42786/0 (socket says 172.21.15.22:42786) 2024-09-19T08:15:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.932+0000 7f857f59e640 1 -- 172.21.15.22:0/1766741634 learned_addr learned my addr 172.21.15.22:0/1766741634 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.932+0000 7f857659c640 1 --2- 172.21.15.22:0/1766741634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437862480 0x55f43786a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.932+0000 7f857ed9d640 1 --2- 172.21.15.22:0/1766741634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437861b60 0x55f437869bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.933+0000 7f857f59e640 1 -- 172.21.15.22:0/1766741634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437862480 msgr2=0x55f43786a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.933+0000 7f857f59e640 1 --2- 172.21.15.22:0/1766741634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437862480 0x55f43786a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.933+0000 7f857f59e640 1 -- 172.21.15.22:0/1766741634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437861b60 msgr2=0x55f437869bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.933+0000 7f857f59e640 1 --2- 172.21.15.22:0/1766741634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437861b60 0x55f437869bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.933+0000 7f857f59e640 1 -- 172.21.15.22:0/1766741634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4375386a0 con 0x55f43786cd40 2024-09-19T08:15:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.933+0000 7f857659c640 1 --2- 172.21.15.22:0/1766741634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437862480 0x55f43786a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.933+0000 7f857f59e640 1 --2- 172.21.15.22:0/1766741634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f43786a670 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8578004ad0 tx=0x7f85780095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e220e0e8330c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.933+0000 7f85667fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.933+0000 7f85667fc640 1 -- 172.21.15.22:0/1766741634 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578051070 con 0x55f43786cd40 2024-09-19T08:15:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.933+0000 7f85667fc640 1 -- 172.21.15.22:0/1766741634 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8578003b70 con 0x55f43786cd40 2024-09-19T08:15:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.933+0000 7f85873fc740 1 -- 172.21.15.22:0/1766741634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8578041030 con 0x55f43786cd40 2024-09-19T08:15:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.933+0000 7f85667fc640 1 -- 172.21.15.22:0/1766741634 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8578034030 con 0x55f43786cd40 2024-09-19T08:15:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.933+0000 7f85873fc740 10 client.15300 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.933+0000 7f85873fc740 1 -- 172.21.15.22:0/1766741634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f437883cf0 con 0x55f43786cd40 2024-09-19T08:15:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.934+0000 7f85667fc640 1 -- 172.21.15.22:0/1766741634 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8578041030 con 0x55f43786cd40 2024-09-19T08:15:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.934+0000 7f85667fc640 1 -- 172.21.15.22:0/1766741634 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8578005020 con 0x55f43786cd40 2024-09-19T08:15:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.934+0000 7f85873fc740 10 client.15300 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.934+0000 7f85873fc740 10 client.15300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:15:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.934+0000 7f85873fc740 20 client.15300 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.934+0000 7f85873fc740 1 --2- 172.21.15.22:0/1766741634 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f437885040 0x55f437887430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.934+0000 7f85873fc740 4 client.15300 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:15:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.934+0000 7f85873fc740 1 -- 172.21.15.22:0/1766741634 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4377d47f0 con 0x55f437885040 2024-09-19T08:15:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.934+0000 7f857ed9d640 1 --2- 172.21.15.22:0/1766741634 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f437885040 0x55f437887430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.935+0000 7f85667fc640 10 client.15300 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.934+0000 7f857ed9d640 1 --2- 172.21.15.22:0/1766741634 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f437885040 0x55f437887430 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75ace8187ec16188 server_cookie=7c197b7fe991fa8a in_seq=0 out_seq=0 2024-09-19T08:15:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.935+0000 7f85667fc640 1 -- 172.21.15.22:0/1766741634 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4377d47f0 con 0x55f437885040 2024-09-19T08:15:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.935+0000 7f85667fc640 10 client.15300 handle_command_reply: tid=0 2024-09-19T08:15:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.935+0000 7f85873fc740 1 client.15300 shutdown 2024-09-19T08:15:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.935+0000 7f85873fc740 1 -- 172.21.15.22:0/1766741634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 msgr2=0x55f43786a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.935+0000 7f85873fc740 1 --2- 172.21.15.22:0/1766741634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f43786a670 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8578004ad0 tx=0x7f85780095c0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.935+0000 7f85873fc740 1 -- 172.21.15.22:0/1766741634 shutdown_connections 2024-09-19T08:15:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.935+0000 7f85873fc740 1 --2- 172.21.15.22:0/1766741634 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f437885040 0x55f437887430 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.935+0000 7f85873fc740 1 --2- 172.21.15.22:0/1766741634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f43786a670 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.935+0000 7f85873fc740 1 --2- 172.21.15.22:0/1766741634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437862480 0x55f43786a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.935+0000 7f85873fc740 1 --2- 172.21.15.22:0/1766741634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437861b60 0x55f437869bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.935+0000 7f85873fc740 1 -- 172.21.15.22:0/1766741634 >> 172.21.15.22:0/1766741634 conn(0x55f43785e0f0 msgr2=0x55f437863dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.935+0000 7f85667fc640 0 client.15300 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.936+0000 7f85873fc740 1 -- 172.21.15.22:0/1766741634 shutdown_connections 2024-09-19T08:15:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.936+0000 7f85873fc740 1 -- 172.21.15.22:0/1766741634 wait complete. 2024-09-19T08:15:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.936+0000 7f85873fc740 20 client.15300 trim_cache size 0 max 16384 2024-09-19T08:15:10.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.972+0000 7f85873fc740 1 Processor -- start 2024-09-19T08:15:10.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.972+0000 7f85873fc740 1 -- start start 2024-09-19T08:15:10.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f85873fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f437861b60 0x55f437863cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f85873fc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437862480 0x55f43788a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f85873fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f43786cd40 0x55f43788a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f85873fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f85801adf80 con 0x55f437861b60 2024-09-19T08:15:10.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f85873fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f85801ab850 con 0x55f43786cd40 2024-09-19T08:15:10.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f85873fc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85801a96a0 con 0x55f437862480 2024-09-19T08:15:10.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f857ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f437861b60 0x55f437863cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f857ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f437861b60 0x55f437863cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42802/0 (socket says 172.21.15.22:42802) 2024-09-19T08:15:10.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f857ed9d640 1 -- 172.21.15.22:0/2801848160 learned_addr learned my addr 172.21.15.22:0/2801848160 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:10.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f857659c640 1 --2- 172.21.15.22:0/2801848160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437862480 0x55f43788a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f857f59e640 1 --2- 172.21.15.22:0/2801848160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f43786cd40 0x55f43788a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f857ed9d640 1 -- 172.21.15.22:0/2801848160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437862480 msgr2=0x55f43788a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:10.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f857ed9d640 1 --2- 172.21.15.22:0/2801848160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437862480 0x55f43788a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f857ed9d640 1 -- 172.21.15.22:0/2801848160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f43786cd40 msgr2=0x55f43788a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:10.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f857ed9d640 1 --2- 172.21.15.22:0/2801848160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f43786cd40 0x55f43788a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.973+0000 7f857ed9d640 1 -- 172.21.15.22:0/2801848160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f437859830 con 0x55f437861b60 2024-09-19T08:15:10.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.974+0000 7f857ed9d640 1 --2- 172.21.15.22:0/2801848160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f437861b60 0x55f437863cc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f85700ce010 tx=0x7f8570012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c0fa3f1dd8fa6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:10.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.974+0000 7f8574d99640 1 -- 172.21.15.22:0/2801848160 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85700a0700 con 0x55f437861b60 2024-09-19T08:15:10.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.974+0000 7f8574d99640 1 -- 172.21.15.22:0/2801848160 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85700a4fc0 con 0x55f437861b60 2024-09-19T08:15:10.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.974+0000 7f8574d99640 1 -- 172.21.15.22:0/2801848160 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570098a70 con 0x55f437861b60 2024-09-19T08:15:10.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.975+0000 7f85873fc740 1 -- 172.21.15.22:0/2801848160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f437861b60 msgr2=0x55f437863cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.975+0000 7f85873fc740 1 --2- 172.21.15.22:0/2801848160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f437861b60 0x55f437863cc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f85700ce010 tx=0x7f8570012040 comp rx=0 tx=0).stop 2024-09-19T08:15:10.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.975+0000 7f85873fc740 1 -- 172.21.15.22:0/2801848160 shutdown_connections 2024-09-19T08:15:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.975+0000 7f85873fc740 1 --2- 172.21.15.22:0/2801848160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f43786cd40 0x55f43788a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.975+0000 7f85873fc740 1 --2- 172.21.15.22:0/2801848160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437862480 0x55f43788a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.975+0000 7f85873fc740 1 --2- 172.21.15.22:0/2801848160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f437861b60 0x55f437863cc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.975+0000 7f85873fc740 1 -- 172.21.15.22:0/2801848160 >> 172.21.15.22:0/2801848160 conn(0x55f43785e0f0 msgr2=0x55f4378635b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.975+0000 7f85873fc740 1 -- 172.21.15.22:0/2801848160 shutdown_connections 2024-09-19T08:15:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.975+0000 7f85873fc740 1 -- 172.21.15.22:0/2801848160 wait complete. 2024-09-19T08:15:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.975+0000 7f85873fc740 1 -- start start 2024-09-19T08:15:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.975+0000 7f85873fc740 1 Processor -- start 2024-09-19T08:15:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f85873fc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437861b60 0x55f437887e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f85873fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437862480 0x55f437888340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f85873fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f437888880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f85873fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f85801a9d10 con 0x55f43786cd40 2024-09-19T08:15:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f85873fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f85801ab180 con 0x55f437862480 2024-09-19T08:15:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f85873fc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85801aa3e0 con 0x55f437861b60 2024-09-19T08:15:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f85873fc740 10 client.0 _ll_register_callbacks cb 0x7f8570009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f857ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437861b60 0x55f437887e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f857ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437861b60 0x55f437887e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:47146/0 (socket says 172.21.15.22:47146) 2024-09-19T08:15:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f857ed9d640 1 -- 172.21.15.22:0/214504624 learned_addr learned my addr 172.21.15.22:0/214504624 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f857f59e640 1 --2- 172.21.15.22:0/214504624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f437888880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f857659c640 1 --2- 172.21.15.22:0/214504624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437862480 0x55f437888340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f857ed9d640 1 -- 172.21.15.22:0/214504624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437862480 msgr2=0x55f437888340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f857ed9d640 1 --2- 172.21.15.22:0/214504624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437862480 0x55f437888340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f857ed9d640 1 -- 172.21.15.22:0/214504624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 msgr2=0x55f437888880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f857ed9d640 1 --2- 172.21.15.22:0/214504624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f437888880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f857ed9d640 1 -- 172.21.15.22:0/214504624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4378ee980 con 0x55f437861b60 2024-09-19T08:15:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f857f59e640 1 --2- 172.21.15.22:0/214504624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f437888880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f857ed9d640 1 --2- 172.21.15.22:0/214504624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437861b60 0x55f437887e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f85700a4a90 tx=0x7f857009c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a97c1afb826b00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.976+0000 7f85667fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:15:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.977+0000 7f85667fc640 1 -- 172.21.15.22:0/214504624 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85700ba060 con 0x55f437861b60 2024-09-19T08:15:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.977+0000 7f85667fc640 1 -- 172.21.15.22:0/214504624 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f857009d360 con 0x55f437861b60 2024-09-19T08:15:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.977+0000 7f85873fc740 1 -- 172.21.15.22:0/214504624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85700a4fc0 con 0x55f437861b60 2024-09-19T08:15:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.977+0000 7f85667fc640 1 -- 172.21.15.22:0/214504624 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f85700ac050 con 0x55f437861b60 2024-09-19T08:15:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.977+0000 7f85873fc740 10 client.24967 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.977+0000 7f85873fc740 1 -- 172.21.15.22:0/214504624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f4378efca0 con 0x55f437861b60 2024-09-19T08:15:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.977+0000 7f85667fc640 1 -- 172.21.15.22:0/214504624 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85700a4fc0 con 0x55f437861b60 2024-09-19T08:15:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.977+0000 7f85667fc640 1 -- 172.21.15.22:0/214504624 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f857009b850 con 0x55f437861b60 2024-09-19T08:15:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.977+0000 7f85873fc740 10 client.24967 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.977+0000 7f85873fc740 10 client.24967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:15:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.977+0000 7f85873fc740 20 client.24967 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.977+0000 7f85873fc740 1 --2- 172.21.15.22:0/214504624 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f4378f0260 0x55f4378f2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.977+0000 7f85873fc740 4 client.24967 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:15:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.977+0000 7f85873fc740 1 -- 172.21.15.22:0/214504624 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f4377ced00 con 0x55f4378f0260 2024-09-19T08:15:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.978+0000 7f857659c640 1 --2- 172.21.15.22:0/214504624 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f4378f0260 0x55f4378f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.978+0000 7f857659c640 1 --2- 172.21.15.22:0/214504624 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f4378f0260 0x55f4378f2650 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e12e3ac3c2f8e8 server_cookie=68a4104133bfa95e in_seq=0 out_seq=0 2024-09-19T08:15:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.978+0000 7f85667fc640 10 client.24967 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.979+0000 7f85667fc640 1 -- 172.21.15.22:0/214504624 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f4377ced00 con 0x55f4378f0260 2024-09-19T08:15:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.979+0000 7f85667fc640 10 client.24967 handle_command_reply: tid=0 2024-09-19T08:15:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.979+0000 7f85873fc740 1 client.24967 shutdown 2024-09-19T08:15:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.979+0000 7f85873fc740 1 -- 172.21.15.22:0/214504624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437861b60 msgr2=0x55f437887e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.979+0000 7f85873fc740 1 --2- 172.21.15.22:0/214504624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437861b60 0x55f437887e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f85700a4a90 tx=0x7f857009c960 comp rx=0 tx=0).stop 2024-09-19T08:15:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.980+0000 7f85873fc740 1 -- 172.21.15.22:0/214504624 shutdown_connections 2024-09-19T08:15:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.980+0000 7f85873fc740 1 --2- 172.21.15.22:0/214504624 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f4378f0260 0x55f4378f2650 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.980+0000 7f85873fc740 1 --2- 172.21.15.22:0/214504624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f43786cd40 0x55f437888880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.980+0000 7f85667fc640 0 client.24967 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.980+0000 7f85873fc740 1 --2- 172.21.15.22:0/214504624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f437862480 0x55f437888340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.980+0000 7f85873fc740 1 --2- 172.21.15.22:0/214504624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f437861b60 0x55f437887e00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.980+0000 7f85873fc740 1 -- 172.21.15.22:0/214504624 >> 172.21.15.22:0/214504624 conn(0x55f43785e0f0 msgr2=0x55f43785d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.980+0000 7f85873fc740 1 -- 172.21.15.22:0/214504624 shutdown_connections 2024-09-19T08:15:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.980+0000 7f85873fc740 1 -- 172.21.15.22:0/214504624 wait complete. 2024-09-19T08:15:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.980+0000 7f85873fc740 20 client.24967 trim_cache size 0 max 16384 2024-09-19T08:15:10.986 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:15:10.986 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-19T08:15:10.986 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "190afc4d-70c2-4bfa-9608-f13ca79efbf7", 2024-09-19T08:15:10.987 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-19T08:15:10.987 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:15:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.981+0000 7f8585023640 1 -- 172.21.15.22:0/1744838919 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f856c0a34d0 msgr2=0x7f856c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.981+0000 7f8585023640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f856c0a34d0 0x7f856c0a5990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8580102440 tx=0x7f8568005f90 comp rx=0 tx=0).stop 2024-09-19T08:15:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.981+0000 7f8585023640 1 -- 172.21.15.22:0/1744838919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8580102d50 msgr2=0x7f85801009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.981+0000 7f8585023640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8580102d50 0x7f85801009b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f857000e9f0 tx=0x7f857000eeb0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.981+0000 7f8585023640 1 -- 172.21.15.22:0/1744838919 shutdown_connections 2024-09-19T08:15:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.981+0000 7f8585023640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f856c0a34d0 0x7f856c0a5990 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.981+0000 7f8585023640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858010c180 0x7f8580101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.981+0000 7f8585023640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8580103670 0x7f8580100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.981+0000 7f8585023640 1 --2- 172.21.15.22:0/1744838919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8580102d50 0x7f85801009b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.981+0000 7f8585023640 1 -- 172.21.15.22:0/1744838919 >> 172.21.15.22:0/1744838919 conn(0x7f85800fc7d0 msgr2=0x7f85800fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.981+0000 7f8585023640 1 -- 172.21.15.22:0/1744838919 shutdown_connections 2024-09-19T08:15:10.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:10.981+0000 7f8585023640 1 -- 172.21.15.22:0/1744838919 wait complete. 2024-09-19T08:15:11.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.125+0000 7fa3c7da1640 1 -- 172.21.15.22:0/3258662420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fa3c019b890 con 0x7fa3c0101ad0 2024-09-19T08:15:11.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.128+0000 7fa3b6ffd640 1 -- 172.21.15.22:0/3258662420 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7fa3c019b890 con 0x7fa3c0101ad0 2024-09-19T08:15:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.130+0000 7fa3c7da1640 1 -- 172.21.15.22:0/3258662420 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3ac0a3600 msgr2=0x7fa3ac0a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.130+0000 7fa3c7da1640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3ac0a3600 0x7fa3ac0a5ac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009900 tx=0x7fa3b00058d0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.130+0000 7fa3c7da1640 1 -- 172.21.15.22:0/3258662420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3c0101ad0 msgr2=0x7fa3c019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.130+0000 7fa3c7da1640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3c0101ad0 0x7fa3c019a540 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00d950 tx=0x7fa3bc00de10 comp rx=0 tx=0).stop 2024-09-19T08:15:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.130+0000 7fa3c7da1640 1 -- 172.21.15.22:0/3258662420 shutdown_connections 2024-09-19T08:15:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.130+0000 7fa3c7da1640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3ac0a3600 0x7fa3ac0a5ac0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.130+0000 7fa3c7da1640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c0107b70 0x7fa3c019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.130+0000 7fa3c7da1640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c0102470 0x7fa3c019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.130+0000 7fa3c7da1640 1 --2- 172.21.15.22:0/3258662420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3c0101ad0 0x7fa3c019a540 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.131+0000 7fa3c7da1640 1 -- 172.21.15.22:0/3258662420 >> 172.21.15.22:0/3258662420 conn(0x7fa3c00fd7d0 msgr2=0x7fa3c00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.131+0000 7fa3c7da1640 1 -- 172.21.15.22:0/3258662420 shutdown_connections 2024-09-19T08:15:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.131+0000 7fa3c7da1640 1 -- 172.21.15.22:0/3258662420 wait complete. 2024-09-19T08:15:11.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-19T08:15:11.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.251+0000 7f6382cb7640 1 -- 172.21.15.22:0/1125839895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c1080f0 msgr2=0x7f637c10ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:11.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.251+0000 7f6382cb7640 1 --2- 172.21.15.22:0/1125839895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c1080f0 0x7f637c10ea40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6374009cd0 tx=0x7f637402a970 comp rx=0 tx=0).stop 2024-09-19T08:15:11.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.251+0000 7f6382cb7640 1 -- 172.21.15.22:0/1125839895 shutdown_connections 2024-09-19T08:15:11.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.251+0000 7f6382cb7640 1 --2- 172.21.15.22:0/1125839895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c1080f0 0x7f637c10ea40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.251+0000 7f6382cb7640 1 --2- 172.21.15.22:0/1125839895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f637c101150 0x7f637c107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.251+0000 7f6382cb7640 1 --2- 172.21.15.22:0/1125839895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f637c100830 0x7f637c100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.251+0000 7f6382cb7640 1 -- 172.21.15.22:0/1125839895 >> 172.21.15.22:0/1125839895 conn(0x7f637c075ec0 msgr2=0x7f637c0762d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:11.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.251+0000 7f6382cb7640 1 -- 172.21.15.22:0/1125839895 shutdown_connections 2024-09-19T08:15:11.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.251+0000 7f6382cb7640 1 -- 172.21.15.22:0/1125839895 wait complete. 2024-09-19T08:15:11.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6382cb7640 1 Processor -- start 2024-09-19T08:15:11.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6382cb7640 1 -- start start 2024-09-19T08:15:11.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6382cb7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c100830 0x7f637c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:11.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6382cb7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f637c101150 0x7f637c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:11.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6382cb7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f637c1080f0 0x7f637c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:11.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6382cb7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f637c111f20 con 0x7f637c100830 2024-09-19T08:15:11.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6382cb7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f637c111da0 con 0x7f637c101150 2024-09-19T08:15:11.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6382cb7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f637c1120a0 con 0x7f637c1080f0 2024-09-19T08:15:11.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6380a53640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c100830 0x7f637c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:11.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6380a53640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c100830 0x7f637c198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42828/0 (socket says 172.21.15.22:42828) 2024-09-19T08:15:11.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6380a53640 1 -- 172.21.15.22:0/718722872 learned_addr learned my addr 172.21.15.22:0/718722872 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:11.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6381254640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f637c1080f0 0x7f637c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:11.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f637bfff640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f637c101150 0x7f637c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:11.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6380a53640 1 -- 172.21.15.22:0/718722872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f637c1080f0 msgr2=0x7f637c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:11.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6380a53640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f637c1080f0 0x7f637c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6380a53640 1 -- 172.21.15.22:0/718722872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f637c101150 msgr2=0x7f637c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:11.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6380a53640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f637c101150 0x7f637c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6380a53640 1 -- 172.21.15.22:0/718722872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637c19d340 con 0x7f637c100830 2024-09-19T08:15:11.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f6381254640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f637c1080f0 0x7f637c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:11.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.252+0000 7f637bfff640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f637c101150 0x7f637c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:11.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.253+0000 7f6380a53640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c100830 0x7f637c198350 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f636c00e9f0 tx=0x7f636c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:11.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.253+0000 7f6379ffb640 1 -- 172.21.15.22:0/718722872 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c00cd80 con 0x7f637c100830 2024-09-19T08:15:11.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.253+0000 7f6379ffb640 1 -- 172.21.15.22:0/718722872 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f636c005500 con 0x7f637c100830 2024-09-19T08:15:11.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.253+0000 7f6382cb7640 1 -- 172.21.15.22:0/718722872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637c19d630 con 0x7f637c100830 2024-09-19T08:15:11.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.253+0000 7f6379ffb640 1 -- 172.21.15.22:0/718722872 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c010690 con 0x7f637c100830 2024-09-19T08:15:11.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.253+0000 7f6382cb7640 1 -- 172.21.15.22:0/718722872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f637c1a4ec0 con 0x7f637c100830 2024-09-19T08:15:11.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.254+0000 7f6382cb7640 1 -- 172.21.15.22:0/718722872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f637c100c10 con 0x7f637c100830 2024-09-19T08:15:11.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.255+0000 7f6379ffb640 1 -- 172.21.15.22:0/718722872 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f636c0046c0 con 0x7f637c100830 2024-09-19T08:15:11.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.256+0000 7f6379ffb640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f63640a3580 0x7f63640a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:11.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.256+0000 7f637bfff640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f63640a3580 0x7f63640a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:11.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.256+0000 7f6379ffb640 1 -- 172.21.15.22:0/718722872 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f636c0d0560 con 0x7f637c100830 2024-09-19T08:15:11.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.256+0000 7f637bfff640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f63640a3580 0x7f63640a5a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6370005fd0 tx=0x7f6370005850 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:11.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.261+0000 7f6379ffb640 1 -- 172.21.15.22:0/718722872 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f636c09cca0 con 0x7f637c100830 2024-09-19T08:15:11.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.463+0000 7f6382cb7640 1 -- 172.21.15.22:0/718722872 --> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f637c0ffce0 con 0x7f63640a3580 2024-09-19T08:15:11.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.524+0000 7f6379ffb640 1 -- 172.21.15.22:0/718722872 <== mgr.15141 v2:172.21.15.22:6800/497251935 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f637c0ffce0 con 0x7f63640a3580 2024-09-19T08:15:11.525 INFO:teuthology.orchestra.run.smithi022.stdout:Schedule set for path / 2024-09-19T08:15:11.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.527+0000 7f6382cb7640 1 -- 172.21.15.22:0/718722872 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f63640a3580 msgr2=0x7f63640a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:11.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.527+0000 7f6382cb7640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f63640a3580 0x7f63640a5a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6370005fd0 tx=0x7f6370005850 comp rx=0 tx=0).stop 2024-09-19T08:15:11.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.527+0000 7f6382cb7640 1 -- 172.21.15.22:0/718722872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c100830 msgr2=0x7f637c198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:11.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.527+0000 7f6382cb7640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c100830 0x7f637c198350 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f636c00e9f0 tx=0x7f636c00edf0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.527+0000 7f6382cb7640 1 -- 172.21.15.22:0/718722872 shutdown_connections 2024-09-19T08:15:11.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.527+0000 7f6382cb7640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f63640a3580 0x7f63640a5a40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.527+0000 7f6382cb7640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f637c1080f0 0x7f637c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.527+0000 7f6382cb7640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f637c101150 0x7f637c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.527+0000 7f6382cb7640 1 --2- 172.21.15.22:0/718722872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c100830 0x7f637c198350 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.527+0000 7f6382cb7640 1 -- 172.21.15.22:0/718722872 >> 172.21.15.22:0/718722872 conn(0x7f637c075ec0 msgr2=0x7f637c0ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:11.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.527+0000 7f6382cb7640 1 -- 172.21.15.22:0/718722872 shutdown_connections 2024-09-19T08:15:11.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.527+0000 7f6382cb7640 1 -- 172.21.15.22:0/718722872 wait complete. 2024-09-19T08:15:11.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-19T08:15:11.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.645+0000 7f80c8168640 1 -- 172.21.15.22:0/566599579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c01035c0 msgr2=0x7f80c010baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:11.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.645+0000 7f80c8168640 1 --2- 172.21.15.22:0/566599579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c01035c0 0x7f80c010baa0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009900 tx=0x7f80b002a480 comp rx=0 tx=0).stop 2024-09-19T08:15:11.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.645+0000 7f80c8168640 1 -- 172.21.15.22:0/566599579 shutdown_connections 2024-09-19T08:15:11.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.645+0000 7f80c8168640 1 --2- 172.21.15.22:0/566599579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80c010c0c0 0x7f80c010e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.645+0000 7f80c8168640 1 --2- 172.21.15.22:0/566599579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c01035c0 0x7f80c010baa0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.645+0000 7f80c8168640 1 --2- 172.21.15.22:0/566599579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c0102ca0 0x7f80c0103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.645+0000 7f80c8168640 1 -- 172.21.15.22:0/566599579 >> 172.21.15.22:0/566599579 conn(0x7f80c00fc6f0 msgr2=0x7f80c00feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:11.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.645+0000 7f80c8168640 1 -- 172.21.15.22:0/566599579 shutdown_connections 2024-09-19T08:15:11.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.645+0000 7f80c8168640 1 -- 172.21.15.22:0/566599579 wait complete. 2024-09-19T08:15:11.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c8168640 1 Processor -- start 2024-09-19T08:15:11.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c8168640 1 -- start start 2024-09-19T08:15:11.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c8168640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c0102ca0 0x7f80c0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:11.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c8168640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c01035c0 0x7f80c0198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:11.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c8168640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80c010c0c0 0x7f80c019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:11.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c8168640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f80c0111ee0 con 0x7f80c01035c0 2024-09-19T08:15:11.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c8168640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f80c0111d60 con 0x7f80c010c0c0 2024-09-19T08:15:11.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c8168640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f80c0112060 con 0x7f80c0102ca0 2024-09-19T08:15:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c5703640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c01035c0 0x7f80c0198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c5703640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c01035c0 0x7f80c0198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42836/0 (socket says 172.21.15.22:42836) 2024-09-19T08:15:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c5703640 1 -- 172.21.15.22:0/4154638998 learned_addr learned my addr 172.21.15.22:0/4154638998 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c5f04640 1 --2- 172.21.15.22:0/4154638998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c0102ca0 0x7f80c0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c6705640 1 --2- 172.21.15.22:0/4154638998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80c010c0c0 0x7f80c019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c5703640 1 -- 172.21.15.22:0/4154638998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c0102ca0 msgr2=0x7f80c0198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.646+0000 7f80c5703640 1 --2- 172.21.15.22:0/4154638998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c0102ca0 0x7f80c0198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.647+0000 7f80c5703640 1 -- 172.21.15.22:0/4154638998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80c010c0c0 msgr2=0x7f80c019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.647+0000 7f80c5703640 1 --2- 172.21.15.22:0/4154638998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80c010c0c0 0x7f80c019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.647+0000 7f80c5703640 1 -- 172.21.15.22:0/4154638998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c019d2f0 con 0x7f80c01035c0 2024-09-19T08:15:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.647+0000 7f80c5703640 1 --2- 172.21.15.22:0/4154638998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c01035c0 0x7f80c0198840 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f80b0006010 tx=0x7f80b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.647+0000 7f80b6ffd640 1 -- 172.21.15.22:0/4154638998 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b003f030 con 0x7f80c01035c0 2024-09-19T08:15:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.647+0000 7f80c8168640 1 -- 172.21.15.22:0/4154638998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c019d580 con 0x7f80c01035c0 2024-09-19T08:15:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.647+0000 7f80b6ffd640 1 -- 172.21.15.22:0/4154638998 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80b0003ad0 con 0x7f80c01035c0 2024-09-19T08:15:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.647+0000 7f80b6ffd640 1 -- 172.21.15.22:0/4154638998 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b0035070 con 0x7f80c01035c0 2024-09-19T08:15:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.647+0000 7f80c8168640 1 -- 172.21.15.22:0/4154638998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c01a4e70 con 0x7f80c01035c0 2024-09-19T08:15:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.648+0000 7f80c8168640 1 -- 172.21.15.22:0/4154638998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c006abe0 con 0x7f80c01035c0 2024-09-19T08:15:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.649+0000 7f80b6ffd640 1 -- 172.21.15.22:0/4154638998 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f80b0042030 con 0x7f80c01035c0 2024-09-19T08:15:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.652+0000 7f80b6ffd640 1 --2- 172.21.15.22:0/4154638998 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f80ac0a3580 0x7f80ac0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.652+0000 7f80c5f04640 1 --2- 172.21.15.22:0/4154638998 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f80ac0a3580 0x7f80ac0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.652+0000 7f80b6ffd640 1 -- 172.21.15.22:0/4154638998 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f80b00f0740 con 0x7f80c01035c0 2024-09-19T08:15:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.652+0000 7f80c5f04640 1 --2- 172.21.15.22:0/4154638998 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f80ac0a3580 0x7f80ac0a5a40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009bd0 tx=0x7f80bc005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:11.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.654+0000 7f80b6ffd640 1 -- 172.21.15.22:0/4154638998 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f80b00bc960 con 0x7f80c01035c0 2024-09-19T08:15:11.674 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:15:11.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3549112373' entity='client.admin' 2024-09-19T08:15:11.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:11.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:11.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:11.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:11.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/4224074455' entity='client.admin' 2024-09-19T08:15:11.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:11.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:11.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:11.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:11.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3258662420' entity='client.admin' 2024-09-19T08:15:11.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:11.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:11.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:11.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:11 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3549112373' entity='client.admin' 2024-09-19T08:15:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/4224074455' entity='client.admin' 2024-09-19T08:15:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3258662420' entity='client.admin' 2024-09-19T08:15:11.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:11.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:11.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:11.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:11 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3549112373' entity='client.admin' 2024-09-19T08:15:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/4224074455' entity='client.admin' 2024-09-19T08:15:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:11.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:11.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3258662420' entity='client.admin' 2024-09-19T08:15:11.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:15:11.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:15:11.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:15:11.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:11 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:15:11.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.855+0000 7f80c8168640 1 -- 172.21.15.22:0/4154638998 --> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f80c01070f0 con 0x7f80ac0a3580 2024-09-19T08:15:11.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.917+0000 7f80b6ffd640 1 -- 172.21.15.22:0/4154638998 <== mgr.15141 v2:172.21.15.22:6800/497251935 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f80c01070f0 con 0x7f80ac0a3580 2024-09-19T08:15:11.918 INFO:teuthology.orchestra.run.smithi022.stdout:Retention added to path / 2024-09-19T08:15:11.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.919+0000 7f80c8168640 1 -- 172.21.15.22:0/4154638998 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f80ac0a3580 msgr2=0x7f80ac0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.919+0000 7f80c8168640 1 --2- 172.21.15.22:0/4154638998 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f80ac0a3580 0x7f80ac0a5a40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009bd0 tx=0x7f80bc005f90 comp rx=0 tx=0).stop 2024-09-19T08:15:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.919+0000 7f80c8168640 1 -- 172.21.15.22:0/4154638998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c01035c0 msgr2=0x7f80c0198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.919+0000 7f80c8168640 1 --2- 172.21.15.22:0/4154638998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c01035c0 0x7f80c0198840 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f80b0006010 tx=0x7f80b0004270 comp rx=0 tx=0).stop 2024-09-19T08:15:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.919+0000 7f80c8168640 1 -- 172.21.15.22:0/4154638998 shutdown_connections 2024-09-19T08:15:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.919+0000 7f80c8168640 1 --2- 172.21.15.22:0/4154638998 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f80ac0a3580 0x7f80ac0a5a40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.919+0000 7f80c8168640 1 --2- 172.21.15.22:0/4154638998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80c010c0c0 0x7f80c019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.919+0000 7f80c8168640 1 --2- 172.21.15.22:0/4154638998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c01035c0 0x7f80c0198840 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.919+0000 7f80c8168640 1 --2- 172.21.15.22:0/4154638998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c0102ca0 0x7f80c0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.919+0000 7f80c8168640 1 -- 172.21.15.22:0/4154638998 >> 172.21.15.22:0/4154638998 conn(0x7f80c00fc6f0 msgr2=0x7f80c00fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:11.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.919+0000 7f80c8168640 1 -- 172.21.15.22:0/4154638998 shutdown_connections 2024-09-19T08:15:11.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:11.919+0000 7f80c8168640 1 -- 172.21.15.22:0/4154638998 wait complete. 2024-09-19T08:15:11.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T08:15:11.940 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:12.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.045+0000 7f902b0a6640 1 -- 172.21.15.22:0/3200497656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9024106a00 msgr2=0x7f9024106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.045+0000 7f902b0a6640 1 --2- 172.21.15.22:0/3200497656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9024106a00 0x7f9024106de0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9018009930 tx=0x7f901802a420 comp rx=0 tx=0).stop 2024-09-19T08:15:12.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.045+0000 7f902b0a6640 1 -- 172.21.15.22:0/3200497656 shutdown_connections 2024-09-19T08:15:12.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.045+0000 7f902b0a6640 1 --2- 172.21.15.22:0/3200497656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9024101390 0x7f902410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.045+0000 7f902b0a6640 1 --2- 172.21.15.22:0/3200497656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90241009f0 0x7f9024100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.045+0000 7f902b0a6640 1 --2- 172.21.15.22:0/3200497656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9024106a00 0x7f9024106de0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.045+0000 7f902b0a6640 1 -- 172.21.15.22:0/3200497656 >> 172.21.15.22:0/3200497656 conn(0x7f90240fc6b0 msgr2=0x7f90240fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:12.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.045+0000 7f902b0a6640 1 -- 172.21.15.22:0/3200497656 shutdown_connections 2024-09-19T08:15:12.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.045+0000 7f902b0a6640 1 -- 172.21.15.22:0/3200497656 wait complete. 2024-09-19T08:15:12.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.046+0000 7f902b0a6640 1 Processor -- start 2024-09-19T08:15:12.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.046+0000 7f902b0a6640 1 -- start start 2024-09-19T08:15:12.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.046+0000 7f902b0a6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90241009f0 0x7f9024198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.046+0000 7f902b0a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9024101390 0x7f9024198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f902b0a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9024106a00 0x7f902419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f902b0a6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9024110e00 con 0x7f9024101390 2024-09-19T08:15:12.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f902b0a6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9024110c80 con 0x7f90241009f0 2024-09-19T08:15:12.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f902b0a6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9024110f80 con 0x7f9024106a00 2024-09-19T08:15:12.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f9023fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9024101390 0x7f9024198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f9023fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9024101390 0x7f9024198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42858/0 (socket says 172.21.15.22:42858) 2024-09-19T08:15:12.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f9023fff640 1 -- 172.21.15.22:0/600248863 learned_addr learned my addr 172.21.15.22:0/600248863 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:12.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f9028e42640 1 --2- 172.21.15.22:0/600248863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90241009f0 0x7f9024198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f9029643640 1 --2- 172.21.15.22:0/600248863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9024106a00 0x7f902419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f9023fff640 1 -- 172.21.15.22:0/600248863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9024106a00 msgr2=0x7f902419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f9023fff640 1 --2- 172.21.15.22:0/600248863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9024106a00 0x7f902419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f9023fff640 1 -- 172.21.15.22:0/600248863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90241009f0 msgr2=0x7f9024198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f9023fff640 1 --2- 172.21.15.22:0/600248863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90241009f0 0x7f9024198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f9023fff640 1 -- 172.21.15.22:0/600248863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902419d410 con 0x7f9024101390 2024-09-19T08:15:12.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f9023fff640 1 --2- 172.21.15.22:0/600248863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9024101390 0x7f9024198960 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f901000d950 tx=0x7f901000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:12.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f9021ffb640 1 -- 172.21.15.22:0/600248863 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010017030 con 0x7f9024101390 2024-09-19T08:15:12.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f902b0a6640 1 -- 172.21.15.22:0/600248863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f902419d700 con 0x7f9024101390 2024-09-19T08:15:12.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f9021ffb640 1 -- 172.21.15.22:0/600248863 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9010004020 con 0x7f9024101390 2024-09-19T08:15:12.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.047+0000 7f9021ffb640 1 -- 172.21.15.22:0/600248863 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100047a0 con 0x7f9024101390 2024-09-19T08:15:12.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.048+0000 7f902b0a6640 1 -- 172.21.15.22:0/600248863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90241a4f90 con 0x7f9024101390 2024-09-19T08:15:12.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.049+0000 7f902b0a6640 1 -- 172.21.15.22:0/600248863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9024100e50 con 0x7f9024101390 2024-09-19T08:15:12.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.050+0000 7f9021ffb640 1 -- 172.21.15.22:0/600248863 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f90100041d0 con 0x7f9024101390 2024-09-19T08:15:12.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.050+0000 7f9021ffb640 1 --2- 172.21.15.22:0/600248863 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f90140a3580 0x7f90140a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.051+0000 7f9028e42640 1 --2- 172.21.15.22:0/600248863 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f90140a3580 0x7f90140a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.051+0000 7f9021ffb640 1 -- 172.21.15.22:0/600248863 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f90100cfa50 con 0x7f9024101390 2024-09-19T08:15:12.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.051+0000 7f9028e42640 1 --2- 172.21.15.22:0/600248863 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f90140a3580 0x7f90140a5a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9018000c00 tx=0x7f9018005a20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:12.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.056+0000 7f9021ffb640 1 -- 172.21.15.22:0/600248863 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f901009c1f0 con 0x7f9024101390 2024-09-19T08:15:12.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.255+0000 7f902b0a6640 1 -- 172.21.15.22:0/600248863 --> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9024109fd0 con 0x7f90140a3580 2024-09-19T08:15:12.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.262+0000 7f9021ffb640 1 -- 172.21.15.22:0/600248863 <== mgr.15141 v2:172.21.15.22:6800/497251935 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f9024109fd0 con 0x7f90140a3580 2024-09-19T08:15:12.263 INFO:teuthology.orchestra.run.smithi022.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T08:15:11", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-19T08:15:12.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.264+0000 7f902b0a6640 1 -- 172.21.15.22:0/600248863 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f90140a3580 msgr2=0x7f90140a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.264+0000 7f902b0a6640 1 --2- 172.21.15.22:0/600248863 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f90140a3580 0x7f90140a5a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9018000c00 tx=0x7f9018005a20 comp rx=0 tx=0).stop 2024-09-19T08:15:12.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.264+0000 7f902b0a6640 1 -- 172.21.15.22:0/600248863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9024101390 msgr2=0x7f9024198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.264+0000 7f902b0a6640 1 --2- 172.21.15.22:0/600248863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9024101390 0x7f9024198960 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f901000d950 tx=0x7f901000de10 comp rx=0 tx=0).stop 2024-09-19T08:15:12.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.264+0000 7f902b0a6640 1 -- 172.21.15.22:0/600248863 shutdown_connections 2024-09-19T08:15:12.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.264+0000 7f902b0a6640 1 --2- 172.21.15.22:0/600248863 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f90140a3580 0x7f90140a5a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.264+0000 7f902b0a6640 1 --2- 172.21.15.22:0/600248863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9024106a00 0x7f902419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.264+0000 7f902b0a6640 1 --2- 172.21.15.22:0/600248863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9024101390 0x7f9024198960 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.264+0000 7f902b0a6640 1 --2- 172.21.15.22:0/600248863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90241009f0 0x7f9024198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.264+0000 7f902b0a6640 1 -- 172.21.15.22:0/600248863 >> 172.21.15.22:0/600248863 conn(0x7f90240fc6b0 msgr2=0x7f902406a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:12.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.264+0000 7f902b0a6640 1 -- 172.21.15.22:0/600248863 shutdown_connections 2024-09-19T08:15:12.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.264+0000 7f902b0a6640 1 -- 172.21.15.22:0/600248863 wait complete. 2024-09-19T08:15:12.278 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T08:15:12.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.425+0000 7fad861e4640 1 -- 172.21.15.22:0/3494909248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad80101350 msgr2=0x7fad8010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.425+0000 7fad861e4640 1 --2- 172.21.15.22:0/3494909248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad80101350 0x7fad8010d880 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fad78009cd0 tx=0x7fad7802a8e0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.425+0000 7fad861e4640 1 -- 172.21.15.22:0/3494909248 shutdown_connections 2024-09-19T08:15:12.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.425+0000 7fad861e4640 1 --2- 172.21.15.22:0/3494909248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad80101350 0x7fad8010d880 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.425+0000 7fad861e4640 1 --2- 172.21.15.22:0/3494909248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad801009b0 0x7fad80100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.425+0000 7fad861e4640 1 --2- 172.21.15.22:0/3494909248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad801069c0 0x7fad80106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.425+0000 7fad861e4640 1 -- 172.21.15.22:0/3494909248 >> 172.21.15.22:0/3494909248 conn(0x7fad80077990 msgr2=0x7fad800fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:12.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad861e4640 1 -- 172.21.15.22:0/3494909248 shutdown_connections 2024-09-19T08:15:12.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad861e4640 1 -- 172.21.15.22:0/3494909248 wait complete. 2024-09-19T08:15:12.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad861e4640 1 Processor -- start 2024-09-19T08:15:12.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad861e4640 1 -- start start 2024-09-19T08:15:12.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad861e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad801009b0 0x7fad801972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad861e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80101350 0x7fad80197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad861e4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad801069c0 0x7fad8019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad861e4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fad80110e50 con 0x7fad801069c0 2024-09-19T08:15:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad861e4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad80110cd0 con 0x7fad80101350 2024-09-19T08:15:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad861e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fad80110fd0 con 0x7fad801009b0 2024-09-19T08:15:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad7ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad801069c0 0x7fad8019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad7ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad801069c0 0x7fad8019bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42884/0 (socket says 172.21.15.22:42884) 2024-09-19T08:15:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad7f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad801009b0 0x7fad801972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad7ffff640 1 -- 172.21.15.22:0/762399650 learned_addr learned my addr 172.21.15.22:0/762399650 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad76ffd640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80101350 0x7fad80197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad7f7fe640 1 -- 172.21.15.22:0/762399650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80101350 msgr2=0x7fad80197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad7f7fe640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80101350 0x7fad80197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad7f7fe640 1 -- 172.21.15.22:0/762399650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad801069c0 msgr2=0x7fad8019bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad7f7fe640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad801069c0 0x7fad8019bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad7f7fe640 1 -- 172.21.15.22:0/762399650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8019c2b0 con 0x7fad801009b0 2024-09-19T08:15:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad76ffd640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80101350 0x7fad80197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad7ffff640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad801069c0 0x7fad8019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.426+0000 7fad7f7fe640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad801009b0 0x7fad801972c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fad7000d720 tx=0x7fad7000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.428+0000 7fad7d7fa640 1 -- 172.21.15.22:0/762399650 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70017070 con 0x7fad801009b0 2024-09-19T08:15:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.428+0000 7fad861e4640 1 -- 172.21.15.22:0/762399650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8019c5a0 con 0x7fad801009b0 2024-09-19T08:15:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.428+0000 7fad7d7fa640 1 -- 172.21.15.22:0/762399650 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad7000de30 con 0x7fad801009b0 2024-09-19T08:15:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.428+0000 7fad7d7fa640 1 -- 172.21.15.22:0/762399650 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad700047a0 con 0x7fad801009b0 2024-09-19T08:15:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.428+0000 7fad861e4640 1 -- 172.21.15.22:0/762399650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad801a3e30 con 0x7fad801009b0 2024-09-19T08:15:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.429+0000 7fad861e4640 1 -- 172.21.15.22:0/762399650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad80100e10 con 0x7fad801009b0 2024-09-19T08:15:12.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.430+0000 7fad7d7fa640 1 -- 172.21.15.22:0/762399650 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fad700099d0 con 0x7fad801009b0 2024-09-19T08:15:12.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.431+0000 7fad7d7fa640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fad6c0a3580 0x7fad6c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.431+0000 7fad76ffd640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fad6c0a3580 0x7fad6c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.431+0000 7fad7d7fa640 1 -- 172.21.15.22:0/762399650 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fad700cff00 con 0x7fad801009b0 2024-09-19T08:15:12.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.431+0000 7fad76ffd640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fad6c0a3580 0x7fad6c0a5a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fad80105f40 tx=0x7fad6800b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:12.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.434+0000 7fad7d7fa640 1 -- 172.21.15.22:0/762399650 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fad70020020 con 0x7fad801009b0 2024-09-19T08:15:12.478 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:12 smithi022 ceph-mon[30963]: scrub queued for path: / 2024-09-19T08:15:12.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:12 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:15:12.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:12 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:15:12.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:12 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:15:12.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:12 smithi022 ceph-mon[30963]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 796 B/s wr, 17 op/s 2024-09-19T08:15:12.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:12 smithi022 ceph-mon[30963]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:15:12.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:12 smithi022 ceph-mon[30963]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:15:12.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.478+0000 7f3debb96640 1 -- 172.21.15.22:0/941203802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de4103740 msgr2=0x7f3de4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.478+0000 7f3debb96640 1 --2- 172.21.15.22:0/941203802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de4103740 0x7f3de4107b80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009930 tx=0x7f3dd802a420 comp rx=0 tx=0).stop 2024-09-19T08:15:12.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.478+0000 7f3debb96640 1 -- 172.21.15.22:0/941203802 shutdown_connections 2024-09-19T08:15:12.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.478+0000 7f3debb96640 1 --2- 172.21.15.22:0/941203802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3de4108290 0x7f3de410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.478+0000 7f3debb96640 1 --2- 172.21.15.22:0/941203802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de4103740 0x7f3de4107b80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.478+0000 7f3debb96640 1 --2- 172.21.15.22:0/941203802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de4102d90 0x7f3de4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.478+0000 7f3debb96640 1 -- 172.21.15.22:0/941203802 >> 172.21.15.22:0/941203802 conn(0x7f3de40fc7d0 msgr2=0x7f3de40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:12.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.478+0000 7f3debb96640 1 -- 172.21.15.22:0/941203802 shutdown_connections 2024-09-19T08:15:12.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.478+0000 7f3debb96640 1 -- 172.21.15.22:0/941203802 wait complete. 2024-09-19T08:15:12.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.479+0000 7f3debb96640 1 Processor -- start 2024-09-19T08:15:12.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.479+0000 7f3debb96640 1 -- start start 2024-09-19T08:15:12.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.479+0000 7f3debb96640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3de4102d90 0x7f3de41a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.479+0000 7f3debb96640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de4103740 0x7f3de41a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.479+0000 7f3debb96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de4108290 0x7f3de419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.479+0000 7f3debb96640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3de4112090 con 0x7f3de4108290 2024-09-19T08:15:12.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.479+0000 7f3debb96640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3de4111f10 con 0x7f3de4102d90 2024-09-19T08:15:12.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.479+0000 7f3debb96640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3de4112210 con 0x7f3de4103740 2024-09-19T08:15:12.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.479+0000 7f3dea133640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de4108290 0x7f3de419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3dea133640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de4108290 0x7f3de419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42896/0 (socket says 172.21.15.22:42896) 2024-09-19T08:15:12.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3dea133640 1 -- 172.21.15.22:0/512241586 learned_addr learned my addr 172.21.15.22:0/512241586 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:12.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3de9932640 1 --2- 172.21.15.22:0/512241586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3de4102d90 0x7f3de41a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3de9131640 1 --2- 172.21.15.22:0/512241586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de4103740 0x7f3de41a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3dea133640 1 -- 172.21.15.22:0/512241586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de4103740 msgr2=0x7f3de41a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3dea133640 1 --2- 172.21.15.22:0/512241586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de4103740 0x7f3de41a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3dea133640 1 -- 172.21.15.22:0/512241586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3de4102d90 msgr2=0x7f3de41a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3dea133640 1 --2- 172.21.15.22:0/512241586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3de4102d90 0x7f3de41a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3dea133640 1 -- 172.21.15.22:0/512241586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de419ad50 con 0x7f3de4108290 2024-09-19T08:15:12.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3dea133640 1 --2- 172.21.15.22:0/512241586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de4108290 0x7f3de419a620 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400eb10 tx=0x7f3dd400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:12.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3de2ffd640 1 -- 172.21.15.22:0/512241586 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4004b20 con 0x7f3de4108290 2024-09-19T08:15:12.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3debb96640 1 -- 172.21.15.22:0/512241586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de419b040 con 0x7f3de4108290 2024-09-19T08:15:12.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3de2ffd640 1 -- 172.21.15.22:0/512241586 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dd4004cc0 con 0x7f3de4108290 2024-09-19T08:15:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3de2ffd640 1 -- 172.21.15.22:0/512241586 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4010760 con 0x7f3de4108290 2024-09-19T08:15:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.480+0000 7f3debb96640 1 -- 172.21.15.22:0/512241586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de41a7330 con 0x7f3de4108290 2024-09-19T08:15:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.484+0000 7f3dedf6f740 1 Processor -- start 2024-09-19T08:15:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.484+0000 7f3de2ffd640 1 -- 172.21.15.22:0/512241586 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f3dd4005020 con 0x7f3de4108290 2024-09-19T08:15:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.484+0000 7f3dedf6f740 1 -- start start 2024-09-19T08:15:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.484+0000 7f3dedf6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33d4a0 0x560ddb33d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.484+0000 7f3dedf6f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb33ddc0 0x560ddb348140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.484+0000 7f3dedf6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb348680 0x560ddb34aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.484+0000 7f3dedf6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560ddb2a32e0 con 0x560ddb348680 2024-09-19T08:15:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.484+0000 7f3dedf6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560ddb24ace0 con 0x560ddb33d4a0 2024-09-19T08:15:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.484+0000 7f3dedf6f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560ddb29f2a0 con 0x560ddb33ddc0 2024-09-19T08:15:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de9131640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb348680 0x560ddb34aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de9932640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb33ddc0 0x560ddb348140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de9131640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb348680 0x560ddb34aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42910/0 (socket says 172.21.15.22:42910) 2024-09-19T08:15:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de9131640 1 -- 172.21.15.22:0/3910612610 learned_addr learned my addr 172.21.15.22:0/3910612610 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de9932640 1 --2- 172.21.15.22:0/3910612610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb33ddc0 0x560ddb348140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:47240/0 (socket says 172.21.15.22:47240) 2024-09-19T08:15:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de9131640 1 --2- 172.21.15.22:0/3910612610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33d4a0 0x560ddb33d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de9932640 1 -- 172.21.15.22:0/3910612610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33d4a0 msgr2=0x560ddb33d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de9932640 1 --2- 172.21.15.22:0/3910612610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33d4a0 0x560ddb33d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de2ffd640 1 --2- 172.21.15.22:0/512241586 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3dcc0a34d0 0x7f3dcc0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de9932640 1 -- 172.21.15.22:0/3910612610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb348680 msgr2=0x560ddb34aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de9932640 1 --2- 172.21.15.22:0/3910612610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb348680 0x560ddb34aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de9932640 1 -- 172.21.15.22:0/3910612610 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ddb22f440 con 0x560ddb33ddc0 2024-09-19T08:15:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de9131640 1 --2- 172.21.15.22:0/3910612610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb348680 0x560ddb34aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3dea133640 1 --2- 172.21.15.22:0/512241586 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3dcc0a34d0 0x7f3dcc0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de9932640 1 --2- 172.21.15.22:0/3910612610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb33ddc0 0x560ddb348140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009c00 tx=0x7f3ddc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7fa72b5b71ac275 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:12.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3de2ffd640 1 -- 172.21.15.22:0/512241586 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3dd40d04b0 con 0x7f3de4108290 2024-09-19T08:15:12.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.485+0000 7f3dcbfff640 1 -- 172.21.15.22:0/3910612610 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc004230 con 0x560ddb33ddc0 2024-09-19T08:15:12.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.486+0000 7f3dcbfff640 1 -- 172.21.15.22:0/3910612610 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ddc0043d0 con 0x560ddb33ddc0 2024-09-19T08:15:12.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.486+0000 7f3dcbfff640 1 -- 172.21.15.22:0/3910612610 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc004d20 con 0x560ddb33ddc0 2024-09-19T08:15:12.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.486+0000 7f3dedf6f740 1 -- 172.21.15.22:0/3910612610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb33ddc0 msgr2=0x560ddb348140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.486+0000 7f3dea133640 1 --2- 172.21.15.22:0/512241586 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3dcc0a34d0 0x7f3dcc0a5990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40cdaf0 tx=0x7f3dd401bb30 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:12.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.486+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/3910612610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb33ddc0 0x560ddb348140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009c00 tx=0x7f3ddc011810 comp rx=0 tx=0).stop 2024-09-19T08:15:12.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.486+0000 7f3dedf6f740 1 -- 172.21.15.22:0/3910612610 shutdown_connections 2024-09-19T08:15:12.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.486+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/3910612610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb348680 0x560ddb34aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.486+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/3910612610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb33ddc0 0x560ddb348140 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.486+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/3910612610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33d4a0 0x560ddb33d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.486+0000 7f3dedf6f740 1 -- 172.21.15.22:0/3910612610 >> 172.21.15.22:0/3910612610 conn(0x560ddb339a30 msgr2=0x560ddb339e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:12.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.487+0000 7f3dedf6f740 1 -- 172.21.15.22:0/3910612610 shutdown_connections 2024-09-19T08:15:12.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.487+0000 7f3dedf6f740 1 -- 172.21.15.22:0/3910612610 wait complete. 2024-09-19T08:15:12.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.487+0000 7f3dedf6f740 1 -- start start 2024-09-19T08:15:12.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.487+0000 7f3dedf6f740 1 Processor -- start 2024-09-19T08:15:12.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3dedf6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb33d4a0 0x560ddb347fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3dedf6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 0x560ddb345250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3dedf6f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb348680 0x560ddb345790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3dedf6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560ddb34b250 con 0x560ddb33d4a0 2024-09-19T08:15:12.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3dedf6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560ddb25e050 con 0x560ddb33ddc0 2024-09-19T08:15:12.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3dedf6f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560ddb34b3d0 con 0x560ddb348680 2024-09-19T08:15:12.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3dedf6f740 10 client.0 _ll_register_callbacks cb 0x560ddb29e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:12.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3de9131640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb33d4a0 0x560ddb347fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3de9131640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb33d4a0 0x560ddb347fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42926/0 (socket says 172.21.15.22:42926) 2024-09-19T08:15:12.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3de9131640 1 -- 172.21.15.22:0/4205746038 learned_addr learned my addr 172.21.15.22:0/4205746038 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:12.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3de9932640 1 --2- 172.21.15.22:0/4205746038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 0x560ddb345250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3de9131640 1 --2- 172.21.15.22:0/4205746038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb348680 0x560ddb345790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3de9131640 1 -- 172.21.15.22:0/4205746038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb348680 msgr2=0x560ddb345790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:12.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3de9131640 1 --2- 172.21.15.22:0/4205746038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb348680 0x560ddb345790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3de9131640 1 -- 172.21.15.22:0/4205746038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 msgr2=0x560ddb345250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:12.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3de9131640 1 --2- 172.21.15.22:0/4205746038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 0x560ddb345250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3de9131640 1 -- 172.21.15.22:0/4205746038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ddb0143c0 con 0x560ddb33d4a0 2024-09-19T08:15:12.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3de9932640 1 --2- 172.21.15.22:0/4205746038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 0x560ddb345250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:12.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3de9131640 1 --2- 172.21.15.22:0/4205746038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb33d4a0 0x560ddb347fa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8005780 tx=0x7f3dd802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c22fce64988d5b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:12.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3dc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:12.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.488+0000 7f3dc97fa640 1 -- 172.21.15.22:0/4205746038 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8041030 con 0x560ddb33d4a0 2024-09-19T08:15:12.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.489+0000 7f3dc97fa640 1 -- 172.21.15.22:0/4205746038 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3dd8004450 con 0x560ddb33d4a0 2024-09-19T08:15:12.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.489+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4205746038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ddc0043d0 con 0x560ddb33d4a0 2024-09-19T08:15:12.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.489+0000 7f3dc97fa640 1 -- 172.21.15.22:0/4205746038 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3dd8035050 con 0x560ddb33d4a0 2024-09-19T08:15:12.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.489+0000 7f3dedf6f740 10 client.15342 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:12.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.489+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4205746038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560ddb356c00 con 0x560ddb33d4a0 2024-09-19T08:15:12.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.489+0000 7f3dc97fa640 1 -- 172.21.15.22:0/4205746038 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ddc0043d0 con 0x560ddb33d4a0 2024-09-19T08:15:12.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.489+0000 7f3dc97fa640 1 -- 172.21.15.22:0/4205746038 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3dd80547d0 con 0x560ddb33d4a0 2024-09-19T08:15:12.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.489+0000 7f3dedf6f740 10 client.15342 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:12.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.489+0000 7f3dedf6f740 10 client.15342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:15:12.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.489+0000 7f3dedf6f740 20 client.15342 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:12.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.489+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/4205746038 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560ddb358440 0x560ddb35a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.489+0000 7f3dedf6f740 4 client.15342 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:15:12.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.489+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4205746038 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ddb2b0850 con 0x560ddb358440 2024-09-19T08:15:12.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.490+0000 7f3de9932640 1 --2- 172.21.15.22:0/4205746038 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560ddb358440 0x560ddb35a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.490+0000 7f3de9932640 1 --2- 172.21.15.22:0/4205746038 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560ddb358440 0x560ddb35a870 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86777cf8837f9ece server_cookie=2d50d7467787ffa9 in_seq=0 out_seq=0 2024-09-19T08:15:12.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.490+0000 7f3dc97fa640 10 client.15342 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:12.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.491+0000 7f3dc97fa640 1 -- 172.21.15.22:0/4205746038 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ddb2b0850 con 0x560ddb358440 2024-09-19T08:15:12.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.491+0000 7f3dc97fa640 10 client.15342 handle_command_reply: tid=0 2024-09-19T08:15:12.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.491+0000 7f3dedf6f740 1 client.15342 shutdown 2024-09-19T08:15:12.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.491+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4205746038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb33d4a0 msgr2=0x560ddb347fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.491+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/4205746038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb33d4a0 0x560ddb347fa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8005780 tx=0x7f3dd802a420 comp rx=0 tx=0).stop 2024-09-19T08:15:12.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.492+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4205746038 shutdown_connections 2024-09-19T08:15:12.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.492+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/4205746038 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560ddb358440 0x560ddb35a870 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.492+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/4205746038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb348680 0x560ddb345790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.492+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/4205746038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 0x560ddb345250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.492+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/4205746038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb33d4a0 0x560ddb347fa0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.492+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4205746038 >> 172.21.15.22:0/4205746038 conn(0x560ddb339a30 msgr2=0x560ddb33f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:12.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.492+0000 7f3dc97fa640 0 client.15342 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:12.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.492+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4205746038 shutdown_connections 2024-09-19T08:15:12.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.492+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4205746038 wait complete. 2024-09-19T08:15:12.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.492+0000 7f3dedf6f740 20 client.15342 trim_cache size 0 max 16384 2024-09-19T08:15:12.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.509+0000 7f3dedf6f740 1 Processor -- start 2024-09-19T08:15:12.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.509+0000 7f3dedf6f740 1 -- start start 2024-09-19T08:15:12.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.509+0000 7f3dedf6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb33d4a0 0x560ddb3544e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.509+0000 7f3dedf6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 0x560ddb360180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.509+0000 7f3dedf6f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb348680 0x560ddb3606c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.509+0000 7f3dedf6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41b23c0 con 0x560ddb33d4a0 2024-09-19T08:15:12.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.509+0000 7f3dedf6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41afc90 con 0x560ddb33ddc0 2024-09-19T08:15:12.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.509+0000 7f3dedf6f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41adae0 con 0x560ddb348680 2024-09-19T08:15:12.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.510+0000 7f3de9131640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb33d4a0 0x560ddb3544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.510+0000 7f3de9131640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb33d4a0 0x560ddb3544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42934/0 (socket says 172.21.15.22:42934) 2024-09-19T08:15:12.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.510+0000 7f3de9131640 1 -- 172.21.15.22:0/1721886578 learned_addr learned my addr 172.21.15.22:0/1721886578 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:12.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.510+0000 7f3de9932640 1 --2- 172.21.15.22:0/1721886578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 0x560ddb360180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.510+0000 7f3de9131640 1 --2- 172.21.15.22:0/1721886578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb348680 0x560ddb3606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.510+0000 7f3de9131640 1 -- 172.21.15.22:0/1721886578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb348680 msgr2=0x560ddb3606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:12.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.510+0000 7f3de9131640 1 --2- 172.21.15.22:0/1721886578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb348680 0x560ddb3606c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.510+0000 7f3de9131640 1 -- 172.21.15.22:0/1721886578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 msgr2=0x560ddb360180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:12.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.510+0000 7f3de9131640 1 --2- 172.21.15.22:0/1721886578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 0x560ddb360180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.510+0000 7f3de9131640 1 -- 172.21.15.22:0/1721886578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ddb348310 con 0x560ddb33d4a0 2024-09-19T08:15:12.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.510+0000 7f3de9131640 1 --2- 172.21.15.22:0/1721886578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb33d4a0 0x560ddb3544e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8004720 tx=0x7f3dd8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b584d27f704200a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:12.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.510+0000 7f3dcb7fe640 1 -- 172.21.15.22:0/1721886578 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8041040 con 0x560ddb33d4a0 2024-09-19T08:15:12.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.510+0000 7f3dcb7fe640 1 -- 172.21.15.22:0/1721886578 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dd8054950 con 0x560ddb33d4a0 2024-09-19T08:15:12.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.510+0000 7f3dcb7fe640 1 -- 172.21.15.22:0/1721886578 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd804d8e0 con 0x560ddb33d4a0 2024-09-19T08:15:12.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.511+0000 7f3dedf6f740 1 -- 172.21.15.22:0/1721886578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb33d4a0 msgr2=0x560ddb3544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.511+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/1721886578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb33d4a0 0x560ddb3544e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8004720 tx=0x7f3dd8005fd0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.511+0000 7f3dedf6f740 1 -- 172.21.15.22:0/1721886578 shutdown_connections 2024-09-19T08:15:12.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.511+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/1721886578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb348680 0x560ddb3606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.511+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/1721886578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 0x560ddb360180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.511+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/1721886578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb33d4a0 0x560ddb3544e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.511+0000 7f3dedf6f740 1 -- 172.21.15.22:0/1721886578 >> 172.21.15.22:0/1721886578 conn(0x560ddb339a30 msgr2=0x560ddb340850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:12.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.511+0000 7f3dedf6f740 1 -- 172.21.15.22:0/1721886578 shutdown_connections 2024-09-19T08:15:12.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.511+0000 7f3dedf6f740 1 -- 172.21.15.22:0/1721886578 wait complete. 2024-09-19T08:15:12.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.512+0000 7f3dedf6f740 1 -- start start 2024-09-19T08:15:12.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.512+0000 7f3dedf6f740 1 Processor -- start 2024-09-19T08:15:12.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.512+0000 7f3dedf6f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb33d4a0 0x560ddb34b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.512+0000 7f3dedf6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 0x560ddb34bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.512+0000 7f3dedf6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb348680 0x560ddb3c1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.512+0000 7f3dedf6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41ae150 con 0x560ddb348680 2024-09-19T08:15:12.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.512+0000 7f3dedf6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41af5c0 con 0x560ddb33ddc0 2024-09-19T08:15:12.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.512+0000 7f3dedf6f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41ae820 con 0x560ddb33d4a0 2024-09-19T08:15:12.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.512+0000 7f3dedf6f740 10 client.0 _ll_register_callbacks cb 0x7f3dd8031710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:12.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.512+0000 7f3de9131640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb348680 0x560ddb3c1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.512+0000 7f3de9131640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb348680 0x560ddb3c1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42946/0 (socket says 172.21.15.22:42946) 2024-09-19T08:15:12.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.512+0000 7f3de9131640 1 -- 172.21.15.22:0/4240246633 learned_addr learned my addr 172.21.15.22:0/4240246633 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:12.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3de9131640 1 --2- 172.21.15.22:0/4240246633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb33d4a0 0x560ddb34b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3de9932640 1 --2- 172.21.15.22:0/4240246633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 0x560ddb34bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3de9131640 1 -- 172.21.15.22:0/4240246633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb33d4a0 msgr2=0x560ddb34b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:12.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3de9131640 1 --2- 172.21.15.22:0/4240246633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb33d4a0 0x560ddb34b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3de9131640 1 -- 172.21.15.22:0/4240246633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 msgr2=0x560ddb34bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:12.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3de9131640 1 --2- 172.21.15.22:0/4240246633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 0x560ddb34bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3de9131640 1 -- 172.21.15.22:0/4240246633 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ddb3c1680 con 0x560ddb348680 2024-09-19T08:15:12.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3de9932640 1 --2- 172.21.15.22:0/4240246633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 0x560ddb34bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:12.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3de9131640 1 --2- 172.21.15.22:0/4240246633 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb348680 0x560ddb3c1140 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8054ba0 tx=0x7f3dd8054f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd54358d51e7238 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:12.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3dc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:12.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3dc97fa640 1 -- 172.21.15.22:0/4240246633 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8066020 con 0x560ddb348680 2024-09-19T08:15:12.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3dc97fa640 1 -- 172.21.15.22:0/4240246633 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3dd8054470 con 0x560ddb348680 2024-09-19T08:15:12.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4240246633 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dd8054950 con 0x560ddb348680 2024-09-19T08:15:12.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3dc97fa640 1 -- 172.21.15.22:0/4240246633 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3dd8054670 con 0x560ddb348680 2024-09-19T08:15:12.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3dedf6f740 10 client.15348 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:12.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4240246633 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560ddb3c2890 con 0x560ddb348680 2024-09-19T08:15:12.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.513+0000 7f3dc97fa640 1 -- 172.21.15.22:0/4240246633 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dd8054950 con 0x560ddb348680 2024-09-19T08:15:12.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.514+0000 7f3dc97fa640 1 -- 172.21.15.22:0/4240246633 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3dd8069020 con 0x560ddb348680 2024-09-19T08:15:12.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.514+0000 7f3dedf6f740 10 client.15348 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:12.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.514+0000 7f3dedf6f740 10 client.15348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:15:12.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.514+0000 7f3dedf6f740 20 client.15348 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:12.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.514+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/4240246633 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560ddb3c2e20 0x560ddb3c5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:12.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.514+0000 7f3dedf6f740 4 client.15348 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:15:12.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.514+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4240246633 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560ddb2aad60 con 0x560ddb3c2e20 2024-09-19T08:15:12.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.514+0000 7f3de9932640 1 --2- 172.21.15.22:0/4240246633 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560ddb3c2e20 0x560ddb3c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:12.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.514+0000 7f3de9932640 1 --2- 172.21.15.22:0/4240246633 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560ddb3c2e20 0x560ddb3c5240 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aea7c24154e86bc7 server_cookie=d084add02b4f1480 in_seq=0 out_seq=0 2024-09-19T08:15:12.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.514+0000 7f3dc97fa640 10 client.15348 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:12.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dc97fa640 1 -- 172.21.15.22:0/4240246633 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560ddb2aad60 con 0x560ddb3c2e20 2024-09-19T08:15:12.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dc97fa640 10 client.15348 handle_command_reply: tid=0 2024-09-19T08:15:12.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dedf6f740 1 client.15348 shutdown 2024-09-19T08:15:12.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4240246633 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb348680 msgr2=0x560ddb3c1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/4240246633 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb348680 0x560ddb3c1140 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8054ba0 tx=0x7f3dd8054f90 comp rx=0 tx=0).stop 2024-09-19T08:15:12.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4240246633 shutdown_connections 2024-09-19T08:15:12.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/4240246633 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560ddb3c2e20 0x560ddb3c5240 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/4240246633 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ddb348680 0x560ddb3c1140 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/4240246633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ddb33ddc0 0x560ddb34bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dc97fa640 0 client.15348 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:12.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dedf6f740 1 --2- 172.21.15.22:0/4240246633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560ddb33d4a0 0x560ddb34b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4240246633 >> 172.21.15.22:0/4240246633 conn(0x560ddb339a30 msgr2=0x560ddb33ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:12.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4240246633 shutdown_connections 2024-09-19T08:15:12.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dedf6f740 1 -- 172.21.15.22:0/4240246633 wait complete. 2024-09-19T08:15:12.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.515+0000 7f3dedf6f740 20 client.15348 trim_cache size 0 max 16384 2024-09-19T08:15:12.520 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:15:12.520 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:15:12.520 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-19T08:15:12.521 INFO:teuthology.orchestra.run.smithi022.stdout: "190afc4d-70c2-4bfa-9608-f13ca79efbf7": { 2024-09-19T08:15:12.521 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-19T08:15:12.521 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "190afc4d-70c2-4bfa-9608-f13ca79efbf7", 2024-09-19T08:15:12.521 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-09-19T08:15:12.521 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:15:12.521 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:15:12.521 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:15:12.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.516+0000 7f3debb96640 1 -- 172.21.15.22:0/512241586 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3dcc0a34d0 msgr2=0x7f3dcc0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.516+0000 7f3debb96640 1 --2- 172.21.15.22:0/512241586 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3dcc0a34d0 0x7f3dcc0a5990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40cdaf0 tx=0x7f3dd401bb30 comp rx=0 tx=0).stop 2024-09-19T08:15:12.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.516+0000 7f3debb96640 1 -- 172.21.15.22:0/512241586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de4108290 msgr2=0x7f3de419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.516+0000 7f3debb96640 1 --2- 172.21.15.22:0/512241586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de4108290 0x7f3de419a620 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400eb10 tx=0x7f3dd400c2d0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.516+0000 7f3debb96640 1 -- 172.21.15.22:0/512241586 shutdown_connections 2024-09-19T08:15:12.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.516+0000 7f3debb96640 1 --2- 172.21.15.22:0/512241586 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3dcc0a34d0 0x7f3dcc0a5990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.516+0000 7f3debb96640 1 --2- 172.21.15.22:0/512241586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de4108290 0x7f3de419a620 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.516+0000 7f3debb96640 1 --2- 172.21.15.22:0/512241586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de4103740 0x7f3de41a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.516+0000 7f3debb96640 1 --2- 172.21.15.22:0/512241586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3de4102d90 0x7f3de41a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.516+0000 7f3debb96640 1 -- 172.21.15.22:0/512241586 >> 172.21.15.22:0/512241586 conn(0x7f3de40fc7d0 msgr2=0x7f3de410d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:12.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.516+0000 7f3debb96640 1 -- 172.21.15.22:0/512241586 shutdown_connections 2024-09-19T08:15:12.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.516+0000 7f3debb96640 1 -- 172.21.15.22:0/512241586 wait complete. 2024-09-19T08:15:12.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.654+0000 7fad861e4640 1 -- 172.21.15.22:0/762399650 --> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fad800ff6a0 con 0x7fad6c0a3580 2024-09-19T08:15:12.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.660+0000 7fad7d7fa640 1 -- 172.21.15.22:0/762399650 <== mgr.15141 v2:172.21.15.22:6800/497251935 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fad800ff6a0 con 0x7fad6c0a3580 2024-09-19T08:15:12.660 INFO:teuthology.orchestra.run.smithi022.stdout:/ 1m 6m3h 2024-09-19T08:15:12.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.662+0000 7fad861e4640 1 -- 172.21.15.22:0/762399650 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fad6c0a3580 msgr2=0x7fad6c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.662+0000 7fad861e4640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fad6c0a3580 0x7fad6c0a5a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fad80105f40 tx=0x7fad6800b390 comp rx=0 tx=0).stop 2024-09-19T08:15:12.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.662+0000 7fad861e4640 1 -- 172.21.15.22:0/762399650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad801009b0 msgr2=0x7fad801972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:12.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.662+0000 7fad861e4640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad801009b0 0x7fad801972c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fad7000d720 tx=0x7fad7000dbe0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.663+0000 7fad861e4640 1 -- 172.21.15.22:0/762399650 shutdown_connections 2024-09-19T08:15:12.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.663+0000 7fad861e4640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fad6c0a3580 0x7fad6c0a5a40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.663+0000 7fad861e4640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad801069c0 0x7fad8019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.663+0000 7fad861e4640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80101350 0x7fad80197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.663+0000 7fad861e4640 1 --2- 172.21.15.22:0/762399650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad801009b0 0x7fad801972c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:12.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.663+0000 7fad861e4640 1 -- 172.21.15.22:0/762399650 >> 172.21.15.22:0/762399650 conn(0x7fad80077990 msgr2=0x7fad800fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:12.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.663+0000 7fad861e4640 1 -- 172.21.15.22:0/762399650 shutdown_connections 2024-09-19T08:15:12.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:12.663+0000 7fad861e4640 1 -- 172.21.15.22:0/762399650 wait complete. 2024-09-19T08:15:12.673 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-19T08:15:12.701 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-19T08:15:12.712 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-19T08:15:12.752 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T08:15:12.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:15:12.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:12 smithi157 ceph-mon[35143]: scrub queued for path: / 2024-09-19T08:15:12.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:12 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:15:12.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:12 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:15:12.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:12 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:15:12.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:12 smithi157 ceph-mon[35143]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 796 B/s wr, 17 op/s 2024-09-19T08:15:12.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:12 smithi157 ceph-mon[35143]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:15:12.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:12 smithi157 ceph-mon[35143]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:15:12.838 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:12 smithi064 ceph-mon[34900]: scrub queued for path: / 2024-09-19T08:15:12.839 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:12 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:15:12.839 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:12 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:15:12.839 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:12 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:15:12.839 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:12 smithi064 ceph-mon[34900]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 796 B/s wr, 17 op/s 2024-09-19T08:15:12.839 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:12 smithi064 ceph-mon[34900]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:15:12.839 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:12 smithi064 ceph-mon[34900]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:15:13.193 INFO:tasks.cephfs.filesystem:scrub status for tag:190afc4d-70c2-4bfa-9608-f13ca79efbf7 - {'path': '/', 'tag': '190afc4d-70c2-4bfa-9608-f13ca79efbf7', 'options': 'recursive,force'} 2024-09-19T08:15:13.241 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:13.642 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:13 smithi022 ceph-mon[30963]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:15:13.642 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:13 smithi022 ceph-mon[30963]: from='client.24991 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:15:13.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.723+0000 7fa3fc348640 1 -- 172.21.15.22:0/2960127012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4102790 msgr2=0x7fa3f41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:13.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.723+0000 7fa3fc348640 1 --2- 172.21.15.22:0/2960127012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4102790 0x7fa3f41067e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009900 tx=0x7fa3e402a410 comp rx=0 tx=0).stop 2024-09-19T08:15:13.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.723+0000 7fa3fc348640 1 -- 172.21.15.22:0/2960127012 shutdown_connections 2024-09-19T08:15:13.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.723+0000 7fa3fc348640 1 --2- 172.21.15.22:0/2960127012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f4106f10 0x7fa3f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:13.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.723+0000 7fa3fc348640 1 --2- 172.21.15.22:0/2960127012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4102790 0x7fa3f41067e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:13.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.723+0000 7fa3fc348640 1 --2- 172.21.15.22:0/2960127012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f4101de0 0x7fa3f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:13.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.723+0000 7fa3fc348640 1 -- 172.21.15.22:0/2960127012 >> 172.21.15.22:0/2960127012 conn(0x7fa3f40fd8c0 msgr2=0x7fa3f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:13.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.723+0000 7fa3fc348640 1 -- 172.21.15.22:0/2960127012 shutdown_connections 2024-09-19T08:15:13.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.723+0000 7fa3fc348640 1 -- 172.21.15.22:0/2960127012 wait complete. 2024-09-19T08:15:13.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fc348640 1 Processor -- start 2024-09-19T08:15:13.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fc348640 1 -- start start 2024-09-19T08:15:13.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fc348640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4101de0 0x7fa3f4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:13.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fc348640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f4102790 0x7fa3f4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:13.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fc348640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f4106f10 0x7fa3f419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:13.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fc348640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4111f70 con 0x7fa3f4101de0 2024-09-19T08:15:13.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fc348640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4111df0 con 0x7fa3f4106f10 2024-09-19T08:15:13.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fc348640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41120f0 con 0x7fa3f4102790 2024-09-19T08:15:13.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fa0e4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4101de0 0x7fa3f4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:13.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fa0e4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4101de0 0x7fa3f4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42960/0 (socket says 172.21.15.22:42960) 2024-09-19T08:15:13.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fa0e4640 1 -- 172.21.15.22:0/1230974404 learned_addr learned my addr 172.21.15.22:0/1230974404 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:13.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fa8e5640 1 --2- 172.21.15.22:0/1230974404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f4106f10 0x7fa3f419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:13.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3f98e3640 1 --2- 172.21.15.22:0/1230974404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f4102790 0x7fa3f4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:13.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fa0e4640 1 -- 172.21.15.22:0/1230974404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f4102790 msgr2=0x7fa3f4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:13.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fa0e4640 1 --2- 172.21.15.22:0/1230974404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f4102790 0x7fa3f4198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:13.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fa0e4640 1 -- 172.21.15.22:0/1230974404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f4106f10 msgr2=0x7fa3f419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:13.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fa0e4640 1 --2- 172.21.15.22:0/1230974404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f4106f10 0x7fa3f419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:13.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.724+0000 7fa3fa0e4640 1 -- 172.21.15.22:0/1230974404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f419d3e0 con 0x7fa3f4101de0 2024-09-19T08:15:13.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.725+0000 7fa3fa0e4640 1 --2- 172.21.15.22:0/1230974404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4101de0 0x7fa3f4198430 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000d950 tx=0x7fa3f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:13.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.725+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/1230974404 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0017070 con 0x7fa3f4101de0 2024-09-19T08:15:13.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.725+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/1230974404 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3f0009c30 con 0x7fa3f4101de0 2024-09-19T08:15:13.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.725+0000 7fa3fc348640 1 -- 172.21.15.22:0/1230974404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f419d6d0 con 0x7fa3f4101de0 2024-09-19T08:15:13.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.725+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/1230974404 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f000f6b0 con 0x7fa3f4101de0 2024-09-19T08:15:13.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.725+0000 7fa3fc348640 1 -- 172.21.15.22:0/1230974404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f41a4fb0 con 0x7fa3f4101de0 2024-09-19T08:15:13.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.726+0000 7fa3fc348640 1 -- 172.21.15.22:0/1230974404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f41021c0 con 0x7fa3f4101de0 2024-09-19T08:15:13.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.727+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/1230974404 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fa3f00109d0 con 0x7fa3f4101de0 2024-09-19T08:15:13.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.728+0000 7fa3eb7fe640 1 --2- 172.21.15.22:0/1230974404 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3e00a3550 0x7fa3e00a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:13.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.728+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/1230974404 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa3f00cfc00 con 0x7fa3f4101de0 2024-09-19T08:15:13.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.728+0000 7fa3f98e3640 1 --2- 172.21.15.22:0/1230974404 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3e00a3550 0x7fa3e00a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:13.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.728+0000 7fa3f98e3640 1 --2- 172.21.15.22:0/1230974404 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3e00a3550 0x7fa3e00a5a10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40096c0 tx=0x7fa3e4000bc0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:13.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.732+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/1230974404 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa3f00d6080 con 0x7fa3f4101de0 2024-09-19T08:15:13.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:13 smithi157 ceph-mon[35143]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:15:13.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:13 smithi157 ceph-mon[35143]: from='client.24991 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:15:13.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:13 smithi064 ceph-mon[34900]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:15:13.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:13 smithi064 ceph-mon[34900]: from='client.24991 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:15:13.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.908+0000 7fa3fc348640 1 -- 172.21.15.22:0/1230974404 --> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3f4062630 con 0x7fa3e00a3550 2024-09-19T08:15:13.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.914+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/1230974404 <== mgr.15141 v2:172.21.15.22:6800/497251935 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258053 (secure 0 0 0) 0x7fa3f4062630 con 0x7fa3e00a3550 2024-09-19T08:15:13.917 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:13.918 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T08:15:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.917+0000 7fa3fc348640 1 -- 172.21.15.22:0/1230974404 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3e00a3550 msgr2=0x7fa3e00a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.917+0000 7fa3fc348640 1 --2- 172.21.15.22:0/1230974404 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3e00a3550 0x7fa3e00a5a10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40096c0 tx=0x7fa3e4000bc0 comp rx=0 tx=0).stop 2024-09-19T08:15:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.917+0000 7fa3fc348640 1 -- 172.21.15.22:0/1230974404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4101de0 msgr2=0x7fa3f4198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.917+0000 7fa3fc348640 1 --2- 172.21.15.22:0/1230974404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4101de0 0x7fa3f4198430 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000d950 tx=0x7fa3f000de10 comp rx=0 tx=0).stop 2024-09-19T08:15:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.918+0000 7fa3fc348640 1 -- 172.21.15.22:0/1230974404 shutdown_connections 2024-09-19T08:15:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.918+0000 7fa3fc348640 1 --2- 172.21.15.22:0/1230974404 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3e00a3550 0x7fa3e00a5a10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.918+0000 7fa3fc348640 1 --2- 172.21.15.22:0/1230974404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f4106f10 0x7fa3f419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.918+0000 7fa3fc348640 1 --2- 172.21.15.22:0/1230974404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3f4102790 0x7fa3f4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.918+0000 7fa3fc348640 1 --2- 172.21.15.22:0/1230974404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4101de0 0x7fa3f4198430 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.918+0000 7fa3fc348640 1 -- 172.21.15.22:0/1230974404 >> 172.21.15.22:0/1230974404 conn(0x7fa3f40fd8c0 msgr2=0x7fa3f40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.918+0000 7fa3fc348640 1 -- 172.21.15.22:0/1230974404 shutdown_connections 2024-09-19T08:15:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:13.918+0000 7fa3fc348640 1 -- 172.21.15.22:0/1230974404 wait complete. 2024-09-19T08:15:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:14 smithi022 ceph-mon[30963]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-09-19T08:15:14.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:14 smithi022 ceph-mon[30963]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:14.580 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-19T08:15:13.187675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389488,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1318,"num_read_kb":1080,"num_write":626,"num_write_kb":5304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1242042,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":964,"ondisk_log_size":964,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381672,"kb_used_data":11596,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560152,"statfs":{"total":1151940427776,"available":1151549595648,"internally_reserved":0,"allocated":11874304,"data_stored":9421896,"data_compressed":317170,"data_compressed_allocated":417792,"data_compressed_original":4038656,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201076,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":398,"num_read_kb":460,"num_write":146,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.994869"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309162+0000","last_change":"2024-09-19T08:13:00.853947+0000","last_active":"2024-09-19T08:15:01.309162+0000","last_peered":"2024-09-19T08:15:01.309162+0000","last_clean":"2024-09-19T08:15:01.309162+0000","last_became_active":"2024-09-19T08:13:00.852951+0000","last_became_peered":"2024-09-19T08:13:00.852951+0000","last_unstale":"2024-09-19T08:15:01.309162+0000","last_undegraded":"2024-09-19T08:15:01.309162+0000","last_fullsized":"2024-09-19T08:15:01.309162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:25.786828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308625+0000","last_change":"2024-09-19T08:13:00.853389+0000","last_active":"2024-09-19T08:15:01.308625+0000","last_peered":"2024-09-19T08:15:01.308625+0000","last_clean":"2024-09-19T08:15:01.308625+0000","last_became_active":"2024-09-19T08:13:00.853102+0000","last_became_peered":"2024-09-19T08:13:00.853102+0000","last_unstale":"2024-09-19T08:15:01.308625+0000","last_undegraded":"2024-09-19T08:15:01.308625+0000","last_fullsized":"2024-09-19T08:15:01.308625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:42.773917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308869+0000","last_change":"2024-09-19T08:13:00.846544+0000","last_active":"2024-09-19T08:15:01.308869+0000","last_peered":"2024-09-19T08:15:01.308869+0000","last_clean":"2024-09-19T08:15:01.308869+0000","last_became_active":"2024-09-19T08:13:00.846229+0000","last_became_peered":"2024-09-19T08:13:00.846229+0000","last_unstale":"2024-09-19T08:15:01.308869+0000","last_undegraded":"2024-09-19T08:15:01.308869+0000","last_fullsized":"2024-09-19T08:15:01.308869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:43.471684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172938+0000","last_change":"2024-09-19T08:13:00.848149+0000","last_active":"2024-09-19T08:15:01.172938+0000","last_peered":"2024-09-19T08:15:01.172938+0000","last_clean":"2024-09-19T08:15:01.172938+0000","last_became_active":"2024-09-19T08:13:00.847744+0000","last_became_peered":"2024-09-19T08:13:00.847744+0000","last_unstale":"2024-09-19T08:15:01.172938+0000","last_undegraded":"2024-09-19T08:15:01.172938+0000","last_fullsized":"2024-09-19T08:15:01.172938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:01.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263000+0000","last_change":"2024-09-19T08:13:00.841862+0000","last_active":"2024-09-19T08:15:01.263000+0000","last_peered":"2024-09-19T08:15:01.263000+0000","last_clean":"2024-09-19T08:15:01.263000+0000","last_became_active":"2024-09-19T08:13:00.841331+0000","last_became_peered":"2024-09-19T08:13:00.841331+0000","last_unstale":"2024-09-19T08:15:01.263000+0000","last_undegraded":"2024-09-19T08:15:01.263000+0000","last_fullsized":"2024-09-19T08:15:01.263000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:54.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271477+0000","last_change":"2024-09-19T08:13:00.849626+0000","last_active":"2024-09-19T08:15:01.271477+0000","last_peered":"2024-09-19T08:15:01.271477+0000","last_clean":"2024-09-19T08:15:01.271477+0000","last_became_active":"2024-09-19T08:13:00.849034+0000","last_became_peered":"2024-09-19T08:13:00.849034+0000","last_unstale":"2024-09-19T08:15:01.271477+0000","last_undegraded":"2024-09-19T08:15:01.271477+0000","last_fullsized":"2024-09-19T08:15:01.271477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:39.118978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308024+0000","last_change":"2024-09-19T08:13:00.847522+0000","last_active":"2024-09-19T08:15:01.308024+0000","last_peered":"2024-09-19T08:15:01.308024+0000","last_clean":"2024-09-19T08:15:01.308024+0000","last_became_active":"2024-09-19T08:13:00.847117+0000","last_became_peered":"2024-09-19T08:13:00.847117+0000","last_unstale":"2024-09-19T08:15:01.308024+0000","last_undegraded":"2024-09-19T08:15:01.308024+0000","last_fullsized":"2024-09-19T08:15:01.308024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.836385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170733+0000","last_change":"2024-09-19T08:13:00.849544+0000","last_active":"2024-09-19T08:15:01.170733+0000","last_peered":"2024-09-19T08:15:01.170733+0000","last_clean":"2024-09-19T08:15:01.170733+0000","last_became_active":"2024-09-19T08:13:00.849170+0000","last_became_peered":"2024-09-19T08:13:00.849170+0000","last_unstale":"2024-09-19T08:15:01.170733+0000","last_undegraded":"2024-09-19T08:15:01.170733+0000","last_fullsized":"2024-09-19T08:15:01.170733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:28.371965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223727+0000","last_change":"2024-09-19T08:13:00.856052+0000","last_active":"2024-09-19T08:15:01.223727+0000","last_peered":"2024-09-19T08:15:01.223727+0000","last_clean":"2024-09-19T08:15:01.223727+0000","last_became_active":"2024-09-19T08:13:00.855291+0000","last_became_peered":"2024-09-19T08:13:00.855291+0000","last_unstale":"2024-09-19T08:15:01.223727+0000","last_undegraded":"2024-09-19T08:15:01.223727+0000","last_fullsized":"2024-09-19T08:15:01.223727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:18.581867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271887+0000","last_change":"2024-09-19T08:13:00.850145+0000","last_active":"2024-09-19T08:15:01.271887+0000","last_peered":"2024-09-19T08:15:01.271887+0000","last_clean":"2024-09-19T08:15:01.271887+0000","last_became_active":"2024-09-19T08:13:00.849798+0000","last_became_peered":"2024-09-19T08:13:00.849798+0000","last_unstale":"2024-09-19T08:15:01.271887+0000","last_undegraded":"2024-09-19T08:15:01.271887+0000","last_fullsized":"2024-09-19T08:15:01.271887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:21.400000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308275+0000","last_change":"2024-09-19T08:13:00.859153+0000","last_active":"2024-09-19T08:15:01.308275+0000","last_peered":"2024-09-19T08:15:01.308275+0000","last_clean":"2024-09-19T08:15:01.308275+0000","last_became_active":"2024-09-19T08:13:00.858473+0000","last_became_peered":"2024-09-19T08:13:00.858473+0000","last_unstale":"2024-09-19T08:15:01.308275+0000","last_undegraded":"2024-09-19T08:15:01.308275+0000","last_fullsized":"2024-09-19T08:15:01.308275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:07.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172922+0000","last_change":"2024-09-19T08:13:00.850797+0000","last_active":"2024-09-19T08:15:01.172922+0000","last_peered":"2024-09-19T08:15:01.172922+0000","last_clean":"2024-09-19T08:15:01.172922+0000","last_became_active":"2024-09-19T08:13:00.850393+0000","last_became_peered":"2024-09-19T08:13:00.850393+0000","last_unstale":"2024-09-19T08:15:01.172922+0000","last_undegraded":"2024-09-19T08:15:01.172922+0000","last_fullsized":"2024-09-19T08:15:01.172922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:59.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310082+0000","last_change":"2024-09-19T08:13:00.850835+0000","last_active":"2024-09-19T08:15:01.310082+0000","last_peered":"2024-09-19T08:15:01.310082+0000","last_clean":"2024-09-19T08:15:01.310082+0000","last_became_active":"2024-09-19T08:13:00.850518+0000","last_became_peered":"2024-09-19T08:13:00.850518+0000","last_unstale":"2024-09-19T08:15:01.310082+0000","last_undegraded":"2024-09-19T08:15:01.310082+0000","last_fullsized":"2024-09-19T08:15:01.310082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.178530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308450+0000","last_change":"2024-09-19T08:13:00.853934+0000","last_active":"2024-09-19T08:15:01.308450+0000","last_peered":"2024-09-19T08:15:01.308450+0000","last_clean":"2024-09-19T08:15:01.308450+0000","last_became_active":"2024-09-19T08:13:00.853604+0000","last_became_peered":"2024-09-19T08:13:00.853604+0000","last_unstale":"2024-09-19T08:15:01.308450+0000","last_undegraded":"2024-09-19T08:15:01.308450+0000","last_fullsized":"2024-09-19T08:15:01.308450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:43.622526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308262+0000","last_change":"2024-09-19T08:14:07.741370+0000","last_active":"2024-09-19T08:15:01.308262+0000","last_peered":"2024-09-19T08:15:01.308262+0000","last_clean":"2024-09-19T08:15:01.308262+0000","last_became_active":"2024-09-19T08:14:07.741116+0000","last_became_peered":"2024-09-19T08:14:07.741116+0000","last_unstale":"2024-09-19T08:15:01.308262+0000","last_undegraded":"2024-09-19T08:15:01.308262+0000","last_fullsized":"2024-09-19T08:15:01.308262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:45.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223320+0000","last_change":"2024-09-19T08:13:00.850266+0000","last_active":"2024-09-19T08:15:01.223320+0000","last_peered":"2024-09-19T08:15:01.223320+0000","last_clean":"2024-09-19T08:15:01.223320+0000","last_became_active":"2024-09-19T08:13:00.849928+0000","last_became_peered":"2024-09-19T08:13:00.849928+0000","last_unstale":"2024-09-19T08:15:01.223320+0000","last_undegraded":"2024-09-19T08:15:01.223320+0000","last_fullsized":"2024-09-19T08:15:01.223320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:38.231331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309177+0000","last_change":"2024-09-19T08:13:00.852551+0000","last_active":"2024-09-19T08:15:01.309177+0000","last_peered":"2024-09-19T08:15:01.309177+0000","last_clean":"2024-09-19T08:15:01.309177+0000","last_became_active":"2024-09-19T08:13:00.852225+0000","last_became_peered":"2024-09-19T08:13:00.852225+0000","last_unstale":"2024-09-19T08:15:01.309177+0000","last_undegraded":"2024-09-19T08:15:01.309177+0000","last_fullsized":"2024-09-19T08:15:01.309177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:33.115108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.261006+0000","last_change":"2024-09-19T08:13:00.854033+0000","last_active":"2024-09-19T08:15:01.261006+0000","last_peered":"2024-09-19T08:15:01.261006+0000","last_clean":"2024-09-19T08:15:01.261006+0000","last_became_active":"2024-09-19T08:13:00.853660+0000","last_became_peered":"2024-09-19T08:13:00.853660+0000","last_unstale":"2024-09-19T08:15:01.261006+0000","last_undegraded":"2024-09-19T08:15:01.261006+0000","last_fullsized":"2024-09-19T08:15:01.261006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:35.602420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308765+0000","last_change":"2024-09-19T08:13:00.850658+0000","last_active":"2024-09-19T08:15:01.308765+0000","last_peered":"2024-09-19T08:15:01.308765+0000","last_clean":"2024-09-19T08:15:01.308765+0000","last_became_active":"2024-09-19T08:13:00.850324+0000","last_became_peered":"2024-09-19T08:13:00.850324+0000","last_unstale":"2024-09-19T08:15:01.308765+0000","last_undegraded":"2024-09-19T08:15:01.308765+0000","last_fullsized":"2024-09-19T08:15:01.308765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:59:49.004605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308718+0000","last_change":"2024-09-19T08:14:07.741837+0000","last_active":"2024-09-19T08:15:01.308718+0000","last_peered":"2024-09-19T08:15:01.308718+0000","last_clean":"2024-09-19T08:15:01.308718+0000","last_became_active":"2024-09-19T08:14:07.741398+0000","last_became_peered":"2024-09-19T08:14:07.741398+0000","last_unstale":"2024-09-19T08:15:01.308718+0000","last_undegraded":"2024-09-19T08:15:01.308718+0000","last_fullsized":"2024-09-19T08:15:01.308718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:56.716663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310236+0000","last_change":"2024-09-19T08:13:00.855215+0000","last_active":"2024-09-19T08:15:01.310236+0000","last_peered":"2024-09-19T08:15:01.310236+0000","last_clean":"2024-09-19T08:15:01.310236+0000","last_became_active":"2024-09-19T08:13:00.854878+0000","last_became_peered":"2024-09-19T08:13:00.854878+0000","last_unstale":"2024-09-19T08:15:01.310236+0000","last_undegraded":"2024-09-19T08:15:01.310236+0000","last_fullsized":"2024-09-19T08:15:01.310236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:48.216066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223444+0000","last_change":"2024-09-19T08:13:00.856995+0000","last_active":"2024-09-19T08:15:01.223444+0000","last_peered":"2024-09-19T08:15:01.223444+0000","last_clean":"2024-09-19T08:15:01.223444+0000","last_became_active":"2024-09-19T08:13:00.856645+0000","last_became_peered":"2024-09-19T08:13:00.856645+0000","last_unstale":"2024-09-19T08:15:01.223444+0000","last_undegraded":"2024-09-19T08:15:01.223444+0000","last_fullsized":"2024-09-19T08:15:01.223444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:20.699293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170753+0000","last_change":"2024-09-19T08:13:00.853362+0000","last_active":"2024-09-19T08:15:01.170753+0000","last_peered":"2024-09-19T08:15:01.170753+0000","last_clean":"2024-09-19T08:15:01.170753+0000","last_became_active":"2024-09-19T08:13:00.853046+0000","last_became_peered":"2024-09-19T08:13:00.853046+0000","last_unstale":"2024-09-19T08:15:01.170753+0000","last_undegraded":"2024-09-19T08:15:01.170753+0000","last_fullsized":"2024-09-19T08:15:01.170753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172297+0000","last_change":"2024-09-19T08:13:00.845175+0000","last_active":"2024-09-19T08:15:01.172297+0000","last_peered":"2024-09-19T08:15:01.172297+0000","last_clean":"2024-09-19T08:15:01.172297+0000","last_became_active":"2024-09-19T08:13:00.844857+0000","last_became_peered":"2024-09-19T08:13:00.844857+0000","last_unstale":"2024-09-19T08:15:01.172297+0000","last_undegraded":"2024-09-19T08:15:01.172297+0000","last_fullsized":"2024-09-19T08:15:01.172297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:40.818956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171025+0000","last_change":"2024-09-19T08:13:00.847769+0000","last_active":"2024-09-19T08:15:01.171025+0000","last_peered":"2024-09-19T08:15:01.171025+0000","last_clean":"2024-09-19T08:15:01.171025+0000","last_became_active":"2024-09-19T08:13:00.847454+0000","last_became_peered":"2024-09-19T08:13:00.847454+0000","last_unstale":"2024-09-19T08:15:01.171025+0000","last_undegraded":"2024-09-19T08:15:01.171025+0000","last_fullsized":"2024-09-19T08:15:01.171025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.583051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308881+0000","last_change":"2024-09-19T08:13:00.859882+0000","last_active":"2024-09-19T08:15:01.308881+0000","last_peered":"2024-09-19T08:15:01.308881+0000","last_clean":"2024-09-19T08:15:01.308881+0000","last_became_active":"2024-09-19T08:13:00.859491+0000","last_became_peered":"2024-09-19T08:13:00.859491+0000","last_unstale":"2024-09-19T08:15:01.308881+0000","last_undegraded":"2024-09-19T08:15:01.308881+0000","last_fullsized":"2024-09-19T08:15:01.308881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:13.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308686+0000","last_change":"2024-09-19T08:13:00.845584+0000","last_active":"2024-09-19T08:15:01.308686+0000","last_peered":"2024-09-19T08:15:01.308686+0000","last_clean":"2024-09-19T08:15:01.308686+0000","last_became_active":"2024-09-19T08:13:00.845166+0000","last_became_peered":"2024-09-19T08:13:00.845166+0000","last_unstale":"2024-09-19T08:15:01.308686+0000","last_undegraded":"2024-09-19T08:15:01.308686+0000","last_fullsized":"2024-09-19T08:15:01.308686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:02.705793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310539+0000","last_change":"2024-09-19T08:13:00.838069+0000","last_active":"2024-09-19T08:15:01.310539+0000","last_peered":"2024-09-19T08:15:01.310539+0000","last_clean":"2024-09-19T08:15:01.310539+0000","last_became_active":"2024-09-19T08:13:00.837736+0000","last_became_peered":"2024-09-19T08:13:00.837736+0000","last_unstale":"2024-09-19T08:15:01.310539+0000","last_undegraded":"2024-09-19T08:15:01.310539+0000","last_fullsized":"2024-09-19T08:15:01.310539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:01.388290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308795+0000","last_change":"2024-09-19T08:13:00.841507+0000","last_active":"2024-09-19T08:15:01.308795+0000","last_peered":"2024-09-19T08:15:01.308795+0000","last_clean":"2024-09-19T08:15:01.308795+0000","last_became_active":"2024-09-19T08:13:00.841161+0000","last_became_peered":"2024-09-19T08:13:00.841161+0000","last_unstale":"2024-09-19T08:15:01.308795+0000","last_undegraded":"2024-09-19T08:15:01.308795+0000","last_fullsized":"2024-09-19T08:15:01.308795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:41.862290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308530+0000","last_change":"2024-09-19T08:13:00.847657+0000","last_active":"2024-09-19T08:15:01.308530+0000","last_peered":"2024-09-19T08:15:01.308530+0000","last_clean":"2024-09-19T08:15:01.308530+0000","last_became_active":"2024-09-19T08:13:00.846849+0000","last_became_peered":"2024-09-19T08:13:00.846849+0000","last_unstale":"2024-09-19T08:15:01.308530+0000","last_undegraded":"2024-09-19T08:15:01.308530+0000","last_fullsized":"2024-09-19T08:15:01.308530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:35.219913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309056+0000","last_change":"2024-09-19T08:13:00.845971+0000","last_active":"2024-09-19T08:15:01.309056+0000","last_peered":"2024-09-19T08:15:01.309056+0000","last_clean":"2024-09-19T08:15:01.309056+0000","last_became_active":"2024-09-19T08:13:00.845626+0000","last_became_peered":"2024-09-19T08:13:00.845626+0000","last_unstale":"2024-09-19T08:15:01.309056+0000","last_undegraded":"2024-09-19T08:15:01.309056+0000","last_fullsized":"2024-09-19T08:15:01.309056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:36.945627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308432+0000","last_change":"2024-09-19T08:14:07.738450+0000","last_active":"2024-09-19T08:15:01.308432+0000","last_peered":"2024-09-19T08:15:01.308432+0000","last_clean":"2024-09-19T08:15:01.308432+0000","last_became_active":"2024-09-19T08:14:07.738028+0000","last_became_peered":"2024-09-19T08:14:07.738028+0000","last_unstale":"2024-09-19T08:15:01.308432+0000","last_undegraded":"2024-09-19T08:15:01.308432+0000","last_fullsized":"2024-09-19T08:15:01.308432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:08.659158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262954+0000","last_change":"2024-09-19T08:13:00.848325+0000","last_active":"2024-09-19T08:15:01.262954+0000","last_peered":"2024-09-19T08:15:01.262954+0000","last_clean":"2024-09-19T08:15:01.262954+0000","last_became_active":"2024-09-19T08:13:00.848015+0000","last_became_peered":"2024-09-19T08:13:00.848015+0000","last_unstale":"2024-09-19T08:15:01.262954+0000","last_undegraded":"2024-09-19T08:15:01.262954+0000","last_fullsized":"2024-09-19T08:15:01.262954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:10.948451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'9","reported_seq":160,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.051779+0000","last_change":"2024-09-19T08:12:58.681134+0000","last_active":"2024-09-19T08:15:11.051779+0000","last_peered":"2024-09-19T08:15:11.051779+0000","last_clean":"2024-09-19T08:15:11.051779+0000","last_became_active":"2024-09-19T08:12:58.680764+0000","last_became_peered":"2024-09-19T08:12:58.680764+0000","last_unstale":"2024-09-19T08:15:11.051779+0000","last_undegraded":"2024-09-19T08:15:11.051779+0000","last_fullsized":"2024-09-19T08:15:11.051779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:24.939263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172533+0000","last_change":"2024-09-19T08:13:00.844319+0000","last_active":"2024-09-19T08:15:01.172533+0000","last_peered":"2024-09-19T08:15:01.172533+0000","last_clean":"2024-09-19T08:15:01.172533+0000","last_became_active":"2024-09-19T08:13:00.843994+0000","last_became_peered":"2024-09-19T08:13:00.843994+0000","last_unstale":"2024-09-19T08:15:01.172533+0000","last_undegraded":"2024-09-19T08:15:01.172533+0000","last_fullsized":"2024-09-19T08:15:01.172533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:46.937466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271532+0000","last_change":"2024-09-19T08:12:58.676541+0000","last_active":"2024-09-19T08:15:01.271532+0000","last_peered":"2024-09-19T08:15:01.271532+0000","last_clean":"2024-09-19T08:15:01.271532+0000","last_became_active":"2024-09-19T08:12:58.675988+0000","last_became_peered":"2024-09-19T08:12:58.675988+0000","last_unstale":"2024-09-19T08:15:01.271532+0000","last_undegraded":"2024-09-19T08:15:01.271532+0000","last_fullsized":"2024-09-19T08:15:01.271532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:01.434986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309788+0000","last_change":"2024-09-19T08:12:58.677510+0000","last_active":"2024-09-19T08:15:01.309788+0000","last_peered":"2024-09-19T08:15:01.309788+0000","last_clean":"2024-09-19T08:15:01.309788+0000","last_became_active":"2024-09-19T08:12:58.677200+0000","last_became_peered":"2024-09-19T08:12:58.677200+0000","last_unstale":"2024-09-19T08:15:01.309788+0000","last_undegraded":"2024-09-19T08:15:01.309788+0000","last_fullsized":"2024-09-19T08:15:01.309788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:36.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170825+0000","last_change":"2024-09-19T08:13:00.853430+0000","last_active":"2024-09-19T08:15:01.170825+0000","last_peered":"2024-09-19T08:15:01.170825+0000","last_clean":"2024-09-19T08:15:01.170825+0000","last_became_active":"2024-09-19T08:13:00.853096+0000","last_became_peered":"2024-09-19T08:13:00.853096+0000","last_unstale":"2024-09-19T08:15:01.170825+0000","last_undegraded":"2024-09-19T08:15:01.170825+0000","last_fullsized":"2024-09-19T08:15:01.170825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:07.147021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172384+0000","last_change":"2024-09-19T08:12:58.680081+0000","last_active":"2024-09-19T08:15:01.172384+0000","last_peered":"2024-09-19T08:15:01.172384+0000","last_clean":"2024-09-19T08:15:01.172384+0000","last_became_active":"2024-09-19T08:12:58.679704+0000","last_became_peered":"2024-09-19T08:12:58.679704+0000","last_unstale":"2024-09-19T08:15:01.172384+0000","last_undegraded":"2024-09-19T08:15:01.172384+0000","last_fullsized":"2024-09-19T08:15:01.172384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:32.942586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.260317+0000","last_change":"2024-09-19T08:13:00.849394+0000","last_active":"2024-09-19T08:15:01.260317+0000","last_peered":"2024-09-19T08:15:01.260317+0000","last_clean":"2024-09-19T08:15:01.260317+0000","last_became_active":"2024-09-19T08:13:00.849051+0000","last_became_peered":"2024-09-19T08:13:00.849051+0000","last_unstale":"2024-09-19T08:15:01.260317+0000","last_undegraded":"2024-09-19T08:15:01.260317+0000","last_fullsized":"2024-09-19T08:15:01.260317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:23.012928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'5","reported_seq":150,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.051165+0000","last_change":"2024-09-19T08:12:58.676121+0000","last_active":"2024-09-19T08:15:11.051165+0000","last_peered":"2024-09-19T08:15:11.051165+0000","last_clean":"2024-09-19T08:15:11.051165+0000","last_became_active":"2024-09-19T08:12:58.675693+0000","last_became_peered":"2024-09-19T08:12:58.675693+0000","last_unstale":"2024-09-19T08:15:11.051165+0000","last_undegraded":"2024-09-19T08:15:11.051165+0000","last_fullsized":"2024-09-19T08:15:11.051165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:34.882190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223583+0000","last_change":"2024-09-19T08:13:00.858735+0000","last_active":"2024-09-19T08:15:01.223583+0000","last_peered":"2024-09-19T08:15:01.223583+0000","last_clean":"2024-09-19T08:15:01.223583+0000","last_became_active":"2024-09-19T08:13:00.858218+0000","last_became_peered":"2024-09-19T08:13:00.858218+0000","last_unstale":"2024-09-19T08:15:01.223583+0000","last_undegraded":"2024-09-19T08:15:01.223583+0000","last_fullsized":"2024-09-19T08:15:01.223583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:33.356810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308176+0000","last_change":"2024-09-19T08:14:07.743785+0000","last_active":"2024-09-19T08:15:01.308176+0000","last_peered":"2024-09-19T08:15:01.308176+0000","last_clean":"2024-09-19T08:15:01.308176+0000","last_became_active":"2024-09-19T08:14:07.739531+0000","last_became_peered":"2024-09-19T08:14:07.739531+0000","last_unstale":"2024-09-19T08:15:01.308176+0000","last_undegraded":"2024-09-19T08:15:01.308176+0000","last_fullsized":"2024-09-19T08:15:01.308176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:43.367704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308976+0000","last_change":"2024-09-19T08:13:00.843852+0000","last_active":"2024-09-19T08:15:01.308976+0000","last_peered":"2024-09-19T08:15:01.308976+0000","last_clean":"2024-09-19T08:15:01.308976+0000","last_became_active":"2024-09-19T08:13:00.843497+0000","last_became_peered":"2024-09-19T08:13:00.843497+0000","last_unstale":"2024-09-19T08:15:01.308976+0000","last_undegraded":"2024-09-19T08:15:01.308976+0000","last_fullsized":"2024-09-19T08:15:01.308976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:26.380024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172698+0000","last_change":"2024-09-19T08:12:58.717225+0000","last_active":"2024-09-19T08:15:01.172698+0000","last_peered":"2024-09-19T08:15:01.172698+0000","last_clean":"2024-09-19T08:15:01.172698+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:15:01.172698+0000","last_undegraded":"2024-09-19T08:15:01.172698+0000","last_fullsized":"2024-09-19T08:15:01.172698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:43.201907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223446+0000","last_change":"2024-09-19T08:13:00.845613+0000","last_active":"2024-09-19T08:15:01.223446+0000","last_peered":"2024-09-19T08:15:01.223446+0000","last_clean":"2024-09-19T08:15:01.223446+0000","last_became_active":"2024-09-19T08:13:00.845260+0000","last_became_peered":"2024-09-19T08:13:00.845260+0000","last_unstale":"2024-09-19T08:15:01.223446+0000","last_undegraded":"2024-09-19T08:15:01.223446+0000","last_fullsized":"2024-09-19T08:15:01.223446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:03:47.033972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'39","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.834802+0000","last_change":"2024-09-19T08:14:07.749616+0000","last_active":"2024-09-19T08:15:11.834802+0000","last_peered":"2024-09-19T08:15:11.834802+0000","last_clean":"2024-09-19T08:15:11.834802+0000","last_became_active":"2024-09-19T08:14:07.740298+0000","last_became_peered":"2024-09-19T08:14:07.740298+0000","last_unstale":"2024-09-19T08:15:11.834802+0000","last_undegraded":"2024-09-19T08:15:11.834802+0000","last_fullsized":"2024-09-19T08:15:11.834802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:28.796855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70880,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":249,"num_read_kb":194,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61172,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263216+0000","last_change":"2024-09-19T08:13:00.855603+0000","last_active":"2024-09-19T08:15:01.263216+0000","last_peered":"2024-09-19T08:15:01.263216+0000","last_clean":"2024-09-19T08:15:01.263216+0000","last_became_active":"2024-09-19T08:13:00.855241+0000","last_became_peered":"2024-09-19T08:13:00.855241+0000","last_unstale":"2024-09-19T08:15:01.263216+0000","last_undegraded":"2024-09-19T08:15:01.263216+0000","last_fullsized":"2024-09-19T08:15:01.263216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:28.811847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308653+0000","last_change":"2024-09-19T08:12:58.675897+0000","last_active":"2024-09-19T08:15:01.308653+0000","last_peered":"2024-09-19T08:15:01.308653+0000","last_clean":"2024-09-19T08:15:01.308653+0000","last_became_active":"2024-09-19T08:12:58.675525+0000","last_became_peered":"2024-09-19T08:12:58.675525+0000","last_unstale":"2024-09-19T08:15:01.308653+0000","last_undegraded":"2024-09-19T08:15:01.308653+0000","last_fullsized":"2024-09-19T08:15:01.308653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:54.518064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309636+0000","last_change":"2024-09-19T08:13:00.856420+0000","last_active":"2024-09-19T08:15:01.309636+0000","last_peered":"2024-09-19T08:15:01.309636+0000","last_clean":"2024-09-19T08:15:01.309636+0000","last_became_active":"2024-09-19T08:13:00.856100+0000","last_became_peered":"2024-09-19T08:13:00.856100+0000","last_unstale":"2024-09-19T08:15:01.309636+0000","last_undegraded":"2024-09-19T08:15:01.309636+0000","last_fullsized":"2024-09-19T08:15:01.309636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:58.684930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172292+0000","last_change":"2024-09-19T08:12:58.680778+0000","last_active":"2024-09-19T08:15:01.172292+0000","last_peered":"2024-09-19T08:15:01.172292+0000","last_clean":"2024-09-19T08:15:01.172292+0000","last_became_active":"2024-09-19T08:12:58.680364+0000","last_became_peered":"2024-09-19T08:12:58.680364+0000","last_unstale":"2024-09-19T08:15:01.172292+0000","last_undegraded":"2024-09-19T08:15:01.172292+0000","last_fullsized":"2024-09-19T08:15:01.172292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:10.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308288+0000","last_change":"2024-09-19T08:13:00.853415+0000","last_active":"2024-09-19T08:15:01.308288+0000","last_peered":"2024-09-19T08:15:01.308288+0000","last_clean":"2024-09-19T08:15:01.308288+0000","last_became_active":"2024-09-19T08:13:00.853101+0000","last_became_peered":"2024-09-19T08:13:00.853101+0000","last_unstale":"2024-09-19T08:15:01.308288+0000","last_undegraded":"2024-09-19T08:15:01.308288+0000","last_fullsized":"2024-09-19T08:15:01.308288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:42.375394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":159,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.050571+0000","last_change":"2024-09-19T08:12:58.680312+0000","last_active":"2024-09-19T08:15:11.050571+0000","last_peered":"2024-09-19T08:15:11.050571+0000","last_clean":"2024-09-19T08:15:11.050571+0000","last_became_active":"2024-09-19T08:12:58.679870+0000","last_became_peered":"2024-09-19T08:12:58.679870+0000","last_unstale":"2024-09-19T08:15:11.050571+0000","last_undegraded":"2024-09-19T08:15:11.050571+0000","last_fullsized":"2024-09-19T08:15:11.050571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:42.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308034+0000","last_change":"2024-09-19T08:13:00.853920+0000","last_active":"2024-09-19T08:15:01.308034+0000","last_peered":"2024-09-19T08:15:01.308034+0000","last_clean":"2024-09-19T08:15:01.308034+0000","last_became_active":"2024-09-19T08:13:00.853594+0000","last_became_peered":"2024-09-19T08:13:00.853594+0000","last_unstale":"2024-09-19T08:15:01.308034+0000","last_undegraded":"2024-09-19T08:15:01.308034+0000","last_fullsized":"2024-09-19T08:15:01.308034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:31.746133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308308+0000","last_change":"2024-09-19T08:14:07.751888+0000","last_active":"2024-09-19T08:15:01.308308+0000","last_peered":"2024-09-19T08:15:01.308308+0000","last_clean":"2024-09-19T08:15:01.308308+0000","last_became_active":"2024-09-19T08:14:07.737201+0000","last_became_peered":"2024-09-19T08:14:07.737201+0000","last_unstale":"2024-09-19T08:15:01.308308+0000","last_undegraded":"2024-09-19T08:15:01.308308+0000","last_fullsized":"2024-09-19T08:15:01.308308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:44.095987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172245+0000","last_change":"2024-09-19T08:13:00.847061+0000","last_active":"2024-09-19T08:15:01.172245+0000","last_peered":"2024-09-19T08:15:01.172245+0000","last_clean":"2024-09-19T08:15:01.172245+0000","last_became_active":"2024-09-19T08:13:00.846742+0000","last_became_peered":"2024-09-19T08:13:00.846742+0000","last_unstale":"2024-09-19T08:15:01.172245+0000","last_undegraded":"2024-09-19T08:15:01.172245+0000","last_fullsized":"2024-09-19T08:15:01.172245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:21.785585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309033+0000","last_change":"2024-09-19T08:14:07.755277+0000","last_active":"2024-09-19T08:15:01.309033+0000","last_peered":"2024-09-19T08:15:01.309033+0000","last_clean":"2024-09-19T08:15:01.309033+0000","last_became_active":"2024-09-19T08:14:07.739773+0000","last_became_peered":"2024-09-19T08:14:07.739773+0000","last_unstale":"2024-09-19T08:15:01.309033+0000","last_undegraded":"2024-09-19T08:15:01.309033+0000","last_fullsized":"2024-09-19T08:15:01.309033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:56.848251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308865+0000","last_change":"2024-09-19T08:13:00.844940+0000","last_active":"2024-09-19T08:15:01.308865+0000","last_peered":"2024-09-19T08:15:01.308865+0000","last_clean":"2024-09-19T08:15:01.308865+0000","last_became_active":"2024-09-19T08:13:00.844454+0000","last_became_peered":"2024-09-19T08:13:00.844454+0000","last_unstale":"2024-09-19T08:15:01.308865+0000","last_undegraded":"2024-09-19T08:15:01.308865+0000","last_fullsized":"2024-09-19T08:15:01.308865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:15.577624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309967+0000","last_change":"2024-09-19T08:12:58.673301+0000","last_active":"2024-09-19T08:15:01.309967+0000","last_peered":"2024-09-19T08:15:01.309967+0000","last_clean":"2024-09-19T08:15:01.309967+0000","last_became_active":"2024-09-19T08:12:58.672981+0000","last_became_peered":"2024-09-19T08:12:58.672981+0000","last_unstale":"2024-09-19T08:15:01.309967+0000","last_undegraded":"2024-09-19T08:15:01.309967+0000","last_fullsized":"2024-09-19T08:15:01.309967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:03.963712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308223+0000","last_change":"2024-09-19T08:13:00.856620+0000","last_active":"2024-09-19T08:15:01.308223+0000","last_peered":"2024-09-19T08:15:01.308223+0000","last_clean":"2024-09-19T08:15:01.308223+0000","last_became_active":"2024-09-19T08:13:00.856310+0000","last_became_peered":"2024-09-19T08:13:00.856310+0000","last_unstale":"2024-09-19T08:15:01.308223+0000","last_undegraded":"2024-09-19T08:15:01.308223+0000","last_fullsized":"2024-09-19T08:15:01.308223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:16.918768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'65","reported_seq":118,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.911955+0000","last_change":"2024-09-19T08:12:58.672282+0000","last_active":"2024-09-19T08:15:11.911955+0000","last_peered":"2024-09-19T08:15:11.911955+0000","last_clean":"2024-09-19T08:15:11.911955+0000","last_became_active":"2024-09-19T08:12:58.671891+0000","last_became_peered":"2024-09-19T08:12:58.671891+0000","last_unstale":"2024-09-19T08:15:11.911955+0000","last_undegraded":"2024-09-19T08:15:11.911955+0000","last_fullsized":"2024-09-19T08:15:11.911955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:41.731571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":31,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308179+0000","last_change":"2024-09-19T08:13:00.848107+0000","last_active":"2024-09-19T08:15:01.308179+0000","last_peered":"2024-09-19T08:15:01.308179+0000","last_clean":"2024-09-19T08:15:01.308179+0000","last_became_active":"2024-09-19T08:13:00.847632+0000","last_became_peered":"2024-09-19T08:13:00.847632+0000","last_unstale":"2024-09-19T08:15:01.308179+0000","last_undegraded":"2024-09-19T08:15:01.308179+0000","last_fullsized":"2024-09-19T08:15:01.308179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.491122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308878+0000","last_change":"2024-09-19T08:12:59.179200+0000","last_active":"2024-09-19T08:15:01.308878+0000","last_peered":"2024-09-19T08:15:01.308878+0000","last_clean":"2024-09-19T08:15:01.308878+0000","last_became_active":"2024-09-19T08:12:59.178762+0000","last_became_peered":"2024-09-19T08:12:59.178762+0000","last_unstale":"2024-09-19T08:15:01.308878+0000","last_undegraded":"2024-09-19T08:15:01.308878+0000","last_fullsized":"2024-09-19T08:15:01.308878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:08.424106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308358+0000","last_change":"2024-09-19T08:13:00.855870+0000","last_active":"2024-09-19T08:15:01.308358+0000","last_peered":"2024-09-19T08:15:01.308358+0000","last_clean":"2024-09-19T08:15:01.308358+0000","last_became_active":"2024-09-19T08:13:00.855538+0000","last_became_peered":"2024-09-19T08:13:00.855538+0000","last_unstale":"2024-09-19T08:15:01.308358+0000","last_undegraded":"2024-09-19T08:15:01.308358+0000","last_fullsized":"2024-09-19T08:15:01.308358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"86'148","reported_seq":395,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:03.309732+0000","last_change":"2024-09-19T08:11:48.661333+0000","last_active":"2024-09-19T08:15:03.309732+0000","last_peered":"2024-09-19T08:15:03.309732+0000","last_clean":"2024-09-19T08:15:03.309732+0000","last_became_active":"2024-09-19T08:11:48.660873+0000","last_became_peered":"2024-09-19T08:11:48.660873+0000","last_unstale":"2024-09-19T08:15:03.309732+0000","last_undegraded":"2024-09-19T08:15:03.309732+0000","last_fullsized":"2024-09-19T08:15:03.309732+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_clean_scrub_stamp":"2024-09-19T08:08:25.372155+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:19.401292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172985+0000","last_change":"2024-09-19T08:12:58.675237+0000","last_active":"2024-09-19T08:15:01.172985+0000","last_peered":"2024-09-19T08:15:01.172985+0000","last_clean":"2024-09-19T08:15:01.172985+0000","last_became_active":"2024-09-19T08:12:58.674910+0000","last_became_peered":"2024-09-19T08:12:58.674910+0000","last_unstale":"2024-09-19T08:15:01.172985+0000","last_undegraded":"2024-09-19T08:15:01.172985+0000","last_fullsized":"2024-09-19T08:15:01.172985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:47.038328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171081+0000","last_change":"2024-09-19T08:13:00.846754+0000","last_active":"2024-09-19T08:15:01.171081+0000","last_peered":"2024-09-19T08:15:01.171081+0000","last_clean":"2024-09-19T08:15:01.171081+0000","last_became_active":"2024-09-19T08:13:00.846452+0000","last_became_peered":"2024-09-19T08:13:00.846452+0000","last_unstale":"2024-09-19T08:15:01.171081+0000","last_undegraded":"2024-09-19T08:15:01.171081+0000","last_fullsized":"2024-09-19T08:15:01.171081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:22.929689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308239+0000","last_change":"2024-09-19T08:12:58.678958+0000","last_active":"2024-09-19T08:15:01.308239+0000","last_peered":"2024-09-19T08:15:01.308239+0000","last_clean":"2024-09-19T08:15:01.308239+0000","last_became_active":"2024-09-19T08:12:58.678592+0000","last_became_peered":"2024-09-19T08:12:58.678592+0000","last_unstale":"2024-09-19T08:15:01.308239+0000","last_undegraded":"2024-09-19T08:15:01.308239+0000","last_fullsized":"2024-09-19T08:15:01.308239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:58.374606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309700+0000","last_change":"2024-09-19T08:13:00.840973+0000","last_active":"2024-09-19T08:15:01.309700+0000","last_peered":"2024-09-19T08:15:01.309700+0000","last_clean":"2024-09-19T08:15:01.309700+0000","last_became_active":"2024-09-19T08:13:00.840619+0000","last_became_peered":"2024-09-19T08:13:00.840619+0000","last_unstale":"2024-09-19T08:15:01.309700+0000","last_undegraded":"2024-09-19T08:15:01.309700+0000","last_fullsized":"2024-09-19T08:15:01.309700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":152,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.050056+0000","last_change":"2024-09-19T08:12:58.715972+0000","last_active":"2024-09-19T08:15:11.050056+0000","last_peered":"2024-09-19T08:15:11.050056+0000","last_clean":"2024-09-19T08:15:11.050056+0000","last_became_active":"2024-09-19T08:12:58.715630+0000","last_became_peered":"2024-09-19T08:12:58.715630+0000","last_unstale":"2024-09-19T08:15:11.050056+0000","last_undegraded":"2024-09-19T08:15:11.050056+0000","last_fullsized":"2024-09-19T08:15:11.050056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:35.637121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172244+0000","last_change":"2024-09-19T08:13:00.847023+0000","last_active":"2024-09-19T08:15:01.172244+0000","last_peered":"2024-09-19T08:15:01.172244+0000","last_clean":"2024-09-19T08:15:01.172244+0000","last_became_active":"2024-09-19T08:13:00.846387+0000","last_became_peered":"2024-09-19T08:13:00.846387+0000","last_unstale":"2024-09-19T08:15:01.172244+0000","last_undegraded":"2024-09-19T08:15:01.172244+0000","last_fullsized":"2024-09-19T08:15:01.172244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:12.187929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308283+0000","last_change":"2024-09-19T08:12:58.681062+0000","last_active":"2024-09-19T08:15:01.308283+0000","last_peered":"2024-09-19T08:15:01.308283+0000","last_clean":"2024-09-19T08:15:01.308283+0000","last_became_active":"2024-09-19T08:12:58.680524+0000","last_became_peered":"2024-09-19T08:12:58.680524+0000","last_unstale":"2024-09-19T08:15:01.308283+0000","last_undegraded":"2024-09-19T08:15:01.308283+0000","last_fullsized":"2024-09-19T08:15:01.308283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271807+0000","last_change":"2024-09-19T08:13:00.851523+0000","last_active":"2024-09-19T08:15:01.271807+0000","last_peered":"2024-09-19T08:15:01.271807+0000","last_clean":"2024-09-19T08:15:01.271807+0000","last_became_active":"2024-09-19T08:13:00.851068+0000","last_became_peered":"2024-09-19T08:13:00.851068+0000","last_unstale":"2024-09-19T08:15:01.271807+0000","last_undegraded":"2024-09-19T08:15:01.271807+0000","last_fullsized":"2024-09-19T08:15:01.271807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:44.905469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.260925+0000","last_change":"2024-09-19T08:12:58.718234+0000","last_active":"2024-09-19T08:15:01.260925+0000","last_peered":"2024-09-19T08:15:01.260925+0000","last_clean":"2024-09-19T08:15:01.260925+0000","last_became_active":"2024-09-19T08:12:58.717840+0000","last_became_peered":"2024-09-19T08:12:58.717840+0000","last_unstale":"2024-09-19T08:15:01.260925+0000","last_undegraded":"2024-09-19T08:15:01.260925+0000","last_fullsized":"2024-09-19T08:15:01.260925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:05.283770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271686+0000","last_change":"2024-09-19T08:13:00.848305+0000","last_active":"2024-09-19T08:15:01.271686+0000","last_peered":"2024-09-19T08:15:01.271686+0000","last_clean":"2024-09-19T08:15:01.271686+0000","last_became_active":"2024-09-19T08:13:00.847981+0000","last_became_peered":"2024-09-19T08:13:00.847981+0000","last_unstale":"2024-09-19T08:15:01.271686+0000","last_undegraded":"2024-09-19T08:15:01.271686+0000","last_fullsized":"2024-09-19T08:15:01.271686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:04.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263351+0000","last_change":"2024-09-19T08:12:58.715943+0000","last_active":"2024-09-19T08:15:01.263351+0000","last_peered":"2024-09-19T08:15:01.263351+0000","last_clean":"2024-09-19T08:15:01.263351+0000","last_became_active":"2024-09-19T08:12:58.715600+0000","last_became_peered":"2024-09-19T08:12:58.715600+0000","last_unstale":"2024-09-19T08:15:01.263351+0000","last_undegraded":"2024-09-19T08:15:01.263351+0000","last_fullsized":"2024-09-19T08:15:01.263351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:19.142271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310659+0000","last_change":"2024-09-19T08:13:00.840944+0000","last_active":"2024-09-19T08:15:01.310659+0000","last_peered":"2024-09-19T08:15:01.310659+0000","last_clean":"2024-09-19T08:15:01.310659+0000","last_became_active":"2024-09-19T08:13:00.840608+0000","last_became_peered":"2024-09-19T08:13:00.840608+0000","last_unstale":"2024-09-19T08:15:01.310659+0000","last_undegraded":"2024-09-19T08:15:01.310659+0000","last_fullsized":"2024-09-19T08:15:01.310659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:25.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'97","reported_seq":208,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.918689+0000","last_change":"2024-09-19T08:12:58.672929+0000","last_active":"2024-09-19T08:15:11.918689+0000","last_peered":"2024-09-19T08:15:11.918689+0000","last_clean":"2024-09-19T08:15:11.918689+0000","last_became_active":"2024-09-19T08:12:58.672487+0000","last_became_peered":"2024-09-19T08:12:58.672487+0000","last_unstale":"2024-09-19T08:15:11.918689+0000","last_undegraded":"2024-09-19T08:15:11.918689+0000","last_fullsized":"2024-09-19T08:15:11.918689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:33.042735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":177,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.261959+0000","last_change":"2024-09-19T08:14:07.738260+0000","last_active":"2024-09-19T08:15:01.261959+0000","last_peered":"2024-09-19T08:15:01.261959+0000","last_clean":"2024-09-19T08:15:01.261959+0000","last_became_active":"2024-09-19T08:14:07.737915+0000","last_became_peered":"2024-09-19T08:14:07.737915+0000","last_unstale":"2024-09-19T08:15:01.261959+0000","last_undegraded":"2024-09-19T08:15:01.261959+0000","last_fullsized":"2024-09-19T08:15:01.261959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:02.500657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.173307+0000","last_change":"2024-09-19T08:12:59.179510+0000","last_active":"2024-09-19T08:15:01.173307+0000","last_peered":"2024-09-19T08:15:01.173307+0000","last_clean":"2024-09-19T08:15:01.173307+0000","last_became_active":"2024-09-19T08:12:59.179157+0000","last_became_peered":"2024-09-19T08:12:59.179157+0000","last_unstale":"2024-09-19T08:15:01.173307+0000","last_undegraded":"2024-09-19T08:15:01.173307+0000","last_fullsized":"2024-09-19T08:15:01.173307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:22.981289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308670+0000","last_change":"2024-09-19T08:13:00.841853+0000","last_active":"2024-09-19T08:15:01.308670+0000","last_peered":"2024-09-19T08:15:01.308670+0000","last_clean":"2024-09-19T08:15:01.308670+0000","last_became_active":"2024-09-19T08:13:00.841518+0000","last_became_peered":"2024-09-19T08:13:00.841518+0000","last_unstale":"2024-09-19T08:15:01.308670+0000","last_undegraded":"2024-09-19T08:15:01.308670+0000","last_fullsized":"2024-09-19T08:15:01.308670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:05.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223446+0000","last_change":"2024-09-19T08:12:58.716882+0000","last_active":"2024-09-19T08:15:01.223446+0000","last_peered":"2024-09-19T08:15:01.223446+0000","last_clean":"2024-09-19T08:15:01.223446+0000","last_became_active":"2024-09-19T08:12:58.716545+0000","last_became_peered":"2024-09-19T08:12:58.716545+0000","last_unstale":"2024-09-19T08:15:01.223446+0000","last_undegraded":"2024-09-19T08:15:01.223446+0000","last_fullsized":"2024-09-19T08:15:01.223446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:14.469109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262999+0000","last_change":"2024-09-19T08:13:00.850484+0000","last_active":"2024-09-19T08:15:01.262999+0000","last_peered":"2024-09-19T08:15:01.262999+0000","last_clean":"2024-09-19T08:15:01.262999+0000","last_became_active":"2024-09-19T08:13:00.850049+0000","last_became_peered":"2024-09-19T08:13:00.850049+0000","last_unstale":"2024-09-19T08:15:01.262999+0000","last_undegraded":"2024-09-19T08:15:01.262999+0000","last_fullsized":"2024-09-19T08:15:01.262999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:24.424111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170992+0000","last_change":"2024-09-19T08:12:58.669928+0000","last_active":"2024-09-19T08:15:01.170992+0000","last_peered":"2024-09-19T08:15:01.170992+0000","last_clean":"2024-09-19T08:15:01.170992+0000","last_became_active":"2024-09-19T08:12:58.669614+0000","last_became_peered":"2024-09-19T08:12:58.669614+0000","last_unstale":"2024-09-19T08:15:01.170992+0000","last_undegraded":"2024-09-19T08:15:01.170992+0000","last_fullsized":"2024-09-19T08:15:01.170992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:50.924779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308498+0000","last_change":"2024-09-19T08:13:00.853520+0000","last_active":"2024-09-19T08:15:01.308498+0000","last_peered":"2024-09-19T08:15:01.308498+0000","last_clean":"2024-09-19T08:15:01.308498+0000","last_became_active":"2024-09-19T08:13:00.853184+0000","last_became_peered":"2024-09-19T08:13:00.853184+0000","last_unstale":"2024-09-19T08:15:01.308498+0000","last_undegraded":"2024-09-19T08:15:01.308498+0000","last_fullsized":"2024-09-19T08:15:01.308498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:45.160439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:02.882016+0000","last_change":"2024-09-19T08:12:58.674359+0000","last_active":"2024-09-19T08:15:02.882016+0000","last_peered":"2024-09-19T08:15:02.882016+0000","last_clean":"2024-09-19T08:15:02.882016+0000","last_became_active":"2024-09-19T08:12:58.673890+0000","last_became_peered":"2024-09-19T08:12:58.673890+0000","last_unstale":"2024-09-19T08:15:02.882016+0000","last_undegraded":"2024-09-19T08:15:02.882016+0000","last_fullsized":"2024-09-19T08:15:02.882016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:54.939959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171020+0000","last_change":"2024-09-19T08:13:00.851071+0000","last_active":"2024-09-19T08:15:01.171020+0000","last_peered":"2024-09-19T08:15:01.171020+0000","last_clean":"2024-09-19T08:15:01.171020+0000","last_became_active":"2024-09-19T08:13:00.850756+0000","last_became_peered":"2024-09-19T08:13:00.850756+0000","last_unstale":"2024-09-19T08:15:01.171020+0000","last_undegraded":"2024-09-19T08:15:01.171020+0000","last_fullsized":"2024-09-19T08:15:01.171020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:52.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263074+0000","last_change":"2024-09-19T08:12:58.681466+0000","last_active":"2024-09-19T08:15:01.263074+0000","last_peered":"2024-09-19T08:15:01.263074+0000","last_clean":"2024-09-19T08:15:01.263074+0000","last_became_active":"2024-09-19T08:12:58.681113+0000","last_became_peered":"2024-09-19T08:12:58.681113+0000","last_unstale":"2024-09-19T08:15:01.263074+0000","last_undegraded":"2024-09-19T08:15:01.263074+0000","last_fullsized":"2024-09-19T08:15:01.263074+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:49.610828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308435+0000","last_change":"2024-09-19T08:13:00.851272+0000","last_active":"2024-09-19T08:15:01.308435+0000","last_peered":"2024-09-19T08:15:01.308435+0000","last_clean":"2024-09-19T08:15:01.308435+0000","last_became_active":"2024-09-19T08:13:00.849098+0000","last_became_peered":"2024-09-19T08:13:00.849098+0000","last_unstale":"2024-09-19T08:15:01.308435+0000","last_undegraded":"2024-09-19T08:15:01.308435+0000","last_fullsized":"2024-09-19T08:15:01.308435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:05.643992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172513+0000","last_change":"2024-09-19T08:14:07.738913+0000","last_active":"2024-09-19T08:15:01.172513+0000","last_peered":"2024-09-19T08:15:01.172513+0000","last_clean":"2024-09-19T08:15:01.172513+0000","last_became_active":"2024-09-19T08:14:07.738560+0000","last_became_peered":"2024-09-19T08:14:07.738560+0000","last_unstale":"2024-09-19T08:15:01.172513+0000","last_undegraded":"2024-09-19T08:15:01.172513+0000","last_fullsized":"2024-09-19T08:15:01.172513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:35.731290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263192+0000","last_change":"2024-09-19T08:12:58.674610+0000","last_active":"2024-09-19T08:15:01.263192+0000","last_peered":"2024-09-19T08:15:01.263192+0000","last_clean":"2024-09-19T08:15:01.263192+0000","last_became_active":"2024-09-19T08:12:58.674247+0000","last_became_peered":"2024-09-19T08:12:58.674247+0000","last_unstale":"2024-09-19T08:15:01.263192+0000","last_undegraded":"2024-09-19T08:15:01.263192+0000","last_fullsized":"2024-09-19T08:15:01.263192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:16.323574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223395+0000","last_change":"2024-09-19T08:13:00.858512+0000","last_active":"2024-09-19T08:15:01.223395+0000","last_peered":"2024-09-19T08:15:01.223395+0000","last_clean":"2024-09-19T08:15:01.223395+0000","last_became_active":"2024-09-19T08:13:00.857897+0000","last_became_peered":"2024-09-19T08:13:00.857897+0000","last_unstale":"2024-09-19T08:15:01.223395+0000","last_undegraded":"2024-09-19T08:15:01.223395+0000","last_fullsized":"2024-09-19T08:15:01.223395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263238+0000","last_change":"2024-09-19T08:12:58.682207+0000","last_active":"2024-09-19T08:15:01.263238+0000","last_peered":"2024-09-19T08:15:01.263238+0000","last_clean":"2024-09-19T08:15:01.263238+0000","last_became_active":"2024-09-19T08:12:58.681842+0000","last_became_peered":"2024-09-19T08:12:58.681842+0000","last_unstale":"2024-09-19T08:15:01.263238+0000","last_undegraded":"2024-09-19T08:15:01.263238+0000","last_fullsized":"2024-09-19T08:15:01.263238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.825858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308892+0000","last_change":"2024-09-19T08:13:00.846748+0000","last_active":"2024-09-19T08:15:01.308892+0000","last_peered":"2024-09-19T08:15:01.308892+0000","last_clean":"2024-09-19T08:15:01.308892+0000","last_became_active":"2024-09-19T08:13:00.846368+0000","last_became_peered":"2024-09-19T08:13:00.846368+0000","last_unstale":"2024-09-19T08:15:01.308892+0000","last_undegraded":"2024-09-19T08:15:01.308892+0000","last_fullsized":"2024-09-19T08:15:01.308892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:59.526512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171334+0000","last_change":"2024-09-19T08:12:58.716118+0000","last_active":"2024-09-19T08:15:01.171334+0000","last_peered":"2024-09-19T08:15:01.171334+0000","last_clean":"2024-09-19T08:15:01.171334+0000","last_became_active":"2024-09-19T08:12:58.715711+0000","last_became_peered":"2024-09-19T08:12:58.715711+0000","last_unstale":"2024-09-19T08:15:01.171334+0000","last_undegraded":"2024-09-19T08:15:01.171334+0000","last_fullsized":"2024-09-19T08:15:01.171334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:29.439887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171226+0000","last_change":"2024-09-19T08:13:00.848729+0000","last_active":"2024-09-19T08:15:01.171226+0000","last_peered":"2024-09-19T08:15:01.171226+0000","last_clean":"2024-09-19T08:15:01.171226+0000","last_became_active":"2024-09-19T08:13:00.848470+0000","last_became_peered":"2024-09-19T08:13:00.848470+0000","last_unstale":"2024-09-19T08:15:01.171226+0000","last_undegraded":"2024-09-19T08:15:01.171226+0000","last_fullsized":"2024-09-19T08:15:01.171226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:30.653101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:07.185959+0000","last_change":"2024-09-19T08:12:58.681654+0000","last_active":"2024-09-19T08:15:07.185959+0000","last_peered":"2024-09-19T08:15:07.185959+0000","last_clean":"2024-09-19T08:15:07.185959+0000","last_became_active":"2024-09-19T08:12:58.681284+0000","last_became_peered":"2024-09-19T08:12:58.681284+0000","last_unstale":"2024-09-19T08:15:07.185959+0000","last_undegraded":"2024-09-19T08:15:07.185959+0000","last_fullsized":"2024-09-19T08:15:07.185959+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.429930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798886,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1081,"num_read_kb":750,"num_write":423,"num_write_kb":2364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":61306,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":618496,"data_stored":2396658,"data_compressed":113084,"data_compressed_allocated":147456,"data_compressed_original":2158592,"omap_allocated":0,"internal_metadata":0},"log_size":314,"ondisk_log_size":314,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":109176,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481830,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":454888,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710764,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1138688,"data_stored":1446975,"data_compressed":59949,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874289,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":848206,"data_compressed":28006,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103222,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":958464,"data_stored":847992,"data_compressed":27863,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332155,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":1049397,"data_compressed":45109,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495680,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":933888,"data_stored":454379,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757318,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1052672,"data_stored":1375685,"data_compressed":60311,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":527324,"data_compressed":18328,"data_compressed_allocated":24576,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":979506,"data_compressed":34019,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":456842,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":525052,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":455650,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":73152,"data_compressed":9611,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":331028,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":393820,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":402318,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:15:14.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:15:14.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:14 smithi157 ceph-mon[35143]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-09-19T08:15:14.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:14 smithi157 ceph-mon[35143]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:14.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:14 smithi064 ceph-mon[34900]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-09-19T08:15:14.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:14 smithi064 ceph-mon[34900]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:14.850 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:15.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.343+0000 7f5175197640 1 -- 172.21.15.22:0/816372603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5170101420 msgr2=0x7f517010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:15.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.343+0000 7f5175197640 1 --2- 172.21.15.22:0/816372603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5170101420 0x7f517010d950 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5168009cd0 tx=0x7f516802a8e0 comp rx=0 tx=0).stop 2024-09-19T08:15:15.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.343+0000 7f5175197640 1 -- 172.21.15.22:0/816372603 shutdown_connections 2024-09-19T08:15:15.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.343+0000 7f5175197640 1 --2- 172.21.15.22:0/816372603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5170101420 0x7f517010d950 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:15.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.343+0000 7f5175197640 1 --2- 172.21.15.22:0/816372603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5170100a80 0x7f5170100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:15.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.343+0000 7f5175197640 1 --2- 172.21.15.22:0/816372603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5170106a80 0x7f5170106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:15.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.343+0000 7f5175197640 1 -- 172.21.15.22:0/816372603 >> 172.21.15.22:0/816372603 conn(0x7f51700fc7a0 msgr2=0x7f51700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:15.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.343+0000 7f5175197640 1 -- 172.21.15.22:0/816372603 shutdown_connections 2024-09-19T08:15:15.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.343+0000 7f5175197640 1 -- 172.21.15.22:0/816372603 wait complete. 2024-09-19T08:15:15.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.343+0000 7f5175197640 1 Processor -- start 2024-09-19T08:15:15.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f5175197640 1 -- start start 2024-09-19T08:15:15.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f5175197640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5170100a80 0x7f5170199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:15.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f5175197640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5170101420 0x7f51701997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:15.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f5175197640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5170106a80 0x7f51701a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:15.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f5175197640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5170110e80 con 0x7f5170101420 2024-09-19T08:15:15.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f5175197640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5170110d00 con 0x7f5170100a80 2024-09-19T08:15:15.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f5175197640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5170111000 con 0x7f5170106a80 2024-09-19T08:15:15.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f516e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5170101420 0x7f51701997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:15.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f516e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5170101420 0x7f51701997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42986/0 (socket says 172.21.15.22:42986) 2024-09-19T08:15:15.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f516e59c640 1 -- 172.21.15.22:0/2598030669 learned_addr learned my addr 172.21.15.22:0/2598030669 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:15.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f516f59e640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5170106a80 0x7f51701a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:15.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f516ed9d640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5170100a80 0x7f5170199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:15.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f516e59c640 1 -- 172.21.15.22:0/2598030669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5170106a80 msgr2=0x7f51701a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:15.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f516e59c640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5170106a80 0x7f51701a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:15.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f516e59c640 1 -- 172.21.15.22:0/2598030669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5170100a80 msgr2=0x7f5170199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:15.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f516e59c640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5170100a80 0x7f5170199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:15.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f516e59c640 1 -- 172.21.15.22:0/2598030669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51701a4850 con 0x7f5170101420 2024-09-19T08:15:15.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.344+0000 7f516ed9d640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5170100a80 0x7f5170199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:15.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.345+0000 7f516e59c640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5170101420 0x7f51701997d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f515800d950 tx=0x7f515800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:15.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.345+0000 7f5167fff640 1 -- 172.21.15.22:0/2598030669 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158017070 con 0x7f5170101420 2024-09-19T08:15:15.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.345+0000 7f5175197640 1 -- 172.21.15.22:0/2598030669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51701a4a20 con 0x7f5170101420 2024-09-19T08:15:15.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.345+0000 7f5167fff640 1 -- 172.21.15.22:0/2598030669 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5158009c30 con 0x7f5170101420 2024-09-19T08:15:15.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.345+0000 7f5167fff640 1 -- 172.21.15.22:0/2598030669 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515800f6b0 con 0x7f5170101420 2024-09-19T08:15:15.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.345+0000 7f5175197640 1 -- 172.21.15.22:0/2598030669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51701a4f60 con 0x7f5170101420 2024-09-19T08:15:15.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.346+0000 7f5175197640 1 -- 172.21.15.22:0/2598030669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5170100ee0 con 0x7f5170101420 2024-09-19T08:15:15.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.347+0000 7f5167fff640 1 -- 172.21.15.22:0/2598030669 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f51580109d0 con 0x7f5170101420 2024-09-19T08:15:15.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.348+0000 7f5167fff640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f515c0a34d0 0x7f515c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:15.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.348+0000 7f516ed9d640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f515c0a34d0 0x7f515c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:15.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.348+0000 7f5167fff640 1 -- 172.21.15.22:0/2598030669 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f51580cfc30 con 0x7f5170101420 2024-09-19T08:15:15.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.349+0000 7f516ed9d640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f515c0a34d0 0x7f515c0a5990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5160009ba0 tx=0x7f5160005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:15.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.352+0000 7f5167fff640 1 -- 172.21.15.22:0/2598030669 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5158009de0 con 0x7f5170101420 2024-09-19T08:15:15.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.524+0000 7f5175197640 1 -- 172.21.15.22:0/2598030669 --> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5170109060 con 0x7f515c0a34d0 2024-09-19T08:15:15.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.531+0000 7f5167fff640 1 -- 172.21.15.22:0/2598030669 <== mgr.15141 v2:172.21.15.22:6800/497251935 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258054 (secure 0 0 0) 0x7f5160016070 con 0x7f515c0a34d0 2024-09-19T08:15:15.533 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:15.534 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T08:15:15.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.535+0000 7f5175197640 1 -- 172.21.15.22:0/2598030669 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f515c0a34d0 msgr2=0x7f515c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:15.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.535+0000 7f5175197640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f515c0a34d0 0x7f515c0a5990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5160009ba0 tx=0x7f5160005f90 comp rx=0 tx=0).stop 2024-09-19T08:15:15.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.535+0000 7f5175197640 1 -- 172.21.15.22:0/2598030669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5170101420 msgr2=0x7f51701997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:15.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.535+0000 7f5175197640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5170101420 0x7f51701997d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f515800d950 tx=0x7f515800de10 comp rx=0 tx=0).stop 2024-09-19T08:15:15.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.535+0000 7f5175197640 1 -- 172.21.15.22:0/2598030669 shutdown_connections 2024-09-19T08:15:15.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.535+0000 7f5175197640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f515c0a34d0 0x7f515c0a5990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:15.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.535+0000 7f5175197640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5170106a80 0x7f51701a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:15.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.535+0000 7f5175197640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5170101420 0x7f51701997d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:15.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.535+0000 7f5175197640 1 --2- 172.21.15.22:0/2598030669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5170100a80 0x7f5170199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:15.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.535+0000 7f5175197640 1 -- 172.21.15.22:0/2598030669 >> 172.21.15.22:0/2598030669 conn(0x7f51700fc7a0 msgr2=0x7f51700fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:15.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.535+0000 7f5175197640 1 -- 172.21.15.22:0/2598030669 shutdown_connections 2024-09-19T08:15:15.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:15.535+0000 7f5175197640 1 -- 172.21.15.22:0/2598030669 wait complete. 2024-09-19T08:15:16.233 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-19T08:15:15.188192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389488,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1318,"num_read_kb":1080,"num_write":626,"num_write_kb":5304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1242042,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":964,"ondisk_log_size":964,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381672,"kb_used_data":11596,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560152,"statfs":{"total":1151940427776,"available":1151549595648,"internally_reserved":0,"allocated":11874304,"data_stored":9421896,"data_compressed":317170,"data_compressed_allocated":417792,"data_compressed_original":4038656,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201076,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":398,"num_read_kb":460,"num_write":146,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002688"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309162+0000","last_change":"2024-09-19T08:13:00.853947+0000","last_active":"2024-09-19T08:15:01.309162+0000","last_peered":"2024-09-19T08:15:01.309162+0000","last_clean":"2024-09-19T08:15:01.309162+0000","last_became_active":"2024-09-19T08:13:00.852951+0000","last_became_peered":"2024-09-19T08:13:00.852951+0000","last_unstale":"2024-09-19T08:15:01.309162+0000","last_undegraded":"2024-09-19T08:15:01.309162+0000","last_fullsized":"2024-09-19T08:15:01.309162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:25.786828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308625+0000","last_change":"2024-09-19T08:13:00.853389+0000","last_active":"2024-09-19T08:15:01.308625+0000","last_peered":"2024-09-19T08:15:01.308625+0000","last_clean":"2024-09-19T08:15:01.308625+0000","last_became_active":"2024-09-19T08:13:00.853102+0000","last_became_peered":"2024-09-19T08:13:00.853102+0000","last_unstale":"2024-09-19T08:15:01.308625+0000","last_undegraded":"2024-09-19T08:15:01.308625+0000","last_fullsized":"2024-09-19T08:15:01.308625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:42.773917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308869+0000","last_change":"2024-09-19T08:13:00.846544+0000","last_active":"2024-09-19T08:15:01.308869+0000","last_peered":"2024-09-19T08:15:01.308869+0000","last_clean":"2024-09-19T08:15:01.308869+0000","last_became_active":"2024-09-19T08:13:00.846229+0000","last_became_peered":"2024-09-19T08:13:00.846229+0000","last_unstale":"2024-09-19T08:15:01.308869+0000","last_undegraded":"2024-09-19T08:15:01.308869+0000","last_fullsized":"2024-09-19T08:15:01.308869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:43.471684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172938+0000","last_change":"2024-09-19T08:13:00.848149+0000","last_active":"2024-09-19T08:15:01.172938+0000","last_peered":"2024-09-19T08:15:01.172938+0000","last_clean":"2024-09-19T08:15:01.172938+0000","last_became_active":"2024-09-19T08:13:00.847744+0000","last_became_peered":"2024-09-19T08:13:00.847744+0000","last_unstale":"2024-09-19T08:15:01.172938+0000","last_undegraded":"2024-09-19T08:15:01.172938+0000","last_fullsized":"2024-09-19T08:15:01.172938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:01.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263000+0000","last_change":"2024-09-19T08:13:00.841862+0000","last_active":"2024-09-19T08:15:01.263000+0000","last_peered":"2024-09-19T08:15:01.263000+0000","last_clean":"2024-09-19T08:15:01.263000+0000","last_became_active":"2024-09-19T08:13:00.841331+0000","last_became_peered":"2024-09-19T08:13:00.841331+0000","last_unstale":"2024-09-19T08:15:01.263000+0000","last_undegraded":"2024-09-19T08:15:01.263000+0000","last_fullsized":"2024-09-19T08:15:01.263000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:54.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271477+0000","last_change":"2024-09-19T08:13:00.849626+0000","last_active":"2024-09-19T08:15:01.271477+0000","last_peered":"2024-09-19T08:15:01.271477+0000","last_clean":"2024-09-19T08:15:01.271477+0000","last_became_active":"2024-09-19T08:13:00.849034+0000","last_became_peered":"2024-09-19T08:13:00.849034+0000","last_unstale":"2024-09-19T08:15:01.271477+0000","last_undegraded":"2024-09-19T08:15:01.271477+0000","last_fullsized":"2024-09-19T08:15:01.271477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:39.118978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308024+0000","last_change":"2024-09-19T08:13:00.847522+0000","last_active":"2024-09-19T08:15:01.308024+0000","last_peered":"2024-09-19T08:15:01.308024+0000","last_clean":"2024-09-19T08:15:01.308024+0000","last_became_active":"2024-09-19T08:13:00.847117+0000","last_became_peered":"2024-09-19T08:13:00.847117+0000","last_unstale":"2024-09-19T08:15:01.308024+0000","last_undegraded":"2024-09-19T08:15:01.308024+0000","last_fullsized":"2024-09-19T08:15:01.308024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.836385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170733+0000","last_change":"2024-09-19T08:13:00.849544+0000","last_active":"2024-09-19T08:15:01.170733+0000","last_peered":"2024-09-19T08:15:01.170733+0000","last_clean":"2024-09-19T08:15:01.170733+0000","last_became_active":"2024-09-19T08:13:00.849170+0000","last_became_peered":"2024-09-19T08:13:00.849170+0000","last_unstale":"2024-09-19T08:15:01.170733+0000","last_undegraded":"2024-09-19T08:15:01.170733+0000","last_fullsized":"2024-09-19T08:15:01.170733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:28.371965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223727+0000","last_change":"2024-09-19T08:13:00.856052+0000","last_active":"2024-09-19T08:15:01.223727+0000","last_peered":"2024-09-19T08:15:01.223727+0000","last_clean":"2024-09-19T08:15:01.223727+0000","last_became_active":"2024-09-19T08:13:00.855291+0000","last_became_peered":"2024-09-19T08:13:00.855291+0000","last_unstale":"2024-09-19T08:15:01.223727+0000","last_undegraded":"2024-09-19T08:15:01.223727+0000","last_fullsized":"2024-09-19T08:15:01.223727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:18.581867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271887+0000","last_change":"2024-09-19T08:13:00.850145+0000","last_active":"2024-09-19T08:15:01.271887+0000","last_peered":"2024-09-19T08:15:01.271887+0000","last_clean":"2024-09-19T08:15:01.271887+0000","last_became_active":"2024-09-19T08:13:00.849798+0000","last_became_peered":"2024-09-19T08:13:00.849798+0000","last_unstale":"2024-09-19T08:15:01.271887+0000","last_undegraded":"2024-09-19T08:15:01.271887+0000","last_fullsized":"2024-09-19T08:15:01.271887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:21.400000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308275+0000","last_change":"2024-09-19T08:13:00.859153+0000","last_active":"2024-09-19T08:15:01.308275+0000","last_peered":"2024-09-19T08:15:01.308275+0000","last_clean":"2024-09-19T08:15:01.308275+0000","last_became_active":"2024-09-19T08:13:00.858473+0000","last_became_peered":"2024-09-19T08:13:00.858473+0000","last_unstale":"2024-09-19T08:15:01.308275+0000","last_undegraded":"2024-09-19T08:15:01.308275+0000","last_fullsized":"2024-09-19T08:15:01.308275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:07.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172922+0000","last_change":"2024-09-19T08:13:00.850797+0000","last_active":"2024-09-19T08:15:01.172922+0000","last_peered":"2024-09-19T08:15:01.172922+0000","last_clean":"2024-09-19T08:15:01.172922+0000","last_became_active":"2024-09-19T08:13:00.850393+0000","last_became_peered":"2024-09-19T08:13:00.850393+0000","last_unstale":"2024-09-19T08:15:01.172922+0000","last_undegraded":"2024-09-19T08:15:01.172922+0000","last_fullsized":"2024-09-19T08:15:01.172922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:59.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310082+0000","last_change":"2024-09-19T08:13:00.850835+0000","last_active":"2024-09-19T08:15:01.310082+0000","last_peered":"2024-09-19T08:15:01.310082+0000","last_clean":"2024-09-19T08:15:01.310082+0000","last_became_active":"2024-09-19T08:13:00.850518+0000","last_became_peered":"2024-09-19T08:13:00.850518+0000","last_unstale":"2024-09-19T08:15:01.310082+0000","last_undegraded":"2024-09-19T08:15:01.310082+0000","last_fullsized":"2024-09-19T08:15:01.310082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.178530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308450+0000","last_change":"2024-09-19T08:13:00.853934+0000","last_active":"2024-09-19T08:15:01.308450+0000","last_peered":"2024-09-19T08:15:01.308450+0000","last_clean":"2024-09-19T08:15:01.308450+0000","last_became_active":"2024-09-19T08:13:00.853604+0000","last_became_peered":"2024-09-19T08:13:00.853604+0000","last_unstale":"2024-09-19T08:15:01.308450+0000","last_undegraded":"2024-09-19T08:15:01.308450+0000","last_fullsized":"2024-09-19T08:15:01.308450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:43.622526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308262+0000","last_change":"2024-09-19T08:14:07.741370+0000","last_active":"2024-09-19T08:15:01.308262+0000","last_peered":"2024-09-19T08:15:01.308262+0000","last_clean":"2024-09-19T08:15:01.308262+0000","last_became_active":"2024-09-19T08:14:07.741116+0000","last_became_peered":"2024-09-19T08:14:07.741116+0000","last_unstale":"2024-09-19T08:15:01.308262+0000","last_undegraded":"2024-09-19T08:15:01.308262+0000","last_fullsized":"2024-09-19T08:15:01.308262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:45.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223320+0000","last_change":"2024-09-19T08:13:00.850266+0000","last_active":"2024-09-19T08:15:01.223320+0000","last_peered":"2024-09-19T08:15:01.223320+0000","last_clean":"2024-09-19T08:15:01.223320+0000","last_became_active":"2024-09-19T08:13:00.849928+0000","last_became_peered":"2024-09-19T08:13:00.849928+0000","last_unstale":"2024-09-19T08:15:01.223320+0000","last_undegraded":"2024-09-19T08:15:01.223320+0000","last_fullsized":"2024-09-19T08:15:01.223320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:38.231331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309177+0000","last_change":"2024-09-19T08:13:00.852551+0000","last_active":"2024-09-19T08:15:01.309177+0000","last_peered":"2024-09-19T08:15:01.309177+0000","last_clean":"2024-09-19T08:15:01.309177+0000","last_became_active":"2024-09-19T08:13:00.852225+0000","last_became_peered":"2024-09-19T08:13:00.852225+0000","last_unstale":"2024-09-19T08:15:01.309177+0000","last_undegraded":"2024-09-19T08:15:01.309177+0000","last_fullsized":"2024-09-19T08:15:01.309177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:33.115108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.261006+0000","last_change":"2024-09-19T08:13:00.854033+0000","last_active":"2024-09-19T08:15:01.261006+0000","last_peered":"2024-09-19T08:15:01.261006+0000","last_clean":"2024-09-19T08:15:01.261006+0000","last_became_active":"2024-09-19T08:13:00.853660+0000","last_became_peered":"2024-09-19T08:13:00.853660+0000","last_unstale":"2024-09-19T08:15:01.261006+0000","last_undegraded":"2024-09-19T08:15:01.261006+0000","last_fullsized":"2024-09-19T08:15:01.261006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:35.602420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308765+0000","last_change":"2024-09-19T08:13:00.850658+0000","last_active":"2024-09-19T08:15:01.308765+0000","last_peered":"2024-09-19T08:15:01.308765+0000","last_clean":"2024-09-19T08:15:01.308765+0000","last_became_active":"2024-09-19T08:13:00.850324+0000","last_became_peered":"2024-09-19T08:13:00.850324+0000","last_unstale":"2024-09-19T08:15:01.308765+0000","last_undegraded":"2024-09-19T08:15:01.308765+0000","last_fullsized":"2024-09-19T08:15:01.308765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:59:49.004605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308718+0000","last_change":"2024-09-19T08:14:07.741837+0000","last_active":"2024-09-19T08:15:01.308718+0000","last_peered":"2024-09-19T08:15:01.308718+0000","last_clean":"2024-09-19T08:15:01.308718+0000","last_became_active":"2024-09-19T08:14:07.741398+0000","last_became_peered":"2024-09-19T08:14:07.741398+0000","last_unstale":"2024-09-19T08:15:01.308718+0000","last_undegraded":"2024-09-19T08:15:01.308718+0000","last_fullsized":"2024-09-19T08:15:01.308718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:56.716663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310236+0000","last_change":"2024-09-19T08:13:00.855215+0000","last_active":"2024-09-19T08:15:01.310236+0000","last_peered":"2024-09-19T08:15:01.310236+0000","last_clean":"2024-09-19T08:15:01.310236+0000","last_became_active":"2024-09-19T08:13:00.854878+0000","last_became_peered":"2024-09-19T08:13:00.854878+0000","last_unstale":"2024-09-19T08:15:01.310236+0000","last_undegraded":"2024-09-19T08:15:01.310236+0000","last_fullsized":"2024-09-19T08:15:01.310236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:48.216066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223444+0000","last_change":"2024-09-19T08:13:00.856995+0000","last_active":"2024-09-19T08:15:01.223444+0000","last_peered":"2024-09-19T08:15:01.223444+0000","last_clean":"2024-09-19T08:15:01.223444+0000","last_became_active":"2024-09-19T08:13:00.856645+0000","last_became_peered":"2024-09-19T08:13:00.856645+0000","last_unstale":"2024-09-19T08:15:01.223444+0000","last_undegraded":"2024-09-19T08:15:01.223444+0000","last_fullsized":"2024-09-19T08:15:01.223444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:20.699293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170753+0000","last_change":"2024-09-19T08:13:00.853362+0000","last_active":"2024-09-19T08:15:01.170753+0000","last_peered":"2024-09-19T08:15:01.170753+0000","last_clean":"2024-09-19T08:15:01.170753+0000","last_became_active":"2024-09-19T08:13:00.853046+0000","last_became_peered":"2024-09-19T08:13:00.853046+0000","last_unstale":"2024-09-19T08:15:01.170753+0000","last_undegraded":"2024-09-19T08:15:01.170753+0000","last_fullsized":"2024-09-19T08:15:01.170753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172297+0000","last_change":"2024-09-19T08:13:00.845175+0000","last_active":"2024-09-19T08:15:01.172297+0000","last_peered":"2024-09-19T08:15:01.172297+0000","last_clean":"2024-09-19T08:15:01.172297+0000","last_became_active":"2024-09-19T08:13:00.844857+0000","last_became_peered":"2024-09-19T08:13:00.844857+0000","last_unstale":"2024-09-19T08:15:01.172297+0000","last_undegraded":"2024-09-19T08:15:01.172297+0000","last_fullsized":"2024-09-19T08:15:01.172297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:40.818956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171025+0000","last_change":"2024-09-19T08:13:00.847769+0000","last_active":"2024-09-19T08:15:01.171025+0000","last_peered":"2024-09-19T08:15:01.171025+0000","last_clean":"2024-09-19T08:15:01.171025+0000","last_became_active":"2024-09-19T08:13:00.847454+0000","last_became_peered":"2024-09-19T08:13:00.847454+0000","last_unstale":"2024-09-19T08:15:01.171025+0000","last_undegraded":"2024-09-19T08:15:01.171025+0000","last_fullsized":"2024-09-19T08:15:01.171025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.583051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308881+0000","last_change":"2024-09-19T08:13:00.859882+0000","last_active":"2024-09-19T08:15:01.308881+0000","last_peered":"2024-09-19T08:15:01.308881+0000","last_clean":"2024-09-19T08:15:01.308881+0000","last_became_active":"2024-09-19T08:13:00.859491+0000","last_became_peered":"2024-09-19T08:13:00.859491+0000","last_unstale":"2024-09-19T08:15:01.308881+0000","last_undegraded":"2024-09-19T08:15:01.308881+0000","last_fullsized":"2024-09-19T08:15:01.308881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:13.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308686+0000","last_change":"2024-09-19T08:13:00.845584+0000","last_active":"2024-09-19T08:15:01.308686+0000","last_peered":"2024-09-19T08:15:01.308686+0000","last_clean":"2024-09-19T08:15:01.308686+0000","last_became_active":"2024-09-19T08:13:00.845166+0000","last_became_peered":"2024-09-19T08:13:00.845166+0000","last_unstale":"2024-09-19T08:15:01.308686+0000","last_undegraded":"2024-09-19T08:15:01.308686+0000","last_fullsized":"2024-09-19T08:15:01.308686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:02.705793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310539+0000","last_change":"2024-09-19T08:13:00.838069+0000","last_active":"2024-09-19T08:15:01.310539+0000","last_peered":"2024-09-19T08:15:01.310539+0000","last_clean":"2024-09-19T08:15:01.310539+0000","last_became_active":"2024-09-19T08:13:00.837736+0000","last_became_peered":"2024-09-19T08:13:00.837736+0000","last_unstale":"2024-09-19T08:15:01.310539+0000","last_undegraded":"2024-09-19T08:15:01.310539+0000","last_fullsized":"2024-09-19T08:15:01.310539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:01.388290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308795+0000","last_change":"2024-09-19T08:13:00.841507+0000","last_active":"2024-09-19T08:15:01.308795+0000","last_peered":"2024-09-19T08:15:01.308795+0000","last_clean":"2024-09-19T08:15:01.308795+0000","last_became_active":"2024-09-19T08:13:00.841161+0000","last_became_peered":"2024-09-19T08:13:00.841161+0000","last_unstale":"2024-09-19T08:15:01.308795+0000","last_undegraded":"2024-09-19T08:15:01.308795+0000","last_fullsized":"2024-09-19T08:15:01.308795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:41.862290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308530+0000","last_change":"2024-09-19T08:13:00.847657+0000","last_active":"2024-09-19T08:15:01.308530+0000","last_peered":"2024-09-19T08:15:01.308530+0000","last_clean":"2024-09-19T08:15:01.308530+0000","last_became_active":"2024-09-19T08:13:00.846849+0000","last_became_peered":"2024-09-19T08:13:00.846849+0000","last_unstale":"2024-09-19T08:15:01.308530+0000","last_undegraded":"2024-09-19T08:15:01.308530+0000","last_fullsized":"2024-09-19T08:15:01.308530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:35.219913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309056+0000","last_change":"2024-09-19T08:13:00.845971+0000","last_active":"2024-09-19T08:15:01.309056+0000","last_peered":"2024-09-19T08:15:01.309056+0000","last_clean":"2024-09-19T08:15:01.309056+0000","last_became_active":"2024-09-19T08:13:00.845626+0000","last_became_peered":"2024-09-19T08:13:00.845626+0000","last_unstale":"2024-09-19T08:15:01.309056+0000","last_undegraded":"2024-09-19T08:15:01.309056+0000","last_fullsized":"2024-09-19T08:15:01.309056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:36.945627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308432+0000","last_change":"2024-09-19T08:14:07.738450+0000","last_active":"2024-09-19T08:15:01.308432+0000","last_peered":"2024-09-19T08:15:01.308432+0000","last_clean":"2024-09-19T08:15:01.308432+0000","last_became_active":"2024-09-19T08:14:07.738028+0000","last_became_peered":"2024-09-19T08:14:07.738028+0000","last_unstale":"2024-09-19T08:15:01.308432+0000","last_undegraded":"2024-09-19T08:15:01.308432+0000","last_fullsized":"2024-09-19T08:15:01.308432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:08.659158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262954+0000","last_change":"2024-09-19T08:13:00.848325+0000","last_active":"2024-09-19T08:15:01.262954+0000","last_peered":"2024-09-19T08:15:01.262954+0000","last_clean":"2024-09-19T08:15:01.262954+0000","last_became_active":"2024-09-19T08:13:00.848015+0000","last_became_peered":"2024-09-19T08:13:00.848015+0000","last_unstale":"2024-09-19T08:15:01.262954+0000","last_undegraded":"2024-09-19T08:15:01.262954+0000","last_fullsized":"2024-09-19T08:15:01.262954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:10.948451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'9","reported_seq":160,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.051779+0000","last_change":"2024-09-19T08:12:58.681134+0000","last_active":"2024-09-19T08:15:11.051779+0000","last_peered":"2024-09-19T08:15:11.051779+0000","last_clean":"2024-09-19T08:15:11.051779+0000","last_became_active":"2024-09-19T08:12:58.680764+0000","last_became_peered":"2024-09-19T08:12:58.680764+0000","last_unstale":"2024-09-19T08:15:11.051779+0000","last_undegraded":"2024-09-19T08:15:11.051779+0000","last_fullsized":"2024-09-19T08:15:11.051779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:24.939263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172533+0000","last_change":"2024-09-19T08:13:00.844319+0000","last_active":"2024-09-19T08:15:01.172533+0000","last_peered":"2024-09-19T08:15:01.172533+0000","last_clean":"2024-09-19T08:15:01.172533+0000","last_became_active":"2024-09-19T08:13:00.843994+0000","last_became_peered":"2024-09-19T08:13:00.843994+0000","last_unstale":"2024-09-19T08:15:01.172533+0000","last_undegraded":"2024-09-19T08:15:01.172533+0000","last_fullsized":"2024-09-19T08:15:01.172533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:46.937466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271532+0000","last_change":"2024-09-19T08:12:58.676541+0000","last_active":"2024-09-19T08:15:01.271532+0000","last_peered":"2024-09-19T08:15:01.271532+0000","last_clean":"2024-09-19T08:15:01.271532+0000","last_became_active":"2024-09-19T08:12:58.675988+0000","last_became_peered":"2024-09-19T08:12:58.675988+0000","last_unstale":"2024-09-19T08:15:01.271532+0000","last_undegraded":"2024-09-19T08:15:01.271532+0000","last_fullsized":"2024-09-19T08:15:01.271532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:01.434986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309788+0000","last_change":"2024-09-19T08:12:58.677510+0000","last_active":"2024-09-19T08:15:01.309788+0000","last_peered":"2024-09-19T08:15:01.309788+0000","last_clean":"2024-09-19T08:15:01.309788+0000","last_became_active":"2024-09-19T08:12:58.677200+0000","last_became_peered":"2024-09-19T08:12:58.677200+0000","last_unstale":"2024-09-19T08:15:01.309788+0000","last_undegraded":"2024-09-19T08:15:01.309788+0000","last_fullsized":"2024-09-19T08:15:01.309788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:36.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170825+0000","last_change":"2024-09-19T08:13:00.853430+0000","last_active":"2024-09-19T08:15:01.170825+0000","last_peered":"2024-09-19T08:15:01.170825+0000","last_clean":"2024-09-19T08:15:01.170825+0000","last_became_active":"2024-09-19T08:13:00.853096+0000","last_became_peered":"2024-09-19T08:13:00.853096+0000","last_unstale":"2024-09-19T08:15:01.170825+0000","last_undegraded":"2024-09-19T08:15:01.170825+0000","last_fullsized":"2024-09-19T08:15:01.170825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:07.147021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172384+0000","last_change":"2024-09-19T08:12:58.680081+0000","last_active":"2024-09-19T08:15:01.172384+0000","last_peered":"2024-09-19T08:15:01.172384+0000","last_clean":"2024-09-19T08:15:01.172384+0000","last_became_active":"2024-09-19T08:12:58.679704+0000","last_became_peered":"2024-09-19T08:12:58.679704+0000","last_unstale":"2024-09-19T08:15:01.172384+0000","last_undegraded":"2024-09-19T08:15:01.172384+0000","last_fullsized":"2024-09-19T08:15:01.172384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:32.942586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.260317+0000","last_change":"2024-09-19T08:13:00.849394+0000","last_active":"2024-09-19T08:15:01.260317+0000","last_peered":"2024-09-19T08:15:01.260317+0000","last_clean":"2024-09-19T08:15:01.260317+0000","last_became_active":"2024-09-19T08:13:00.849051+0000","last_became_peered":"2024-09-19T08:13:00.849051+0000","last_unstale":"2024-09-19T08:15:01.260317+0000","last_undegraded":"2024-09-19T08:15:01.260317+0000","last_fullsized":"2024-09-19T08:15:01.260317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:23.012928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'5","reported_seq":150,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.051165+0000","last_change":"2024-09-19T08:12:58.676121+0000","last_active":"2024-09-19T08:15:11.051165+0000","last_peered":"2024-09-19T08:15:11.051165+0000","last_clean":"2024-09-19T08:15:11.051165+0000","last_became_active":"2024-09-19T08:12:58.675693+0000","last_became_peered":"2024-09-19T08:12:58.675693+0000","last_unstale":"2024-09-19T08:15:11.051165+0000","last_undegraded":"2024-09-19T08:15:11.051165+0000","last_fullsized":"2024-09-19T08:15:11.051165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:34.882190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223583+0000","last_change":"2024-09-19T08:13:00.858735+0000","last_active":"2024-09-19T08:15:01.223583+0000","last_peered":"2024-09-19T08:15:01.223583+0000","last_clean":"2024-09-19T08:15:01.223583+0000","last_became_active":"2024-09-19T08:13:00.858218+0000","last_became_peered":"2024-09-19T08:13:00.858218+0000","last_unstale":"2024-09-19T08:15:01.223583+0000","last_undegraded":"2024-09-19T08:15:01.223583+0000","last_fullsized":"2024-09-19T08:15:01.223583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:33.356810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308176+0000","last_change":"2024-09-19T08:14:07.743785+0000","last_active":"2024-09-19T08:15:01.308176+0000","last_peered":"2024-09-19T08:15:01.308176+0000","last_clean":"2024-09-19T08:15:01.308176+0000","last_became_active":"2024-09-19T08:14:07.739531+0000","last_became_peered":"2024-09-19T08:14:07.739531+0000","last_unstale":"2024-09-19T08:15:01.308176+0000","last_undegraded":"2024-09-19T08:15:01.308176+0000","last_fullsized":"2024-09-19T08:15:01.308176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:43.367704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308976+0000","last_change":"2024-09-19T08:13:00.843852+0000","last_active":"2024-09-19T08:15:01.308976+0000","last_peered":"2024-09-19T08:15:01.308976+0000","last_clean":"2024-09-19T08:15:01.308976+0000","last_became_active":"2024-09-19T08:13:00.843497+0000","last_became_peered":"2024-09-19T08:13:00.843497+0000","last_unstale":"2024-09-19T08:15:01.308976+0000","last_undegraded":"2024-09-19T08:15:01.308976+0000","last_fullsized":"2024-09-19T08:15:01.308976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:26.380024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172698+0000","last_change":"2024-09-19T08:12:58.717225+0000","last_active":"2024-09-19T08:15:01.172698+0000","last_peered":"2024-09-19T08:15:01.172698+0000","last_clean":"2024-09-19T08:15:01.172698+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:15:01.172698+0000","last_undegraded":"2024-09-19T08:15:01.172698+0000","last_fullsized":"2024-09-19T08:15:01.172698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:43.201907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223446+0000","last_change":"2024-09-19T08:13:00.845613+0000","last_active":"2024-09-19T08:15:01.223446+0000","last_peered":"2024-09-19T08:15:01.223446+0000","last_clean":"2024-09-19T08:15:01.223446+0000","last_became_active":"2024-09-19T08:13:00.845260+0000","last_became_peered":"2024-09-19T08:13:00.845260+0000","last_unstale":"2024-09-19T08:15:01.223446+0000","last_undegraded":"2024-09-19T08:15:01.223446+0000","last_fullsized":"2024-09-19T08:15:01.223446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:03:47.033972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'39","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.834802+0000","last_change":"2024-09-19T08:14:07.749616+0000","last_active":"2024-09-19T08:15:11.834802+0000","last_peered":"2024-09-19T08:15:11.834802+0000","last_clean":"2024-09-19T08:15:11.834802+0000","last_became_active":"2024-09-19T08:14:07.740298+0000","last_became_peered":"2024-09-19T08:14:07.740298+0000","last_unstale":"2024-09-19T08:15:11.834802+0000","last_undegraded":"2024-09-19T08:15:11.834802+0000","last_fullsized":"2024-09-19T08:15:11.834802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:28.796855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70880,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":249,"num_read_kb":194,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61172,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263216+0000","last_change":"2024-09-19T08:13:00.855603+0000","last_active":"2024-09-19T08:15:01.263216+0000","last_peered":"2024-09-19T08:15:01.263216+0000","last_clean":"2024-09-19T08:15:01.263216+0000","last_became_active":"2024-09-19T08:13:00.855241+0000","last_became_peered":"2024-09-19T08:13:00.855241+0000","last_unstale":"2024-09-19T08:15:01.263216+0000","last_undegraded":"2024-09-19T08:15:01.263216+0000","last_fullsized":"2024-09-19T08:15:01.263216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:28.811847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308653+0000","last_change":"2024-09-19T08:12:58.675897+0000","last_active":"2024-09-19T08:15:01.308653+0000","last_peered":"2024-09-19T08:15:01.308653+0000","last_clean":"2024-09-19T08:15:01.308653+0000","last_became_active":"2024-09-19T08:12:58.675525+0000","last_became_peered":"2024-09-19T08:12:58.675525+0000","last_unstale":"2024-09-19T08:15:01.308653+0000","last_undegraded":"2024-09-19T08:15:01.308653+0000","last_fullsized":"2024-09-19T08:15:01.308653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:54.518064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309636+0000","last_change":"2024-09-19T08:13:00.856420+0000","last_active":"2024-09-19T08:15:01.309636+0000","last_peered":"2024-09-19T08:15:01.309636+0000","last_clean":"2024-09-19T08:15:01.309636+0000","last_became_active":"2024-09-19T08:13:00.856100+0000","last_became_peered":"2024-09-19T08:13:00.856100+0000","last_unstale":"2024-09-19T08:15:01.309636+0000","last_undegraded":"2024-09-19T08:15:01.309636+0000","last_fullsized":"2024-09-19T08:15:01.309636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:58.684930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172292+0000","last_change":"2024-09-19T08:12:58.680778+0000","last_active":"2024-09-19T08:15:01.172292+0000","last_peered":"2024-09-19T08:15:01.172292+0000","last_clean":"2024-09-19T08:15:01.172292+0000","last_became_active":"2024-09-19T08:12:58.680364+0000","last_became_peered":"2024-09-19T08:12:58.680364+0000","last_unstale":"2024-09-19T08:15:01.172292+0000","last_undegraded":"2024-09-19T08:15:01.172292+0000","last_fullsized":"2024-09-19T08:15:01.172292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:10.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308288+0000","last_change":"2024-09-19T08:13:00.853415+0000","last_active":"2024-09-19T08:15:01.308288+0000","last_peered":"2024-09-19T08:15:01.308288+0000","last_clean":"2024-09-19T08:15:01.308288+0000","last_became_active":"2024-09-19T08:13:00.853101+0000","last_became_peered":"2024-09-19T08:13:00.853101+0000","last_unstale":"2024-09-19T08:15:01.308288+0000","last_undegraded":"2024-09-19T08:15:01.308288+0000","last_fullsized":"2024-09-19T08:15:01.308288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:42.375394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":159,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.050571+0000","last_change":"2024-09-19T08:12:58.680312+0000","last_active":"2024-09-19T08:15:11.050571+0000","last_peered":"2024-09-19T08:15:11.050571+0000","last_clean":"2024-09-19T08:15:11.050571+0000","last_became_active":"2024-09-19T08:12:58.679870+0000","last_became_peered":"2024-09-19T08:12:58.679870+0000","last_unstale":"2024-09-19T08:15:11.050571+0000","last_undegraded":"2024-09-19T08:15:11.050571+0000","last_fullsized":"2024-09-19T08:15:11.050571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:42.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308034+0000","last_change":"2024-09-19T08:13:00.853920+0000","last_active":"2024-09-19T08:15:01.308034+0000","last_peered":"2024-09-19T08:15:01.308034+0000","last_clean":"2024-09-19T08:15:01.308034+0000","last_became_active":"2024-09-19T08:13:00.853594+0000","last_became_peered":"2024-09-19T08:13:00.853594+0000","last_unstale":"2024-09-19T08:15:01.308034+0000","last_undegraded":"2024-09-19T08:15:01.308034+0000","last_fullsized":"2024-09-19T08:15:01.308034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:31.746133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308308+0000","last_change":"2024-09-19T08:14:07.751888+0000","last_active":"2024-09-19T08:15:01.308308+0000","last_peered":"2024-09-19T08:15:01.308308+0000","last_clean":"2024-09-19T08:15:01.308308+0000","last_became_active":"2024-09-19T08:14:07.737201+0000","last_became_peered":"2024-09-19T08:14:07.737201+0000","last_unstale":"2024-09-19T08:15:01.308308+0000","last_undegraded":"2024-09-19T08:15:01.308308+0000","last_fullsized":"2024-09-19T08:15:01.308308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:44.095987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172245+0000","last_change":"2024-09-19T08:13:00.847061+0000","last_active":"2024-09-19T08:15:01.172245+0000","last_peered":"2024-09-19T08:15:01.172245+0000","last_clean":"2024-09-19T08:15:01.172245+0000","last_became_active":"2024-09-19T08:13:00.846742+0000","last_became_peered":"2024-09-19T08:13:00.846742+0000","last_unstale":"2024-09-19T08:15:01.172245+0000","last_undegraded":"2024-09-19T08:15:01.172245+0000","last_fullsized":"2024-09-19T08:15:01.172245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:21.785585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309033+0000","last_change":"2024-09-19T08:14:07.755277+0000","last_active":"2024-09-19T08:15:01.309033+0000","last_peered":"2024-09-19T08:15:01.309033+0000","last_clean":"2024-09-19T08:15:01.309033+0000","last_became_active":"2024-09-19T08:14:07.739773+0000","last_became_peered":"2024-09-19T08:14:07.739773+0000","last_unstale":"2024-09-19T08:15:01.309033+0000","last_undegraded":"2024-09-19T08:15:01.309033+0000","last_fullsized":"2024-09-19T08:15:01.309033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:56.848251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308865+0000","last_change":"2024-09-19T08:13:00.844940+0000","last_active":"2024-09-19T08:15:01.308865+0000","last_peered":"2024-09-19T08:15:01.308865+0000","last_clean":"2024-09-19T08:15:01.308865+0000","last_became_active":"2024-09-19T08:13:00.844454+0000","last_became_peered":"2024-09-19T08:13:00.844454+0000","last_unstale":"2024-09-19T08:15:01.308865+0000","last_undegraded":"2024-09-19T08:15:01.308865+0000","last_fullsized":"2024-09-19T08:15:01.308865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:15.577624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309967+0000","last_change":"2024-09-19T08:12:58.673301+0000","last_active":"2024-09-19T08:15:01.309967+0000","last_peered":"2024-09-19T08:15:01.309967+0000","last_clean":"2024-09-19T08:15:01.309967+0000","last_became_active":"2024-09-19T08:12:58.672981+0000","last_became_peered":"2024-09-19T08:12:58.672981+0000","last_unstale":"2024-09-19T08:15:01.309967+0000","last_undegraded":"2024-09-19T08:15:01.309967+0000","last_fullsized":"2024-09-19T08:15:01.309967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:03.963712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308223+0000","last_change":"2024-09-19T08:13:00.856620+0000","last_active":"2024-09-19T08:15:01.308223+0000","last_peered":"2024-09-19T08:15:01.308223+0000","last_clean":"2024-09-19T08:15:01.308223+0000","last_became_active":"2024-09-19T08:13:00.856310+0000","last_became_peered":"2024-09-19T08:13:00.856310+0000","last_unstale":"2024-09-19T08:15:01.308223+0000","last_undegraded":"2024-09-19T08:15:01.308223+0000","last_fullsized":"2024-09-19T08:15:01.308223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:16.918768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'65","reported_seq":118,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.911955+0000","last_change":"2024-09-19T08:12:58.672282+0000","last_active":"2024-09-19T08:15:11.911955+0000","last_peered":"2024-09-19T08:15:11.911955+0000","last_clean":"2024-09-19T08:15:11.911955+0000","last_became_active":"2024-09-19T08:12:58.671891+0000","last_became_peered":"2024-09-19T08:12:58.671891+0000","last_unstale":"2024-09-19T08:15:11.911955+0000","last_undegraded":"2024-09-19T08:15:11.911955+0000","last_fullsized":"2024-09-19T08:15:11.911955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:41.731571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":31,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308179+0000","last_change":"2024-09-19T08:13:00.848107+0000","last_active":"2024-09-19T08:15:01.308179+0000","last_peered":"2024-09-19T08:15:01.308179+0000","last_clean":"2024-09-19T08:15:01.308179+0000","last_became_active":"2024-09-19T08:13:00.847632+0000","last_became_peered":"2024-09-19T08:13:00.847632+0000","last_unstale":"2024-09-19T08:15:01.308179+0000","last_undegraded":"2024-09-19T08:15:01.308179+0000","last_fullsized":"2024-09-19T08:15:01.308179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.491122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308878+0000","last_change":"2024-09-19T08:12:59.179200+0000","last_active":"2024-09-19T08:15:01.308878+0000","last_peered":"2024-09-19T08:15:01.308878+0000","last_clean":"2024-09-19T08:15:01.308878+0000","last_became_active":"2024-09-19T08:12:59.178762+0000","last_became_peered":"2024-09-19T08:12:59.178762+0000","last_unstale":"2024-09-19T08:15:01.308878+0000","last_undegraded":"2024-09-19T08:15:01.308878+0000","last_fullsized":"2024-09-19T08:15:01.308878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:08.424106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308358+0000","last_change":"2024-09-19T08:13:00.855870+0000","last_active":"2024-09-19T08:15:01.308358+0000","last_peered":"2024-09-19T08:15:01.308358+0000","last_clean":"2024-09-19T08:15:01.308358+0000","last_became_active":"2024-09-19T08:13:00.855538+0000","last_became_peered":"2024-09-19T08:13:00.855538+0000","last_unstale":"2024-09-19T08:15:01.308358+0000","last_undegraded":"2024-09-19T08:15:01.308358+0000","last_fullsized":"2024-09-19T08:15:01.308358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"86'148","reported_seq":395,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:03.309732+0000","last_change":"2024-09-19T08:11:48.661333+0000","last_active":"2024-09-19T08:15:03.309732+0000","last_peered":"2024-09-19T08:15:03.309732+0000","last_clean":"2024-09-19T08:15:03.309732+0000","last_became_active":"2024-09-19T08:11:48.660873+0000","last_became_peered":"2024-09-19T08:11:48.660873+0000","last_unstale":"2024-09-19T08:15:03.309732+0000","last_undegraded":"2024-09-19T08:15:03.309732+0000","last_fullsized":"2024-09-19T08:15:03.309732+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_clean_scrub_stamp":"2024-09-19T08:08:25.372155+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:19.401292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172985+0000","last_change":"2024-09-19T08:12:58.675237+0000","last_active":"2024-09-19T08:15:01.172985+0000","last_peered":"2024-09-19T08:15:01.172985+0000","last_clean":"2024-09-19T08:15:01.172985+0000","last_became_active":"2024-09-19T08:12:58.674910+0000","last_became_peered":"2024-09-19T08:12:58.674910+0000","last_unstale":"2024-09-19T08:15:01.172985+0000","last_undegraded":"2024-09-19T08:15:01.172985+0000","last_fullsized":"2024-09-19T08:15:01.172985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:47.038328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171081+0000","last_change":"2024-09-19T08:13:00.846754+0000","last_active":"2024-09-19T08:15:01.171081+0000","last_peered":"2024-09-19T08:15:01.171081+0000","last_clean":"2024-09-19T08:15:01.171081+0000","last_became_active":"2024-09-19T08:13:00.846452+0000","last_became_peered":"2024-09-19T08:13:00.846452+0000","last_unstale":"2024-09-19T08:15:01.171081+0000","last_undegraded":"2024-09-19T08:15:01.171081+0000","last_fullsized":"2024-09-19T08:15:01.171081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:22.929689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308239+0000","last_change":"2024-09-19T08:12:58.678958+0000","last_active":"2024-09-19T08:15:01.308239+0000","last_peered":"2024-09-19T08:15:01.308239+0000","last_clean":"2024-09-19T08:15:01.308239+0000","last_became_active":"2024-09-19T08:12:58.678592+0000","last_became_peered":"2024-09-19T08:12:58.678592+0000","last_unstale":"2024-09-19T08:15:01.308239+0000","last_undegraded":"2024-09-19T08:15:01.308239+0000","last_fullsized":"2024-09-19T08:15:01.308239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:58.374606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309700+0000","last_change":"2024-09-19T08:13:00.840973+0000","last_active":"2024-09-19T08:15:01.309700+0000","last_peered":"2024-09-19T08:15:01.309700+0000","last_clean":"2024-09-19T08:15:01.309700+0000","last_became_active":"2024-09-19T08:13:00.840619+0000","last_became_peered":"2024-09-19T08:13:00.840619+0000","last_unstale":"2024-09-19T08:15:01.309700+0000","last_undegraded":"2024-09-19T08:15:01.309700+0000","last_fullsized":"2024-09-19T08:15:01.309700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":152,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.050056+0000","last_change":"2024-09-19T08:12:58.715972+0000","last_active":"2024-09-19T08:15:11.050056+0000","last_peered":"2024-09-19T08:15:11.050056+0000","last_clean":"2024-09-19T08:15:11.050056+0000","last_became_active":"2024-09-19T08:12:58.715630+0000","last_became_peered":"2024-09-19T08:12:58.715630+0000","last_unstale":"2024-09-19T08:15:11.050056+0000","last_undegraded":"2024-09-19T08:15:11.050056+0000","last_fullsized":"2024-09-19T08:15:11.050056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:35.637121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172244+0000","last_change":"2024-09-19T08:13:00.847023+0000","last_active":"2024-09-19T08:15:01.172244+0000","last_peered":"2024-09-19T08:15:01.172244+0000","last_clean":"2024-09-19T08:15:01.172244+0000","last_became_active":"2024-09-19T08:13:00.846387+0000","last_became_peered":"2024-09-19T08:13:00.846387+0000","last_unstale":"2024-09-19T08:15:01.172244+0000","last_undegraded":"2024-09-19T08:15:01.172244+0000","last_fullsized":"2024-09-19T08:15:01.172244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:12.187929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308283+0000","last_change":"2024-09-19T08:12:58.681062+0000","last_active":"2024-09-19T08:15:01.308283+0000","last_peered":"2024-09-19T08:15:01.308283+0000","last_clean":"2024-09-19T08:15:01.308283+0000","last_became_active":"2024-09-19T08:12:58.680524+0000","last_became_peered":"2024-09-19T08:12:58.680524+0000","last_unstale":"2024-09-19T08:15:01.308283+0000","last_undegraded":"2024-09-19T08:15:01.308283+0000","last_fullsized":"2024-09-19T08:15:01.308283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271807+0000","last_change":"2024-09-19T08:13:00.851523+0000","last_active":"2024-09-19T08:15:01.271807+0000","last_peered":"2024-09-19T08:15:01.271807+0000","last_clean":"2024-09-19T08:15:01.271807+0000","last_became_active":"2024-09-19T08:13:00.851068+0000","last_became_peered":"2024-09-19T08:13:00.851068+0000","last_unstale":"2024-09-19T08:15:01.271807+0000","last_undegraded":"2024-09-19T08:15:01.271807+0000","last_fullsized":"2024-09-19T08:15:01.271807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:44.905469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.260925+0000","last_change":"2024-09-19T08:12:58.718234+0000","last_active":"2024-09-19T08:15:01.260925+0000","last_peered":"2024-09-19T08:15:01.260925+0000","last_clean":"2024-09-19T08:15:01.260925+0000","last_became_active":"2024-09-19T08:12:58.717840+0000","last_became_peered":"2024-09-19T08:12:58.717840+0000","last_unstale":"2024-09-19T08:15:01.260925+0000","last_undegraded":"2024-09-19T08:15:01.260925+0000","last_fullsized":"2024-09-19T08:15:01.260925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:05.283770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271686+0000","last_change":"2024-09-19T08:13:00.848305+0000","last_active":"2024-09-19T08:15:01.271686+0000","last_peered":"2024-09-19T08:15:01.271686+0000","last_clean":"2024-09-19T08:15:01.271686+0000","last_became_active":"2024-09-19T08:13:00.847981+0000","last_became_peered":"2024-09-19T08:13:00.847981+0000","last_unstale":"2024-09-19T08:15:01.271686+0000","last_undegraded":"2024-09-19T08:15:01.271686+0000","last_fullsized":"2024-09-19T08:15:01.271686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:04.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263351+0000","last_change":"2024-09-19T08:12:58.715943+0000","last_active":"2024-09-19T08:15:01.263351+0000","last_peered":"2024-09-19T08:15:01.263351+0000","last_clean":"2024-09-19T08:15:01.263351+0000","last_became_active":"2024-09-19T08:12:58.715600+0000","last_became_peered":"2024-09-19T08:12:58.715600+0000","last_unstale":"2024-09-19T08:15:01.263351+0000","last_undegraded":"2024-09-19T08:15:01.263351+0000","last_fullsized":"2024-09-19T08:15:01.263351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:19.142271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310659+0000","last_change":"2024-09-19T08:13:00.840944+0000","last_active":"2024-09-19T08:15:01.310659+0000","last_peered":"2024-09-19T08:15:01.310659+0000","last_clean":"2024-09-19T08:15:01.310659+0000","last_became_active":"2024-09-19T08:13:00.840608+0000","last_became_peered":"2024-09-19T08:13:00.840608+0000","last_unstale":"2024-09-19T08:15:01.310659+0000","last_undegraded":"2024-09-19T08:15:01.310659+0000","last_fullsized":"2024-09-19T08:15:01.310659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:25.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'97","reported_seq":208,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.918689+0000","last_change":"2024-09-19T08:12:58.672929+0000","last_active":"2024-09-19T08:15:11.918689+0000","last_peered":"2024-09-19T08:15:11.918689+0000","last_clean":"2024-09-19T08:15:11.918689+0000","last_became_active":"2024-09-19T08:12:58.672487+0000","last_became_peered":"2024-09-19T08:12:58.672487+0000","last_unstale":"2024-09-19T08:15:11.918689+0000","last_undegraded":"2024-09-19T08:15:11.918689+0000","last_fullsized":"2024-09-19T08:15:11.918689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:33.042735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":177,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.261959+0000","last_change":"2024-09-19T08:14:07.738260+0000","last_active":"2024-09-19T08:15:01.261959+0000","last_peered":"2024-09-19T08:15:01.261959+0000","last_clean":"2024-09-19T08:15:01.261959+0000","last_became_active":"2024-09-19T08:14:07.737915+0000","last_became_peered":"2024-09-19T08:14:07.737915+0000","last_unstale":"2024-09-19T08:15:01.261959+0000","last_undegraded":"2024-09-19T08:15:01.261959+0000","last_fullsized":"2024-09-19T08:15:01.261959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:02.500657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.173307+0000","last_change":"2024-09-19T08:12:59.179510+0000","last_active":"2024-09-19T08:15:01.173307+0000","last_peered":"2024-09-19T08:15:01.173307+0000","last_clean":"2024-09-19T08:15:01.173307+0000","last_became_active":"2024-09-19T08:12:59.179157+0000","last_became_peered":"2024-09-19T08:12:59.179157+0000","last_unstale":"2024-09-19T08:15:01.173307+0000","last_undegraded":"2024-09-19T08:15:01.173307+0000","last_fullsized":"2024-09-19T08:15:01.173307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:22.981289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308670+0000","last_change":"2024-09-19T08:13:00.841853+0000","last_active":"2024-09-19T08:15:01.308670+0000","last_peered":"2024-09-19T08:15:01.308670+0000","last_clean":"2024-09-19T08:15:01.308670+0000","last_became_active":"2024-09-19T08:13:00.841518+0000","last_became_peered":"2024-09-19T08:13:00.841518+0000","last_unstale":"2024-09-19T08:15:01.308670+0000","last_undegraded":"2024-09-19T08:15:01.308670+0000","last_fullsized":"2024-09-19T08:15:01.308670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:05.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223446+0000","last_change":"2024-09-19T08:12:58.716882+0000","last_active":"2024-09-19T08:15:01.223446+0000","last_peered":"2024-09-19T08:15:01.223446+0000","last_clean":"2024-09-19T08:15:01.223446+0000","last_became_active":"2024-09-19T08:12:58.716545+0000","last_became_peered":"2024-09-19T08:12:58.716545+0000","last_unstale":"2024-09-19T08:15:01.223446+0000","last_undegraded":"2024-09-19T08:15:01.223446+0000","last_fullsized":"2024-09-19T08:15:01.223446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:14.469109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262999+0000","last_change":"2024-09-19T08:13:00.850484+0000","last_active":"2024-09-19T08:15:01.262999+0000","last_peered":"2024-09-19T08:15:01.262999+0000","last_clean":"2024-09-19T08:15:01.262999+0000","last_became_active":"2024-09-19T08:13:00.850049+0000","last_became_peered":"2024-09-19T08:13:00.850049+0000","last_unstale":"2024-09-19T08:15:01.262999+0000","last_undegraded":"2024-09-19T08:15:01.262999+0000","last_fullsized":"2024-09-19T08:15:01.262999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:24.424111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170992+0000","last_change":"2024-09-19T08:12:58.669928+0000","last_active":"2024-09-19T08:15:01.170992+0000","last_peered":"2024-09-19T08:15:01.170992+0000","last_clean":"2024-09-19T08:15:01.170992+0000","last_became_active":"2024-09-19T08:12:58.669614+0000","last_became_peered":"2024-09-19T08:12:58.669614+0000","last_unstale":"2024-09-19T08:15:01.170992+0000","last_undegraded":"2024-09-19T08:15:01.170992+0000","last_fullsized":"2024-09-19T08:15:01.170992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:50.924779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308498+0000","last_change":"2024-09-19T08:13:00.853520+0000","last_active":"2024-09-19T08:15:01.308498+0000","last_peered":"2024-09-19T08:15:01.308498+0000","last_clean":"2024-09-19T08:15:01.308498+0000","last_became_active":"2024-09-19T08:13:00.853184+0000","last_became_peered":"2024-09-19T08:13:00.853184+0000","last_unstale":"2024-09-19T08:15:01.308498+0000","last_undegraded":"2024-09-19T08:15:01.308498+0000","last_fullsized":"2024-09-19T08:15:01.308498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:45.160439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:02.882016+0000","last_change":"2024-09-19T08:12:58.674359+0000","last_active":"2024-09-19T08:15:02.882016+0000","last_peered":"2024-09-19T08:15:02.882016+0000","last_clean":"2024-09-19T08:15:02.882016+0000","last_became_active":"2024-09-19T08:12:58.673890+0000","last_became_peered":"2024-09-19T08:12:58.673890+0000","last_unstale":"2024-09-19T08:15:02.882016+0000","last_undegraded":"2024-09-19T08:15:02.882016+0000","last_fullsized":"2024-09-19T08:15:02.882016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:54.939959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171020+0000","last_change":"2024-09-19T08:13:00.851071+0000","last_active":"2024-09-19T08:15:01.171020+0000","last_peered":"2024-09-19T08:15:01.171020+0000","last_clean":"2024-09-19T08:15:01.171020+0000","last_became_active":"2024-09-19T08:13:00.850756+0000","last_became_peered":"2024-09-19T08:13:00.850756+0000","last_unstale":"2024-09-19T08:15:01.171020+0000","last_undegraded":"2024-09-19T08:15:01.171020+0000","last_fullsized":"2024-09-19T08:15:01.171020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:52.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263074+0000","last_change":"2024-09-19T08:12:58.681466+0000","last_active":"2024-09-19T08:15:01.263074+0000","last_peered":"2024-09-19T08:15:01.263074+0000","last_clean":"2024-09-19T08:15:01.263074+0000","last_became_active":"2024-09-19T08:12:58.681113+0000","last_became_peered":"2024-09-19T08:12:58.681113+0000","last_unstale":"2024-09-19T08:15:01.263074+0000","last_undegraded":"2024-09-19T08:15:01.263074+0000","last_fullsized":"2024-09-19T08:15:01.263074+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:49.610828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308435+0000","last_change":"2024-09-19T08:13:00.851272+0000","last_active":"2024-09-19T08:15:01.308435+0000","last_peered":"2024-09-19T08:15:01.308435+0000","last_clean":"2024-09-19T08:15:01.308435+0000","last_became_active":"2024-09-19T08:13:00.849098+0000","last_became_peered":"2024-09-19T08:13:00.849098+0000","last_unstale":"2024-09-19T08:15:01.308435+0000","last_undegraded":"2024-09-19T08:15:01.308435+0000","last_fullsized":"2024-09-19T08:15:01.308435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:05.643992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172513+0000","last_change":"2024-09-19T08:14:07.738913+0000","last_active":"2024-09-19T08:15:01.172513+0000","last_peered":"2024-09-19T08:15:01.172513+0000","last_clean":"2024-09-19T08:15:01.172513+0000","last_became_active":"2024-09-19T08:14:07.738560+0000","last_became_peered":"2024-09-19T08:14:07.738560+0000","last_unstale":"2024-09-19T08:15:01.172513+0000","last_undegraded":"2024-09-19T08:15:01.172513+0000","last_fullsized":"2024-09-19T08:15:01.172513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:35.731290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263192+0000","last_change":"2024-09-19T08:12:58.674610+0000","last_active":"2024-09-19T08:15:01.263192+0000","last_peered":"2024-09-19T08:15:01.263192+0000","last_clean":"2024-09-19T08:15:01.263192+0000","last_became_active":"2024-09-19T08:12:58.674247+0000","last_became_peered":"2024-09-19T08:12:58.674247+0000","last_unstale":"2024-09-19T08:15:01.263192+0000","last_undegraded":"2024-09-19T08:15:01.263192+0000","last_fullsized":"2024-09-19T08:15:01.263192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:16.323574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223395+0000","last_change":"2024-09-19T08:13:00.858512+0000","last_active":"2024-09-19T08:15:01.223395+0000","last_peered":"2024-09-19T08:15:01.223395+0000","last_clean":"2024-09-19T08:15:01.223395+0000","last_became_active":"2024-09-19T08:13:00.857897+0000","last_became_peered":"2024-09-19T08:13:00.857897+0000","last_unstale":"2024-09-19T08:15:01.223395+0000","last_undegraded":"2024-09-19T08:15:01.223395+0000","last_fullsized":"2024-09-19T08:15:01.223395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263238+0000","last_change":"2024-09-19T08:12:58.682207+0000","last_active":"2024-09-19T08:15:01.263238+0000","last_peered":"2024-09-19T08:15:01.263238+0000","last_clean":"2024-09-19T08:15:01.263238+0000","last_became_active":"2024-09-19T08:12:58.681842+0000","last_became_peered":"2024-09-19T08:12:58.681842+0000","last_unstale":"2024-09-19T08:15:01.263238+0000","last_undegraded":"2024-09-19T08:15:01.263238+0000","last_fullsized":"2024-09-19T08:15:01.263238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.825858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308892+0000","last_change":"2024-09-19T08:13:00.846748+0000","last_active":"2024-09-19T08:15:01.308892+0000","last_peered":"2024-09-19T08:15:01.308892+0000","last_clean":"2024-09-19T08:15:01.308892+0000","last_became_active":"2024-09-19T08:13:00.846368+0000","last_became_peered":"2024-09-19T08:13:00.846368+0000","last_unstale":"2024-09-19T08:15:01.308892+0000","last_undegraded":"2024-09-19T08:15:01.308892+0000","last_fullsized":"2024-09-19T08:15:01.308892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:59.526512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171334+0000","last_change":"2024-09-19T08:12:58.716118+0000","last_active":"2024-09-19T08:15:01.171334+0000","last_peered":"2024-09-19T08:15:01.171334+0000","last_clean":"2024-09-19T08:15:01.171334+0000","last_became_active":"2024-09-19T08:12:58.715711+0000","last_became_peered":"2024-09-19T08:12:58.715711+0000","last_unstale":"2024-09-19T08:15:01.171334+0000","last_undegraded":"2024-09-19T08:15:01.171334+0000","last_fullsized":"2024-09-19T08:15:01.171334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:29.439887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171226+0000","last_change":"2024-09-19T08:13:00.848729+0000","last_active":"2024-09-19T08:15:01.171226+0000","last_peered":"2024-09-19T08:15:01.171226+0000","last_clean":"2024-09-19T08:15:01.171226+0000","last_became_active":"2024-09-19T08:13:00.848470+0000","last_became_peered":"2024-09-19T08:13:00.848470+0000","last_unstale":"2024-09-19T08:15:01.171226+0000","last_undegraded":"2024-09-19T08:15:01.171226+0000","last_fullsized":"2024-09-19T08:15:01.171226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:30.653101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:07.185959+0000","last_change":"2024-09-19T08:12:58.681654+0000","last_active":"2024-09-19T08:15:07.185959+0000","last_peered":"2024-09-19T08:15:07.185959+0000","last_clean":"2024-09-19T08:15:07.185959+0000","last_became_active":"2024-09-19T08:12:58.681284+0000","last_became_peered":"2024-09-19T08:12:58.681284+0000","last_unstale":"2024-09-19T08:15:07.185959+0000","last_undegraded":"2024-09-19T08:15:07.185959+0000","last_fullsized":"2024-09-19T08:15:07.185959+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.429930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798886,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1081,"num_read_kb":750,"num_write":423,"num_write_kb":2364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":61306,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":618496,"data_stored":2396658,"data_compressed":113084,"data_compressed_allocated":147456,"data_compressed_original":2158592,"omap_allocated":0,"internal_metadata":0},"log_size":314,"ondisk_log_size":314,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":109176,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481830,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":454888,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710764,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1138688,"data_stored":1446975,"data_compressed":59949,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874289,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":848206,"data_compressed":28006,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103222,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":958464,"data_stored":847992,"data_compressed":27863,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332155,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":1049397,"data_compressed":45109,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495680,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":933888,"data_stored":454379,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757318,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1052672,"data_stored":1375685,"data_compressed":60311,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":527324,"data_compressed":18328,"data_compressed_allocated":24576,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":979506,"data_compressed":34019,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":456842,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":525052,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":455650,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":73152,"data_compressed":9611,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":331028,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":393820,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":402318,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:15:16.238 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T08:15:16.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:15:16.502 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:16.742 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:16 smithi022 ceph-mon[30963]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-19T08:15:16.742 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:16 smithi022 ceph-mon[30963]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:16.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:16 smithi157 ceph-mon[35143]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-19T08:15:16.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:16 smithi157 ceph-mon[35143]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:16.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:16 smithi064 ceph-mon[34900]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-19T08:15:16.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:16 smithi064 ceph-mon[34900]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:16.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.960+0000 7f44c8ea8640 1 -- 172.21.15.22:0/379723723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44c4108290 msgr2=0x7f44c410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:16.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.960+0000 7f44c8ea8640 1 --2- 172.21.15.22:0/379723723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44c4108290 0x7f44c410ec30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009cd0 tx=0x7f44bc02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:15:16.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.960+0000 7f44c8ea8640 1 -- 172.21.15.22:0/379723723 shutdown_connections 2024-09-19T08:15:16.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.960+0000 7f44c8ea8640 1 --2- 172.21.15.22:0/379723723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44c4108290 0x7f44c410ec30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:16.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.960+0000 7f44c8ea8640 1 --2- 172.21.15.22:0/379723723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44c4103740 0x7f44c4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:16.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.960+0000 7f44c8ea8640 1 --2- 172.21.15.22:0/379723723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c4102d90 0x7f44c4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:16.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.960+0000 7f44c8ea8640 1 -- 172.21.15.22:0/379723723 >> 172.21.15.22:0/379723723 conn(0x7f44c40fc7d0 msgr2=0x7f44c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.960+0000 7f44c8ea8640 1 -- 172.21.15.22:0/379723723 shutdown_connections 2024-09-19T08:15:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.960+0000 7f44c8ea8640 1 -- 172.21.15.22:0/379723723 wait complete. 2024-09-19T08:15:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.961+0000 7f44c8ea8640 1 Processor -- start 2024-09-19T08:15:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.961+0000 7f44c8ea8640 1 -- start start 2024-09-19T08:15:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.961+0000 7f44c8ea8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c4102d90 0x7f44c41009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.961+0000 7f44c8ea8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44c4103740 0x7f44c4100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.961+0000 7f44c8ea8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44c4108290 0x7f44c4101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.961+0000 7f44c8ea8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4112090 con 0x7f44c4103740 2024-09-19T08:15:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.961+0000 7f44c8ea8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4111f10 con 0x7f44c4108290 2024-09-19T08:15:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.961+0000 7f44c8ea8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4112210 con 0x7f44c4102d90 2024-09-19T08:15:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.961+0000 7f44c1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44c4103740 0x7f44c4100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.961+0000 7f44c1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44c4103740 0x7f44c4100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42996/0 (socket says 172.21.15.22:42996) 2024-09-19T08:15:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.961+0000 7f44c1d9b640 1 -- 172.21.15.22:0/2195016430 learned_addr learned my addr 172.21.15.22:0/2195016430 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.961+0000 7f44c2d9d640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44c4108290 0x7f44c4101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.961+0000 7f44c259c640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c4102d90 0x7f44c41009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.962+0000 7f44c1d9b640 1 -- 172.21.15.22:0/2195016430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c4102d90 msgr2=0x7f44c41009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.962+0000 7f44c1d9b640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c4102d90 0x7f44c41009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.962+0000 7f44c1d9b640 1 -- 172.21.15.22:0/2195016430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44c4108290 msgr2=0x7f44c4101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.962+0000 7f44c1d9b640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44c4108290 0x7f44c4101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.962+0000 7f44c1d9b640 1 -- 172.21.15.22:0/2195016430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c41a6a30 con 0x7f44c4103740 2024-09-19T08:15:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.962+0000 7f44c259c640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c4102d90 0x7f44c41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.962+0000 7f44c2d9d640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44c4108290 0x7f44c4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.962+0000 7f44c1d9b640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44c4103740 0x7f44c4100f30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f44b8005fd0 tx=0x7f44b800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.962+0000 7f44b37fe640 1 -- 172.21.15.22:0/2195016430 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8017070 con 0x7f44c4103740 2024-09-19T08:15:16.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.962+0000 7f44b37fe640 1 -- 172.21.15.22:0/2195016430 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b8009c30 con 0x7f44c4103740 2024-09-19T08:15:16.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.962+0000 7f44c8ea8640 1 -- 172.21.15.22:0/2195016430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c41a6d20 con 0x7f44c4103740 2024-09-19T08:15:16.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.962+0000 7f44b37fe640 1 -- 172.21.15.22:0/2195016430 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8010460 con 0x7f44c4103740 2024-09-19T08:15:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.962+0000 7f44c8ea8640 1 -- 172.21.15.22:0/2195016430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c41a7250 con 0x7f44c4103740 2024-09-19T08:15:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.963+0000 7f44c8ea8640 1 -- 172.21.15.22:0/2195016430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c406a7f0 con 0x7f44c4103740 2024-09-19T08:15:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.965+0000 7f44b37fe640 1 -- 172.21.15.22:0/2195016430 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f44b8003e70 con 0x7f44c4103740 2024-09-19T08:15:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.965+0000 7f44b37fe640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f44ac0a3550 0x7f44ac0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.966+0000 7f44c259c640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f44ac0a3550 0x7f44ac0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.966+0000 7f44b37fe640 1 -- 172.21.15.22:0/2195016430 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f44b80cfb40 con 0x7f44c4103740 2024-09-19T08:15:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.966+0000 7f44c259c640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f44ac0a3550 0x7f44ac0a5a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44b40058c0 tx=0x7f44b4005850 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:16.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:16.970+0000 7f44b37fe640 1 -- 172.21.15.22:0/2195016430 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f44b8020030 con 0x7f44c4103740 2024-09-19T08:15:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:17.142+0000 7f44c8ea8640 1 -- 172.21.15.22:0/2195016430 --> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44c410a310 con 0x7f44ac0a3550 2024-09-19T08:15:17.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:17.149+0000 7f44b37fe640 1 -- 172.21.15.22:0/2195016430 <== mgr.15141 v2:172.21.15.22:6800/497251935 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258054 (secure 0 0 0) 0x7f44b400bca0 con 0x7f44ac0a3550 2024-09-19T08:15:17.151 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:17.152 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T08:15:17.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:17.152+0000 7f44c8ea8640 1 -- 172.21.15.22:0/2195016430 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f44ac0a3550 msgr2=0x7f44ac0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:17.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:17.152+0000 7f44c8ea8640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f44ac0a3550 0x7f44ac0a5a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44b40058c0 tx=0x7f44b4005850 comp rx=0 tx=0).stop 2024-09-19T08:15:17.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:17.152+0000 7f44c8ea8640 1 -- 172.21.15.22:0/2195016430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44c4103740 msgr2=0x7f44c4100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:17.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:17.152+0000 7f44c8ea8640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44c4103740 0x7f44c4100f30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f44b8005fd0 tx=0x7f44b800dde0 comp rx=0 tx=0).stop 2024-09-19T08:15:17.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:17.152+0000 7f44c8ea8640 1 -- 172.21.15.22:0/2195016430 shutdown_connections 2024-09-19T08:15:17.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:17.152+0000 7f44c8ea8640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f44ac0a3550 0x7f44ac0a5a10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:17.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:17.152+0000 7f44c8ea8640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44c4108290 0x7f44c4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:17.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:17.152+0000 7f44c8ea8640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44c4103740 0x7f44c4100f30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:17.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:17.152+0000 7f44c8ea8640 1 --2- 172.21.15.22:0/2195016430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c4102d90 0x7f44c41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:17.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:17.152+0000 7f44c8ea8640 1 -- 172.21.15.22:0/2195016430 >> 172.21.15.22:0/2195016430 conn(0x7f44c40fc7d0 msgr2=0x7f44c4105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:17.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:17.152+0000 7f44c8ea8640 1 -- 172.21.15.22:0/2195016430 shutdown_connections 2024-09-19T08:15:17.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:17.152+0000 7f44c8ea8640 1 -- 172.21.15.22:0/2195016430 wait complete. 2024-09-19T08:15:17.813 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-19T08:15:15.188192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389488,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1318,"num_read_kb":1080,"num_write":626,"num_write_kb":5304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1242042,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":964,"ondisk_log_size":964,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381672,"kb_used_data":11596,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560152,"statfs":{"total":1151940427776,"available":1151549595648,"internally_reserved":0,"allocated":11874304,"data_stored":9421896,"data_compressed":317170,"data_compressed_allocated":417792,"data_compressed_original":4038656,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201076,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":398,"num_read_kb":460,"num_write":146,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002688"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309162+0000","last_change":"2024-09-19T08:13:00.853947+0000","last_active":"2024-09-19T08:15:01.309162+0000","last_peered":"2024-09-19T08:15:01.309162+0000","last_clean":"2024-09-19T08:15:01.309162+0000","last_became_active":"2024-09-19T08:13:00.852951+0000","last_became_peered":"2024-09-19T08:13:00.852951+0000","last_unstale":"2024-09-19T08:15:01.309162+0000","last_undegraded":"2024-09-19T08:15:01.309162+0000","last_fullsized":"2024-09-19T08:15:01.309162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:25.786828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308625+0000","last_change":"2024-09-19T08:13:00.853389+0000","last_active":"2024-09-19T08:15:01.308625+0000","last_peered":"2024-09-19T08:15:01.308625+0000","last_clean":"2024-09-19T08:15:01.308625+0000","last_became_active":"2024-09-19T08:13:00.853102+0000","last_became_peered":"2024-09-19T08:13:00.853102+0000","last_unstale":"2024-09-19T08:15:01.308625+0000","last_undegraded":"2024-09-19T08:15:01.308625+0000","last_fullsized":"2024-09-19T08:15:01.308625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:42.773917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308869+0000","last_change":"2024-09-19T08:13:00.846544+0000","last_active":"2024-09-19T08:15:01.308869+0000","last_peered":"2024-09-19T08:15:01.308869+0000","last_clean":"2024-09-19T08:15:01.308869+0000","last_became_active":"2024-09-19T08:13:00.846229+0000","last_became_peered":"2024-09-19T08:13:00.846229+0000","last_unstale":"2024-09-19T08:15:01.308869+0000","last_undegraded":"2024-09-19T08:15:01.308869+0000","last_fullsized":"2024-09-19T08:15:01.308869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:43.471684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172938+0000","last_change":"2024-09-19T08:13:00.848149+0000","last_active":"2024-09-19T08:15:01.172938+0000","last_peered":"2024-09-19T08:15:01.172938+0000","last_clean":"2024-09-19T08:15:01.172938+0000","last_became_active":"2024-09-19T08:13:00.847744+0000","last_became_peered":"2024-09-19T08:13:00.847744+0000","last_unstale":"2024-09-19T08:15:01.172938+0000","last_undegraded":"2024-09-19T08:15:01.172938+0000","last_fullsized":"2024-09-19T08:15:01.172938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:01.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263000+0000","last_change":"2024-09-19T08:13:00.841862+0000","last_active":"2024-09-19T08:15:01.263000+0000","last_peered":"2024-09-19T08:15:01.263000+0000","last_clean":"2024-09-19T08:15:01.263000+0000","last_became_active":"2024-09-19T08:13:00.841331+0000","last_became_peered":"2024-09-19T08:13:00.841331+0000","last_unstale":"2024-09-19T08:15:01.263000+0000","last_undegraded":"2024-09-19T08:15:01.263000+0000","last_fullsized":"2024-09-19T08:15:01.263000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:54.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271477+0000","last_change":"2024-09-19T08:13:00.849626+0000","last_active":"2024-09-19T08:15:01.271477+0000","last_peered":"2024-09-19T08:15:01.271477+0000","last_clean":"2024-09-19T08:15:01.271477+0000","last_became_active":"2024-09-19T08:13:00.849034+0000","last_became_peered":"2024-09-19T08:13:00.849034+0000","last_unstale":"2024-09-19T08:15:01.271477+0000","last_undegraded":"2024-09-19T08:15:01.271477+0000","last_fullsized":"2024-09-19T08:15:01.271477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:39.118978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308024+0000","last_change":"2024-09-19T08:13:00.847522+0000","last_active":"2024-09-19T08:15:01.308024+0000","last_peered":"2024-09-19T08:15:01.308024+0000","last_clean":"2024-09-19T08:15:01.308024+0000","last_became_active":"2024-09-19T08:13:00.847117+0000","last_became_peered":"2024-09-19T08:13:00.847117+0000","last_unstale":"2024-09-19T08:15:01.308024+0000","last_undegraded":"2024-09-19T08:15:01.308024+0000","last_fullsized":"2024-09-19T08:15:01.308024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.836385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170733+0000","last_change":"2024-09-19T08:13:00.849544+0000","last_active":"2024-09-19T08:15:01.170733+0000","last_peered":"2024-09-19T08:15:01.170733+0000","last_clean":"2024-09-19T08:15:01.170733+0000","last_became_active":"2024-09-19T08:13:00.849170+0000","last_became_peered":"2024-09-19T08:13:00.849170+0000","last_unstale":"2024-09-19T08:15:01.170733+0000","last_undegraded":"2024-09-19T08:15:01.170733+0000","last_fullsized":"2024-09-19T08:15:01.170733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:28.371965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223727+0000","last_change":"2024-09-19T08:13:00.856052+0000","last_active":"2024-09-19T08:15:01.223727+0000","last_peered":"2024-09-19T08:15:01.223727+0000","last_clean":"2024-09-19T08:15:01.223727+0000","last_became_active":"2024-09-19T08:13:00.855291+0000","last_became_peered":"2024-09-19T08:13:00.855291+0000","last_unstale":"2024-09-19T08:15:01.223727+0000","last_undegraded":"2024-09-19T08:15:01.223727+0000","last_fullsized":"2024-09-19T08:15:01.223727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:18.581867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271887+0000","last_change":"2024-09-19T08:13:00.850145+0000","last_active":"2024-09-19T08:15:01.271887+0000","last_peered":"2024-09-19T08:15:01.271887+0000","last_clean":"2024-09-19T08:15:01.271887+0000","last_became_active":"2024-09-19T08:13:00.849798+0000","last_became_peered":"2024-09-19T08:13:00.849798+0000","last_unstale":"2024-09-19T08:15:01.271887+0000","last_undegraded":"2024-09-19T08:15:01.271887+0000","last_fullsized":"2024-09-19T08:15:01.271887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:21.400000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308275+0000","last_change":"2024-09-19T08:13:00.859153+0000","last_active":"2024-09-19T08:15:01.308275+0000","last_peered":"2024-09-19T08:15:01.308275+0000","last_clean":"2024-09-19T08:15:01.308275+0000","last_became_active":"2024-09-19T08:13:00.858473+0000","last_became_peered":"2024-09-19T08:13:00.858473+0000","last_unstale":"2024-09-19T08:15:01.308275+0000","last_undegraded":"2024-09-19T08:15:01.308275+0000","last_fullsized":"2024-09-19T08:15:01.308275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:07.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172922+0000","last_change":"2024-09-19T08:13:00.850797+0000","last_active":"2024-09-19T08:15:01.172922+0000","last_peered":"2024-09-19T08:15:01.172922+0000","last_clean":"2024-09-19T08:15:01.172922+0000","last_became_active":"2024-09-19T08:13:00.850393+0000","last_became_peered":"2024-09-19T08:13:00.850393+0000","last_unstale":"2024-09-19T08:15:01.172922+0000","last_undegraded":"2024-09-19T08:15:01.172922+0000","last_fullsized":"2024-09-19T08:15:01.172922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:59.099331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310082+0000","last_change":"2024-09-19T08:13:00.850835+0000","last_active":"2024-09-19T08:15:01.310082+0000","last_peered":"2024-09-19T08:15:01.310082+0000","last_clean":"2024-09-19T08:15:01.310082+0000","last_became_active":"2024-09-19T08:13:00.850518+0000","last_became_peered":"2024-09-19T08:13:00.850518+0000","last_unstale":"2024-09-19T08:15:01.310082+0000","last_undegraded":"2024-09-19T08:15:01.310082+0000","last_fullsized":"2024-09-19T08:15:01.310082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.178530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308450+0000","last_change":"2024-09-19T08:13:00.853934+0000","last_active":"2024-09-19T08:15:01.308450+0000","last_peered":"2024-09-19T08:15:01.308450+0000","last_clean":"2024-09-19T08:15:01.308450+0000","last_became_active":"2024-09-19T08:13:00.853604+0000","last_became_peered":"2024-09-19T08:13:00.853604+0000","last_unstale":"2024-09-19T08:15:01.308450+0000","last_undegraded":"2024-09-19T08:15:01.308450+0000","last_fullsized":"2024-09-19T08:15:01.308450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:43.622526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308262+0000","last_change":"2024-09-19T08:14:07.741370+0000","last_active":"2024-09-19T08:15:01.308262+0000","last_peered":"2024-09-19T08:15:01.308262+0000","last_clean":"2024-09-19T08:15:01.308262+0000","last_became_active":"2024-09-19T08:14:07.741116+0000","last_became_peered":"2024-09-19T08:14:07.741116+0000","last_unstale":"2024-09-19T08:15:01.308262+0000","last_undegraded":"2024-09-19T08:15:01.308262+0000","last_fullsized":"2024-09-19T08:15:01.308262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:45.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223320+0000","last_change":"2024-09-19T08:13:00.850266+0000","last_active":"2024-09-19T08:15:01.223320+0000","last_peered":"2024-09-19T08:15:01.223320+0000","last_clean":"2024-09-19T08:15:01.223320+0000","last_became_active":"2024-09-19T08:13:00.849928+0000","last_became_peered":"2024-09-19T08:13:00.849928+0000","last_unstale":"2024-09-19T08:15:01.223320+0000","last_undegraded":"2024-09-19T08:15:01.223320+0000","last_fullsized":"2024-09-19T08:15:01.223320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:38.231331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309177+0000","last_change":"2024-09-19T08:13:00.852551+0000","last_active":"2024-09-19T08:15:01.309177+0000","last_peered":"2024-09-19T08:15:01.309177+0000","last_clean":"2024-09-19T08:15:01.309177+0000","last_became_active":"2024-09-19T08:13:00.852225+0000","last_became_peered":"2024-09-19T08:13:00.852225+0000","last_unstale":"2024-09-19T08:15:01.309177+0000","last_undegraded":"2024-09-19T08:15:01.309177+0000","last_fullsized":"2024-09-19T08:15:01.309177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:33.115108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.261006+0000","last_change":"2024-09-19T08:13:00.854033+0000","last_active":"2024-09-19T08:15:01.261006+0000","last_peered":"2024-09-19T08:15:01.261006+0000","last_clean":"2024-09-19T08:15:01.261006+0000","last_became_active":"2024-09-19T08:13:00.853660+0000","last_became_peered":"2024-09-19T08:13:00.853660+0000","last_unstale":"2024-09-19T08:15:01.261006+0000","last_undegraded":"2024-09-19T08:15:01.261006+0000","last_fullsized":"2024-09-19T08:15:01.261006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:35.602420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308765+0000","last_change":"2024-09-19T08:13:00.850658+0000","last_active":"2024-09-19T08:15:01.308765+0000","last_peered":"2024-09-19T08:15:01.308765+0000","last_clean":"2024-09-19T08:15:01.308765+0000","last_became_active":"2024-09-19T08:13:00.850324+0000","last_became_peered":"2024-09-19T08:13:00.850324+0000","last_unstale":"2024-09-19T08:15:01.308765+0000","last_undegraded":"2024-09-19T08:15:01.308765+0000","last_fullsized":"2024-09-19T08:15:01.308765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:59:49.004605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308718+0000","last_change":"2024-09-19T08:14:07.741837+0000","last_active":"2024-09-19T08:15:01.308718+0000","last_peered":"2024-09-19T08:15:01.308718+0000","last_clean":"2024-09-19T08:15:01.308718+0000","last_became_active":"2024-09-19T08:14:07.741398+0000","last_became_peered":"2024-09-19T08:14:07.741398+0000","last_unstale":"2024-09-19T08:15:01.308718+0000","last_undegraded":"2024-09-19T08:15:01.308718+0000","last_fullsized":"2024-09-19T08:15:01.308718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:56.716663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310236+0000","last_change":"2024-09-19T08:13:00.855215+0000","last_active":"2024-09-19T08:15:01.310236+0000","last_peered":"2024-09-19T08:15:01.310236+0000","last_clean":"2024-09-19T08:15:01.310236+0000","last_became_active":"2024-09-19T08:13:00.854878+0000","last_became_peered":"2024-09-19T08:13:00.854878+0000","last_unstale":"2024-09-19T08:15:01.310236+0000","last_undegraded":"2024-09-19T08:15:01.310236+0000","last_fullsized":"2024-09-19T08:15:01.310236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:48.216066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223444+0000","last_change":"2024-09-19T08:13:00.856995+0000","last_active":"2024-09-19T08:15:01.223444+0000","last_peered":"2024-09-19T08:15:01.223444+0000","last_clean":"2024-09-19T08:15:01.223444+0000","last_became_active":"2024-09-19T08:13:00.856645+0000","last_became_peered":"2024-09-19T08:13:00.856645+0000","last_unstale":"2024-09-19T08:15:01.223444+0000","last_undegraded":"2024-09-19T08:15:01.223444+0000","last_fullsized":"2024-09-19T08:15:01.223444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:20.699293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170753+0000","last_change":"2024-09-19T08:13:00.853362+0000","last_active":"2024-09-19T08:15:01.170753+0000","last_peered":"2024-09-19T08:15:01.170753+0000","last_clean":"2024-09-19T08:15:01.170753+0000","last_became_active":"2024-09-19T08:13:00.853046+0000","last_became_peered":"2024-09-19T08:13:00.853046+0000","last_unstale":"2024-09-19T08:15:01.170753+0000","last_undegraded":"2024-09-19T08:15:01.170753+0000","last_fullsized":"2024-09-19T08:15:01.170753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172297+0000","last_change":"2024-09-19T08:13:00.845175+0000","last_active":"2024-09-19T08:15:01.172297+0000","last_peered":"2024-09-19T08:15:01.172297+0000","last_clean":"2024-09-19T08:15:01.172297+0000","last_became_active":"2024-09-19T08:13:00.844857+0000","last_became_peered":"2024-09-19T08:13:00.844857+0000","last_unstale":"2024-09-19T08:15:01.172297+0000","last_undegraded":"2024-09-19T08:15:01.172297+0000","last_fullsized":"2024-09-19T08:15:01.172297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:40.818956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171025+0000","last_change":"2024-09-19T08:13:00.847769+0000","last_active":"2024-09-19T08:15:01.171025+0000","last_peered":"2024-09-19T08:15:01.171025+0000","last_clean":"2024-09-19T08:15:01.171025+0000","last_became_active":"2024-09-19T08:13:00.847454+0000","last_became_peered":"2024-09-19T08:13:00.847454+0000","last_unstale":"2024-09-19T08:15:01.171025+0000","last_undegraded":"2024-09-19T08:15:01.171025+0000","last_fullsized":"2024-09-19T08:15:01.171025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.583051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308881+0000","last_change":"2024-09-19T08:13:00.859882+0000","last_active":"2024-09-19T08:15:01.308881+0000","last_peered":"2024-09-19T08:15:01.308881+0000","last_clean":"2024-09-19T08:15:01.308881+0000","last_became_active":"2024-09-19T08:13:00.859491+0000","last_became_peered":"2024-09-19T08:13:00.859491+0000","last_unstale":"2024-09-19T08:15:01.308881+0000","last_undegraded":"2024-09-19T08:15:01.308881+0000","last_fullsized":"2024-09-19T08:15:01.308881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:13.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308686+0000","last_change":"2024-09-19T08:13:00.845584+0000","last_active":"2024-09-19T08:15:01.308686+0000","last_peered":"2024-09-19T08:15:01.308686+0000","last_clean":"2024-09-19T08:15:01.308686+0000","last_became_active":"2024-09-19T08:13:00.845166+0000","last_became_peered":"2024-09-19T08:13:00.845166+0000","last_unstale":"2024-09-19T08:15:01.308686+0000","last_undegraded":"2024-09-19T08:15:01.308686+0000","last_fullsized":"2024-09-19T08:15:01.308686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:02.705793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310539+0000","last_change":"2024-09-19T08:13:00.838069+0000","last_active":"2024-09-19T08:15:01.310539+0000","last_peered":"2024-09-19T08:15:01.310539+0000","last_clean":"2024-09-19T08:15:01.310539+0000","last_became_active":"2024-09-19T08:13:00.837736+0000","last_became_peered":"2024-09-19T08:13:00.837736+0000","last_unstale":"2024-09-19T08:15:01.310539+0000","last_undegraded":"2024-09-19T08:15:01.310539+0000","last_fullsized":"2024-09-19T08:15:01.310539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:01.388290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308795+0000","last_change":"2024-09-19T08:13:00.841507+0000","last_active":"2024-09-19T08:15:01.308795+0000","last_peered":"2024-09-19T08:15:01.308795+0000","last_clean":"2024-09-19T08:15:01.308795+0000","last_became_active":"2024-09-19T08:13:00.841161+0000","last_became_peered":"2024-09-19T08:13:00.841161+0000","last_unstale":"2024-09-19T08:15:01.308795+0000","last_undegraded":"2024-09-19T08:15:01.308795+0000","last_fullsized":"2024-09-19T08:15:01.308795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:41.862290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308530+0000","last_change":"2024-09-19T08:13:00.847657+0000","last_active":"2024-09-19T08:15:01.308530+0000","last_peered":"2024-09-19T08:15:01.308530+0000","last_clean":"2024-09-19T08:15:01.308530+0000","last_became_active":"2024-09-19T08:13:00.846849+0000","last_became_peered":"2024-09-19T08:13:00.846849+0000","last_unstale":"2024-09-19T08:15:01.308530+0000","last_undegraded":"2024-09-19T08:15:01.308530+0000","last_fullsized":"2024-09-19T08:15:01.308530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:35.219913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309056+0000","last_change":"2024-09-19T08:13:00.845971+0000","last_active":"2024-09-19T08:15:01.309056+0000","last_peered":"2024-09-19T08:15:01.309056+0000","last_clean":"2024-09-19T08:15:01.309056+0000","last_became_active":"2024-09-19T08:13:00.845626+0000","last_became_peered":"2024-09-19T08:13:00.845626+0000","last_unstale":"2024-09-19T08:15:01.309056+0000","last_undegraded":"2024-09-19T08:15:01.309056+0000","last_fullsized":"2024-09-19T08:15:01.309056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:36.945627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308432+0000","last_change":"2024-09-19T08:14:07.738450+0000","last_active":"2024-09-19T08:15:01.308432+0000","last_peered":"2024-09-19T08:15:01.308432+0000","last_clean":"2024-09-19T08:15:01.308432+0000","last_became_active":"2024-09-19T08:14:07.738028+0000","last_became_peered":"2024-09-19T08:14:07.738028+0000","last_unstale":"2024-09-19T08:15:01.308432+0000","last_undegraded":"2024-09-19T08:15:01.308432+0000","last_fullsized":"2024-09-19T08:15:01.308432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:08.659158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262954+0000","last_change":"2024-09-19T08:13:00.848325+0000","last_active":"2024-09-19T08:15:01.262954+0000","last_peered":"2024-09-19T08:15:01.262954+0000","last_clean":"2024-09-19T08:15:01.262954+0000","last_became_active":"2024-09-19T08:13:00.848015+0000","last_became_peered":"2024-09-19T08:13:00.848015+0000","last_unstale":"2024-09-19T08:15:01.262954+0000","last_undegraded":"2024-09-19T08:15:01.262954+0000","last_fullsized":"2024-09-19T08:15:01.262954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:10.948451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'9","reported_seq":160,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.051779+0000","last_change":"2024-09-19T08:12:58.681134+0000","last_active":"2024-09-19T08:15:11.051779+0000","last_peered":"2024-09-19T08:15:11.051779+0000","last_clean":"2024-09-19T08:15:11.051779+0000","last_became_active":"2024-09-19T08:12:58.680764+0000","last_became_peered":"2024-09-19T08:12:58.680764+0000","last_unstale":"2024-09-19T08:15:11.051779+0000","last_undegraded":"2024-09-19T08:15:11.051779+0000","last_fullsized":"2024-09-19T08:15:11.051779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:24.939263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172533+0000","last_change":"2024-09-19T08:13:00.844319+0000","last_active":"2024-09-19T08:15:01.172533+0000","last_peered":"2024-09-19T08:15:01.172533+0000","last_clean":"2024-09-19T08:15:01.172533+0000","last_became_active":"2024-09-19T08:13:00.843994+0000","last_became_peered":"2024-09-19T08:13:00.843994+0000","last_unstale":"2024-09-19T08:15:01.172533+0000","last_undegraded":"2024-09-19T08:15:01.172533+0000","last_fullsized":"2024-09-19T08:15:01.172533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:46.937466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271532+0000","last_change":"2024-09-19T08:12:58.676541+0000","last_active":"2024-09-19T08:15:01.271532+0000","last_peered":"2024-09-19T08:15:01.271532+0000","last_clean":"2024-09-19T08:15:01.271532+0000","last_became_active":"2024-09-19T08:12:58.675988+0000","last_became_peered":"2024-09-19T08:12:58.675988+0000","last_unstale":"2024-09-19T08:15:01.271532+0000","last_undegraded":"2024-09-19T08:15:01.271532+0000","last_fullsized":"2024-09-19T08:15:01.271532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:01.434986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309788+0000","last_change":"2024-09-19T08:12:58.677510+0000","last_active":"2024-09-19T08:15:01.309788+0000","last_peered":"2024-09-19T08:15:01.309788+0000","last_clean":"2024-09-19T08:15:01.309788+0000","last_became_active":"2024-09-19T08:12:58.677200+0000","last_became_peered":"2024-09-19T08:12:58.677200+0000","last_unstale":"2024-09-19T08:15:01.309788+0000","last_undegraded":"2024-09-19T08:15:01.309788+0000","last_fullsized":"2024-09-19T08:15:01.309788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:36.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170825+0000","last_change":"2024-09-19T08:13:00.853430+0000","last_active":"2024-09-19T08:15:01.170825+0000","last_peered":"2024-09-19T08:15:01.170825+0000","last_clean":"2024-09-19T08:15:01.170825+0000","last_became_active":"2024-09-19T08:13:00.853096+0000","last_became_peered":"2024-09-19T08:13:00.853096+0000","last_unstale":"2024-09-19T08:15:01.170825+0000","last_undegraded":"2024-09-19T08:15:01.170825+0000","last_fullsized":"2024-09-19T08:15:01.170825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:07.147021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172384+0000","last_change":"2024-09-19T08:12:58.680081+0000","last_active":"2024-09-19T08:15:01.172384+0000","last_peered":"2024-09-19T08:15:01.172384+0000","last_clean":"2024-09-19T08:15:01.172384+0000","last_became_active":"2024-09-19T08:12:58.679704+0000","last_became_peered":"2024-09-19T08:12:58.679704+0000","last_unstale":"2024-09-19T08:15:01.172384+0000","last_undegraded":"2024-09-19T08:15:01.172384+0000","last_fullsized":"2024-09-19T08:15:01.172384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:32.942586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.260317+0000","last_change":"2024-09-19T08:13:00.849394+0000","last_active":"2024-09-19T08:15:01.260317+0000","last_peered":"2024-09-19T08:15:01.260317+0000","last_clean":"2024-09-19T08:15:01.260317+0000","last_became_active":"2024-09-19T08:13:00.849051+0000","last_became_peered":"2024-09-19T08:13:00.849051+0000","last_unstale":"2024-09-19T08:15:01.260317+0000","last_undegraded":"2024-09-19T08:15:01.260317+0000","last_fullsized":"2024-09-19T08:15:01.260317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:23.012928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'5","reported_seq":150,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.051165+0000","last_change":"2024-09-19T08:12:58.676121+0000","last_active":"2024-09-19T08:15:11.051165+0000","last_peered":"2024-09-19T08:15:11.051165+0000","last_clean":"2024-09-19T08:15:11.051165+0000","last_became_active":"2024-09-19T08:12:58.675693+0000","last_became_peered":"2024-09-19T08:12:58.675693+0000","last_unstale":"2024-09-19T08:15:11.051165+0000","last_undegraded":"2024-09-19T08:15:11.051165+0000","last_fullsized":"2024-09-19T08:15:11.051165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:34.882190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223583+0000","last_change":"2024-09-19T08:13:00.858735+0000","last_active":"2024-09-19T08:15:01.223583+0000","last_peered":"2024-09-19T08:15:01.223583+0000","last_clean":"2024-09-19T08:15:01.223583+0000","last_became_active":"2024-09-19T08:13:00.858218+0000","last_became_peered":"2024-09-19T08:13:00.858218+0000","last_unstale":"2024-09-19T08:15:01.223583+0000","last_undegraded":"2024-09-19T08:15:01.223583+0000","last_fullsized":"2024-09-19T08:15:01.223583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:33.356810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308176+0000","last_change":"2024-09-19T08:14:07.743785+0000","last_active":"2024-09-19T08:15:01.308176+0000","last_peered":"2024-09-19T08:15:01.308176+0000","last_clean":"2024-09-19T08:15:01.308176+0000","last_became_active":"2024-09-19T08:14:07.739531+0000","last_became_peered":"2024-09-19T08:14:07.739531+0000","last_unstale":"2024-09-19T08:15:01.308176+0000","last_undegraded":"2024-09-19T08:15:01.308176+0000","last_fullsized":"2024-09-19T08:15:01.308176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:43.367704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308976+0000","last_change":"2024-09-19T08:13:00.843852+0000","last_active":"2024-09-19T08:15:01.308976+0000","last_peered":"2024-09-19T08:15:01.308976+0000","last_clean":"2024-09-19T08:15:01.308976+0000","last_became_active":"2024-09-19T08:13:00.843497+0000","last_became_peered":"2024-09-19T08:13:00.843497+0000","last_unstale":"2024-09-19T08:15:01.308976+0000","last_undegraded":"2024-09-19T08:15:01.308976+0000","last_fullsized":"2024-09-19T08:15:01.308976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:26.380024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172698+0000","last_change":"2024-09-19T08:12:58.717225+0000","last_active":"2024-09-19T08:15:01.172698+0000","last_peered":"2024-09-19T08:15:01.172698+0000","last_clean":"2024-09-19T08:15:01.172698+0000","last_became_active":"2024-09-19T08:12:58.716869+0000","last_became_peered":"2024-09-19T08:12:58.716869+0000","last_unstale":"2024-09-19T08:15:01.172698+0000","last_undegraded":"2024-09-19T08:15:01.172698+0000","last_fullsized":"2024-09-19T08:15:01.172698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:43.201907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223446+0000","last_change":"2024-09-19T08:13:00.845613+0000","last_active":"2024-09-19T08:15:01.223446+0000","last_peered":"2024-09-19T08:15:01.223446+0000","last_clean":"2024-09-19T08:15:01.223446+0000","last_became_active":"2024-09-19T08:13:00.845260+0000","last_became_peered":"2024-09-19T08:13:00.845260+0000","last_unstale":"2024-09-19T08:15:01.223446+0000","last_undegraded":"2024-09-19T08:15:01.223446+0000","last_fullsized":"2024-09-19T08:15:01.223446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:03:47.033972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'39","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.834802+0000","last_change":"2024-09-19T08:14:07.749616+0000","last_active":"2024-09-19T08:15:11.834802+0000","last_peered":"2024-09-19T08:15:11.834802+0000","last_clean":"2024-09-19T08:15:11.834802+0000","last_became_active":"2024-09-19T08:14:07.740298+0000","last_became_peered":"2024-09-19T08:14:07.740298+0000","last_unstale":"2024-09-19T08:15:11.834802+0000","last_undegraded":"2024-09-19T08:15:11.834802+0000","last_fullsized":"2024-09-19T08:15:11.834802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:28.796855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70880,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":249,"num_read_kb":194,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61172,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263216+0000","last_change":"2024-09-19T08:13:00.855603+0000","last_active":"2024-09-19T08:15:01.263216+0000","last_peered":"2024-09-19T08:15:01.263216+0000","last_clean":"2024-09-19T08:15:01.263216+0000","last_became_active":"2024-09-19T08:13:00.855241+0000","last_became_peered":"2024-09-19T08:13:00.855241+0000","last_unstale":"2024-09-19T08:15:01.263216+0000","last_undegraded":"2024-09-19T08:15:01.263216+0000","last_fullsized":"2024-09-19T08:15:01.263216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:28.811847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308653+0000","last_change":"2024-09-19T08:12:58.675897+0000","last_active":"2024-09-19T08:15:01.308653+0000","last_peered":"2024-09-19T08:15:01.308653+0000","last_clean":"2024-09-19T08:15:01.308653+0000","last_became_active":"2024-09-19T08:12:58.675525+0000","last_became_peered":"2024-09-19T08:12:58.675525+0000","last_unstale":"2024-09-19T08:15:01.308653+0000","last_undegraded":"2024-09-19T08:15:01.308653+0000","last_fullsized":"2024-09-19T08:15:01.308653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:54.518064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309636+0000","last_change":"2024-09-19T08:13:00.856420+0000","last_active":"2024-09-19T08:15:01.309636+0000","last_peered":"2024-09-19T08:15:01.309636+0000","last_clean":"2024-09-19T08:15:01.309636+0000","last_became_active":"2024-09-19T08:13:00.856100+0000","last_became_peered":"2024-09-19T08:13:00.856100+0000","last_unstale":"2024-09-19T08:15:01.309636+0000","last_undegraded":"2024-09-19T08:15:01.309636+0000","last_fullsized":"2024-09-19T08:15:01.309636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:58.684930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172292+0000","last_change":"2024-09-19T08:12:58.680778+0000","last_active":"2024-09-19T08:15:01.172292+0000","last_peered":"2024-09-19T08:15:01.172292+0000","last_clean":"2024-09-19T08:15:01.172292+0000","last_became_active":"2024-09-19T08:12:58.680364+0000","last_became_peered":"2024-09-19T08:12:58.680364+0000","last_unstale":"2024-09-19T08:15:01.172292+0000","last_undegraded":"2024-09-19T08:15:01.172292+0000","last_fullsized":"2024-09-19T08:15:01.172292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:10.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308288+0000","last_change":"2024-09-19T08:13:00.853415+0000","last_active":"2024-09-19T08:15:01.308288+0000","last_peered":"2024-09-19T08:15:01.308288+0000","last_clean":"2024-09-19T08:15:01.308288+0000","last_became_active":"2024-09-19T08:13:00.853101+0000","last_became_peered":"2024-09-19T08:13:00.853101+0000","last_unstale":"2024-09-19T08:15:01.308288+0000","last_undegraded":"2024-09-19T08:15:01.308288+0000","last_fullsized":"2024-09-19T08:15:01.308288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:42.375394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":159,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.050571+0000","last_change":"2024-09-19T08:12:58.680312+0000","last_active":"2024-09-19T08:15:11.050571+0000","last_peered":"2024-09-19T08:15:11.050571+0000","last_clean":"2024-09-19T08:15:11.050571+0000","last_became_active":"2024-09-19T08:12:58.679870+0000","last_became_peered":"2024-09-19T08:12:58.679870+0000","last_unstale":"2024-09-19T08:15:11.050571+0000","last_undegraded":"2024-09-19T08:15:11.050571+0000","last_fullsized":"2024-09-19T08:15:11.050571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:42.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308034+0000","last_change":"2024-09-19T08:13:00.853920+0000","last_active":"2024-09-19T08:15:01.308034+0000","last_peered":"2024-09-19T08:15:01.308034+0000","last_clean":"2024-09-19T08:15:01.308034+0000","last_became_active":"2024-09-19T08:13:00.853594+0000","last_became_peered":"2024-09-19T08:13:00.853594+0000","last_unstale":"2024-09-19T08:15:01.308034+0000","last_undegraded":"2024-09-19T08:15:01.308034+0000","last_fullsized":"2024-09-19T08:15:01.308034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:31.746133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308308+0000","last_change":"2024-09-19T08:14:07.751888+0000","last_active":"2024-09-19T08:15:01.308308+0000","last_peered":"2024-09-19T08:15:01.308308+0000","last_clean":"2024-09-19T08:15:01.308308+0000","last_became_active":"2024-09-19T08:14:07.737201+0000","last_became_peered":"2024-09-19T08:14:07.737201+0000","last_unstale":"2024-09-19T08:15:01.308308+0000","last_undegraded":"2024-09-19T08:15:01.308308+0000","last_fullsized":"2024-09-19T08:15:01.308308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:44.095987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172245+0000","last_change":"2024-09-19T08:13:00.847061+0000","last_active":"2024-09-19T08:15:01.172245+0000","last_peered":"2024-09-19T08:15:01.172245+0000","last_clean":"2024-09-19T08:15:01.172245+0000","last_became_active":"2024-09-19T08:13:00.846742+0000","last_became_peered":"2024-09-19T08:13:00.846742+0000","last_unstale":"2024-09-19T08:15:01.172245+0000","last_undegraded":"2024-09-19T08:15:01.172245+0000","last_fullsized":"2024-09-19T08:15:01.172245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:21.785585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309033+0000","last_change":"2024-09-19T08:14:07.755277+0000","last_active":"2024-09-19T08:15:01.309033+0000","last_peered":"2024-09-19T08:15:01.309033+0000","last_clean":"2024-09-19T08:15:01.309033+0000","last_became_active":"2024-09-19T08:14:07.739773+0000","last_became_peered":"2024-09-19T08:14:07.739773+0000","last_unstale":"2024-09-19T08:15:01.309033+0000","last_undegraded":"2024-09-19T08:15:01.309033+0000","last_fullsized":"2024-09-19T08:15:01.309033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:56.848251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308865+0000","last_change":"2024-09-19T08:13:00.844940+0000","last_active":"2024-09-19T08:15:01.308865+0000","last_peered":"2024-09-19T08:15:01.308865+0000","last_clean":"2024-09-19T08:15:01.308865+0000","last_became_active":"2024-09-19T08:13:00.844454+0000","last_became_peered":"2024-09-19T08:13:00.844454+0000","last_unstale":"2024-09-19T08:15:01.308865+0000","last_undegraded":"2024-09-19T08:15:01.308865+0000","last_fullsized":"2024-09-19T08:15:01.308865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:15.577624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309967+0000","last_change":"2024-09-19T08:12:58.673301+0000","last_active":"2024-09-19T08:15:01.309967+0000","last_peered":"2024-09-19T08:15:01.309967+0000","last_clean":"2024-09-19T08:15:01.309967+0000","last_became_active":"2024-09-19T08:12:58.672981+0000","last_became_peered":"2024-09-19T08:12:58.672981+0000","last_unstale":"2024-09-19T08:15:01.309967+0000","last_undegraded":"2024-09-19T08:15:01.309967+0000","last_fullsized":"2024-09-19T08:15:01.309967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:03.963712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308223+0000","last_change":"2024-09-19T08:13:00.856620+0000","last_active":"2024-09-19T08:15:01.308223+0000","last_peered":"2024-09-19T08:15:01.308223+0000","last_clean":"2024-09-19T08:15:01.308223+0000","last_became_active":"2024-09-19T08:13:00.856310+0000","last_became_peered":"2024-09-19T08:13:00.856310+0000","last_unstale":"2024-09-19T08:15:01.308223+0000","last_undegraded":"2024-09-19T08:15:01.308223+0000","last_fullsized":"2024-09-19T08:15:01.308223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:19:16.918768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'65","reported_seq":118,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.911955+0000","last_change":"2024-09-19T08:12:58.672282+0000","last_active":"2024-09-19T08:15:11.911955+0000","last_peered":"2024-09-19T08:15:11.911955+0000","last_clean":"2024-09-19T08:15:11.911955+0000","last_became_active":"2024-09-19T08:12:58.671891+0000","last_became_peered":"2024-09-19T08:12:58.671891+0000","last_unstale":"2024-09-19T08:15:11.911955+0000","last_undegraded":"2024-09-19T08:15:11.911955+0000","last_fullsized":"2024-09-19T08:15:11.911955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:41.731571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":31,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308179+0000","last_change":"2024-09-19T08:13:00.848107+0000","last_active":"2024-09-19T08:15:01.308179+0000","last_peered":"2024-09-19T08:15:01.308179+0000","last_clean":"2024-09-19T08:15:01.308179+0000","last_became_active":"2024-09-19T08:13:00.847632+0000","last_became_peered":"2024-09-19T08:13:00.847632+0000","last_unstale":"2024-09-19T08:15:01.308179+0000","last_undegraded":"2024-09-19T08:15:01.308179+0000","last_fullsized":"2024-09-19T08:15:01.308179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:34.491122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308878+0000","last_change":"2024-09-19T08:12:59.179200+0000","last_active":"2024-09-19T08:15:01.308878+0000","last_peered":"2024-09-19T08:15:01.308878+0000","last_clean":"2024-09-19T08:15:01.308878+0000","last_became_active":"2024-09-19T08:12:59.178762+0000","last_became_peered":"2024-09-19T08:12:59.178762+0000","last_unstale":"2024-09-19T08:15:01.308878+0000","last_undegraded":"2024-09-19T08:15:01.308878+0000","last_fullsized":"2024-09-19T08:15:01.308878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:08.424106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308358+0000","last_change":"2024-09-19T08:13:00.855870+0000","last_active":"2024-09-19T08:15:01.308358+0000","last_peered":"2024-09-19T08:15:01.308358+0000","last_clean":"2024-09-19T08:15:01.308358+0000","last_became_active":"2024-09-19T08:13:00.855538+0000","last_became_peered":"2024-09-19T08:13:00.855538+0000","last_unstale":"2024-09-19T08:15:01.308358+0000","last_undegraded":"2024-09-19T08:15:01.308358+0000","last_fullsized":"2024-09-19T08:15:01.308358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"86'148","reported_seq":395,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:03.309732+0000","last_change":"2024-09-19T08:11:48.661333+0000","last_active":"2024-09-19T08:15:03.309732+0000","last_peered":"2024-09-19T08:15:03.309732+0000","last_clean":"2024-09-19T08:15:03.309732+0000","last_became_active":"2024-09-19T08:11:48.660873+0000","last_became_peered":"2024-09-19T08:11:48.660873+0000","last_unstale":"2024-09-19T08:15:03.309732+0000","last_undegraded":"2024-09-19T08:15:03.309732+0000","last_fullsized":"2024-09-19T08:15:03.309732+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:08:25.372155+0000","last_clean_scrub_stamp":"2024-09-19T08:08:25.372155+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:19.401292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172985+0000","last_change":"2024-09-19T08:12:58.675237+0000","last_active":"2024-09-19T08:15:01.172985+0000","last_peered":"2024-09-19T08:15:01.172985+0000","last_clean":"2024-09-19T08:15:01.172985+0000","last_became_active":"2024-09-19T08:12:58.674910+0000","last_became_peered":"2024-09-19T08:12:58.674910+0000","last_unstale":"2024-09-19T08:15:01.172985+0000","last_undegraded":"2024-09-19T08:15:01.172985+0000","last_fullsized":"2024-09-19T08:15:01.172985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:47.038328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171081+0000","last_change":"2024-09-19T08:13:00.846754+0000","last_active":"2024-09-19T08:15:01.171081+0000","last_peered":"2024-09-19T08:15:01.171081+0000","last_clean":"2024-09-19T08:15:01.171081+0000","last_became_active":"2024-09-19T08:13:00.846452+0000","last_became_peered":"2024-09-19T08:13:00.846452+0000","last_unstale":"2024-09-19T08:15:01.171081+0000","last_undegraded":"2024-09-19T08:15:01.171081+0000","last_fullsized":"2024-09-19T08:15:01.171081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:22.929689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308239+0000","last_change":"2024-09-19T08:12:58.678958+0000","last_active":"2024-09-19T08:15:01.308239+0000","last_peered":"2024-09-19T08:15:01.308239+0000","last_clean":"2024-09-19T08:15:01.308239+0000","last_became_active":"2024-09-19T08:12:58.678592+0000","last_became_peered":"2024-09-19T08:12:58.678592+0000","last_unstale":"2024-09-19T08:15:01.308239+0000","last_undegraded":"2024-09-19T08:15:01.308239+0000","last_fullsized":"2024-09-19T08:15:01.308239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:58.374606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.309700+0000","last_change":"2024-09-19T08:13:00.840973+0000","last_active":"2024-09-19T08:15:01.309700+0000","last_peered":"2024-09-19T08:15:01.309700+0000","last_clean":"2024-09-19T08:15:01.309700+0000","last_became_active":"2024-09-19T08:13:00.840619+0000","last_became_peered":"2024-09-19T08:13:00.840619+0000","last_unstale":"2024-09-19T08:15:01.309700+0000","last_undegraded":"2024-09-19T08:15:01.309700+0000","last_fullsized":"2024-09-19T08:15:01.309700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":152,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.050056+0000","last_change":"2024-09-19T08:12:58.715972+0000","last_active":"2024-09-19T08:15:11.050056+0000","last_peered":"2024-09-19T08:15:11.050056+0000","last_clean":"2024-09-19T08:15:11.050056+0000","last_became_active":"2024-09-19T08:12:58.715630+0000","last_became_peered":"2024-09-19T08:12:58.715630+0000","last_unstale":"2024-09-19T08:15:11.050056+0000","last_undegraded":"2024-09-19T08:15:11.050056+0000","last_fullsized":"2024-09-19T08:15:11.050056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:35.637121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172244+0000","last_change":"2024-09-19T08:13:00.847023+0000","last_active":"2024-09-19T08:15:01.172244+0000","last_peered":"2024-09-19T08:15:01.172244+0000","last_clean":"2024-09-19T08:15:01.172244+0000","last_became_active":"2024-09-19T08:13:00.846387+0000","last_became_peered":"2024-09-19T08:13:00.846387+0000","last_unstale":"2024-09-19T08:15:01.172244+0000","last_undegraded":"2024-09-19T08:15:01.172244+0000","last_fullsized":"2024-09-19T08:15:01.172244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:12.187929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308283+0000","last_change":"2024-09-19T08:12:58.681062+0000","last_active":"2024-09-19T08:15:01.308283+0000","last_peered":"2024-09-19T08:15:01.308283+0000","last_clean":"2024-09-19T08:15:01.308283+0000","last_became_active":"2024-09-19T08:12:58.680524+0000","last_became_peered":"2024-09-19T08:12:58.680524+0000","last_unstale":"2024-09-19T08:15:01.308283+0000","last_undegraded":"2024-09-19T08:15:01.308283+0000","last_fullsized":"2024-09-19T08:15:01.308283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.668912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271807+0000","last_change":"2024-09-19T08:13:00.851523+0000","last_active":"2024-09-19T08:15:01.271807+0000","last_peered":"2024-09-19T08:15:01.271807+0000","last_clean":"2024-09-19T08:15:01.271807+0000","last_became_active":"2024-09-19T08:13:00.851068+0000","last_became_peered":"2024-09-19T08:13:00.851068+0000","last_unstale":"2024-09-19T08:15:01.271807+0000","last_undegraded":"2024-09-19T08:15:01.271807+0000","last_fullsized":"2024-09-19T08:15:01.271807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:44.905469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.260925+0000","last_change":"2024-09-19T08:12:58.718234+0000","last_active":"2024-09-19T08:15:01.260925+0000","last_peered":"2024-09-19T08:15:01.260925+0000","last_clean":"2024-09-19T08:15:01.260925+0000","last_became_active":"2024-09-19T08:12:58.717840+0000","last_became_peered":"2024-09-19T08:12:58.717840+0000","last_unstale":"2024-09-19T08:15:01.260925+0000","last_undegraded":"2024-09-19T08:15:01.260925+0000","last_fullsized":"2024-09-19T08:15:01.260925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:05.283770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.271686+0000","last_change":"2024-09-19T08:13:00.848305+0000","last_active":"2024-09-19T08:15:01.271686+0000","last_peered":"2024-09-19T08:15:01.271686+0000","last_clean":"2024-09-19T08:15:01.271686+0000","last_became_active":"2024-09-19T08:13:00.847981+0000","last_became_peered":"2024-09-19T08:13:00.847981+0000","last_unstale":"2024-09-19T08:15:01.271686+0000","last_undegraded":"2024-09-19T08:15:01.271686+0000","last_fullsized":"2024-09-19T08:15:01.271686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:04.681875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263351+0000","last_change":"2024-09-19T08:12:58.715943+0000","last_active":"2024-09-19T08:15:01.263351+0000","last_peered":"2024-09-19T08:15:01.263351+0000","last_clean":"2024-09-19T08:15:01.263351+0000","last_became_active":"2024-09-19T08:12:58.715600+0000","last_became_peered":"2024-09-19T08:12:58.715600+0000","last_unstale":"2024-09-19T08:15:01.263351+0000","last_undegraded":"2024-09-19T08:15:01.263351+0000","last_fullsized":"2024-09-19T08:15:01.263351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:19.142271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.310659+0000","last_change":"2024-09-19T08:13:00.840944+0000","last_active":"2024-09-19T08:15:01.310659+0000","last_peered":"2024-09-19T08:15:01.310659+0000","last_clean":"2024-09-19T08:15:01.310659+0000","last_became_active":"2024-09-19T08:13:00.840608+0000","last_became_peered":"2024-09-19T08:13:00.840608+0000","last_unstale":"2024-09-19T08:15:01.310659+0000","last_undegraded":"2024-09-19T08:15:01.310659+0000","last_fullsized":"2024-09-19T08:15:01.310659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:25.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'97","reported_seq":208,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:11.918689+0000","last_change":"2024-09-19T08:12:58.672929+0000","last_active":"2024-09-19T08:15:11.918689+0000","last_peered":"2024-09-19T08:15:11.918689+0000","last_clean":"2024-09-19T08:15:11.918689+0000","last_became_active":"2024-09-19T08:12:58.672487+0000","last_became_peered":"2024-09-19T08:12:58.672487+0000","last_unstale":"2024-09-19T08:15:11.918689+0000","last_undegraded":"2024-09-19T08:15:11.918689+0000","last_fullsized":"2024-09-19T08:15:11.918689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:33.042735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":177,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.261959+0000","last_change":"2024-09-19T08:14:07.738260+0000","last_active":"2024-09-19T08:15:01.261959+0000","last_peered":"2024-09-19T08:15:01.261959+0000","last_clean":"2024-09-19T08:15:01.261959+0000","last_became_active":"2024-09-19T08:14:07.737915+0000","last_became_peered":"2024-09-19T08:14:07.737915+0000","last_unstale":"2024-09-19T08:15:01.261959+0000","last_undegraded":"2024-09-19T08:15:01.261959+0000","last_fullsized":"2024-09-19T08:15:01.261959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:02.500657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.173307+0000","last_change":"2024-09-19T08:12:59.179510+0000","last_active":"2024-09-19T08:15:01.173307+0000","last_peered":"2024-09-19T08:15:01.173307+0000","last_clean":"2024-09-19T08:15:01.173307+0000","last_became_active":"2024-09-19T08:12:59.179157+0000","last_became_peered":"2024-09-19T08:12:59.179157+0000","last_unstale":"2024-09-19T08:15:01.173307+0000","last_undegraded":"2024-09-19T08:15:01.173307+0000","last_fullsized":"2024-09-19T08:15:01.173307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:22.981289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308670+0000","last_change":"2024-09-19T08:13:00.841853+0000","last_active":"2024-09-19T08:15:01.308670+0000","last_peered":"2024-09-19T08:15:01.308670+0000","last_clean":"2024-09-19T08:15:01.308670+0000","last_became_active":"2024-09-19T08:13:00.841518+0000","last_became_peered":"2024-09-19T08:13:00.841518+0000","last_unstale":"2024-09-19T08:15:01.308670+0000","last_undegraded":"2024-09-19T08:15:01.308670+0000","last_fullsized":"2024-09-19T08:15:01.308670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:05.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223446+0000","last_change":"2024-09-19T08:12:58.716882+0000","last_active":"2024-09-19T08:15:01.223446+0000","last_peered":"2024-09-19T08:15:01.223446+0000","last_clean":"2024-09-19T08:15:01.223446+0000","last_became_active":"2024-09-19T08:12:58.716545+0000","last_became_peered":"2024-09-19T08:12:58.716545+0000","last_unstale":"2024-09-19T08:15:01.223446+0000","last_undegraded":"2024-09-19T08:15:01.223446+0000","last_fullsized":"2024-09-19T08:15:01.223446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:14.469109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.262999+0000","last_change":"2024-09-19T08:13:00.850484+0000","last_active":"2024-09-19T08:15:01.262999+0000","last_peered":"2024-09-19T08:15:01.262999+0000","last_clean":"2024-09-19T08:15:01.262999+0000","last_became_active":"2024-09-19T08:13:00.850049+0000","last_became_peered":"2024-09-19T08:13:00.850049+0000","last_unstale":"2024-09-19T08:15:01.262999+0000","last_undegraded":"2024-09-19T08:15:01.262999+0000","last_fullsized":"2024-09-19T08:15:01.262999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:24.424111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.170992+0000","last_change":"2024-09-19T08:12:58.669928+0000","last_active":"2024-09-19T08:15:01.170992+0000","last_peered":"2024-09-19T08:15:01.170992+0000","last_clean":"2024-09-19T08:15:01.170992+0000","last_became_active":"2024-09-19T08:12:58.669614+0000","last_became_peered":"2024-09-19T08:12:58.669614+0000","last_unstale":"2024-09-19T08:15:01.170992+0000","last_undegraded":"2024-09-19T08:15:01.170992+0000","last_fullsized":"2024-09-19T08:15:01.170992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:50.924779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308498+0000","last_change":"2024-09-19T08:13:00.853520+0000","last_active":"2024-09-19T08:15:01.308498+0000","last_peered":"2024-09-19T08:15:01.308498+0000","last_clean":"2024-09-19T08:15:01.308498+0000","last_became_active":"2024-09-19T08:13:00.853184+0000","last_became_peered":"2024-09-19T08:13:00.853184+0000","last_unstale":"2024-09-19T08:15:01.308498+0000","last_undegraded":"2024-09-19T08:15:01.308498+0000","last_fullsized":"2024-09-19T08:15:01.308498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:45.160439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:02.882016+0000","last_change":"2024-09-19T08:12:58.674359+0000","last_active":"2024-09-19T08:15:02.882016+0000","last_peered":"2024-09-19T08:15:02.882016+0000","last_clean":"2024-09-19T08:15:02.882016+0000","last_became_active":"2024-09-19T08:12:58.673890+0000","last_became_peered":"2024-09-19T08:12:58.673890+0000","last_unstale":"2024-09-19T08:15:02.882016+0000","last_undegraded":"2024-09-19T08:15:02.882016+0000","last_fullsized":"2024-09-19T08:15:02.882016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:54.939959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171020+0000","last_change":"2024-09-19T08:13:00.851071+0000","last_active":"2024-09-19T08:15:01.171020+0000","last_peered":"2024-09-19T08:15:01.171020+0000","last_clean":"2024-09-19T08:15:01.171020+0000","last_became_active":"2024-09-19T08:13:00.850756+0000","last_became_peered":"2024-09-19T08:13:00.850756+0000","last_unstale":"2024-09-19T08:15:01.171020+0000","last_undegraded":"2024-09-19T08:15:01.171020+0000","last_fullsized":"2024-09-19T08:15:01.171020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:52.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263074+0000","last_change":"2024-09-19T08:12:58.681466+0000","last_active":"2024-09-19T08:15:01.263074+0000","last_peered":"2024-09-19T08:15:01.263074+0000","last_clean":"2024-09-19T08:15:01.263074+0000","last_became_active":"2024-09-19T08:12:58.681113+0000","last_became_peered":"2024-09-19T08:12:58.681113+0000","last_unstale":"2024-09-19T08:15:01.263074+0000","last_undegraded":"2024-09-19T08:15:01.263074+0000","last_fullsized":"2024-09-19T08:15:01.263074+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:49.610828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308435+0000","last_change":"2024-09-19T08:13:00.851272+0000","last_active":"2024-09-19T08:15:01.308435+0000","last_peered":"2024-09-19T08:15:01.308435+0000","last_clean":"2024-09-19T08:15:01.308435+0000","last_became_active":"2024-09-19T08:13:00.849098+0000","last_became_peered":"2024-09-19T08:13:00.849098+0000","last_unstale":"2024-09-19T08:15:01.308435+0000","last_undegraded":"2024-09-19T08:15:01.308435+0000","last_fullsized":"2024-09-19T08:15:01.308435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:05.643992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.172513+0000","last_change":"2024-09-19T08:14:07.738913+0000","last_active":"2024-09-19T08:15:01.172513+0000","last_peered":"2024-09-19T08:15:01.172513+0000","last_clean":"2024-09-19T08:15:01.172513+0000","last_became_active":"2024-09-19T08:14:07.738560+0000","last_became_peered":"2024-09-19T08:14:07.738560+0000","last_unstale":"2024-09-19T08:15:01.172513+0000","last_undegraded":"2024-09-19T08:15:01.172513+0000","last_fullsized":"2024-09-19T08:15:01.172513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:35.731290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263192+0000","last_change":"2024-09-19T08:12:58.674610+0000","last_active":"2024-09-19T08:15:01.263192+0000","last_peered":"2024-09-19T08:15:01.263192+0000","last_clean":"2024-09-19T08:15:01.263192+0000","last_became_active":"2024-09-19T08:12:58.674247+0000","last_became_peered":"2024-09-19T08:12:58.674247+0000","last_unstale":"2024-09-19T08:15:01.263192+0000","last_undegraded":"2024-09-19T08:15:01.263192+0000","last_fullsized":"2024-09-19T08:15:01.263192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:16.323574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.223395+0000","last_change":"2024-09-19T08:13:00.858512+0000","last_active":"2024-09-19T08:15:01.223395+0000","last_peered":"2024-09-19T08:15:01.223395+0000","last_clean":"2024-09-19T08:15:01.223395+0000","last_became_active":"2024-09-19T08:13:00.857897+0000","last_became_peered":"2024-09-19T08:13:00.857897+0000","last_unstale":"2024-09-19T08:15:01.223395+0000","last_undegraded":"2024-09-19T08:15:01.223395+0000","last_fullsized":"2024-09-19T08:15:01.223395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.263238+0000","last_change":"2024-09-19T08:12:58.682207+0000","last_active":"2024-09-19T08:15:01.263238+0000","last_peered":"2024-09-19T08:15:01.263238+0000","last_clean":"2024-09-19T08:15:01.263238+0000","last_became_active":"2024-09-19T08:12:58.681842+0000","last_became_peered":"2024-09-19T08:12:58.681842+0000","last_unstale":"2024-09-19T08:15:01.263238+0000","last_undegraded":"2024-09-19T08:15:01.263238+0000","last_fullsized":"2024-09-19T08:15:01.263238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.825858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.308892+0000","last_change":"2024-09-19T08:13:00.846748+0000","last_active":"2024-09-19T08:15:01.308892+0000","last_peered":"2024-09-19T08:15:01.308892+0000","last_clean":"2024-09-19T08:15:01.308892+0000","last_became_active":"2024-09-19T08:13:00.846368+0000","last_became_peered":"2024-09-19T08:13:00.846368+0000","last_unstale":"2024-09-19T08:15:01.308892+0000","last_undegraded":"2024-09-19T08:15:01.308892+0000","last_fullsized":"2024-09-19T08:15:01.308892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:59.526512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171334+0000","last_change":"2024-09-19T08:12:58.716118+0000","last_active":"2024-09-19T08:15:01.171334+0000","last_peered":"2024-09-19T08:15:01.171334+0000","last_clean":"2024-09-19T08:15:01.171334+0000","last_became_active":"2024-09-19T08:12:58.715711+0000","last_became_peered":"2024-09-19T08:12:58.715711+0000","last_unstale":"2024-09-19T08:15:01.171334+0000","last_undegraded":"2024-09-19T08:15:01.171334+0000","last_fullsized":"2024-09-19T08:15:01.171334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:29.439887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:01.171226+0000","last_change":"2024-09-19T08:13:00.848729+0000","last_active":"2024-09-19T08:15:01.171226+0000","last_peered":"2024-09-19T08:15:01.171226+0000","last_clean":"2024-09-19T08:15:01.171226+0000","last_became_active":"2024-09-19T08:13:00.848470+0000","last_became_peered":"2024-09-19T08:13:00.848470+0000","last_unstale":"2024-09-19T08:15:01.171226+0000","last_undegraded":"2024-09-19T08:15:01.171226+0000","last_fullsized":"2024-09-19T08:15:01.171226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:59.824436+0000","last_clean_scrub_stamp":"2024-09-19T08:12:59.824436+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:30.653101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T08:15:07.185959+0000","last_change":"2024-09-19T08:12:58.681654+0000","last_active":"2024-09-19T08:15:07.185959+0000","last_peered":"2024-09-19T08:15:07.185959+0000","last_clean":"2024-09-19T08:15:07.185959+0000","last_became_active":"2024-09-19T08:12:58.681284+0000","last_became_peered":"2024-09-19T08:12:58.681284+0000","last_unstale":"2024-09-19T08:15:07.185959+0000","last_undegraded":"2024-09-19T08:15:07.185959+0000","last_fullsized":"2024-09-19T08:15:07.185959+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:12:57.658970+0000","last_clean_scrub_stamp":"2024-09-19T08:12:57.658970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:09.429930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798886,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1081,"num_read_kb":750,"num_write":423,"num_write_kb":2364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":61306,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":618496,"data_stored":2396658,"data_compressed":113084,"data_compressed_allocated":147456,"data_compressed_original":2158592,"omap_allocated":0,"internal_metadata":0},"log_size":314,"ondisk_log_size":314,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":109176,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481830,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":454888,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710764,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1138688,"data_stored":1446975,"data_compressed":59949,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874289,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":848206,"data_compressed":28006,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103222,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":958464,"data_stored":847992,"data_compressed":27863,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332155,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":1049397,"data_compressed":45109,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495680,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":933888,"data_stored":454379,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757318,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1052672,"data_stored":1375685,"data_compressed":60311,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":527324,"data_compressed":18328,"data_compressed_allocated":24576,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":979506,"data_compressed":34019,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":456842,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":525052,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":455650,"data_compressed":8717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36392,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":73152,"data_compressed":9611,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":331028,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":393820,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":402318,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:15:17.866 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:15:18.129 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:18.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:18 smithi022 ceph-mon[30963]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:18.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:18 smithi022 ceph-mon[30963]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T08:15:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.653+0000 7f5c63e7e640 1 -- 172.21.15.22:0/1664516642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c10c180 msgr2=0x7f5c5c10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.653+0000 7f5c63e7e640 1 --2- 172.21.15.22:0/1664516642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c10c180 0x7f5c5c10e600 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009ca0 tx=0x7f5c5402a950 comp rx=0 tx=0).stop 2024-09-19T08:15:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.653+0000 7f5c63e7e640 1 -- 172.21.15.22:0/1664516642 shutdown_connections 2024-09-19T08:15:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.653+0000 7f5c63e7e640 1 --2- 172.21.15.22:0/1664516642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c10c180 0x7f5c5c10e600 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.653+0000 7f5c63e7e640 1 --2- 172.21.15.22:0/1664516642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c5c1036c0 0x7f5c5c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.653+0000 7f5c63e7e640 1 --2- 172.21.15.22:0/1664516642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c5c102da0 0x7f5c5c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.653+0000 7f5c63e7e640 1 -- 172.21.15.22:0/1664516642 >> 172.21.15.22:0/1664516642 conn(0x7f5c5c0fc7a0 msgr2=0x7f5c5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.653+0000 7f5c63e7e640 1 -- 172.21.15.22:0/1664516642 shutdown_connections 2024-09-19T08:15:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.653+0000 7f5c63e7e640 1 -- 172.21.15.22:0/1664516642 wait complete. 2024-09-19T08:15:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c63e7e640 1 Processor -- start 2024-09-19T08:15:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c63e7e640 1 -- start start 2024-09-19T08:15:18.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c63e7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c5c102da0 0x7f5c5c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:18.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c63e7e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c1036c0 0x7f5c5c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:18.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c63e7e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c5c10c180 0x7f5c5c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:18.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c63e7e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c111f60 con 0x7f5c5c1036c0 2024-09-19T08:15:18.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c63e7e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c111de0 con 0x7f5c5c102da0 2024-09-19T08:15:18.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c63e7e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1120e0 con 0x7f5c5c10c180 2024-09-19T08:15:18.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c61419640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c1036c0 0x7f5c5c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:18.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c61419640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c1036c0 0x7f5c5c1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58632/0 (socket says 172.21.15.22:58632) 2024-09-19T08:15:18.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c61419640 1 -- 172.21.15.22:0/666984396 learned_addr learned my addr 172.21.15.22:0/666984396 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:18.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c61c1a640 1 --2- 172.21.15.22:0/666984396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c5c102da0 0x7f5c5c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:18.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c6241b640 1 --2- 172.21.15.22:0/666984396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c5c10c180 0x7f5c5c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:18.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c61419640 1 -- 172.21.15.22:0/666984396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c5c10c180 msgr2=0x7f5c5c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:18.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c61419640 1 --2- 172.21.15.22:0/666984396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c5c10c180 0x7f5c5c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:18.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.654+0000 7f5c61419640 1 -- 172.21.15.22:0/666984396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c5c102da0 msgr2=0x7f5c5c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:18.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.655+0000 7f5c61419640 1 --2- 172.21.15.22:0/666984396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c5c102da0 0x7f5c5c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:18.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.655+0000 7f5c61419640 1 -- 172.21.15.22:0/666984396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c19d350 con 0x7f5c5c1036c0 2024-09-19T08:15:18.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.655+0000 7f5c61419640 1 --2- 172.21.15.22:0/666984396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c1036c0 0x7f5c5c1988e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00c920 tx=0x7f5c4c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:18.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.655+0000 7f5c52ffd640 1 -- 172.21.15.22:0/666984396 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c007c70 con 0x7f5c5c1036c0 2024-09-19T08:15:18.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.655+0000 7f5c63e7e640 1 -- 172.21.15.22:0/666984396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c19d640 con 0x7f5c5c1036c0 2024-09-19T08:15:18.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.655+0000 7f5c52ffd640 1 -- 172.21.15.22:0/666984396 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c4c007e10 con 0x7f5c5c1036c0 2024-09-19T08:15:18.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.655+0000 7f5c52ffd640 1 -- 172.21.15.22:0/666984396 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c018890 con 0x7f5c5c1036c0 2024-09-19T08:15:18.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.655+0000 7f5c63e7e640 1 -- 172.21.15.22:0/666984396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c1a4f20 con 0x7f5c5c1036c0 2024-09-19T08:15:18.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.656+0000 7f5c63e7e640 1 -- 172.21.15.22:0/666984396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c5c103180 con 0x7f5c5c1036c0 2024-09-19T08:15:18.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.657+0000 7f5c52ffd640 1 -- 172.21.15.22:0/666984396 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f5c4c018a50 con 0x7f5c5c1036c0 2024-09-19T08:15:18.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.658+0000 7f5c52ffd640 1 --2- 172.21.15.22:0/666984396 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5c480a34d0 0x7f5c480a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:18.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.658+0000 7f5c61c1a640 1 --2- 172.21.15.22:0/666984396 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5c480a34d0 0x7f5c480a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:18.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.658+0000 7f5c52ffd640 1 -- 172.21.15.22:0/666984396 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5c4c0d02e0 con 0x7f5c5c1036c0 2024-09-19T08:15:18.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.659+0000 7f5c61c1a640 1 --2- 172.21.15.22:0/666984396 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5c480a34d0 0x7f5c480a5990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800bbe0 tx=0x7f5c5800b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:18.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.662+0000 7f5c52ffd640 1 -- 172.21.15.22:0/666984396 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5c4c014030 con 0x7f5c5c1036c0 2024-09-19T08:15:18.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:18 smithi157 ceph-mon[35143]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:18.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:18 smithi157 ceph-mon[35143]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T08:15:18.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:18 smithi064 ceph-mon[34900]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:15:18.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:18 smithi064 ceph-mon[34900]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T08:15:18.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.876+0000 7f5c63e7e640 1 -- 172.21.15.22:0/666984396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c5c1a5180 con 0x7f5c5c1036c0 2024-09-19T08:15:18.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.877+0000 7f5c52ffd640 1 -- 172.21.15.22:0/666984396 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11906 (secure 0 0 0) 0x7f5c4c09cab0 con 0x7f5c5c1036c0 2024-09-19T08:15:18.878 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:18.878 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"btime":"2024-09-19T08:15:00:109390+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:15:00.059208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:15:18.879 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-09-19T08:15:18.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.879+0000 7f5c63e7e640 1 -- 172.21.15.22:0/666984396 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5c480a34d0 msgr2=0x7f5c480a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:18.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.879+0000 7f5c63e7e640 1 --2- 172.21.15.22:0/666984396 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5c480a34d0 0x7f5c480a5990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800bbe0 tx=0x7f5c5800b390 comp rx=0 tx=0).stop 2024-09-19T08:15:18.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.879+0000 7f5c63e7e640 1 -- 172.21.15.22:0/666984396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c1036c0 msgr2=0x7f5c5c1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:18.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.879+0000 7f5c63e7e640 1 --2- 172.21.15.22:0/666984396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c1036c0 0x7f5c5c1988e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00c920 tx=0x7f5c4c00cde0 comp rx=0 tx=0).stop 2024-09-19T08:15:18.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.880+0000 7f5c63e7e640 1 -- 172.21.15.22:0/666984396 shutdown_connections 2024-09-19T08:15:18.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.880+0000 7f5c63e7e640 1 --2- 172.21.15.22:0/666984396 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5c480a34d0 0x7f5c480a5990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:18.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.880+0000 7f5c63e7e640 1 --2- 172.21.15.22:0/666984396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c5c10c180 0x7f5c5c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:18.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.880+0000 7f5c63e7e640 1 --2- 172.21.15.22:0/666984396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c1036c0 0x7f5c5c1988e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:18.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.880+0000 7f5c63e7e640 1 --2- 172.21.15.22:0/666984396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c5c102da0 0x7f5c5c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:18.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.880+0000 7f5c63e7e640 1 -- 172.21.15.22:0/666984396 >> 172.21.15.22:0/666984396 conn(0x7f5c5c0fc7a0 msgr2=0x7f5c5c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:18.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.880+0000 7f5c63e7e640 1 -- 172.21.15.22:0/666984396 shutdown_connections 2024-09-19T08:15:18.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:18.880+0000 7f5c63e7e640 1 -- 172.21.15.22:0/666984396 wait complete. 2024-09-19T08:15:19.577 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:19 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/666984396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:19.610 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:15:19.738 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:19 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/666984396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:19.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:19 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/666984396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:19.873 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:20.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.355+0000 7fe8718b1640 1 -- 172.21.15.22:0/2952824173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe86c102ef0 msgr2=0x7fe86c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:20.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.355+0000 7fe8718b1640 1 --2- 172.21.15.22:0/2952824173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe86c102ef0 0x7fe86c10da40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe864009ca0 tx=0x7fe86402a950 comp rx=0 tx=0).stop 2024-09-19T08:15:20.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.355+0000 7fe8718b1640 1 -- 172.21.15.22:0/2952824173 shutdown_connections 2024-09-19T08:15:20.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.355+0000 7fe8718b1640 1 --2- 172.21.15.22:0/2952824173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe86c102ef0 0x7fe86c10da40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:20.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.355+0000 7fe8718b1640 1 --2- 172.21.15.22:0/2952824173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe86c102570 0x7fe86c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:20.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.355+0000 7fe8718b1640 1 --2- 172.21.15.22:0/2952824173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe86c100e20 0x7fe86c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:20.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.355+0000 7fe8718b1640 1 -- 172.21.15.22:0/2952824173 >> 172.21.15.22:0/2952824173 conn(0x7fe86c0fc7d0 msgr2=0x7fe86c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:20.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.355+0000 7fe8718b1640 1 -- 172.21.15.22:0/2952824173 shutdown_connections 2024-09-19T08:15:20.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.355+0000 7fe8718b1640 1 -- 172.21.15.22:0/2952824173 wait complete. 2024-09-19T08:15:20.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.355+0000 7fe8718b1640 1 Processor -- start 2024-09-19T08:15:20.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe8718b1640 1 -- start start 2024-09-19T08:15:20.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe8718b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe86c100e20 0x7fe86c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:20.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe8718b1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe86c102570 0x7fe86c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:20.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe8718b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe86c102ef0 0x7fe86c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:20.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe8718b1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c110f80 con 0x7fe86c102ef0 2024-09-19T08:15:20.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe8718b1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c110e00 con 0x7fe86c100e20 2024-09-19T08:15:20.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe8718b1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c111100 con 0x7fe86c102570 2024-09-19T08:15:20.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe86b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe86c102ef0 0x7fe86c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:20.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe86b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe86c102ef0 0x7fe86c19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58650/0 (socket says 172.21.15.22:58650) 2024-09-19T08:15:20.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe86b7fe640 1 -- 172.21.15.22:0/2295433070 learned_addr learned my addr 172.21.15.22:0/2295433070 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:20.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe86a7fc640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe86c102570 0x7fe86c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe86affd640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe86c100e20 0x7fe86c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe86b7fe640 1 -- 172.21.15.22:0/2295433070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe86c102570 msgr2=0x7fe86c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe86b7fe640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe86c102570 0x7fe86c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe86b7fe640 1 -- 172.21.15.22:0/2295433070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe86c100e20 msgr2=0x7fe86c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe86b7fe640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe86c100e20 0x7fe86c198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:20.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe86b7fe640 1 -- 172.21.15.22:0/2295433070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c19d4b0 con 0x7fe86c102ef0 2024-09-19T08:15:20.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.356+0000 7fe86affd640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe86c100e20 0x7fe86c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:20.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.357+0000 7fe86b7fe640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe86c102ef0 0x7fe86c19cdd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe864004af0 tx=0x7fe86402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:20.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.357+0000 7fe8708af640 1 -- 172.21.15.22:0/2295433070 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864034040 con 0x7fe86c102ef0 2024-09-19T08:15:20.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.357+0000 7fe8708af640 1 -- 172.21.15.22:0/2295433070 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe864030920 con 0x7fe86c102ef0 2024-09-19T08:15:20.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.357+0000 7fe8718b1640 1 -- 172.21.15.22:0/2295433070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86c19d740 con 0x7fe86c102ef0 2024-09-19T08:15:20.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.357+0000 7fe8708af640 1 -- 172.21.15.22:0/2295433070 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864004820 con 0x7fe86c102ef0 2024-09-19T08:15:20.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.357+0000 7fe8718b1640 1 -- 172.21.15.22:0/2295433070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe86c1a5080 con 0x7fe86c102ef0 2024-09-19T08:15:20.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.358+0000 7fe8718b1640 1 -- 172.21.15.22:0/2295433070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe86c101200 con 0x7fe86c102ef0 2024-09-19T08:15:20.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.359+0000 7fe8708af640 1 -- 172.21.15.22:0/2295433070 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fe864037020 con 0x7fe86c102ef0 2024-09-19T08:15:20.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.360+0000 7fe8708af640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe8540a3550 0x7fe8540a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:20.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.360+0000 7fe8708af640 1 -- 172.21.15.22:0/2295433070 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe8640f0dd0 con 0x7fe86c102ef0 2024-09-19T08:15:20.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.360+0000 7fe86affd640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe8540a3550 0x7fe8540a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:20.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.361+0000 7fe86affd640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe8540a3550 0x7fe8540a5a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0058c0 tx=0x7fe85c005850 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:20.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.363+0000 7fe8708af640 1 -- 172.21.15.22:0/2295433070 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe8640bd680 con 0x7fe86c102ef0 2024-09-19T08:15:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.595+0000 7fe8718b1640 1 -- 172.21.15.22:0/2295433070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe86c199860 con 0x7fe86c102ef0 2024-09-19T08:15:20.596 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:20 smithi022 ceph-mon[30963]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T08:15:20.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.596+0000 7fe8708af640 1 -- 172.21.15.22:0/2295433070 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11906 (secure 0 0 0) 0x7fe86c199860 con 0x7fe86c102ef0 2024-09-19T08:15:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:20.597 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"btime":"2024-09-19T08:15:00:109390+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:15:00.059208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:15:20.597 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-09-19T08:15:20.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.598+0000 7fe8718b1640 1 -- 172.21.15.22:0/2295433070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe8540a3550 msgr2=0x7fe8540a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:20.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.598+0000 7fe8718b1640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe8540a3550 0x7fe8540a5a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0058c0 tx=0x7fe85c005850 comp rx=0 tx=0).stop 2024-09-19T08:15:20.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.598+0000 7fe8718b1640 1 -- 172.21.15.22:0/2295433070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe86c102ef0 msgr2=0x7fe86c19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:20.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.598+0000 7fe8718b1640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe86c102ef0 0x7fe86c19cdd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe864004af0 tx=0x7fe86402f5b0 comp rx=0 tx=0).stop 2024-09-19T08:15:20.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.598+0000 7fe8718b1640 1 -- 172.21.15.22:0/2295433070 shutdown_connections 2024-09-19T08:15:20.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.598+0000 7fe8718b1640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe8540a3550 0x7fe8540a5a10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:20.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.598+0000 7fe8718b1640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe86c102ef0 0x7fe86c19cdd0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:20.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.598+0000 7fe8718b1640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe86c102570 0x7fe86c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:20.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.598+0000 7fe8718b1640 1 --2- 172.21.15.22:0/2295433070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe86c100e20 0x7fe86c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:20.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.598+0000 7fe8718b1640 1 -- 172.21.15.22:0/2295433070 >> 172.21.15.22:0/2295433070 conn(0x7fe86c0fc7d0 msgr2=0x7fe86c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:20.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.599+0000 7fe8718b1640 1 -- 172.21.15.22:0/2295433070 shutdown_connections 2024-09-19T08:15:20.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:20.599+0000 7fe8718b1640 1 -- 172.21.15.22:0/2295433070 wait complete. 2024-09-19T08:15:20.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:20 smithi157 ceph-mon[35143]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T08:15:20.755 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:20 smithi064 ceph-mon[34900]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T08:15:21.284 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:15:21.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:21 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2295433070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:21.542 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:21.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:21 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2295433070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:21.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:21 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2295433070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.039+0000 7f9d52512640 1 -- 172.21.15.22:0/2302391046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c103660 msgr2=0x7f9d4c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.039+0000 7f9d52512640 1 --2- 172.21.15.22:0/2302391046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c103660 0x7f9d4c10bb50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009900 tx=0x7f9d4002a480 comp rx=0 tx=0).stop 2024-09-19T08:15:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.039+0000 7f9d52512640 1 -- 172.21.15.22:0/2302391046 shutdown_connections 2024-09-19T08:15:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.039+0000 7f9d52512640 1 --2- 172.21.15.22:0/2302391046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d4c10c170 0x7f9d4c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.039+0000 7f9d52512640 1 --2- 172.21.15.22:0/2302391046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c103660 0x7f9d4c10bb50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:22.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.039+0000 7f9d52512640 1 --2- 172.21.15.22:0/2302391046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d4c102d40 0x7f9d4c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:22.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.039+0000 7f9d52512640 1 -- 172.21.15.22:0/2302391046 >> 172.21.15.22:0/2302391046 conn(0x7f9d4c0fc7a0 msgr2=0x7f9d4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:22.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.039+0000 7f9d52512640 1 -- 172.21.15.22:0/2302391046 shutdown_connections 2024-09-19T08:15:22.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.039+0000 7f9d52512640 1 -- 172.21.15.22:0/2302391046 wait complete. 2024-09-19T08:15:22.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.040+0000 7f9d52512640 1 Processor -- start 2024-09-19T08:15:22.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.040+0000 7f9d52512640 1 -- start start 2024-09-19T08:15:22.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.040+0000 7f9d52512640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d4c102d40 0x7f9d4c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:22.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.040+0000 7f9d52512640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d4c103660 0x7f9d4c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:22.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.040+0000 7f9d52512640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c10c170 0x7f9d4c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:22.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.040+0000 7f9d52512640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c111f70 con 0x7f9d4c10c170 2024-09-19T08:15:22.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.040+0000 7f9d52512640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c111df0 con 0x7f9d4c103660 2024-09-19T08:15:22.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.040+0000 7f9d52512640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c1120f0 con 0x7f9d4c102d40 2024-09-19T08:15:22.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.040+0000 7f9d50aaf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c10c170 0x7f9d4c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:22.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.040+0000 7f9d50aaf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c10c170 0x7f9d4c101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58666/0 (socket says 172.21.15.22:58666) 2024-09-19T08:15:22.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.040+0000 7f9d4b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d4c103660 0x7f9d4c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:22.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.040+0000 7f9d50aaf640 1 -- 172.21.15.22:0/1188235040 learned_addr learned my addr 172.21.15.22:0/1188235040 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:22.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.040+0000 7f9d4bfff640 1 --2- 172.21.15.22:0/1188235040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d4c102d40 0x7f9d4c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:22.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.041+0000 7f9d50aaf640 1 -- 172.21.15.22:0/1188235040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d4c102d40 msgr2=0x7f9d4c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:22.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.041+0000 7f9d50aaf640 1 --2- 172.21.15.22:0/1188235040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d4c102d40 0x7f9d4c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:22.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.041+0000 7f9d50aaf640 1 -- 172.21.15.22:0/1188235040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d4c103660 msgr2=0x7f9d4c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:22.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.041+0000 7f9d50aaf640 1 --2- 172.21.15.22:0/1188235040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d4c103660 0x7f9d4c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:22.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.041+0000 7f9d50aaf640 1 -- 172.21.15.22:0/1188235040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4c1a6930 con 0x7f9d4c10c170 2024-09-19T08:15:22.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.041+0000 7f9d4bfff640 1 --2- 172.21.15.22:0/1188235040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d4c102d40 0x7f9d4c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:22.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.041+0000 7f9d50aaf640 1 --2- 172.21.15.22:0/1188235040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c10c170 0x7f9d4c101420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400cad0 tx=0x7f9d4400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:22.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.041+0000 7f9d497fa640 1 -- 172.21.15.22:0/1188235040 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4400ae20 con 0x7f9d4c10c170 2024-09-19T08:15:22.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.041+0000 7f9d497fa640 1 -- 172.21.15.22:0/1188235040 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d44005020 con 0x7f9d4c10c170 2024-09-19T08:15:22.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.041+0000 7f9d52512640 1 -- 172.21.15.22:0/1188235040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4c1a6c20 con 0x7f9d4c10c170 2024-09-19T08:15:22.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.041+0000 7f9d497fa640 1 -- 172.21.15.22:0/1188235040 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4400f6f0 con 0x7f9d4c10c170 2024-09-19T08:15:22.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.041+0000 7f9d52512640 1 -- 172.21.15.22:0/1188235040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d4c1a7130 con 0x7f9d4c10c170 2024-09-19T08:15:22.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.042+0000 7f9d52512640 1 -- 172.21.15.22:0/1188235040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d4c103120 con 0x7f9d4c10c170 2024-09-19T08:15:22.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.043+0000 7f9d497fa640 1 -- 172.21.15.22:0/1188235040 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f9d440051d0 con 0x7f9d4c10c170 2024-09-19T08:15:22.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.044+0000 7f9d497fa640 1 --2- 172.21.15.22:0/1188235040 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9d340a34d0 0x7f9d340a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:22.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.044+0000 7f9d4bfff640 1 --2- 172.21.15.22:0/1188235040 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9d340a34d0 0x7f9d340a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:22.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.044+0000 7f9d497fa640 1 -- 172.21.15.22:0/1188235040 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9d440d02a0 con 0x7f9d4c10c170 2024-09-19T08:15:22.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.044+0000 7f9d4bfff640 1 --2- 172.21.15.22:0/1188235040 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9d340a34d0 0x7f9d340a5990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009c20 tx=0x7f9d3c005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:22.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.048+0000 7f9d497fa640 1 -- 172.21.15.22:0/1188235040 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9d44014030 con 0x7f9d4c10c170 2024-09-19T08:15:22.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.226+0000 7f9d52512640 1 -- 172.21.15.22:0/1188235040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d4c1071e0 con 0x7f9d4c10c170 2024-09-19T08:15:22.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.228+0000 7f9d497fa640 1 -- 172.21.15.22:0/1188235040 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+23100 (secure 0 0 0) 0x7f9d4409ca40 con 0x7f9d4c10c170 2024-09-19T08:15:22.229 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:22.229 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":86,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","created":"2024-09-19T08:05:19.476952+0000","modified":"2024-09-19T08:15:01.165514+0000","last_up_change":"2024-09-19T08:12:10.339586+0000","last_in_change":"2024-09-19T08:11:47.653576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:08:25.175759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:12:57.608787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:12:59.824327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ed67f2d-9b28-427a-b69f-676e7a2abcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6803","nonce":3442715307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6805","nonce":3442715307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6809","nonce":3442715307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6807","nonce":3442715307}]},"public_addr":"172.21.15.22:6803/3442715307","cluster_addr":"172.21.15.22:6805/3442715307","heartbeat_back_addr":"172.21.15.22:6809/3442715307","heartbeat_front_addr":"172.21.15.22:6807/3442715307","state":["exists","up"]},{"osd":1,"uuid":"bff3e283-4ab1-49d7-86d8-45d361995122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6801","nonce":2379700228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6803","nonce":2379700228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6807","nonce":2379700228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6805","nonce":2379700228}]},"public_addr":"172.21.15.64:6801/2379700228","cluster_addr":"172.21.15.64:6803/2379700228","heartbeat_back_addr":"172.21.15.64:6807/2379700228","heartbeat_front_addr":"172.21.15.64:6805/2379700228","state":["exists","up"]},{"osd":2,"uuid":"81421d08-7dc6-419b-a566-dee1e1e13b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6801","nonce":3363263867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6803","nonce":3363263867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6807","nonce":3363263867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6805","nonce":3363263867}]},"public_addr":"172.21.15.157:6801/3363263867","cluster_addr":"172.21.15.157:6803/3363263867","heartbeat_back_addr":"172.21.15.157:6807/3363263867","heartbeat_front_addr":"172.21.15.157:6805/3363263867","state":["exists","up"]},{"osd":3,"uuid":"34debfd0-3226-4d78-b9e6-e03dbc5190f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6811","nonce":4225229456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6813","nonce":4225229456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6817","nonce":4225229456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6815","nonce":4225229456}]},"public_addr":"172.21.15.22:6811/4225229456","cluster_addr":"172.21.15.22:6813/4225229456","heartbeat_back_addr":"172.21.15.22:6817/4225229456","heartbeat_front_addr":"172.21.15.22:6815/4225229456","state":["exists","up"]},{"osd":4,"uuid":"1293332c-d772-4ca9-bf76-79e0b29451ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6809","nonce":1858090423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6811","nonce":1858090423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6815","nonce":1858090423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6813","nonce":1858090423}]},"public_addr":"172.21.15.64:6809/1858090423","cluster_addr":"172.21.15.64:6811/1858090423","heartbeat_back_addr":"172.21.15.64:6815/1858090423","heartbeat_front_addr":"172.21.15.64:6813/1858090423","state":["exists","up"]},{"osd":5,"uuid":"240a6c42-89a2-41c1-948c-b09914b5722e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6809","nonce":2900701212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2900701212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2900701212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2900701212}]},"public_addr":"172.21.15.157:6809/2900701212","cluster_addr":"172.21.15.157:6811/2900701212","heartbeat_back_addr":"172.21.15.157:6815/2900701212","heartbeat_front_addr":"172.21.15.157:6813/2900701212","state":["exists","up"]},{"osd":6,"uuid":"80199030-109e-41c4-bc1c-731755464770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6819","nonce":4264426242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6821","nonce":4264426242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6825","nonce":4264426242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6823","nonce":4264426242}]},"public_addr":"172.21.15.22:6819/4264426242","cluster_addr":"172.21.15.22:6821/4264426242","heartbeat_back_addr":"172.21.15.22:6825/4264426242","heartbeat_front_addr":"172.21.15.22:6823/4264426242","state":["exists","up"]},{"osd":7,"uuid":"dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6817","nonce":3980224103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6819","nonce":3980224103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6823","nonce":3980224103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6821","nonce":3980224103}]},"public_addr":"172.21.15.64:6817/3980224103","cluster_addr":"172.21.15.64:6819/3980224103","heartbeat_back_addr":"172.21.15.64:6823/3980224103","heartbeat_front_addr":"172.21.15.64:6821/3980224103","state":["exists","up"]},{"osd":8,"uuid":"764b1e6e-48c9-49f9-8124-bc13bcba3a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6817","nonce":973678477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6819","nonce":973678477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6823","nonce":973678477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6821","nonce":973678477}]},"public_addr":"172.21.15.157:6817/973678477","cluster_addr":"172.21.15.157:6819/973678477","heartbeat_back_addr":"172.21.15.157:6823/973678477","heartbeat_front_addr":"172.21.15.157:6821/973678477","state":["exists","up"]},{"osd":9,"uuid":"82914818-df47-4374-9e3e-2f71fc76e78b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6827","nonce":710033990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6829","nonce":710033990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6833","nonce":710033990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6831","nonce":710033990}]},"public_addr":"172.21.15.22:6827/710033990","cluster_addr":"172.21.15.22:6829/710033990","heartbeat_back_addr":"172.21.15.22:6833/710033990","heartbeat_front_addr":"172.21.15.22:6831/710033990","state":["exists","up"]},{"osd":10,"uuid":"36ae177d-c454-4cba-ba21-cbeee0f10dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6825","nonce":373412752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6827","nonce":373412752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6831","nonce":373412752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6829","nonce":373412752}]},"public_addr":"172.21.15.64:6825/373412752","cluster_addr":"172.21.15.64:6827/373412752","heartbeat_back_addr":"172.21.15.64:6831/373412752","heartbeat_front_addr":"172.21.15.64:6829/373412752","state":["exists","up"]},{"osd":11,"uuid":"cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6825","nonce":1918889194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6827","nonce":1918889194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6831","nonce":1918889194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6829","nonce":1918889194}]},"public_addr":"172.21.15.157:6825/1918889194","cluster_addr":"172.21.15.157:6827/1918889194","heartbeat_back_addr":"172.21.15.157:6831/1918889194","heartbeat_front_addr":"172.21.15.157:6829/1918889194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:31.378218+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:56.794167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:20.790267+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:47.039106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:11.483842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:38.151466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:03.553308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:27.865321+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:53.086530+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:18.946661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:43.697243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:12:08.936588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/422860013":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/2837500116":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/1845420423":"2024-09-20T08:15:01.165466+0000","172.21.15.22:6800/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/3623845502":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2851750481":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2133059277":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/384027596":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/586163474":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/3712426751":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/1866276413":"2024-09-20T08:15:01.165466+0000","172.21.15.22:6801/1866276413":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/155709052":"2024-09-20T08:15:01.165466+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:15:22.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.230+0000 7f9d52512640 1 -- 172.21.15.22:0/1188235040 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9d340a34d0 msgr2=0x7f9d340a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:22.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.230+0000 7f9d52512640 1 --2- 172.21.15.22:0/1188235040 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9d340a34d0 0x7f9d340a5990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009c20 tx=0x7f9d3c005f90 comp rx=0 tx=0).stop 2024-09-19T08:15:22.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.230+0000 7f9d52512640 1 -- 172.21.15.22:0/1188235040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c10c170 msgr2=0x7f9d4c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:22.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.230+0000 7f9d52512640 1 --2- 172.21.15.22:0/1188235040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c10c170 0x7f9d4c101420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400cad0 tx=0x7f9d4400cf90 comp rx=0 tx=0).stop 2024-09-19T08:15:22.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.230+0000 7f9d52512640 1 -- 172.21.15.22:0/1188235040 shutdown_connections 2024-09-19T08:15:22.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.230+0000 7f9d52512640 1 --2- 172.21.15.22:0/1188235040 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9d340a34d0 0x7f9d340a5990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:22.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.230+0000 7f9d52512640 1 --2- 172.21.15.22:0/1188235040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c10c170 0x7f9d4c101420 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:22.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.230+0000 7f9d52512640 1 --2- 172.21.15.22:0/1188235040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d4c103660 0x7f9d4c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:22.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.230+0000 7f9d52512640 1 --2- 172.21.15.22:0/1188235040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d4c102d40 0x7f9d4c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:22.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.230+0000 7f9d52512640 1 -- 172.21.15.22:0/1188235040 >> 172.21.15.22:0/1188235040 conn(0x7f9d4c0fc7a0 msgr2=0x7f9d4c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:22.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.231+0000 7f9d52512640 1 -- 172.21.15.22:0/1188235040 shutdown_connections 2024-09-19T08:15:22.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:22.231+0000 7f9d52512640 1 -- 172.21.15.22:0/1188235040 wait complete. 2024-09-19T08:15:22.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:22 smithi022 ceph-mon[30963]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-19T08:15:22.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:22 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:15:22.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:22 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1188235040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:15:22.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:22 smithi157 ceph-mon[35143]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-19T08:15:22.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:22 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:15:22.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:22 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1188235040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:15:22.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:22 smithi064 ceph-mon[34900]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-19T08:15:22.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:22 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:15:22.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:22 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1188235040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:15:22.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-19T08:15:23.247 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:24.462 INFO:teuthology.orchestra.run.smithi022.stdout:{"run_dir":"/var/run/ceph"} 2024-09-19T08:15:24.463 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-19T08:15:24.463 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1747 2024-09-19T08:15:24.464 DEBUG:tasks.quiescer:Yielding 2024-09-19T08:15:24.464 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T08:15:24.483 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T08:15:24.484 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T08:15:24.484 INFO:tasks.workunit:timeout=3h 2024-09-19T08:15:24.484 INFO:tasks.workunit:cleanup=True 2024-09-19T08:15:24.484 DEBUG:teuthology.orchestra.run.smithi022:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T08:15:24.485 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:15:24.503 INFO:teuthology.orchestra.run.smithi022.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T08:15:24.503 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-19T08:15:24.503 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-19T08:15:24.503 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:15:24.503 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-19T08:15:24.503 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-19 08:13:20.612370596 +0000 2024-09-19T08:15:24.503 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-19 08:13:20.612370596 +0000 2024-09-19T08:15:24.503 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-19 08:14:59.299963835 +0000 2024-09-19T08:15:24.503 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-19 08:13:20.612370596 +0000 2024-09-19T08:15:24.503 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T08:15:24.504 DEBUG:teuthology.orchestra.run.smithi022:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T08:15:24.579 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T08:15:24.636 INFO:tasks.workunit.client.0.smithi022.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T08:15:24.748 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:24.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:24 smithi157 ceph-mon[35143]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-19T08:15:24.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:24 smithi022 ceph-mon[30963]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-19T08:15:24.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:24 smithi064 ceph-mon[34900]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-19T08:15:25.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.269+0000 7f7628463640 1 -- 172.21.15.22:0/2692149812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7620100e20 msgr2=0x7f7620101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.269+0000 7f7628463640 1 --2- 172.21.15.22:0/2692149812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7620100e20 0x7f7620101200 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f761c009900 tx=0x7f761c02a480 comp rx=0 tx=0).stop 2024-09-19T08:15:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.269+0000 7f7628463640 1 -- 172.21.15.22:0/2692149812 shutdown_connections 2024-09-19T08:15:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.269+0000 7f7628463640 1 --2- 172.21.15.22:0/2692149812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7620102ef0 0x7f762010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.269+0000 7f7628463640 1 --2- 172.21.15.22:0/2692149812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7620102570 0x7f76201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.269+0000 7f7628463640 1 --2- 172.21.15.22:0/2692149812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7620100e20 0x7f7620101200 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.269+0000 7f7628463640 1 -- 172.21.15.22:0/2692149812 >> 172.21.15.22:0/2692149812 conn(0x7f76200fc7d0 msgr2=0x7f76200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.269+0000 7f7628463640 1 -- 172.21.15.22:0/2692149812 shutdown_connections 2024-09-19T08:15:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.269+0000 7f7628463640 1 -- 172.21.15.22:0/2692149812 wait complete. 2024-09-19T08:15:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.270+0000 7f7628463640 1 Processor -- start 2024-09-19T08:15:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.270+0000 7f7628463640 1 -- start start 2024-09-19T08:15:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.270+0000 7f7628463640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7620100e20 0x7f76200fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:25.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.270+0000 7f7628463640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7620102570 0x7f76200fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:25.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.270+0000 7f7628463640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7620102ef0 0x7f76200fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:25.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.270+0000 7f7628463640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7620110f50 con 0x7f7620100e20 2024-09-19T08:15:25.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.270+0000 7f7628463640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7620110dd0 con 0x7f7620102570 2024-09-19T08:15:25.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.270+0000 7f7628463640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f76201110d0 con 0x7f7620102ef0 2024-09-19T08:15:25.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.270+0000 7f76261ff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7620100e20 0x7f76200fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:25.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.270+0000 7f76261ff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7620100e20 0x7f76200fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58682/0 (socket says 172.21.15.22:58682) 2024-09-19T08:15:25.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.270+0000 7f76261ff640 1 -- 172.21.15.22:0/3301766658 learned_addr learned my addr 172.21.15.22:0/3301766658 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:25.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.271+0000 7f7626a00640 1 --2- 172.21.15.22:0/3301766658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7620102ef0 0x7f76200fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:25.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.271+0000 7f76259fe640 1 --2- 172.21.15.22:0/3301766658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7620102570 0x7f76200fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:25.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.271+0000 7f76261ff640 1 -- 172.21.15.22:0/3301766658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7620102ef0 msgr2=0x7f76200fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:25.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.271+0000 7f76261ff640 1 --2- 172.21.15.22:0/3301766658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7620102ef0 0x7f76200fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:25.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.271+0000 7f76261ff640 1 -- 172.21.15.22:0/3301766658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7620102570 msgr2=0x7f76200fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.271+0000 7f76261ff640 1 --2- 172.21.15.22:0/3301766658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7620102570 0x7f76200fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.271+0000 7f76261ff640 1 -- 172.21.15.22:0/3301766658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76201a6940 con 0x7f7620100e20 2024-09-19T08:15:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.271+0000 7f76261ff640 1 --2- 172.21.15.22:0/3301766658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7620100e20 0x7f76200fcc80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f761c006010 tx=0x7f761c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.271+0000 7f76177fe640 1 -- 172.21.15.22:0/3301766658 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c03f070 con 0x7f7620100e20 2024-09-19T08:15:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.271+0000 7f76177fe640 1 -- 172.21.15.22:0/3301766658 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f761c031e30 con 0x7f7620100e20 2024-09-19T08:15:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.271+0000 7f7628463640 1 -- 172.21.15.22:0/3301766658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76201a6bd0 con 0x7f7620100e20 2024-09-19T08:15:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.271+0000 7f76177fe640 1 -- 172.21.15.22:0/3301766658 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c035030 con 0x7f7620100e20 2024-09-19T08:15:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.271+0000 7f7628463640 1 -- 172.21.15.22:0/3301766658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76201a7100 con 0x7f7620100e20 2024-09-19T08:15:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.272+0000 7f7628463640 1 -- 172.21.15.22:0/3301766658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7620101200 con 0x7f7620100e20 2024-09-19T08:15:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.273+0000 7f76177fe640 1 -- 172.21.15.22:0/3301766658 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f761c042070 con 0x7f7620100e20 2024-09-19T08:15:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.274+0000 7f76177fe640 1 --2- 172.21.15.22:0/3301766658 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f760c0a3550 0x7f760c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.274+0000 7f76259fe640 1 --2- 172.21.15.22:0/3301766658 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f760c0a3550 0x7f760c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.274+0000 7f76177fe640 1 -- 172.21.15.22:0/3301766658 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f761c0f0870 con 0x7f7620100e20 2024-09-19T08:15:25.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.275+0000 7f76259fe640 1 --2- 172.21.15.22:0/3301766658 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f760c0a3550 0x7f760c0a5a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7610005e10 tx=0x7f7610005da0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:25.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.278+0000 7f76177fe640 1 -- 172.21.15.22:0/3301766658 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f761c0bd0e0 con 0x7f7620100e20 2024-09-19T08:15:25.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.491+0000 7f7628463640 1 -- 172.21.15.22:0/3301766658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f762010a0a0 con 0x7f7620100e20 2024-09-19T08:15:25.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.492+0000 7f76177fe640 1 -- 172.21.15.22:0/3301766658 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11906 (secure 0 0 0) 0x7f761c0bc9e0 con 0x7f7620100e20 2024-09-19T08:15:25.492 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:25.492 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"btime":"2024-09-19T08:15:00:109390+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:15:00.059208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:15:25.492 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-09-19T08:15:25.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.494+0000 7f7628463640 1 -- 172.21.15.22:0/3301766658 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f760c0a3550 msgr2=0x7f760c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:25.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.494+0000 7f7628463640 1 --2- 172.21.15.22:0/3301766658 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f760c0a3550 0x7f760c0a5a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7610005e10 tx=0x7f7610005da0 comp rx=0 tx=0).stop 2024-09-19T08:15:25.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.494+0000 7f7628463640 1 -- 172.21.15.22:0/3301766658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7620100e20 msgr2=0x7f76200fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:25.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.494+0000 7f7628463640 1 --2- 172.21.15.22:0/3301766658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7620100e20 0x7f76200fcc80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f761c006010 tx=0x7f761c003c80 comp rx=0 tx=0).stop 2024-09-19T08:15:25.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.495+0000 7f7628463640 1 -- 172.21.15.22:0/3301766658 shutdown_connections 2024-09-19T08:15:25.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.495+0000 7f7628463640 1 --2- 172.21.15.22:0/3301766658 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f760c0a3550 0x7f760c0a5a10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:25.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.495+0000 7f7628463640 1 --2- 172.21.15.22:0/3301766658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7620102ef0 0x7f76200fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:25.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.495+0000 7f7628463640 1 --2- 172.21.15.22:0/3301766658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7620102570 0x7f76200fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:25.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.495+0000 7f7628463640 1 --2- 172.21.15.22:0/3301766658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7620100e20 0x7f76200fcc80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:25.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.495+0000 7f7628463640 1 -- 172.21.15.22:0/3301766658 >> 172.21.15.22:0/3301766658 conn(0x7f76200fc7d0 msgr2=0x7f762006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:25.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.495+0000 7f7628463640 1 -- 172.21.15.22:0/3301766658 shutdown_connections 2024-09-19T08:15:25.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:25.495+0000 7f7628463640 1 -- 172.21.15.22:0/3301766658 wait complete. 2024-09-19T08:15:26.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T08:13:01.503449+0000', 'modified': '2024-09-19T08:15:00.059208+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24481, 'mds_1': 24479, 'mds_2': 24460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6835/3333087560', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 3333087560}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 3333087560}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6835/4158350679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 4158350679}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 4158350679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6837/2142121431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6836', 'nonce': 2142121431}, {'type': 'v1', 'addr': '172.21.15.64:6837', 'nonce': 2142121431}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.157:6837/892624399', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 892624399}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 892624399}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.64:6839/358484749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6838', 'nonce': 358484749}, {'type': 'v1', 'addr': '172.21.15.64:6839', 'nonce': 358484749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.157:6839/3802797238', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 3802797238}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 3802797238}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24481, 'qdb_cluster': [24481, 24479, 24460]} 2024-09-19T08:15:26.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T08:15:26.203 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T08:15:26.505 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:26.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:26 smithi022 ceph-mon[30963]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T08:15:26.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:26 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3301766658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:26.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:26 smithi157 ceph-mon[35143]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T08:15:26.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:26 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3301766658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:26.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:26 smithi064 ceph-mon[34900]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T08:15:26.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:26 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3301766658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:27.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.020+0000 7f1c332dc640 1 -- 172.21.15.22:0/102864872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c102d90 msgr2=0x7f1c2c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:27.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.020+0000 7f1c332dc640 1 --2- 172.21.15.22:0/102864872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c102d90 0x7f1c2c103170 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009930 tx=0x7f1c1c02a420 comp rx=0 tx=0).stop 2024-09-19T08:15:27.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.020+0000 7f1c332dc640 1 -- 172.21.15.22:0/102864872 shutdown_connections 2024-09-19T08:15:27.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.020+0000 7f1c332dc640 1 --2- 172.21.15.22:0/102864872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c108290 0x7f1c2c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.020+0000 7f1c332dc640 1 --2- 172.21.15.22:0/102864872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c2c103740 0x7f1c2c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.020+0000 7f1c332dc640 1 --2- 172.21.15.22:0/102864872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c102d90 0x7f1c2c103170 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.020+0000 7f1c332dc640 1 -- 172.21.15.22:0/102864872 >> 172.21.15.22:0/102864872 conn(0x7f1c2c0fc7d0 msgr2=0x7f1c2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.020+0000 7f1c332dc640 1 -- 172.21.15.22:0/102864872 shutdown_connections 2024-09-19T08:15:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.020+0000 7f1c332dc640 1 -- 172.21.15.22:0/102864872 wait complete. 2024-09-19T08:15:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.020+0000 7f1c332dc640 1 Processor -- start 2024-09-19T08:15:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.020+0000 7f1c332dc640 1 -- start start 2024-09-19T08:15:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c332dc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c102d90 0x7f1c2c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c332dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c2c103740 0x7f1c2c100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c332dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c108290 0x7f1c2c101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c332dc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c112090 con 0x7f1c2c102d90 2024-09-19T08:15:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c332dc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c111f10 con 0x7f1c2c103740 2024-09-19T08:15:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c332dc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c112210 con 0x7f1c2c108290 2024-09-19T08:15:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c31078640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c102d90 0x7f1c2c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c31078640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c102d90 0x7f1c2c1009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58692/0 (socket says 172.21.15.22:58692) 2024-09-19T08:15:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c31078640 1 -- 172.21.15.22:0/2612368596 learned_addr learned my addr 172.21.15.22:0/2612368596 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c30877640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c2c103740 0x7f1c2c100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c31879640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c108290 0x7f1c2c101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c31078640 1 -- 172.21.15.22:0/2612368596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c108290 msgr2=0x7f1c2c101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c31078640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c108290 0x7f1c2c101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c31078640 1 -- 172.21.15.22:0/2612368596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c2c103740 msgr2=0x7f1c2c100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c31078640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c2c103740 0x7f1c2c100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c31078640 1 -- 172.21.15.22:0/2612368596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c1a6a10 con 0x7f1c2c102d90 2024-09-19T08:15:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c31879640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c108290 0x7f1c2c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c31078640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c102d90 0x7f1c2c1009f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009230 tx=0x7f1c1c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.021+0000 7f1c227fc640 1 -- 172.21.15.22:0/2612368596 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c03f030 con 0x7f1c2c102d90 2024-09-19T08:15:27.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.022+0000 7f1c332dc640 1 -- 172.21.15.22:0/2612368596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2c1a6ca0 con 0x7f1c2c102d90 2024-09-19T08:15:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.022+0000 7f1c227fc640 1 -- 172.21.15.22:0/2612368596 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c003ad0 con 0x7f1c2c102d90 2024-09-19T08:15:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.022+0000 7f1c227fc640 1 -- 172.21.15.22:0/2612368596 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c035070 con 0x7f1c2c102d90 2024-09-19T08:15:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.022+0000 7f1c332dc640 1 -- 172.21.15.22:0/2612368596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2c1a7180 con 0x7f1c2c102d90 2024-09-19T08:15:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.024+0000 7f1c227fc640 1 -- 172.21.15.22:0/2612368596 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f1c1c042030 con 0x7f1c2c102d90 2024-09-19T08:15:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.024+0000 7f1c356b5740 1 Processor -- start 2024-09-19T08:15:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.024+0000 7f1c356b5740 1 -- start start 2024-09-19T08:15:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c356b5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752e99d0 0x5566752e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c356b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752ea2f0 0x5566752f4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c356b5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752f4bb0 0x5566752f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c356b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556675250090 con 0x5566752ea2f0 2024-09-19T08:15:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c356b5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5566751f7a70 con 0x5566752f4bb0 2024-09-19T08:15:27.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c356b5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55667524c050 con 0x5566752e99d0 2024-09-19T08:15:27.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c31078640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752ea2f0 0x5566752f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c31078640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752ea2f0 0x5566752f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58700/0 (socket says 172.21.15.22:58700) 2024-09-19T08:15:27.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c227fc640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f1c140a34d0 0x7f1c140a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c31078640 1 -- 172.21.15.22:0/1509258287 learned_addr learned my addr 172.21.15.22:0/1509258287 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:27.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c31879640 1 --2- 172.21.15.22:0/1509258287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752e99d0 0x5566752e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c30877640 1 --2- 172.21.15.22:0/1509258287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752f4bb0 0x5566752f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c31879640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f1c140a34d0 0x7f1c140a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c227fc640 1 -- 172.21.15.22:0/2612368596 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1c1c0f0440 con 0x7f1c2c102d90 2024-09-19T08:15:27.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c31078640 1 -- 172.21.15.22:0/1509258287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752e99d0 msgr2=0x5566752e9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:27.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c31078640 1 --2- 172.21.15.22:0/1509258287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752e99d0 0x5566752e9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c31078640 1 -- 172.21.15.22:0/1509258287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752f4bb0 msgr2=0x5566752f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:27.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.025+0000 7f1c31078640 1 --2- 172.21.15.22:0/1509258287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752f4bb0 0x5566752f6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.026+0000 7f1c31078640 1 -- 172.21.15.22:0/1509258287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556675204130 con 0x5566752ea2f0 2024-09-19T08:15:27.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.026+0000 7f1c31078640 1 --2- 172.21.15.22:0/1509258287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752ea2f0 0x5566752f4670 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0bc2b0 tx=0x7f1c1c0b8d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0fc8449e0b2de20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:27.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.026+0000 7f1c31879640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f1c140a34d0 0x7f1c140a5990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c24007310 tx=0x7f1c240072a0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:27.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.026+0000 7f1c137fe640 1 -- 172.21.15.22:0/1509258287 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c049070 con 0x5566752ea2f0 2024-09-19T08:15:27.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.026+0000 7f1c137fe640 1 -- 172.21.15.22:0/1509258287 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c0bccb0 con 0x5566752ea2f0 2024-09-19T08:15:27.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.026+0000 7f1c137fe640 1 -- 172.21.15.22:0/1509258287 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0bc810 con 0x5566752ea2f0 2024-09-19T08:15:27.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.027+0000 7f1c356b5740 1 -- 172.21.15.22:0/1509258287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752ea2f0 msgr2=0x5566752f4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:27.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.027+0000 7f1c356b5740 1 --2- 172.21.15.22:0/1509258287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752ea2f0 0x5566752f4670 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0bc2b0 tx=0x7f1c1c0b8d50 comp rx=0 tx=0).stop 2024-09-19T08:15:27.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.027+0000 7f1c356b5740 1 -- 172.21.15.22:0/1509258287 shutdown_connections 2024-09-19T08:15:27.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.027+0000 7f1c356b5740 1 --2- 172.21.15.22:0/1509258287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752f4bb0 0x5566752f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.027+0000 7f1c356b5740 1 --2- 172.21.15.22:0/1509258287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752ea2f0 0x5566752f4670 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.027+0000 7f1c356b5740 1 --2- 172.21.15.22:0/1509258287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752e99d0 0x5566752e9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.027+0000 7f1c356b5740 1 -- 172.21.15.22:0/1509258287 >> 172.21.15.22:0/1509258287 conn(0x5566752e5f60 msgr2=0x5566752e6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:27.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.027+0000 7f1c356b5740 1 -- 172.21.15.22:0/1509258287 shutdown_connections 2024-09-19T08:15:27.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.027+0000 7f1c356b5740 1 -- 172.21.15.22:0/1509258287 wait complete. 2024-09-19T08:15:27.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.027+0000 7f1c356b5740 1 -- start start 2024-09-19T08:15:27.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.027+0000 7f1c356b5740 1 Processor -- start 2024-09-19T08:15:27.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c356b5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752e99d0 0x5566752f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c356b5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752ea2f0 0x5566752f1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c356b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752f4bb0 0x5566752f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c356b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5566752f7540 con 0x5566752f4bb0 2024-09-19T08:15:27.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c356b5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55667520af20 con 0x5566752e99d0 2024-09-19T08:15:27.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c356b5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5566752f76c0 con 0x5566752ea2f0 2024-09-19T08:15:27.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c356b5740 10 client.0 _ll_register_callbacks cb 0x5566751d4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:27.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c31879640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752f4bb0 0x5566752f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c30877640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752ea2f0 0x5566752f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c30877640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752ea2f0 0x5566752f1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:44946/0 (socket says 172.21.15.22:44946) 2024-09-19T08:15:27.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c30877640 1 -- 172.21.15.22:0/4155212659 learned_addr learned my addr 172.21.15.22:0/4155212659 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:27.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c31078640 1 --2- 172.21.15.22:0/4155212659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752e99d0 0x5566752f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c31879640 1 -- 172.21.15.22:0/4155212659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752ea2f0 msgr2=0x5566752f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:27.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c31879640 1 --2- 172.21.15.22:0/4155212659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752ea2f0 0x5566752f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c31879640 1 -- 172.21.15.22:0/4155212659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752e99d0 msgr2=0x5566752f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:27.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c31879640 1 --2- 172.21.15.22:0/4155212659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752e99d0 0x5566752f43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c31879640 1 -- 172.21.15.22:0/4155212659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556674fc12b0 con 0x5566752f4bb0 2024-09-19T08:15:27.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c30877640 1 --2- 172.21.15.22:0/4155212659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752ea2f0 0x5566752f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:27.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c31879640 1 --2- 172.21.15.22:0/4155212659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752f4bb0 0x5566752f1c40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400e6f0 tx=0x7f1c24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d924ac8e89897c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:27.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.028+0000 7f1c10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:27.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.029+0000 7f1c10ff9640 1 -- 172.21.15.22:0/4155212659 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24025070 con 0x5566752f4bb0 2024-09-19T08:15:27.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.029+0000 7f1c356b5740 1 -- 172.21.15.22:0/4155212659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c1c0bccb0 con 0x5566752f4bb0 2024-09-19T08:15:27.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.029+0000 7f1c10ff9640 1 -- 172.21.15.22:0/4155212659 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1c24004d20 con 0x5566752f4bb0 2024-09-19T08:15:27.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.029+0000 7f1c10ff9640 1 -- 172.21.15.22:0/4155212659 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c24030050 con 0x5566752f4bb0 2024-09-19T08:15:27.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.029+0000 7f1c356b5740 10 client.15402 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:27.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.029+0000 7f1c356b5740 1 -- 172.21.15.22:0/4155212659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556675302d40 con 0x5566752f4bb0 2024-09-19T08:15:27.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.029+0000 7f1c10ff9640 1 -- 172.21.15.22:0/4155212659 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c0bccb0 con 0x5566752f4bb0 2024-09-19T08:15:27.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.029+0000 7f1c10ff9640 1 -- 172.21.15.22:0/4155212659 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1c240050b0 con 0x5566752f4bb0 2024-09-19T08:15:27.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.029+0000 7f1c356b5740 10 client.15402 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.029+0000 7f1c356b5740 10 client.15402 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:15:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.029+0000 7f1c356b5740 20 client.15402 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.029+0000 7f1c356b5740 1 --2- 172.21.15.22:0/4155212659 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5566753053e0 0x556675306850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.029+0000 7f1c356b5740 4 client.15402 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:15:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.029+0000 7f1c356b5740 1 -- 172.21.15.22:0/4155212659 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55667525d600 con 0x5566753053e0 2024-09-19T08:15:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.030+0000 7f1c31078640 1 --2- 172.21.15.22:0/4155212659 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5566753053e0 0x556675306850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.030+0000 7f1c10ff9640 10 client.15402 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:15:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.030+0000 7f1c31078640 1 --2- 172.21.15.22:0/4155212659 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5566753053e0 0x556675306850 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16868039567ed5e1 server_cookie=739efff948f46977 in_seq=0 out_seq=0 2024-09-19T08:15:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c10ff9640 1 -- 172.21.15.22:0/4155212659 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55667525d600 con 0x5566753053e0 2024-09-19T08:15:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c10ff9640 10 client.15402 handle_command_reply: tid=0 2024-09-19T08:15:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c356b5740 1 client.15402 shutdown 2024-09-19T08:15:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c356b5740 1 -- 172.21.15.22:0/4155212659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752f4bb0 msgr2=0x5566752f1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c356b5740 1 --2- 172.21.15.22:0/4155212659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752f4bb0 0x5566752f1c40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400e6f0 tx=0x7f1c24004a40 comp rx=0 tx=0).stop 2024-09-19T08:15:27.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c356b5740 1 -- 172.21.15.22:0/4155212659 shutdown_connections 2024-09-19T08:15:27.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c356b5740 1 --2- 172.21.15.22:0/4155212659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752f4bb0 0x5566752f1c40 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c356b5740 1 --2- 172.21.15.22:0/4155212659 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5566753053e0 0x556675306850 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c356b5740 1 --2- 172.21.15.22:0/4155212659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752ea2f0 0x5566752f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c356b5740 1 --2- 172.21.15.22:0/4155212659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752e99d0 0x5566752f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c356b5740 1 -- 172.21.15.22:0/4155212659 >> 172.21.15.22:0/4155212659 conn(0x5566752e5f60 msgr2=0x5566752ebc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:27.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c10ff9640 0 client.15402 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:15:27.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c356b5740 1 -- 172.21.15.22:0/4155212659 shutdown_connections 2024-09-19T08:15:27.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c356b5740 1 -- 172.21.15.22:0/4155212659 wait complete. 2024-09-19T08:15:27.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.031+0000 7f1c356b5740 20 client.15402 trim_cache size 0 max 16384 2024-09-19T08:15:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.049+0000 7f1c356b5740 1 Processor -- start 2024-09-19T08:15:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.049+0000 7f1c356b5740 1 -- start start 2024-09-19T08:15:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.049+0000 7f1c356b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752e99d0 0x5566753053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.049+0000 7f1c356b5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752ea2f0 0x556675306180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.049+0000 7f1c356b5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752f4bb0 0x556675304d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.049+0000 7f1c356b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556675304b90 con 0x5566752e99d0 2024-09-19T08:15:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.049+0000 7f1c356b5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556675305260 con 0x5566752ea2f0 2024-09-19T08:15:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.049+0000 7f1c356b5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556675305930 con 0x5566752f4bb0 2024-09-19T08:15:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c31078640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752e99d0 0x5566753053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c31078640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752e99d0 0x5566753053e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58720/0 (socket says 172.21.15.22:58720) 2024-09-19T08:15:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c31078640 1 -- 172.21.15.22:0/4182725110 learned_addr learned my addr 172.21.15.22:0/4182725110 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c31879640 1 --2- 172.21.15.22:0/4182725110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752f4bb0 0x556675304d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c30877640 1 --2- 172.21.15.22:0/4182725110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752ea2f0 0x556675306180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c31078640 1 -- 172.21.15.22:0/4182725110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752f4bb0 msgr2=0x556675304d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c31078640 1 --2- 172.21.15.22:0/4182725110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752f4bb0 0x556675304d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c31078640 1 -- 172.21.15.22:0/4182725110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752ea2f0 msgr2=0x556675306180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c31078640 1 --2- 172.21.15.22:0/4182725110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752ea2f0 0x556675306180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c31078640 1 -- 172.21.15.22:0/4182725110 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566752efc50 con 0x5566752e99d0 2024-09-19T08:15:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c31879640 1 --2- 172.21.15.22:0/4182725110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752f4bb0 0x556675304d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c31078640 1 --2- 172.21.15.22:0/4182725110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752e99d0 0x5566753053e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0b8a10 tx=0x7f1c1c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40903228f35a8207 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c12ffd640 1 -- 172.21.15.22:0/4182725110 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0d5070 con 0x5566752e99d0 2024-09-19T08:15:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c12ffd640 1 -- 172.21.15.22:0/4182725110 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c0a5030 con 0x5566752e99d0 2024-09-19T08:15:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.050+0000 7f1c12ffd640 1 -- 172.21.15.22:0/4182725110 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c049040 con 0x5566752e99d0 2024-09-19T08:15:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.051+0000 7f1c356b5740 1 -- 172.21.15.22:0/4182725110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752e99d0 msgr2=0x5566753053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.051+0000 7f1c356b5740 1 --2- 172.21.15.22:0/4182725110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752e99d0 0x5566753053e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0b8a10 tx=0x7f1c1c009260 comp rx=0 tx=0).stop 2024-09-19T08:15:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.051+0000 7f1c356b5740 1 -- 172.21.15.22:0/4182725110 shutdown_connections 2024-09-19T08:15:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.051+0000 7f1c356b5740 1 --2- 172.21.15.22:0/4182725110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752f4bb0 0x556675304d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.051+0000 7f1c356b5740 1 --2- 172.21.15.22:0/4182725110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752ea2f0 0x556675306180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.051+0000 7f1c356b5740 1 --2- 172.21.15.22:0/4182725110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752e99d0 0x5566753053e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.051+0000 7f1c356b5740 1 -- 172.21.15.22:0/4182725110 >> 172.21.15.22:0/4182725110 conn(0x5566752e5f60 msgr2=0x5566752f16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.051+0000 7f1c356b5740 1 -- 172.21.15.22:0/4182725110 shutdown_connections 2024-09-19T08:15:27.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.051+0000 7f1c356b5740 1 -- 172.21.15.22:0/4182725110 wait complete. 2024-09-19T08:15:27.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.051+0000 7f1c356b5740 1 -- start start 2024-09-19T08:15:27.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c356b5740 1 Processor -- start 2024-09-19T08:15:27.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c356b5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752e99d0 0x556675309470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c356b5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752ea2f0 0x5566753099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c356b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752f4bb0 0x55667530e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c356b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5566753066d0 con 0x5566752f4bb0 2024-09-19T08:15:27.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c356b5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556675306000 con 0x5566752ea2f0 2024-09-19T08:15:27.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c356b5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556675306da0 con 0x5566752e99d0 2024-09-19T08:15:27.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c356b5740 10 client.0 _ll_register_callbacks cb 0x7f1c24010dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:27.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c31879640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752f4bb0 0x55667530e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c31879640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752f4bb0 0x55667530e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58722/0 (socket says 172.21.15.22:58722) 2024-09-19T08:15:27.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c31879640 1 -- 172.21.15.22:0/2326041019 learned_addr learned my addr 172.21.15.22:0/2326041019 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:27.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c30877640 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752ea2f0 0x5566753099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c31078640 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752e99d0 0x556675309470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c31879640 1 -- 172.21.15.22:0/2326041019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752e99d0 msgr2=0x556675309470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:27.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c31879640 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752e99d0 0x556675309470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c31879640 1 -- 172.21.15.22:0/2326041019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752ea2f0 msgr2=0x5566753099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:27.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c31879640 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752ea2f0 0x5566753099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c31879640 1 -- 172.21.15.22:0/2326041019 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55667530f180 con 0x5566752f4bb0 2024-09-19T08:15:27.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c30877640 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752ea2f0 0x5566753099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:27.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.052+0000 7f1c31078640 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752e99d0 0x556675309470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:27.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.053+0000 7f1c31879640 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752f4bb0 0x55667530e950 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005b40 tx=0x7f1c24005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993aed6bf2894df0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:27.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.053+0000 7f1c10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:27.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.053+0000 7f1c10ff9640 1 -- 172.21.15.22:0/2326041019 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2403a020 con 0x5566752f4bb0 2024-09-19T08:15:27.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.053+0000 7f1c10ff9640 1 -- 172.21.15.22:0/2326041019 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1c24004ec0 con 0x5566752f4bb0 2024-09-19T08:15:27.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.053+0000 7f1c356b5740 1 -- 172.21.15.22:0/2326041019 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c1c0a5030 con 0x5566752f4bb0 2024-09-19T08:15:27.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.053+0000 7f1c10ff9640 1 -- 172.21.15.22:0/2326041019 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c24025040 con 0x5566752f4bb0 2024-09-19T08:15:27.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.053+0000 7f1c356b5740 10 client.15408 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:27.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.053+0000 7f1c356b5740 1 -- 172.21.15.22:0/2326041019 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55667536e600 con 0x5566752f4bb0 2024-09-19T08:15:27.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.053+0000 7f1c10ff9640 1 -- 172.21.15.22:0/2326041019 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c0a5030 con 0x5566752f4bb0 2024-09-19T08:15:27.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.053+0000 7f1c10ff9640 1 -- 172.21.15.22:0/2326041019 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1c2401fbd0 con 0x5566752f4bb0 2024-09-19T08:15:27.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.053+0000 7f1c356b5740 10 client.15408 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:27.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.054+0000 7f1c356b5740 10 client.15408 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:15:27.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.054+0000 7f1c356b5740 20 client.15408 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:27.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.054+0000 7f1c356b5740 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5566753730d0 0x556675371de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:27.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.054+0000 7f1c356b5740 4 client.15408 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:15:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.054+0000 7f1c356b5740 1 -- 172.21.15.22:0/2326041019 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "status"}) -- 0x556675257b10 con 0x5566753730d0 2024-09-19T08:15:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.054+0000 7f1c31078640 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5566753730d0 0x556675371de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.054+0000 7f1c31078640 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5566753730d0 0x556675371de0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59195efabcf1478c server_cookie=ea457606437cf22c in_seq=0 out_seq=0 2024-09-19T08:15:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.054+0000 7f1c10ff9640 10 client.15408 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:15:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.054+0000 7f1c10ff9640 1 -- 172.21.15.22:0/2326041019 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x556675257b10 con 0x5566753730d0 2024-09-19T08:15:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.054+0000 7f1c10ff9640 10 client.15408 handle_command_reply: tid=0 2024-09-19T08:15:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.054+0000 7f1c356b5740 1 client.15408 shutdown 2024-09-19T08:15:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.055+0000 7f1c356b5740 1 -- 172.21.15.22:0/2326041019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752f4bb0 msgr2=0x55667530e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.055+0000 7f1c356b5740 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752f4bb0 0x55667530e950 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005b40 tx=0x7f1c24005a20 comp rx=0 tx=0).stop 2024-09-19T08:15:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.055+0000 7f1c356b5740 1 -- 172.21.15.22:0/2326041019 shutdown_connections 2024-09-19T08:15:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.055+0000 7f1c356b5740 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566752f4bb0 0x55667530e950 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.055+0000 7f1c356b5740 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566752ea2f0 0x5566753099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.055+0000 7f1c356b5740 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5566753730d0 0x556675371de0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.055+0000 7f1c356b5740 1 --2- 172.21.15.22:0/2326041019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566752e99d0 0x556675309470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.055+0000 7f1c356b5740 1 -- 172.21.15.22:0/2326041019 >> 172.21.15.22:0/2326041019 conn(0x5566752e5f60 msgr2=0x5566752efe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.055+0000 7f1c10ff9640 0 client.15408 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:15:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.055+0000 7f1c356b5740 1 -- 172.21.15.22:0/2326041019 shutdown_connections 2024-09-19T08:15:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.055+0000 7f1c356b5740 1 -- 172.21.15.22:0/2326041019 wait complete. 2024-09-19T08:15:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.055+0000 7f1c356b5740 20 client.15408 trim_cache size 0 max 16384 2024-09-19T08:15:27.060 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:15:27.060 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "c0ad60c6-765d-11ef-bceb-c7b262605968", 2024-09-19T08:15:27.060 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 2, 2024-09-19T08:15:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24460, 2024-09-19T08:15:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-19T08:15:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-19T08:15:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-19T08:15:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 136.18302201, 2024-09-19T08:15:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 25, 2024-09-19T08:15:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 86, 2024-09-19T08:15:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 85, 2024-09-19T08:15:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 170.29216775699999 2024-09-19T08:15:27.061 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:15:27.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.056+0000 7f1c332dc640 1 -- 172.21.15.22:0/2612368596 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f1c140a34d0 msgr2=0x7f1c140a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:27.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.056+0000 7f1c332dc640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f1c140a34d0 0x7f1c140a5990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c24007310 tx=0x7f1c240072a0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.056+0000 7f1c332dc640 1 -- 172.21.15.22:0/2612368596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c102d90 msgr2=0x7f1c2c1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:27.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.056+0000 7f1c332dc640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c102d90 0x7f1c2c1009f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009230 tx=0x7f1c1c004270 comp rx=0 tx=0).stop 2024-09-19T08:15:27.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.056+0000 7f1c332dc640 1 -- 172.21.15.22:0/2612368596 shutdown_connections 2024-09-19T08:15:27.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.056+0000 7f1c332dc640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f1c140a34d0 0x7f1c140a5990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.056+0000 7f1c332dc640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c108290 0x7f1c2c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.056+0000 7f1c332dc640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c2c103740 0x7f1c2c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.056+0000 7f1c332dc640 1 --2- 172.21.15.22:0/2612368596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c102d90 0x7f1c2c1009f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:27.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.056+0000 7f1c332dc640 1 -- 172.21.15.22:0/2612368596 >> 172.21.15.22:0/2612368596 conn(0x7f1c2c0fc7d0 msgr2=0x7f1c2c105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:27.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.056+0000 7f1c332dc640 1 -- 172.21.15.22:0/2612368596 shutdown_connections 2024-09-19T08:15:27.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:27.056+0000 7f1c332dc640 1 -- 172.21.15.22:0/2612368596 wait complete. 2024-09-19T08:15:27.764 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T08:15:28.034 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:28.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.532+0000 7f6421ba5640 1 -- 172.21.15.22:0/3887170196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f641c103700 msgr2=0x7f641c10bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:28.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.532+0000 7f6421ba5640 1 --2- 172.21.15.22:0/3887170196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f641c103700 0x7f641c10bbf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6410009900 tx=0x7f641002a480 comp rx=0 tx=0).stop 2024-09-19T08:15:28.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.532+0000 7f6421ba5640 1 -- 172.21.15.22:0/3887170196 shutdown_connections 2024-09-19T08:15:28.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.532+0000 7f6421ba5640 1 --2- 172.21.15.22:0/3887170196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f641c10c180 0x7f641c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.532+0000 7f6421ba5640 1 --2- 172.21.15.22:0/3887170196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f641c103700 0x7f641c10bbf0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.532+0000 7f6421ba5640 1 --2- 172.21.15.22:0/3887170196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c102de0 0x7f641c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.532+0000 7f6421ba5640 1 -- 172.21.15.22:0/3887170196 >> 172.21.15.22:0/3887170196 conn(0x7f641c0fc7a0 msgr2=0x7f641c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:28.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.533+0000 7f6421ba5640 1 -- 172.21.15.22:0/3887170196 shutdown_connections 2024-09-19T08:15:28.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.533+0000 7f6421ba5640 1 -- 172.21.15.22:0/3887170196 wait complete. 2024-09-19T08:15:28.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.533+0000 7f6421ba5640 1 Processor -- start 2024-09-19T08:15:28.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.533+0000 7f6421ba5640 1 -- start start 2024-09-19T08:15:28.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f6421ba5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c102de0 0x7f641c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f6421ba5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f641c103700 0x7f641c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f6421ba5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f641c10c180 0x7f641c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f6421ba5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f641c111f30 con 0x7f641c103700 2024-09-19T08:15:28.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f6421ba5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f641c111db0 con 0x7f641c10c180 2024-09-19T08:15:28.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f6421ba5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1120b0 con 0x7f641c102de0 2024-09-19T08:15:28.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f641affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f641c103700 0x7f641c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f641affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f641c103700 0x7f641c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33616/0 (socket says 172.21.15.22:33616) 2024-09-19T08:15:28.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f641affd640 1 -- 172.21.15.22:0/2332753945 learned_addr learned my addr 172.21.15.22:0/2332753945 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:28.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f641b7fe640 1 --2- 172.21.15.22:0/2332753945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c102de0 0x7f641c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f641bfff640 1 --2- 172.21.15.22:0/2332753945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f641c10c180 0x7f641c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f641affd640 1 -- 172.21.15.22:0/2332753945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c102de0 msgr2=0x7f641c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:28.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f641affd640 1 --2- 172.21.15.22:0/2332753945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c102de0 0x7f641c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f641affd640 1 -- 172.21.15.22:0/2332753945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f641c10c180 msgr2=0x7f641c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:28.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f641affd640 1 --2- 172.21.15.22:0/2332753945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f641c10c180 0x7f641c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f641affd640 1 -- 172.21.15.22:0/2332753945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c19d3a0 con 0x7f641c103700 2024-09-19T08:15:28.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.534+0000 7f641affd640 1 --2- 172.21.15.22:0/2332753945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f641c103700 0x7f641c198930 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6410006010 tx=0x7f64100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:28.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.535+0000 7f6418ff9640 1 -- 172.21.15.22:0/2332753945 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641003f030 con 0x7f641c103700 2024-09-19T08:15:28.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.535+0000 7f6418ff9640 1 -- 172.21.15.22:0/2332753945 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6410003ce0 con 0x7f641c103700 2024-09-19T08:15:28.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.535+0000 7f6421ba5640 1 -- 172.21.15.22:0/2332753945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641c19d630 con 0x7f641c103700 2024-09-19T08:15:28.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.535+0000 7f6418ff9640 1 -- 172.21.15.22:0/2332753945 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410035070 con 0x7f641c103700 2024-09-19T08:15:28.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.535+0000 7f6421ba5640 1 -- 172.21.15.22:0/2332753945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f641c1a4f70 con 0x7f641c103700 2024-09-19T08:15:28.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.537+0000 7f6418ff9640 1 -- 172.21.15.22:0/2332753945 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f6410042030 con 0x7f641c103700 2024-09-19T08:15:28.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.537+0000 7f6423f7e740 1 Processor -- start 2024-09-19T08:15:28.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.537+0000 7f6423f7e740 1 -- start start 2024-09-19T08:15:28.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f6423f7e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 0x55ad58e04db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f6423f7e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e052f0 0x55ad58e0f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f6423f7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e0fbb0 0x55ad58e11fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f6423f7e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ad58d6b090 con 0x55ad58e052f0 2024-09-19T08:15:28.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f6423f7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ad58d12a70 con 0x55ad58e0fbb0 2024-09-19T08:15:28.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f6423f7e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ad58d67050 con 0x55ad58e049d0 2024-09-19T08:15:28.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f6418ff9640 1 --2- 172.21.15.22:0/2332753945 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f64040a3550 0x7f64040a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f641b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e052f0 0x55ad58e0f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f641b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e052f0 0x55ad58e0f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33618/0 (socket says 172.21.15.22:33618) 2024-09-19T08:15:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f641b7fe640 1 -- 172.21.15.22:0/2163546849 learned_addr learned my addr 172.21.15.22:0/2163546849 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f641bfff640 1 --2- 172.21.15.22:0/2332753945 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f64040a3550 0x7f64040a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f641bfff640 1 --2- 172.21.15.22:0/2163546849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 0x55ad58e04db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f6418ff9640 1 -- 172.21.15.22:0/2332753945 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f64100f0220 con 0x7f641c103700 2024-09-19T08:15:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f641affd640 1 --2- 172.21.15.22:0/2163546849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e0fbb0 0x55ad58e11fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f641b7fe640 1 -- 172.21.15.22:0/2163546849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 msgr2=0x55ad58e04db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f641b7fe640 1 --2- 172.21.15.22:0/2163546849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 0x55ad58e04db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f641b7fe640 1 -- 172.21.15.22:0/2163546849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e0fbb0 msgr2=0x55ad58e11fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f641b7fe640 1 --2- 172.21.15.22:0/2163546849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e0fbb0 0x55ad58e11fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f641b7fe640 1 -- 172.21.15.22:0/2163546849 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad58d1f130 con 0x55ad58e052f0 2024-09-19T08:15:28.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f641b7fe640 1 --2- 172.21.15.22:0/2163546849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e052f0 0x55ad58e0f670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f640c00ad00 tx=0x7f640c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390a5480463aec74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:28.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.538+0000 7f641bfff640 1 --2- 172.21.15.22:0/2332753945 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f64040a3550 0x7f64040a5a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6414009fd0 tx=0x7f6414009840 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:28.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.539+0000 7f6401ffb640 1 -- 172.21.15.22:0/2163546849 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c01d070 con 0x55ad58e052f0 2024-09-19T08:15:28.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.539+0000 7f6401ffb640 1 -- 172.21.15.22:0/2163546849 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f640c018b20 con 0x55ad58e052f0 2024-09-19T08:15:28.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.539+0000 7f6401ffb640 1 -- 172.21.15.22:0/2163546849 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c010bc0 con 0x55ad58e052f0 2024-09-19T08:15:28.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.539+0000 7f6423f7e740 1 -- 172.21.15.22:0/2163546849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e052f0 msgr2=0x55ad58e0f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:28.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.539+0000 7f6423f7e740 1 --2- 172.21.15.22:0/2163546849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e052f0 0x55ad58e0f670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f640c00ad00 tx=0x7f640c00f790 comp rx=0 tx=0).stop 2024-09-19T08:15:28.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.539+0000 7f6423f7e740 1 -- 172.21.15.22:0/2163546849 shutdown_connections 2024-09-19T08:15:28.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.539+0000 7f6423f7e740 1 --2- 172.21.15.22:0/2163546849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e0fbb0 0x55ad58e11fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.539+0000 7f6423f7e740 1 --2- 172.21.15.22:0/2163546849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e052f0 0x55ad58e0f670 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.539+0000 7f6423f7e740 1 --2- 172.21.15.22:0/2163546849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 0x55ad58e04db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.539+0000 7f6423f7e740 1 -- 172.21.15.22:0/2163546849 >> 172.21.15.22:0/2163546849 conn(0x55ad58e00f60 msgr2=0x55ad58e01340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.539+0000 7f6423f7e740 1 -- 172.21.15.22:0/2163546849 shutdown_connections 2024-09-19T08:15:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.539+0000 7f6423f7e740 1 -- 172.21.15.22:0/2163546849 wait complete. 2024-09-19T08:15:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.540+0000 7f6423f7e740 1 -- start start 2024-09-19T08:15:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.540+0000 7f6423f7e740 1 Processor -- start 2024-09-19T08:15:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.540+0000 7f6423f7e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e049d0 0x55ad58e0f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.540+0000 7f6423f7e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e052f0 0x55ad58e0c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.540+0000 7f6423f7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e0fbb0 0x55ad58e0cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.540+0000 7f6423f7e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ad58e12780 con 0x55ad58e049d0 2024-09-19T08:15:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.540+0000 7f6423f7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ad58d25f20 con 0x55ad58e0fbb0 2024-09-19T08:15:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.540+0000 7f6423f7e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ad58e12900 con 0x55ad58e052f0 2024-09-19T08:15:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.540+0000 7f6423f7e740 10 client.0 _ll_register_callbacks cb 0x55ad58cef030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.540+0000 7f641b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e049d0 0x55ad58e0f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.540+0000 7f641b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e049d0 0x55ad58e0f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33626/0 (socket says 172.21.15.22:33626) 2024-09-19T08:15:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.540+0000 7f641b7fe640 1 -- 172.21.15.22:0/739205523 learned_addr learned my addr 172.21.15.22:0/739205523 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f641bfff640 1 --2- 172.21.15.22:0/739205523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e0fbb0 0x55ad58e0cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f641affd640 1 --2- 172.21.15.22:0/739205523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e052f0 0x55ad58e0c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f641b7fe640 1 -- 172.21.15.22:0/739205523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e052f0 msgr2=0x55ad58e0c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f641b7fe640 1 --2- 172.21.15.22:0/739205523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e052f0 0x55ad58e0c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f641b7fe640 1 -- 172.21.15.22:0/739205523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e0fbb0 msgr2=0x55ad58e0cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f641b7fe640 1 --2- 172.21.15.22:0/739205523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e0fbb0 0x55ad58e0cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f641b7fe640 1 -- 172.21.15.22:0/739205523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad58adc2b0 con 0x55ad58e049d0 2024-09-19T08:15:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f641affd640 1 --2- 172.21.15.22:0/739205523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e052f0 0x55ad58e0c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:15:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f641b7fe640 1 --2- 172.21.15.22:0/739205523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e049d0 0x55ad58e0f3e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f640c00a550 tx=0x7f640c0186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e9ca26560be08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f63faffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f63faffd640 1 -- 172.21.15.22:0/739205523 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c01d040 con 0x55ad58e049d0 2024-09-19T08:15:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f63faffd640 1 -- 172.21.15.22:0/739205523 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f640c0108e0 con 0x55ad58e049d0 2024-09-19T08:15:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f6423f7e740 1 -- 172.21.15.22:0/739205523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f640c018b20 con 0x55ad58e049d0 2024-09-19T08:15:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f63faffd640 1 -- 172.21.15.22:0/739205523 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f640c031050 con 0x55ad58e049d0 2024-09-19T08:15:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f6423f7e740 10 client.15420 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f6423f7e740 1 -- 172.21.15.22:0/739205523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ad58e1e030 con 0x55ad58e049d0 2024-09-19T08:15:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.541+0000 7f63faffd640 1 -- 172.21.15.22:0/739205523 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f640c018b20 con 0x55ad58e049d0 2024-09-19T08:15:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.542+0000 7f63faffd640 1 -- 172.21.15.22:0/739205523 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f640c020030 con 0x55ad58e049d0 2024-09-19T08:15:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.542+0000 7f6423f7e740 10 client.15420 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.542+0000 7f6423f7e740 10 client.15420 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T08:15:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.542+0000 7f6423f7e740 20 client.15420 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.542+0000 7f6423f7e740 1 --2- 172.21.15.22:0/739205523 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55ad58e202b0 0x55ad58e21720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.542+0000 7f6423f7e740 4 client.15420 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:15:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.542+0000 7f6423f7e740 1 -- 172.21.15.22:0/739205523 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad58d78600 con 0x55ad58e202b0 2024-09-19T08:15:28.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.542+0000 7f641affd640 1 --2- 172.21.15.22:0/739205523 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55ad58e202b0 0x55ad58e21720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.542+0000 7f641affd640 1 --2- 172.21.15.22:0/739205523 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55ad58e202b0 0x55ad58e21720 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5aad1d7fcc246a9c server_cookie=ae4b592470683206 in_seq=0 out_seq=0 2024-09-19T08:15:28.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.542+0000 7f63faffd640 10 client.15420 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:15:28.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.543+0000 7f63faffd640 1 -- 172.21.15.22:0/739205523 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad58d78600 con 0x55ad58e202b0 2024-09-19T08:15:28.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.543+0000 7f63faffd640 10 client.15420 handle_command_reply: tid=0 2024-09-19T08:15:28.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.543+0000 7f6423f7e740 1 client.15420 shutdown 2024-09-19T08:15:28.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.543+0000 7f6423f7e740 1 -- 172.21.15.22:0/739205523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e049d0 msgr2=0x55ad58e0f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:28.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.543+0000 7f6423f7e740 1 --2- 172.21.15.22:0/739205523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e049d0 0x55ad58e0f3e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f640c00a550 tx=0x7f640c0186f0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.543+0000 7f6423f7e740 1 -- 172.21.15.22:0/739205523 shutdown_connections 2024-09-19T08:15:28.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.543+0000 7f6423f7e740 1 --2- 172.21.15.22:0/739205523 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55ad58e202b0 0x55ad58e21720 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.543+0000 7f6423f7e740 1 --2- 172.21.15.22:0/739205523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e0fbb0 0x55ad58e0cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.543+0000 7f6423f7e740 1 --2- 172.21.15.22:0/739205523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e052f0 0x55ad58e0c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.543+0000 7f6423f7e740 1 --2- 172.21.15.22:0/739205523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e049d0 0x55ad58e0f3e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.543+0000 7f63faffd640 0 client.15420 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:15:28.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.543+0000 7f6423f7e740 1 -- 172.21.15.22:0/739205523 >> 172.21.15.22:0/739205523 conn(0x55ad58e00f60 msgr2=0x55ad58e06c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:28.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.544+0000 7f6423f7e740 1 -- 172.21.15.22:0/739205523 shutdown_connections 2024-09-19T08:15:28.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.544+0000 7f6423f7e740 1 -- 172.21.15.22:0/739205523 wait complete. 2024-09-19T08:15:28.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.544+0000 7f6423f7e740 20 client.15420 trim_cache size 0 max 16384 2024-09-19T08:15:28.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.562+0000 7f6423f7e740 1 Processor -- start 2024-09-19T08:15:28.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.562+0000 7f6423f7e740 1 -- start start 2024-09-19T08:15:28.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.562+0000 7f6423f7e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 0x55ad58e21720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.562+0000 7f6423f7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e052f0 0x55ad58e29a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.562+0000 7f6423f7e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e0fbb0 0x55ad58e29f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.562+0000 7f6423f7e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ad58e1fa60 con 0x55ad58e0fbb0 2024-09-19T08:15:28.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.562+0000 7f6423f7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ad58e20130 con 0x55ad58e052f0 2024-09-19T08:15:28.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.562+0000 7f6423f7e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ad58e20800 con 0x55ad58e049d0 2024-09-19T08:15:28.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.562+0000 7f641bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e0fbb0 0x55ad58e29f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.562+0000 7f641bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e0fbb0 0x55ad58e29f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33634/0 (socket says 172.21.15.22:33634) 2024-09-19T08:15:28.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.562+0000 7f641bfff640 1 -- 172.21.15.22:0/3380816037 learned_addr learned my addr 172.21.15.22:0/3380816037 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:28.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.562+0000 7f641b7fe640 1 --2- 172.21.15.22:0/3380816037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 0x55ad58e21720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.563+0000 7f641affd640 1 --2- 172.21.15.22:0/3380816037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e052f0 0x55ad58e29a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.563+0000 7f641bfff640 1 -- 172.21.15.22:0/3380816037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 msgr2=0x55ad58e21720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:28.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.563+0000 7f641bfff640 1 --2- 172.21.15.22:0/3380816037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 0x55ad58e21720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.563+0000 7f641bfff640 1 -- 172.21.15.22:0/3380816037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e052f0 msgr2=0x55ad58e29a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:28.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.563+0000 7f641bfff640 1 --2- 172.21.15.22:0/3380816037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e052f0 0x55ad58e29a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.563+0000 7f641bfff640 1 -- 172.21.15.22:0/3380816037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad58e0f750 con 0x55ad58e0fbb0 2024-09-19T08:15:28.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.563+0000 7f641affd640 1 --2- 172.21.15.22:0/3380816037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e052f0 0x55ad58e29a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:28.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.563+0000 7f641bfff640 1 --2- 172.21.15.22:0/3380816037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e0fbb0 0x55ad58e29f50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6414012f30 tx=0x7f6414009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5124507c89b05351 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:28.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.563+0000 7f64017fa640 1 -- 172.21.15.22:0/3380816037 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641402c480 con 0x55ad58e0fbb0 2024-09-19T08:15:28.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.563+0000 7f64017fa640 1 -- 172.21.15.22:0/3380816037 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f641402cb00 con 0x55ad58e0fbb0 2024-09-19T08:15:28.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.563+0000 7f64017fa640 1 -- 172.21.15.22:0/3380816037 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414021a80 con 0x55ad58e0fbb0 2024-09-19T08:15:28.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.564+0000 7f6423f7e740 1 -- 172.21.15.22:0/3380816037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e0fbb0 msgr2=0x55ad58e29f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:28.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.564+0000 7f6423f7e740 1 --2- 172.21.15.22:0/3380816037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e0fbb0 0x55ad58e29f50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6414012f30 tx=0x7f6414009600 comp rx=0 tx=0).stop 2024-09-19T08:15:28.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.564+0000 7f6423f7e740 1 -- 172.21.15.22:0/3380816037 shutdown_connections 2024-09-19T08:15:28.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.564+0000 7f6423f7e740 1 --2- 172.21.15.22:0/3380816037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e0fbb0 0x55ad58e29f50 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.564+0000 7f6423f7e740 1 --2- 172.21.15.22:0/3380816037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e052f0 0x55ad58e29a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.564+0000 7f6423f7e740 1 --2- 172.21.15.22:0/3380816037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 0x55ad58e21720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.564+0000 7f6423f7e740 1 -- 172.21.15.22:0/3380816037 >> 172.21.15.22:0/3380816037 conn(0x55ad58e00f60 msgr2=0x55ad58dffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:28.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.564+0000 7f6423f7e740 1 -- 172.21.15.22:0/3380816037 shutdown_connections 2024-09-19T08:15:28.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.564+0000 7f6423f7e740 1 -- 172.21.15.22:0/3380816037 wait complete. 2024-09-19T08:15:28.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.564+0000 7f6423f7e740 1 -- start start 2024-09-19T08:15:28.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f6423f7e740 1 Processor -- start 2024-09-19T08:15:28.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f6423f7e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 0x55ad58e26f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f6423f7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e052f0 0x55ad58e27440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f6423f7e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e0fbb0 0x55ad58e889c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f6423f7e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ad58e215a0 con 0x55ad58e0fbb0 2024-09-19T08:15:28.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f6423f7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ad58e20ed0 con 0x55ad58e052f0 2024-09-19T08:15:28.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f6423f7e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ad58e21c70 con 0x55ad58e049d0 2024-09-19T08:15:28.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f6423f7e740 10 client.0 _ll_register_callbacks cb 0x7f640c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f641bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e0fbb0 0x55ad58e889c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f641bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e0fbb0 0x55ad58e889c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33646/0 (socket says 172.21.15.22:33646) 2024-09-19T08:15:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f641bfff640 1 -- 172.21.15.22:0/1878356272 learned_addr learned my addr 172.21.15.22:0/1878356272 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f641b7fe640 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 0x55ad58e26f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f641affd640 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e052f0 0x55ad58e27440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f641bfff640 1 -- 172.21.15.22:0/1878356272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 msgr2=0x55ad58e26f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f641bfff640 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 0x55ad58e26f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f641bfff640 1 -- 172.21.15.22:0/1878356272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e052f0 msgr2=0x55ad58e27440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f641bfff640 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e052f0 0x55ad58e27440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f641bfff640 1 -- 172.21.15.22:0/1878356272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad58e88fb0 con 0x55ad58e0fbb0 2024-09-19T08:15:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f641b7fe640 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 0x55ad58e26f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.565+0000 7f641affd640 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e052f0 0x55ad58e27440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f641bfff640 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e0fbb0 0x55ad58e889c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6414010c30 tx=0x7f641402ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba5e5d1629007aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f63faffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f63faffd640 1 -- 172.21.15.22:0/1878356272 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414039060 con 0x55ad58e0fbb0 2024-09-19T08:15:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f63faffd640 1 -- 172.21.15.22:0/1878356272 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f641402c480 con 0x55ad58e0fbb0 2024-09-19T08:15:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f6423f7e740 1 -- 172.21.15.22:0/1878356272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f641402cb00 con 0x55ad58e0fbb0 2024-09-19T08:15:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f63faffd640 1 -- 172.21.15.22:0/1878356272 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6414004e50 con 0x55ad58e0fbb0 2024-09-19T08:15:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f6423f7e740 10 client.15426 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f6423f7e740 1 -- 172.21.15.22:0/1878356272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ad58e8a270 con 0x55ad58e0fbb0 2024-09-19T08:15:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f63faffd640 1 -- 172.21.15.22:0/1878356272 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f641402cb00 con 0x55ad58e0fbb0 2024-09-19T08:15:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f63faffd640 1 -- 172.21.15.22:0/1878356272 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6414033030 con 0x55ad58e0fbb0 2024-09-19T08:15:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f6423f7e740 10 client.15426 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f6423f7e740 10 client.15426 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T08:15:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f6423f7e740 20 client.15426 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f6423f7e740 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55ad58e8e9e0 0x55ad58e8d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f6423f7e740 4 client.15426 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:15:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.566+0000 7f6423f7e740 1 -- 172.21.15.22:0/1878356272 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "status"}) -- 0x55ad58d72b10 con 0x55ad58e8e9e0 2024-09-19T08:15:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.567+0000 7f641b7fe640 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55ad58e8e9e0 0x55ad58e8d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.567+0000 7f641b7fe640 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55ad58e8e9e0 0x55ad58e8d6f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0043823e9f61cae server_cookie=4d6be747988bc72e in_seq=0 out_seq=0 2024-09-19T08:15:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.567+0000 7f63faffd640 10 client.15426 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:15:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.567+0000 7f63faffd640 1 -- 172.21.15.22:0/1878356272 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ad58d72b10 con 0x55ad58e8e9e0 2024-09-19T08:15:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.567+0000 7f63faffd640 10 client.15426 handle_command_reply: tid=0 2024-09-19T08:15:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.567+0000 7f6423f7e740 1 client.15426 shutdown 2024-09-19T08:15:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.567+0000 7f6423f7e740 1 -- 172.21.15.22:0/1878356272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e0fbb0 msgr2=0x55ad58e889c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.567+0000 7f6423f7e740 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e0fbb0 0x55ad58e889c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6414010c30 tx=0x7f641402ccd0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6423f7e740 1 -- 172.21.15.22:0/1878356272 shutdown_connections 2024-09-19T08:15:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6423f7e740 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55ad58e8e9e0 0x55ad58e8d6f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6423f7e740 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad58e0fbb0 0x55ad58e889c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6423f7e740 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ad58e052f0 0x55ad58e27440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6423f7e740 1 --2- 172.21.15.22:0/1878356272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad58e049d0 0x55ad58e26f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f63faffd640 0 client.15426 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:15:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6423f7e740 1 -- 172.21.15.22:0/1878356272 >> 172.21.15.22:0/1878356272 conn(0x55ad58e00f60 msgr2=0x55ad58e1ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6423f7e740 1 -- 172.21.15.22:0/1878356272 shutdown_connections 2024-09-19T08:15:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6423f7e740 1 -- 172.21.15.22:0/1878356272 wait complete. 2024-09-19T08:15:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6423f7e740 20 client.15426 trim_cache size 0 max 16384 2024-09-19T08:15:28.574 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:15:28.574 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "c0ad60c6-765d-11ef-bceb-c7b262605968", 2024-09-19T08:15:28.574 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 1, 2024-09-19T08:15:28.574 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24479, 2024-09-19T08:15:28.574 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-19T08:15:28.574 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-19T08:15:28.574 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-19T08:15:28.574 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 138.70550246600001, 2024-09-19T08:15:28.574 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 25, 2024-09-19T08:15:28.574 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 86, 2024-09-19T08:15:28.574 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 85, 2024-09-19T08:15:28.575 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 157.92055918599999 2024-09-19T08:15:28.575 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:15:28.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6421ba5640 1 -- 172.21.15.22:0/2332753945 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f64040a3550 msgr2=0x7f64040a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:28.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6421ba5640 1 --2- 172.21.15.22:0/2332753945 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f64040a3550 0x7f64040a5a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6414009fd0 tx=0x7f6414009840 comp rx=0 tx=0).stop 2024-09-19T08:15:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6421ba5640 1 -- 172.21.15.22:0/2332753945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f641c103700 msgr2=0x7f641c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6421ba5640 1 --2- 172.21.15.22:0/2332753945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f641c103700 0x7f641c198930 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6410006010 tx=0x7f64100042f0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6421ba5640 1 -- 172.21.15.22:0/2332753945 shutdown_connections 2024-09-19T08:15:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6421ba5640 1 --2- 172.21.15.22:0/2332753945 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f64040a3550 0x7f64040a5a10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6421ba5640 1 --2- 172.21.15.22:0/2332753945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f641c10c180 0x7f641c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6421ba5640 1 --2- 172.21.15.22:0/2332753945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f641c103700 0x7f641c198930 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6421ba5640 1 --2- 172.21.15.22:0/2332753945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641c102de0 0x7f641c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.568+0000 7f6421ba5640 1 -- 172.21.15.22:0/2332753945 >> 172.21.15.22:0/2332753945 conn(0x7f641c0fc7a0 msgr2=0x7f641c0fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.569+0000 7f6421ba5640 1 -- 172.21.15.22:0/2332753945 shutdown_connections 2024-09-19T08:15:28.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:28.569+0000 7f6421ba5640 1 -- 172.21.15.22:0/2332753945 wait complete. 2024-09-19T08:15:28.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:28 smithi157 ceph-mon[35143]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T08:15:28.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:28 smithi022 ceph-mon[30963]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T08:15:28.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:28 smithi064 ceph-mon[34900]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T08:15:29.248 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T08:15:29.509 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:30.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.989+0000 7f5a6d2ca640 1 -- 172.21.15.22:0/3903430509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a68100d20 msgr2=0x7f5a68106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:30.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.989+0000 7f5a6d2ca640 1 --2- 172.21.15.22:0/3903430509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a68100d20 0x7f5a68106860 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009930 tx=0x7f5a5002a420 comp rx=0 tx=0).stop 2024-09-19T08:15:30.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.989+0000 7f5a6d2ca640 1 -- 172.21.15.22:0/3903430509 shutdown_connections 2024-09-19T08:15:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.989+0000 7f5a6d2ca640 1 --2- 172.21.15.22:0/3903430509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a68106fc0 0x7f5a6810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.989+0000 7f5a6d2ca640 1 --2- 172.21.15.22:0/3903430509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a68100d20 0x7f5a68106860 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.989+0000 7f5a6d2ca640 1 --2- 172.21.15.22:0/3903430509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a68100370 0x7f5a68100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.989+0000 7f5a6d2ca640 1 -- 172.21.15.22:0/3903430509 >> 172.21.15.22:0/3903430509 conn(0x7f5a68077210 msgr2=0x7f5a680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.989+0000 7f5a6d2ca640 1 -- 172.21.15.22:0/3903430509 shutdown_connections 2024-09-19T08:15:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.989+0000 7f5a6d2ca640 1 -- 172.21.15.22:0/3903430509 wait complete. 2024-09-19T08:15:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.990+0000 7f5a6d2ca640 1 Processor -- start 2024-09-19T08:15:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.990+0000 7f5a6d2ca640 1 -- start start 2024-09-19T08:15:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.990+0000 7f5a6d2ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a68100370 0x7f5a681012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.990+0000 7f5a6d2ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a68100d20 0x7f5a68101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.990+0000 7f5a6d2ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a68106fc0 0x7f5a68101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.990+0000 7f5a6d2ca640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68111fa0 con 0x7f5a68106fc0 2024-09-19T08:15:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.990+0000 7f5a6d2ca640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68111e20 con 0x7f5a68100370 2024-09-19T08:15:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.990+0000 7f5a6d2ca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68112120 con 0x7f5a68100d20 2024-09-19T08:15:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.990+0000 7f5a677fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a68106fc0 0x7f5a68101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.990+0000 7f5a677fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a68106fc0 0x7f5a68101d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33668/0 (socket says 172.21.15.22:33668) 2024-09-19T08:15:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.990+0000 7f5a677fe640 1 -- 172.21.15.22:0/278378938 learned_addr learned my addr 172.21.15.22:0/278378938 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.991+0000 7f5a66ffd640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a68100370 0x7f5a681012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.991+0000 7f5a667fc640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a68100d20 0x7f5a68101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.991+0000 7f5a677fe640 1 -- 172.21.15.22:0/278378938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a68100d20 msgr2=0x7f5a68101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.991+0000 7f5a677fe640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a68100d20 0x7f5a68101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.991+0000 7f5a677fe640 1 -- 172.21.15.22:0/278378938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a68100370 msgr2=0x7f5a681012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.991+0000 7f5a677fe640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a68100370 0x7f5a681012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.991+0000 7f5a677fe640 1 -- 172.21.15.22:0/278378938 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a681a6940 con 0x7f5a68106fc0 2024-09-19T08:15:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.991+0000 7f5a667fc640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a68100d20 0x7f5a68101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.991+0000 7f5a677fe640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a68106fc0 0x7f5a68101d70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000eb10 tx=0x7f5a6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.991+0000 7f5a5ffff640 1 -- 172.21.15.22:0/278378938 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60004b20 con 0x7f5a68106fc0 2024-09-19T08:15:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.991+0000 7f5a5ffff640 1 -- 172.21.15.22:0/278378938 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a60004cc0 con 0x7f5a68106fc0 2024-09-19T08:15:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.991+0000 7f5a6d2ca640 1 -- 172.21.15.22:0/278378938 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a681a6c30 con 0x7f5a68106fc0 2024-09-19T08:15:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.991+0000 7f5a5ffff640 1 -- 172.21.15.22:0/278378938 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60010760 con 0x7f5a68106fc0 2024-09-19T08:15:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.991+0000 7f5a6d2ca640 1 -- 172.21.15.22:0/278378938 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a681a70f0 con 0x7f5a68106fc0 2024-09-19T08:15:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.994+0000 7f5a5ffff640 1 -- 172.21.15.22:0/278378938 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f5a60005020 con 0x7f5a68106fc0 2024-09-19T08:15:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.994+0000 7f5a6f6a3740 1 Processor -- start 2024-09-19T08:15:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.994+0000 7f5a6f6a3740 1 -- start start 2024-09-19T08:15:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.994+0000 7f5a6f6a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107d9d0 0x5598e107ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.994+0000 7f5a6f6a3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107e2f0 0x5598e1088670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.994+0000 7f5a6f6a3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e1088bb0 0x5598e108afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.994+0000 7f5a6f6a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5598e0fe4090 con 0x5598e107d9d0 2024-09-19T08:15:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.994+0000 7f5a6f6a3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5598e0f8ba70 con 0x5598e107e2f0 2024-09-19T08:15:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.994+0000 7f5a6f6a3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5598e0fe0050 con 0x5598e1088bb0 2024-09-19T08:15:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.994+0000 7f5a667fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107d9d0 0x5598e107ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.994+0000 7f5a667fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107d9d0 0x5598e107ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33672/0 (socket says 172.21.15.22:33672) 2024-09-19T08:15:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.994+0000 7f5a667fc640 1 -- 172.21.15.22:0/1733733476 learned_addr learned my addr 172.21.15.22:0/1733733476 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.994+0000 7f5a66ffd640 1 --2- 172.21.15.22:0/1733733476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107e2f0 0x5598e1088670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.994+0000 7f5a667fc640 1 --2- 172.21.15.22:0/1733733476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e1088bb0 0x5598e108afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.995+0000 7f5a667fc640 1 -- 172.21.15.22:0/1733733476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e1088bb0 msgr2=0x5598e108afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.995+0000 7f5a667fc640 1 --2- 172.21.15.22:0/1733733476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e1088bb0 0x5598e108afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.995+0000 7f5a667fc640 1 -- 172.21.15.22:0/1733733476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107e2f0 msgr2=0x5598e1088670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.995+0000 7f5a667fc640 1 --2- 172.21.15.22:0/1733733476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107e2f0 0x5598e1088670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.995+0000 7f5a667fc640 1 -- 172.21.15.22:0/1733733476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598e0f98130 con 0x5598e107d9d0 2024-09-19T08:15:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.995+0000 7f5a5ffff640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5a540a34d0 0x7f5a540a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.995+0000 7f5a667fc640 1 --2- 172.21.15.22:0/1733733476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107d9d0 0x5598e107ddb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5a50005940 tx=0x7f5a50005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58221b12edeab3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.995+0000 7f5a66ffd640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5a540a34d0 0x7f5a540a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.995+0000 7f5a5ffff640 1 -- 172.21.15.22:0/278378938 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5a600d0560 con 0x7f5a68106fc0 2024-09-19T08:15:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.995+0000 7f5a5d7fa640 1 -- 172.21.15.22:0/1733733476 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a50035050 con 0x5598e107d9d0 2024-09-19T08:15:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.995+0000 7f5a5d7fa640 1 -- 172.21.15.22:0/1733733476 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a50031730 con 0x5598e107d9d0 2024-09-19T08:15:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.995+0000 7f5a5d7fa640 1 -- 172.21.15.22:0/1733733476 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a500046c0 con 0x5598e107d9d0 2024-09-19T08:15:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.996+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/1733733476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107d9d0 msgr2=0x5598e107ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.996+0000 7f5a66ffd640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5a540a34d0 0x7f5a540a5990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x5598e107adc0 tx=0x7f5a58005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.996+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/1733733476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107d9d0 0x5598e107ddb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5a50005940 tx=0x7f5a50005bd0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.996+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/1733733476 shutdown_connections 2024-09-19T08:15:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.996+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/1733733476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e1088bb0 0x5598e108afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.996+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/1733733476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107e2f0 0x5598e1088670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.996+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/1733733476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107d9d0 0x5598e107ddb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.996+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/1733733476 >> 172.21.15.22:0/1733733476 conn(0x5598e1079f60 msgr2=0x5598e107a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.996+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/1733733476 shutdown_connections 2024-09-19T08:15:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.996+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/1733733476 wait complete. 2024-09-19T08:15:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.996+0000 7f5a6f6a3740 1 -- start start 2024-09-19T08:15:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.996+0000 7f5a6f6a3740 1 Processor -- start 2024-09-19T08:15:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a6f6a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107d9d0 0x5598e10884a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a6f6a3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e107e2f0 0x5598e10857f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a6f6a3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e1088bb0 0x5598e1085d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a6f6a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5598e108b780 con 0x5598e107d9d0 2024-09-19T08:15:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a6f6a3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5598e0f9ef20 con 0x5598e1088bb0 2024-09-19T08:15:30.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a6f6a3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5598e108b900 con 0x5598e107e2f0 2024-09-19T08:15:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a6f6a3740 10 client.0 _ll_register_callbacks cb 0x5598e0f68030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a66ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107d9d0 0x5598e10884a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a66ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107d9d0 0x5598e10884a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33678/0 (socket says 172.21.15.22:33678) 2024-09-19T08:15:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a66ffd640 1 -- 172.21.15.22:0/953606547 learned_addr learned my addr 172.21.15.22:0/953606547 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a677fe640 1 --2- 172.21.15.22:0/953606547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e1088bb0 0x5598e1085d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a667fc640 1 --2- 172.21.15.22:0/953606547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e107e2f0 0x5598e10857f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a66ffd640 1 -- 172.21.15.22:0/953606547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e107e2f0 msgr2=0x5598e10857f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a66ffd640 1 --2- 172.21.15.22:0/953606547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e107e2f0 0x5598e10857f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a66ffd640 1 -- 172.21.15.22:0/953606547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e1088bb0 msgr2=0x5598e1085d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a66ffd640 1 --2- 172.21.15.22:0/953606547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e1088bb0 0x5598e1085d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a66ffd640 1 -- 172.21.15.22:0/953606547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598e0d552b0 con 0x5598e107d9d0 2024-09-19T08:15:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a677fe640 1 --2- 172.21.15.22:0/953606547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e1088bb0 0x5598e1085d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a66ffd640 1 --2- 172.21.15.22:0/953606547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107d9d0 0x5598e10884a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009730 tx=0x7f5a58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2886cf48b33f435 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a4e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a4e7fc640 1 -- 172.21.15.22:0/953606547 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5801f640 con 0x5598e107d9d0 2024-09-19T08:15:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a4e7fc640 1 -- 172.21.15.22:0/953606547 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5a5801f7e0 con 0x5598e107d9d0 2024-09-19T08:15:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.997+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/953606547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a50031730 con 0x5598e107d9d0 2024-09-19T08:15:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.998+0000 7f5a4e7fc640 1 -- 172.21.15.22:0/953606547 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5a5801f9e0 con 0x5598e107d9d0 2024-09-19T08:15:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.998+0000 7f5a6f6a3740 10 client.15438 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.998+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/953606547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598e1097150 con 0x5598e107d9d0 2024-09-19T08:15:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.998+0000 7f5a4e7fc640 1 -- 172.21.15.22:0/953606547 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a50031730 con 0x5598e107d9d0 2024-09-19T08:15:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.998+0000 7f5a4e7fc640 1 -- 172.21.15.22:0/953606547 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5a58027070 con 0x5598e107d9d0 2024-09-19T08:15:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.998+0000 7f5a6f6a3740 10 client.15438 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.998+0000 7f5a6f6a3740 10 client.15438 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T08:15:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.998+0000 7f5a6f6a3740 20 client.15438 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.998+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/953606547 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5598e10997e0 0x5598e109ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.998+0000 7f5a6f6a3740 4 client.15438 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:15:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.998+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/953606547 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598e0ff1600 con 0x5598e10997e0 2024-09-19T08:15:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.998+0000 7f5a667fc640 1 --2- 172.21.15.22:0/953606547 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5598e10997e0 0x5598e109ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.999+0000 7f5a4e7fc640 10 client.15438 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.999+0000 7f5a667fc640 1 --2- 172.21.15.22:0/953606547 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5598e10997e0 0x5598e109ac50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=484bdf4f39354a97 server_cookie=66142f2f55fc6189 in_seq=0 out_seq=0 2024-09-19T08:15:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.999+0000 7f5a4e7fc640 1 -- 172.21.15.22:0/953606547 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5598e0ff1600 con 0x5598e10997e0 2024-09-19T08:15:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.999+0000 7f5a4e7fc640 10 client.15438 handle_command_reply: tid=0 2024-09-19T08:15:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.999+0000 7f5a6f6a3740 1 client.15438 shutdown 2024-09-19T08:15:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:29.999+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/953606547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107d9d0 msgr2=0x5598e10884a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.000+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/953606547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107d9d0 0x5598e10884a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009730 tx=0x7f5a58004730 comp rx=0 tx=0).stop 2024-09-19T08:15:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.000+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/953606547 shutdown_connections 2024-09-19T08:15:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.000+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/953606547 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5598e10997e0 0x5598e109ac50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.000+0000 7f5a4e7fc640 0 client.15438 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.000+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/953606547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e1088bb0 0x5598e1085d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.000+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/953606547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e107e2f0 0x5598e10857f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.000+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/953606547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107d9d0 0x5598e10884a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.000+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/953606547 >> 172.21.15.22:0/953606547 conn(0x5598e1079f60 msgr2=0x5598e107fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.000+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/953606547 shutdown_connections 2024-09-19T08:15:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.000+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/953606547 wait complete. 2024-09-19T08:15:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.000+0000 7f5a6f6a3740 20 client.15438 trim_cache size 0 max 16384 2024-09-19T08:15:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.018+0000 7f5a6f6a3740 1 Processor -- start 2024-09-19T08:15:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.018+0000 7f5a6f6a3740 1 -- start start 2024-09-19T08:15:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.018+0000 7f5a6f6a3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e107d9d0 0x5598e109ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.018+0000 7f5a6f6a3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107e2f0 0x5598e1074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.018+0000 7f5a6f6a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e1088bb0 0x5598e1087650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.018+0000 7f5a6f6a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5598e1098f90 con 0x5598e1088bb0 2024-09-19T08:15:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.018+0000 7f5a6f6a3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5598e1099660 con 0x5598e107e2f0 2024-09-19T08:15:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.018+0000 7f5a6f6a3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5598e1099d30 con 0x5598e107d9d0 2024-09-19T08:15:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.018+0000 7f5a677fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e1088bb0 0x5598e1087650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.018+0000 7f5a677fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e1088bb0 0x5598e1087650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33684/0 (socket says 172.21.15.22:33684) 2024-09-19T08:15:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.018+0000 7f5a677fe640 1 -- 172.21.15.22:0/3612104979 learned_addr learned my addr 172.21.15.22:0/3612104979 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.019+0000 7f5a66ffd640 1 --2- 172.21.15.22:0/3612104979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e107d9d0 0x5598e109ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.019+0000 7f5a667fc640 1 --2- 172.21.15.22:0/3612104979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107e2f0 0x5598e1074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.019+0000 7f5a677fe640 1 -- 172.21.15.22:0/3612104979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e107d9d0 msgr2=0x5598e109ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.019+0000 7f5a677fe640 1 --2- 172.21.15.22:0/3612104979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e107d9d0 0x5598e109ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.019+0000 7f5a677fe640 1 -- 172.21.15.22:0/3612104979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107e2f0 msgr2=0x5598e1074ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.019+0000 7f5a677fe640 1 --2- 172.21.15.22:0/3612104979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107e2f0 0x5598e1074ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.019+0000 7f5a677fe640 1 -- 172.21.15.22:0/3612104979 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598e10847a0 con 0x5598e1088bb0 2024-09-19T08:15:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.019+0000 7f5a66ffd640 1 --2- 172.21.15.22:0/3612104979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e107d9d0 0x5598e109ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.019+0000 7f5a677fe640 1 --2- 172.21.15.22:0/3612104979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e1088bb0 0x5598e1087650 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5a6009c590 tx=0x7f5a6009c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8dbb7e5dea3b92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.019+0000 7f5a5cff9640 1 -- 172.21.15.22:0/3612104979 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a600b4070 con 0x5598e1088bb0 2024-09-19T08:15:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.019+0000 7f5a5cff9640 1 -- 172.21.15.22:0/3612104979 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a6009c600 con 0x5598e1088bb0 2024-09-19T08:15:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.019+0000 7f5a5cff9640 1 -- 172.21.15.22:0/3612104979 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60098040 con 0x5598e1088bb0 2024-09-19T08:15:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.020+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/3612104979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e1088bb0 msgr2=0x5598e1087650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.020+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/3612104979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e1088bb0 0x5598e1087650 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5a6009c590 tx=0x7f5a6009c490 comp rx=0 tx=0).stop 2024-09-19T08:15:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.020+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/3612104979 shutdown_connections 2024-09-19T08:15:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.020+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/3612104979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e1088bb0 0x5598e1087650 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.020+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/3612104979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107e2f0 0x5598e1074ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.020+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/3612104979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e107d9d0 0x5598e109ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.020+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/3612104979 >> 172.21.15.22:0/3612104979 conn(0x5598e1079f60 msgr2=0x5598e1086f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.020+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/3612104979 shutdown_connections 2024-09-19T08:15:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.020+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/3612104979 wait complete. 2024-09-19T08:15:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.020+0000 7f5a6f6a3740 1 -- start start 2024-09-19T08:15:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.020+0000 7f5a6f6a3740 1 Processor -- start 2024-09-19T08:15:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a6f6a3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107d9d0 0x5598e10a31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a6f6a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107e2f0 0x5598e10a3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a6f6a3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e1088bb0 0x5598e109e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a6f6a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5598e109aad0 con 0x5598e107e2f0 2024-09-19T08:15:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a6f6a3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5598e109a400 con 0x5598e107d9d0 2024-09-19T08:15:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a6f6a3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5598e109b1a0 con 0x5598e1088bb0 2024-09-19T08:15:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a6f6a3740 10 client.0 _ll_register_callbacks cb 0x7f5a58009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a667fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107e2f0 0x5598e10a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a667fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107e2f0 0x5598e10a3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33698/0 (socket says 172.21.15.22:33698) 2024-09-19T08:15:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a667fc640 1 -- 172.21.15.22:0/2435626897 learned_addr learned my addr 172.21.15.22:0/2435626897 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a677fe640 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e1088bb0 0x5598e109e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a66ffd640 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107d9d0 0x5598e10a31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a667fc640 1 -- 172.21.15.22:0/2435626897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e1088bb0 msgr2=0x5598e109e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a667fc640 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e1088bb0 0x5598e109e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a667fc640 1 -- 172.21.15.22:0/2435626897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107d9d0 msgr2=0x5598e10a31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a667fc640 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107d9d0 0x5598e10a31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a667fc640 1 -- 172.21.15.22:0/2435626897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598e109edb0 con 0x5598e107e2f0 2024-09-19T08:15:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a677fe640 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e1088bb0 0x5598e109e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a66ffd640 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107d9d0 0x5598e10a31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a667fc640 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107e2f0 0x5598e10a3730 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5a50005310 tx=0x7f5a500053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2be93cbb0e73eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a4e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a4e7fc640 1 -- 172.21.15.22:0/2435626897 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a50035040 con 0x5598e107e2f0 2024-09-19T08:15:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a4e7fc640 1 -- 172.21.15.22:0/2435626897 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5a50044040 con 0x5598e107e2f0 2024-09-19T08:15:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.021+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/2435626897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a6009c600 con 0x5598e107e2f0 2024-09-19T08:15:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.022+0000 7f5a4e7fc640 1 -- 172.21.15.22:0/2435626897 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5a50005620 con 0x5598e107e2f0 2024-09-19T08:15:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.022+0000 7f5a6f6a3740 10 client.15444 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.022+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/2435626897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598e1103a70 con 0x5598e107e2f0 2024-09-19T08:15:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.022+0000 7f5a4e7fc640 1 -- 172.21.15.22:0/2435626897 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a6009c600 con 0x5598e107e2f0 2024-09-19T08:15:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.022+0000 7f5a4e7fc640 1 -- 172.21.15.22:0/2435626897 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5a50003920 con 0x5598e107e2f0 2024-09-19T08:15:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.022+0000 7f5a6f6a3740 10 client.15444 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.022+0000 7f5a6f6a3740 10 client.15444 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T08:15:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.022+0000 7f5a6f6a3740 20 client.15444 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.022+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5598e1108120 0x5598e1106e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.022+0000 7f5a6f6a3740 4 client.15444 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:15:30.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.022+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/2435626897 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "status"}) -- 0x5598e0febb10 con 0x5598e1108120 2024-09-19T08:15:30.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.022+0000 7f5a66ffd640 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5598e1108120 0x5598e1106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:30.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a66ffd640 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5598e1108120 0x5598e1106e30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5833f8f4176c339a server_cookie=3055610da958ab47 in_seq=0 out_seq=0 2024-09-19T08:15:30.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a4e7fc640 10 client.15444 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:30.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a4e7fc640 1 -- 172.21.15.22:0/2435626897 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5598e0febb10 con 0x5598e1108120 2024-09-19T08:15:30.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a4e7fc640 10 client.15444 handle_command_reply: tid=0 2024-09-19T08:15:30.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a6f6a3740 1 client.15444 shutdown 2024-09-19T08:15:30.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/2435626897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107e2f0 msgr2=0x5598e10a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:30.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107e2f0 0x5598e10a3730 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5a50005310 tx=0x7f5a500053e0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/2435626897 shutdown_connections 2024-09-19T08:15:30.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5598e1108120 0x5598e1106e30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5598e1088bb0 0x5598e109e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598e107e2f0 0x5598e10a3730 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a6f6a3740 1 --2- 172.21.15.22:0/2435626897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598e107d9d0 0x5598e10a31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/2435626897 >> 172.21.15.22:0/2435626897 conn(0x5598e1079f60 msgr2=0x5598e1078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:30.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a4e7fc640 0 client.15444 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:30.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/2435626897 shutdown_connections 2024-09-19T08:15:30.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.023+0000 7f5a6f6a3740 1 -- 172.21.15.22:0/2435626897 wait complete. 2024-09-19T08:15:30.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.024+0000 7f5a6f6a3740 20 client.15444 trim_cache size 0 max 16384 2024-09-19T08:15:30.030 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:15:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "c0ad60c6-765d-11ef-bceb-c7b262605968", 2024-09-19T08:15:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-19T08:15:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24481, 2024-09-19T08:15:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-19T08:15:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-19T08:15:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-19T08:15:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 147.763449388, 2024-09-19T08:15:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 25, 2024-09-19T08:15:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 86, 2024-09-19T08:15:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 85, 2024-09-19T08:15:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 168.16396631699999 2024-09-19T08:15:30.031 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:15:30.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.024+0000 7f5a6d2ca640 1 -- 172.21.15.22:0/278378938 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5a540a34d0 msgr2=0x7f5a540a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:30.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.024+0000 7f5a6d2ca640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5a540a34d0 0x7f5a540a5990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x5598e107adc0 tx=0x7f5a58005f90 comp rx=0 tx=0).stop 2024-09-19T08:15:30.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.024+0000 7f5a6d2ca640 1 -- 172.21.15.22:0/278378938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a68106fc0 msgr2=0x7f5a68101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:30.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.024+0000 7f5a6d2ca640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a68106fc0 0x7f5a68101d70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000eb10 tx=0x7f5a6000c2d0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.024+0000 7f5a6d2ca640 1 -- 172.21.15.22:0/278378938 shutdown_connections 2024-09-19T08:15:30.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.024+0000 7f5a6d2ca640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5a540a34d0 0x7f5a540a5990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.024+0000 7f5a6d2ca640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a68106fc0 0x7f5a68101d70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.024+0000 7f5a6d2ca640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a68100d20 0x7f5a68101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.024+0000 7f5a6d2ca640 1 --2- 172.21.15.22:0/278378938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a68100370 0x7f5a681012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:30.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.024+0000 7f5a6d2ca640 1 -- 172.21.15.22:0/278378938 >> 172.21.15.22:0/278378938 conn(0x7f5a68077210 msgr2=0x7f5a681051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:30.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.024+0000 7f5a6d2ca640 1 -- 172.21.15.22:0/278378938 shutdown_connections 2024-09-19T08:15:30.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:30.024+0000 7f5a6d2ca640 1 -- 172.21.15.22:0/278378938 wait complete. 2024-09-19T08:15:30.717 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-19T08:15:30.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:30 smithi157 ceph-mon[35143]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-19T08:15:30.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:30 smithi022 ceph-mon[30963]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-19T08:15:30.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:30 smithi064 ceph-mon[34900]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-19T08:15:32.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:32 smithi157 ceph-mon[35143]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-19T08:15:32.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:32 smithi022 ceph-mon[30963]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-19T08:15:32.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:32 smithi064 ceph-mon[34900]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-19T08:15:34.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:34 smithi157 ceph-mon[35143]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T08:15:34.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:34 smithi022 ceph-mon[30963]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T08:15:34.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:34 smithi064 ceph-mon[34900]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T08:15:36.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:36 smithi157 ceph-mon[35143]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-19T08:15:36.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:36 smithi022 ceph-mon[30963]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-19T08:15:36.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:36 smithi064 ceph-mon[34900]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-19T08:15:38.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:38 smithi157 ceph-mon[35143]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-19T08:15:38.763 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:38 smithi022 ceph-mon[30963]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-19T08:15:38.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:38 smithi064 ceph-mon[34900]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-19T08:15:40.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:40 smithi157 ceph-mon[35143]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T08:15:40.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:40 smithi022 ceph-mon[30963]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T08:15:40.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:40 smithi064 ceph-mon[34900]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T08:15:42.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:42 smithi157 ceph-mon[35143]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:15:42.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:42 smithi022 ceph-mon[30963]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:15:42.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:42 smithi064 ceph-mon[34900]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:15:43.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:15:43.457 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:43.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.986+0000 7f9a28d02640 1 -- 172.21.15.22:0/1572812896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a24100a70 msgr2=0x7f9a24100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:43.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.986+0000 7f9a28d02640 1 --2- 172.21.15.22:0/1572812896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a24100a70 0x7f9a24100ed0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009930 tx=0x7f9a1802a420 comp rx=0 tx=0).stop 2024-09-19T08:15:43.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.986+0000 7f9a28d02640 1 -- 172.21.15.22:0/1572812896 shutdown_connections 2024-09-19T08:15:43.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.986+0000 7f9a28d02640 1 --2- 172.21.15.22:0/1572812896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a24101410 0x7f9a2410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:43.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.986+0000 7f9a28d02640 1 --2- 172.21.15.22:0/1572812896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a24100a70 0x7f9a24100ed0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:43.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.986+0000 7f9a28d02640 1 --2- 172.21.15.22:0/1572812896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24106a70 0x7f9a24106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:43.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.986+0000 7f9a28d02640 1 -- 172.21.15.22:0/1572812896 >> 172.21.15.22:0/1572812896 conn(0x7f9a240fc7d0 msgr2=0x7f9a240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:43.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.986+0000 7f9a28d02640 1 -- 172.21.15.22:0/1572812896 shutdown_connections 2024-09-19T08:15:43.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.986+0000 7f9a28d02640 1 -- 172.21.15.22:0/1572812896 wait complete. 2024-09-19T08:15:43.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.987+0000 7f9a28d02640 1 Processor -- start 2024-09-19T08:15:43.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.987+0000 7f9a28d02640 1 -- start start 2024-09-19T08:15:43.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.987+0000 7f9a28d02640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a24100a70 0x7f9a24198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:43.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.987+0000 7f9a28d02640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a24101410 0x7f9a24198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:43.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.987+0000 7f9a28d02640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24106a70 0x7f9a2419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:43.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.987+0000 7f9a28d02640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9a24110e90 con 0x7f9a24101410 2024-09-19T08:15:43.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.987+0000 7f9a28d02640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a24110d10 con 0x7f9a24106a70 2024-09-19T08:15:43.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.987+0000 7f9a28d02640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a24111010 con 0x7f9a24100a70 2024-09-19T08:15:43.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.987+0000 7f9a21d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a24101410 0x7f9a24198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:43.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.987+0000 7f9a21d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a24101410 0x7f9a24198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59976/0 (socket says 172.21.15.22:59976) 2024-09-19T08:15:43.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.987+0000 7f9a21d9b640 1 -- 172.21.15.22:0/3223268498 learned_addr learned my addr 172.21.15.22:0/3223268498 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:43.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.987+0000 7f9a2259c640 1 --2- 172.21.15.22:0/3223268498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a24100a70 0x7f9a24198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:43.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.987+0000 7f9a22d9d640 1 --2- 172.21.15.22:0/3223268498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24106a70 0x7f9a2419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:43.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.988+0000 7f9a21d9b640 1 -- 172.21.15.22:0/3223268498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a24100a70 msgr2=0x7f9a24198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:43.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.988+0000 7f9a21d9b640 1 --2- 172.21.15.22:0/3223268498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a24100a70 0x7f9a24198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:43.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.988+0000 7f9a21d9b640 1 -- 172.21.15.22:0/3223268498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24106a70 msgr2=0x7f9a2419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:43.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.988+0000 7f9a21d9b640 1 --2- 172.21.15.22:0/3223268498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24106a70 0x7f9a2419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:43.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.988+0000 7f9a21d9b640 1 -- 172.21.15.22:0/3223268498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2419d3f0 con 0x7f9a24101410 2024-09-19T08:15:43.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.988+0000 7f9a21d9b640 1 --2- 172.21.15.22:0/3223268498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a24101410 0x7f9a24198980 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009a50 tx=0x7f9a18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:43.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.988+0000 7f9a137fe640 1 -- 172.21.15.22:0/3223268498 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1803f030 con 0x7f9a24101410 2024-09-19T08:15:43.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.988+0000 7f9a28d02640 1 -- 172.21.15.22:0/3223268498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2419d680 con 0x7f9a24101410 2024-09-19T08:15:43.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.988+0000 7f9a137fe640 1 -- 172.21.15.22:0/3223268498 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a18003ad0 con 0x7f9a24101410 2024-09-19T08:15:43.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.988+0000 7f9a137fe640 1 -- 172.21.15.22:0/3223268498 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18035070 con 0x7f9a24101410 2024-09-19T08:15:43.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.988+0000 7f9a28d02640 1 -- 172.21.15.22:0/3223268498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a241a4fc0 con 0x7f9a24101410 2024-09-19T08:15:43.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.991+0000 7f9a137fe640 1 -- 172.21.15.22:0/3223268498 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f9a18042030 con 0x7f9a24101410 2024-09-19T08:15:43.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.991+0000 7f9a2b0db740 1 Processor -- start 2024-09-19T08:15:43.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.992+0000 7f9a2b0db740 1 -- start start 2024-09-19T08:15:43.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.992+0000 7f9a2b0db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x562356688ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:43.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.992+0000 7f9a2b0db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x562356683030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:43.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.992+0000 7f9a2b0db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 0x562356683b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:43.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.992+0000 7f9a2b0db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5623565df2e0 con 0x5623566886e0 2024-09-19T08:15:43.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.992+0000 7f9a2b0db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562356586ce0 con 0x562356689000 2024-09-19T08:15:43.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.992+0000 7f9a2b0db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5623565db2a0 con 0x5623566836a0 2024-09-19T08:15:43.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.992+0000 7f9a22d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x562356688ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:43.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.992+0000 7f9a22d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x562356688ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59980/0 (socket says 172.21.15.22:59980) 2024-09-19T08:15:43.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.992+0000 7f9a22d9d640 1 -- 172.21.15.22:0/512276982 learned_addr learned my addr 172.21.15.22:0/512276982 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:43.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.992+0000 7f9a2259c640 1 --2- 172.21.15.22:0/512276982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x562356683030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:43.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.992+0000 7f9a137fe640 1 --2- 172.21.15.22:0/3223268498 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9a0c0a3550 0x7f9a0c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:43.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a137fe640 1 -- 172.21.15.22:0/3223268498 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9a180f0530 con 0x7f9a24101410 2024-09-19T08:15:43.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a22d9d640 1 --2- 172.21.15.22:0/3223268498 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9a0c0a3550 0x7f9a0c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:43.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a2259c640 1 -- 172.21.15.22:0/512276982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 msgr2=0x562356683b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:43.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a21d9b640 1 --2- 172.21.15.22:0/512276982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 0x562356683b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:44.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a2259c640 1 --2- 172.21.15.22:0/512276982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 0x562356683b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a2259c640 1 -- 172.21.15.22:0/512276982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 msgr2=0x562356688ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:44.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a2259c640 1 --2- 172.21.15.22:0/512276982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x562356688ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a2259c640 1 -- 172.21.15.22:0/512276982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235656b440 con 0x562356689000 2024-09-19T08:15:44.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a22d9d640 1 --2- 172.21.15.22:0/512276982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x562356688ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:44.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a21d9b640 1 --2- 172.21.15.22:0/512276982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 0x562356683b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:15:44.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a2259c640 1 --2- 172.21.15.22:0/512276982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x562356683030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400abf0 tx=0x7f9a1400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d480c26ad0b85e65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:44.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a22d9d640 1 --2- 172.21.15.22:0/3223268498 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9a0c0a3550 0x7f9a0c0a5a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0075f0 tx=0x7f9a1c004a40 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:44.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a10ff9640 1 -- 172.21.15.22:0/512276982 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14005220 con 0x562356689000 2024-09-19T08:15:44.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a10ff9640 1 -- 172.21.15.22:0/512276982 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a14004c80 con 0x562356689000 2024-09-19T08:15:44.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.993+0000 7f9a10ff9640 1 -- 172.21.15.22:0/512276982 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a140187d0 con 0x562356689000 2024-09-19T08:15:44.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.994+0000 7f9a2b0db740 1 -- 172.21.15.22:0/512276982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 msgr2=0x562356683030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:44.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.994+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/512276982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x562356683030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400abf0 tx=0x7f9a1400f790 comp rx=0 tx=0).stop 2024-09-19T08:15:44.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.994+0000 7f9a2b0db740 1 -- 172.21.15.22:0/512276982 shutdown_connections 2024-09-19T08:15:44.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.994+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/512276982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 0x562356683b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.994+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/512276982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x562356683030 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.994+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/512276982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x562356688ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.994+0000 7f9a2b0db740 1 -- 172.21.15.22:0/512276982 >> 172.21.15.22:0/512276982 conn(0x5623566754e0 msgr2=0x5623566758c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:44.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.995+0000 7f9a2b0db740 1 -- 172.21.15.22:0/512276982 shutdown_connections 2024-09-19T08:15:44.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.995+0000 7f9a2b0db740 1 -- 172.21.15.22:0/512276982 wait complete. 2024-09-19T08:15:44.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.995+0000 7f9a2b0db740 1 -- start start 2024-09-19T08:15:44.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.995+0000 7f9a2b0db740 1 Processor -- start 2024-09-19T08:15:44.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.995+0000 7f9a2b0db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 0x56235668ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:44.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.995+0000 7f9a2b0db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x56235668c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:44.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.995+0000 7f9a2b0db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x56235668c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:44.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.995+0000 7f9a2b0db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56235668fa20 con 0x5623566886e0 2024-09-19T08:15:44.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.995+0000 7f9a2b0db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56235659a050 con 0x562356689000 2024-09-19T08:15:44.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.995+0000 7f9a2b0db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56235668fba0 con 0x5623566836a0 2024-09-19T08:15:44.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.995+0000 7f9a2b0db740 10 client.0 _ll_register_callbacks cb 0x5623565da160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:44.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.995+0000 7f9a21d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x56235668c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:44.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.995+0000 7f9a21d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x56235668c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59982/0 (socket says 172.21.15.22:59982) 2024-09-19T08:15:44.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.995+0000 7f9a21d9b640 1 -- 172.21.15.22:0/985125337 learned_addr learned my addr 172.21.15.22:0/985125337 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:44.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.996+0000 7f9a22d9d640 1 --2- 172.21.15.22:0/985125337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x56235668c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:44.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.996+0000 7f9a2259c640 1 --2- 172.21.15.22:0/985125337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 0x56235668ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:44.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.996+0000 7f9a21d9b640 1 -- 172.21.15.22:0/985125337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 msgr2=0x56235668ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.996+0000 7f9a21d9b640 1 --2- 172.21.15.22:0/985125337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 0x56235668ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.996+0000 7f9a21d9b640 1 -- 172.21.15.22:0/985125337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 msgr2=0x56235668c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.996+0000 7f9a21d9b640 1 --2- 172.21.15.22:0/985125337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x56235668c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.996+0000 7f9a21d9b640 1 -- 172.21.15.22:0/985125337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623563503c0 con 0x5623566886e0 2024-09-19T08:15:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.996+0000 7f9a22d9d640 1 --2- 172.21.15.22:0/985125337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x56235668c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.996+0000 7f9a21d9b640 1 --2- 172.21.15.22:0/985125337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x56235668c1e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9a18031550 tx=0x7f9a180bc490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886d09873b69ffb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.996+0000 7f9a09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.996+0000 7f9a09ffb640 1 -- 172.21.15.22:0/985125337 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18049040 con 0x5623566886e0 2024-09-19T08:15:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.996+0000 7f9a2b0db740 1 -- 172.21.15.22:0/985125337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a14004c80 con 0x5623566886e0 2024-09-19T08:15:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.996+0000 7f9a09ffb640 1 -- 172.21.15.22:0/985125337 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9a180b8cb0 con 0x5623566886e0 2024-09-19T08:15:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.997+0000 7f9a09ffb640 1 -- 172.21.15.22:0/985125337 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a180bc640 con 0x5623566886e0 2024-09-19T08:15:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.997+0000 7f9a2b0db740 10 client.15456 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.997+0000 7f9a2b0db740 1 -- 172.21.15.22:0/985125337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56235669b3e0 con 0x5623566886e0 2024-09-19T08:15:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.997+0000 7f9a09ffb640 1 -- 172.21.15.22:0/985125337 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a180c09f0 con 0x5623566886e0 2024-09-19T08:15:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.997+0000 7f9a09ffb640 1 -- 172.21.15.22:0/985125337 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9a180c4eb0 con 0x5623566886e0 2024-09-19T08:15:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.997+0000 7f9a2b0db740 10 client.15456 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.997+0000 7f9a2b0db740 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:15:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.997+0000 7f9a2b0db740 20 client.15456 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.997+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/985125337 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56235669c8d0 0x56235669ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.997+0000 7f9a2b0db740 4 client.15456 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:15:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.997+0000 7f9a2b0db740 1 -- 172.21.15.22:0/985125337 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9a14004c80 con 0x56235669c8d0 2024-09-19T08:15:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.997+0000 7f9a2259c640 1 --2- 172.21.15.22:0/985125337 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56235669c8d0 0x56235669ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.998+0000 7f9a2259c640 1 --2- 172.21.15.22:0/985125337 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56235669c8d0 0x56235669ecc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a752d344400f967a server_cookie=cb77e764ac14b1bf in_seq=0 out_seq=0 2024-09-19T08:15:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.998+0000 7f9a09ffb640 10 client.15456 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a09ffb640 1 -- 172.21.15.22:0/985125337 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9a14004c80 con 0x56235669c8d0 2024-09-19T08:15:44.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a09ffb640 10 client.15456 handle_command_reply: tid=0 2024-09-19T08:15:44.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a2b0db740 1 client.15456 shutdown 2024-09-19T08:15:44.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a2b0db740 1 -- 172.21.15.22:0/985125337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 msgr2=0x56235668c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:44.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/985125337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x56235668c1e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9a18031550 tx=0x7f9a180bc490 comp rx=0 tx=0).stop 2024-09-19T08:15:44.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a2b0db740 1 -- 172.21.15.22:0/985125337 shutdown_connections 2024-09-19T08:15:44.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/985125337 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56235669c8d0 0x56235669ecc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/985125337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x56235668c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/985125337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x56235668c1e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/985125337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 0x56235668ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a2b0db740 1 -- 172.21.15.22:0/985125337 >> 172.21.15.22:0/985125337 conn(0x5623566754e0 msgr2=0x562356674bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:44.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a09ffb640 0 client.15456 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:44.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a2b0db740 1 -- 172.21.15.22:0/985125337 shutdown_connections 2024-09-19T08:15:44.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a2b0db740 1 -- 172.21.15.22:0/985125337 wait complete. 2024-09-19T08:15:44.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:43.999+0000 7f9a2b0db740 20 client.15456 trim_cache size 0 max 16384 2024-09-19T08:15:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.017+0000 7f9a2b0db740 1 Processor -- start 2024-09-19T08:15:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.017+0000 7f9a2b0db740 1 -- start start 2024-09-19T08:15:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.017+0000 7f9a2b0db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566836a0 0x562356683030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.017+0000 7f9a2b0db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623566886e0 0x562356698cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.017+0000 7f9a2b0db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562356689000 0x562356674a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.017+0000 7f9a2b0db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241ade80 con 0x5623566836a0 2024-09-19T08:15:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.017+0000 7f9a2b0db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241ab750 con 0x5623566886e0 2024-09-19T08:15:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.017+0000 7f9a2b0db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241a95a0 con 0x562356689000 2024-09-19T08:15:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.017+0000 7f9a2259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566836a0 0x562356683030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.017+0000 7f9a2259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566836a0 0x562356683030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59992/0 (socket says 172.21.15.22:59992) 2024-09-19T08:15:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.017+0000 7f9a2259c640 1 -- 172.21.15.22:0/1597097945 learned_addr learned my addr 172.21.15.22:0/1597097945 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.018+0000 7f9a22d9d640 1 --2- 172.21.15.22:0/1597097945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562356689000 0x562356674a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.018+0000 7f9a2259c640 1 -- 172.21.15.22:0/1597097945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562356689000 msgr2=0x562356674a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.018+0000 7f9a2259c640 1 --2- 172.21.15.22:0/1597097945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562356689000 0x562356674a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.018+0000 7f9a2259c640 1 -- 172.21.15.22:0/1597097945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623566886e0 msgr2=0x562356698cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.018+0000 7f9a2259c640 1 --2- 172.21.15.22:0/1597097945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623566886e0 0x562356698cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.018+0000 7f9a2259c640 1 -- 172.21.15.22:0/1597097945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235668caf0 con 0x5623566836a0 2024-09-19T08:15:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.018+0000 7f9a22d9d640 1 --2- 172.21.15.22:0/1597097945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562356689000 0x562356674a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:15:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.018+0000 7f9a2259c640 1 --2- 172.21.15.22:0/1597097945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566836a0 0x562356683030 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400ab10 tx=0x7f9a1400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbc14cdb7068f6c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:44.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.018+0000 7f9a0bfff640 1 -- 172.21.15.22:0/1597097945 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a140046e0 con 0x5623566836a0 2024-09-19T08:15:44.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.018+0000 7f9a0bfff640 1 -- 172.21.15.22:0/1597097945 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a14005000 con 0x5623566836a0 2024-09-19T08:15:44.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.018+0000 7f9a0bfff640 1 -- 172.21.15.22:0/1597097945 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14018730 con 0x5623566836a0 2024-09-19T08:15:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.019+0000 7f9a2b0db740 1 -- 172.21.15.22:0/1597097945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566836a0 msgr2=0x562356683030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.019+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/1597097945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566836a0 0x562356683030 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400ab10 tx=0x7f9a1400a2d0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.019+0000 7f9a2b0db740 1 -- 172.21.15.22:0/1597097945 shutdown_connections 2024-09-19T08:15:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.019+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/1597097945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562356689000 0x562356674a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.019+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/1597097945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623566886e0 0x562356698cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.019+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/1597097945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566836a0 0x562356683030 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.019+0000 7f9a2b0db740 1 -- 172.21.15.22:0/1597097945 >> 172.21.15.22:0/1597097945 conn(0x5623566754e0 msgr2=0x562356674350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.019+0000 7f9a2b0db740 1 -- 172.21.15.22:0/1597097945 shutdown_connections 2024-09-19T08:15:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.019+0000 7f9a2b0db740 1 -- 172.21.15.22:0/1597097945 wait complete. 2024-09-19T08:15:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.019+0000 7f9a2b0db740 1 -- start start 2024-09-19T08:15:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.019+0000 7f9a2b0db740 1 Processor -- start 2024-09-19T08:15:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a2b0db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 0x5623566952b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a2b0db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x5623566a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a2b0db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x5623566a4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a2b0db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241a9c10 con 0x5623566886e0 2024-09-19T08:15:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a2b0db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241ab080 con 0x562356689000 2024-09-19T08:15:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a2b0db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241aa2e0 con 0x5623566836a0 2024-09-19T08:15:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a2b0db740 10 client.0 _ll_register_callbacks cb 0x7f9a180edb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a21d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x5623566a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a21d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x5623566a3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59996/0 (socket says 172.21.15.22:59996) 2024-09-19T08:15:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a21d9b640 1 -- 172.21.15.22:0/2189387929 learned_addr learned my addr 172.21.15.22:0/2189387929 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a2259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 0x5623566952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a22d9d640 1 --2- 172.21.15.22:0/2189387929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x5623566a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a21d9b640 1 -- 172.21.15.22:0/2189387929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 msgr2=0x5623566952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a21d9b640 1 --2- 172.21.15.22:0/2189387929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 0x5623566952b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a21d9b640 1 -- 172.21.15.22:0/2189387929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 msgr2=0x5623566a4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a21d9b640 1 --2- 172.21.15.22:0/2189387929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x5623566a4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a21d9b640 1 -- 172.21.15.22:0/2189387929 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623566a4670 con 0x5623566886e0 2024-09-19T08:15:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.020+0000 7f9a22d9d640 1 --2- 172.21.15.22:0/2189387929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x5623566a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.021+0000 7f9a21d9b640 1 --2- 172.21.15.22:0/2189387929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x5623566a3bf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9a180bc5d0 tx=0x7f9a18003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1869340d640c4f18 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.021+0000 7f9a09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.021+0000 7f9a09ffb640 1 -- 172.21.15.22:0/2189387929 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18049070 con 0x5623566886e0 2024-09-19T08:15:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.021+0000 7f9a09ffb640 1 -- 172.21.15.22:0/2189387929 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9a180c4e20 con 0x5623566886e0 2024-09-19T08:15:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.021+0000 7f9a2b0db740 1 -- 172.21.15.22:0/2189387929 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a14005000 con 0x5623566886e0 2024-09-19T08:15:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.021+0000 7f9a09ffb640 1 -- 172.21.15.22:0/2189387929 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a180b88e0 con 0x5623566886e0 2024-09-19T08:15:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.021+0000 7f9a2b0db740 10 client.15462 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.021+0000 7f9a2b0db740 1 -- 172.21.15.22:0/2189387929 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562356707200 con 0x5623566886e0 2024-09-19T08:15:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.021+0000 7f9a09ffb640 1 -- 172.21.15.22:0/2189387929 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a14005000 con 0x5623566886e0 2024-09-19T08:15:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.021+0000 7f9a09ffb640 1 -- 172.21.15.22:0/2189387929 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9a180f0040 con 0x5623566886e0 2024-09-19T08:15:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.021+0000 7f9a2b0db740 10 client.15462 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.021+0000 7f9a2b0db740 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:15:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.021+0000 7f9a2b0db740 20 client.15462 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.021+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/2189387929 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562356707840 0x562356709c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.022+0000 7f9a2b0db740 4 client.15462 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:15:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.022+0000 7f9a2b0db740 1 -- 172.21.15.22:0/2189387929 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5623565ec850 con 0x562356707840 2024-09-19T08:15:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.022+0000 7f9a2259c640 1 --2- 172.21.15.22:0/2189387929 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562356707840 0x562356709c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.022+0000 7f9a09ffb640 10 client.15462 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.022+0000 7f9a2259c640 1 --2- 172.21.15.22:0/2189387929 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562356707840 0x562356709c60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204f0fe54bfd6ba2 server_cookie=9e467027cbc35175 in_seq=0 out_seq=0 2024-09-19T08:15:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.022+0000 7f9a09ffb640 1 -- 172.21.15.22:0/2189387929 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5623565ec850 con 0x562356707840 2024-09-19T08:15:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.022+0000 7f9a09ffb640 10 client.15462 handle_command_reply: tid=0 2024-09-19T08:15:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.022+0000 7f9a2b0db740 1 client.15462 shutdown 2024-09-19T08:15:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.023+0000 7f9a2b0db740 1 -- 172.21.15.22:0/2189387929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 msgr2=0x5623566a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.023+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/2189387929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x5623566a3bf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9a180bc5d0 tx=0x7f9a18003eb0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.023+0000 7f9a2b0db740 1 -- 172.21.15.22:0/2189387929 shutdown_connections 2024-09-19T08:15:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.023+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/2189387929 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562356707840 0x562356709c60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.023+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/2189387929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562356689000 0x5623566a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.023+0000 7f9a09ffb640 0 client.15462 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:44.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.023+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/2189387929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623566886e0 0x5623566a3bf0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.023+0000 7f9a2b0db740 1 --2- 172.21.15.22:0/2189387929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623566836a0 0x5623566952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.023+0000 7f9a2b0db740 1 -- 172.21.15.22:0/2189387929 >> 172.21.15.22:0/2189387929 conn(0x5623566754e0 msgr2=0x562356674b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:44.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.023+0000 7f9a2b0db740 1 -- 172.21.15.22:0/2189387929 shutdown_connections 2024-09-19T08:15:44.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.023+0000 7f9a2b0db740 1 -- 172.21.15.22:0/2189387929 wait complete. 2024-09-19T08:15:44.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.023+0000 7f9a2b0db740 20 client.15462 trim_cache size 0 max 16384 2024-09-19T08:15:44.029 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:15:44.030 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-19T08:15:44.030 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-09-19T08:15:44.030 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:15:44.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.024+0000 7f9a28d02640 1 -- 172.21.15.22:0/3223268498 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9a0c0a3550 msgr2=0x7f9a0c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:44.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.024+0000 7f9a28d02640 1 --2- 172.21.15.22:0/3223268498 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9a0c0a3550 0x7f9a0c0a5a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0075f0 tx=0x7f9a1c004a40 comp rx=0 tx=0).stop 2024-09-19T08:15:44.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.024+0000 7f9a28d02640 1 -- 172.21.15.22:0/3223268498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a24101410 msgr2=0x7f9a24198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:44.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.024+0000 7f9a28d02640 1 --2- 172.21.15.22:0/3223268498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a24101410 0x7f9a24198980 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009a50 tx=0x7f9a18004270 comp rx=0 tx=0).stop 2024-09-19T08:15:44.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.024+0000 7f9a28d02640 1 -- 172.21.15.22:0/3223268498 shutdown_connections 2024-09-19T08:15:44.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.024+0000 7f9a28d02640 1 --2- 172.21.15.22:0/3223268498 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9a0c0a3550 0x7f9a0c0a5a10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.024+0000 7f9a28d02640 1 --2- 172.21.15.22:0/3223268498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24106a70 0x7f9a2419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.024+0000 7f9a28d02640 1 --2- 172.21.15.22:0/3223268498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a24101410 0x7f9a24198980 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.024+0000 7f9a28d02640 1 --2- 172.21.15.22:0/3223268498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a24100a70 0x7f9a24198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:44.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.024+0000 7f9a28d02640 1 -- 172.21.15.22:0/3223268498 >> 172.21.15.22:0/3223268498 conn(0x7f9a240fc7d0 msgr2=0x7f9a2406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:44.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.024+0000 7f9a28d02640 1 -- 172.21.15.22:0/3223268498 shutdown_connections 2024-09-19T08:15:44.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:44.024+0000 7f9a28d02640 1 -- 172.21.15.22:0/3223268498 wait complete. 2024-09-19T08:15:44.616 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:44 smithi022 ceph-mon[30963]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:15:44.765 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:15:44.765 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:15:44.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:44 smithi064 ceph-mon[34900]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:15:45.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:44 smithi157 ceph-mon[35143]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:15:45.051 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:45.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.578+0000 7f99c0556640 1 -- 172.21.15.22:0/1815978772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8103740 msgr2=0x7f99b8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:45.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.578+0000 7f99c0556640 1 --2- 172.21.15.22:0/1815978772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8103740 0x7f99b8107b80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009930 tx=0x7f99a802a420 comp rx=0 tx=0).stop 2024-09-19T08:15:45.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.579+0000 7f99c0556640 1 -- 172.21.15.22:0/1815978772 shutdown_connections 2024-09-19T08:15:45.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.579+0000 7f99c0556640 1 --2- 172.21.15.22:0/1815978772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b8108290 0x7f99b810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:45.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.579+0000 7f99c0556640 1 --2- 172.21.15.22:0/1815978772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8103740 0x7f99b8107b80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:45.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.579+0000 7f99c0556640 1 --2- 172.21.15.22:0/1815978772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b8102d90 0x7f99b8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:45.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.579+0000 7f99c0556640 1 -- 172.21.15.22:0/1815978772 >> 172.21.15.22:0/1815978772 conn(0x7f99b80fc7d0 msgr2=0x7f99b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:45.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.579+0000 7f99c0556640 1 -- 172.21.15.22:0/1815978772 shutdown_connections 2024-09-19T08:15:45.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.579+0000 7f99c0556640 1 -- 172.21.15.22:0/1815978772 wait complete. 2024-09-19T08:15:45.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.579+0000 7f99c0556640 1 Processor -- start 2024-09-19T08:15:45.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.579+0000 7f99c0556640 1 -- start start 2024-09-19T08:15:45.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99c0556640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b8102d90 0x7f99b81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:45.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99c0556640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b8103740 0x7f99b8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:45.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99c0556640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8108290 0x7f99b819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:45.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99c0556640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99b8112050 con 0x7f99b8108290 2024-09-19T08:15:45.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99c0556640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99b8111ed0 con 0x7f99b8103740 2024-09-19T08:15:45.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99c0556640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99b81121d0 con 0x7f99b8102d90 2024-09-19T08:15:45.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99beaf3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8108290 0x7f99b819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:45.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99beaf3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8108290 0x7f99b819ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60002/0 (socket says 172.21.15.22:60002) 2024-09-19T08:15:45.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99beaf3640 1 -- 172.21.15.22:0/1776320159 learned_addr learned my addr 172.21.15.22:0/1776320159 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:45.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99bdaf1640 1 --2- 172.21.15.22:0/1776320159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b8103740 0x7f99b8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:45.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99be2f2640 1 --2- 172.21.15.22:0/1776320159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b8102d90 0x7f99b81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:45.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99beaf3640 1 -- 172.21.15.22:0/1776320159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b8102d90 msgr2=0x7f99b81985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:45.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99beaf3640 1 --2- 172.21.15.22:0/1776320159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b8102d90 0x7f99b81985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:45.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99beaf3640 1 -- 172.21.15.22:0/1776320159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b8103740 msgr2=0x7f99b8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:45.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99beaf3640 1 --2- 172.21.15.22:0/1776320159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b8103740 0x7f99b8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:45.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.580+0000 7f99beaf3640 1 -- 172.21.15.22:0/1776320159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b819d560 con 0x7f99b8108290 2024-09-19T08:15:45.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.581+0000 7f99beaf3640 1 --2- 172.21.15.22:0/1776320159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8108290 0x7f99b819ce80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f99b000eaa0 tx=0x7f99b000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:45.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.581+0000 7f99af7fe640 1 -- 172.21.15.22:0/1776320159 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b000cdb0 con 0x7f99b8108290 2024-09-19T08:15:45.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.581+0000 7f99af7fe640 1 -- 172.21.15.22:0/1776320159 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99b0005020 con 0x7f99b8108290 2024-09-19T08:15:45.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.581+0000 7f99c0556640 1 -- 172.21.15.22:0/1776320159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b819d850 con 0x7f99b8108290 2024-09-19T08:15:45.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.581+0000 7f99af7fe640 1 -- 172.21.15.22:0/1776320159 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b00116b0 con 0x7f99b8108290 2024-09-19T08:15:45.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.581+0000 7f99c0556640 1 -- 172.21.15.22:0/1776320159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b81a5130 con 0x7f99b8108290 2024-09-19T08:15:45.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.582+0000 7f99c0556640 1 -- 172.21.15.22:0/1776320159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b8103170 con 0x7f99b8108290 2024-09-19T08:15:45.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.584+0000 7f99af7fe640 1 -- 172.21.15.22:0/1776320159 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f99b0005560 con 0x7f99b8108290 2024-09-19T08:15:45.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.585+0000 7f99af7fe640 1 --2- 172.21.15.22:0/1776320159 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f99a40a34d0 0x7f99a40a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:45.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.585+0000 7f99be2f2640 1 --2- 172.21.15.22:0/1776320159 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f99a40a34d0 0x7f99a40a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:45.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.585+0000 7f99af7fe640 1 -- 172.21.15.22:0/1776320159 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f99b00d0100 con 0x7f99b8108290 2024-09-19T08:15:45.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.586+0000 7f99be2f2640 1 --2- 172.21.15.22:0/1776320159 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f99a40a34d0 0x7f99a40a5990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f99b400bbe0 tx=0x7f99b400b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:45.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.590+0000 7f99af7fe640 1 -- 172.21.15.22:0/1776320159 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f99b0016030 con 0x7f99b8108290 2024-09-19T08:15:45.718 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.2 2024-09-19T08:15:45.718 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:15:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.813+0000 7f99c0556640 1 -- 172.21.15.22:0/1776320159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99b810a2a0 con 0x7f99b8108290 2024-09-19T08:15:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.814+0000 7f99af7fe640 1 -- 172.21.15.22:0/1776320159 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11906 (secure 0 0 0) 0x7f99b009c250 con 0x7f99b8108290 2024-09-19T08:15:45.814 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:45.815 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"btime":"2024-09-19T08:15:00:109390+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:15:00.059208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:15:45.815 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-09-19T08:15:45.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.818+0000 7f99c0556640 1 -- 172.21.15.22:0/1776320159 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f99a40a34d0 msgr2=0x7f99a40a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:45.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.818+0000 7f99c0556640 1 --2- 172.21.15.22:0/1776320159 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f99a40a34d0 0x7f99a40a5990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f99b400bbe0 tx=0x7f99b400b390 comp rx=0 tx=0).stop 2024-09-19T08:15:45.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.818+0000 7f99c0556640 1 -- 172.21.15.22:0/1776320159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8108290 msgr2=0x7f99b819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:45.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.818+0000 7f99c0556640 1 --2- 172.21.15.22:0/1776320159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8108290 0x7f99b819ce80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f99b000eaa0 tx=0x7f99b000ef60 comp rx=0 tx=0).stop 2024-09-19T08:15:45.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.818+0000 7f99c0556640 1 -- 172.21.15.22:0/1776320159 shutdown_connections 2024-09-19T08:15:45.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.818+0000 7f99c0556640 1 --2- 172.21.15.22:0/1776320159 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f99a40a34d0 0x7f99a40a5990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:45.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.818+0000 7f99c0556640 1 --2- 172.21.15.22:0/1776320159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8108290 0x7f99b819ce80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:45.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.818+0000 7f99c0556640 1 --2- 172.21.15.22:0/1776320159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b8103740 0x7f99b8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:45.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.818+0000 7f99c0556640 1 --2- 172.21.15.22:0/1776320159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b8102d90 0x7f99b81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:45.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.818+0000 7f99c0556640 1 -- 172.21.15.22:0/1776320159 >> 172.21.15.22:0/1776320159 conn(0x7f99b80fc7d0 msgr2=0x7f99b8105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:45.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.818+0000 7f99c0556640 1 -- 172.21.15.22:0/1776320159 shutdown_connections 2024-09-19T08:15:45.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:45.818+0000 7f99c0556640 1 -- 172.21.15.22:0/1776320159 wait complete. 2024-09-19T08:15:46.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:15:46.545 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:46.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:46 smithi022 ceph-mon[30963]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:15:46.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:46 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1776320159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:46.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:46 smithi064 ceph-mon[34900]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:15:46.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:46 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1776320159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:47.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:46 smithi157 ceph-mon[35143]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:15:47.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:46 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1776320159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:47.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.066+0000 7f6c47844640 1 -- 172.21.15.22:0/1561891169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c40101de0 msgr2=0x7f6c401021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:47.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.066+0000 7f6c47844640 1 --2- 172.21.15.22:0/1561891169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c40101de0 0x7f6c401021c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009900 tx=0x7f6c3c02a410 comp rx=0 tx=0).stop 2024-09-19T08:15:47.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.066+0000 7f6c47844640 1 -- 172.21.15.22:0/1561891169 shutdown_connections 2024-09-19T08:15:47.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.066+0000 7f6c47844640 1 --2- 172.21.15.22:0/1561891169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c40106f10 0x7f6c4010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:47.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.066+0000 7f6c47844640 1 --2- 172.21.15.22:0/1561891169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c40102790 0x7f6c401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:47.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.066+0000 7f6c47844640 1 --2- 172.21.15.22:0/1561891169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c40101de0 0x7f6c401021c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:47.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.066+0000 7f6c47844640 1 -- 172.21.15.22:0/1561891169 >> 172.21.15.22:0/1561891169 conn(0x7f6c400fd8c0 msgr2=0x7f6c400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:47.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.066+0000 7f6c47844640 1 -- 172.21.15.22:0/1561891169 shutdown_connections 2024-09-19T08:15:47.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.066+0000 7f6c47844640 1 -- 172.21.15.22:0/1561891169 wait complete. 2024-09-19T08:15:47.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.067+0000 7f6c47844640 1 Processor -- start 2024-09-19T08:15:47.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.067+0000 7f6c47844640 1 -- start start 2024-09-19T08:15:47.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.067+0000 7f6c47844640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c40101de0 0x7f6c40198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:47.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.067+0000 7f6c47844640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c40102790 0x7f6c401989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:47.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.067+0000 7f6c47844640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c40106f10 0x7f6c4019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:47.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.067+0000 7f6c47844640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40111f70 con 0x7f6c40101de0 2024-09-19T08:15:47.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.067+0000 7f6c47844640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40111df0 con 0x7f6c40106f10 2024-09-19T08:15:47.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.067+0000 7f6c47844640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401120f0 con 0x7f6c40102790 2024-09-19T08:15:47.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.067+0000 7f6c455e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c40101de0 0x7f6c40198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:47.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.067+0000 7f6c455e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c40101de0 0x7f6c40198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60020/0 (socket says 172.21.15.22:60020) 2024-09-19T08:15:47.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.067+0000 7f6c455e0640 1 -- 172.21.15.22:0/1495904549 learned_addr learned my addr 172.21.15.22:0/1495904549 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:47.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.067+0000 7f6c44ddf640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c40102790 0x7f6c401989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:47.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.068+0000 7f6c45de1640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c40106f10 0x7f6c4019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:47.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.068+0000 7f6c455e0640 1 -- 172.21.15.22:0/1495904549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c40102790 msgr2=0x7f6c401989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:47.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.068+0000 7f6c455e0640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c40102790 0x7f6c401989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:47.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.068+0000 7f6c455e0640 1 -- 172.21.15.22:0/1495904549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c40106f10 msgr2=0x7f6c4019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:47.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.068+0000 7f6c455e0640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c40106f10 0x7f6c4019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:47.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.068+0000 7f6c455e0640 1 -- 172.21.15.22:0/1495904549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4019d420 con 0x7f6c40101de0 2024-09-19T08:15:47.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.068+0000 7f6c44ddf640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c40102790 0x7f6c401989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:47.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.068+0000 7f6c455e0640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c40101de0 0x7f6c40198470 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c006010 tx=0x7f6c3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:47.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.068+0000 7f6c367fc640 1 -- 172.21.15.22:0/1495904549 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c03f030 con 0x7f6c40101de0 2024-09-19T08:15:47.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.068+0000 7f6c367fc640 1 -- 172.21.15.22:0/1495904549 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3c003ce0 con 0x7f6c40101de0 2024-09-19T08:15:47.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.068+0000 7f6c47844640 1 -- 172.21.15.22:0/1495904549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4019d6b0 con 0x7f6c40101de0 2024-09-19T08:15:47.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.068+0000 7f6c367fc640 1 -- 172.21.15.22:0/1495904549 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c035070 con 0x7f6c40101de0 2024-09-19T08:15:47.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.068+0000 7f6c47844640 1 -- 172.21.15.22:0/1495904549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c401a4ff0 con 0x7f6c40101de0 2024-09-19T08:15:47.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.070+0000 7f6c47844640 1 -- 172.21.15.22:0/1495904549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c401032c0 con 0x7f6c40101de0 2024-09-19T08:15:47.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.071+0000 7f6c367fc640 1 -- 172.21.15.22:0/1495904549 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f6c3c042030 con 0x7f6c40101de0 2024-09-19T08:15:47.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.071+0000 7f6c367fc640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6c2c0a34d0 0x7f6c2c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:47.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.071+0000 7f6c44ddf640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6c2c0a34d0 0x7f6c2c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:47.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.071+0000 7f6c367fc640 1 -- 172.21.15.22:0/1495904549 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6c3c0f0660 con 0x7f6c40101de0 2024-09-19T08:15:47.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.072+0000 7f6c44ddf640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6c2c0a34d0 0x7f6c2c0a5990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6c300058c0 tx=0x7f6c30005850 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:47.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.076+0000 7f6c367fc640 1 -- 172.21.15.22:0/1495904549 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6c3c0bcf90 con 0x7f6c40101de0 2024-09-19T08:15:47.121 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:47.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.325+0000 7f6c47844640 1 -- 172.21.15.22:0/1495904549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c4010a0f0 con 0x7f6c40101de0 2024-09-19T08:15:47.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.326+0000 7f6c367fc640 1 -- 172.21.15.22:0/1495904549 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11906 (secure 0 0 0) 0x7f6c3c0bc890 con 0x7f6c40101de0 2024-09-19T08:15:47.326 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:47.326 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"btime":"2024-09-19T08:15:00:109390+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:15:00.059208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:15:47.326 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-09-19T08:15:47.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.328+0000 7f6c47844640 1 -- 172.21.15.22:0/1495904549 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6c2c0a34d0 msgr2=0x7f6c2c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:47.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.328+0000 7f6c47844640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6c2c0a34d0 0x7f6c2c0a5990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6c300058c0 tx=0x7f6c30005850 comp rx=0 tx=0).stop 2024-09-19T08:15:47.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.328+0000 7f6c47844640 1 -- 172.21.15.22:0/1495904549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c40101de0 msgr2=0x7f6c40198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:47.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.328+0000 7f6c47844640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c40101de0 0x7f6c40198470 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c006010 tx=0x7f6c3c0042f0 comp rx=0 tx=0).stop 2024-09-19T08:15:47.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.328+0000 7f6c47844640 1 -- 172.21.15.22:0/1495904549 shutdown_connections 2024-09-19T08:15:47.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.328+0000 7f6c47844640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6c2c0a34d0 0x7f6c2c0a5990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:47.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.328+0000 7f6c47844640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c40106f10 0x7f6c4019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:47.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.328+0000 7f6c47844640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c40102790 0x7f6c401989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:47.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.328+0000 7f6c47844640 1 --2- 172.21.15.22:0/1495904549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c40101de0 0x7f6c40198470 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:47.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.328+0000 7f6c47844640 1 -- 172.21.15.22:0/1495904549 >> 172.21.15.22:0/1495904549 conn(0x7f6c400fd8c0 msgr2=0x7f6c400fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:47.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.328+0000 7f6c47844640 1 -- 172.21.15.22:0/1495904549 shutdown_connections 2024-09-19T08:15:47.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:47.328+0000 7f6c47844640 1 -- 172.21.15.22:0/1495904549 wait complete. 2024-09-19T08:15:47.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:47 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1495904549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:47.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:47 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1495904549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:48.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:47 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1495904549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:15:48.100 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0' 2024-09-19T08:15:48.101 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0 2024-09-19T08:15:48.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.355+0000 7f6637cb1640 1 -- 172.21.15.22:0/4160227306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6630101420 msgr2=0x7f663010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:48.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.355+0000 7f6637cb1640 1 --2- 172.21.15.22:0/4160227306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6630101420 0x7f663010d950 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6628009cd0 tx=0x7f662802a8e0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.355+0000 7f6637cb1640 1 -- 172.21.15.22:0/4160227306 shutdown_connections 2024-09-19T08:15:48.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.355+0000 7f6637cb1640 1 --2- 172.21.15.22:0/4160227306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6630101420 0x7f663010d950 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.355+0000 7f6637cb1640 1 --2- 172.21.15.22:0/4160227306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6630100a80 0x7f6630100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.355+0000 7f6637cb1640 1 --2- 172.21.15.22:0/4160227306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6630106a80 0x7f6630106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.355+0000 7f6637cb1640 1 -- 172.21.15.22:0/4160227306 >> 172.21.15.22:0/4160227306 conn(0x7f66300fc7a0 msgr2=0x7f66300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:48.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.355+0000 7f6637cb1640 1 -- 172.21.15.22:0/4160227306 shutdown_connections 2024-09-19T08:15:48.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.355+0000 7f6637cb1640 1 -- 172.21.15.22:0/4160227306 wait complete. 2024-09-19T08:15:48.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f6637cb1640 1 Processor -- start 2024-09-19T08:15:48.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f6637cb1640 1 -- start start 2024-09-19T08:15:48.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f6637cb1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6630100a80 0x7f663019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f6637cb1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6630101420 0x7f663019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f6637cb1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6630106a80 0x7f663019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f6637cb1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6630110e80 con 0x7f6630106a80 2024-09-19T08:15:48.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f6637cb1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6630110d00 con 0x7f6630100a80 2024-09-19T08:15:48.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f6637cb1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6630111000 con 0x7f6630101420 2024-09-19T08:15:48.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f663624e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6630106a80 0x7f663019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f6635a4d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6630100a80 0x7f663019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f663624e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6630106a80 0x7f663019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39748/0 (socket says 172.21.15.22:39748) 2024-09-19T08:15:48.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f663624e640 1 -- 172.21.15.22:0/1380931953 learned_addr learned my addr 172.21.15.22:0/1380931953 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:48.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f663524c640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6630101420 0x7f663019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f6635a4d640 1 -- 172.21.15.22:0/1380931953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6630101420 msgr2=0x7f663019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:48.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f6635a4d640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6630101420 0x7f663019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f6635a4d640 1 -- 172.21.15.22:0/1380931953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6630106a80 msgr2=0x7f663019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:48.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f6635a4d640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6630106a80 0x7f663019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.356+0000 7f6635a4d640 1 -- 172.21.15.22:0/1380931953 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663019f5f0 con 0x7f6630100a80 2024-09-19T08:15:48.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.357+0000 7f663624e640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6630106a80 0x7f663019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:48.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.357+0000 7f663524c640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6630101420 0x7f663019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:48.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.357+0000 7f6635a4d640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6630100a80 0x7f663019a640 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f662c00d6d0 tx=0x7f662c00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:48.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.357+0000 7f6626ffd640 1 -- 172.21.15.22:0/1380931953 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c017070 con 0x7f6630100a80 2024-09-19T08:15:48.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.357+0000 7f6637cb1640 1 -- 172.21.15.22:0/1380931953 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663019f8e0 con 0x7f6630100a80 2024-09-19T08:15:48.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.357+0000 7f6637cb1640 1 -- 172.21.15.22:0/1380931953 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66301a71c0 con 0x7f6630100a80 2024-09-19T08:15:48.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.357+0000 7f6626ffd640 1 -- 172.21.15.22:0/1380931953 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662c009820 con 0x7f6630100a80 2024-09-19T08:15:48.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.357+0000 7f6626ffd640 1 -- 172.21.15.22:0/1380931953 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c020060 con 0x7f6630100a80 2024-09-19T08:15:48.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.360+0000 7f663a08a740 1 Processor -- start 2024-09-19T08:15:48.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f663a08a740 1 -- start start 2024-09-19T08:15:48.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f6626ffd640 1 -- 172.21.15.22:0/1380931953 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f662c005220 con 0x7f6630100a80 2024-09-19T08:15:48.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f663a08a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648eb40 0x55c12648d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f663a08a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648d5d0 0x55c12648da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f663a08a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 0x55c12649a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f663a08a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c1263f32f0 con 0x55c12648d5d0 2024-09-19T08:15:48.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f663a08a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c12639acf0 con 0x55c12648eb40 2024-09-19T08:15:48.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f663a08a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c1263ef2b0 con 0x55c126498370 2024-09-19T08:15:48.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f6635a4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648d5d0 0x55c12648da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f6635a4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648d5d0 0x55c12648da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39760/0 (socket says 172.21.15.22:39760) 2024-09-19T08:15:48.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f6635a4d640 1 -- 172.21.15.22:0/2410278836 learned_addr learned my addr 172.21.15.22:0/2410278836 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:48.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f663624e640 1 --2- 172.21.15.22:0/2410278836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648eb40 0x55c12648d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f663524c640 1 --2- 172.21.15.22:0/2410278836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 0x55c12649a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f6635a4d640 1 -- 172.21.15.22:0/2410278836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 msgr2=0x55c12649a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:48.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f6635a4d640 1 --2- 172.21.15.22:0/2410278836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 0x55c12649a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f6635a4d640 1 -- 172.21.15.22:0/2410278836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648eb40 msgr2=0x55c12648d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:48.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f6635a4d640 1 --2- 172.21.15.22:0/2410278836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648eb40 0x55c12648d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.361+0000 7f6635a4d640 1 -- 172.21.15.22:0/2410278836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c12637f3a0 con 0x55c12648d5d0 2024-09-19T08:15:48.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.362+0000 7f663524c640 1 --2- 172.21.15.22:0/2410278836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 0x55c12649a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:15:48.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.362+0000 7f6635a4d640 1 --2- 172.21.15.22:0/2410278836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648d5d0 0x55c12648da10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f662c0cd0a0 tx=0x7f662c0d84f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a087320d95b41241 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:48.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.362+0000 7f6626ffd640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f661c0a34d0 0x7f661c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.362+0000 7f661bfff640 1 -- 172.21.15.22:0/2410278836 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c0e3070 con 0x55c12648d5d0 2024-09-19T08:15:48.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.362+0000 7f663624e640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f661c0a34d0 0x7f661c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.362+0000 7f6626ffd640 1 -- 172.21.15.22:0/1380931953 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f662c0cf9b0 con 0x7f6630100a80 2024-09-19T08:15:48.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.362+0000 7f661bfff640 1 -- 172.21.15.22:0/2410278836 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662c09c150 con 0x55c12648d5d0 2024-09-19T08:15:48.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.362+0000 7f661bfff640 1 -- 172.21.15.22:0/2410278836 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c0cfe60 con 0x55c12648d5d0 2024-09-19T08:15:48.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.363+0000 7f663a08a740 1 -- 172.21.15.22:0/2410278836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648d5d0 msgr2=0x55c12648da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:48.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.363+0000 7f663624e640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f661c0a34d0 0x7f661c0a5990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6628031860 tx=0x7f6628005c50 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:48.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.363+0000 7f663a08a740 1 --2- 172.21.15.22:0/2410278836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648d5d0 0x55c12648da10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f662c0cd0a0 tx=0x7f662c0d84f0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.363+0000 7f663a08a740 1 -- 172.21.15.22:0/2410278836 shutdown_connections 2024-09-19T08:15:48.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.363+0000 7f663a08a740 1 --2- 172.21.15.22:0/2410278836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 0x55c12649a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.363+0000 7f663a08a740 1 --2- 172.21.15.22:0/2410278836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648d5d0 0x55c12648da10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.363+0000 7f663a08a740 1 --2- 172.21.15.22:0/2410278836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648eb40 0x55c12648d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.363+0000 7f663a08a740 1 -- 172.21.15.22:0/2410278836 >> 172.21.15.22:0/2410278836 conn(0x55c126456310 msgr2=0x55c126489c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:48.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.363+0000 7f663a08a740 1 -- 172.21.15.22:0/2410278836 shutdown_connections 2024-09-19T08:15:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.363+0000 7f663a08a740 1 -- 172.21.15.22:0/2410278836 wait complete. 2024-09-19T08:15:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.364+0000 7f663a08a740 1 -- start start 2024-09-19T08:15:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.364+0000 7f663a08a740 1 Processor -- start 2024-09-19T08:15:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.364+0000 7f663a08a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 0x55c126497cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.364+0000 7f663a08a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648eb40 0x55c126494f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663a08a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 0x55c1264954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663524c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648eb40 0x55c126494f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663524c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648eb40 0x55c126494f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39772/0 (socket says 172.21.15.22:39772) 2024-09-19T08:15:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663524c640 1 -- 172.21.15.22:0/2456702211 learned_addr learned my addr 172.21.15.22:0/2456702211 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663a08a740 1 -- 172.21.15.22:0/2456702211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c12649b070 con 0x55c12648eb40 2024-09-19T08:15:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663a08a740 1 -- 172.21.15.22:0/2456702211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c1263ae060 con 0x55c12648d5d0 2024-09-19T08:15:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663a08a740 1 -- 172.21.15.22:0/2456702211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c12649b1f0 con 0x55c126498370 2024-09-19T08:15:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f6635a4d640 1 --2- 172.21.15.22:0/2456702211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 0x55c126497cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663a08a740 10 client.0 _ll_register_callbacks cb 0x55c1261018c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663524c640 1 -- 172.21.15.22:0/2456702211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 msgr2=0x55c1264954e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:15:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663624e640 1 --2- 172.21.15.22:0/2456702211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 0x55c1264954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663524c640 1 --2- 172.21.15.22:0/2456702211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 0x55c1264954e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663524c640 1 -- 172.21.15.22:0/2456702211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 msgr2=0x55c126497cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663524c640 1 --2- 172.21.15.22:0/2456702211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 0x55c126497cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663524c640 1 -- 172.21.15.22:0/2456702211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1261643c0 con 0x55c12648eb40 2024-09-19T08:15:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663a08a740 1 -- 172.21.15.22:0/2456702211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f662c09c150 con 0x55c12648eb40 2024-09-19T08:15:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f663524c640 1 --2- 172.21.15.22:0/2456702211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648eb40 0x55c126494f80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6620010670 tx=0x7f6620010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35acb217630e3ba6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.365+0000 7f66197fa640 10 client.15486 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.366+0000 7f66197fa640 1 -- 172.21.15.22:0/2456702211 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662000ead0 con 0x55c12648eb40 2024-09-19T08:15:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.366+0000 7f6635a4d640 1 --2- 172.21.15.22:0/2456702211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 0x55c126497cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.366+0000 7f66197fa640 1 -- 172.21.15.22:0/2456702211 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f662000ec70 con 0x55c12648eb40 2024-09-19T08:15:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.366+0000 7f66197fa640 1 -- 172.21.15.22:0/2456702211 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f662000ee70 con 0x55c12648eb40 2024-09-19T08:15:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.366+0000 7f663a08a740 10 client.15486 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.366+0000 7f663a08a740 1 -- 172.21.15.22:0/2456702211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55c1264a6a80 con 0x55c12648eb40 2024-09-19T08:15:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.366+0000 7f66197fa640 1 -- 172.21.15.22:0/2456702211 <== mon.0 v2:172.21.15.22:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f662000f040 con 0x55c12648eb40 2024-09-19T08:15:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.366+0000 7f663a08a740 10 client.15486 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.366+0000 7f663a08a740 10 client.15486 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:15:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.366+0000 7f663a08a740 20 client.15486 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.366+0000 7f663a08a740 1 --2- 172.21.15.22:0/2456702211 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55c1264a7e30 0x55c1264aa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.366+0000 7f663a08a740 4 client.15486 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:15:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.366+0000 7f663a08a740 1 -- 172.21.15.22:0/2456702211 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f662c09c150 con 0x55c1264a7e30 2024-09-19T08:15:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.367+0000 7f6635a4d640 1 --2- 172.21.15.22:0/2456702211 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55c1264a7e30 0x55c1264aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.367+0000 7f66197fa640 10 client.15486 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:15:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.367+0000 7f6635a4d640 1 --2- 172.21.15.22:0/2456702211 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55c1264a7e30 0x55c1264aa220 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e87dee45a57c9b5 server_cookie=236d8df1cd4e43f2 in_seq=0 out_seq=0 2024-09-19T08:15:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f66197fa640 1 -- 172.21.15.22:0/2456702211 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f662c09c150 con 0x55c1264a7e30 2024-09-19T08:15:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f66197fa640 10 client.15486 handle_command_reply: tid=0 2024-09-19T08:15:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f663a08a740 1 client.15486 shutdown 2024-09-19T08:15:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f663a08a740 1 -- 172.21.15.22:0/2456702211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648eb40 msgr2=0x55c126494f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f663a08a740 1 --2- 172.21.15.22:0/2456702211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648eb40 0x55c126494f80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6620010670 tx=0x7f6620010b30 comp rx=0 tx=0).stop 2024-09-19T08:15:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f663a08a740 1 -- 172.21.15.22:0/2456702211 shutdown_connections 2024-09-19T08:15:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f663a08a740 1 --2- 172.21.15.22:0/2456702211 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55c1264a7e30 0x55c1264aa220 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f66197fa640 0 client.15486 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:15:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f663a08a740 1 --2- 172.21.15.22:0/2456702211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 0x55c1264954e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f663a08a740 1 --2- 172.21.15.22:0/2456702211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648eb40 0x55c126494f80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f663a08a740 1 --2- 172.21.15.22:0/2456702211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 0x55c126497cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f663a08a740 1 -- 172.21.15.22:0/2456702211 >> 172.21.15.22:0/2456702211 conn(0x55c126456310 msgr2=0x55c12648f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:48.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f663a08a740 1 -- 172.21.15.22:0/2456702211 shutdown_connections 2024-09-19T08:15:48.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f663a08a740 1 -- 172.21.15.22:0/2456702211 wait complete. 2024-09-19T08:15:48.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.368+0000 7f663a08a740 20 client.15486 trim_cache size 0 max 16384 2024-09-19T08:15:48.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.381+0000 7f663a08a740 1 Processor -- start 2024-09-19T08:15:48.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.381+0000 7f663a08a740 1 -- start start 2024-09-19T08:15:48.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f663a08a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 0x55c1264a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f663a08a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c12648eb40 0x55c126494840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f663a08a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c126498370 0x55c126493850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f663a08a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6614010f80 con 0x55c126498370 2024-09-19T08:15:48.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f663a08a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f661400e850 con 0x55c12648d5d0 2024-09-19T08:15:48.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f663a08a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f661400c6a0 con 0x55c12648eb40 2024-09-19T08:15:48.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f663624e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c126498370 0x55c126493850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f663524c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c12648eb40 0x55c126494840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f663624e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c126498370 0x55c126493850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39788/0 (socket says 172.21.15.22:39788) 2024-09-19T08:15:48.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f663624e640 1 -- 172.21.15.22:0/2890138624 learned_addr learned my addr 172.21.15.22:0/2890138624 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:48.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f6635a4d640 1 --2- 172.21.15.22:0/2890138624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 0x55c1264a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f663624e640 1 -- 172.21.15.22:0/2890138624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c12648eb40 msgr2=0x55c126494840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:48.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f663624e640 1 --2- 172.21.15.22:0/2890138624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c12648eb40 0x55c126494840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f663624e640 1 -- 172.21.15.22:0/2890138624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 msgr2=0x55c1264a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:48.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.382+0000 7f663624e640 1 --2- 172.21.15.22:0/2890138624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 0x55c1264a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.383+0000 7f663624e640 1 -- 172.21.15.22:0/2890138624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c12649aad0 con 0x55c126498370 2024-09-19T08:15:48.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.383+0000 7f663624e640 1 --2- 172.21.15.22:0/2890138624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c126498370 0x55c126493850 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6628009df0 tx=0x7f6628040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7fe318c22fde5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:48.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.383+0000 7f661b7fe640 1 -- 172.21.15.22:0/2890138624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628056070 con 0x55c126498370 2024-09-19T08:15:48.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.383+0000 7f661b7fe640 1 -- 172.21.15.22:0/2890138624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6628036040 con 0x55c126498370 2024-09-19T08:15:48.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.383+0000 7f661b7fe640 1 -- 172.21.15.22:0/2890138624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628030aa0 con 0x55c126498370 2024-09-19T08:15:48.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.384+0000 7f663a08a740 1 -- 172.21.15.22:0/2890138624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c126498370 msgr2=0x55c126493850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:48.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.384+0000 7f663a08a740 1 --2- 172.21.15.22:0/2890138624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c126498370 0x55c126493850 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6628009df0 tx=0x7f6628040000 comp rx=0 tx=0).stop 2024-09-19T08:15:48.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.384+0000 7f663a08a740 1 -- 172.21.15.22:0/2890138624 shutdown_connections 2024-09-19T08:15:48.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.384+0000 7f663a08a740 1 --2- 172.21.15.22:0/2890138624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c126498370 0x55c126493850 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.384+0000 7f663a08a740 1 --2- 172.21.15.22:0/2890138624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c12648eb40 0x55c126494840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.384+0000 7f663a08a740 1 --2- 172.21.15.22:0/2890138624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 0x55c1264a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.384+0000 7f663a08a740 1 -- 172.21.15.22:0/2890138624 >> 172.21.15.22:0/2890138624 conn(0x55c126456310 msgr2=0x55c126484a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:48.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.384+0000 7f663a08a740 1 -- 172.21.15.22:0/2890138624 shutdown_connections 2024-09-19T08:15:48.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.384+0000 7f663a08a740 1 -- 172.21.15.22:0/2890138624 wait complete. 2024-09-19T08:15:48.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.384+0000 7f663a08a740 1 -- start start 2024-09-19T08:15:48.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.385+0000 7f663a08a740 1 Processor -- start 2024-09-19T08:15:48.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.385+0000 7f663a08a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 0x55c12649bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.385+0000 7f663a08a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648eb40 0x55c12649c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.385+0000 7f663a08a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 0x55c1265132b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.385+0000 7f663a08a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f661400cd10 con 0x55c12648eb40 2024-09-19T08:15:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.385+0000 7f663a08a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f661400e180 con 0x55c12648d5d0 2024-09-19T08:15:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.385+0000 7f663a08a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f661400d3e0 con 0x55c126498370 2024-09-19T08:15:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.385+0000 7f663a08a740 10 client.0 _ll_register_callbacks cb 0x7f662000b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.385+0000 7f663524c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648eb40 0x55c12649c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.386+0000 7f663524c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648eb40 0x55c12649c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39804/0 (socket says 172.21.15.22:39804) 2024-09-19T08:15:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.386+0000 7f663524c640 1 -- 172.21.15.22:0/3687911063 learned_addr learned my addr 172.21.15.22:0/3687911063 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.386+0000 7f663624e640 1 --2- 172.21.15.22:0/3687911063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 0x55c1265132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.386+0000 7f6635a4d640 1 --2- 172.21.15.22:0/3687911063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 0x55c12649bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.386+0000 7f6635a4d640 1 -- 172.21.15.22:0/3687911063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 msgr2=0x55c1265132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.386+0000 7f6635a4d640 1 --2- 172.21.15.22:0/3687911063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 0x55c1265132b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.386+0000 7f6635a4d640 1 -- 172.21.15.22:0/3687911063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648eb40 msgr2=0x55c12649c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.386+0000 7f6635a4d640 1 --2- 172.21.15.22:0/3687911063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648eb40 0x55c12649c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.386+0000 7f6635a4d640 1 -- 172.21.15.22:0/3687911063 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1265138f0 con 0x55c12648d5d0 2024-09-19T08:15:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.386+0000 7f663524c640 1 --2- 172.21.15.22:0/3687911063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648eb40 0x55c12649c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.386+0000 7f6635a4d640 1 --2- 172.21.15.22:0/3687911063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 0x55c12649bb90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f662c097eb0 tx=0x7f662c005870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=616468f388bc9d08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.386+0000 7f66197fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:15:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.387+0000 7f66197fa640 1 -- 172.21.15.22:0/3687911063 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c0cfe60 con 0x55c12648d5d0 2024-09-19T08:15:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.387+0000 7f66197fa640 1 -- 172.21.15.22:0/3687911063 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f662c099d50 con 0x55c12648d5d0 2024-09-19T08:15:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.387+0000 7f663a08a740 1 -- 172.21.15.22:0/3687911063 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6628036040 con 0x55c12648d5d0 2024-09-19T08:15:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.387+0000 7f66197fa640 1 -- 172.21.15.22:0/3687911063 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f662c09b820 con 0x55c12648d5d0 2024-09-19T08:15:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.387+0000 7f663a08a740 10 client.25178 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.387+0000 7f663a08a740 1 -- 172.21.15.22:0/3687911063 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c12651acc0 con 0x55c12648d5d0 2024-09-19T08:15:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.387+0000 7f66197fa640 1 -- 172.21.15.22:0/3687911063 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6628036040 con 0x55c12648d5d0 2024-09-19T08:15:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.387+0000 7f66197fa640 1 -- 172.21.15.22:0/3687911063 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f662c0980c0 con 0x55c12648d5d0 2024-09-19T08:15:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.388+0000 7f663a08a740 10 client.25178 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.388+0000 7f663a08a740 10 client.25178 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:15:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.388+0000 7f663a08a740 20 client.25178 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.388+0000 7f663a08a740 1 --2- 172.21.15.22:0/3687911063 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55c12651b050 0x55c12651d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.388+0000 7f663a08a740 4 client.25178 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:15:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.388+0000 7f663a08a740 1 -- 172.21.15.22:0/3687911063 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c126400860 con 0x55c12651b050 2024-09-19T08:15:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.388+0000 7f663524c640 1 --2- 172.21.15.22:0/3687911063 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55c12651b050 0x55c12651d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.388+0000 7f663524c640 1 --2- 172.21.15.22:0/3687911063 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55c12651b050 0x55c12651d440 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d9c9d3c53bc1b13 server_cookie=265ac17b873ecad7 in_seq=0 out_seq=0 2024-09-19T08:15:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.388+0000 7f66197fa640 10 client.25178 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:15:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f66197fa640 1 -- 172.21.15.22:0/3687911063 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c126400860 con 0x55c12651b050 2024-09-19T08:15:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f66197fa640 10 client.25178 handle_command_reply: tid=0 2024-09-19T08:15:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f663a08a740 1 client.25178 shutdown 2024-09-19T08:15:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f663a08a740 1 -- 172.21.15.22:0/3687911063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 msgr2=0x55c12649bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f663a08a740 1 --2- 172.21.15.22:0/3687911063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 0x55c12649bb90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f662c097eb0 tx=0x7f662c005870 comp rx=0 tx=0).stop 2024-09-19T08:15:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f663a08a740 1 -- 172.21.15.22:0/3687911063 shutdown_connections 2024-09-19T08:15:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f663a08a740 1 --2- 172.21.15.22:0/3687911063 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55c12651b050 0x55c12651d440 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f663a08a740 1 --2- 172.21.15.22:0/3687911063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c126498370 0x55c1265132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f663a08a740 1 --2- 172.21.15.22:0/3687911063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c12648eb40 0x55c12649c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f66197fa640 0 client.25178 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:15:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f663a08a740 1 --2- 172.21.15.22:0/3687911063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c12648d5d0 0x55c12649bb90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f663a08a740 1 -- 172.21.15.22:0/3687911063 >> 172.21.15.22:0/3687911063 conn(0x55c126456310 msgr2=0x55c1264936e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f663a08a740 1 -- 172.21.15.22:0/3687911063 shutdown_connections 2024-09-19T08:15:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f663a08a740 1 -- 172.21.15.22:0/3687911063 wait complete. 2024-09-19T08:15:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.389+0000 7f663a08a740 20 client.25178 trim_cache size 0 max 16384 2024-09-19T08:15:48.408 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:15:48.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.390+0000 7f6637cb1640 1 -- 172.21.15.22:0/1380931953 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f661c0a34d0 msgr2=0x7f661c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:48.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.390+0000 7f6637cb1640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f661c0a34d0 0x7f661c0a5990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6628031860 tx=0x7f6628005c50 comp rx=0 tx=0).stop 2024-09-19T08:15:48.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.390+0000 7f6637cb1640 1 -- 172.21.15.22:0/1380931953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6630100a80 msgr2=0x7f663019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:48.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.390+0000 7f6637cb1640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6630100a80 0x7f663019a640 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f662c00d6d0 tx=0x7f662c00db90 comp rx=0 tx=0).stop 2024-09-19T08:15:48.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.390+0000 7f6637cb1640 1 -- 172.21.15.22:0/1380931953 shutdown_connections 2024-09-19T08:15:48.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.390+0000 7f6637cb1640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f661c0a34d0 0x7f661c0a5990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.390+0000 7f6637cb1640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6630106a80 0x7f663019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.390+0000 7f6637cb1640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6630101420 0x7f663019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.390+0000 7f6637cb1640 1 --2- 172.21.15.22:0/1380931953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6630100a80 0x7f663019a640 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:48.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.390+0000 7f6637cb1640 1 -- 172.21.15.22:0/1380931953 >> 172.21.15.22:0/1380931953 conn(0x7f66300fc7a0 msgr2=0x7f663006a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:48.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.391+0000 7f6637cb1640 1 -- 172.21.15.22:0/1380931953 shutdown_connections 2024-09-19T08:15:48.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:48.391+0000 7f6637cb1640 1 -- 172.21.15.22:0/1380931953 wait complete. 2024-09-19T08:15:48.718 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:48 smithi022 ceph-mon[30963]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:15:48.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:48 smithi064 ceph-mon[34900]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:15:48.904 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:48 smithi157 ceph-mon[35143]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:15:49.049 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:15:49.092 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:49.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.585+0000 7f9982dbd640 1 -- 172.21.15.22:0/2805597602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f997c102d30 msgr2=0x7f997c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:49.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.585+0000 7f9982dbd640 1 --2- 172.21.15.22:0/2805597602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f997c102d30 0x7f997c103110 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9970009930 tx=0x7f997002a420 comp rx=0 tx=0).stop 2024-09-19T08:15:49.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.586+0000 7f9982dbd640 1 -- 172.21.15.22:0/2805597602 shutdown_connections 2024-09-19T08:15:49.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.586+0000 7f9982dbd640 1 --2- 172.21.15.22:0/2805597602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c1082a0 0x7f997c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.586+0000 7f9982dbd640 1 --2- 172.21.15.22:0/2805597602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c1036e0 0x7f997c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.586+0000 7f9982dbd640 1 --2- 172.21.15.22:0/2805597602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f997c102d30 0x7f997c103110 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.586+0000 7f9982dbd640 1 -- 172.21.15.22:0/2805597602 >> 172.21.15.22:0/2805597602 conn(0x7f997c0fc7d0 msgr2=0x7f997c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:49.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.586+0000 7f9982dbd640 1 -- 172.21.15.22:0/2805597602 shutdown_connections 2024-09-19T08:15:49.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.586+0000 7f9982dbd640 1 -- 172.21.15.22:0/2805597602 wait complete. 2024-09-19T08:15:49.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.587+0000 7f9982dbd640 1 Processor -- start 2024-09-19T08:15:49.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.587+0000 7f9982dbd640 1 -- start start 2024-09-19T08:15:49.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.587+0000 7f9982dbd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c102d30 0x7f997c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.587+0000 7f9982dbd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c1036e0 0x7f997c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.587+0000 7f9982dbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f997c1082a0 0x7f997c19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.587+0000 7f9982dbd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f997c1120f0 con 0x7f997c1082a0 2024-09-19T08:15:49.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.587+0000 7f9982dbd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f997c111f70 con 0x7f997c102d30 2024-09-19T08:15:49.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.587+0000 7f9982dbd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f997c112270 con 0x7f997c1036e0 2024-09-19T08:15:49.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.587+0000 7f998135a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f997c1082a0 0x7f997c19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.587+0000 7f997bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c1036e0 0x7f997c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.587+0000 7f998135a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f997c1082a0 0x7f997c19cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39830/0 (socket says 172.21.15.22:39830) 2024-09-19T08:15:49.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.587+0000 7f998135a640 1 -- 172.21.15.22:0/1448999057 learned_addr learned my addr 172.21.15.22:0/1448999057 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:49.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.587+0000 7f9980b59640 1 --2- 172.21.15.22:0/1448999057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c102d30 0x7f997c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.588+0000 7f997bfff640 1 -- 172.21.15.22:0/1448999057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c102d30 msgr2=0x7f997c198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:49.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.588+0000 7f997bfff640 1 --2- 172.21.15.22:0/1448999057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c102d30 0x7f997c198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.588+0000 7f997bfff640 1 -- 172.21.15.22:0/1448999057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f997c1082a0 msgr2=0x7f997c19cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:49.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.588+0000 7f997bfff640 1 --2- 172.21.15.22:0/1448999057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f997c1082a0 0x7f997c19cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.588+0000 7f997bfff640 1 -- 172.21.15.22:0/1448999057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c19d620 con 0x7f997c1036e0 2024-09-19T08:15:49.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.588+0000 7f998135a640 1 --2- 172.21.15.22:0/1448999057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f997c1082a0 0x7f997c19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:49.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.588+0000 7f997bfff640 1 --2- 172.21.15.22:0/1448999057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c1036e0 0x7f997c198bb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f997c0697d0 tx=0x7f996800db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:49.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.588+0000 7f9979ffb640 1 -- 172.21.15.22:0/1448999057 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968017070 con 0x7f997c1036e0 2024-09-19T08:15:49.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.588+0000 7f9979ffb640 1 -- 172.21.15.22:0/1448999057 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9968009820 con 0x7f997c1036e0 2024-09-19T08:15:49.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.588+0000 7f9982dbd640 1 -- 172.21.15.22:0/1448999057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997c19d910 con 0x7f997c1036e0 2024-09-19T08:15:49.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.588+0000 7f9979ffb640 1 -- 172.21.15.22:0/1448999057 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968004ca0 con 0x7f997c1036e0 2024-09-19T08:15:49.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.589+0000 7f9982dbd640 1 -- 172.21.15.22:0/1448999057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997c1a51f0 con 0x7f997c1036e0 2024-09-19T08:15:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.591+0000 7f9979ffb640 1 -- 172.21.15.22:0/1448999057 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f9968005220 con 0x7f997c1036e0 2024-09-19T08:15:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.591+0000 7f9979ffb640 1 --2- 172.21.15.22:0/1448999057 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f996c0a34d0 0x7f996c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.592+0000 7f9979ffb640 1 -- 172.21.15.22:0/1448999057 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f99680cf7f0 con 0x7f997c1036e0 2024-09-19T08:15:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.592+0000 7f9980b59640 1 --2- 172.21.15.22:0/1448999057 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f996c0a34d0 0x7f996c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.592+0000 7f9985196740 1 Processor -- start 2024-09-19T08:15:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.592+0000 7f9980b59640 1 --2- 172.21.15.22:0/1448999057 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f996c0a34d0 0x7f996c0a5990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9970006010 tx=0x7f9970000bc0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.592+0000 7f9985196740 1 -- start start 2024-09-19T08:15:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.593+0000 7f9985196740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca21600 0x55830ca219e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.593+0000 7f9985196740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca30810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.593+0000 7f997bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca30810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.593+0000 7f9985196740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca30d50 0x55830ca33140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.593+0000 7f997bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca30810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39834/0 (socket says 172.21.15.22:39834) 2024-09-19T08:15:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.593+0000 7f997bfff640 1 -- 172.21.15.22:0/3056891222 learned_addr learned my addr 172.21.15.22:0/3056891222 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.593+0000 7f9985196740 1 -- 172.21.15.22:0/3056891222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55830c9872a0 con 0x55830ca21f20 2024-09-19T08:15:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.593+0000 7f9980b59640 1 --2- 172.21.15.22:0/3056891222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca21600 0x55830ca219e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.593+0000 7f9985196740 1 -- 172.21.15.22:0/3056891222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55830c92eb10 con 0x55830ca21600 2024-09-19T08:15:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.593+0000 7f9985196740 1 -- 172.21.15.22:0/3056891222 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55830c983260 con 0x55830ca30d50 2024-09-19T08:15:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.593+0000 7f998135a640 1 --2- 172.21.15.22:0/3056891222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca30d50 0x55830ca33140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.594+0000 7f997bfff640 1 -- 172.21.15.22:0/3056891222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca30d50 msgr2=0x55830ca33140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.594+0000 7f997bfff640 1 --2- 172.21.15.22:0/3056891222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca30d50 0x55830ca33140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.594+0000 7f997bfff640 1 -- 172.21.15.22:0/3056891222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca21600 msgr2=0x55830ca219e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.594+0000 7f997bfff640 1 --2- 172.21.15.22:0/3056891222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca21600 0x55830ca219e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.594+0000 7f997bfff640 1 -- 172.21.15.22:0/3056891222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55830c6f8b90 con 0x55830ca21f20 2024-09-19T08:15:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.594+0000 7f9980b59640 1 --2- 172.21.15.22:0/3056891222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca21600 0x55830ca219e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.594+0000 7f997bfff640 1 --2- 172.21.15.22:0/3056891222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca30810 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9968097da0 tx=0x7f996809b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9381b724843900e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.594+0000 7f9966ffd640 1 -- 172.21.15.22:0/3056891222 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968020070 con 0x55830ca21f20 2024-09-19T08:15:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.594+0000 7f9966ffd640 1 -- 172.21.15.22:0/3056891222 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99680860b0 con 0x55830ca21f20 2024-09-19T08:15:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.594+0000 7f9966ffd640 1 -- 172.21.15.22:0/3056891222 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996809cbc0 con 0x55830ca21f20 2024-09-19T08:15:49.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.595+0000 7f9985196740 1 -- 172.21.15.22:0/3056891222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 msgr2=0x55830ca30810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:49.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.595+0000 7f9985196740 1 --2- 172.21.15.22:0/3056891222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca30810 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9968097da0 tx=0x7f996809b590 comp rx=0 tx=0).stop 2024-09-19T08:15:49.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.595+0000 7f9985196740 1 -- 172.21.15.22:0/3056891222 shutdown_connections 2024-09-19T08:15:49.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.595+0000 7f9985196740 1 --2- 172.21.15.22:0/3056891222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca30d50 0x55830ca33140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.595+0000 7f9985196740 1 --2- 172.21.15.22:0/3056891222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca30810 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.595+0000 7f9985196740 1 --2- 172.21.15.22:0/3056891222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca21600 0x55830ca219e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.595+0000 7f9985196740 1 -- 172.21.15.22:0/3056891222 >> 172.21.15.22:0/3056891222 conn(0x55830ca1dcf0 msgr2=0x55830ca1e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:49.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.595+0000 7f9985196740 1 -- 172.21.15.22:0/3056891222 shutdown_connections 2024-09-19T08:15:49.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.595+0000 7f9985196740 1 -- 172.21.15.22:0/3056891222 wait complete. 2024-09-19T08:15:49.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.596+0000 7f9985196740 1 -- start start 2024-09-19T08:15:49.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.596+0000 7f9985196740 1 Processor -- start 2024-09-19T08:15:49.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9985196740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca2dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9985196740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca2e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9980b59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca2dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f997bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca2e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9980b59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca2dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:60014/0 (socket says 172.21.15.22:60014) 2024-09-19T08:15:49.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f997bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca2e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39848/0 (socket says 172.21.15.22:39848) 2024-09-19T08:15:49.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9980b59640 1 -- 172.21.15.22:0/1189728445 learned_addr learned my addr 172.21.15.22:0/1189728445 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:49.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9985196740 1 --2- 172.21.15.22:0/1189728445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 0x55830ca2e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9985196740 1 -- 172.21.15.22:0/1189728445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55830ca33920 con 0x55830ca21f20 2024-09-19T08:15:49.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9985196740 1 -- 172.21.15.22:0/1189728445 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55830c90f6e0 con 0x55830ca30d50 2024-09-19T08:15:49.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9985196740 1 -- 172.21.15.22:0/1189728445 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55830ca33aa0 con 0x55830ca21600 2024-09-19T08:15:49.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9985196740 10 client.0 _ll_register_callbacks cb 0x55830c933a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:49.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f998135a640 1 --2- 172.21.15.22:0/1189728445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 0x55830ca2e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9980b59640 1 -- 172.21.15.22:0/1189728445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 msgr2=0x55830ca2e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:49.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9980b59640 1 --2- 172.21.15.22:0/1189728445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 0x55830ca2e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9980b59640 1 -- 172.21.15.22:0/1189728445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 msgr2=0x55830ca2e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9980b59640 1 --2- 172.21.15.22:0/1189728445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca2e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.597+0000 7f9980b59640 1 -- 172.21.15.22:0/1189728445 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55830ca3cf70 con 0x55830ca21600 2024-09-19T08:15:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.598+0000 7f9980b59640 1 --2- 172.21.15.22:0/1189728445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca2dd50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9970033000 tx=0x7f9970009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45135244dab022c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.598+0000 7f995bfff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:15:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.598+0000 7f995bfff640 1 -- 172.21.15.22:0/1189728445 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970035040 con 0x55830ca21600 2024-09-19T08:15:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.598+0000 7f9985196740 1 -- 172.21.15.22:0/1189728445 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99680860b0 con 0x55830ca21600 2024-09-19T08:15:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.598+0000 7f995bfff640 1 -- 172.21.15.22:0/1189728445 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9970004af0 con 0x55830ca21600 2024-09-19T08:15:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.598+0000 7f995bfff640 1 -- 172.21.15.22:0/1189728445 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9970004cf0 con 0x55830ca21600 2024-09-19T08:15:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.598+0000 7f9985196740 10 client.25129 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.598+0000 7f9985196740 1 -- 172.21.15.22:0/1189728445 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55830ca3f930 con 0x55830ca21600 2024-09-19T08:15:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.599+0000 7f995bfff640 1 -- 172.21.15.22:0/1189728445 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99680860b0 con 0x55830ca21600 2024-09-19T08:15:49.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.599+0000 7f995bfff640 1 -- 172.21.15.22:0/1189728445 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f99700038d0 con 0x55830ca21600 2024-09-19T08:15:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.599+0000 7f9985196740 10 client.25129 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.600+0000 7f9985196740 10 client.25129 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:15:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.600+0000 7f9985196740 20 client.25129 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.600+0000 7f9985196740 1 --2- 172.21.15.22:0/1189728445 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55830ca40d40 0x55830ca43170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.600+0000 7f9985196740 4 client.25129 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:15:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.600+0000 7f9985196740 1 -- 172.21.15.22:0/1189728445 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55830c9949d0 con 0x55830ca40d40 2024-09-19T08:15:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.600+0000 7f997bfff640 1 --2- 172.21.15.22:0/1189728445 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55830ca40d40 0x55830ca43170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.600+0000 7f997bfff640 1 --2- 172.21.15.22:0/1189728445 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55830ca40d40 0x55830ca43170 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be8b6d7f6d58b5ff server_cookie=33c2c16f1f486f7a in_seq=0 out_seq=0 2024-09-19T08:15:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.600+0000 7f995bfff640 10 client.25129 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.601+0000 7f995bfff640 1 -- 172.21.15.22:0/1189728445 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55830c9949d0 con 0x55830ca40d40 2024-09-19T08:15:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.601+0000 7f995bfff640 10 client.25129 handle_command_reply: tid=0 2024-09-19T08:15:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.601+0000 7f9985196740 1 client.25129 shutdown 2024-09-19T08:15:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.601+0000 7f9985196740 1 -- 172.21.15.22:0/1189728445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 msgr2=0x55830ca2dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.601+0000 7f9985196740 1 --2- 172.21.15.22:0/1189728445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca2dd50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9970033000 tx=0x7f9970009260 comp rx=0 tx=0).stop 2024-09-19T08:15:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.601+0000 7f9985196740 1 -- 172.21.15.22:0/1189728445 shutdown_connections 2024-09-19T08:15:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.601+0000 7f9985196740 1 --2- 172.21.15.22:0/1189728445 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55830ca40d40 0x55830ca43170 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.601+0000 7f9985196740 1 --2- 172.21.15.22:0/1189728445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 0x55830ca2e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.601+0000 7f9985196740 1 --2- 172.21.15.22:0/1189728445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca2e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.602+0000 7f9985196740 1 --2- 172.21.15.22:0/1189728445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca2dd50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.602+0000 7f9985196740 1 -- 172.21.15.22:0/1189728445 >> 172.21.15.22:0/1189728445 conn(0x55830ca1dcf0 msgr2=0x55830ca23a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.602+0000 7f995bfff640 0 client.25129 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.602+0000 7f9985196740 1 -- 172.21.15.22:0/1189728445 shutdown_connections 2024-09-19T08:15:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.602+0000 7f9985196740 1 -- 172.21.15.22:0/1189728445 wait complete. 2024-09-19T08:15:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.602+0000 7f9985196740 20 client.25129 trim_cache size 0 max 16384 2024-09-19T08:15:49.619 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.620+0000 7f9985196740 1 Processor -- start 2024-09-19T08:15:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.620+0000 7f9985196740 1 -- start start 2024-09-19T08:15:49.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.620+0000 7f9985196740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca2e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.620+0000 7f9985196740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca2e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.620+0000 7f9985196740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 0x55830ca48a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.620+0000 7f9985196740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f997c1ae080 con 0x55830ca21f20 2024-09-19T08:15:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.620+0000 7f9985196740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f997c1ab950 con 0x55830ca30d50 2024-09-19T08:15:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.620+0000 7f9985196740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f997c1a97a0 con 0x55830ca21600 2024-09-19T08:15:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.620+0000 7f997bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca2e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.620+0000 7f997bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca2e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39862/0 (socket says 172.21.15.22:39862) 2024-09-19T08:15:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.620+0000 7f997bfff640 1 -- 172.21.15.22:0/1340107550 learned_addr learned my addr 172.21.15.22:0/1340107550 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.621+0000 7f9980b59640 1 --2- 172.21.15.22:0/1340107550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca2e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.621+0000 7f997bfff640 1 -- 172.21.15.22:0/1340107550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 msgr2=0x55830ca2e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.621+0000 7f997bfff640 1 --2- 172.21.15.22:0/1340107550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca2e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.621+0000 7f997bfff640 1 -- 172.21.15.22:0/1340107550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 msgr2=0x55830ca48a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.621+0000 7f997bfff640 1 --2- 172.21.15.22:0/1340107550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 0x55830ca48a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.621+0000 7f997bfff640 1 -- 172.21.15.22:0/1340107550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55830ca1d790 con 0x55830ca21f20 2024-09-19T08:15:49.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.621+0000 7f9980b59640 1 --2- 172.21.15.22:0/1340107550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca2e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:49.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.621+0000 7f997bfff640 1 --2- 172.21.15.22:0/1340107550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca2e960 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f99680ccd10 tx=0x7f99680099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2c69db4b0455a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:49.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.621+0000 7f99667fc640 1 -- 172.21.15.22:0/1340107550 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968020040 con 0x55830ca21f20 2024-09-19T08:15:49.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.621+0000 7f99667fc640 1 -- 172.21.15.22:0/1340107550 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9968099c40 con 0x55830ca21f20 2024-09-19T08:15:49.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.621+0000 7f99667fc640 1 -- 172.21.15.22:0/1340107550 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996809ce20 con 0x55830ca21f20 2024-09-19T08:15:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.622+0000 7f9985196740 1 -- 172.21.15.22:0/1340107550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 msgr2=0x55830ca2e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.622+0000 7f9985196740 1 --2- 172.21.15.22:0/1340107550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca2e960 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f99680ccd10 tx=0x7f99680099d0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.622+0000 7f9985196740 1 -- 172.21.15.22:0/1340107550 shutdown_connections 2024-09-19T08:15:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.622+0000 7f9985196740 1 --2- 172.21.15.22:0/1340107550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 0x55830ca48a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.622+0000 7f9985196740 1 --2- 172.21.15.22:0/1340107550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca2e960 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.622+0000 7f9985196740 1 --2- 172.21.15.22:0/1340107550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca2e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.622+0000 7f9985196740 1 -- 172.21.15.22:0/1340107550 >> 172.21.15.22:0/1340107550 conn(0x55830ca1dcf0 msgr2=0x55830ca2dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.622+0000 7f9985196740 1 -- 172.21.15.22:0/1340107550 shutdown_connections 2024-09-19T08:15:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.622+0000 7f9985196740 1 -- 172.21.15.22:0/1340107550 wait complete. 2024-09-19T08:15:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.623+0000 7f9985196740 1 -- start start 2024-09-19T08:15:49.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.623+0000 7f9985196740 1 Processor -- start 2024-09-19T08:15:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.623+0000 7f9985196740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca27ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.623+0000 7f9985196740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca38d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.623+0000 7f9985196740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 0x55830ca39270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f9985196740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f997c1a9e10 con 0x55830ca21f20 2024-09-19T08:15:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f9985196740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f997c1ab280 con 0x55830ca30d50 2024-09-19T08:15:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f9985196740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f997c1aa4e0 con 0x55830ca21600 2024-09-19T08:15:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f9985196740 10 client.0 _ll_register_callbacks cb 0x7f9970004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f998135a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 0x55830ca39270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f9980b59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca27ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f9980b59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca27ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:60024/0 (socket says 172.21.15.22:60024) 2024-09-19T08:15:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f998135a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 0x55830ca39270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:37136/0 (socket says 172.21.15.22:37136) 2024-09-19T08:15:49.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f9980b59640 1 -- 172.21.15.22:0/629854174 learned_addr learned my addr 172.21.15.22:0/629854174 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:49.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f997bfff640 1 --2- 172.21.15.22:0/629854174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca38d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f9980b59640 1 -- 172.21.15.22:0/629854174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 msgr2=0x55830ca39270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:49.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f9980b59640 1 --2- 172.21.15.22:0/629854174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 0x55830ca39270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f9980b59640 1 -- 172.21.15.22:0/629854174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 msgr2=0x55830ca38d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:49.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f9980b59640 1 --2- 172.21.15.22:0/629854174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca38d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.624+0000 7f9980b59640 1 -- 172.21.15.22:0/629854174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55830ca25390 con 0x55830ca21600 2024-09-19T08:15:49.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.625+0000 7f9980b59640 1 --2- 172.21.15.22:0/629854174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca27ff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9970003d20 tx=0x7f99700038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1def3d0b1364e82f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:49.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.625+0000 7f995bfff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:15:49.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.625+0000 7f995bfff640 1 -- 172.21.15.22:0/629854174 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970035040 con 0x55830ca21600 2024-09-19T08:15:49.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.625+0000 7f995bfff640 1 -- 172.21.15.22:0/629854174 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f99700329d0 con 0x55830ca21600 2024-09-19T08:15:49.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.625+0000 7f9985196740 1 -- 172.21.15.22:0/629854174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9968099c40 con 0x55830ca21600 2024-09-19T08:15:49.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.625+0000 7f995bfff640 1 -- 172.21.15.22:0/629854174 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9970004e90 con 0x55830ca21600 2024-09-19T08:15:49.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.625+0000 7f9985196740 10 client.25135 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:49.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.625+0000 7f9985196740 1 -- 172.21.15.22:0/629854174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55830cab02c0 con 0x55830ca21600 2024-09-19T08:15:49.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.625+0000 7f995bfff640 1 -- 172.21.15.22:0/629854174 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9968099c40 con 0x55830ca21600 2024-09-19T08:15:49.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.625+0000 7f995bfff640 1 -- 172.21.15.22:0/629854174 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f997006d950 con 0x55830ca21600 2024-09-19T08:15:49.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.625+0000 7f9985196740 10 client.25135 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:49.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.625+0000 7f9985196740 10 client.25135 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:15:49.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.626+0000 7f9985196740 20 client.25135 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:49.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.626+0000 7f9985196740 1 --2- 172.21.15.22:0/629854174 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55830cab09f0 0x55830cab2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:49.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.626+0000 7f9985196740 4 client.25135 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}] 2024-09-19T08:15:49.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.626+0000 7f9985196740 1 -- 172.21.15.22:0/629854174 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}) -- 0x55830c98ed20 con 0x55830cab09f0 2024-09-19T08:15:49.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.626+0000 7f997bfff640 1 --2- 172.21.15.22:0/629854174 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55830cab09f0 0x55830cab2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:49.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.626+0000 7f995bfff640 10 client.25135 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:49.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.626+0000 7f997bfff640 1 --2- 172.21.15.22:0/629854174 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55830cab09f0 0x55830cab2e10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee98e2693469f1 server_cookie=68398f5714fea74d in_seq=0 out_seq=0 2024-09-19T08:15:49.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.639+0000 7f995bfff640 1 -- 172.21.15.22:0/629854174 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55830c98ed20 con 0x55830cab09f0 2024-09-19T08:15:49.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.639+0000 7f995bfff640 10 client.25135 handle_command_reply: tid=0 2024-09-19T08:15:49.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.639+0000 7f9985196740 1 client.25135 shutdown 2024-09-19T08:15:49.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.639+0000 7f9985196740 1 -- 172.21.15.22:0/629854174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 msgr2=0x55830ca27ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:49.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.639+0000 7f9985196740 1 --2- 172.21.15.22:0/629854174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca27ff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9970003d20 tx=0x7f99700038a0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.639+0000 7f9985196740 1 -- 172.21.15.22:0/629854174 shutdown_connections 2024-09-19T08:15:49.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.639+0000 7f9985196740 1 --2- 172.21.15.22:0/629854174 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55830cab09f0 0x55830cab2e10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.639+0000 7f9985196740 1 --2- 172.21.15.22:0/629854174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55830ca30d50 0x55830ca39270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.639+0000 7f9985196740 1 --2- 172.21.15.22:0/629854174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55830ca21f20 0x55830ca38d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.639+0000 7f995bfff640 0 client.25135 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:49.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.639+0000 7f9985196740 1 --2- 172.21.15.22:0/629854174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55830ca21600 0x55830ca27ff0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.639+0000 7f9985196740 1 -- 172.21.15.22:0/629854174 >> 172.21.15.22:0/629854174 conn(0x55830ca1dcf0 msgr2=0x55830ca2e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:49.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.639+0000 7f9985196740 1 -- 172.21.15.22:0/629854174 shutdown_connections 2024-09-19T08:15:49.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.640+0000 7f9985196740 1 -- 172.21.15.22:0/629854174 wait complete. 2024-09-19T08:15:49.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.640+0000 7f9985196740 20 client.25135 trim_cache size 0 max 16384 2024-09-19T08:15:49.643 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:15:49.643 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 25, 2024-09-19T08:15:49.643 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:15:49.643 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 2, 2024-09-19T08:15:49.643 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:15:49.643 INFO:teuthology.orchestra.run.smithi022.stdout: "bdd9f7b8": { 2024-09-19T08:15:49.643 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 2, 2024-09-19T08:15:49.643 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:15:49.643 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:15:49.644 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:15:49.644 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:15:49.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:15:49.644 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:15:49.644 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 131.2, 2024-09-19T08:15:49.644 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:15:49.644 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:15:49.644 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:15:49.644 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:15:49.644 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:15:49.644 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:15:49.644 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:15:49.644 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:15:49.644 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:15:49.645 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:15:49.645 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:15:49.645 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:15:49.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.640+0000 7f9982dbd640 1 -- 172.21.15.22:0/1448999057 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f996c0a34d0 msgr2=0x7f996c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:49.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.640+0000 7f9982dbd640 1 --2- 172.21.15.22:0/1448999057 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f996c0a34d0 0x7f996c0a5990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9970006010 tx=0x7f9970000bc0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.640+0000 7f9982dbd640 1 -- 172.21.15.22:0/1448999057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c1036e0 msgr2=0x7f997c198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.640+0000 7f9982dbd640 1 --2- 172.21.15.22:0/1448999057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c1036e0 0x7f997c198bb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f997c0697d0 tx=0x7f996800db30 comp rx=0 tx=0).stop 2024-09-19T08:15:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.641+0000 7f9982dbd640 1 -- 172.21.15.22:0/1448999057 shutdown_connections 2024-09-19T08:15:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.641+0000 7f9982dbd640 1 --2- 172.21.15.22:0/1448999057 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f996c0a34d0 0x7f996c0a5990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.641+0000 7f9982dbd640 1 --2- 172.21.15.22:0/1448999057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f997c1082a0 0x7f997c19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.641+0000 7f9982dbd640 1 --2- 172.21.15.22:0/1448999057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997c1036e0 0x7f997c198bb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.641+0000 7f9982dbd640 1 --2- 172.21.15.22:0/1448999057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c102d30 0x7f997c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.641+0000 7f9982dbd640 1 -- 172.21.15.22:0/1448999057 >> 172.21.15.22:0/1448999057 conn(0x7f997c0fc7d0 msgr2=0x7f997c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.641+0000 7f9982dbd640 1 -- 172.21.15.22:0/1448999057 shutdown_connections 2024-09-19T08:15:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:49.641+0000 7f9982dbd640 1 -- 172.21.15.22:0/1448999057 wait complete. 2024-09-19T08:15:50.401 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bdd9f7b8', quiesce took 4.7 seconds. Will release after: 6.499999999999999 2024-09-19T08:15:50.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:50 smithi022 ceph-mon[30963]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T08:15:50.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:50 smithi064 ceph-mon[34900]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T08:15:50.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.838+0000 7fc78ced6640 1 -- 172.21.15.22:0/4059813927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc788102570 msgr2=0x7fc7881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:50.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.838+0000 7fc78ced6640 1 --2- 172.21.15.22:0/4059813927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc788102570 0x7fc7881029b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc770009900 tx=0x7fc77002a480 comp rx=0 tx=0).stop 2024-09-19T08:15:50.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.838+0000 7fc78ced6640 1 -- 172.21.15.22:0/4059813927 shutdown_connections 2024-09-19T08:15:50.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.838+0000 7fc78ced6640 1 --2- 172.21.15.22:0/4059813927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc788102ef0 0x7fc78810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.838+0000 7fc78ced6640 1 --2- 172.21.15.22:0/4059813927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc788102570 0x7fc7881029b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.838+0000 7fc78ced6640 1 --2- 172.21.15.22:0/4059813927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc788100e20 0x7fc788101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.838+0000 7fc78ced6640 1 -- 172.21.15.22:0/4059813927 >> 172.21.15.22:0/4059813927 conn(0x7fc7880fc7d0 msgr2=0x7fc7880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:50.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.838+0000 7fc78ced6640 1 -- 172.21.15.22:0/4059813927 shutdown_connections 2024-09-19T08:15:50.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.838+0000 7fc78ced6640 1 -- 172.21.15.22:0/4059813927 wait complete. 2024-09-19T08:15:50.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.839+0000 7fc78ced6640 1 Processor -- start 2024-09-19T08:15:50.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.839+0000 7fc78ced6640 1 -- start start 2024-09-19T08:15:50.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.839+0000 7fc78ced6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc788100e20 0x7fc78819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.839+0000 7fc78ced6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc788102570 0x7fc78819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.839+0000 7fc78ced6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc788102ef0 0x7fc78819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.839+0000 7fc78ced6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc788110f80 con 0x7fc788100e20 2024-09-19T08:15:50.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.839+0000 7fc78ced6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc788110e00 con 0x7fc788102570 2024-09-19T08:15:50.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.839+0000 7fc78ced6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc788111100 con 0x7fc788102ef0 2024-09-19T08:15:50.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.839+0000 7fc78659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc788100e20 0x7fc78819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.839+0000 7fc78659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc788100e20 0x7fc78819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39890/0 (socket says 172.21.15.22:39890) 2024-09-19T08:15:50.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.839+0000 7fc78659c640 1 -- 172.21.15.22:0/205948282 learned_addr learned my addr 172.21.15.22:0/205948282 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:50.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.839+0000 7fc785d9b640 1 --2- 172.21.15.22:0/205948282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc788102570 0x7fc78819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.840+0000 7fc786d9d640 1 --2- 172.21.15.22:0/205948282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc788102ef0 0x7fc78819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.840+0000 7fc78659c640 1 -- 172.21.15.22:0/205948282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc788102ef0 msgr2=0x7fc78819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:50.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.840+0000 7fc78659c640 1 --2- 172.21.15.22:0/205948282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc788102ef0 0x7fc78819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.840+0000 7fc78659c640 1 -- 172.21.15.22:0/205948282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc788102570 msgr2=0x7fc78819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:50.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.840+0000 7fc78659c640 1 --2- 172.21.15.22:0/205948282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc788102570 0x7fc78819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.840+0000 7fc78659c640 1 -- 172.21.15.22:0/205948282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78819f620 con 0x7fc788100e20 2024-09-19T08:15:50.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.840+0000 7fc78659c640 1 --2- 172.21.15.22:0/205948282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc788100e20 0x7fc78819a670 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc77800d950 tx=0x7fc77800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:50.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.840+0000 7fc77f7fe640 1 -- 172.21.15.22:0/205948282 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778017070 con 0x7fc788100e20 2024-09-19T08:15:50.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.840+0000 7fc77f7fe640 1 -- 172.21.15.22:0/205948282 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc778009c30 con 0x7fc788100e20 2024-09-19T08:15:50.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.840+0000 7fc78ced6640 1 -- 172.21.15.22:0/205948282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78819f910 con 0x7fc788100e20 2024-09-19T08:15:50.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.840+0000 7fc77f7fe640 1 -- 172.21.15.22:0/205948282 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77800f6b0 con 0x7fc788100e20 2024-09-19T08:15:50.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.840+0000 7fc78ced6640 1 -- 172.21.15.22:0/205948282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7881a71f0 con 0x7fc788100e20 2024-09-19T08:15:50.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.843+0000 7fc77f7fe640 1 -- 172.21.15.22:0/205948282 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fc7780109d0 con 0x7fc788100e20 2024-09-19T08:15:50.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.844+0000 7fc78f2af740 1 Processor -- start 2024-09-19T08:15:50.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.844+0000 7fc78f2af740 1 -- start start 2024-09-19T08:15:50.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.844+0000 7fc77f7fe640 1 --2- 172.21.15.22:0/205948282 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc7740a34d0 0x7fc7740a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.844+0000 7fc78f2af740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5862b40 0x55e9b5861090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.844+0000 7fc78f2af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b58615d0 0x55e9b5861a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.844+0000 7fc78f2af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b586c370 0x55e9b586e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.844+0000 7fc78f2af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e9b57c72f0 con 0x55e9b58615d0 2024-09-19T08:15:50.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.844+0000 7fc78f2af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e9b576ecf0 con 0x55e9b586c370 2024-09-19T08:15:50.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.844+0000 7fc78f2af740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e9b57c32b0 con 0x55e9b5862b40 2024-09-19T08:15:50.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.844+0000 7fc786d9d640 1 --2- 172.21.15.22:0/205948282 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc7740a34d0 0x7fc7740a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.845+0000 7fc77f7fe640 1 -- 172.21.15.22:0/205948282 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc7780cfc50 con 0x7fc788100e20 2024-09-19T08:15:50.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.845+0000 7fc785d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b58615d0 0x55e9b5861a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.845+0000 7fc785d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b58615d0 0x55e9b5861a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39894/0 (socket says 172.21.15.22:39894) 2024-09-19T08:15:50.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.845+0000 7fc785d9b640 1 -- 172.21.15.22:0/119203098 learned_addr learned my addr 172.21.15.22:0/119203098 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:50.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.845+0000 7fc78659c640 1 --2- 172.21.15.22:0/119203098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5862b40 0x55e9b5861090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.845+0000 7fc786d9d640 1 --2- 172.21.15.22:0/119203098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b586c370 0x55e9b586e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.845+0000 7fc78659c640 1 -- 172.21.15.22:0/119203098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b586c370 msgr2=0x55e9b586e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:50.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.845+0000 7fc78659c640 1 --2- 172.21.15.22:0/119203098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b586c370 0x55e9b586e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.845+0000 7fc78659c640 1 -- 172.21.15.22:0/119203098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b58615d0 msgr2=0x55e9b5861a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:50.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.845+0000 7fc78659c640 1 --2- 172.21.15.22:0/119203098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b58615d0 0x55e9b5861a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.845+0000 7fc78659c640 1 -- 172.21.15.22:0/119203098 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b57533a0 con 0x55e9b5862b40 2024-09-19T08:15:50.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.845+0000 7fc786d9d640 1 --2- 172.21.15.22:0/205948282 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc7740a34d0 0x7fc7740a5990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc7800125e0 tx=0x7fc7800129e0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:50.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.845+0000 7fc78659c640 1 --2- 172.21.15.22:0/119203098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5862b40 0x55e9b5861090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc7780cd230 tx=0x7fc77800bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=872b2ff5ac08d3a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:50.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.846+0000 7fc77cff9640 1 -- 172.21.15.22:0/119203098 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778020070 con 0x55e9b5862b40 2024-09-19T08:15:50.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.846+0000 7fc77cff9640 1 -- 172.21.15.22:0/119203098 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc778009de0 con 0x55e9b5862b40 2024-09-19T08:15:50.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.846+0000 7fc77cff9640 1 -- 172.21.15.22:0/119203098 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778010460 con 0x55e9b5862b40 2024-09-19T08:15:50.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.846+0000 7fc78f2af740 1 -- 172.21.15.22:0/119203098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5862b40 msgr2=0x55e9b5861090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:50.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.846+0000 7fc78f2af740 1 --2- 172.21.15.22:0/119203098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5862b40 0x55e9b5861090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc7780cd230 tx=0x7fc77800bf60 comp rx=0 tx=0).stop 2024-09-19T08:15:50.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.847+0000 7fc78f2af740 1 -- 172.21.15.22:0/119203098 shutdown_connections 2024-09-19T08:15:50.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.847+0000 7fc78f2af740 1 --2- 172.21.15.22:0/119203098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b586c370 0x55e9b586e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.847+0000 7fc78f2af740 1 --2- 172.21.15.22:0/119203098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b58615d0 0x55e9b5861a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.847+0000 7fc78f2af740 1 --2- 172.21.15.22:0/119203098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5862b40 0x55e9b5861090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.847+0000 7fc78f2af740 1 -- 172.21.15.22:0/119203098 >> 172.21.15.22:0/119203098 conn(0x55e9b582a310 msgr2=0x55e9b585dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:50.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.847+0000 7fc78f2af740 1 -- 172.21.15.22:0/119203098 shutdown_connections 2024-09-19T08:15:50.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.847+0000 7fc78f2af740 1 -- 172.21.15.22:0/119203098 wait complete. 2024-09-19T08:15:50.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.847+0000 7fc78f2af740 1 -- start start 2024-09-19T08:15:50.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.847+0000 7fc78f2af740 1 Processor -- start 2024-09-19T08:15:50.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.847+0000 7fc78f2af740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b58615d0 0x55e9b586bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc78f2af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b5862b40 0x55e9b5869060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc78f2af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b586c370 0x55e9b58697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc78f2af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e9b586f070 con 0x55e9b586c370 2024-09-19T08:15:50.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc78f2af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e9b5782060 con 0x55e9b5862b40 2024-09-19T08:15:50.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc78f2af740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e9b586f1f0 con 0x55e9b58615d0 2024-09-19T08:15:50.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc78f2af740 10 client.0 _ll_register_callbacks cb 0x55e9b54d58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:50.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc786d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b586c370 0x55e9b58697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc786d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b586c370 0x55e9b58697f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39896/0 (socket says 172.21.15.22:39896) 2024-09-19T08:15:50.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc786d9d640 1 -- 172.21.15.22:0/3634533646 learned_addr learned my addr 172.21.15.22:0/3634533646 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:50.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc78659c640 1 --2- 172.21.15.22:0/3634533646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b58615d0 0x55e9b586bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc785d9b640 1 --2- 172.21.15.22:0/3634533646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b5862b40 0x55e9b5869060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc786d9d640 1 -- 172.21.15.22:0/3634533646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b58615d0 msgr2=0x55e9b586bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:50.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc786d9d640 1 --2- 172.21.15.22:0/3634533646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b58615d0 0x55e9b586bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc786d9d640 1 -- 172.21.15.22:0/3634533646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b5862b40 msgr2=0x55e9b5869060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:50.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc786d9d640 1 --2- 172.21.15.22:0/3634533646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b5862b40 0x55e9b5869060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc786d9d640 1 -- 172.21.15.22:0/3634533646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b55383c0 con 0x55e9b586c370 2024-09-19T08:15:50.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc786d9d640 1 --2- 172.21.15.22:0/3634533646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b586c370 0x55e9b58697f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc780010de0 tx=0x7fc780004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0c3021962865dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:50.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc76dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:50.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.848+0000 7fc76dffb640 1 -- 172.21.15.22:0/3634533646 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780004b20 con 0x55e9b586c370 2024-09-19T08:15:50.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.849+0000 7fc76dffb640 1 -- 172.21.15.22:0/3634533646 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc780004cc0 con 0x55e9b586c370 2024-09-19T08:15:50.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.849+0000 7fc78f2af740 1 -- 172.21.15.22:0/3634533646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc778009de0 con 0x55e9b586c370 2024-09-19T08:15:50.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.849+0000 7fc76dffb640 1 -- 172.21.15.22:0/3634533646 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc780005020 con 0x55e9b586c370 2024-09-19T08:15:50.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.849+0000 7fc78f2af740 10 client.15522 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:50.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.849+0000 7fc78f2af740 1 -- 172.21.15.22:0/3634533646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e9b587ab60 con 0x55e9b586c370 2024-09-19T08:15:50.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.849+0000 7fc76dffb640 1 -- 172.21.15.22:0/3634533646 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc778009de0 con 0x55e9b586c370 2024-09-19T08:15:50.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.849+0000 7fc76dffb640 1 -- 172.21.15.22:0/3634533646 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc7800286e0 con 0x55e9b586c370 2024-09-19T08:15:50.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.849+0000 7fc78f2af740 10 client.15522 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:50.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.849+0000 7fc78f2af740 10 client.15522 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:15:50.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.849+0000 7fc78f2af740 20 client.15522 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:50.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.849+0000 7fc78f2af740 1 --2- 172.21.15.22:0/3634533646 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55e9b587c020 0x55e9b587e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.849+0000 7fc78f2af740 4 client.15522 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:15:50.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.849+0000 7fc78f2af740 1 -- 172.21.15.22:0/3634533646 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9b57d4860 con 0x55e9b587c020 2024-09-19T08:15:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.850+0000 7fc78659c640 1 --2- 172.21.15.22:0/3634533646 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55e9b587c020 0x55e9b587e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.850+0000 7fc76dffb640 10 client.15522 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:15:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.850+0000 7fc78659c640 1 --2- 172.21.15.22:0/3634533646 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55e9b587c020 0x55e9b587e410 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a063410b2722973 server_cookie=7fe6f592286ebf4a in_seq=0 out_seq=0 2024-09-19T08:15:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc76dffb640 1 -- 172.21.15.22:0/3634533646 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9b57d4860 con 0x55e9b587c020 2024-09-19T08:15:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc76dffb640 10 client.15522 handle_command_reply: tid=0 2024-09-19T08:15:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc78f2af740 1 client.15522 shutdown 2024-09-19T08:15:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc78f2af740 1 -- 172.21.15.22:0/3634533646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b586c370 msgr2=0x55e9b58697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc78f2af740 1 --2- 172.21.15.22:0/3634533646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b586c370 0x55e9b58697f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc780010de0 tx=0x7fc780004ab0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc78f2af740 1 -- 172.21.15.22:0/3634533646 shutdown_connections 2024-09-19T08:15:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc78f2af740 1 --2- 172.21.15.22:0/3634533646 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55e9b587c020 0x55e9b587e410 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc78f2af740 1 --2- 172.21.15.22:0/3634533646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b586c370 0x55e9b58697f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc78f2af740 1 --2- 172.21.15.22:0/3634533646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b5862b40 0x55e9b5869060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc78f2af740 1 --2- 172.21.15.22:0/3634533646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b58615d0 0x55e9b586bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc78f2af740 1 -- 172.21.15.22:0/3634533646 >> 172.21.15.22:0/3634533646 conn(0x55e9b582a310 msgr2=0x55e9b5863960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc76dffb640 0 client.15522 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:15:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc78f2af740 1 -- 172.21.15.22:0/3634533646 shutdown_connections 2024-09-19T08:15:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc78f2af740 1 -- 172.21.15.22:0/3634533646 wait complete. 2024-09-19T08:15:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.851+0000 7fc78f2af740 20 client.15522 trim_cache size 0 max 16384 2024-09-19T08:15:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc78f2af740 1 Processor -- start 2024-09-19T08:15:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc78f2af740 1 -- start start 2024-09-19T08:15:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc78f2af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b58615d0 0x55e9b586afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc78f2af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b5862b40 0x55e9b5864120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc78f2af740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b586c370 0x55e9b5864660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc78f2af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc7881b22d0 con 0x55e9b5862b40 2024-09-19T08:15:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc78f2af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc7881afba0 con 0x55e9b58615d0 2024-09-19T08:15:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc78f2af740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc7881ad9f0 con 0x55e9b586c370 2024-09-19T08:15:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc785d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b5862b40 0x55e9b5864120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc785d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b5862b40 0x55e9b5864120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39900/0 (socket says 172.21.15.22:39900) 2024-09-19T08:15:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc785d9b640 1 -- 172.21.15.22:0/602238188 learned_addr learned my addr 172.21.15.22:0/602238188 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc78659c640 1 --2- 172.21.15.22:0/602238188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b58615d0 0x55e9b586afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc786d9d640 1 --2- 172.21.15.22:0/602238188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b586c370 0x55e9b5864660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc785d9b640 1 -- 172.21.15.22:0/602238188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b586c370 msgr2=0x55e9b5864660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc785d9b640 1 --2- 172.21.15.22:0/602238188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b586c370 0x55e9b5864660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc785d9b640 1 -- 172.21.15.22:0/602238188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b58615d0 msgr2=0x55e9b586afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc785d9b640 1 --2- 172.21.15.22:0/602238188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b58615d0 0x55e9b586afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.866+0000 7fc785d9b640 1 -- 172.21.15.22:0/602238188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b585d4f0 con 0x55e9b5862b40 2024-09-19T08:15:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.867+0000 7fc785d9b640 1 --2- 172.21.15.22:0/602238188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b5862b40 0x55e9b5864120 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc770006010 tx=0x7fc7700043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc0f2cd4206bc9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.867+0000 7fc76ffff640 1 -- 172.21.15.22:0/602238188 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770035040 con 0x55e9b5862b40 2024-09-19T08:15:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.867+0000 7fc76ffff640 1 -- 172.21.15.22:0/602238188 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc770004750 con 0x55e9b5862b40 2024-09-19T08:15:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.867+0000 7fc76ffff640 1 -- 172.21.15.22:0/602238188 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7700038c0 con 0x55e9b5862b40 2024-09-19T08:15:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.868+0000 7fc78f2af740 1 -- 172.21.15.22:0/602238188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b5862b40 msgr2=0x55e9b5864120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.868+0000 7fc78f2af740 1 --2- 172.21.15.22:0/602238188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b5862b40 0x55e9b5864120 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc770006010 tx=0x7fc7700043e0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.868+0000 7fc78f2af740 1 -- 172.21.15.22:0/602238188 shutdown_connections 2024-09-19T08:15:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.868+0000 7fc78f2af740 1 --2- 172.21.15.22:0/602238188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b586c370 0x55e9b5864660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.868+0000 7fc78f2af740 1 --2- 172.21.15.22:0/602238188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b5862b40 0x55e9b5864120 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.868+0000 7fc78f2af740 1 --2- 172.21.15.22:0/602238188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b58615d0 0x55e9b586afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.868+0000 7fc78f2af740 1 -- 172.21.15.22:0/602238188 >> 172.21.15.22:0/602238188 conn(0x55e9b582a310 msgr2=0x55e9b586a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.868+0000 7fc78f2af740 1 -- 172.21.15.22:0/602238188 shutdown_connections 2024-09-19T08:15:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.868+0000 7fc78f2af740 1 -- 172.21.15.22:0/602238188 wait complete. 2024-09-19T08:15:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.868+0000 7fc78f2af740 1 -- start start 2024-09-19T08:15:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc78f2af740 1 Processor -- start 2024-09-19T08:15:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc78f2af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b58615d0 0x55e9b58708d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc78f2af740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5862b40 0x55e9b5870e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc78f2af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b586c370 0x55e9b58e5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc78f2af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc7881ae060 con 0x55e9b58615d0 2024-09-19T08:15:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc78f2af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc7881af4d0 con 0x55e9b586c370 2024-09-19T08:15:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc78f2af740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc7881ae730 con 0x55e9b5862b40 2024-09-19T08:15:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc78659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b58615d0 0x55e9b58708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc78f2af740 10 client.0 _ll_register_callbacks cb 0x7fc780010ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:50.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc78659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b58615d0 0x55e9b58708d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39912/0 (socket says 172.21.15.22:39912) 2024-09-19T08:15:50.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc78659c640 1 -- 172.21.15.22:0/3276715959 learned_addr learned my addr 172.21.15.22:0/3276715959 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:50.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc786d9d640 1 --2- 172.21.15.22:0/3276715959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b586c370 0x55e9b58e5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc785d9b640 1 --2- 172.21.15.22:0/3276715959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5862b40 0x55e9b5870e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc78659c640 1 -- 172.21.15.22:0/3276715959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5862b40 msgr2=0x55e9b5870e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.869+0000 7fc78659c640 1 --2- 172.21.15.22:0/3276715959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5862b40 0x55e9b5870e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.870+0000 7fc78659c640 1 -- 172.21.15.22:0/3276715959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b586c370 msgr2=0x55e9b58e5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.870+0000 7fc78659c640 1 --2- 172.21.15.22:0/3276715959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b586c370 0x55e9b58e5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.870+0000 7fc78659c640 1 -- 172.21.15.22:0/3276715959 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b58e61f0 con 0x55e9b58615d0 2024-09-19T08:15:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.870+0000 7fc78659c640 1 --2- 172.21.15.22:0/3276715959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b58615d0 0x55e9b58708d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc778009610 tx=0x7fc77809b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee45b07a89777974 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.870+0000 7fc76dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.870+0000 7fc76dffb640 1 -- 172.21.15.22:0/3276715959 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778014030 con 0x55e9b58615d0 2024-09-19T08:15:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.870+0000 7fc76dffb640 1 -- 172.21.15.22:0/3276715959 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc7780cd510 con 0x55e9b58615d0 2024-09-19T08:15:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.870+0000 7fc78f2af740 1 -- 172.21.15.22:0/3276715959 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc770004750 con 0x55e9b58615d0 2024-09-19T08:15:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.870+0000 7fc76dffb640 1 -- 172.21.15.22:0/3276715959 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc778020040 con 0x55e9b58615d0 2024-09-19T08:15:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.870+0000 7fc78f2af740 10 client.15528 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.871+0000 7fc78f2af740 1 -- 172.21.15.22:0/3276715959 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e9b58e7490 con 0x55e9b58615d0 2024-09-19T08:15:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.871+0000 7fc76dffb640 1 -- 172.21.15.22:0/3276715959 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc770004750 con 0x55e9b58615d0 2024-09-19T08:15:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.871+0000 7fc76dffb640 1 -- 172.21.15.22:0/3276715959 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc77809b140 con 0x55e9b58615d0 2024-09-19T08:15:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.871+0000 7fc78f2af740 10 client.15528 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.871+0000 7fc78f2af740 10 client.15528 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:15:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.871+0000 7fc78f2af740 20 client.15528 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.871+0000 7fc78f2af740 1 --2- 172.21.15.22:0/3276715959 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55e9b58e7770 0x55e9b58e9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.871+0000 7fc78f2af740 4 client.15528 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:15:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.871+0000 7fc78f2af740 1 -- 172.21.15.22:0/3276715959 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9b57ced70 con 0x55e9b58e7770 2024-09-19T08:15:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.871+0000 7fc785d9b640 1 --2- 172.21.15.22:0/3276715959 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55e9b58e7770 0x55e9b58e9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.872+0000 7fc785d9b640 1 --2- 172.21.15.22:0/3276715959 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55e9b58e7770 0x55e9b58e9b60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=747684154fdaee6c server_cookie=e5ee0860c51b8b61 in_seq=0 out_seq=0 2024-09-19T08:15:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.872+0000 7fc76dffb640 10 client.15528 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:15:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.872+0000 7fc76dffb640 1 -- 172.21.15.22:0/3276715959 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9b57ced70 con 0x55e9b58e7770 2024-09-19T08:15:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.872+0000 7fc76dffb640 10 client.15528 handle_command_reply: tid=0 2024-09-19T08:15:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.872+0000 7fc78f2af740 1 client.15528 shutdown 2024-09-19T08:15:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.872+0000 7fc78f2af740 1 -- 172.21.15.22:0/3276715959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b58615d0 msgr2=0x55e9b58708d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.872+0000 7fc78f2af740 1 --2- 172.21.15.22:0/3276715959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b58615d0 0x55e9b58708d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc778009610 tx=0x7fc77809b960 comp rx=0 tx=0).stop 2024-09-19T08:15:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.872+0000 7fc78f2af740 1 -- 172.21.15.22:0/3276715959 shutdown_connections 2024-09-19T08:15:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.872+0000 7fc78f2af740 1 --2- 172.21.15.22:0/3276715959 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55e9b58e7770 0x55e9b58e9b60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.872+0000 7fc78f2af740 1 --2- 172.21.15.22:0/3276715959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9b586c370 0x55e9b58e5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.872+0000 7fc76dffb640 0 client.15528 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:15:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.873+0000 7fc78f2af740 1 --2- 172.21.15.22:0/3276715959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9b5862b40 0x55e9b5870e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.873+0000 7fc78f2af740 1 --2- 172.21.15.22:0/3276715959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9b58615d0 0x55e9b58708d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.873+0000 7fc78f2af740 1 -- 172.21.15.22:0/3276715959 >> 172.21.15.22:0/3276715959 conn(0x55e9b582a310 msgr2=0x55e9b585cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.873+0000 7fc78f2af740 1 -- 172.21.15.22:0/3276715959 shutdown_connections 2024-09-19T08:15:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.873+0000 7fc78f2af740 1 -- 172.21.15.22:0/3276715959 wait complete. 2024-09-19T08:15:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.873+0000 7fc78f2af740 20 client.15528 trim_cache size 0 max 16384 2024-09-19T08:15:50.880 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:15:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.873+0000 7fc78ced6640 1 -- 172.21.15.22:0/205948282 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc7740a34d0 msgr2=0x7fc7740a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.873+0000 7fc78ced6640 1 --2- 172.21.15.22:0/205948282 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc7740a34d0 0x7fc7740a5990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc7800125e0 tx=0x7fc7800129e0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.873+0000 7fc78ced6640 1 -- 172.21.15.22:0/205948282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc788100e20 msgr2=0x7fc78819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.874+0000 7fc78ced6640 1 --2- 172.21.15.22:0/205948282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc788100e20 0x7fc78819a670 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc77800d950 tx=0x7fc77800de10 comp rx=0 tx=0).stop 2024-09-19T08:15:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.874+0000 7fc78ced6640 1 -- 172.21.15.22:0/205948282 shutdown_connections 2024-09-19T08:15:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.874+0000 7fc78ced6640 1 --2- 172.21.15.22:0/205948282 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc7740a34d0 0x7fc7740a5990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.874+0000 7fc78ced6640 1 --2- 172.21.15.22:0/205948282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc788102ef0 0x7fc78819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.874+0000 7fc78ced6640 1 --2- 172.21.15.22:0/205948282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc788102570 0x7fc78819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.874+0000 7fc78ced6640 1 --2- 172.21.15.22:0/205948282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc788100e20 0x7fc78819a670 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.874+0000 7fc78ced6640 1 -- 172.21.15.22:0/205948282 >> 172.21.15.22:0/205948282 conn(0x7fc7880fc7d0 msgr2=0x7fc78806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.874+0000 7fc78ced6640 1 -- 172.21.15.22:0/205948282 shutdown_connections 2024-09-19T08:15:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:50.874+0000 7fc78ced6640 1 -- 172.21.15.22:0/205948282 wait complete. 2024-09-19T08:15:51.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:50 smithi157 ceph-mon[35143]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T08:15:51.754 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:15:52.017 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:52.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.513+0000 7fad1b411640 1 -- 172.21.15.22:0/3563394998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad14101410 msgr2=0x7fad1410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:52.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.513+0000 7fad1b411640 1 --2- 172.21.15.22:0/3563394998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad14101410 0x7fad1410d940 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009cd0 tx=0x7fad0c02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.513+0000 7fad1b411640 1 -- 172.21.15.22:0/3563394998 shutdown_connections 2024-09-19T08:15:52.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.513+0000 7fad1b411640 1 --2- 172.21.15.22:0/3563394998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad14101410 0x7fad1410d940 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.513+0000 7fad1b411640 1 --2- 172.21.15.22:0/3563394998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad14100a70 0x7fad14100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.513+0000 7fad1b411640 1 --2- 172.21.15.22:0/3563394998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad14106a70 0x7fad14106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.513+0000 7fad1b411640 1 -- 172.21.15.22:0/3563394998 >> 172.21.15.22:0/3563394998 conn(0x7fad140fc790 msgr2=0x7fad140febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:52.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.514+0000 7fad1b411640 1 -- 172.21.15.22:0/3563394998 shutdown_connections 2024-09-19T08:15:52.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.514+0000 7fad1b411640 1 -- 172.21.15.22:0/3563394998 wait complete. 2024-09-19T08:15:52.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.514+0000 7fad1b411640 1 Processor -- start 2024-09-19T08:15:52.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.514+0000 7fad1b411640 1 -- start start 2024-09-19T08:15:52.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad1b411640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad14100a70 0x7fad14198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad1b411640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad14101410 0x7fad14198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad1b411640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad14106a70 0x7fad1419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad1b411640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fad14110e70 con 0x7fad14106a70 2024-09-19T08:15:52.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad1b411640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad14110cf0 con 0x7fad14101410 2024-09-19T08:15:52.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad1b411640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fad14110ff0 con 0x7fad14100a70 2024-09-19T08:15:52.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad199ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad14106a70 0x7fad1419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad189ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad14101410 0x7fad14198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad199ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad14106a70 0x7fad1419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39918/0 (socket says 172.21.15.22:39918) 2024-09-19T08:15:52.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad199ae640 1 -- 172.21.15.22:0/3525703008 learned_addr learned my addr 172.21.15.22:0/3525703008 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:52.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad191ad640 1 --2- 172.21.15.22:0/3525703008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad14100a70 0x7fad14198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad199ae640 1 -- 172.21.15.22:0/3525703008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad14100a70 msgr2=0x7fad14198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:52.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad199ae640 1 --2- 172.21.15.22:0/3525703008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad14100a70 0x7fad14198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad199ae640 1 -- 172.21.15.22:0/3525703008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad14101410 msgr2=0x7fad14198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:52.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad199ae640 1 --2- 172.21.15.22:0/3525703008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad14101410 0x7fad14198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad199ae640 1 -- 172.21.15.22:0/3525703008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1419d3c0 con 0x7fad14106a70 2024-09-19T08:15:52.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.515+0000 7fad199ae640 1 --2- 172.21.15.22:0/3525703008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad14106a70 0x7fad1419cce0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fad0c005cc0 tx=0x7fad0c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:52.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.516+0000 7fad0a7fc640 1 -- 172.21.15.22:0/3525703008 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c004040 con 0x7fad14106a70 2024-09-19T08:15:52.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.516+0000 7fad1b411640 1 -- 172.21.15.22:0/3525703008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1419d650 con 0x7fad14106a70 2024-09-19T08:15:52.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.516+0000 7fad0a7fc640 1 -- 172.21.15.22:0/3525703008 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad0c0046c0 con 0x7fad14106a70 2024-09-19T08:15:52.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.516+0000 7fad0a7fc640 1 -- 172.21.15.22:0/3525703008 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c046480 con 0x7fad14106a70 2024-09-19T08:15:52.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.516+0000 7fad1b411640 1 -- 172.21.15.22:0/3525703008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad141a4f90 con 0x7fad14106a70 2024-09-19T08:15:52.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.519+0000 7fad0a7fc640 1 -- 172.21.15.22:0/3525703008 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fad0c0305a0 con 0x7fad14106a70 2024-09-19T08:15:52.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.519+0000 7fad1d7ea740 1 Processor -- start 2024-09-19T08:15:52.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.519+0000 7fad1d7ea740 1 -- start start 2024-09-19T08:15:52.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.519+0000 7fad1d7ea740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b25b40 0x55c510b24090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.519+0000 7fad1d7ea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b245d0 0x55c510b24a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.519+0000 7fad1d7ea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b33780 0x55c510b35b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.519+0000 7fad1d7ea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c510a8a2f0 con 0x55c510b33780 2024-09-19T08:15:52.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.519+0000 7fad1d7ea740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c510a31cf0 con 0x55c510b25b40 2024-09-19T08:15:52.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.519+0000 7fad1d7ea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c510a862b0 con 0x55c510b245d0 2024-09-19T08:15:52.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.519+0000 7fad189ac640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b33780 0x55c510b35b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.519+0000 7fad191ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b245d0 0x55c510b24a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.520+0000 7fad189ac640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b33780 0x55c510b35b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39926/0 (socket says 172.21.15.22:39926) 2024-09-19T08:15:52.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.520+0000 7fad189ac640 1 -- 172.21.15.22:0/895001763 learned_addr learned my addr 172.21.15.22:0/895001763 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:52.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.520+0000 7fad189ac640 1 --2- 172.21.15.22:0/895001763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b25b40 0x55c510b24090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.520+0000 7fad0a7fc640 1 --2- 172.21.15.22:0/3525703008 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fad000a34d0 0x7fad000a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.520+0000 7fad191ad640 1 -- 172.21.15.22:0/895001763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b25b40 msgr2=0x55c510b24090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:52.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.520+0000 7fad191ad640 1 --2- 172.21.15.22:0/895001763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b25b40 0x55c510b24090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.520+0000 7fad191ad640 1 -- 172.21.15.22:0/895001763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b33780 msgr2=0x55c510b35b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:52.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.520+0000 7fad191ad640 1 --2- 172.21.15.22:0/895001763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b33780 0x55c510b35b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.520+0000 7fad191ad640 1 -- 172.21.15.22:0/895001763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c510a163a0 con 0x55c510b245d0 2024-09-19T08:15:52.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.520+0000 7fad189ac640 1 --2- 172.21.15.22:0/895001763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b33780 0x55c510b35b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:52.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.520+0000 7fad0a7fc640 1 -- 172.21.15.22:0/3525703008 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fad0c02e030 con 0x7fad14106a70 2024-09-19T08:15:52.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.520+0000 7fad199ae640 1 --2- 172.21.15.22:0/3525703008 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fad000a34d0 0x7fad000a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.520+0000 7fad191ad640 1 --2- 172.21.15.22:0/895001763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b245d0 0x55c510b24a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fad1000bb70 tx=0x7fad100107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dc0a41badac3174 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:52.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.521+0000 7fad199ae640 1 --2- 172.21.15.22:0/3525703008 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fad000a34d0 0x7fad000a5990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0eed80 tx=0x7fad0c02a8e0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:52.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.521+0000 7facff7fe640 1 -- 172.21.15.22:0/895001763 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10004030 con 0x55c510b245d0 2024-09-19T08:15:52.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.521+0000 7facff7fe640 1 -- 172.21.15.22:0/895001763 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad100041d0 con 0x55c510b245d0 2024-09-19T08:15:52.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.521+0000 7facff7fe640 1 -- 172.21.15.22:0/895001763 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad100116a0 con 0x55c510b245d0 2024-09-19T08:15:52.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.522+0000 7fad1d7ea740 1 -- 172.21.15.22:0/895001763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b245d0 msgr2=0x55c510b24a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:52.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.522+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/895001763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b245d0 0x55c510b24a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fad1000bb70 tx=0x7fad100107a0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.522+0000 7fad1d7ea740 1 -- 172.21.15.22:0/895001763 shutdown_connections 2024-09-19T08:15:52.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.522+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/895001763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b33780 0x55c510b35b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.522+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/895001763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b245d0 0x55c510b24a10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.522+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/895001763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b25b40 0x55c510b24090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.522+0000 7fad1d7ea740 1 -- 172.21.15.22:0/895001763 >> 172.21.15.22:0/895001763 conn(0x55c510aed310 msgr2=0x55c510b20c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:52.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.522+0000 7fad1d7ea740 1 -- 172.21.15.22:0/895001763 shutdown_connections 2024-09-19T08:15:52.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.522+0000 7fad1d7ea740 1 -- 172.21.15.22:0/895001763 wait complete. 2024-09-19T08:15:52.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.522+0000 7fad1d7ea740 1 -- start start 2024-09-19T08:15:52.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.523+0000 7fad1d7ea740 1 Processor -- start 2024-09-19T08:15:52.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.523+0000 7fad1d7ea740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b245d0 0x55c510b331a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.523+0000 7fad1d7ea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b25b40 0x55c510b30450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.523+0000 7fad1d7ea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b33780 0x55c510b30a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.523+0000 7fad1d7ea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c510b36480 con 0x55c510b33780 2024-09-19T08:15:52.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.523+0000 7fad1d7ea740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c510a45060 con 0x55c510b245d0 2024-09-19T08:15:52.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.523+0000 7fad1d7ea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c510b36600 con 0x55c510b25b40 2024-09-19T08:15:52.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.523+0000 7fad1d7ea740 10 client.0 _ll_register_callbacks cb 0x55c5107988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:52.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.523+0000 7fad189ac640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b33780 0x55c510b30a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.523+0000 7fad189ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b245d0 0x55c510b331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.523+0000 7fad191ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b25b40 0x55c510b30450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.523+0000 7fad189ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b245d0 0x55c510b331a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:37242/0 (socket says 172.21.15.22:37242) 2024-09-19T08:15:52.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.523+0000 7fad189ac640 1 -- 172.21.15.22:0/2593087611 learned_addr learned my addr 172.21.15.22:0/2593087611 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:52.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.524+0000 7fad189ac640 1 -- 172.21.15.22:0/2593087611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b25b40 msgr2=0x55c510b30450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:52.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.524+0000 7fad189ac640 1 --2- 172.21.15.22:0/2593087611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b25b40 0x55c510b30450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.524+0000 7fad189ac640 1 -- 172.21.15.22:0/2593087611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b33780 msgr2=0x55c510b30a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:52.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.524+0000 7fad189ac640 1 --2- 172.21.15.22:0/2593087611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b33780 0x55c510b30a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.524+0000 7fad189ac640 1 -- 172.21.15.22:0/2593087611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5107fb3c0 con 0x55c510b245d0 2024-09-19T08:15:52.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.524+0000 7fad191ad640 1 --2- 172.21.15.22:0/2593087611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b25b40 0x55c510b30450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:52.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.524+0000 7fad189ac640 1 --2- 172.21.15.22:0/2593087611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b245d0 0x55c510b331a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad04012d90 tx=0x7fad0400b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac0778ad8cec684d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:52.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.524+0000 7facfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:15:52.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.524+0000 7facfcff9640 1 -- 172.21.15.22:0/2593087611 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04004050 con 0x55c510b245d0 2024-09-19T08:15:52.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.524+0000 7facfcff9640 1 -- 172.21.15.22:0/2593087611 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fad040041f0 con 0x55c510b245d0 2024-09-19T08:15:52.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.524+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2593087611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad100041d0 con 0x55c510b245d0 2024-09-19T08:15:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.524+0000 7facfcff9640 1 -- 172.21.15.22:0/2593087611 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fad0402c050 con 0x55c510b245d0 2024-09-19T08:15:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.524+0000 7fad1d7ea740 10 client.25223 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.524+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2593087611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c510b42000 con 0x55c510b245d0 2024-09-19T08:15:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.525+0000 7facfcff9640 1 -- 172.21.15.22:0/2593087611 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad100041d0 con 0x55c510b245d0 2024-09-19T08:15:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.525+0000 7facfcff9640 1 -- 172.21.15.22:0/2593087611 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fad040243f0 con 0x55c510b245d0 2024-09-19T08:15:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.525+0000 7fad1d7ea740 10 client.25223 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.525+0000 7fad1d7ea740 10 client.25223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:15:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.525+0000 7fad1d7ea740 20 client.25223 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.525+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2593087611 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c510b43840 0x55c510b45cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.525+0000 7fad1d7ea740 4 client.25223 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:15:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.525+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2593087611 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c510a97860 con 0x55c510b43840 2024-09-19T08:15:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.525+0000 7fad191ad640 1 --2- 172.21.15.22:0/2593087611 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c510b43840 0x55c510b45cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.526+0000 7facfcff9640 10 client.25223 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.526+0000 7fad191ad640 1 --2- 172.21.15.22:0/2593087611 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c510b43840 0x55c510b45cc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f78aa225ff102109 server_cookie=9cd5df654725f042 in_seq=0 out_seq=0 2024-09-19T08:15:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.526+0000 7facfcff9640 1 -- 172.21.15.22:0/2593087611 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c510a97860 con 0x55c510b43840 2024-09-19T08:15:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.526+0000 7facfcff9640 10 client.25223 handle_command_reply: tid=0 2024-09-19T08:15:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.527+0000 7fad1d7ea740 1 client.25223 shutdown 2024-09-19T08:15:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.527+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2593087611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b245d0 msgr2=0x55c510b331a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.527+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2593087611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b245d0 0x55c510b331a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad04012d90 tx=0x7fad0400b7c0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.527+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2593087611 shutdown_connections 2024-09-19T08:15:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.527+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2593087611 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c510b43840 0x55c510b45cc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.527+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2593087611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b33780 0x55c510b30a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.527+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2593087611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b25b40 0x55c510b30450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.527+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2593087611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b245d0 0x55c510b331a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.527+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2593087611 >> 172.21.15.22:0/2593087611 conn(0x55c510aed310 msgr2=0x55c510b26960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.527+0000 7facfcff9640 0 client.25223 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.527+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2593087611 shutdown_connections 2024-09-19T08:15:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.527+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2593087611 wait complete. 2024-09-19T08:15:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.527+0000 7fad1d7ea740 20 client.25223 trim_cache size 0 max 16384 2024-09-19T08:15:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.540+0000 7fad1d7ea740 1 Processor -- start 2024-09-19T08:15:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.540+0000 7fad1d7ea740 1 -- start start 2024-09-19T08:15:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.540+0000 7fad1d7ea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b245d0 0x55c510b3f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.540+0000 7fad1d7ea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b25b40 0x55c510b2fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.540+0000 7fad1d7ea740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b33780 0x55c510b2b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.540+0000 7fad1d7ea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fad141ade80 con 0x55c510b25b40 2024-09-19T08:15:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.540+0000 7fad1d7ea740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad141ab750 con 0x55c510b33780 2024-09-19T08:15:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.540+0000 7fad1d7ea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fad141a95a0 con 0x55c510b245d0 2024-09-19T08:15:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.540+0000 7fad191ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b25b40 0x55c510b2fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.540+0000 7fad191ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b25b40 0x55c510b2fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39946/0 (socket says 172.21.15.22:39946) 2024-09-19T08:15:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.540+0000 7fad191ad640 1 -- 172.21.15.22:0/2697179457 learned_addr learned my addr 172.21.15.22:0/2697179457 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.540+0000 7fad189ac640 1 --2- 172.21.15.22:0/2697179457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b33780 0x55c510b2b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.540+0000 7fad189ac640 1 --2- 172.21.15.22:0/2697179457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b245d0 0x55c510b3f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.541+0000 7fad191ad640 1 -- 172.21.15.22:0/2697179457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b245d0 msgr2=0x55c510b3f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.541+0000 7fad191ad640 1 --2- 172.21.15.22:0/2697179457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b245d0 0x55c510b3f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.541+0000 7fad191ad640 1 -- 172.21.15.22:0/2697179457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b33780 msgr2=0x55c510b2b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.541+0000 7fad191ad640 1 --2- 172.21.15.22:0/2697179457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b33780 0x55c510b2b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.541+0000 7fad191ad640 1 -- 172.21.15.22:0/2697179457 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c510b253a0 con 0x55c510b25b40 2024-09-19T08:15:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.541+0000 7fad189ac640 1 --2- 172.21.15.22:0/2697179457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b245d0 0x55c510b3f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:15:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.541+0000 7fad191ad640 1 --2- 172.21.15.22:0/2697179457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b25b40 0x55c510b2fd10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fad1001a000 tx=0x7fad10003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a052091e5621bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.541+0000 7facfeffd640 1 -- 172.21.15.22:0/2697179457 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1002c020 con 0x55c510b25b40 2024-09-19T08:15:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.541+0000 7facfeffd640 1 -- 172.21.15.22:0/2697179457 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad10005520 con 0x55c510b25b40 2024-09-19T08:15:52.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.541+0000 7facfeffd640 1 -- 172.21.15.22:0/2697179457 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad100116a0 con 0x55c510b25b40 2024-09-19T08:15:52.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.542+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2697179457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b25b40 msgr2=0x55c510b2fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:52.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.542+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2697179457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b25b40 0x55c510b2fd10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fad1001a000 tx=0x7fad10003e30 comp rx=0 tx=0).stop 2024-09-19T08:15:52.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.542+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2697179457 shutdown_connections 2024-09-19T08:15:52.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.542+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2697179457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b33780 0x55c510b2b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.542+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2697179457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b25b40 0x55c510b2fd10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.542+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2697179457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b245d0 0x55c510b3f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.542+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2697179457 >> 172.21.15.22:0/2697179457 conn(0x55c510aed310 msgr2=0x55c510b20700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.542+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2697179457 shutdown_connections 2024-09-19T08:15:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.542+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2697179457 wait complete. 2024-09-19T08:15:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.542+0000 7fad1d7ea740 1 -- start start 2024-09-19T08:15:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.542+0000 7fad1d7ea740 1 Processor -- start 2024-09-19T08:15:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad1d7ea740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b245d0 0x55c510b4a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad1d7ea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b25b40 0x55c510b4ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad1d7ea740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b33780 0x55c510b4b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad1d7ea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fad141a9c10 con 0x55c510b25b40 2024-09-19T08:15:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad1d7ea740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad141ab080 con 0x55c510b245d0 2024-09-19T08:15:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad1d7ea740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fad141aa2e0 con 0x55c510b33780 2024-09-19T08:15:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad1d7ea740 10 client.0 _ll_register_callbacks cb 0x7fad0400bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:52.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad191ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b25b40 0x55c510b4ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad191ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b25b40 0x55c510b4ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39962/0 (socket says 172.21.15.22:39962) 2024-09-19T08:15:52.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad191ad640 1 -- 172.21.15.22:0/2294063466 learned_addr learned my addr 172.21.15.22:0/2294063466 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:52.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad189ac640 1 --2- 172.21.15.22:0/2294063466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b245d0 0x55c510b4a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad189ac640 1 --2- 172.21.15.22:0/2294063466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b33780 0x55c510b4b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad191ad640 1 -- 172.21.15.22:0/2294063466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b33780 msgr2=0x55c510b4b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:52.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad191ad640 1 --2- 172.21.15.22:0/2294063466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b33780 0x55c510b4b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad191ad640 1 -- 172.21.15.22:0/2294063466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b245d0 msgr2=0x55c510b4a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:52.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad191ad640 1 --2- 172.21.15.22:0/2294063466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b245d0 0x55c510b4a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad191ad640 1 -- 172.21.15.22:0/2294063466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c510b4bca0 con 0x55c510b25b40 2024-09-19T08:15:52.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad189ac640 1 --2- 172.21.15.22:0/2294063466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b245d0 0x55c510b4a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:52.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7fad191ad640 1 --2- 172.21.15.22:0/2294063466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b25b40 0x55c510b4ace0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fad100057a0 tx=0x7fad1000ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3afc903e4b0f341 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:52.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.543+0000 7facfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:52.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.544+0000 7facfcff9640 1 -- 172.21.15.22:0/2294063466 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1002c040 con 0x55c510b25b40 2024-09-19T08:15:52.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.544+0000 7facfcff9640 1 -- 172.21.15.22:0/2294063466 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fad10005240 con 0x55c510b25b40 2024-09-19T08:15:52.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.544+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2294063466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad10005520 con 0x55c510b25b40 2024-09-19T08:15:52.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.544+0000 7facfcff9640 1 -- 172.21.15.22:0/2294063466 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fad100035f0 con 0x55c510b25b40 2024-09-19T08:15:52.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.544+0000 7fad1d7ea740 10 client.15546 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:52.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.544+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2294063466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c510b3b380 con 0x55c510b25b40 2024-09-19T08:15:52.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.544+0000 7facfcff9640 1 -- 172.21.15.22:0/2294063466 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad10005520 con 0x55c510b25b40 2024-09-19T08:15:52.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.544+0000 7facfcff9640 1 -- 172.21.15.22:0/2294063466 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fad10038030 con 0x55c510b25b40 2024-09-19T08:15:52.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.544+0000 7fad1d7ea740 10 client.15546 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:52.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.544+0000 7fad1d7ea740 10 client.15546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:15:52.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.544+0000 7fad1d7ea740 20 client.15546 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:52.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.544+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2294063466 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c510b3b890 0x55c510b3bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:52.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.544+0000 7fad1d7ea740 4 client.15546 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:15:52.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.544+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2294063466 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c510a91d70 con 0x55c510b3b890 2024-09-19T08:15:52.570 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:15:52.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.545+0000 7fad189ac640 1 --2- 172.21.15.22:0/2294063466 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c510b3b890 0x55c510b3bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:52.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.545+0000 7facfcff9640 10 client.15546 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:52.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.545+0000 7fad189ac640 1 --2- 172.21.15.22:0/2294063466 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c510b3b890 0x55c510b3bca0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceb276391218bbfc server_cookie=2df5611dd9a11f1d in_seq=0 out_seq=0 2024-09-19T08:15:52.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.545+0000 7facfcff9640 1 -- 172.21.15.22:0/2294063466 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c510a91d70 con 0x55c510b3b890 2024-09-19T08:15:52.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.545+0000 7facfcff9640 10 client.15546 handle_command_reply: tid=0 2024-09-19T08:15:52.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.545+0000 7fad1d7ea740 1 client.15546 shutdown 2024-09-19T08:15:52.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.546+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2294063466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b25b40 msgr2=0x55c510b4ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:52.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.546+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2294063466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b25b40 0x55c510b4ace0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fad100057a0 tx=0x7fad1000ba70 comp rx=0 tx=0).stop 2024-09-19T08:15:52.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.546+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2294063466 shutdown_connections 2024-09-19T08:15:52.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.546+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2294063466 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c510b3b890 0x55c510b3bca0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.546+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2294063466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c510b33780 0x55c510b4b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.546+0000 7facfcff9640 0 client.15546 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:52.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.546+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2294063466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c510b25b40 0x55c510b4ace0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.546+0000 7fad1d7ea740 1 --2- 172.21.15.22:0/2294063466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c510b245d0 0x55c510b4a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.546+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2294063466 >> 172.21.15.22:0/2294063466 conn(0x55c510aed310 msgr2=0x55c510b2b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.546+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2294063466 shutdown_connections 2024-09-19T08:15:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.546+0000 7fad1d7ea740 1 -- 172.21.15.22:0/2294063466 wait complete. 2024-09-19T08:15:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.546+0000 7fad1d7ea740 20 client.15546 trim_cache size 0 max 16384 2024-09-19T08:15:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.547+0000 7fad1b411640 1 -- 172.21.15.22:0/3525703008 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fad000a34d0 msgr2=0x7fad000a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.547+0000 7fad1b411640 1 --2- 172.21.15.22:0/3525703008 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fad000a34d0 0x7fad000a5990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0eed80 tx=0x7fad0c02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.547+0000 7fad1b411640 1 -- 172.21.15.22:0/3525703008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad14106a70 msgr2=0x7fad1419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.547+0000 7fad1b411640 1 --2- 172.21.15.22:0/3525703008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad14106a70 0x7fad1419cce0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fad0c005cc0 tx=0x7fad0c003970 comp rx=0 tx=0).stop 2024-09-19T08:15:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.547+0000 7fad1b411640 1 -- 172.21.15.22:0/3525703008 shutdown_connections 2024-09-19T08:15:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.547+0000 7fad1b411640 1 --2- 172.21.15.22:0/3525703008 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fad000a34d0 0x7fad000a5990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.547+0000 7fad1b411640 1 --2- 172.21.15.22:0/3525703008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad14106a70 0x7fad1419cce0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.547+0000 7fad1b411640 1 --2- 172.21.15.22:0/3525703008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad14101410 0x7fad14198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.547+0000 7fad1b411640 1 --2- 172.21.15.22:0/3525703008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad14100a70 0x7fad14198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.547+0000 7fad1b411640 1 -- 172.21.15.22:0/3525703008 >> 172.21.15.22:0/3525703008 conn(0x7fad140fc790 msgr2=0x7fad1406a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:52.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.547+0000 7fad1b411640 1 -- 172.21.15.22:0/3525703008 shutdown_connections 2024-09-19T08:15:52.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:52.547+0000 7fad1b411640 1 -- 172.21.15.22:0/3525703008 wait complete. 2024-09-19T08:15:52.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:52 smithi022 ceph-mon[30963]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:15:52.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:52 smithi064 ceph-mon[34900]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:15:53.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:52 smithi157 ceph-mon[35143]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:15:54.379 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:15:54.379 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:15:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:54.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:54 smithi022 ceph-mon[30963]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-19T08:15:54.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:54 smithi064 ceph-mon[34900]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-19T08:15:55.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:54 smithi157 ceph-mon[35143]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-19T08:15:55.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.161+0000 7fa9caeab640 1 -- 172.21.15.22:0/2513679161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c41009d0 msgr2=0x7fa9c4100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:55.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.161+0000 7fa9caeab640 1 --2- 172.21.15.22:0/2513679161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c41009d0 0x7fa9c4100db0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009900 tx=0x7fa9ac02a410 comp rx=0 tx=0).stop 2024-09-19T08:15:55.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.161+0000 7fa9caeab640 1 -- 172.21.15.22:0/2513679161 shutdown_connections 2024-09-19T08:15:55.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.161+0000 7fa9caeab640 1 --2- 172.21.15.22:0/2513679161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9c410e4f0 0x7fa9c4110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.161+0000 7fa9caeab640 1 --2- 172.21.15.22:0/2513679161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c4101380 0x7fa9c410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.161+0000 7fa9caeab640 1 --2- 172.21.15.22:0/2513679161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c41009d0 0x7fa9c4100db0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.161+0000 7fa9caeab640 1 -- 172.21.15.22:0/2513679161 >> 172.21.15.22:0/2513679161 conn(0x7fa9c40fc7d0 msgr2=0x7fa9c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:55.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.161+0000 7fa9caeab640 1 -- 172.21.15.22:0/2513679161 shutdown_connections 2024-09-19T08:15:55.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.161+0000 7fa9caeab640 1 -- 172.21.15.22:0/2513679161 wait complete. 2024-09-19T08:15:55.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.162+0000 7fa9caeab640 1 Processor -- start 2024-09-19T08:15:55.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.162+0000 7fa9caeab640 1 -- start start 2024-09-19T08:15:55.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.162+0000 7fa9caeab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c41009d0 0x7fa9c41a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.162+0000 7fa9caeab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9c4101380 0x7fa9c41a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.162+0000 7fa9caeab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c410e4f0 0x7fa9c419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.162+0000 7fa9caeab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c4114250 con 0x7fa9c410e4f0 2024-09-19T08:15:55.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.162+0000 7fa9caeab640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41140d0 con 0x7fa9c41009d0 2024-09-19T08:15:55.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.162+0000 7fa9caeab640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41143d0 con 0x7fa9c4101380 2024-09-19T08:15:55.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.162+0000 7fa9c9448640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c410e4f0 0x7fa9c419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.162+0000 7fa9c9448640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c410e4f0 0x7fa9c419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39988/0 (socket says 172.21.15.22:39988) 2024-09-19T08:15:55.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.162+0000 7fa9c3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9c4101380 0x7fa9c41a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.162+0000 7fa9c9448640 1 -- 172.21.15.22:0/17751032 learned_addr learned my addr 172.21.15.22:0/17751032 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:55.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.162+0000 7fa9c8c47640 1 --2- 172.21.15.22:0/17751032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c41009d0 0x7fa9c41a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.163+0000 7fa9c9448640 1 -- 172.21.15.22:0/17751032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9c4101380 msgr2=0x7fa9c41a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:55.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.163+0000 7fa9c9448640 1 --2- 172.21.15.22:0/17751032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9c4101380 0x7fa9c41a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.163+0000 7fa9c9448640 1 -- 172.21.15.22:0/17751032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c41009d0 msgr2=0x7fa9c41a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:55.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.163+0000 7fa9c9448640 1 --2- 172.21.15.22:0/17751032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c41009d0 0x7fa9c41a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.163+0000 7fa9c9448640 1 -- 172.21.15.22:0/17751032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c419ae90 con 0x7fa9c410e4f0 2024-09-19T08:15:55.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.163+0000 7fa9c9448640 1 --2- 172.21.15.22:0/17751032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c410e4f0 0x7fa9c419a660 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00e9d0 tx=0x7fa9bc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:55.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.163+0000 7fa9c1ffb640 1 -- 172.21.15.22:0/17751032 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc00cd80 con 0x7fa9c410e4f0 2024-09-19T08:15:55.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.163+0000 7fa9c1ffb640 1 -- 172.21.15.22:0/17751032 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9bc005020 con 0x7fa9c410e4f0 2024-09-19T08:15:55.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.163+0000 7fa9caeab640 1 -- 172.21.15.22:0/17751032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c419b180 con 0x7fa9c410e4f0 2024-09-19T08:15:55.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.163+0000 7fa9c1ffb640 1 -- 172.21.15.22:0/17751032 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc0116d0 con 0x7fa9c410e4f0 2024-09-19T08:15:55.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.163+0000 7fa9caeab640 1 -- 172.21.15.22:0/17751032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c41a7320 con 0x7fa9c410e4f0 2024-09-19T08:15:55.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.165+0000 7fa9c1ffb640 1 -- 172.21.15.22:0/17751032 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fa9bc0051d0 con 0x7fa9c410e4f0 2024-09-19T08:15:55.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.166+0000 7fa9cd284740 1 Processor -- start 2024-09-19T08:15:55.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.166+0000 7fa9cd284740 1 -- start start 2024-09-19T08:15:55.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.166+0000 7fa9c1ffb640 1 --2- 172.21.15.22:0/17751032 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa9b00a34d0 0x7fa9b00a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.166+0000 7fa9cd284740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5db60 0x55fa5cf5df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.166+0000 7fa9cd284740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf5e480 0x55fa5cf68800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.166+0000 7fa9cd284740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 0x55fa5cf6b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.166+0000 7fa9cd284740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fa5cec3280 con 0x55fa5cf5db60 2024-09-19T08:15:55.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.166+0000 7fa9cd284740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fa5ce6ab10 con 0x55fa5cf5e480 2024-09-19T08:15:55.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.166+0000 7fa9cd284740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fa5cebf240 con 0x55fa5cf68d40 2024-09-19T08:15:55.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.166+0000 7fa9c3fff640 1 --2- 172.21.15.22:0/17751032 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa9b00a34d0 0x7fa9b00a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.166+0000 7fa9c1ffb640 1 -- 172.21.15.22:0/17751032 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa9bc0d05b0 con 0x7fa9c410e4f0 2024-09-19T08:15:55.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.166+0000 7fa9c8c47640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5db60 0x55fa5cf5df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.167+0000 7fa9c8c47640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5db60 0x55fa5cf5df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39996/0 (socket says 172.21.15.22:39996) 2024-09-19T08:15:55.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.167+0000 7fa9c8c47640 1 -- 172.21.15.22:0/1036178976 learned_addr learned my addr 172.21.15.22:0/1036178976 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:55.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.167+0000 7fa9c9448640 1 --2- 172.21.15.22:0/1036178976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 0x55fa5cf6b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.167+0000 7fa9c3fff640 1 --2- 172.21.15.22:0/1036178976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf5e480 0x55fa5cf68800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.167+0000 7fa9c3fff640 1 --2- 172.21.15.22:0/17751032 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa9b00a34d0 0x7fa9b00a5990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4102520 tx=0x7fa9b40103d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:55.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.167+0000 7fa9c8c47640 1 -- 172.21.15.22:0/1036178976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 msgr2=0x55fa5cf6b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:55.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.167+0000 7fa9c8c47640 1 --2- 172.21.15.22:0/1036178976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 0x55fa5cf6b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.167+0000 7fa9c8c47640 1 -- 172.21.15.22:0/1036178976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf5e480 msgr2=0x55fa5cf68800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:55.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.167+0000 7fa9c8c47640 1 --2- 172.21.15.22:0/1036178976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf5e480 0x55fa5cf68800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.167+0000 7fa9c8c47640 1 -- 172.21.15.22:0/1036178976 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5ce771f0 con 0x55fa5cf5db60 2024-09-19T08:15:55.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.167+0000 7fa9c8c47640 1 --2- 172.21.15.22:0/1036178976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5db60 0x55fa5cf5df40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009900 tx=0x7fa9ac005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f9831f343cd200 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:55.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.167+0000 7fa9baffd640 1 -- 172.21.15.22:0/1036178976 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac03f070 con 0x55fa5cf5db60 2024-09-19T08:15:55.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.167+0000 7fa9baffd640 1 -- 172.21.15.22:0/1036178976 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac004030 con 0x55fa5cf5db60 2024-09-19T08:15:55.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.167+0000 7fa9baffd640 1 -- 172.21.15.22:0/1036178976 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac035070 con 0x55fa5cf5db60 2024-09-19T08:15:55.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.168+0000 7fa9cd284740 1 -- 172.21.15.22:0/1036178976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5db60 msgr2=0x55fa5cf5df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:55.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.168+0000 7fa9cd284740 1 --2- 172.21.15.22:0/1036178976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5db60 0x55fa5cf5df40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009900 tx=0x7fa9ac005e80 comp rx=0 tx=0).stop 2024-09-19T08:15:55.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.168+0000 7fa9cd284740 1 -- 172.21.15.22:0/1036178976 shutdown_connections 2024-09-19T08:15:55.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.168+0000 7fa9cd284740 1 --2- 172.21.15.22:0/1036178976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 0x55fa5cf6b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.168+0000 7fa9cd284740 1 --2- 172.21.15.22:0/1036178976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf5e480 0x55fa5cf68800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.168+0000 7fa9cd284740 1 --2- 172.21.15.22:0/1036178976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5db60 0x55fa5cf5df40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.168+0000 7fa9cd284740 1 -- 172.21.15.22:0/1036178976 >> 172.21.15.22:0/1036178976 conn(0x55fa5cf5a0f0 msgr2=0x55fa5cf5a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:55.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.169+0000 7fa9cd284740 1 -- 172.21.15.22:0/1036178976 shutdown_connections 2024-09-19T08:15:55.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.169+0000 7fa9cd284740 1 -- 172.21.15.22:0/1036178976 wait complete. 2024-09-19T08:15:55.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.169+0000 7fa9cd284740 1 -- start start 2024-09-19T08:15:55.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.169+0000 7fa9cd284740 1 Processor -- start 2024-09-19T08:15:55.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.169+0000 7fa9cd284740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf5db60 0x55fa5cf65c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.169+0000 7fa9cd284740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5e480 0x55fa5cf66160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.169+0000 7fa9cd284740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf68d40 0x55fa5cf666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.169+0000 7fa9cd284740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fa5cf6b910 con 0x55fa5cf5e480 2024-09-19T08:15:55.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.169+0000 7fa9cd284740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fa5ce7df10 con 0x55fa5cf68d40 2024-09-19T08:15:55.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.169+0000 7fa9cd284740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fa5cf6ba90 con 0x55fa5cf5db60 2024-09-19T08:15:55.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.169+0000 7fa9cd284740 10 client.0 _ll_register_callbacks cb 0x55fa5ce46a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:55.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9c3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5e480 0x55fa5cf66160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9c3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5e480 0x55fa5cf66160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40012/0 (socket says 172.21.15.22:40012) 2024-09-19T08:15:55.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9c9448640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf68d40 0x55fa5cf666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9c3fff640 1 -- 172.21.15.22:0/1075865602 learned_addr learned my addr 172.21.15.22:0/1075865602 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:55.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9c8c47640 1 --2- 172.21.15.22:0/1075865602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf5db60 0x55fa5cf65c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9c3fff640 1 -- 172.21.15.22:0/1075865602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf5db60 msgr2=0x55fa5cf65c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:55.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9c3fff640 1 --2- 172.21.15.22:0/1075865602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf5db60 0x55fa5cf65c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9c3fff640 1 -- 172.21.15.22:0/1075865602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf68d40 msgr2=0x55fa5cf666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:55.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9c3fff640 1 --2- 172.21.15.22:0/1075865602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf68d40 0x55fa5cf666a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9c3fff640 1 -- 172.21.15.22:0/1075865602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa5cc346a0 con 0x55fa5cf5e480 2024-09-19T08:15:55.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9c9448640 1 --2- 172.21.15.22:0/1075865602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf68d40 0x55fa5cf666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:15:55.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9c3fff640 1 --2- 172.21.15.22:0/1075865602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5e480 0x55fa5cf66160 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4013860 tx=0x7fa9b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73b4cdd4f7c59ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:55.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:55.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9a3fff640 1 -- 172.21.15.22:0/1075865602 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4013e60 con 0x55fa5cf5e480 2024-09-19T08:15:55.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9a3fff640 1 -- 172.21.15.22:0/1075865602 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa9b401f9c0 con 0x55fa5cf5e480 2024-09-19T08:15:55.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.170+0000 7fa9cd284740 1 -- 172.21.15.22:0/1075865602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9ac004030 con 0x55fa5cf5e480 2024-09-19T08:15:55.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.171+0000 7fa9a3fff640 1 -- 172.21.15.22:0/1075865602 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9b401fc00 con 0x55fa5cf5e480 2024-09-19T08:15:55.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.171+0000 7fa9cd284740 10 client.15558 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:55.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.171+0000 7fa9cd284740 1 -- 172.21.15.22:0/1075865602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa5cf77570 con 0x55fa5cf5e480 2024-09-19T08:15:55.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.171+0000 7fa9a3fff640 1 -- 172.21.15.22:0/1075865602 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac004030 con 0x55fa5cf5e480 2024-09-19T08:15:55.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.171+0000 7fa9a3fff640 1 -- 172.21.15.22:0/1075865602 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa9b4026070 con 0x55fa5cf5e480 2024-09-19T08:15:55.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.171+0000 7fa9cd284740 10 client.15558 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:55.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.171+0000 7fa9cd284740 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:15:55.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.171+0000 7fa9cd284740 20 client.15558 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:55.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.171+0000 7fa9cd284740 1 --2- 172.21.15.22:0/1075865602 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fa5cf78ad0 0x55fa5cf7aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.171+0000 7fa9cd284740 4 client.15558 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:15:55.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.171+0000 7fa9cd284740 1 -- 172.21.15.22:0/1075865602 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa5ced07f0 con 0x55fa5cf78ad0 2024-09-19T08:15:55.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.171+0000 7fa9c8c47640 1 --2- 172.21.15.22:0/1075865602 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fa5cf78ad0 0x55fa5cf7aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.172+0000 7fa9a3fff640 10 client.15558 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:55.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.172+0000 7fa9c8c47640 1 --2- 172.21.15.22:0/1075865602 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fa5cf78ad0 0x55fa5cf7aef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e59c45d15a7d9e server_cookie=22322fe99109ce9 in_seq=0 out_seq=0 2024-09-19T08:15:55.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9a3fff640 1 -- 172.21.15.22:0/1075865602 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa5ced07f0 con 0x55fa5cf78ad0 2024-09-19T08:15:55.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9a3fff640 10 client.15558 handle_command_reply: tid=0 2024-09-19T08:15:55.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9cd284740 1 client.15558 shutdown 2024-09-19T08:15:55.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9cd284740 1 -- 172.21.15.22:0/1075865602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5e480 msgr2=0x55fa5cf66160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:55.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9cd284740 1 --2- 172.21.15.22:0/1075865602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5e480 0x55fa5cf66160 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4013860 tx=0x7fa9b4004730 comp rx=0 tx=0).stop 2024-09-19T08:15:55.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9cd284740 1 -- 172.21.15.22:0/1075865602 shutdown_connections 2024-09-19T08:15:55.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9cd284740 1 --2- 172.21.15.22:0/1075865602 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fa5cf78ad0 0x55fa5cf7aef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9cd284740 1 --2- 172.21.15.22:0/1075865602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf68d40 0x55fa5cf666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9cd284740 1 --2- 172.21.15.22:0/1075865602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5e480 0x55fa5cf66160 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9a3fff640 0 client.15558 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:55.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9cd284740 1 --2- 172.21.15.22:0/1075865602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf5db60 0x55fa5cf65c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9cd284740 1 -- 172.21.15.22:0/1075865602 >> 172.21.15.22:0/1075865602 conn(0x55fa5cf5a0f0 msgr2=0x55fa5cf59270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:55.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9cd284740 1 -- 172.21.15.22:0/1075865602 shutdown_connections 2024-09-19T08:15:55.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9cd284740 1 -- 172.21.15.22:0/1075865602 wait complete. 2024-09-19T08:15:55.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.173+0000 7fa9cd284740 20 client.15558 trim_cache size 0 max 16384 2024-09-19T08:15:55.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.199+0000 7fa9cd284740 1 Processor -- start 2024-09-19T08:15:55.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.199+0000 7fa9cd284740 1 -- start start 2024-09-19T08:15:55.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.199+0000 7fa9cd284740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5db60 0x55fa5cf67a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.199+0000 7fa9cd284740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf5e480 0x55fa5cf807c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.199+0000 7fa9cd284740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 0x55fa5cf80d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.199+0000 7fa9cd284740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41b22c0 con 0x55fa5cf5db60 2024-09-19T08:15:55.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.199+0000 7fa9cd284740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41afb90 con 0x55fa5cf5e480 2024-09-19T08:15:55.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.199+0000 7fa9cd284740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41ad9e0 con 0x55fa5cf68d40 2024-09-19T08:15:55.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9c8c47640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5db60 0x55fa5cf67a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9c8c47640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5db60 0x55fa5cf67a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40018/0 (socket says 172.21.15.22:40018) 2024-09-19T08:15:55.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9c8c47640 1 -- 172.21.15.22:0/2227618043 learned_addr learned my addr 172.21.15.22:0/2227618043 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:55.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9c3fff640 1 --2- 172.21.15.22:0/2227618043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf5e480 0x55fa5cf807c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9c9448640 1 --2- 172.21.15.22:0/2227618043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 0x55fa5cf80d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9c8c47640 1 -- 172.21.15.22:0/2227618043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 msgr2=0x55fa5cf80d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:55.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9c8c47640 1 --2- 172.21.15.22:0/2227618043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 0x55fa5cf80d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9c8c47640 1 -- 172.21.15.22:0/2227618043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf5e480 msgr2=0x55fa5cf807c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:55.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9c8c47640 1 --2- 172.21.15.22:0/2227618043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf5e480 0x55fa5cf807c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9c8c47640 1 -- 172.21.15.22:0/2227618043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5cf609d0 con 0x55fa5cf5db60 2024-09-19T08:15:55.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9c3fff640 1 --2- 172.21.15.22:0/2227618043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf5e480 0x55fa5cf807c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:55.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9c8c47640 1 --2- 172.21.15.22:0/2227618043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5db60 0x55fa5cf67a80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac005c40 tx=0x7fa9ac005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3d13e096d45f6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:55.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9ba7fc640 1 -- 172.21.15.22:0/2227618043 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac035050 con 0x55fa5cf5db60 2024-09-19T08:15:55.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9ba7fc640 1 -- 172.21.15.22:0/2227618043 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac004040 con 0x55fa5cf5db60 2024-09-19T08:15:55.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.200+0000 7fa9ba7fc640 1 -- 172.21.15.22:0/2227618043 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac03f040 con 0x55fa5cf5db60 2024-09-19T08:15:55.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.201+0000 7fa9cd284740 1 -- 172.21.15.22:0/2227618043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5db60 msgr2=0x55fa5cf67a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:55.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.201+0000 7fa9cd284740 1 --2- 172.21.15.22:0/2227618043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5db60 0x55fa5cf67a80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac005c40 tx=0x7fa9ac005bd0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.201+0000 7fa9cd284740 1 -- 172.21.15.22:0/2227618043 shutdown_connections 2024-09-19T08:15:55.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.201+0000 7fa9cd284740 1 --2- 172.21.15.22:0/2227618043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 0x55fa5cf80d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.201+0000 7fa9cd284740 1 --2- 172.21.15.22:0/2227618043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cf5e480 0x55fa5cf807c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.201+0000 7fa9cd284740 1 --2- 172.21.15.22:0/2227618043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5db60 0x55fa5cf67a80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.201+0000 7fa9cd284740 1 -- 172.21.15.22:0/2227618043 >> 172.21.15.22:0/2227618043 conn(0x55fa5cf5a0f0 msgr2=0x55fa5cf67370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:55.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.201+0000 7fa9cd284740 1 -- 172.21.15.22:0/2227618043 shutdown_connections 2024-09-19T08:15:55.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.202+0000 7fa9cd284740 1 -- 172.21.15.22:0/2227618043 wait complete. 2024-09-19T08:15:55.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.202+0000 7fa9cd284740 1 -- start start 2024-09-19T08:15:55.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.202+0000 7fa9cd284740 1 Processor -- start 2024-09-19T08:15:55.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9cd284740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5e480 0x55fa5cf6d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9cd284740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 0x55fa5cf6d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9cd284740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cfe0780 0x55fa5cfe2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9cd284740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41ae050 con 0x55fa5cf5e480 2024-09-19T08:15:55.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9cd284740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41af4c0 con 0x55fa5cfe0780 2024-09-19T08:15:55.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9cd284740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41ae720 con 0x55fa5cf68d40 2024-09-19T08:15:55.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9cd284740 10 client.0 _ll_register_callbacks cb 0x7fa9b4010d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:55.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9c8c47640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5e480 0x55fa5cf6d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9c8c47640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5e480 0x55fa5cf6d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40034/0 (socket says 172.21.15.22:40034) 2024-09-19T08:15:55.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9c8c47640 1 -- 172.21.15.22:0/3954952813 learned_addr learned my addr 172.21.15.22:0/3954952813 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9c9448640 1 --2- 172.21.15.22:0/3954952813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cfe0780 0x55fa5cfe2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9c3fff640 1 --2- 172.21.15.22:0/3954952813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 0x55fa5cf6d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9c8c47640 1 -- 172.21.15.22:0/3954952813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 msgr2=0x55fa5cf6d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9c8c47640 1 --2- 172.21.15.22:0/3954952813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 0x55fa5cf6d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9c8c47640 1 -- 172.21.15.22:0/3954952813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cfe0780 msgr2=0x55fa5cfe2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9c8c47640 1 --2- 172.21.15.22:0/3954952813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cfe0780 0x55fa5cfe2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.203+0000 7fa9c8c47640 1 -- 172.21.15.22:0/3954952813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa5cfe31b0 con 0x55fa5cf5e480 2024-09-19T08:15:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.204+0000 7fa9c8c47640 1 --2- 172.21.15.22:0/3954952813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5e480 0x55fa5cf6d200 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac045040 tx=0x7fa9ac032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70af09ea24ad5778 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.204+0000 7fa9a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:55.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.204+0000 7fa9a3fff640 1 -- 172.21.15.22:0/3954952813 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac035070 con 0x55fa5cf5e480 2024-09-19T08:15:55.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.204+0000 7fa9a3fff640 1 -- 172.21.15.22:0/3954952813 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa9ac0315b0 con 0x55fa5cf5e480 2024-09-19T08:15:55.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.204+0000 7fa9cd284740 1 -- 172.21.15.22:0/3954952813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9ac004040 con 0x55fa5cf5e480 2024-09-19T08:15:55.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.204+0000 7fa9a3fff640 1 -- 172.21.15.22:0/3954952813 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9ac061050 con 0x55fa5cf5e480 2024-09-19T08:15:55.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.204+0000 7fa9cd284740 10 client.15564 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:55.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.204+0000 7fa9cd284740 1 -- 172.21.15.22:0/3954952813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa5cfecce0 con 0x55fa5cf5e480 2024-09-19T08:15:55.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.204+0000 7fa9a3fff640 1 -- 172.21.15.22:0/3954952813 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac004040 con 0x55fa5cf5e480 2024-09-19T08:15:55.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.205+0000 7fa9a3fff640 1 -- 172.21.15.22:0/3954952813 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa9ac03b430 con 0x55fa5cf5e480 2024-09-19T08:15:55.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.205+0000 7fa9cd284740 10 client.15564 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:55.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.205+0000 7fa9cd284740 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:15:55.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.205+0000 7fa9cd284740 20 client.15564 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:55.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.205+0000 7fa9cd284740 1 --2- 172.21.15.22:0/3954952813 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fa5cfecf10 0x55fa5cfef300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:55.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.205+0000 7fa9cd284740 4 client.15564 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:15:55.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.205+0000 7fa9cd284740 1 -- 172.21.15.22:0/3954952813 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fa5cecad00 con 0x55fa5cfecf10 2024-09-19T08:15:55.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.205+0000 7fa9c3fff640 1 --2- 172.21.15.22:0/3954952813 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fa5cfecf10 0x55fa5cfef300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:55.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.206+0000 7fa9a3fff640 10 client.15564 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:55.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.206+0000 7fa9c3fff640 1 --2- 172.21.15.22:0/3954952813 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fa5cfecf10 0x55fa5cfef300 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c533dfee0c98df91 server_cookie=d132697e4491a124 in_seq=0 out_seq=0 2024-09-19T08:15:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.207+0000 7fa9a3fff640 1 -- 172.21.15.22:0/3954952813 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fa5cecad00 con 0x55fa5cfecf10 2024-09-19T08:15:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.207+0000 7fa9a3fff640 10 client.15564 handle_command_reply: tid=0 2024-09-19T08:15:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.207+0000 7fa9cd284740 1 client.15564 shutdown 2024-09-19T08:15:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.207+0000 7fa9cd284740 1 -- 172.21.15.22:0/3954952813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5e480 msgr2=0x55fa5cf6d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.207+0000 7fa9cd284740 1 --2- 172.21.15.22:0/3954952813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5e480 0x55fa5cf6d200 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac045040 tx=0x7fa9ac032fc0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.207+0000 7fa9cd284740 1 -- 172.21.15.22:0/3954952813 shutdown_connections 2024-09-19T08:15:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.207+0000 7fa9cd284740 1 --2- 172.21.15.22:0/3954952813 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fa5cfecf10 0x55fa5cfef300 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.207+0000 7fa9cd284740 1 --2- 172.21.15.22:0/3954952813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa5cfe0780 0x55fa5cfe2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.207+0000 7fa9cd284740 1 --2- 172.21.15.22:0/3954952813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa5cf68d40 0x55fa5cf6d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.207+0000 7fa9a3fff640 0 client.15564 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:55.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.207+0000 7fa9cd284740 1 --2- 172.21.15.22:0/3954952813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa5cf5e480 0x55fa5cf6d200 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.207+0000 7fa9cd284740 1 -- 172.21.15.22:0/3954952813 >> 172.21.15.22:0/3954952813 conn(0x55fa5cf5a0f0 msgr2=0x55fa5cf5fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:55.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.208+0000 7fa9cd284740 1 -- 172.21.15.22:0/3954952813 shutdown_connections 2024-09-19T08:15:55.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.208+0000 7fa9cd284740 1 -- 172.21.15.22:0/3954952813 wait complete. 2024-09-19T08:15:55.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.208+0000 7fa9cd284740 20 client.15564 trim_cache size 0 max 16384 2024-09-19T08:15:55.213 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:15:55.213 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-19T08:15:55.213 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "626af7f5-722f-4200-8888-fd90c131d2ee", 2024-09-19T08:15:55.213 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-19T08:15:55.214 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:15:55.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.208+0000 7fa9caeab640 1 -- 172.21.15.22:0/17751032 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa9b00a34d0 msgr2=0x7fa9b00a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:55.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.208+0000 7fa9caeab640 1 --2- 172.21.15.22:0/17751032 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa9b00a34d0 0x7fa9b00a5990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4102520 tx=0x7fa9b40103d0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.208+0000 7fa9caeab640 1 -- 172.21.15.22:0/17751032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c410e4f0 msgr2=0x7fa9c419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:55.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.208+0000 7fa9caeab640 1 --2- 172.21.15.22:0/17751032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c410e4f0 0x7fa9c419a660 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00e9d0 tx=0x7fa9bc00ee90 comp rx=0 tx=0).stop 2024-09-19T08:15:55.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.209+0000 7fa9caeab640 1 -- 172.21.15.22:0/17751032 shutdown_connections 2024-09-19T08:15:55.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.209+0000 7fa9caeab640 1 --2- 172.21.15.22:0/17751032 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa9b00a34d0 0x7fa9b00a5990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.209+0000 7fa9caeab640 1 --2- 172.21.15.22:0/17751032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9c410e4f0 0x7fa9c419a660 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.209+0000 7fa9caeab640 1 --2- 172.21.15.22:0/17751032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9c4101380 0x7fa9c41a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.209+0000 7fa9caeab640 1 --2- 172.21.15.22:0/17751032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c41009d0 0x7fa9c41a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:55.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.209+0000 7fa9caeab640 1 -- 172.21.15.22:0/17751032 >> 172.21.15.22:0/17751032 conn(0x7fa9c40fc7d0 msgr2=0x7fa9c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:55.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.209+0000 7fa9caeab640 1 -- 172.21.15.22:0/17751032 shutdown_connections 2024-09-19T08:15:55.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:55.209+0000 7fa9caeab640 1 -- 172.21.15.22:0/17751032 wait complete. 2024-09-19T08:15:56.091 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:15:56.372 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:56.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:56 smithi022 ceph-mon[30963]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T08:15:56.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:56 smithi022 ceph-mon[30963]: scrub queued for path: / 2024-09-19T08:15:56.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:56 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:15:56.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:56 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:15:56.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:56 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:15:56.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:56 smithi064 ceph-mon[34900]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T08:15:56.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:56 smithi064 ceph-mon[34900]: scrub queued for path: / 2024-09-19T08:15:56.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:56 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:15:56.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:56 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:15:56.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:56 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:15:56.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.843+0000 7f7b0cf5d640 1 -- 172.21.15.22:0/2177467158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b08102570 msgr2=0x7f7b081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:56.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.843+0000 7f7b0cf5d640 1 --2- 172.21.15.22:0/2177467158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b08102570 0x7f7b081029b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009900 tx=0x7f7afc02a480 comp rx=0 tx=0).stop 2024-09-19T08:15:56.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.843+0000 7f7b0cf5d640 1 -- 172.21.15.22:0/2177467158 shutdown_connections 2024-09-19T08:15:56.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.843+0000 7f7b0cf5d640 1 --2- 172.21.15.22:0/2177467158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b08102ef0 0x7f7b0810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.843+0000 7f7b0cf5d640 1 --2- 172.21.15.22:0/2177467158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b08102570 0x7f7b081029b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.843+0000 7f7b0cf5d640 1 --2- 172.21.15.22:0/2177467158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b08100e20 0x7f7b08101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.843+0000 7f7b0cf5d640 1 -- 172.21.15.22:0/2177467158 >> 172.21.15.22:0/2177467158 conn(0x7f7b080fc7d0 msgr2=0x7f7b080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.844+0000 7f7b0cf5d640 1 -- 172.21.15.22:0/2177467158 shutdown_connections 2024-09-19T08:15:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.844+0000 7f7b0cf5d640 1 -- 172.21.15.22:0/2177467158 wait complete. 2024-09-19T08:15:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.845+0000 7f7b0cf5d640 1 Processor -- start 2024-09-19T08:15:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.845+0000 7f7b0cf5d640 1 -- start start 2024-09-19T08:15:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.845+0000 7f7b0cf5d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b08100e20 0x7f7b08199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.845+0000 7f7b0cf5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b08102570 0x7f7b08199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.845+0000 7f7b0cf5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b08102ef0 0x7f7b081a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.845+0000 7f7b0cf5d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7b08110f80 con 0x7f7b08102570 2024-09-19T08:15:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.845+0000 7f7b0cf5d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7b08110e00 con 0x7f7b08102ef0 2024-09-19T08:15:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.845+0000 7f7b0cf5d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7b08111100 con 0x7f7b08100e20 2024-09-19T08:15:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.845+0000 7f7b05d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b08102570 0x7f7b08199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.845+0000 7f7b05d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b08102570 0x7f7b08199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40046/0 (socket says 172.21.15.22:40046) 2024-09-19T08:15:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.845+0000 7f7b05d9b640 1 -- 172.21.15.22:0/1956827500 learned_addr learned my addr 172.21.15.22:0/1956827500 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.845+0000 7f7b06d9d640 1 --2- 172.21.15.22:0/1956827500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b08102ef0 0x7f7b081a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.845+0000 7f7b0659c640 1 --2- 172.21.15.22:0/1956827500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b08100e20 0x7f7b08199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.846+0000 7f7b06d9d640 1 -- 172.21.15.22:0/1956827500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b08100e20 msgr2=0x7f7b08199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.846+0000 7f7b06d9d640 1 --2- 172.21.15.22:0/1956827500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b08100e20 0x7f7b08199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.846+0000 7f7b06d9d640 1 -- 172.21.15.22:0/1956827500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b08102570 msgr2=0x7f7b08199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.846+0000 7f7b06d9d640 1 --2- 172.21.15.22:0/1956827500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b08102570 0x7f7b08199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.846+0000 7f7b06d9d640 1 -- 172.21.15.22:0/1956827500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b08197140 con 0x7f7b08102ef0 2024-09-19T08:15:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.846+0000 7f7b06d9d640 1 --2- 172.21.15.22:0/1956827500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b08102ef0 0x7f7b081a43d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000ec50 tx=0x7f7b0000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.846+0000 7f7af77fe640 1 -- 172.21.15.22:0/1956827500 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00003b70 con 0x7f7b08102ef0 2024-09-19T08:15:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.846+0000 7f7af77fe640 1 -- 172.21.15.22:0/1956827500 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b00003d10 con 0x7f7b08102ef0 2024-09-19T08:15:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.846+0000 7f7b0cf5d640 1 -- 172.21.15.22:0/1956827500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b081973d0 con 0x7f7b08102ef0 2024-09-19T08:15:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.846+0000 7f7af77fe640 1 -- 172.21.15.22:0/1956827500 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00010930 con 0x7f7b08102ef0 2024-09-19T08:15:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.846+0000 7f7b0cf5d640 1 -- 172.21.15.22:0/1956827500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b08197910 con 0x7f7b08102ef0 2024-09-19T08:15:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.849+0000 7f7b0f336740 1 Processor -- start 2024-09-19T08:15:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.850+0000 7f7af77fe640 1 -- 172.21.15.22:0/1956827500 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f7b00003fc0 con 0x7f7b08102ef0 2024-09-19T08:15:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.850+0000 7f7b0f336740 1 -- start start 2024-09-19T08:15:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b0f336740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258ac4a0 0x559a258ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b0f336740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258acdc0 0x559a258b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b0f336740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258b7680 0x559a258b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b0f336740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559a258122e0 con 0x559a258acdc0 2024-09-19T08:15:56.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b0f336740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a257b9ce0 con 0x559a258b7680 2024-09-19T08:15:56.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b0f336740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559a2580e2a0 con 0x559a258ac4a0 2024-09-19T08:15:56.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b05d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258ac4a0 0x559a258ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b05d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258ac4a0 0x559a258ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:60256/0 (socket says 172.21.15.22:60256) 2024-09-19T08:15:56.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b05d9b640 1 -- 172.21.15.22:0/1986853453 learned_addr learned my addr 172.21.15.22:0/1986853453 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:56.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b05d9b640 1 --2- 172.21.15.22:0/1986853453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258b7680 0x559a258b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b05d9b640 1 -- 172.21.15.22:0/1986853453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258b7680 msgr2=0x559a258b9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:56.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b05d9b640 1 --2- 172.21.15.22:0/1986853453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258b7680 0x559a258b9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b05d9b640 1 -- 172.21.15.22:0/1986853453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258acdc0 msgr2=0x559a258b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:56.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b05d9b640 1 --2- 172.21.15.22:0/1986853453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258acdc0 0x559a258b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.851+0000 7f7b05d9b640 1 -- 172.21.15.22:0/1986853453 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a2579e440 con 0x559a258ac4a0 2024-09-19T08:15:56.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.852+0000 7f7b05d9b640 1 --2- 172.21.15.22:0/1986853453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258ac4a0 0x559a258ac880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7afc000c00 tx=0x7f7afc02a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98a8add9f1bb3726 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.852+0000 7f7af4ff9640 1 -- 172.21.15.22:0/1986853453 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc035030 con 0x559a258ac4a0 2024-09-19T08:15:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.852+0000 7f7af4ff9640 1 -- 172.21.15.22:0/1986853453 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7afc003b80 con 0x559a258ac4a0 2024-09-19T08:15:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.852+0000 7f7af4ff9640 1 -- 172.21.15.22:0/1986853453 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc03d9f0 con 0x559a258ac4a0 2024-09-19T08:15:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.852+0000 7f7af77fe640 1 --2- 172.21.15.22:0/1956827500 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7af00a3620 0x7f7af00a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.853+0000 7f7af77fe640 1 -- 172.21.15.22:0/1956827500 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7b000d0380 con 0x7f7b08102ef0 2024-09-19T08:15:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.853+0000 7f7b0659c640 1 --2- 172.21.15.22:0/1956827500 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7af00a3620 0x7f7af00a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.853+0000 7f7b0f336740 1 -- 172.21.15.22:0/1986853453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258ac4a0 msgr2=0x559a258ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.854+0000 7f7b0f336740 1 --2- 172.21.15.22:0/1986853453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258ac4a0 0x559a258ac880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7afc000c00 tx=0x7f7afc02a480 comp rx=0 tx=0).stop 2024-09-19T08:15:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.854+0000 7f7b0659c640 1 --2- 172.21.15.22:0/1956827500 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7af00a3620 0x7f7af00a5ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x559a258a9890 tx=0x7f7af8009840 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.854+0000 7f7b0f336740 1 -- 172.21.15.22:0/1986853453 shutdown_connections 2024-09-19T08:15:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.854+0000 7f7b0f336740 1 --2- 172.21.15.22:0/1986853453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258b7680 0x559a258b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.854+0000 7f7b0f336740 1 --2- 172.21.15.22:0/1986853453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258acdc0 0x559a258b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.854+0000 7f7b0f336740 1 --2- 172.21.15.22:0/1986853453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258ac4a0 0x559a258ac880 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.854+0000 7f7b0f336740 1 -- 172.21.15.22:0/1986853453 >> 172.21.15.22:0/1986853453 conn(0x559a258a8a30 msgr2=0x559a258a8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.854+0000 7f7b0f336740 1 -- 172.21.15.22:0/1986853453 shutdown_connections 2024-09-19T08:15:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.854+0000 7f7b0f336740 1 -- 172.21.15.22:0/1986853453 wait complete. 2024-09-19T08:15:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.854+0000 7f7b0f336740 1 -- start start 2024-09-19T08:15:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.854+0000 7f7b0f336740 1 Processor -- start 2024-09-19T08:15:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.855+0000 7f7b0f336740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258ac4a0 0x559a258b7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.855+0000 7f7b0f336740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258acdc0 0x559a258b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.855+0000 7f7b0f336740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258b7680 0x559a258b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.855+0000 7f7b0f336740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559a258ba250 con 0x559a258ac4a0 2024-09-19T08:15:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.855+0000 7f7b0f336740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a257cd050 con 0x559a258b7680 2024-09-19T08:15:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.855+0000 7f7b0f336740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559a258ba3d0 con 0x559a258acdc0 2024-09-19T08:15:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.855+0000 7f7b0f336740 10 client.0 _ll_register_callbacks cb 0x559a2580d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.855+0000 7f7b05d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258acdc0 0x559a258b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.855+0000 7f7b05d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258acdc0 0x559a258b42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:60268/0 (socket says 172.21.15.22:60268) 2024-09-19T08:15:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.855+0000 7f7b05d9b640 1 -- 172.21.15.22:0/4144439002 learned_addr learned my addr 172.21.15.22:0/4144439002 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.855+0000 7f7b06d9d640 1 --2- 172.21.15.22:0/4144439002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258b7680 0x559a258b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.855+0000 7f7b05d9b640 1 -- 172.21.15.22:0/4144439002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258b7680 msgr2=0x559a258b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.855+0000 7f7b05d9b640 1 --2- 172.21.15.22:0/4144439002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258b7680 0x559a258b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.855+0000 7f7b05d9b640 1 -- 172.21.15.22:0/4144439002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258ac4a0 msgr2=0x559a258b7040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:15:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.856+0000 7f7b0659c640 1 --2- 172.21.15.22:0/4144439002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258ac4a0 0x559a258b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.856+0000 7f7b05d9b640 1 --2- 172.21.15.22:0/4144439002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258ac4a0 0x559a258b7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.856+0000 7f7b05d9b640 1 -- 172.21.15.22:0/4144439002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a255833c0 con 0x559a258acdc0 2024-09-19T08:15:56.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.856+0000 7f7b0659c640 1 --2- 172.21.15.22:0/4144439002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258ac4a0 0x559a258b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:15:56.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.857+0000 7f7b05d9b640 1 --2- 172.21.15.22:0/4144439002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258acdc0 0x559a258b42f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7afc005920 tx=0x7f7afc003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=167486ea1eb91653 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:56.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.857+0000 7f7aedffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:15:56.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.858+0000 7f7aedffb640 1 -- 172.21.15.22:0/4144439002 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc04e070 con 0x559a258acdc0 2024-09-19T08:15:56.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.858+0000 7f7aedffb640 1 -- 172.21.15.22:0/4144439002 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7afc03d770 con 0x559a258acdc0 2024-09-19T08:15:56.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.858+0000 7f7b0f336740 1 -- 172.21.15.22:0/4144439002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7afc003b80 con 0x559a258acdc0 2024-09-19T08:15:56.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.858+0000 7f7aedffb640 1 -- 172.21.15.22:0/4144439002 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7afc058050 con 0x559a258acdc0 2024-09-19T08:15:56.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.858+0000 7f7b0f336740 10 client.25195 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:56.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.858+0000 7f7b0f336740 1 -- 172.21.15.22:0/4144439002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a258ba250 con 0x559a258acdc0 2024-09-19T08:15:56.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.858+0000 7f7aedffb640 1 -- 172.21.15.22:0/4144439002 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7afc003b80 con 0x559a258acdc0 2024-09-19T08:15:56.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.859+0000 7f7aedffb640 1 -- 172.21.15.22:0/4144439002 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7afc031ad0 con 0x559a258acdc0 2024-09-19T08:15:56.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.859+0000 7f7b0f336740 10 client.25195 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:56.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.859+0000 7f7b0f336740 10 client.25195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:15:56.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.859+0000 7f7b0f336740 20 client.25195 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:56.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.859+0000 7f7b0f336740 1 --2- 172.21.15.22:0/4144439002 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x559a258c6e20 0x559a258c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.859+0000 7f7b0f336740 4 client.25195 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:15:56.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.859+0000 7f7b0f336740 1 -- 172.21.15.22:0/4144439002 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a2581f850 con 0x559a258c6e20 2024-09-19T08:15:56.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.860+0000 7f7b0659c640 1 --2- 172.21.15.22:0/4144439002 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x559a258c6e20 0x559a258c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.860+0000 7f7b0659c640 1 --2- 172.21.15.22:0/4144439002 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x559a258c6e20 0x559a258c9210 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=175ef193d710ea68 server_cookie=8db24a28317e4473 in_seq=0 out_seq=0 2024-09-19T08:15:56.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.860+0000 7f7aedffb640 10 client.25195 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:56.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.861+0000 7f7aedffb640 1 -- 172.21.15.22:0/4144439002 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a2581f850 con 0x559a258c6e20 2024-09-19T08:15:56.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.861+0000 7f7aedffb640 10 client.25195 handle_command_reply: tid=0 2024-09-19T08:15:56.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.861+0000 7f7b0f336740 1 client.25195 shutdown 2024-09-19T08:15:56.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.861+0000 7f7b0f336740 1 -- 172.21.15.22:0/4144439002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258acdc0 msgr2=0x559a258b42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:56.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.861+0000 7f7b0f336740 1 --2- 172.21.15.22:0/4144439002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258acdc0 0x559a258b42f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7afc005920 tx=0x7f7afc003f60 comp rx=0 tx=0).stop 2024-09-19T08:15:56.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.861+0000 7f7b0f336740 1 -- 172.21.15.22:0/4144439002 shutdown_connections 2024-09-19T08:15:56.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.861+0000 7f7b0f336740 1 --2- 172.21.15.22:0/4144439002 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x559a258c6e20 0x559a258c9210 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.861+0000 7f7b0f336740 1 --2- 172.21.15.22:0/4144439002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258b7680 0x559a258b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.861+0000 7f7b0f336740 1 --2- 172.21.15.22:0/4144439002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258acdc0 0x559a258b42f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.861+0000 7f7b0f336740 1 --2- 172.21.15.22:0/4144439002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258ac4a0 0x559a258b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.861+0000 7f7b0f336740 1 -- 172.21.15.22:0/4144439002 >> 172.21.15.22:0/4144439002 conn(0x559a258a8a30 msgr2=0x559a258ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:56.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.861+0000 7f7aedffb640 0 client.25195 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:56.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.862+0000 7f7b0f336740 1 -- 172.21.15.22:0/4144439002 shutdown_connections 2024-09-19T08:15:56.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.862+0000 7f7b0f336740 1 -- 172.21.15.22:0/4144439002 wait complete. 2024-09-19T08:15:56.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.862+0000 7f7b0f336740 20 client.25195 trim_cache size 0 max 16384 2024-09-19T08:15:56.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.880+0000 7f7b0f336740 1 Processor -- start 2024-09-19T08:15:56.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.880+0000 7f7b0f336740 1 -- start start 2024-09-19T08:15:56.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.880+0000 7f7b0f336740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258ac4a0 0x559a258c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.880+0000 7f7b0f336740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258acdc0 0x559a258ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.880+0000 7f7b0f336740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258b7680 0x559a258cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.880+0000 7f7b0f336740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b6650 con 0x559a258b7680 2024-09-19T08:15:56.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.880+0000 7f7b0f336740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b3f20 con 0x559a258acdc0 2024-09-19T08:15:56.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.880+0000 7f7b0f336740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081a8340 con 0x559a258ac4a0 2024-09-19T08:15:56.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.880+0000 7f7b06d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258b7680 0x559a258cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.880+0000 7f7b06d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258b7680 0x559a258cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40072/0 (socket says 172.21.15.22:40072) 2024-09-19T08:15:56.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.881+0000 7f7b06d9d640 1 -- 172.21.15.22:0/765810050 learned_addr learned my addr 172.21.15.22:0/765810050 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:56.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.881+0000 7f7b0659c640 1 --2- 172.21.15.22:0/765810050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258ac4a0 0x559a258c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.881+0000 7f7b05d9b640 1 --2- 172.21.15.22:0/765810050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258acdc0 0x559a258ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.881+0000 7f7b06d9d640 1 -- 172.21.15.22:0/765810050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258ac4a0 msgr2=0x559a258c34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:56.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.881+0000 7f7b06d9d640 1 --2- 172.21.15.22:0/765810050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258ac4a0 0x559a258c34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.881+0000 7f7b06d9d640 1 -- 172.21.15.22:0/765810050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258acdc0 msgr2=0x559a258ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:56.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.881+0000 7f7b06d9d640 1 --2- 172.21.15.22:0/765810050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258acdc0 0x559a258ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.881+0000 7f7b06d9d640 1 -- 172.21.15.22:0/765810050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a258b73b0 con 0x559a258b7680 2024-09-19T08:15:56.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.881+0000 7f7b0659c640 1 --2- 172.21.15.22:0/765810050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258ac4a0 0x559a258c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:15:56.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.881+0000 7f7b06d9d640 1 --2- 172.21.15.22:0/765810050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258b7680 0x559a258cee40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7b000cda70 tx=0x7f7b00098780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b7470b4858815f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:56.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.881+0000 7f7aeffff640 1 -- 172.21.15.22:0/765810050 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b000b4070 con 0x559a258b7680 2024-09-19T08:15:56.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.881+0000 7f7aeffff640 1 -- 172.21.15.22:0/765810050 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0009cb20 con 0x559a258b7680 2024-09-19T08:15:56.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.881+0000 7f7aeffff640 1 -- 172.21.15.22:0/765810050 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b000b5260 con 0x559a258b7680 2024-09-19T08:15:56.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.882+0000 7f7b0f336740 1 -- 172.21.15.22:0/765810050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258b7680 msgr2=0x559a258cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:56.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.883+0000 7f7b0f336740 1 --2- 172.21.15.22:0/765810050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258b7680 0x559a258cee40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7b000cda70 tx=0x7f7b00098780 comp rx=0 tx=0).stop 2024-09-19T08:15:56.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.883+0000 7f7b0f336740 1 -- 172.21.15.22:0/765810050 shutdown_connections 2024-09-19T08:15:56.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.883+0000 7f7b0f336740 1 --2- 172.21.15.22:0/765810050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258b7680 0x559a258cee40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.883+0000 7f7b0f336740 1 --2- 172.21.15.22:0/765810050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258acdc0 0x559a258ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.883+0000 7f7b0f336740 1 --2- 172.21.15.22:0/765810050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258ac4a0 0x559a258c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.883+0000 7f7b0f336740 1 -- 172.21.15.22:0/765810050 >> 172.21.15.22:0/765810050 conn(0x559a258a8a30 msgr2=0x559a258b6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:56.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.883+0000 7f7b0f336740 1 -- 172.21.15.22:0/765810050 shutdown_connections 2024-09-19T08:15:56.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.883+0000 7f7b0f336740 1 -- 172.21.15.22:0/765810050 wait complete. 2024-09-19T08:15:56.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.883+0000 7f7b0f336740 1 -- start start 2024-09-19T08:15:56.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.883+0000 7f7b0f336740 1 Processor -- start 2024-09-19T08:15:56.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0f336740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258ac4a0 0x559a258bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0f336740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258acdc0 0x559a258bd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0f336740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258b7680 0x559a258bd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0f336740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081a89b0 con 0x559a258ac4a0 2024-09-19T08:15:56.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0f336740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b3850 con 0x559a258acdc0 2024-09-19T08:15:56.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0f336740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b2ab0 con 0x559a258b7680 2024-09-19T08:15:56.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0f336740 10 client.0 _ll_register_callbacks cb 0x7f7afc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:15:56.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258ac4a0 0x559a258bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258ac4a0 0x559a258bfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40088/0 (socket says 172.21.15.22:40088) 2024-09-19T08:15:56.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0659c640 1 -- 172.21.15.22:0/79054411 learned_addr learned my addr 172.21.15.22:0/79054411 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:56.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b05d9b640 1 --2- 172.21.15.22:0/79054411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258acdc0 0x559a258bd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b06d9d640 1 --2- 172.21.15.22:0/79054411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258b7680 0x559a258bd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0659c640 1 -- 172.21.15.22:0/79054411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258b7680 msgr2=0x559a258bd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:56.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0659c640 1 --2- 172.21.15.22:0/79054411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258b7680 0x559a258bd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0659c640 1 -- 172.21.15.22:0/79054411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258acdc0 msgr2=0x559a258bd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:15:56.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0659c640 1 --2- 172.21.15.22:0/79054411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258acdc0 0x559a258bd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0659c640 1 -- 172.21.15.22:0/79054411 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a258acbf0 con 0x559a258ac4a0 2024-09-19T08:15:56.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7b0659c640 1 --2- 172.21.15.22:0/79054411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258ac4a0 0x559a258bfdb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x559a258c98a0 tx=0x7f7af8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536605d3507a45c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:56.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.884+0000 7f7aedffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:15:56.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.885+0000 7f7aedffb640 1 -- 172.21.15.22:0/79054411 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8025700 con 0x559a258ac4a0 2024-09-19T08:15:56.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.885+0000 7f7aedffb640 1 -- 172.21.15.22:0/79054411 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7af80258a0 con 0x559a258ac4a0 2024-09-19T08:15:56.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.885+0000 7f7b0f336740 1 -- 172.21.15.22:0/79054411 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b0009cb20 con 0x559a258ac4a0 2024-09-19T08:15:56.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.885+0000 7f7aedffb640 1 -- 172.21.15.22:0/79054411 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7af8025aa0 con 0x559a258ac4a0 2024-09-19T08:15:56.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.885+0000 7f7b0f336740 10 client.15576 fetch_fsmap learned FSMap version 25 2024-09-19T08:15:56.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.885+0000 7f7b0f336740 1 -- 172.21.15.22:0/79054411 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a259317a0 con 0x559a258ac4a0 2024-09-19T08:15:56.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.885+0000 7f7aedffb640 1 -- 172.21.15.22:0/79054411 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0009cb20 con 0x559a258ac4a0 2024-09-19T08:15:56.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.886+0000 7f7aedffb640 1 -- 172.21.15.22:0/79054411 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7af802f410 con 0x559a258ac4a0 2024-09-19T08:15:56.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.886+0000 7f7b0f336740 10 client.15576 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:15:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.886+0000 7f7b0f336740 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:15:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.886+0000 7f7b0f336740 20 client.15576 populate_metadata read hostname 'smithi022' 2024-09-19T08:15:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.886+0000 7f7b0f336740 1 --2- 172.21.15.22:0/79054411 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x559a25931e50 0x559a25934270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.886+0000 7f7b0f336740 4 client.15576 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:15:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.886+0000 7f7b0f336740 1 -- 172.21.15.22:0/79054411 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a25819d60 con 0x559a25931e50 2024-09-19T08:15:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.886+0000 7f7b05d9b640 1 --2- 172.21.15.22:0/79054411 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x559a25931e50 0x559a25934270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.887+0000 7f7aedffb640 10 client.15576 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.887+0000 7f7b05d9b640 1 --2- 172.21.15.22:0/79054411 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x559a25931e50 0x559a25934270 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7df59fdb1a06b58 server_cookie=661fa4bbb67341b2 in_seq=0 out_seq=0 2024-09-19T08:15:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.887+0000 7f7aedffb640 1 -- 172.21.15.22:0/79054411 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559a25819d60 con 0x559a25931e50 2024-09-19T08:15:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.887+0000 7f7aedffb640 10 client.15576 handle_command_reply: tid=0 2024-09-19T08:15:56.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.887+0000 7f7b0f336740 1 client.15576 shutdown 2024-09-19T08:15:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.887+0000 7f7b0f336740 1 -- 172.21.15.22:0/79054411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258ac4a0 msgr2=0x559a258bfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.887+0000 7f7b0f336740 1 --2- 172.21.15.22:0/79054411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258ac4a0 0x559a258bfdb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x559a258c98a0 tx=0x7f7af8005f90 comp rx=0 tx=0).stop 2024-09-19T08:15:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.887+0000 7f7b0f336740 1 -- 172.21.15.22:0/79054411 shutdown_connections 2024-09-19T08:15:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.887+0000 7f7b0f336740 1 --2- 172.21.15.22:0/79054411 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x559a25931e50 0x559a25934270 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.887+0000 7f7b0f336740 1 --2- 172.21.15.22:0/79054411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559a258b7680 0x559a258bd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.887+0000 7f7b0f336740 1 --2- 172.21.15.22:0/79054411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a258acdc0 0x559a258bd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.887+0000 7f7b0f336740 1 --2- 172.21.15.22:0/79054411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a258ac4a0 0x559a258bfdb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.887+0000 7f7b0f336740 1 -- 172.21.15.22:0/79054411 >> 172.21.15.22:0/79054411 conn(0x559a258a8a30 msgr2=0x559a258b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.888+0000 7f7aedffb640 0 client.15576 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:15:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.888+0000 7f7b0f336740 1 -- 172.21.15.22:0/79054411 shutdown_connections 2024-09-19T08:15:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.888+0000 7f7b0f336740 1 -- 172.21.15.22:0/79054411 wait complete. 2024-09-19T08:15:56.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.888+0000 7f7b0f336740 20 client.15576 trim_cache size 0 max 16384 2024-09-19T08:15:56.899 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:15:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-19T08:15:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-09-19T08:15:56.899 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:15:56.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.889+0000 7f7b0cf5d640 1 -- 172.21.15.22:0/1956827500 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7af00a3620 msgr2=0x7f7af00a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:56.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.889+0000 7f7b0cf5d640 1 --2- 172.21.15.22:0/1956827500 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7af00a3620 0x7f7af00a5ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x559a258a9890 tx=0x7f7af8009840 comp rx=0 tx=0).stop 2024-09-19T08:15:56.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.889+0000 7f7b0cf5d640 1 -- 172.21.15.22:0/1956827500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b08102ef0 msgr2=0x7f7b081a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:56.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.889+0000 7f7b0cf5d640 1 --2- 172.21.15.22:0/1956827500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b08102ef0 0x7f7b081a43d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000ec50 tx=0x7f7b0000c6a0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.889+0000 7f7b0cf5d640 1 -- 172.21.15.22:0/1956827500 shutdown_connections 2024-09-19T08:15:56.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.889+0000 7f7b0cf5d640 1 --2- 172.21.15.22:0/1956827500 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7af00a3620 0x7f7af00a5ae0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.889+0000 7f7b0cf5d640 1 --2- 172.21.15.22:0/1956827500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b08102ef0 0x7f7b081a43d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.889+0000 7f7b0cf5d640 1 --2- 172.21.15.22:0/1956827500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b08102570 0x7f7b08199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.889+0000 7f7b0cf5d640 1 --2- 172.21.15.22:0/1956827500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b08100e20 0x7f7b08199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:56.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.889+0000 7f7b0cf5d640 1 -- 172.21.15.22:0/1956827500 >> 172.21.15.22:0/1956827500 conn(0x7f7b080fc7d0 msgr2=0x7f7b080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:56.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.889+0000 7f7b0cf5d640 1 -- 172.21.15.22:0/1956827500 shutdown_connections 2024-09-19T08:15:56.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:56.889+0000 7f7b0cf5d640 1 -- 172.21.15.22:0/1956827500 wait complete. 2024-09-19T08:15:56.902 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:15:57.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:56 smithi157 ceph-mon[35143]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T08:15:57.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:56 smithi157 ceph-mon[35143]: scrub queued for path: / 2024-09-19T08:15:57.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:56 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:15:57.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:56 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:15:57.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:56 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:15:57.730 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:57 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:15:57.737 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:57.774 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:15:57.774 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:15:57.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:57 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:15:58.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:57 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:15:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.292+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/283238740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a68106f10 msgr2=0x7f0a6810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.292+0000 7f0a6e8bc640 1 --2- 172.21.15.22:0/283238740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a68106f10 0x7f0a6810ea90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009ca0 tx=0x7f0a6002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.292+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/283238740 shutdown_connections 2024-09-19T08:15:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.292+0000 7f0a6e8bc640 1 --2- 172.21.15.22:0/283238740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a68106f10 0x7f0a6810ea90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.292+0000 7f0a6e8bc640 1 --2- 172.21.15.22:0/283238740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a68102790 0x7f0a681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.292+0000 7f0a6e8bc640 1 --2- 172.21.15.22:0/283238740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a68101de0 0x7f0a681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.292+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/283238740 >> 172.21.15.22:0/283238740 conn(0x7f0a680fd8e0 msgr2=0x7f0a680ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.292+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/283238740 shutdown_connections 2024-09-19T08:15:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.292+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/283238740 wait complete. 2024-09-19T08:15:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.293+0000 7f0a6e8bc640 1 Processor -- start 2024-09-19T08:15:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.293+0000 7f0a6e8bc640 1 -- start start 2024-09-19T08:15:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.294+0000 7f0a6e8bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a68101de0 0x7f0a6819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.294+0000 7f0a6e8bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a68102790 0x7f0a6819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.294+0000 7f0a6e8bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a68106f10 0x7f0a6819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.294+0000 7f0a6e8bc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68111f70 con 0x7f0a68101de0 2024-09-19T08:15:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.294+0000 7f0a6e8bc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68111df0 con 0x7f0a68102790 2024-09-19T08:15:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.294+0000 7f0a6e8bc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a681120f0 con 0x7f0a68106f10 2024-09-19T08:15:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.294+0000 7f0a67fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a68101de0 0x7f0a6819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.294+0000 7f0a67fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a68101de0 0x7f0a6819a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33908/0 (socket says 172.21.15.22:33908) 2024-09-19T08:15:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.294+0000 7f0a67fff640 1 -- 172.21.15.22:0/1278541653 learned_addr learned my addr 172.21.15.22:0/1278541653 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.294+0000 7f0a677fe640 1 --2- 172.21.15.22:0/1278541653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a68102790 0x7f0a6819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.294+0000 7f0a6ce59640 1 --2- 172.21.15.22:0/1278541653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a68106f10 0x7f0a6819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.295+0000 7f0a67fff640 1 -- 172.21.15.22:0/1278541653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a68106f10 msgr2=0x7f0a6819ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.295+0000 7f0a67fff640 1 --2- 172.21.15.22:0/1278541653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a68106f10 0x7f0a6819ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.295+0000 7f0a67fff640 1 -- 172.21.15.22:0/1278541653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a68102790 msgr2=0x7f0a6819a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.295+0000 7f0a67fff640 1 --2- 172.21.15.22:0/1278541653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a68102790 0x7f0a6819a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.295+0000 7f0a67fff640 1 -- 172.21.15.22:0/1278541653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6819f310 con 0x7f0a68101de0 2024-09-19T08:15:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.295+0000 7f0a67fff640 1 --2- 172.21.15.22:0/1278541653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a68101de0 0x7f0a6819a360 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800c920 tx=0x7f0a5800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.295+0000 7f0a657fa640 1 -- 172.21.15.22:0/1278541653 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58007c70 con 0x7f0a68101de0 2024-09-19T08:15:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.295+0000 7f0a657fa640 1 -- 172.21.15.22:0/1278541653 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a58007e10 con 0x7f0a68101de0 2024-09-19T08:15:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.295+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/1278541653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6819f600 con 0x7f0a68101de0 2024-09-19T08:15:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.296+0000 7f0a657fa640 1 -- 172.21.15.22:0/1278541653 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58018890 con 0x7f0a68101de0 2024-09-19T08:15:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.296+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/1278541653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a681a6ee0 con 0x7f0a68101de0 2024-09-19T08:15:58.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.297+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/1278541653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a6819b6c0 con 0x7f0a68101de0 2024-09-19T08:15:58.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.298+0000 7f0a657fa640 1 -- 172.21.15.22:0/1278541653 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f0a58018b30 con 0x7f0a68101de0 2024-09-19T08:15:58.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.299+0000 7f0a657fa640 1 --2- 172.21.15.22:0/1278541653 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0a500a3550 0x7f0a500a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:58.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.299+0000 7f0a657fa640 1 -- 172.21.15.22:0/1278541653 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0a580cffc0 con 0x7f0a68101de0 2024-09-19T08:15:58.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.299+0000 7f0a677fe640 1 --2- 172.21.15.22:0/1278541653 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0a500a3550 0x7f0a500a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:58.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.300+0000 7f0a677fe640 1 --2- 172.21.15.22:0/1278541653 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0a500a3550 0x7f0a500a5a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a680ffd20 tx=0x7f0a5c005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:58.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.304+0000 7f0a657fa640 1 -- 172.21.15.22:0/1278541653 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0a58014030 con 0x7f0a68101de0 2024-09-19T08:15:58.346 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:15:58.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:58 smithi022 ceph-mon[30963]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-19T08:15:58.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.684+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/1278541653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a681a7320 con 0x7f0a68101de0 2024-09-19T08:15:58.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.685+0000 7f0a657fa640 1 -- 172.21.15.22:0/1278541653 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11906 (secure 0 0 0) 0x7f0a58018e20 con 0x7f0a68101de0 2024-09-19T08:15:58.686 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:58.686 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"btime":"2024-09-19T08:15:00:109390+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:15:00.059208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:15:58.686 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-09-19T08:15:58.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.688+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/1278541653 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0a500a3550 msgr2=0x7f0a500a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:58.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.688+0000 7f0a6e8bc640 1 --2- 172.21.15.22:0/1278541653 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0a500a3550 0x7f0a500a5a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a680ffd20 tx=0x7f0a5c005f90 comp rx=0 tx=0).stop 2024-09-19T08:15:58.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.688+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/1278541653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a68101de0 msgr2=0x7f0a6819a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:58.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.688+0000 7f0a6e8bc640 1 --2- 172.21.15.22:0/1278541653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a68101de0 0x7f0a6819a360 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800c920 tx=0x7f0a5800cde0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.688+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/1278541653 shutdown_connections 2024-09-19T08:15:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.688+0000 7f0a6e8bc640 1 --2- 172.21.15.22:0/1278541653 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0a500a3550 0x7f0a500a5a10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.688+0000 7f0a6e8bc640 1 --2- 172.21.15.22:0/1278541653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a68106f10 0x7f0a6819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.688+0000 7f0a6e8bc640 1 --2- 172.21.15.22:0/1278541653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a68102790 0x7f0a6819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.688+0000 7f0a6e8bc640 1 --2- 172.21.15.22:0/1278541653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a68101de0 0x7f0a6819a360 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.688+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/1278541653 >> 172.21.15.22:0/1278541653 conn(0x7f0a680fd8e0 msgr2=0x7f0a6810ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.689+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/1278541653 shutdown_connections 2024-09-19T08:15:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.689+0000 7f0a6e8bc640 1 -- 172.21.15.22:0/1278541653 wait complete. 2024-09-19T08:15:58.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:58 smithi064 ceph-mon[34900]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-19T08:15:58.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.857+0000 7f3322136640 1 -- 172.21.15.22:0/1275887235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f331c101dd0 msgr2=0x7f331c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.857+0000 7f3322136640 1 --2- 172.21.15.22:0/1275887235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f331c101dd0 0x7f331c1021b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f330c009900 tx=0x7f330c02a410 comp rx=0 tx=0).stop 2024-09-19T08:15:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.857+0000 7f3322136640 1 -- 172.21.15.22:0/1275887235 shutdown_connections 2024-09-19T08:15:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.857+0000 7f3322136640 1 --2- 172.21.15.22:0/1275887235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f331c106f00 0x7f331c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.857+0000 7f3322136640 1 --2- 172.21.15.22:0/1275887235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f331c102780 0x7f331c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.857+0000 7f3322136640 1 --2- 172.21.15.22:0/1275887235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f331c101dd0 0x7f331c1021b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.858+0000 7f3322136640 1 -- 172.21.15.22:0/1275887235 >> 172.21.15.22:0/1275887235 conn(0x7f331c0fd890 msgr2=0x7f331c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.858+0000 7f3322136640 1 -- 172.21.15.22:0/1275887235 shutdown_connections 2024-09-19T08:15:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.858+0000 7f3322136640 1 -- 172.21.15.22:0/1275887235 wait complete. 2024-09-19T08:15:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.858+0000 7f3322136640 1 Processor -- start 2024-09-19T08:15:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.859+0000 7f3322136640 1 -- start start 2024-09-19T08:15:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.859+0000 7f3322136640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f331c101dd0 0x7f331c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.859+0000 7f3322136640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f331c102780 0x7f331c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.859+0000 7f3322136640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f331c106f00 0x7f331c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.859+0000 7f3322136640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f331c111fb0 con 0x7f331c102780 2024-09-19T08:15:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.859+0000 7f3322136640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f331c111e30 con 0x7f331c101dd0 2024-09-19T08:15:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.859+0000 7f3322136640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f331c112130 con 0x7f331c106f00 2024-09-19T08:15:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.859+0000 7f331affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f331c102780 0x7f331c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.859+0000 7f331affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f331c102780 0x7f331c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33916/0 (socket says 172.21.15.22:33916) 2024-09-19T08:15:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.859+0000 7f331affd640 1 -- 172.21.15.22:0/1665361621 learned_addr learned my addr 172.21.15.22:0/1665361621 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:15:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.859+0000 7f331b7fe640 1 --2- 172.21.15.22:0/1665361621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f331c101dd0 0x7f331c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.860+0000 7f331affd640 1 -- 172.21.15.22:0/1665361621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f331c106f00 msgr2=0x7f331c074f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:15:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.860+0000 7f331affd640 1 --2- 172.21.15.22:0/1665361621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f331c106f00 0x7f331c074f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.860+0000 7f331affd640 1 -- 172.21.15.22:0/1665361621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f331c101dd0 msgr2=0x7f331c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.860+0000 7f331affd640 1 --2- 172.21.15.22:0/1665361621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f331c101dd0 0x7f331c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.860+0000 7f331affd640 1 -- 172.21.15.22:0/1665361621 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331c1a68f0 con 0x7f331c102780 2024-09-19T08:15:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.860+0000 7f331affd640 1 --2- 172.21.15.22:0/1665361621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f331c102780 0x7f331c074a30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f331000e9f0 tx=0x7f331000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.860+0000 7f3318ff9640 1 -- 172.21.15.22:0/1665361621 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331000cd80 con 0x7f331c102780 2024-09-19T08:15:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.860+0000 7f3318ff9640 1 -- 172.21.15.22:0/1665361621 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3310010910 con 0x7f331c102780 2024-09-19T08:15:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.860+0000 7f3322136640 1 -- 172.21.15.22:0/1665361621 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331c1a6be0 con 0x7f331c102780 2024-09-19T08:15:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.860+0000 7f3318ff9640 1 -- 172.21.15.22:0/1665361621 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3310018990 con 0x7f331c102780 2024-09-19T08:15:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.860+0000 7f3322136640 1 -- 172.21.15.22:0/1665361621 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f331c1a7170 con 0x7f331c102780 2024-09-19T08:15:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.861+0000 7f3322136640 1 -- 172.21.15.22:0/1665361621 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3308005290 con 0x7f331c102780 2024-09-19T08:15:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.864+0000 7f3318ff9640 1 -- 172.21.15.22:0/1665361621 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f3310018b30 con 0x7f331c102780 2024-09-19T08:15:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.865+0000 7f3318ff9640 1 --2- 172.21.15.22:0/1665361621 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f33040a3550 0x7f33040a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:15:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.865+0000 7f3318ff9640 1 -- 172.21.15.22:0/1665361621 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f33100d0430 con 0x7f331c102780 2024-09-19T08:15:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.865+0000 7f331b7fe640 1 --2- 172.21.15.22:0/1665361621 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f33040a3550 0x7f33040a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:15:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.865+0000 7f331b7fe640 1 --2- 172.21.15.22:0/1665361621 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f33040a3550 0x7f33040a5a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f330c000c00 tx=0x7f330c005b50 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:15:58.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:58.869+0000 7f3318ff9640 1 -- 172.21.15.22:0/1665361621 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f331009cbd0 con 0x7f331c102780 2024-09-19T08:15:59.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:58 smithi157 ceph-mon[35143]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-19T08:15:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:59.251+0000 7f3322136640 1 -- 172.21.15.22:0/1665361621 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3308005580 con 0x7f331c102780 2024-09-19T08:15:59.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:59.252+0000 7f3318ff9640 1 -- 172.21.15.22:0/1665361621 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11906 (secure 0 0 0) 0x7f331009c4d0 con 0x7f331c102780 2024-09-19T08:15:59.252 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:15:59.252 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"btime":"2024-09-19T08:15:00:109390+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:15:00.059208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:15:59.252 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-09-19T08:15:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:59.255+0000 7f3322136640 1 -- 172.21.15.22:0/1665361621 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f33040a3550 msgr2=0x7f33040a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:59.255+0000 7f3322136640 1 --2- 172.21.15.22:0/1665361621 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f33040a3550 0x7f33040a5a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f330c000c00 tx=0x7f330c005b50 comp rx=0 tx=0).stop 2024-09-19T08:15:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:59.255+0000 7f3322136640 1 -- 172.21.15.22:0/1665361621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f331c102780 msgr2=0x7f331c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:15:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:59.255+0000 7f3322136640 1 --2- 172.21.15.22:0/1665361621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f331c102780 0x7f331c074a30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f331000e9f0 tx=0x7f331000eeb0 comp rx=0 tx=0).stop 2024-09-19T08:15:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:59.255+0000 7f3322136640 1 -- 172.21.15.22:0/1665361621 shutdown_connections 2024-09-19T08:15:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:59.255+0000 7f3322136640 1 --2- 172.21.15.22:0/1665361621 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f33040a3550 0x7f33040a5a10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:59.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:59.255+0000 7f3322136640 1 --2- 172.21.15.22:0/1665361621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f331c106f00 0x7f331c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:59.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:59.255+0000 7f3322136640 1 --2- 172.21.15.22:0/1665361621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f331c102780 0x7f331c074a30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:59.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:59.255+0000 7f3322136640 1 --2- 172.21.15.22:0/1665361621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f331c101dd0 0x7f331c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:15:59.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:59.255+0000 7f3322136640 1 -- 172.21.15.22:0/1665361621 >> 172.21.15.22:0/1665361621 conn(0x7f331c0fd890 msgr2=0x7f331c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:15:59.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:59.256+0000 7f3322136640 1 -- 172.21.15.22:0/1665361621 shutdown_connections 2024-09-19T08:15:59.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:15:59.256+0000 7f3322136640 1 -- 172.21.15.22:0/1665361621 wait complete. 2024-09-19T08:15:59.571 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id bdd9f7b8 --release --await' 2024-09-19T08:15:59.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id bdd9f7b8 --release --await 2024-09-19T08:15:59.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:59 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1278541653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:00.663 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:15:59 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1665361621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:00.664 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:59 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1278541653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:00.664 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:15:59 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1665361621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:00.665 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:59 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1278541653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:00.665 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:15:59 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1665361621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:00.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:16:00.667 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:00.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:00 smithi022 ceph-mon[30963]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T08:16:00.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:00 smithi064 ceph-mon[34900]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T08:16:00.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.920+0000 7fa5f8456640 1 -- 172.21.15.22:0/1384279141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5f010eb10 msgr2=0x7fa5f010eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.920+0000 7fa5f8456640 1 --2- 172.21.15.22:0/1384279141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5f010eb10 0x7fa5f010eef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00b9f0 tx=0x7fa5ec029ea0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.920+0000 7fa5f8456640 1 -- 172.21.15.22:0/1384279141 shutdown_connections 2024-09-19T08:16:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.920+0000 7fa5f8456640 1 --2- 172.21.15.22:0/1384279141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0109b90 0x7fa5f01158a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.920+0000 7fa5f8456640 1 --2- 172.21.15.22:0/1384279141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f01091f0 0x7fa5f0109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.920+0000 7fa5f8456640 1 --2- 172.21.15.22:0/1384279141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5f010eb10 0x7fa5f010eef0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.920+0000 7fa5f8456640 1 -- 172.21.15.22:0/1384279141 >> 172.21.15.22:0/1384279141 conn(0x7fa5f006e2d0 msgr2=0x7fa5f006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.921+0000 7fa5f8456640 1 -- 172.21.15.22:0/1384279141 shutdown_connections 2024-09-19T08:16:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.921+0000 7fa5f8456640 1 -- 172.21.15.22:0/1384279141 wait complete. 2024-09-19T08:16:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.922+0000 7fa5f8456640 1 Processor -- start 2024-09-19T08:16:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.922+0000 7fa5f8456640 1 -- start start 2024-09-19T08:16:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.922+0000 7fa5f8456640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5f01091f0 0x7fa5f01a71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.922+0000 7fa5f8456640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f0109b90 0x7fa5f01a7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.922+0000 7fa5f8456640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f010eb10 0x7fa5f01abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.922+0000 7fa5f8456640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01185a0 con 0x7fa5f010eb10 2024-09-19T08:16:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.922+0000 7fa5f8456640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f0118420 con 0x7fa5f01091f0 2024-09-19T08:16:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.922+0000 7fa5f8456640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f0118720 con 0x7fa5f0109b90 2024-09-19T08:16:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.922+0000 7fa5f59f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f0109b90 0x7fa5f01a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.923+0000 7fa5f59f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f0109b90 0x7fa5f01a7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42606/0 (socket says 172.21.15.22:42606) 2024-09-19T08:16:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.923+0000 7fa5f59f1640 1 -- 172.21.15.22:0/1117334987 learned_addr learned my addr 172.21.15.22:0/1117334987 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.923+0000 7fa5f61f2640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5f01091f0 0x7fa5f01a71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.923+0000 7fa5f69f3640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f010eb10 0x7fa5f01abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.923+0000 7fa5f61f2640 1 -- 172.21.15.22:0/1117334987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f0109b90 msgr2=0x7fa5f01a7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.923+0000 7fa5f61f2640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f0109b90 0x7fa5f01a7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.923+0000 7fa5f61f2640 1 -- 172.21.15.22:0/1117334987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f010eb10 msgr2=0x7fa5f01abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.923+0000 7fa5f61f2640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f010eb10 0x7fa5f01abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.923+0000 7fa5f61f2640 1 -- 172.21.15.22:0/1117334987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f01ac180 con 0x7fa5f01091f0 2024-09-19T08:16:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.923+0000 7fa5f59f1640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f0109b90 0x7fa5f01a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.923+0000 7fa5f69f3640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f010eb10 0x7fa5f01abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.923+0000 7fa5f61f2640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5f01091f0 0x7fa5f01a71d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec035040 tx=0x7fa5ec0326f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.924+0000 7fa5e77fe640 1 -- 172.21.15.22:0/1117334987 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec03f030 con 0x7fa5f01091f0 2024-09-19T08:16:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.924+0000 7fa5e77fe640 1 -- 172.21.15.22:0/1117334987 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5ec02ada0 con 0x7fa5f01091f0 2024-09-19T08:16:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.924+0000 7fa5e77fe640 1 -- 172.21.15.22:0/1117334987 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec037070 con 0x7fa5f01091f0 2024-09-19T08:16:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.924+0000 7fa5f8456640 1 -- 172.21.15.22:0/1117334987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f01ac410 con 0x7fa5f01091f0 2024-09-19T08:16:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.924+0000 7fa5f8456640 1 -- 172.21.15.22:0/1117334987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f01ac8a0 con 0x7fa5f01091f0 2024-09-19T08:16:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.927+0000 7fa5e77fe640 1 -- 172.21.15.22:0/1117334987 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fa5ec042070 con 0x7fa5f01091f0 2024-09-19T08:16:00.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.927+0000 7fa5fa82f740 1 Processor -- start 2024-09-19T08:16:00.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.927+0000 7fa5fa82f740 1 -- start start 2024-09-19T08:16:00.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.928+0000 7fa5fa82f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814060 0x564b68814440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.928+0000 7fa5fa82f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814980 0x564b6881ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.928+0000 7fa5fa82f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b6881f240 0x564b68821630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.928+0000 7fa5fa82f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564b68779620 con 0x564b68814060 2024-09-19T08:16:00.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.928+0000 7fa5fa82f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564b68720e50 con 0x564b68814980 2024-09-19T08:16:00.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.928+0000 7fa5fa82f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564b687755e0 con 0x564b6881f240 2024-09-19T08:16:00.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.928+0000 7fa5f69f3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814060 0x564b68814440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.928+0000 7fa5f69f3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814060 0x564b68814440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33964/0 (socket says 172.21.15.22:33964) 2024-09-19T08:16:00.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.928+0000 7fa5f69f3640 1 -- 172.21.15.22:0/1935669409 learned_addr learned my addr 172.21.15.22:0/1935669409 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:00.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.928+0000 7fa5f59f1640 1 --2- 172.21.15.22:0/1935669409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b6881f240 0x564b68821630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.928+0000 7fa5f69f3640 1 -- 172.21.15.22:0/1935669409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b6881f240 msgr2=0x564b68821630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:00.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.928+0000 7fa5f69f3640 1 --2- 172.21.15.22:0/1935669409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b6881f240 0x564b68821630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.928+0000 7fa5f69f3640 1 -- 172.21.15.22:0/1935669409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814980 msgr2=0x564b6881ed00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:16:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.929+0000 7fa5f61f2640 1 --2- 172.21.15.22:0/1935669409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814980 0x564b6881ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.930+0000 7fa5f69f3640 1 --2- 172.21.15.22:0/1935669409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814980 0x564b6881ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.930+0000 7fa5f69f3640 1 -- 172.21.15.22:0/1935669409 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b68711d40 con 0x564b68814060 2024-09-19T08:16:00.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.930+0000 7fa5e77fe640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa5dc0a3550 0x7fa5dc0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.930+0000 7fa5f69f3640 1 --2- 172.21.15.22:0/1935669409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814060 0x564b68814440 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800b530 tx=0x7fa5e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffd6bb4ed623791 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.931+0000 7fa5e77fe640 1 -- 172.21.15.22:0/1117334987 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa5ec0ef5e0 con 0x7fa5f01091f0 2024-09-19T08:16:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.931+0000 7fa5f61f2640 1 --2- 172.21.15.22:0/1935669409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814980 0x564b6881ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:16:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.931+0000 7fa5e4dea640 1 -- 172.21.15.22:0/1935669409 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8008860 con 0x564b68814060 2024-09-19T08:16:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.931+0000 7fa5e4dea640 1 -- 172.21.15.22:0/1935669409 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5e801c070 con 0x564b68814060 2024-09-19T08:16:00.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.931+0000 7fa5f61f2640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa5dc0a3550 0x7fa5dc0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.932+0000 7fa5e4dea640 1 -- 172.21.15.22:0/1935669409 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80163d0 con 0x564b68814060 2024-09-19T08:16:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.932+0000 7fa5fa82f740 1 -- 172.21.15.22:0/1935669409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814060 msgr2=0x564b68814440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.932+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/1935669409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814060 0x564b68814440 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800b530 tx=0x7fa5e800a620 comp rx=0 tx=0).stop 2024-09-19T08:16:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.932+0000 7fa5f61f2640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa5dc0a3550 0x7fa5dc0a5a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0ecd30 tx=0x7fa5ec005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:00.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.932+0000 7fa5fa82f740 1 -- 172.21.15.22:0/1935669409 shutdown_connections 2024-09-19T08:16:00.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.932+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/1935669409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b6881f240 0x564b68821630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.932+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/1935669409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814980 0x564b6881ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.932+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/1935669409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814060 0x564b68814440 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.932+0000 7fa5fa82f740 1 -- 172.21.15.22:0/1935669409 >> 172.21.15.22:0/1935669409 conn(0x564b688105f0 msgr2=0x564b688109d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:00.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.932+0000 7fa5fa82f740 1 -- 172.21.15.22:0/1935669409 shutdown_connections 2024-09-19T08:16:00.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.932+0000 7fa5fa82f740 1 -- 172.21.15.22:0/1935669409 wait complete. 2024-09-19T08:16:00.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.933+0000 7fa5fa82f740 1 -- start start 2024-09-19T08:16:00.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.933+0000 7fa5fa82f740 1 Processor -- start 2024-09-19T08:16:00.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.934+0000 7fa5fa82f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b68814060 0x564b6881c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.934+0000 7fa5fa82f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814980 0x564b6881c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.934+0000 7fa5fa82f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b6881f240 0x564b6881ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.934+0000 7fa5f59f1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814980 0x564b6881c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.934+0000 7fa5f59f1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814980 0x564b6881c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33968/0 (socket says 172.21.15.22:33968) 2024-09-19T08:16:00.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.934+0000 7fa5f59f1640 1 -- 172.21.15.22:0/3878144048 learned_addr learned my addr 172.21.15.22:0/3878144048 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:00.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.934+0000 7fa5fa82f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564b68821eb0 con 0x564b68814980 2024-09-19T08:16:00.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.934+0000 7fa5fa82f740 1 -- 172.21.15.22:0/3878144048 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564b68701f10 con 0x564b6881f240 2024-09-19T08:16:00.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.934+0000 7fa5fa82f740 1 -- 172.21.15.22:0/3878144048 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564b68822030 con 0x564b68814060 2024-09-19T08:16:00.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.934+0000 7fa5fa82f740 10 client.0 _ll_register_callbacks cb 0x564b68725db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:00.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.935+0000 7fa5f59f1640 1 -- 172.21.15.22:0/3878144048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b68814060 msgr2=0x564b6881c000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:16:00.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.935+0000 7fa5f59f1640 1 --2- 172.21.15.22:0/3878144048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b68814060 0x564b6881c000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.935+0000 7fa5f59f1640 1 -- 172.21.15.22:0/3878144048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b6881f240 msgr2=0x564b6881ca80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:16:00.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.935+0000 7fa5f59f1640 1 --2- 172.21.15.22:0/3878144048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b6881f240 0x564b6881ca80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.935+0000 7fa5f59f1640 1 -- 172.21.15.22:0/3878144048 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b68488580 con 0x564b68814980 2024-09-19T08:16:00.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.935+0000 7fa5f59f1640 1 --2- 172.21.15.22:0/3878144048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814980 0x564b6881c540 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00095d0 tx=0x7fa5e0009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60366a3da8565e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:00.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.935+0000 7fa5d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:00.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.936+0000 7fa5d9ffb640 1 -- 172.21.15.22:0/3878144048 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e000d790 con 0x564b68814980 2024-09-19T08:16:00.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.936+0000 7fa5fa82f740 1 -- 172.21.15.22:0/3878144048 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e801c070 con 0x564b68814980 2024-09-19T08:16:00.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.936+0000 7fa5d9ffb640 1 -- 172.21.15.22:0/3878144048 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa5e000d930 con 0x564b68814980 2024-09-19T08:16:00.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.937+0000 7fa5d9ffb640 1 -- 172.21.15.22:0/3878144048 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa5e0027050 con 0x564b68814980 2024-09-19T08:16:00.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.937+0000 7fa5fa82f740 10 client.15606 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:00.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.937+0000 7fa5fa82f740 1 -- 172.21.15.22:0/3878144048 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564b68552960 con 0x564b68814980 2024-09-19T08:16:00.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.938+0000 7fa5d9ffb640 1 -- 172.21.15.22:0/3878144048 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5e801c070 con 0x564b68814980 2024-09-19T08:16:00.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.938+0000 7fa5d9ffb640 1 -- 172.21.15.22:0/3878144048 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa5e00193d0 con 0x564b68814980 2024-09-19T08:16:00.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.939+0000 7fa5fa82f740 10 client.15606 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:00.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.939+0000 7fa5fa82f740 10 client.15606 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:16:00.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.939+0000 7fa5fa82f740 20 client.15606 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:00.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.939+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/3878144048 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564b6882ee50 0x564b68831240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.939+0000 7fa5fa82f740 4 client.15606 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:16:00.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.939+0000 7fa5fa82f740 1 -- 172.21.15.22:0/3878144048 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b68786b90 con 0x564b6882ee50 2024-09-19T08:16:00.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.939+0000 7fa5f69f3640 1 --2- 172.21.15.22:0/3878144048 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564b6882ee50 0x564b68831240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.940+0000 7fa5f69f3640 1 --2- 172.21.15.22:0/3878144048 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564b6882ee50 0x564b68831240 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d66c43c762104fb server_cookie=f19bf4802535f6f7 in_seq=0 out_seq=0 2024-09-19T08:16:00.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.940+0000 7fa5d9ffb640 10 client.15606 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.941+0000 7fa5d9ffb640 1 -- 172.21.15.22:0/3878144048 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b68786b90 con 0x564b6882ee50 2024-09-19T08:16:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.941+0000 7fa5d9ffb640 10 client.15606 handle_command_reply: tid=0 2024-09-19T08:16:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.941+0000 7fa5fa82f740 1 client.15606 shutdown 2024-09-19T08:16:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.941+0000 7fa5fa82f740 1 -- 172.21.15.22:0/3878144048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814980 msgr2=0x564b6881c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.941+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/3878144048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814980 0x564b6881c540 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00095d0 tx=0x7fa5e0009a90 comp rx=0 tx=0).stop 2024-09-19T08:16:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.942+0000 7fa5fa82f740 1 -- 172.21.15.22:0/3878144048 shutdown_connections 2024-09-19T08:16:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.942+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/3878144048 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564b6882ee50 0x564b68831240 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.942+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/3878144048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b6881f240 0x564b6881ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.942+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/3878144048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814980 0x564b6881c540 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.942+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/3878144048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b68814060 0x564b6881c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.942+0000 7fa5fa82f740 1 -- 172.21.15.22:0/3878144048 >> 172.21.15.22:0/3878144048 conn(0x564b688105f0 msgr2=0x564b68816320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.942+0000 7fa5d9ffb640 0 client.15606 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.942+0000 7fa5fa82f740 1 -- 172.21.15.22:0/3878144048 shutdown_connections 2024-09-19T08:16:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.942+0000 7fa5fa82f740 1 -- 172.21.15.22:0/3878144048 wait complete. 2024-09-19T08:16:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.942+0000 7fa5fa82f740 20 client.15606 trim_cache size 0 max 16384 2024-09-19T08:16:00.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.971+0000 7fa5fa82f740 1 Processor -- start 2024-09-19T08:16:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.972+0000 7fa5fa82f740 1 -- start start 2024-09-19T08:16:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.972+0000 7fa5fa82f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814060 0x564b68836b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.972+0000 7fa5fa82f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814980 0x564b68837090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.972+0000 7fa5fa82f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b6881f240 0x564b68819280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.972+0000 7fa5fa82f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01bf050 con 0x564b68814980 2024-09-19T08:16:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.972+0000 7fa5fa82f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01bc920 con 0x564b68814060 2024-09-19T08:16:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.972+0000 7fa5fa82f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01ba770 con 0x564b6881f240 2024-09-19T08:16:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.972+0000 7fa5f69f3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814060 0x564b68836b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.973+0000 7fa5f59f1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814980 0x564b68837090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.973+0000 7fa5f59f1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814980 0x564b68837090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33974/0 (socket says 172.21.15.22:33974) 2024-09-19T08:16:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.973+0000 7fa5f59f1640 1 -- 172.21.15.22:0/851648561 learned_addr learned my addr 172.21.15.22:0/851648561 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.973+0000 7fa5f69f3640 1 --2- 172.21.15.22:0/851648561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b6881f240 0x564b68819280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.973+0000 7fa5f59f1640 1 -- 172.21.15.22:0/851648561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b6881f240 msgr2=0x564b68819280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:00.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.973+0000 7fa5f59f1640 1 --2- 172.21.15.22:0/851648561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b6881f240 0x564b68819280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.973+0000 7fa5f59f1640 1 -- 172.21.15.22:0/851648561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814060 msgr2=0x564b68836b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:00.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.973+0000 7fa5f59f1640 1 --2- 172.21.15.22:0/851648561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814060 0x564b68836b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.973+0000 7fa5f59f1640 1 -- 172.21.15.22:0/851648561 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b68810420 con 0x564b68814980 2024-09-19T08:16:00.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.974+0000 7fa5f59f1640 1 --2- 172.21.15.22:0/851648561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814980 0x564b68837090 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00095a0 tx=0x7fa5e0009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3feb7686f7b81ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:00.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.974+0000 7fa5dbfff640 1 -- 172.21.15.22:0/851648561 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e00193d0 con 0x564b68814980 2024-09-19T08:16:00.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.974+0000 7fa5dbfff640 1 -- 172.21.15.22:0/851648561 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5e002f070 con 0x564b68814980 2024-09-19T08:16:00.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.974+0000 7fa5dbfff640 1 -- 172.21.15.22:0/851648561 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e000d790 con 0x564b68814980 2024-09-19T08:16:00.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.975+0000 7fa5fa82f740 1 -- 172.21.15.22:0/851648561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814980 msgr2=0x564b68837090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:00.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.975+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/851648561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814980 0x564b68837090 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00095a0 tx=0x7fa5e0009280 comp rx=0 tx=0).stop 2024-09-19T08:16:00.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.975+0000 7fa5fa82f740 1 -- 172.21.15.22:0/851648561 shutdown_connections 2024-09-19T08:16:00.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.975+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/851648561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b6881f240 0x564b68819280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.975+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/851648561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b68814980 0x564b68837090 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.975+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/851648561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814060 0x564b68836b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.975+0000 7fa5fa82f740 1 -- 172.21.15.22:0/851648561 >> 172.21.15.22:0/851648561 conn(0x564b688105f0 msgr2=0x564b6881a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:00.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.976+0000 7fa5fa82f740 1 -- 172.21.15.22:0/851648561 shutdown_connections 2024-09-19T08:16:00.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.976+0000 7fa5fa82f740 1 -- 172.21.15.22:0/851648561 wait complete. 2024-09-19T08:16:00.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.976+0000 7fa5fa82f740 1 -- start start 2024-09-19T08:16:00.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.976+0000 7fa5fa82f740 1 Processor -- start 2024-09-19T08:16:00.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.977+0000 7fa5fa82f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b68814060 0x564b68827300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.977+0000 7fa5fa82f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814980 0x564b68827840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.977+0000 7fa5fa82f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b6881f240 0x564b688242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.977+0000 7fa5fa82f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01bade0 con 0x564b6881f240 2024-09-19T08:16:00.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.977+0000 7fa5fa82f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01bc250 con 0x564b68814980 2024-09-19T08:16:00.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.977+0000 7fa5fa82f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01bb4b0 con 0x564b68814060 2024-09-19T08:16:00.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.977+0000 7fa5fa82f740 10 client.0 _ll_register_callbacks cb 0x7fa5e000f510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:00.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.977+0000 7fa5f69f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b68814060 0x564b68827300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.978+0000 7fa5f69f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b68814060 0x564b68827300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42650/0 (socket says 172.21.15.22:42650) 2024-09-19T08:16:00.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.978+0000 7fa5f69f3640 1 -- 172.21.15.22:0/1664918707 learned_addr learned my addr 172.21.15.22:0/1664918707 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:00.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.978+0000 7fa5f69f3640 1 --2- 172.21.15.22:0/1664918707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b6881f240 0x564b688242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.978+0000 7fa5f59f1640 1 --2- 172.21.15.22:0/1664918707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814980 0x564b68827840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.978+0000 7fa5f59f1640 1 -- 172.21.15.22:0/1664918707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b68814060 msgr2=0x564b68827300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:00.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.978+0000 7fa5f59f1640 1 --2- 172.21.15.22:0/1664918707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b68814060 0x564b68827300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.978+0000 7fa5f59f1640 1 -- 172.21.15.22:0/1664918707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b6881f240 msgr2=0x564b688242d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:00.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.978+0000 7fa5f59f1640 1 --2- 172.21.15.22:0/1664918707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b6881f240 0x564b688242d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.978+0000 7fa5f59f1640 1 -- 172.21.15.22:0/1664918707 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b6882d950 con 0x564b68814980 2024-09-19T08:16:00.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.979+0000 7fa5f69f3640 1 --2- 172.21.15.22:0/1664918707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b6881f240 0x564b688242d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:00.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.979+0000 7fa5f59f1640 1 --2- 172.21.15.22:0/1664918707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814980 0x564b68827840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00095d0 tx=0x7fa5e0005690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55d6f6d05071686b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:00.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.979+0000 7fa5d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:16:00.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.979+0000 7fa5d9ffb640 1 -- 172.21.15.22:0/1664918707 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e00195e0 con 0x564b68814980 2024-09-19T08:16:00.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.979+0000 7fa5d9ffb640 1 -- 172.21.15.22:0/1664918707 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa5e000d790 con 0x564b68814980 2024-09-19T08:16:00.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.980+0000 7fa5fa82f740 1 -- 172.21.15.22:0/1664918707 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e002f070 con 0x564b68814980 2024-09-19T08:16:00.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.980+0000 7fa5d9ffb640 1 -- 172.21.15.22:0/1664918707 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa5e000d990 con 0x564b68814980 2024-09-19T08:16:00.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.980+0000 7fa5fa82f740 10 client.25286 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:00.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.980+0000 7fa5fa82f740 1 -- 172.21.15.22:0/1664918707 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564b68813e90 con 0x564b68814980 2024-09-19T08:16:00.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.980+0000 7fa5d9ffb640 1 -- 172.21.15.22:0/1664918707 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5e0037030 con 0x564b68814980 2024-09-19T08:16:00.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.981+0000 7fa5d9ffb640 1 -- 172.21.15.22:0/1664918707 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa5e001aaf0 con 0x564b68814980 2024-09-19T08:16:00.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.981+0000 7fa5fa82f740 10 client.25286 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:00.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.981+0000 7fa5fa82f740 10 client.25286 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:16:00.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.981+0000 7fa5fa82f740 20 client.25286 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:00.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.981+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/1664918707 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564b6889a470 0x564b6889c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:00.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.981+0000 7fa5fa82f740 4 client.25286 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bdd9f7b8", "release": true, "await": true}] 2024-09-19T08:16:00.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.981+0000 7fa5fa82f740 1 -- 172.21.15.22:0/1664918707 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bdd9f7b8", "release": true, "await": true}) -- 0x7fa5e002f070 con 0x564b6889a470 2024-09-19T08:16:00.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.981+0000 7fa5f69f3640 1 --2- 172.21.15.22:0/1664918707 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564b6889a470 0x564b6889c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:00.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.981+0000 7fa5d9ffb640 10 client.25286 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:00.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.981+0000 7fa5f69f3640 1 --2- 172.21.15.22:0/1664918707 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564b6889a470 0x564b6889c890 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc97ebf401ab834b server_cookie=e77a21c26898094c in_seq=0 out_seq=0 2024-09-19T08:16:00.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.984+0000 7fa5d9ffb640 1 -- 172.21.15.22:0/1664918707 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7fa5e002f070 con 0x564b6889a470 2024-09-19T08:16:00.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.984+0000 7fa5d9ffb640 10 client.25286 handle_command_reply: tid=0 2024-09-19T08:16:00.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.984+0000 7fa5fa82f740 1 client.25286 shutdown 2024-09-19T08:16:00.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.984+0000 7fa5fa82f740 1 -- 172.21.15.22:0/1664918707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814980 msgr2=0x564b68827840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:00.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.984+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/1664918707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814980 0x564b68827840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00095d0 tx=0x7fa5e0005690 comp rx=0 tx=0).stop 2024-09-19T08:16:00.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.985+0000 7fa5fa82f740 1 -- 172.21.15.22:0/1664918707 shutdown_connections 2024-09-19T08:16:00.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.985+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/1664918707 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564b6889a470 0x564b6889c890 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.985+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/1664918707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b6881f240 0x564b688242d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.985+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/1664918707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b68814980 0x564b68827840 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.985+0000 7fa5fa82f740 1 --2- 172.21.15.22:0/1664918707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564b68814060 0x564b68827300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.985+0000 7fa5fa82f740 1 -- 172.21.15.22:0/1664918707 >> 172.21.15.22:0/1664918707 conn(0x564b688105f0 msgr2=0x564b6880ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:00.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.985+0000 7fa5d9ffb640 0 client.25286 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:00.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.985+0000 7fa5fa82f740 1 -- 172.21.15.22:0/1664918707 shutdown_connections 2024-09-19T08:16:00.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.986+0000 7fa5fa82f740 1 -- 172.21.15.22:0/1664918707 wait complete. 2024-09-19T08:16:00.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.986+0000 7fa5fa82f740 20 client.25286 trim_cache size 0 max 16384 2024-09-19T08:16:00.987 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:16:00.987 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 25, 2024-09-19T08:16:00.987 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:16:00.987 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 4, 2024-09-19T08:16:00.987 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:16:00.987 INFO:teuthology.orchestra.run.smithi022.stdout: "bdd9f7b8": { 2024-09-19T08:16:00.987 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 4, 2024-09-19T08:16:00.987 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:16:00.987 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:16:00.987 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:16:00.987 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:16:00.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:16:00.987 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:16:00.988 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 131.2, 2024-09-19T08:16:00.988 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:16:00.988 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:16:00.988 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:16:00.988 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:16:00.988 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:16:00.988 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:16:00.988 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:00.988 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:00.988 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:00.988 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:00.988 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:00.988 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:16:00.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.986+0000 7fa5f8456640 1 -- 172.21.15.22:0/1117334987 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa5dc0a3550 msgr2=0x7fa5dc0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:00.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.987+0000 7fa5f8456640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa5dc0a3550 0x7fa5dc0a5a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0ecd30 tx=0x7fa5ec005f90 comp rx=0 tx=0).stop 2024-09-19T08:16:00.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.987+0000 7fa5f8456640 1 -- 172.21.15.22:0/1117334987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5f01091f0 msgr2=0x7fa5f01a71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:00.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.987+0000 7fa5f8456640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5f01091f0 0x7fa5f01a71d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec035040 tx=0x7fa5ec0326f0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.987+0000 7fa5f8456640 1 -- 172.21.15.22:0/1117334987 shutdown_connections 2024-09-19T08:16:00.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.987+0000 7fa5f8456640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa5dc0a3550 0x7fa5dc0a5a10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.987+0000 7fa5f8456640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f010eb10 0x7fa5f01abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.987+0000 7fa5f8456640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f0109b90 0x7fa5f01a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.987+0000 7fa5f8456640 1 --2- 172.21.15.22:0/1117334987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5f01091f0 0x7fa5f01a71d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:00.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.987+0000 7fa5f8456640 1 -- 172.21.15.22:0/1117334987 >> 172.21.15.22:0/1117334987 conn(0x7fa5f006e2d0 msgr2=0x7fa5f0072470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:00.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.987+0000 7fa5f8456640 1 -- 172.21.15.22:0/1117334987 shutdown_connections 2024-09-19T08:16:00.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:00.987+0000 7fa5f8456640 1 -- 172.21.15.22:0/1117334987 wait complete. 2024-09-19T08:16:01.002 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:00 smithi157 ceph-mon[35143]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T08:16:01.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:16:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:16:00.932+0000 7f98268a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:16:01.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:16:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:16:00.932+0000 7f98268a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:16:01.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:16:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:16:00.932+0000 7f98268a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:16:01.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:16:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:16:00.932+0000 7f98268a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:16:01.257 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:16:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:16:00.932+0000 7f98268a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:16:01.711 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bdd9f7b8', total seconds elapsed: 16.0 2024-09-19T08:16:01.747 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:02.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:02 smithi064 ceph-mon[34900]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T08:16:02.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:02 smithi064 ceph-mon[34900]: mgrmap e25: x(active, since 60s), standbys: z, y 2024-09-19T08:16:02.844 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:02 smithi022 ceph-mon[30963]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T08:16:02.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:02 smithi022 ceph-mon[30963]: mgrmap e25: x(active, since 60s), standbys: z, y 2024-09-19T08:16:03.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:02 smithi157 ceph-mon[35143]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T08:16:03.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:02 smithi157 ceph-mon[35143]: mgrmap e25: x(active, since 60s), standbys: z, y 2024-09-19T08:16:03.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.976+0000 7f02abbf7640 1 -- 172.21.15.22:0/784938561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02a4103740 msgr2=0x7f02a4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:03.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.976+0000 7f02abbf7640 1 --2- 172.21.15.22:0/784938561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02a4103740 0x7f02a4107b80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0294009930 tx=0x7f029402a420 comp rx=0 tx=0).stop 2024-09-19T08:16:03.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.976+0000 7f02abbf7640 1 -- 172.21.15.22:0/784938561 shutdown_connections 2024-09-19T08:16:03.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.976+0000 7f02abbf7640 1 --2- 172.21.15.22:0/784938561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a4108290 0x7f02a410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.976+0000 7f02abbf7640 1 --2- 172.21.15.22:0/784938561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02a4103740 0x7f02a4107b80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.976+0000 7f02abbf7640 1 --2- 172.21.15.22:0/784938561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02a4102d90 0x7f02a4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.976+0000 7f02abbf7640 1 -- 172.21.15.22:0/784938561 >> 172.21.15.22:0/784938561 conn(0x7f02a40fc7d0 msgr2=0x7f02a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.977+0000 7f02abbf7640 1 -- 172.21.15.22:0/784938561 shutdown_connections 2024-09-19T08:16:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.977+0000 7f02abbf7640 1 -- 172.21.15.22:0/784938561 wait complete. 2024-09-19T08:16:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.977+0000 7f02abbf7640 1 Processor -- start 2024-09-19T08:16:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.977+0000 7f02abbf7640 1 -- start start 2024-09-19T08:16:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.978+0000 7f02abbf7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02a4102d90 0x7f02a41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.978+0000 7f02a9993640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02a4102d90 0x7f02a41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.978+0000 7f02a9993640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02a4102d90 0x7f02a41984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34008/0 (socket says 172.21.15.22:34008) 2024-09-19T08:16:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.978+0000 7f02abbf7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02a4103740 0x7f02a4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.978+0000 7f02a9993640 1 -- 172.21.15.22:0/3718706220 learned_addr learned my addr 172.21.15.22:0/3718706220 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:03.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.978+0000 7f02abbf7640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a4108290 0x7f02a419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.978+0000 7f02abbf7640 1 -- 172.21.15.22:0/3718706220 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f02a4112090 con 0x7f02a4102d90 2024-09-19T08:16:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.978+0000 7f02abbf7640 1 -- 172.21.15.22:0/3718706220 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f02a4111f10 con 0x7f02a4103740 2024-09-19T08:16:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.978+0000 7f02abbf7640 1 -- 172.21.15.22:0/3718706220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f02a4112210 con 0x7f02a4108290 2024-09-19T08:16:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.978+0000 7f02aa194640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a4108290 0x7f02a419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.978+0000 7f02a9192640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02a4103740 0x7f02a4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.979+0000 7f02a9993640 1 -- 172.21.15.22:0/3718706220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a4108290 msgr2=0x7f02a419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.979+0000 7f02a9993640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a4108290 0x7f02a419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.979+0000 7f02a9993640 1 -- 172.21.15.22:0/3718706220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02a4103740 msgr2=0x7f02a4198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.979+0000 7f02a9993640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02a4103740 0x7f02a4198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.979+0000 7f02a9993640 1 -- 172.21.15.22:0/3718706220 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a419d480 con 0x7f02a4102d90 2024-09-19T08:16:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.979+0000 7f02a9993640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02a4102d90 0x7f02a41984d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f02a000c920 tx=0x7f02a000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.979+0000 7f029affd640 1 -- 172.21.15.22:0/3718706220 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0007c70 con 0x7f02a4102d90 2024-09-19T08:16:03.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.979+0000 7f02abbf7640 1 -- 172.21.15.22:0/3718706220 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a419d770 con 0x7f02a4102d90 2024-09-19T08:16:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.979+0000 7f029affd640 1 -- 172.21.15.22:0/3718706220 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a0007e10 con 0x7f02a4102d90 2024-09-19T08:16:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.979+0000 7f029affd640 1 -- 172.21.15.22:0/3718706220 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0018890 con 0x7f02a4102d90 2024-09-19T08:16:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.979+0000 7f02abbf7640 1 -- 172.21.15.22:0/3718706220 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a41a5050 con 0x7f02a4102d90 2024-09-19T08:16:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.982+0000 7f029affd640 1 -- 172.21.15.22:0/3718706220 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f02a0018a50 con 0x7f02a4102d90 2024-09-19T08:16:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.982+0000 7f02adfd0740 1 Processor -- start 2024-09-19T08:16:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.982+0000 7f02adfd0740 1 -- start start 2024-09-19T08:16:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.982+0000 7f02adfd0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2bb40 0x562d79c2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.982+0000 7f02adfd0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2a5d0 0x562d79c2aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.982+0000 7f02adfd0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c39780 0x562d79c3bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.982+0000 7f02adfd0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562d79b902f0 con 0x562d79c39780 2024-09-19T08:16:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02adfd0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562d79b37cf0 con 0x562d79c2bb40 2024-09-19T08:16:03.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02adfd0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562d79b8c2b0 con 0x562d79c2a5d0 2024-09-19T08:16:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02a9192640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c39780 0x562d79c3bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f029affd640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f02900a35e0 0x7f02900a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02a9192640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c39780 0x562d79c3bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34018/0 (socket says 172.21.15.22:34018) 2024-09-19T08:16:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02a9192640 1 -- 172.21.15.22:0/1646391529 learned_addr learned my addr 172.21.15.22:0/1646391529 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02aa194640 1 --2- 172.21.15.22:0/1646391529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2bb40 0x562d79c2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02a9993640 1 --2- 172.21.15.22:0/1646391529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2a5d0 0x562d79c2aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f029affd640 1 -- 172.21.15.22:0/3718706220 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f02a00d00c0 con 0x7f02a4102d90 2024-09-19T08:16:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02aa194640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f02900a35e0 0x7f02900a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02a9192640 1 -- 172.21.15.22:0/1646391529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2a5d0 msgr2=0x562d79c2aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02a9192640 1 --2- 172.21.15.22:0/1646391529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2a5d0 0x562d79c2aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02a9192640 1 -- 172.21.15.22:0/1646391529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2bb40 msgr2=0x562d79c2a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02a9192640 1 --2- 172.21.15.22:0/1646391529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2bb40 0x562d79c2a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02a9192640 1 -- 172.21.15.22:0/1646391529 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d79b1c3a0 con 0x562d79c39780 2024-09-19T08:16:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02a9993640 1 --2- 172.21.15.22:0/1646391529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2a5d0 0x562d79c2aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:16:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.983+0000 7f02a9192640 1 --2- 172.21.15.22:0/1646391529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c39780 0x562d79c3bb70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0294009900 tx=0x7f02940316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be5fc03407101c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.984+0000 7f02aa194640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f02900a35e0 0x7f02900a5aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f029c005cc0 tx=0x7f029c004ab0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.984+0000 7f028ffff640 1 -- 172.21.15.22:0/1646391529 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294040070 con 0x562d79c39780 2024-09-19T08:16:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.984+0000 7f028ffff640 1 -- 172.21.15.22:0/1646391529 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0294005020 con 0x562d79c39780 2024-09-19T08:16:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.984+0000 7f028ffff640 1 -- 172.21.15.22:0/1646391529 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294035040 con 0x562d79c39780 2024-09-19T08:16:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.985+0000 7f02adfd0740 1 -- 172.21.15.22:0/1646391529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c39780 msgr2=0x562d79c3bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.985+0000 7f02adfd0740 1 --2- 172.21.15.22:0/1646391529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c39780 0x562d79c3bb70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0294009900 tx=0x7f02940316e0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.985+0000 7f02adfd0740 1 -- 172.21.15.22:0/1646391529 shutdown_connections 2024-09-19T08:16:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.985+0000 7f02adfd0740 1 --2- 172.21.15.22:0/1646391529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c39780 0x562d79c3bb70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.985+0000 7f02adfd0740 1 --2- 172.21.15.22:0/1646391529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2a5d0 0x562d79c2aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.985+0000 7f02adfd0740 1 --2- 172.21.15.22:0/1646391529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2bb40 0x562d79c2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.985+0000 7f02adfd0740 1 -- 172.21.15.22:0/1646391529 >> 172.21.15.22:0/1646391529 conn(0x562d79bf3310 msgr2=0x562d79c26c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.985+0000 7f02adfd0740 1 -- 172.21.15.22:0/1646391529 shutdown_connections 2024-09-19T08:16:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.985+0000 7f02adfd0740 1 -- 172.21.15.22:0/1646391529 wait complete. 2024-09-19T08:16:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.986+0000 7f02adfd0740 1 -- start start 2024-09-19T08:16:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.986+0000 7f02adfd0740 1 Processor -- start 2024-09-19T08:16:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.986+0000 7f02adfd0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2a5d0 0x562d79c36620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.986+0000 7f02adfd0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2bb40 0x562d79c36b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.986+0000 7f02adfd0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c370a0 0x562d79c458d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.986+0000 7f02adfd0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562d79c3c420 con 0x562d79c370a0 2024-09-19T08:16:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.986+0000 7f02adfd0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562d79b4b060 con 0x562d79c2a5d0 2024-09-19T08:16:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.986+0000 7f02adfd0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562d79c3c5a0 con 0x562d79c2bb40 2024-09-19T08:16:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.987+0000 7f02aa194640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c370a0 0x562d79c458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.987+0000 7f02aa194640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c370a0 0x562d79c458d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34020/0 (socket says 172.21.15.22:34020) 2024-09-19T08:16:03.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.987+0000 7f02aa194640 1 -- 172.21.15.22:0/420018722 learned_addr learned my addr 172.21.15.22:0/420018722 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.987+0000 7f02a9993640 1 --2- 172.21.15.22:0/420018722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2a5d0 0x562d79c36620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.987+0000 7f02a9192640 1 --2- 172.21.15.22:0/420018722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2bb40 0x562d79c36b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.987+0000 7f02aa194640 1 -- 172.21.15.22:0/420018722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2bb40 msgr2=0x562d79c36b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.987+0000 7f02aa194640 1 --2- 172.21.15.22:0/420018722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2bb40 0x562d79c36b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.987+0000 7f02aa194640 1 -- 172.21.15.22:0/420018722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2a5d0 msgr2=0x562d79c36620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.987+0000 7f02aa194640 1 --2- 172.21.15.22:0/420018722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2a5d0 0x562d79c36620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.987+0000 7f02aa194640 1 -- 172.21.15.22:0/420018722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d799013c0 con 0x562d79c370a0 2024-09-19T08:16:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.987+0000 7f02aa194640 1 --2- 172.21.15.22:0/420018722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c370a0 0x562d79c458d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f029c011fb0 tx=0x7f029c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ac4c7e8cf49c17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.987+0000 7f028d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.987+0000 7f02adfd0740 10 client.0 _ll_register_callbacks cb 0x562d7989e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.987+0000 7f02adfd0740 1 -- 172.21.15.22:0/420018722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0294005020 con 0x562d79c370a0 2024-09-19T08:16:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.988+0000 7f028d7fa640 1 -- 172.21.15.22:0/420018722 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c01d810 con 0x562d79c370a0 2024-09-19T08:16:03.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.988+0000 7f028d7fa640 1 -- 172.21.15.22:0/420018722 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f029c01d9b0 con 0x562d79c370a0 2024-09-19T08:16:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.988+0000 7f028d7fa640 1 -- 172.21.15.22:0/420018722 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f029c01dbb0 con 0x562d79c370a0 2024-09-19T08:16:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.988+0000 7f02adfd0740 10 client.15624 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.988+0000 7f02adfd0740 1 -- 172.21.15.22:0/420018722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=25}) -- 0x562d79c482b0 con 0x562d79c370a0 2024-09-19T08:16:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.988+0000 7f028d7fa640 1 -- 172.21.15.22:0/420018722 <== mon.0 v2:172.21.15.22:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f029c031420 con 0x562d79c370a0 2024-09-19T08:16:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.988+0000 7f02adfd0740 10 client.15624 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.988+0000 7f02adfd0740 10 client.15624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:16:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.988+0000 7f02adfd0740 20 client.15624 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.988+0000 7f02adfd0740 1 --2- 172.21.15.22:0/420018722 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x562d79c496d0 0x562d79c4bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.988+0000 7f02adfd0740 4 client.15624 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:16:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.988+0000 7f02adfd0740 1 -- 172.21.15.22:0/420018722 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0294005020 con 0x562d79c496d0 2024-09-19T08:16:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.989+0000 7f02a9993640 1 --2- 172.21.15.22:0/420018722 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x562d79c496d0 0x562d79c4bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.989+0000 7f028d7fa640 10 client.15624 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:16:03.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.989+0000 7f02a9993640 1 --2- 172.21.15.22:0/420018722 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x562d79c496d0 0x562d79c4bb90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5c30b90994fb506 server_cookie=c1631dca9ee892e4 in_seq=0 out_seq=0 2024-09-19T08:16:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.990+0000 7f028d7fa640 1 -- 172.21.15.22:0/420018722 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0294005020 con 0x562d79c496d0 2024-09-19T08:16:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.990+0000 7f028d7fa640 10 client.15624 handle_command_reply: tid=0 2024-09-19T08:16:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.990+0000 7f02adfd0740 1 client.15624 shutdown 2024-09-19T08:16:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.990+0000 7f02adfd0740 1 -- 172.21.15.22:0/420018722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c370a0 msgr2=0x562d79c458d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.990+0000 7f02adfd0740 1 --2- 172.21.15.22:0/420018722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c370a0 0x562d79c458d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f029c011fb0 tx=0x7f029c004a40 comp rx=0 tx=0).stop 2024-09-19T08:16:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.990+0000 7f02adfd0740 1 -- 172.21.15.22:0/420018722 shutdown_connections 2024-09-19T08:16:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.990+0000 7f02adfd0740 1 --2- 172.21.15.22:0/420018722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c370a0 0x562d79c458d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.990+0000 7f02adfd0740 1 --2- 172.21.15.22:0/420018722 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x562d79c496d0 0x562d79c4bb90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.990+0000 7f02adfd0740 1 --2- 172.21.15.22:0/420018722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2bb40 0x562d79c36b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.990+0000 7f02adfd0740 1 --2- 172.21.15.22:0/420018722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2a5d0 0x562d79c36620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.990+0000 7f028d7fa640 0 client.15624 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:16:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.990+0000 7f02adfd0740 1 -- 172.21.15.22:0/420018722 >> 172.21.15.22:0/420018722 conn(0x562d79bf3310 msgr2=0x562d79c2c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.990+0000 7f02adfd0740 1 -- 172.21.15.22:0/420018722 shutdown_connections 2024-09-19T08:16:03.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.991+0000 7f02adfd0740 1 -- 172.21.15.22:0/420018722 wait complete. 2024-09-19T08:16:03.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:02.991+0000 7f02adfd0740 20 client.15624 trim_cache size 0 max 16384 2024-09-19T08:16:03.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.017+0000 7f02adfd0740 1 Processor -- start 2024-09-19T08:16:03.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.017+0000 7f02adfd0740 1 -- start start 2024-09-19T08:16:03.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.017+0000 7f02adfd0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2a5d0 0x562d79c386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.017+0000 7f02adfd0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c2bb40 0x562d79c38be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.017+0000 7f02adfd0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c370a0 0x562d79c3ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.017+0000 7f02adfd0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41ade50 con 0x562d79c2bb40 2024-09-19T08:16:03.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.017+0000 7f02adfd0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41ab720 con 0x562d79c370a0 2024-09-19T08:16:03.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.017+0000 7f02adfd0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41a9570 con 0x562d79c2a5d0 2024-09-19T08:16:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.017+0000 7f02a9192640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c2bb40 0x562d79c38be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.017+0000 7f02a9192640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c2bb40 0x562d79c38be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34026/0 (socket says 172.21.15.22:34026) 2024-09-19T08:16:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.017+0000 7f02a9192640 1 -- 172.21.15.22:0/544484710 learned_addr learned my addr 172.21.15.22:0/544484710 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.017+0000 7f02aa194640 1 --2- 172.21.15.22:0/544484710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c370a0 0x562d79c3ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.017+0000 7f02a9993640 1 --2- 172.21.15.22:0/544484710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2a5d0 0x562d79c386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.018+0000 7f02a9993640 1 -- 172.21.15.22:0/544484710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c370a0 msgr2=0x562d79c3ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.018+0000 7f02a9993640 1 --2- 172.21.15.22:0/544484710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c370a0 0x562d79c3ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.018+0000 7f02a9993640 1 -- 172.21.15.22:0/544484710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c2bb40 msgr2=0x562d79c38be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.018+0000 7f02a9993640 1 --2- 172.21.15.22:0/544484710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c2bb40 0x562d79c38be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.018+0000 7f02a9993640 1 -- 172.21.15.22:0/544484710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d79c21f90 con 0x562d79c2a5d0 2024-09-19T08:16:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.018+0000 7f02a9993640 1 --2- 172.21.15.22:0/544484710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2a5d0 0x562d79c386a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f02a00cd5e0 tx=0x7f02a000ff30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a68d78d289bb5c61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:03.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.018+0000 7f028f7fe640 1 -- 172.21.15.22:0/544484710 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a00b4070 con 0x562d79c2a5d0 2024-09-19T08:16:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.018+0000 7f028f7fe640 1 -- 172.21.15.22:0/544484710 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a00a4de0 con 0x562d79c2a5d0 2024-09-19T08:16:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.018+0000 7f028f7fe640 1 -- 172.21.15.22:0/544484710 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a00bc0b0 con 0x562d79c2a5d0 2024-09-19T08:16:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.019+0000 7f02adfd0740 1 -- 172.21.15.22:0/544484710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2a5d0 msgr2=0x562d79c386a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.019+0000 7f02adfd0740 1 --2- 172.21.15.22:0/544484710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2a5d0 0x562d79c386a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f02a00cd5e0 tx=0x7f02a000ff30 comp rx=0 tx=0).stop 2024-09-19T08:16:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.019+0000 7f02adfd0740 1 -- 172.21.15.22:0/544484710 shutdown_connections 2024-09-19T08:16:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.019+0000 7f02adfd0740 1 --2- 172.21.15.22:0/544484710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c370a0 0x562d79c3ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.019+0000 7f02adfd0740 1 --2- 172.21.15.22:0/544484710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c2bb40 0x562d79c38be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.019+0000 7f02adfd0740 1 --2- 172.21.15.22:0/544484710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c2a5d0 0x562d79c386a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.019+0000 7f02adfd0740 1 -- 172.21.15.22:0/544484710 >> 172.21.15.22:0/544484710 conn(0x562d79bf3310 msgr2=0x562d79c2cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.019+0000 7f02adfd0740 1 -- 172.21.15.22:0/544484710 shutdown_connections 2024-09-19T08:16:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.019+0000 7f02adfd0740 1 -- 172.21.15.22:0/544484710 wait complete. 2024-09-19T08:16:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.020+0000 7f02adfd0740 1 -- start start 2024-09-19T08:16:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.020+0000 7f02adfd0740 1 Processor -- start 2024-09-19T08:16:03.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.020+0000 7f02adfd0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2a5d0 0x562d79c2d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02adfd0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c2bb40 0x562d79c2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02a9993640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2a5d0 0x562d79c2d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02a9192640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c2bb40 0x562d79c2d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02a9993640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2a5d0 0x562d79c2d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:60278/0 (socket says 172.21.15.22:60278) 2024-09-19T08:16:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02a9993640 1 -- 172.21.15.22:0/2520643315 learned_addr learned my addr 172.21.15.22:0/2520643315 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02adfd0740 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c370a0 0x562d79c2dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02adfd0740 1 -- 172.21.15.22:0/2520643315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41a9be0 con 0x562d79c2bb40 2024-09-19T08:16:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02adfd0740 1 -- 172.21.15.22:0/2520643315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41ab050 con 0x562d79c2a5d0 2024-09-19T08:16:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02adfd0740 1 -- 172.21.15.22:0/2520643315 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41aa2b0 con 0x562d79c370a0 2024-09-19T08:16:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02adfd0740 10 client.0 _ll_register_callbacks cb 0x7f029c00bd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02aa194640 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c370a0 0x562d79c2dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02a9993640 1 -- 172.21.15.22:0/2520643315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c370a0 msgr2=0x562d79c2dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02a9993640 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c370a0 0x562d79c2dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02a9993640 1 -- 172.21.15.22:0/2520643315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c2bb40 msgr2=0x562d79c2d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02a9993640 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c2bb40 0x562d79c2d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02a9993640 1 -- 172.21.15.22:0/2520643315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d79c418f0 con 0x562d79c2a5d0 2024-09-19T08:16:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02aa194640 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c370a0 0x562d79c2dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:16:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.021+0000 7f02a9192640 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c2bb40 0x562d79c2d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.022+0000 7f02a9993640 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2a5d0 0x562d79c2d220 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f02a00cd720 tx=0x7f02a009c070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6ebee4bbb26e0db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.022+0000 7f028d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:16:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.022+0000 7f028d7fa640 1 -- 172.21.15.22:0/2520643315 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a00b4040 con 0x562d79c2a5d0 2024-09-19T08:16:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.022+0000 7f028d7fa640 1 -- 172.21.15.22:0/2520643315 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f02a000fb30 con 0x562d79c2a5d0 2024-09-19T08:16:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.022+0000 7f02adfd0740 1 -- 172.21.15.22:0/2520643315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02a00a4de0 con 0x562d79c2a5d0 2024-09-19T08:16:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.022+0000 7f028d7fa640 1 -- 172.21.15.22:0/2520643315 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f02a00bc0b0 con 0x562d79c2a5d0 2024-09-19T08:16:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.022+0000 7f02adfd0740 10 client.25301 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:03.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.022+0000 7f02adfd0740 1 -- 172.21.15.22:0/2520643315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d79cb91d0 con 0x562d79c2a5d0 2024-09-19T08:16:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.022+0000 7f028d7fa640 1 -- 172.21.15.22:0/2520643315 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a00bc250 con 0x562d79c2a5d0 2024-09-19T08:16:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.023+0000 7f028d7fa640 1 -- 172.21.15.22:0/2520643315 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f02a0096ce0 con 0x562d79c2a5d0 2024-09-19T08:16:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.023+0000 7f02adfd0740 10 client.25301 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.023+0000 7f02adfd0740 10 client.25301 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:16:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.023+0000 7f02adfd0740 20 client.25301 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.023+0000 7f02adfd0740 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x562d79cb98f0 0x562d79cbbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.024+0000 7f02adfd0740 4 client.25301 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:16:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.024+0000 7f02adfd0740 1 -- 172.21.15.22:0/2520643315 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f02a00a4de0 con 0x562d79cb98f0 2024-09-19T08:16:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.024+0000 7f02a9192640 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x562d79cb98f0 0x562d79cbbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.024+0000 7f02a9192640 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x562d79cb98f0 0x562d79cbbce0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a935374324abfd1f server_cookie=4863bce4f67f1a0c in_seq=0 out_seq=0 2024-09-19T08:16:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.024+0000 7f028d7fa640 10 client.25301 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:16:03.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f028d7fa640 1 -- 172.21.15.22:0/2520643315 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f02a00a4de0 con 0x562d79cb98f0 2024-09-19T08:16:03.785 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:16:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f028d7fa640 10 client.25301 handle_command_reply: tid=0 2024-09-19T08:16:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f02adfd0740 1 client.25301 shutdown 2024-09-19T08:16:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f02adfd0740 1 -- 172.21.15.22:0/2520643315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2a5d0 msgr2=0x562d79c2d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f02adfd0740 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2a5d0 0x562d79c2d220 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f02a00cd720 tx=0x7f02a009c070 comp rx=0 tx=0).stop 2024-09-19T08:16:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f02adfd0740 1 -- 172.21.15.22:0/2520643315 shutdown_connections 2024-09-19T08:16:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f02adfd0740 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x562d79cb98f0 0x562d79cbbce0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f02adfd0740 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d79c370a0 0x562d79c2dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f02adfd0740 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d79c2bb40 0x562d79c2d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f02adfd0740 1 --2- 172.21.15.22:0/2520643315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d79c2a5d0 0x562d79c2d220 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f02adfd0740 1 -- 172.21.15.22:0/2520643315 >> 172.21.15.22:0/2520643315 conn(0x562d79bf3310 msgr2=0x562d79c36380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f028d7fa640 0 client.25301 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:16:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f02adfd0740 1 -- 172.21.15.22:0/2520643315 shutdown_connections 2024-09-19T08:16:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f02adfd0740 1 -- 172.21.15.22:0/2520643315 wait complete. 2024-09-19T08:16:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.025+0000 7f02adfd0740 20 client.25301 trim_cache size 0 max 16384 2024-09-19T08:16:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.026+0000 7f02abbf7640 1 -- 172.21.15.22:0/3718706220 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f02900a35e0 msgr2=0x7f02900a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.026+0000 7f02abbf7640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f02900a35e0 0x7f02900a5aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f029c005cc0 tx=0x7f029c004ab0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.026+0000 7f02abbf7640 1 -- 172.21.15.22:0/3718706220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02a4102d90 msgr2=0x7f02a41984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.026+0000 7f02abbf7640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02a4102d90 0x7f02a41984d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f02a000c920 tx=0x7f02a000cde0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.026+0000 7f02abbf7640 1 -- 172.21.15.22:0/3718706220 shutdown_connections 2024-09-19T08:16:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.026+0000 7f02abbf7640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f02900a35e0 0x7f02900a5aa0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.026+0000 7f02abbf7640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a4108290 0x7f02a419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.026+0000 7f02abbf7640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02a4103740 0x7f02a4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.026+0000 7f02abbf7640 1 --2- 172.21.15.22:0/3718706220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02a4102d90 0x7f02a41984d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.026+0000 7f02abbf7640 1 -- 172.21.15.22:0/3718706220 >> 172.21.15.22:0/3718706220 conn(0x7f02a40fc7d0 msgr2=0x7f02a40fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.026+0000 7f02abbf7640 1 -- 172.21.15.22:0/3718706220 shutdown_connections 2024-09-19T08:16:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:03.026+0000 7f02abbf7640 1 -- 172.21.15.22:0/3718706220 wait complete. 2024-09-19T08:16:04.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:04 smithi064 ceph-mon[34900]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T08:16:05.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:04 smithi157 ceph-mon[35143]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T08:16:05.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:04 smithi022 ceph-mon[30963]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T08:16:06.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:06 smithi064 ceph-mon[34900]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T08:16:07.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:06 smithi157 ceph-mon[35143]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T08:16:07.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:06 smithi022 ceph-mon[30963]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T08:16:07.481 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:16:07.918 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:08.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.393+0000 7f3f15378640 1 -- 172.21.15.22:0/4185756866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10106a90 msgr2=0x7f3f10106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.393+0000 7f3f15378640 1 --2- 172.21.15.22:0/4185756866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10106a90 0x7f3f10106e70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009930 tx=0x7f3f0002a420 comp rx=0 tx=0).stop 2024-09-19T08:16:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.393+0000 7f3f15378640 1 -- 172.21.15.22:0/4185756866 shutdown_connections 2024-09-19T08:16:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.393+0000 7f3f15378640 1 --2- 172.21.15.22:0/4185756866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f10101430 0x7f3f1010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.394+0000 7f3f15378640 1 --2- 172.21.15.22:0/4185756866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f10100a90 0x7f3f10100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.394+0000 7f3f15378640 1 --2- 172.21.15.22:0/4185756866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10106a90 0x7f3f10106e70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.394+0000 7f3f15378640 1 -- 172.21.15.22:0/4185756866 >> 172.21.15.22:0/4185756866 conn(0x7f3f100fc7d0 msgr2=0x7f3f100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.394+0000 7f3f15378640 1 -- 172.21.15.22:0/4185756866 shutdown_connections 2024-09-19T08:16:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.394+0000 7f3f15378640 1 -- 172.21.15.22:0/4185756866 wait complete. 2024-09-19T08:16:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.394+0000 7f3f15378640 1 Processor -- start 2024-09-19T08:16:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.395+0000 7f3f15378640 1 -- start start 2024-09-19T08:16:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.395+0000 7f3f15378640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f10100a90 0x7f3f10199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.395+0000 7f3f15378640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10101430 0x7f3f10199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.395+0000 7f3f15378640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f10106a90 0x7f3f101a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.395+0000 7f3f15378640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10110e50 con 0x7f3f10101430 2024-09-19T08:16:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.395+0000 7f3f15378640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10110cd0 con 0x7f3f10106a90 2024-09-19T08:16:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.395+0000 7f3f15378640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10110fd0 con 0x7f3f10100a90 2024-09-19T08:16:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.395+0000 7f3f0e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10101430 0x7f3f10199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.395+0000 7f3f0e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10101430 0x7f3f10199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32982/0 (socket says 172.21.15.22:32982) 2024-09-19T08:16:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.395+0000 7f3f0e7fc640 1 -- 172.21.15.22:0/4265612978 learned_addr learned my addr 172.21.15.22:0/4265612978 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.395+0000 7f3f0effd640 1 --2- 172.21.15.22:0/4265612978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f10100a90 0x7f3f10199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.396+0000 7f3f0effd640 1 -- 172.21.15.22:0/4265612978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f10106a90 msgr2=0x7f3f101a43a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:16:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.396+0000 7f3f0effd640 1 --2- 172.21.15.22:0/4265612978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f10106a90 0x7f3f101a43a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.396+0000 7f3f0effd640 1 -- 172.21.15.22:0/4265612978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10101430 msgr2=0x7f3f10199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.396+0000 7f3f0effd640 1 --2- 172.21.15.22:0/4265612978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10101430 0x7f3f10199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.396+0000 7f3f0effd640 1 -- 172.21.15.22:0/4265612978 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f101a48e0 con 0x7f3f10100a90 2024-09-19T08:16:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.396+0000 7f3f0e7fc640 1 --2- 172.21.15.22:0/4265612978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10101430 0x7f3f10199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.396+0000 7f3f0effd640 1 --2- 172.21.15.22:0/4265612978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f10100a90 0x7f3f10199320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f00005940 tx=0x7f3f000040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.396+0000 7f3ef7fff640 1 -- 172.21.15.22:0/4265612978 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0003f070 con 0x7f3f10100a90 2024-09-19T08:16:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.397+0000 7f3ef7fff640 1 -- 172.21.15.22:0/4265612978 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f000044c0 con 0x7f3f10100a90 2024-09-19T08:16:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.397+0000 7f3ef7fff640 1 -- 172.21.15.22:0/4265612978 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00035070 con 0x7f3f10100a90 2024-09-19T08:16:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.397+0000 7f3f15378640 1 -- 172.21.15.22:0/4265612978 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f101a4ab0 con 0x7f3f10100a90 2024-09-19T08:16:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.397+0000 7f3f15378640 1 -- 172.21.15.22:0/4265612978 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f101a4f60 con 0x7f3f10100a90 2024-09-19T08:16:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.400+0000 7f3f17751740 1 Processor -- start 2024-09-19T08:16:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.400+0000 7f3f17751740 1 -- start start 2024-09-19T08:16:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.401+0000 7f3f17751740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa34b40 0x561f1aa33090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.401+0000 7f3f17751740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 0x561f1aa33a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.401+0000 7f3f17751740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa3e370 0x561f1aa40760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.401+0000 7f3f17751740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561f1a9992f0 con 0x561f1aa335d0 2024-09-19T08:16:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.401+0000 7f3f17751740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561f1a940cf0 con 0x561f1aa3e370 2024-09-19T08:16:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.401+0000 7f3f17751740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561f1a9952b0 con 0x561f1aa34b40 2024-09-19T08:16:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.401+0000 7f3ef7fff640 1 -- 172.21.15.22:0/4265612978 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f3f0003b460 con 0x7f3f10100a90 2024-09-19T08:16:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.401+0000 7f3f0e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa3e370 0x561f1aa40760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.401+0000 7f3f0e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa3e370 0x561f1aa40760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:50486/0 (socket says 172.21.15.22:50486) 2024-09-19T08:16:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.401+0000 7f3f0e7fc640 1 -- 172.21.15.22:0/1181831639 learned_addr learned my addr 172.21.15.22:0/1181831639 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.401+0000 7f3f0f7fe640 1 --2- 172.21.15.22:0/1181831639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa34b40 0x561f1aa33090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.402+0000 7f3f0e7fc640 1 -- 172.21.15.22:0/1181831639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa34b40 msgr2=0x561f1aa33090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.402+0000 7f3f0e7fc640 1 --2- 172.21.15.22:0/1181831639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa34b40 0x561f1aa33090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.402+0000 7f3f0e7fc640 1 -- 172.21.15.22:0/1181831639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 msgr2=0x561f1aa33a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.402+0000 7f3f0effd640 1 --2- 172.21.15.22:0/1181831639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 0x561f1aa33a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.402+0000 7f3f0e7fc640 1 --2- 172.21.15.22:0/1181831639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 0x561f1aa33a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.402+0000 7f3f0e7fc640 1 -- 172.21.15.22:0/1181831639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f1a9253a0 con 0x561f1aa3e370 2024-09-19T08:16:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.402+0000 7f3f0effd640 1 --2- 172.21.15.22:0/1181831639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 0x561f1aa33a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:16:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.403+0000 7f3f0e7fc640 1 --2- 172.21.15.22:0/1181831639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa3e370 0x561f1aa40760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3f04010500 tx=0x7f3f04010900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fdc467c6dbe77d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.403+0000 7f3ef7fff640 1 --2- 172.21.15.22:0/4265612978 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3ef80a35e0 0x7f3ef80a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.403+0000 7f3ef57fa640 1 -- 172.21.15.22:0/1181831639 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04011820 con 0x561f1aa3e370 2024-09-19T08:16:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.403+0000 7f3ef7fff640 1 -- 172.21.15.22:0/4265612978 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3f000f0450 con 0x7f3f10100a90 2024-09-19T08:16:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.403+0000 7f3ef57fa640 1 -- 172.21.15.22:0/1181831639 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f0401c480 con 0x561f1aa3e370 2024-09-19T08:16:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.403+0000 7f3ef57fa640 1 -- 172.21.15.22:0/1181831639 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0401a5e0 con 0x561f1aa3e370 2024-09-19T08:16:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.403+0000 7f3f0f7fe640 1 --2- 172.21.15.22:0/4265612978 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3ef80a35e0 0x7f3ef80a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.404+0000 7f3f17751740 1 -- 172.21.15.22:0/1181831639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa3e370 msgr2=0x561f1aa40760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.404+0000 7f3f0f7fe640 1 --2- 172.21.15.22:0/4265612978 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3ef80a35e0 0x7f3ef80a5aa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f08005fd0 tx=0x7f3f08012040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.404+0000 7f3f17751740 1 --2- 172.21.15.22:0/1181831639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa3e370 0x561f1aa40760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3f04010500 tx=0x7f3f04010900 comp rx=0 tx=0).stop 2024-09-19T08:16:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.404+0000 7f3f17751740 1 -- 172.21.15.22:0/1181831639 shutdown_connections 2024-09-19T08:16:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.404+0000 7f3f17751740 1 --2- 172.21.15.22:0/1181831639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa3e370 0x561f1aa40760 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.404+0000 7f3f17751740 1 --2- 172.21.15.22:0/1181831639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 0x561f1aa33a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.404+0000 7f3f17751740 1 --2- 172.21.15.22:0/1181831639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa34b40 0x561f1aa33090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.404+0000 7f3f17751740 1 -- 172.21.15.22:0/1181831639 >> 172.21.15.22:0/1181831639 conn(0x561f1a9fc310 msgr2=0x561f1aa2fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.404+0000 7f3f17751740 1 -- 172.21.15.22:0/1181831639 shutdown_connections 2024-09-19T08:16:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.405+0000 7f3f17751740 1 -- 172.21.15.22:0/1181831639 wait complete. 2024-09-19T08:16:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.405+0000 7f3f17751740 1 -- start start 2024-09-19T08:16:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.405+0000 7f3f17751740 1 Processor -- start 2024-09-19T08:16:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.406+0000 7f3f17751740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 0x561f1aa3dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.406+0000 7f3f17751740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa34b40 0x561f1aa3af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.406+0000 7f3f17751740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa3e370 0x561f1aa3b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.406+0000 7f3f0effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 0x561f1aa3dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.406+0000 7f3f0e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa34b40 0x561f1aa3af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.406+0000 7f3f0effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 0x561f1aa3dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33012/0 (socket says 172.21.15.22:33012) 2024-09-19T08:16:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.406+0000 7f3f0effd640 1 -- 172.21.15.22:0/3888719184 learned_addr learned my addr 172.21.15.22:0/3888719184 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.406+0000 7f3f17751740 1 -- 172.21.15.22:0/3888719184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561f1aa41070 con 0x561f1aa335d0 2024-09-19T08:16:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.406+0000 7f3f0f7fe640 1 --2- 172.21.15.22:0/3888719184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa3e370 0x561f1aa3b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.406+0000 7f3f17751740 1 -- 172.21.15.22:0/3888719184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561f1a954060 con 0x561f1aa34b40 2024-09-19T08:16:08.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.406+0000 7f3f17751740 1 -- 172.21.15.22:0/3888719184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561f1aa411f0 con 0x561f1aa3e370 2024-09-19T08:16:08.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.407+0000 7f3f17751740 10 client.0 _ll_register_callbacks cb 0x561f1a6a78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:08.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.407+0000 7f3f0e7fc640 1 -- 172.21.15.22:0/3888719184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa3e370 msgr2=0x561f1aa3b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:08.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.407+0000 7f3f0e7fc640 1 --2- 172.21.15.22:0/3888719184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa3e370 0x561f1aa3b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.407+0000 7f3f0e7fc640 1 -- 172.21.15.22:0/3888719184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 msgr2=0x561f1aa3dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:08.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.407+0000 7f3f0e7fc640 1 --2- 172.21.15.22:0/3888719184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 0x561f1aa3dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.407+0000 7f3f0e7fc640 1 -- 172.21.15.22:0/3888719184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f1a70a3c0 con 0x561f1aa34b40 2024-09-19T08:16:08.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.407+0000 7f3f0e7fc640 1 --2- 172.21.15.22:0/3888719184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa34b40 0x561f1aa3af70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f04010500 tx=0x7f3f0401c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d20567d4f9b56a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:08.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.407+0000 7f3ef2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:16:08.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.408+0000 7f3ef2ff5640 1 -- 172.21.15.22:0/3888719184 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0402b060 con 0x561f1aa34b40 2024-09-19T08:16:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.408+0000 7f3ef2ff5640 1 -- 172.21.15.22:0/3888719184 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3f04004250 con 0x561f1aa34b40 2024-09-19T08:16:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.408+0000 7f3f17751740 1 -- 172.21.15.22:0/3888719184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f0401c480 con 0x561f1aa34b40 2024-09-19T08:16:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.408+0000 7f3ef2ff5640 1 -- 172.21.15.22:0/3888719184 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3f0400bd20 con 0x561f1aa34b40 2024-09-19T08:16:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.408+0000 7f3f17751740 10 client.25310 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.408+0000 7f3f17751740 1 -- 172.21.15.22:0/3888719184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f1aa4c9d0 con 0x561f1aa34b40 2024-09-19T08:16:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.408+0000 7f3ef2ff5640 1 -- 172.21.15.22:0/3888719184 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f04004490 con 0x561f1aa34b40 2024-09-19T08:16:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.409+0000 7f3ef2ff5640 1 -- 172.21.15.22:0/3888719184 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3f040038f0 con 0x561f1aa34b40 2024-09-19T08:16:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.409+0000 7f3f17751740 10 client.25310 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.409+0000 7f3f17751740 10 client.25310 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:16:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.409+0000 7f3f17751740 20 client.25310 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.409+0000 7f3f17751740 1 --2- 172.21.15.22:0/3888719184 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x561f1aa4ddd0 0x561f1aa501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.409+0000 7f3f17751740 4 client.25310 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:16:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.409+0000 7f3f17751740 1 -- 172.21.15.22:0/3888719184 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3f0401c480 con 0x561f1aa4ddd0 2024-09-19T08:16:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.409+0000 7f3f0effd640 1 --2- 172.21.15.22:0/3888719184 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x561f1aa4ddd0 0x561f1aa501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.410+0000 7f3ef2ff5640 10 client.25310 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:16:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.410+0000 7f3f0effd640 1 --2- 172.21.15.22:0/3888719184 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x561f1aa4ddd0 0x561f1aa501c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e2cb2099806f57f server_cookie=2dcf45104e421a8f in_seq=0 out_seq=0 2024-09-19T08:16:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3ef2ff5640 1 -- 172.21.15.22:0/3888719184 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f3f0401c480 con 0x561f1aa4ddd0 2024-09-19T08:16:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3ef2ff5640 10 client.25310 handle_command_reply: tid=0 2024-09-19T08:16:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3f17751740 1 client.25310 shutdown 2024-09-19T08:16:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3f17751740 1 -- 172.21.15.22:0/3888719184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa34b40 msgr2=0x561f1aa3af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3f17751740 1 --2- 172.21.15.22:0/3888719184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa34b40 0x561f1aa3af70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f04010500 tx=0x7f3f0401c860 comp rx=0 tx=0).stop 2024-09-19T08:16:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3f17751740 1 -- 172.21.15.22:0/3888719184 shutdown_connections 2024-09-19T08:16:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3f17751740 1 --2- 172.21.15.22:0/3888719184 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x561f1aa4ddd0 0x561f1aa501c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3f17751740 1 --2- 172.21.15.22:0/3888719184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa3e370 0x561f1aa3b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3f17751740 1 --2- 172.21.15.22:0/3888719184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa34b40 0x561f1aa3af70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3f17751740 1 --2- 172.21.15.22:0/3888719184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 0x561f1aa3dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3f17751740 1 -- 172.21.15.22:0/3888719184 >> 172.21.15.22:0/3888719184 conn(0x561f1a9fc310 msgr2=0x561f1aa35960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:08.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3ef2ff5640 0 client.25310 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:16:08.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3f17751740 1 -- 172.21.15.22:0/3888719184 shutdown_connections 2024-09-19T08:16:08.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3f17751740 1 -- 172.21.15.22:0/3888719184 wait complete. 2024-09-19T08:16:08.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.411+0000 7f3f17751740 20 client.25310 trim_cache size 0 max 16384 2024-09-19T08:16:08.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.425+0000 7f3f17751740 1 Processor -- start 2024-09-19T08:16:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.425+0000 7f3f17751740 1 -- start start 2024-09-19T08:16:08.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.425+0000 7f3f17751740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 0x561f1aa4a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.425+0000 7f3f17751740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa34b40 0x561f1aa362f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.425+0000 7f3f17751740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa3e370 0x561f1aa55790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.425+0000 7f3f17751740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101adfe0 con 0x561f1aa335d0 2024-09-19T08:16:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.425+0000 7f3f17751740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101ab8b0 con 0x561f1aa3e370 2024-09-19T08:16:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.425+0000 7f3f17751740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101a9700 con 0x561f1aa34b40 2024-09-19T08:16:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.425+0000 7f3f0effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 0x561f1aa4a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.426+0000 7f3f0f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa3e370 0x561f1aa55790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.426+0000 7f3f0f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa3e370 0x561f1aa55790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:50494/0 (socket says 172.21.15.22:50494) 2024-09-19T08:16:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.426+0000 7f3f0f7fe640 1 -- 172.21.15.22:0/1852508613 learned_addr learned my addr 172.21.15.22:0/1852508613 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:08.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.426+0000 7f3f0e7fc640 1 --2- 172.21.15.22:0/1852508613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa34b40 0x561f1aa362f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.426+0000 7f3f0f7fe640 1 -- 172.21.15.22:0/1852508613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa34b40 msgr2=0x561f1aa362f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.426+0000 7f3f0f7fe640 1 --2- 172.21.15.22:0/1852508613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa34b40 0x561f1aa362f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.426+0000 7f3f0f7fe640 1 -- 172.21.15.22:0/1852508613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 msgr2=0x561f1aa4a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.426+0000 7f3f0f7fe640 1 --2- 172.21.15.22:0/1852508613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 0x561f1aa4a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.426+0000 7f3f0f7fe640 1 -- 172.21.15.22:0/1852508613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f1aa30cf0 con 0x561f1aa3e370 2024-09-19T08:16:08.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.426+0000 7f3f0e7fc640 1 --2- 172.21.15.22:0/1852508613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa34b40 0x561f1aa362f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.426+0000 7f3f0f7fe640 1 --2- 172.21.15.22:0/1852508613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa3e370 0x561f1aa55790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3f08003e20 tx=0x7f3f08018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf969bfb935e2e9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.427+0000 7f3ef4ff9640 1 -- 172.21.15.22:0/1852508613 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08022040 con 0x561f1aa3e370 2024-09-19T08:16:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.427+0000 7f3ef4ff9640 1 -- 172.21.15.22:0/1852508613 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f0802c070 con 0x561f1aa3e370 2024-09-19T08:16:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.427+0000 7f3ef4ff9640 1 -- 172.21.15.22:0/1852508613 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800b9e0 con 0x561f1aa3e370 2024-09-19T08:16:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.428+0000 7f3f17751740 1 -- 172.21.15.22:0/1852508613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa3e370 msgr2=0x561f1aa55790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.428+0000 7f3f17751740 1 --2- 172.21.15.22:0/1852508613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa3e370 0x561f1aa55790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3f08003e20 tx=0x7f3f08018000 comp rx=0 tx=0).stop 2024-09-19T08:16:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.428+0000 7f3f17751740 1 -- 172.21.15.22:0/1852508613 shutdown_connections 2024-09-19T08:16:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.428+0000 7f3f17751740 1 --2- 172.21.15.22:0/1852508613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa3e370 0x561f1aa55790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.428+0000 7f3f17751740 1 --2- 172.21.15.22:0/1852508613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa34b40 0x561f1aa362f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.428+0000 7f3f17751740 1 --2- 172.21.15.22:0/1852508613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa335d0 0x561f1aa4a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.428+0000 7f3f17751740 1 -- 172.21.15.22:0/1852508613 >> 172.21.15.22:0/1852508613 conn(0x561f1a9fc310 msgr2=0x561f1aa3a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:08.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.428+0000 7f3f17751740 1 -- 172.21.15.22:0/1852508613 shutdown_connections 2024-09-19T08:16:08.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.428+0000 7f3f17751740 1 -- 172.21.15.22:0/1852508613 wait complete. 2024-09-19T08:16:08.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.428+0000 7f3f17751740 1 -- start start 2024-09-19T08:16:08.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.429+0000 7f3f17751740 1 Processor -- start 2024-09-19T08:16:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.429+0000 7f3f17751740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa335d0 0x561f1aa46810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.429+0000 7f3f17751740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa34b40 0x561f1aa46d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.429+0000 7f3f17751740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa3e370 0x561f1aa42da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.429+0000 7f3f0e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa34b40 0x561f1aa46d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.429+0000 7f3f17751740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101a9d70 con 0x561f1aa34b40 2024-09-19T08:16:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.429+0000 7f3f0e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa34b40 0x561f1aa46d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33034/0 (socket says 172.21.15.22:33034) 2024-09-19T08:16:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.429+0000 7f3f0e7fc640 1 -- 172.21.15.22:0/1897792450 learned_addr learned my addr 172.21.15.22:0/1897792450 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.429+0000 7f3f0effd640 1 --2- 172.21.15.22:0/1897792450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa335d0 0x561f1aa46810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.429+0000 7f3f17751740 1 -- 172.21.15.22:0/1897792450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101ab1e0 con 0x561f1aa335d0 2024-09-19T08:16:08.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.429+0000 7f3f0f7fe640 1 --2- 172.21.15.22:0/1897792450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa3e370 0x561f1aa42da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.430+0000 7f3f17751740 1 -- 172.21.15.22:0/1897792450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101aa440 con 0x561f1aa3e370 2024-09-19T08:16:08.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.430+0000 7f3f17751740 10 client.0 _ll_register_callbacks cb 0x7f3f040103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:08.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.430+0000 7f3f0e7fc640 1 -- 172.21.15.22:0/1897792450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa3e370 msgr2=0x561f1aa42da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:08.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.430+0000 7f3f0e7fc640 1 --2- 172.21.15.22:0/1897792450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa3e370 0x561f1aa42da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.430+0000 7f3f0e7fc640 1 -- 172.21.15.22:0/1897792450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa335d0 msgr2=0x561f1aa46810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:08.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.430+0000 7f3f0e7fc640 1 --2- 172.21.15.22:0/1897792450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa335d0 0x561f1aa46810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.430+0000 7f3f0e7fc640 1 -- 172.21.15.22:0/1897792450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f1aa2f4f0 con 0x561f1aa34b40 2024-09-19T08:16:08.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.430+0000 7f3f0f7fe640 1 --2- 172.21.15.22:0/1897792450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa3e370 0x561f1aa42da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:08.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.430+0000 7f3f0e7fc640 1 --2- 172.21.15.22:0/1897792450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa34b40 0x561f1aa46d50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400bee0 tx=0x7f3f0400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0263e7add09705f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:08.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.430+0000 7f3ef2ff5640 10 client.15645 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:08.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.431+0000 7f3ef2ff5640 1 -- 172.21.15.22:0/1897792450 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0402b040 con 0x561f1aa34b40 2024-09-19T08:16:08.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.430+0000 7f3f17751740 1 -- 172.21.15.22:0/1897792450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f0802c070 con 0x561f1aa34b40 2024-09-19T08:16:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.431+0000 7f3ef2ff5640 1 -- 172.21.15.22:0/1897792450 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3f040035f0 con 0x561f1aa34b40 2024-09-19T08:16:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.431+0000 7f3ef2ff5640 1 -- 172.21.15.22:0/1897792450 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3f0403e050 con 0x561f1aa34b40 2024-09-19T08:16:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.431+0000 7f3f17751740 10 client.15645 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.431+0000 7f3f17751740 1 -- 172.21.15.22:0/1897792450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=25}) -- 0x561f1aa30960 con 0x561f1aa34b40 2024-09-19T08:16:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.432+0000 7f3ef2ff5640 1 -- 172.21.15.22:0/1897792450 <== mon.0 v2:172.21.15.22:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3f040037f0 con 0x561f1aa34b40 2024-09-19T08:16:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.432+0000 7f3f17751740 10 client.15645 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.432+0000 7f3f17751740 10 client.15645 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:16:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.432+0000 7f3f17751740 20 client.15645 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.432+0000 7f3f17751740 1 --2- 172.21.15.22:0/1897792450 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x561f1aab94f0 0x561f1aabb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.432+0000 7f3f17751740 4 client.15645 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:16:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.432+0000 7f3f17751740 1 -- 172.21.15.22:0/1897792450 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3f0802c070 con 0x561f1aab94f0 2024-09-19T08:16:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.432+0000 7f3f0effd640 1 --2- 172.21.15.22:0/1897792450 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x561f1aab94f0 0x561f1aabb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3ef2ff5640 10 client.15645 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:16:08.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.432+0000 7f3f0effd640 1 --2- 172.21.15.22:0/1897792450 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x561f1aab94f0 0x561f1aabb8e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e41fa8e024952ce2 server_cookie=e938fdcc945cc3fa in_seq=0 out_seq=0 2024-09-19T08:16:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3ef2ff5640 1 -- 172.21.15.22:0/1897792450 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3f0802c070 con 0x561f1aab94f0 2024-09-19T08:16:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3ef2ff5640 10 client.15645 handle_command_reply: tid=0 2024-09-19T08:16:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3f17751740 1 client.15645 shutdown 2024-09-19T08:16:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3f17751740 1 -- 172.21.15.22:0/1897792450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa34b40 msgr2=0x561f1aa46d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3f17751740 1 --2- 172.21.15.22:0/1897792450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa34b40 0x561f1aa46d50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400bee0 tx=0x7f3f0400b260 comp rx=0 tx=0).stop 2024-09-19T08:16:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3f17751740 1 -- 172.21.15.22:0/1897792450 shutdown_connections 2024-09-19T08:16:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3f17751740 1 --2- 172.21.15.22:0/1897792450 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x561f1aab94f0 0x561f1aabb8e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3ef2ff5640 0 client.15645 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:16:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3f17751740 1 --2- 172.21.15.22:0/1897792450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f1aa3e370 0x561f1aa42da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3f17751740 1 --2- 172.21.15.22:0/1897792450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f1aa34b40 0x561f1aa46d50 secure :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400bee0 tx=0x7f3f0400b260 comp rx=0 tx=0).stop 2024-09-19T08:16:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3f17751740 1 --2- 172.21.15.22:0/1897792450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f1aa335d0 0x561f1aa46810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3f17751740 1 -- 172.21.15.22:0/1897792450 >> 172.21.15.22:0/1897792450 conn(0x561f1a9fc310 msgr2=0x561f1aa3a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:08.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3f17751740 1 -- 172.21.15.22:0/1897792450 shutdown_connections 2024-09-19T08:16:08.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3f17751740 1 -- 172.21.15.22:0/1897792450 wait complete. 2024-09-19T08:16:08.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.433+0000 7f3f17751740 20 client.15645 trim_cache size 0 max 16384 2024-09-19T08:16:08.439 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:16:08.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.434+0000 7f3f15378640 1 -- 172.21.15.22:0/4265612978 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3ef80a35e0 msgr2=0x7f3ef80a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:08.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.434+0000 7f3f15378640 1 --2- 172.21.15.22:0/4265612978 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3ef80a35e0 0x7f3ef80a5aa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f08005fd0 tx=0x7f3f08012040 comp rx=0 tx=0).stop 2024-09-19T08:16:08.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.434+0000 7f3f15378640 1 -- 172.21.15.22:0/4265612978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f10100a90 msgr2=0x7f3f10199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:08.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.434+0000 7f3f15378640 1 --2- 172.21.15.22:0/4265612978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f10100a90 0x7f3f10199320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f00005940 tx=0x7f3f000040c0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.434+0000 7f3f15378640 1 -- 172.21.15.22:0/4265612978 shutdown_connections 2024-09-19T08:16:08.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.434+0000 7f3f15378640 1 --2- 172.21.15.22:0/4265612978 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3ef80a35e0 0x7f3ef80a5aa0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.434+0000 7f3f15378640 1 --2- 172.21.15.22:0/4265612978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f10106a90 0x7f3f101a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.434+0000 7f3f15378640 1 --2- 172.21.15.22:0/4265612978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f10101430 0x7f3f10199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.434+0000 7f3f15378640 1 --2- 172.21.15.22:0/4265612978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f10100a90 0x7f3f10199320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:08.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.434+0000 7f3f15378640 1 -- 172.21.15.22:0/4265612978 >> 172.21.15.22:0/4265612978 conn(0x7f3f100fc7d0 msgr2=0x7f3f100fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:08.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.435+0000 7f3f15378640 1 -- 172.21.15.22:0/4265612978 shutdown_connections 2024-09-19T08:16:08.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:08.435+0000 7f3f15378640 1 -- 172.21.15.22:0/4265612978 wait complete. 2024-09-19T08:16:08.836 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:08 smithi022 ceph-mon[30963]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T08:16:08.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:08 smithi064 ceph-mon[34900]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T08:16:09.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:08 smithi157 ceph-mon[35143]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T08:16:09.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:16:09.681 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:10.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.215+0000 7f6734b93640 1 -- 172.21.15.22:0/1111319142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67301082e0 msgr2=0x7f673010ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:10.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.215+0000 7f6734b93640 1 --2- 172.21.15.22:0/1111319142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67301082e0 0x7f673010ec50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6728009cd0 tx=0x7f672802a8e0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.215+0000 7f6734b93640 1 -- 172.21.15.22:0/1111319142 shutdown_connections 2024-09-19T08:16:10.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.215+0000 7f6734b93640 1 --2- 172.21.15.22:0/1111319142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67301082e0 0x7f673010ec50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.215+0000 7f6734b93640 1 --2- 172.21.15.22:0/1111319142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6730103760 0x7f6730107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.215+0000 7f6734b93640 1 --2- 172.21.15.22:0/1111319142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6730102db0 0x7f6730103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.215+0000 7f6734b93640 1 -- 172.21.15.22:0/1111319142 >> 172.21.15.22:0/1111319142 conn(0x7f67300fc7d0 msgr2=0x7f67300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:10.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.215+0000 7f6734b93640 1 -- 172.21.15.22:0/1111319142 shutdown_connections 2024-09-19T08:16:10.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.215+0000 7f6734b93640 1 -- 172.21.15.22:0/1111319142 wait complete. 2024-09-19T08:16:10.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.216+0000 7f6734b93640 1 Processor -- start 2024-09-19T08:16:10.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.217+0000 7f6734b93640 1 -- start start 2024-09-19T08:16:10.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.217+0000 7f6734b93640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6730102db0 0x7f67301985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.217+0000 7f6734b93640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6730103760 0x7f6730198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.217+0000 7f6734b93640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67301082e0 0x7f673019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.217+0000 7f6734b93640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67301120b0 con 0x7f67301082e0 2024-09-19T08:16:10.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.217+0000 7f6734b93640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6730111f30 con 0x7f6730102db0 2024-09-19T08:16:10.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.217+0000 7f6734b93640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6730112230 con 0x7f6730103760 2024-09-19T08:16:10.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.217+0000 7f672dd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6730103760 0x7f6730198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.217+0000 7f672dd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6730103760 0x7f6730198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:54738/0 (socket says 172.21.15.22:54738) 2024-09-19T08:16:10.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.217+0000 7f672dd9b640 1 -- 172.21.15.22:0/135679370 learned_addr learned my addr 172.21.15.22:0/135679370 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:10.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.218+0000 7f672ed9d640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67301082e0 0x7f673019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.218+0000 7f672dd9b640 1 -- 172.21.15.22:0/135679370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6730102db0 msgr2=0x7f67301985b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:16:10.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.218+0000 7f672e59c640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6730102db0 0x7f67301985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.218+0000 7f672dd9b640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6730102db0 0x7f67301985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.218+0000 7f672dd9b640 1 -- 172.21.15.22:0/135679370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67301082e0 msgr2=0x7f673019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:10.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.218+0000 7f672dd9b640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67301082e0 0x7f673019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.218+0000 7f672dd9b640 1 -- 172.21.15.22:0/135679370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673019d560 con 0x7f6730103760 2024-09-19T08:16:10.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.218+0000 7f672ed9d640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67301082e0 0x7f673019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:10.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.218+0000 7f672e59c640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6730102db0 0x7f67301985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:16:10.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.218+0000 7f672dd9b640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6730103760 0x7f6730198af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f672400d950 tx=0x7f672400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:10.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.219+0000 7f671f7fe640 1 -- 172.21.15.22:0/135679370 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724017070 con 0x7f6730103760 2024-09-19T08:16:10.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.219+0000 7f671f7fe640 1 -- 172.21.15.22:0/135679370 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6724009c30 con 0x7f6730103760 2024-09-19T08:16:10.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.219+0000 7f6734b93640 1 -- 172.21.15.22:0/135679370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673019d850 con 0x7f6730103760 2024-09-19T08:16:10.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.219+0000 7f671f7fe640 1 -- 172.21.15.22:0/135679370 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67240047a0 con 0x7f6730103760 2024-09-19T08:16:10.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.219+0000 7f6734b93640 1 -- 172.21.15.22:0/135679370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67301a5130 con 0x7f6730103760 2024-09-19T08:16:10.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.222+0000 7f6736f6c740 1 Processor -- start 2024-09-19T08:16:10.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.222+0000 7f6736f6c740 1 -- start start 2024-09-19T08:16:10.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.222+0000 7f6736f6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb10cb40 0x5561fb10b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.222+0000 7f6736f6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10b5d0 0x5561fb10ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.222+0000 7f6736f6c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb11a780 0x5561fb11cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.222+0000 7f6736f6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5561fb0712f0 con 0x5561fb10cb40 2024-09-19T08:16:10.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.222+0000 7f6736f6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5561fb018cf0 con 0x5561fb10b5d0 2024-09-19T08:16:10.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.222+0000 7f6736f6c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5561fb06d2b0 con 0x5561fb11a780 2024-09-19T08:16:10.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.223+0000 7f672ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb10cb40 0x5561fb10b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.223+0000 7f672ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb10cb40 0x5561fb10b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33060/0 (socket says 172.21.15.22:33060) 2024-09-19T08:16:10.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.223+0000 7f672ed9d640 1 -- 172.21.15.22:0/4294258003 learned_addr learned my addr 172.21.15.22:0/4294258003 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:10.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.223+0000 7f672e59c640 1 --2- 172.21.15.22:0/4294258003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10b5d0 0x5561fb10ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.223+0000 7f671f7fe640 1 -- 172.21.15.22:0/135679370 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f6724004020 con 0x7f6730103760 2024-09-19T08:16:10.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.223+0000 7f672ed9d640 1 -- 172.21.15.22:0/4294258003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb11a780 msgr2=0x5561fb11cb70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:16:10.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.223+0000 7f672ed9d640 1 --2- 172.21.15.22:0/4294258003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb11a780 0x5561fb11cb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.223+0000 7f672ed9d640 1 -- 172.21.15.22:0/4294258003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10b5d0 msgr2=0x5561fb10ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:10.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.223+0000 7f672ed9d640 1 --2- 172.21.15.22:0/4294258003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10b5d0 0x5561fb10ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.223+0000 7f672ed9d640 1 -- 172.21.15.22:0/4294258003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561faffd3a0 con 0x5561fb10cb40 2024-09-19T08:16:10.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.223+0000 7f672ed9d640 1 --2- 172.21.15.22:0/4294258003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb10cb40 0x5561fb10b090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6728009cd0 tx=0x7f6728031910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f0da14af3b51bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:10.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.224+0000 7f671cff9640 1 -- 172.21.15.22:0/4294258003 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728048070 con 0x5561fb10cb40 2024-09-19T08:16:10.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.224+0000 7f671cff9640 1 -- 172.21.15.22:0/4294258003 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6728036070 con 0x5561fb10cb40 2024-09-19T08:16:10.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.224+0000 7f671cff9640 1 -- 172.21.15.22:0/4294258003 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728004840 con 0x5561fb10cb40 2024-09-19T08:16:10.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.224+0000 7f6736f6c740 1 -- 172.21.15.22:0/4294258003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb10cb40 msgr2=0x5561fb10b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:10.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.224+0000 7f6736f6c740 1 --2- 172.21.15.22:0/4294258003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb10cb40 0x5561fb10b090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6728009cd0 tx=0x7f6728031910 comp rx=0 tx=0).stop 2024-09-19T08:16:10.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.225+0000 7f6736f6c740 1 -- 172.21.15.22:0/4294258003 shutdown_connections 2024-09-19T08:16:10.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.225+0000 7f6736f6c740 1 --2- 172.21.15.22:0/4294258003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb11a780 0x5561fb11cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.225+0000 7f6736f6c740 1 --2- 172.21.15.22:0/4294258003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10b5d0 0x5561fb10ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.225+0000 7f6736f6c740 1 --2- 172.21.15.22:0/4294258003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb10cb40 0x5561fb10b090 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.225+0000 7f6736f6c740 1 -- 172.21.15.22:0/4294258003 >> 172.21.15.22:0/4294258003 conn(0x5561fb0d4310 msgr2=0x5561fb107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:10.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.225+0000 7f6736f6c740 1 -- 172.21.15.22:0/4294258003 shutdown_connections 2024-09-19T08:16:10.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.225+0000 7f6736f6c740 1 -- 172.21.15.22:0/4294258003 wait complete. 2024-09-19T08:16:10.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.225+0000 7f6736f6c740 1 -- start start 2024-09-19T08:16:10.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.225+0000 7f6736f6c740 1 Processor -- start 2024-09-19T08:16:10.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f671f7fe640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f67180a34f0 0x7f67180a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f6736f6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb10b5d0 0x5561fb1175c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f672e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb10b5d0 0x5561fb1175c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f672e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb10b5d0 0x5561fb1175c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33062/0 (socket says 172.21.15.22:33062) 2024-09-19T08:16:10.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f672e59c640 1 -- 172.21.15.22:0/1885815133 learned_addr learned my addr 172.21.15.22:0/1885815133 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:10.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f6736f6c740 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10cb40 0x5561fb117b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f6736f6c740 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb11a780 0x5561fb118040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f6736f6c740 1 -- 172.21.15.22:0/1885815133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5561fb11d420 con 0x5561fb10b5d0 2024-09-19T08:16:10.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f6736f6c740 1 -- 172.21.15.22:0/1885815133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5561fb02c060 con 0x5561fb10cb40 2024-09-19T08:16:10.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f6736f6c740 1 -- 172.21.15.22:0/1885815133 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5561fb11d5a0 con 0x5561fb11a780 2024-09-19T08:16:10.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f6736f6c740 10 client.0 _ll_register_callbacks cb 0x5561fad7f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:10.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f672dd9b640 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10cb40 0x5561fb117b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f672e59c640 1 -- 172.21.15.22:0/1885815133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb11a780 msgr2=0x5561fb118040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:16:10.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f672e59c640 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb11a780 0x5561fb118040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f672e59c640 1 -- 172.21.15.22:0/1885815133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10cb40 msgr2=0x5561fb117b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:10.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f672e59c640 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10cb40 0x5561fb117b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f672e59c640 1 -- 172.21.15.22:0/1885815133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561fade23c0 con 0x5561fb10b5d0 2024-09-19T08:16:10.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.226+0000 7f672dd9b640 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10cb40 0x5561fb117b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:10.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.227+0000 7f671f7fe640 1 -- 172.21.15.22:0/135679370 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f67240cfc20 con 0x7f6730103760 2024-09-19T08:16:10.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.227+0000 7f672e59c640 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb10b5d0 0x5561fb1175c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6720010670 tx=0x7f6720010a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9662528bcaaf16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:10.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.227+0000 7f6715ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:10.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.227+0000 7f6715ffb640 1 -- 172.21.15.22:0/1885815133 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672000e9a0 con 0x5561fb10b5d0 2024-09-19T08:16:10.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.227+0000 7f6715ffb640 1 -- 172.21.15.22:0/1885815133 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f672000eb40 con 0x5561fb10b5d0 2024-09-19T08:16:10.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.227+0000 7f6736f6c740 1 -- 172.21.15.22:0/1885815133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6728036070 con 0x5561fb10b5d0 2024-09-19T08:16:10.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.227+0000 7f672ed9d640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f67180a34f0 0x7f67180a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.227+0000 7f6715ffb640 1 -- 172.21.15.22:0/1885815133 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f672000ed40 con 0x5561fb10b5d0 2024-09-19T08:16:10.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.227+0000 7f6736f6c740 10 client.15657 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:10.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.227+0000 7f6736f6c740 1 -- 172.21.15.22:0/1885815133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5561fb1290a0 con 0x5561fb10b5d0 2024-09-19T08:16:10.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.227+0000 7f6715ffb640 1 -- 172.21.15.22:0/1885815133 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6720012930 con 0x5561fb10b5d0 2024-09-19T08:16:10.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.228+0000 7f672ed9d640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f67180a34f0 0x7f67180a59b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f67280042e0 tx=0x7f672802a8e0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:10.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.228+0000 7f6715ffb640 1 -- 172.21.15.22:0/1885815133 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f672000f040 con 0x5561fb10b5d0 2024-09-19T08:16:10.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.228+0000 7f6736f6c740 10 client.15657 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.228+0000 7f6736f6c740 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:16:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.228+0000 7f6736f6c740 20 client.15657 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.228+0000 7f6736f6c740 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5561fb12a560 0x5561fb12c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.228+0000 7f6736f6c740 4 client.15657 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:16:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.228+0000 7f6736f6c740 1 -- 172.21.15.22:0/1885815133 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6728036070 con 0x5561fb12a560 2024-09-19T08:16:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.228+0000 7f672dd9b640 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5561fb12a560 0x5561fb12c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.229+0000 7f6715ffb640 10 client.15657 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.229+0000 7f672dd9b640 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5561fb12a560 0x5561fb12c950 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a2fdecff0e5663 server_cookie=da18c50938004173 in_seq=0 out_seq=0 2024-09-19T08:16:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6715ffb640 1 -- 172.21.15.22:0/1885815133 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6728036070 con 0x5561fb12a560 2024-09-19T08:16:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6715ffb640 10 client.15657 handle_command_reply: tid=0 2024-09-19T08:16:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6736f6c740 1 client.15657 shutdown 2024-09-19T08:16:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6736f6c740 1 -- 172.21.15.22:0/1885815133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb10b5d0 msgr2=0x5561fb1175c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6736f6c740 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb10b5d0 0x5561fb1175c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6720010670 tx=0x7f6720010a70 comp rx=0 tx=0).stop 2024-09-19T08:16:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6736f6c740 1 -- 172.21.15.22:0/1885815133 shutdown_connections 2024-09-19T08:16:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6736f6c740 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5561fb12a560 0x5561fb12c950 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6715ffb640 0 client.15657 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6736f6c740 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb11a780 0x5561fb118040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6736f6c740 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10cb40 0x5561fb117b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6736f6c740 1 --2- 172.21.15.22:0/1885815133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb10b5d0 0x5561fb1175c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6736f6c740 1 -- 172.21.15.22:0/1885815133 >> 172.21.15.22:0/1885815133 conn(0x5561fb0d4310 msgr2=0x5561fb10d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6736f6c740 1 -- 172.21.15.22:0/1885815133 shutdown_connections 2024-09-19T08:16:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6736f6c740 1 -- 172.21.15.22:0/1885815133 wait complete. 2024-09-19T08:16:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.230+0000 7f6736f6c740 20 client.15657 trim_cache size 0 max 16384 2024-09-19T08:16:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.256+0000 7f6736f6c740 1 Processor -- start 2024-09-19T08:16:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.256+0000 7f6736f6c740 1 -- start start 2024-09-19T08:16:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f6736f6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10cb40 0x5561fb11bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f6736f6c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb11a780 0x5561fb11c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f6736f6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb12a560 0x5561fb1064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f6736f6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67301ae000 con 0x5561fb12a560 2024-09-19T08:16:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f6736f6c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f67301ab8d0 con 0x5561fb10cb40 2024-09-19T08:16:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f6736f6c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f67301a9720 con 0x5561fb11a780 2024-09-19T08:16:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f672ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb12a560 0x5561fb1064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f672ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb12a560 0x5561fb1064e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33076/0 (socket says 172.21.15.22:33076) 2024-09-19T08:16:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f672ed9d640 1 -- 172.21.15.22:0/3419352165 learned_addr learned my addr 172.21.15.22:0/3419352165 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f672dd9b640 1 --2- 172.21.15.22:0/3419352165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb11a780 0x5561fb11c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f672e59c640 1 --2- 172.21.15.22:0/3419352165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10cb40 0x5561fb11bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f672ed9d640 1 -- 172.21.15.22:0/3419352165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb11a780 msgr2=0x5561fb11c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f672ed9d640 1 --2- 172.21.15.22:0/3419352165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb11a780 0x5561fb11c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f672ed9d640 1 -- 172.21.15.22:0/3419352165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10cb40 msgr2=0x5561fb11bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f672ed9d640 1 --2- 172.21.15.22:0/3419352165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10cb40 0x5561fb11bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.257+0000 7f672ed9d640 1 -- 172.21.15.22:0/3419352165 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561fb11d470 con 0x5561fb12a560 2024-09-19T08:16:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.258+0000 7f672ed9d640 1 --2- 172.21.15.22:0/3419352165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb12a560 0x5561fb1064e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6728046040 tx=0x7f6728009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77a2cd22e43c7ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.258+0000 7f6717fff640 1 -- 172.21.15.22:0/3419352165 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728048040 con 0x5561fb12a560 2024-09-19T08:16:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.258+0000 7f6717fff640 1 -- 172.21.15.22:0/3419352165 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6728036040 con 0x5561fb12a560 2024-09-19T08:16:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.258+0000 7f6717fff640 1 -- 172.21.15.22:0/3419352165 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672804e6b0 con 0x5561fb12a560 2024-09-19T08:16:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.259+0000 7f6736f6c740 1 -- 172.21.15.22:0/3419352165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb12a560 msgr2=0x5561fb1064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.259+0000 7f6736f6c740 1 --2- 172.21.15.22:0/3419352165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb12a560 0x5561fb1064e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6728046040 tx=0x7f6728009a20 comp rx=0 tx=0).stop 2024-09-19T08:16:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.259+0000 7f6736f6c740 1 -- 172.21.15.22:0/3419352165 shutdown_connections 2024-09-19T08:16:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.259+0000 7f6736f6c740 1 --2- 172.21.15.22:0/3419352165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb12a560 0x5561fb1064e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.259+0000 7f6736f6c740 1 --2- 172.21.15.22:0/3419352165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb11a780 0x5561fb11c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.260+0000 7f6736f6c740 1 --2- 172.21.15.22:0/3419352165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb10cb40 0x5561fb11bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.260+0000 7f6736f6c740 1 -- 172.21.15.22:0/3419352165 >> 172.21.15.22:0/3419352165 conn(0x5561fb0d4310 msgr2=0x5561fb1166b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.260+0000 7f6736f6c740 1 -- 172.21.15.22:0/3419352165 shutdown_connections 2024-09-19T08:16:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.260+0000 7f6736f6c740 1 -- 172.21.15.22:0/3419352165 wait complete. 2024-09-19T08:16:10.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.260+0000 7f6736f6c740 1 -- start start 2024-09-19T08:16:10.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.261+0000 7f6736f6c740 1 Processor -- start 2024-09-19T08:16:10.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.261+0000 7f6736f6c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb10cb40 0x5561fb122830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.261+0000 7f6736f6c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb11a780 0x5561fb122d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.261+0000 7f6736f6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb12a560 0x5561fb113830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.262+0000 7f672e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb10cb40 0x5561fb122830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.262+0000 7f672e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb10cb40 0x5561fb122830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:54760/0 (socket says 172.21.15.22:54760) 2024-09-19T08:16:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.262+0000 7f672dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb11a780 0x5561fb122d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.262+0000 7f672dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb11a780 0x5561fb122d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:50558/0 (socket says 172.21.15.22:50558) 2024-09-19T08:16:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.262+0000 7f672e59c640 1 -- 172.21.15.22:0/1773671622 learned_addr learned my addr 172.21.15.22:0/1773671622 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.262+0000 7f672ed9d640 1 --2- 172.21.15.22:0/1773671622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb12a560 0x5561fb113830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.262+0000 7f6736f6c740 1 -- 172.21.15.22:0/1773671622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67301a9d90 con 0x5561fb12a560 2024-09-19T08:16:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.262+0000 7f6736f6c740 1 -- 172.21.15.22:0/1773671622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f67301ab200 con 0x5561fb11a780 2024-09-19T08:16:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.262+0000 7f6736f6c740 1 -- 172.21.15.22:0/1773671622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f67301aa460 con 0x5561fb10cb40 2024-09-19T08:16:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.263+0000 7f6736f6c740 10 client.0 _ll_register_callbacks cb 0x7f67200094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.263+0000 7f672ed9d640 1 -- 172.21.15.22:0/1773671622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb10cb40 msgr2=0x5561fb122830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.263+0000 7f672ed9d640 1 --2- 172.21.15.22:0/1773671622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb10cb40 0x5561fb122830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.263+0000 7f672ed9d640 1 -- 172.21.15.22:0/1773671622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb11a780 msgr2=0x5561fb122d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.263+0000 7f672ed9d640 1 --2- 172.21.15.22:0/1773671622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb11a780 0x5561fb122d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.263+0000 7f672ed9d640 1 -- 172.21.15.22:0/1773671622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561fb113d70 con 0x5561fb12a560 2024-09-19T08:16:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.263+0000 7f672ed9d640 1 --2- 172.21.15.22:0/1773671622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb12a560 0x5561fb113830 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6728009bf0 tx=0x7f6728031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd3e2e2985a2d3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.263+0000 7f6715ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.263+0000 7f6736f6c740 1 -- 172.21.15.22:0/1773671622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6728036040 con 0x5561fb12a560 2024-09-19T08:16:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.264+0000 7f6715ffb640 1 -- 172.21.15.22:0/1773671622 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728066060 con 0x5561fb12a560 2024-09-19T08:16:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.264+0000 7f6715ffb640 1 -- 172.21.15.22:0/1773671622 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6728004720 con 0x5561fb12a560 2024-09-19T08:16:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.264+0000 7f6715ffb640 1 -- 172.21.15.22:0/1773671622 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6728048050 con 0x5561fb12a560 2024-09-19T08:16:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.264+0000 7f6736f6c740 10 client.15663 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.264+0000 7f6736f6c740 1 -- 172.21.15.22:0/1773671622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5561fb114f20 con 0x5561fb12a560 2024-09-19T08:16:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.265+0000 7f6715ffb640 1 -- 172.21.15.22:0/1773671622 <== mon.0 v2:172.21.15.22:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6728042030 con 0x5561fb12a560 2024-09-19T08:16:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.265+0000 7f6736f6c740 10 client.15663 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.265+0000 7f6736f6c740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:16:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.265+0000 7f6736f6c740 20 client.15663 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.265+0000 7f6736f6c740 1 --2- 172.21.15.22:0/1773671622 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5561fb1150f0 0x5561fb1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.265+0000 7f6736f6c740 4 client.15663 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:16:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.265+0000 7f6736f6c740 1 -- 172.21.15.22:0/1773671622 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561fb07e860 con 0x5561fb1150f0 2024-09-19T08:16:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.265+0000 7f672e59c640 1 --2- 172.21.15.22:0/1773671622 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5561fb1150f0 0x5561fb1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.266+0000 7f6715ffb640 10 client.15663 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.266+0000 7f672e59c640 1 --2- 172.21.15.22:0/1773671622 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5561fb1150f0 0x5561fb1981c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f24eb5f453a6ea server_cookie=f58dd0fc0cf32adb in_seq=0 out_seq=0 2024-09-19T08:16:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.266+0000 7f6715ffb640 1 -- 172.21.15.22:0/1773671622 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561fb07e860 con 0x5561fb1150f0 2024-09-19T08:16:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.266+0000 7f6715ffb640 10 client.15663 handle_command_reply: tid=0 2024-09-19T08:16:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.266+0000 7f6736f6c740 1 client.15663 shutdown 2024-09-19T08:16:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.266+0000 7f6736f6c740 1 -- 172.21.15.22:0/1773671622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb12a560 msgr2=0x5561fb113830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:10.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.266+0000 7f6736f6c740 1 --2- 172.21.15.22:0/1773671622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb12a560 0x5561fb113830 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6728009bf0 tx=0x7f6728031000 comp rx=0 tx=0).stop 2024-09-19T08:16:10.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.266+0000 7f6736f6c740 1 -- 172.21.15.22:0/1773671622 shutdown_connections 2024-09-19T08:16:10.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.266+0000 7f6736f6c740 1 --2- 172.21.15.22:0/1773671622 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5561fb1150f0 0x5561fb1981c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.281 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:16:10.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.266+0000 7f6715ffb640 0 client.15663 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:10.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.267+0000 7f6736f6c740 1 --2- 172.21.15.22:0/1773671622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561fb12a560 0x5561fb113830 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.267+0000 7f6736f6c740 1 --2- 172.21.15.22:0/1773671622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561fb11a780 0x5561fb122d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.267+0000 7f6736f6c740 1 --2- 172.21.15.22:0/1773671622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5561fb10cb40 0x5561fb122830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.267+0000 7f6736f6c740 1 -- 172.21.15.22:0/1773671622 >> 172.21.15.22:0/1773671622 conn(0x5561fb0d4310 msgr2=0x5561fb117580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:10.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.267+0000 7f6736f6c740 1 -- 172.21.15.22:0/1773671622 shutdown_connections 2024-09-19T08:16:10.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.267+0000 7f6736f6c740 1 -- 172.21.15.22:0/1773671622 wait complete. 2024-09-19T08:16:10.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.267+0000 7f6736f6c740 20 client.15663 trim_cache size 0 max 16384 2024-09-19T08:16:10.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.267+0000 7f6734b93640 1 -- 172.21.15.22:0/135679370 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f67180a34f0 msgr2=0x7f67180a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:10.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.267+0000 7f6734b93640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f67180a34f0 0x7f67180a59b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f67280042e0 tx=0x7f672802a8e0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.267+0000 7f6734b93640 1 -- 172.21.15.22:0/135679370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6730103760 msgr2=0x7f6730198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:10.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.267+0000 7f6734b93640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6730103760 0x7f6730198af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f672400d950 tx=0x7f672400de10 comp rx=0 tx=0).stop 2024-09-19T08:16:10.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.268+0000 7f6734b93640 1 -- 172.21.15.22:0/135679370 shutdown_connections 2024-09-19T08:16:10.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.268+0000 7f6734b93640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f67180a34f0 0x7f67180a59b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.268+0000 7f6734b93640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67301082e0 0x7f673019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.268+0000 7f6734b93640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6730103760 0x7f6730198af0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.268+0000 7f6734b93640 1 --2- 172.21.15.22:0/135679370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6730102db0 0x7f67301985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:10.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.268+0000 7f6734b93640 1 -- 172.21.15.22:0/135679370 >> 172.21.15.22:0/135679370 conn(0x7f67300fc7d0 msgr2=0x7f6730106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:10.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.269+0000 7f6734b93640 1 -- 172.21.15.22:0/135679370 shutdown_connections 2024-09-19T08:16:10.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:10.269+0000 7f6734b93640 1 -- 172.21.15.22:0/135679370 wait complete. 2024-09-19T08:16:10.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:10 smithi064 ceph-mon[34900]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T08:16:11.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:10 smithi157 ceph-mon[35143]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T08:16:11.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:10 smithi022 ceph-mon[30963]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T08:16:11.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:11 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:16:12.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:11 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:16:12.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:11 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:16:12.057 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:16:12.058 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:16:12.439 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:12.911 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:12 smithi022 ceph-mon[30963]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T08:16:12.911 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:12 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:16:12.911 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:12 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:16:12.911 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:12 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:16:12.912 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:12 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:16:12.912 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:12 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:16:13.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.048+0000 7fd9590f4640 1 -- 172.21.15.22:0/4045631075 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944031940 con 0x7fd954102570 2024-09-19T08:16:13.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.049+0000 7fd95c35a640 1 -- 172.21.15.22:0/4045631075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd954102570 msgr2=0x7fd9541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:13.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.049+0000 7fd95c35a640 1 --2- 172.21.15.22:0/4045631075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd954102570 0x7fd9541029b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd944009900 tx=0x7fd94402a480 comp rx=0 tx=0).stop 2024-09-19T08:16:13.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.049+0000 7fd95c35a640 1 -- 172.21.15.22:0/4045631075 shutdown_connections 2024-09-19T08:16:13.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.049+0000 7fd95c35a640 1 --2- 172.21.15.22:0/4045631075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd954102ef0 0x7fd95410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.049+0000 7fd95c35a640 1 --2- 172.21.15.22:0/4045631075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd954102570 0x7fd9541029b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.049+0000 7fd95c35a640 1 --2- 172.21.15.22:0/4045631075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd954100e20 0x7fd954101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.049+0000 7fd95c35a640 1 -- 172.21.15.22:0/4045631075 >> 172.21.15.22:0/4045631075 conn(0x7fd9540fc7d0 msgr2=0x7fd9540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:13.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.049+0000 7fd95c35a640 1 -- 172.21.15.22:0/4045631075 shutdown_connections 2024-09-19T08:16:13.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.049+0000 7fd95c35a640 1 -- 172.21.15.22:0/4045631075 wait complete. 2024-09-19T08:16:13.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.050+0000 7fd95c35a640 1 Processor -- start 2024-09-19T08:16:13.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.050+0000 7fd95c35a640 1 -- start start 2024-09-19T08:16:13.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95c35a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd954100e20 0x7fd9541963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95c35a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd954102570 0x7fd954196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95c35a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd954102ef0 0x7fd95419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95c35a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd95410fe70 con 0x7fd954102ef0 2024-09-19T08:16:13.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95c35a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd95410fcf0 con 0x7fd954100e20 2024-09-19T08:16:13.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95c35a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd95410fff0 con 0x7fd954102570 2024-09-19T08:16:13.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95a8f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd954102ef0 0x7fd95419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95a8f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd954102ef0 0x7fd95419acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33124/0 (socket says 172.21.15.22:33124) 2024-09-19T08:16:13.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95a8f7640 1 -- 172.21.15.22:0/3903971696 learned_addr learned my addr 172.21.15.22:0/3903971696 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:13.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95a0f6640 1 --2- 172.21.15.22:0/3903971696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd954100e20 0x7fd9541963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd9598f5640 1 --2- 172.21.15.22:0/3903971696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd954102570 0x7fd954196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95a0f6640 1 -- 172.21.15.22:0/3903971696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd954102570 msgr2=0x7fd954196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:13.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95a0f6640 1 --2- 172.21.15.22:0/3903971696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd954102570 0x7fd954196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95a0f6640 1 -- 172.21.15.22:0/3903971696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd954102ef0 msgr2=0x7fd95419acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:13.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95a0f6640 1 --2- 172.21.15.22:0/3903971696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd954102ef0 0x7fd95419acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.051+0000 7fd95a0f6640 1 -- 172.21.15.22:0/3903971696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95419b3a0 con 0x7fd954100e20 2024-09-19T08:16:13.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.052+0000 7fd95a0f6640 1 --2- 172.21.15.22:0/3903971696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd954100e20 0x7fd9541963f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd95000d720 tx=0x7fd95000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:13.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.052+0000 7fd94b7fe640 1 -- 172.21.15.22:0/3903971696 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd950017070 con 0x7fd954100e20 2024-09-19T08:16:13.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.052+0000 7fd95c35a640 1 -- 172.21.15.22:0/3903971696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95419b690 con 0x7fd954100e20 2024-09-19T08:16:13.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.052+0000 7fd95c35a640 1 -- 172.21.15.22:0/3903971696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9541a2f70 con 0x7fd954100e20 2024-09-19T08:16:13.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.052+0000 7fd94b7fe640 1 -- 172.21.15.22:0/3903971696 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd95000de30 con 0x7fd954100e20 2024-09-19T08:16:13.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.052+0000 7fd94b7fe640 1 -- 172.21.15.22:0/3903971696 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9500047a0 con 0x7fd954100e20 2024-09-19T08:16:13.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.055+0000 7fd95e733740 1 Processor -- start 2024-09-19T08:16:13.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.056+0000 7fd95e733740 1 -- start start 2024-09-19T08:16:13.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.056+0000 7fd95e733740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf37aa80 0x5605bf37ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.056+0000 7fd95e733740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf37b3a0 0x5605bf38e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.056+0000 7fd95e733740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf38e550 0x5605bf390940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.056+0000 7fd95e733740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5605bf2e0280 con 0x5605bf38e550 2024-09-19T08:16:13.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.056+0000 7fd95e733740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5605bf287b10 con 0x5605bf37aa80 2024-09-19T08:16:13.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.056+0000 7fd95e733740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5605bf2dc240 con 0x5605bf37b3a0 2024-09-19T08:16:13.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.056+0000 7fd9598f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf38e550 0x5605bf390940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.056+0000 7fd9598f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf38e550 0x5605bf390940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33132/0 (socket says 172.21.15.22:33132) 2024-09-19T08:16:13.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.056+0000 7fd9598f5640 1 -- 172.21.15.22:0/4185549982 learned_addr learned my addr 172.21.15.22:0/4185549982 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:13.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.056+0000 7fd94b7fe640 1 -- 172.21.15.22:0/3903971696 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fd9500099d0 con 0x7fd954100e20 2024-09-19T08:16:13.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.057+0000 7fd95a8f7640 1 --2- 172.21.15.22:0/4185549982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf37aa80 0x5605bf37ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.057+0000 7fd9598f5640 1 -- 172.21.15.22:0/4185549982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf37b3a0 msgr2=0x5605bf38e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:13.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.057+0000 7fd9598f5640 1 --2- 172.21.15.22:0/4185549982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf37b3a0 0x5605bf38e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.057+0000 7fd9598f5640 1 -- 172.21.15.22:0/4185549982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf37aa80 msgr2=0x5605bf37ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:13.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.057+0000 7fd9598f5640 1 --2- 172.21.15.22:0/4185549982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf37aa80 0x5605bf37ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.057+0000 7fd9598f5640 1 -- 172.21.15.22:0/4185549982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605bf2941f0 con 0x5605bf38e550 2024-09-19T08:16:13.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.057+0000 7fd9598f5640 1 --2- 172.21.15.22:0/4185549982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf38e550 0x5605bf390940 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd94402a9a0 tx=0x7fd944004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb7b18ab6f6f782 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:13.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.058+0000 7fd948dea640 1 -- 172.21.15.22:0/4185549982 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944035070 con 0x5605bf38e550 2024-09-19T08:16:13.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.058+0000 7fd948dea640 1 -- 172.21.15.22:0/4185549982 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd944003ad0 con 0x5605bf38e550 2024-09-19T08:16:13.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.058+0000 7fd948dea640 1 -- 172.21.15.22:0/4185549982 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944031be0 con 0x5605bf38e550 2024-09-19T08:16:13.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.058+0000 7fd94b7fe640 1 --2- 172.21.15.22:0/3903971696 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd9400a35d0 0x7fd9400a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.059+0000 7fd95a8f7640 1 --2- 172.21.15.22:0/3903971696 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd9400a35d0 0x7fd9400a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.059+0000 7fd94b7fe640 1 -- 172.21.15.22:0/3903971696 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd9500cfaf0 con 0x7fd954100e20 2024-09-19T08:16:13.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.059+0000 7fd95e733740 1 -- 172.21.15.22:0/4185549982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf38e550 msgr2=0x5605bf390940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:13.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.059+0000 7fd95e733740 1 --2- 172.21.15.22:0/4185549982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf38e550 0x5605bf390940 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd94402a9a0 tx=0x7fd944004040 comp rx=0 tx=0).stop 2024-09-19T08:16:13.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.059+0000 7fd95a8f7640 1 --2- 172.21.15.22:0/3903971696 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd9400a35d0 0x7fd9400a5a90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009370 tx=0x7fd94c009760 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:13.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.060+0000 7fd95e733740 1 -- 172.21.15.22:0/4185549982 shutdown_connections 2024-09-19T08:16:13.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.060+0000 7fd95e733740 1 --2- 172.21.15.22:0/4185549982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf38e550 0x5605bf390940 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.060+0000 7fd95e733740 1 --2- 172.21.15.22:0/4185549982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf37b3a0 0x5605bf38e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.060+0000 7fd95e733740 1 --2- 172.21.15.22:0/4185549982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf37aa80 0x5605bf37ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.060+0000 7fd95e733740 1 -- 172.21.15.22:0/4185549982 >> 172.21.15.22:0/4185549982 conn(0x5605bf3770f0 msgr2=0x5605bf3774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:13.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.060+0000 7fd95e733740 1 -- 172.21.15.22:0/4185549982 shutdown_connections 2024-09-19T08:16:13.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.060+0000 7fd95e733740 1 -- 172.21.15.22:0/4185549982 wait complete. 2024-09-19T08:16:13.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.060+0000 7fd95e733740 1 -- start start 2024-09-19T08:16:13.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.061+0000 7fd95e733740 1 Processor -- start 2024-09-19T08:16:13.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.061+0000 7fd95e733740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37aa80 0x5605bf38b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.061+0000 7fd95e733740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf37b3a0 0x5605bf38baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.061+0000 7fd95e733740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf38e550 0x5605bf38bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.061+0000 7fd95e733740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5605bf391000 con 0x5605bf37aa80 2024-09-19T08:16:13.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.061+0000 7fd95e733740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5605bf29af10 con 0x5605bf37b3a0 2024-09-19T08:16:13.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.061+0000 7fd95e733740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5605bf391180 con 0x5605bf38e550 2024-09-19T08:16:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.061+0000 7fd95e733740 10 client.0 _ll_register_callbacks cb 0x5605bf263a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.061+0000 7fd95a8f7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf38e550 0x5605bf38bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.061+0000 7fd95a8f7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf38e550 0x5605bf38bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:54784/0 (socket says 172.21.15.22:54784) 2024-09-19T08:16:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.061+0000 7fd95a8f7640 1 -- 172.21.15.22:0/3056488290 learned_addr learned my addr 172.21.15.22:0/3056488290 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.061+0000 7fd95a0f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37aa80 0x5605bf38b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.062+0000 7fd95a8f7640 1 -- 172.21.15.22:0/3056488290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf37b3a0 msgr2=0x5605bf38baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.062+0000 7fd95a8f7640 1 --2- 172.21.15.22:0/3056488290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf37b3a0 0x5605bf38baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.062+0000 7fd95a8f7640 1 -- 172.21.15.22:0/3056488290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37aa80 msgr2=0x5605bf38b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.062+0000 7fd95a8f7640 1 --2- 172.21.15.22:0/3056488290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37aa80 0x5605bf38b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.062+0000 7fd95a8f7640 1 -- 172.21.15.22:0/3056488290 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605bf0516a0 con 0x5605bf38e550 2024-09-19T08:16:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.062+0000 7fd95a0f6640 1 --2- 172.21.15.22:0/3056488290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37aa80 0x5605bf38b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.062+0000 7fd95a8f7640 1 --2- 172.21.15.22:0/3056488290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf38e550 0x5605bf38bfe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd94c0108c0 tx=0x7fd94c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44e30b73987eb725 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.062+0000 7fd93dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:16:13.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.062+0000 7fd93dffb640 1 -- 172.21.15.22:0/3056488290 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c01fbc0 con 0x5605bf38e550 2024-09-19T08:16:13.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.062+0000 7fd93dffb640 1 -- 172.21.15.22:0/3056488290 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd94c01fd60 con 0x5605bf38e550 2024-09-19T08:16:13.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.062+0000 7fd95e733740 1 -- 172.21.15.22:0/3056488290 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd944003ad0 con 0x5605bf38e550 2024-09-19T08:16:13.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.062+0000 7fd93dffb640 1 -- 172.21.15.22:0/3056488290 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd94c003970 con 0x5605bf38e550 2024-09-19T08:16:13.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.062+0000 7fd95e733740 10 client.25273 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:13.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.063+0000 7fd95e733740 1 -- 172.21.15.22:0/3056488290 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5605bf39ce00 con 0x5605bf38e550 2024-09-19T08:16:13.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.063+0000 7fd93dffb640 1 -- 172.21.15.22:0/3056488290 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd94c0278d0 con 0x5605bf38e550 2024-09-19T08:16:13.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.063+0000 7fd93dffb640 1 -- 172.21.15.22:0/3056488290 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd94c026070 con 0x5605bf38e550 2024-09-19T08:16:13.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.063+0000 7fd95e733740 10 client.25273 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:13.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.063+0000 7fd95e733740 10 client.25273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:16:13.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.063+0000 7fd95e733740 20 client.25273 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:13.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.063+0000 7fd95e733740 1 --2- 172.21.15.22:0/3056488290 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5605bf39e460 0x5605bf3a0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.063+0000 7fd95e733740 4 client.25273 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:16:13.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.063+0000 7fd95e733740 1 -- 172.21.15.22:0/3056488290 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd944003ad0 con 0x5605bf39e460 2024-09-19T08:16:13.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.063+0000 7fd95a0f6640 1 --2- 172.21.15.22:0/3056488290 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5605bf39e460 0x5605bf3a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.064+0000 7fd95a0f6640 1 --2- 172.21.15.22:0/3056488290 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5605bf39e460 0x5605bf3a0900 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c698a8b2067735a7 server_cookie=832bfd9ded813a1e in_seq=0 out_seq=0 2024-09-19T08:16:13.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.064+0000 7fd93dffb640 10 client.25273 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:13.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd93dffb640 1 -- 172.21.15.22:0/3056488290 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd944003ad0 con 0x5605bf39e460 2024-09-19T08:16:13.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd93dffb640 10 client.25273 handle_command_reply: tid=0 2024-09-19T08:16:13.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd95e733740 1 client.25273 shutdown 2024-09-19T08:16:13.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd95e733740 1 -- 172.21.15.22:0/3056488290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf38e550 msgr2=0x5605bf38bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:13.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd95e733740 1 --2- 172.21.15.22:0/3056488290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf38e550 0x5605bf38bfe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd94c0108c0 tx=0x7fd94c004ab0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd95e733740 1 -- 172.21.15.22:0/3056488290 shutdown_connections 2024-09-19T08:16:13.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd95e733740 1 --2- 172.21.15.22:0/3056488290 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5605bf39e460 0x5605bf3a0900 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd95e733740 1 --2- 172.21.15.22:0/3056488290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf38e550 0x5605bf38bfe0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd95e733740 1 --2- 172.21.15.22:0/3056488290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf37b3a0 0x5605bf38baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd93dffb640 0 client.25273 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:13.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd95e733740 1 --2- 172.21.15.22:0/3056488290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37aa80 0x5605bf38b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd95e733740 1 -- 172.21.15.22:0/3056488290 >> 172.21.15.22:0/3056488290 conn(0x5605bf3770f0 msgr2=0x5605bf37cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:13.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd95e733740 1 -- 172.21.15.22:0/3056488290 shutdown_connections 2024-09-19T08:16:13.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd95e733740 1 -- 172.21.15.22:0/3056488290 wait complete. 2024-09-19T08:16:13.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.065+0000 7fd95e733740 20 client.25273 trim_cache size 0 max 16384 2024-09-19T08:16:13.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:12 smithi064 ceph-mon[34900]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T08:16:13.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:12 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:16:13.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:12 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:16:13.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:12 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:16:13.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:12 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:16:13.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:12 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:16:13.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.100+0000 7fd95e733740 1 Processor -- start 2024-09-19T08:16:13.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.100+0000 7fd95e733740 1 -- start start 2024-09-19T08:16:13.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.100+0000 7fd95e733740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37aa80 0x5605bf38be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.100+0000 7fd95e733740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf37b3a0 0x5605bf3a6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.100+0000 7fd95e733740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf38e550 0x5605bf3a6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.100+0000 7fd95e733740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541adf40 con 0x5605bf37aa80 2024-09-19T08:16:13.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.100+0000 7fd95e733740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541ab810 con 0x5605bf38e550 2024-09-19T08:16:13.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.100+0000 7fd95e733740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541a9660 con 0x5605bf37b3a0 2024-09-19T08:16:13.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.100+0000 7fd95a0f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37aa80 0x5605bf38be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.100+0000 7fd95a0f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37aa80 0x5605bf38be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33154/0 (socket says 172.21.15.22:33154) 2024-09-19T08:16:13.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.100+0000 7fd95a0f6640 1 -- 172.21.15.22:0/1850183655 learned_addr learned my addr 172.21.15.22:0/1850183655 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:13.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.100+0000 7fd95a8f7640 1 --2- 172.21.15.22:0/1850183655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf38e550 0x5605bf3a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.100+0000 7fd9598f5640 1 --2- 172.21.15.22:0/1850183655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf37b3a0 0x5605bf3a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.101+0000 7fd95a0f6640 1 -- 172.21.15.22:0/1850183655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf37b3a0 msgr2=0x5605bf3a6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:13.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.101+0000 7fd95a0f6640 1 --2- 172.21.15.22:0/1850183655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf37b3a0 0x5605bf3a6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.101+0000 7fd95a0f6640 1 -- 172.21.15.22:0/1850183655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf38e550 msgr2=0x5605bf3a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:13.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.101+0000 7fd95a0f6640 1 --2- 172.21.15.22:0/1850183655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf38e550 0x5605bf3a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.101+0000 7fd95a0f6640 1 -- 172.21.15.22:0/1850183655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605bf37b1d0 con 0x5605bf37aa80 2024-09-19T08:16:13.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.101+0000 7fd95a0f6640 1 --2- 172.21.15.22:0/1850183655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37aa80 0x5605bf38be30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd950004770 tx=0x7fd950086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5ac7a0fc695e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:13.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.101+0000 7fd93ffff640 1 -- 172.21.15.22:0/1850183655 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9500a4810 con 0x5605bf37aa80 2024-09-19T08:16:13.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.101+0000 7fd93ffff640 1 -- 172.21.15.22:0/1850183655 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd950015070 con 0x5605bf37aa80 2024-09-19T08:16:13.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.101+0000 7fd93ffff640 1 -- 172.21.15.22:0/1850183655 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9500860b0 con 0x5605bf37aa80 2024-09-19T08:16:13.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.102+0000 7fd95e733740 1 -- 172.21.15.22:0/1850183655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37aa80 msgr2=0x5605bf38be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:13.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.102+0000 7fd95e733740 1 --2- 172.21.15.22:0/1850183655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37aa80 0x5605bf38be30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd950004770 tx=0x7fd950086040 comp rx=0 tx=0).stop 2024-09-19T08:16:13.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.102+0000 7fd95e733740 1 -- 172.21.15.22:0/1850183655 shutdown_connections 2024-09-19T08:16:13.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.102+0000 7fd95e733740 1 --2- 172.21.15.22:0/1850183655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf38e550 0x5605bf3a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.102+0000 7fd95e733740 1 --2- 172.21.15.22:0/1850183655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf37b3a0 0x5605bf3a6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.102+0000 7fd95e733740 1 --2- 172.21.15.22:0/1850183655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37aa80 0x5605bf38be30 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.103+0000 7fd95e733740 1 -- 172.21.15.22:0/1850183655 >> 172.21.15.22:0/1850183655 conn(0x5605bf3770f0 msgr2=0x5605bf38d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:13.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.103+0000 7fd95e733740 1 -- 172.21.15.22:0/1850183655 shutdown_connections 2024-09-19T08:16:13.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.103+0000 7fd95e733740 1 -- 172.21.15.22:0/1850183655 wait complete. 2024-09-19T08:16:13.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.103+0000 7fd95e733740 1 -- start start 2024-09-19T08:16:13.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.103+0000 7fd95e733740 1 Processor -- start 2024-09-19T08:16:13.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd95e733740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf37aa80 0x5605bf392870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd95e733740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37b3a0 0x5605bf392db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd9598f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37b3a0 0x5605bf392db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd9598f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37b3a0 0x5605bf392db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33158/0 (socket says 172.21.15.22:33158) 2024-09-19T08:16:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd95e733740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf38e550 0x5605bf4081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd95e733740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541a9cd0 con 0x5605bf37b3a0 2024-09-19T08:16:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd95e733740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541ab140 con 0x5605bf37aa80 2024-09-19T08:16:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd95e733740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541aa3a0 con 0x5605bf38e550 2024-09-19T08:16:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd9598f5640 1 -- 172.21.15.22:0/1825638832 learned_addr learned my addr 172.21.15.22:0/1825638832 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd95e733740 10 client.0 _ll_register_callbacks cb 0x7fd94c009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd95a8f7640 1 --2- 172.21.15.22:0/1825638832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf38e550 0x5605bf4081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd9598f5640 1 -- 172.21.15.22:0/1825638832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf38e550 msgr2=0x5605bf4081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd9598f5640 1 --2- 172.21.15.22:0/1825638832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf38e550 0x5605bf4081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd9598f5640 1 -- 172.21.15.22:0/1825638832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf37aa80 msgr2=0x5605bf392870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd9598f5640 1 --2- 172.21.15.22:0/1825638832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf37aa80 0x5605bf392870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.104+0000 7fd9598f5640 1 -- 172.21.15.22:0/1825638832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605bf408720 con 0x5605bf37b3a0 2024-09-19T08:16:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.105+0000 7fd9598f5640 1 --2- 172.21.15.22:0/1825638832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37b3a0 0x5605bf392db0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd94402a9a0 tx=0x7fd944031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d5265a403fcdf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.105+0000 7fd93dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.105+0000 7fd93dffb640 1 -- 172.21.15.22:0/1825638832 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944050030 con 0x5605bf37b3a0 2024-09-19T08:16:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.105+0000 7fd93dffb640 1 -- 172.21.15.22:0/1825638832 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd944049040 con 0x5605bf37b3a0 2024-09-19T08:16:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.105+0000 7fd95e733740 1 -- 172.21.15.22:0/1825638832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd950015070 con 0x5605bf37b3a0 2024-09-19T08:16:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.106+0000 7fd93dffb640 1 -- 172.21.15.22:0/1825638832 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd94405a050 con 0x5605bf37b3a0 2024-09-19T08:16:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.106+0000 7fd95e733740 10 client.15681 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.106+0000 7fd95e733740 1 -- 172.21.15.22:0/1825638832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5605bf4099e0 con 0x5605bf37b3a0 2024-09-19T08:16:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.106+0000 7fd93dffb640 1 -- 172.21.15.22:0/1825638832 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd950015070 con 0x5605bf37b3a0 2024-09-19T08:16:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.106+0000 7fd93dffb640 1 -- 172.21.15.22:0/1825638832 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd944003d10 con 0x5605bf37b3a0 2024-09-19T08:16:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.106+0000 7fd95e733740 10 client.15681 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.106+0000 7fd95e733740 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:16:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.106+0000 7fd95e733740 20 client.15681 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.106+0000 7fd95e733740 1 --2- 172.21.15.22:0/1825638832 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5605bf40a160 0x5605bf40c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.106+0000 7fd95e733740 4 client.15681 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:16:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.106+0000 7fd95e733740 1 -- 172.21.15.22:0/1825638832 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5605bf2ed7f0 con 0x5605bf40a160 2024-09-19T08:16:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.107+0000 7fd95a0f6640 1 --2- 172.21.15.22:0/1825638832 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5605bf40a160 0x5605bf40c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:13.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.107+0000 7fd93dffb640 10 client.15681 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:13.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.107+0000 7fd95a0f6640 1 --2- 172.21.15.22:0/1825638832 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5605bf40a160 0x5605bf40c580 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=678aa1625ea97ec7 server_cookie=8fe661b3fc99b92e in_seq=0 out_seq=0 2024-09-19T08:16:13.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.109+0000 7fd93dffb640 1 -- 172.21.15.22:0/1825638832 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5605bf2ed7f0 con 0x5605bf40a160 2024-09-19T08:16:13.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.109+0000 7fd93dffb640 10 client.15681 handle_command_reply: tid=0 2024-09-19T08:16:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.109+0000 7fd95e733740 1 client.15681 shutdown 2024-09-19T08:16:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.109+0000 7fd95e733740 1 -- 172.21.15.22:0/1825638832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37b3a0 msgr2=0x5605bf392db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.109+0000 7fd95e733740 1 --2- 172.21.15.22:0/1825638832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37b3a0 0x5605bf392db0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd94402a9a0 tx=0x7fd944031750 comp rx=0 tx=0).stop 2024-09-19T08:16:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.109+0000 7fd95e733740 1 -- 172.21.15.22:0/1825638832 shutdown_connections 2024-09-19T08:16:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.109+0000 7fd95e733740 1 --2- 172.21.15.22:0/1825638832 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5605bf40a160 0x5605bf40c580 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.109+0000 7fd95e733740 1 --2- 172.21.15.22:0/1825638832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5605bf38e550 0x5605bf4081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.109+0000 7fd95e733740 1 --2- 172.21.15.22:0/1825638832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5605bf37b3a0 0x5605bf392db0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.109+0000 7fd95e733740 1 --2- 172.21.15.22:0/1825638832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605bf37aa80 0x5605bf392870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.109+0000 7fd95e733740 1 -- 172.21.15.22:0/1825638832 >> 172.21.15.22:0/1825638832 conn(0x5605bf3770f0 msgr2=0x5605bf38cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.109+0000 7fd93dffb640 0 client.15681 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.109+0000 7fd95e733740 1 -- 172.21.15.22:0/1825638832 shutdown_connections 2024-09-19T08:16:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.110+0000 7fd95e733740 1 -- 172.21.15.22:0/1825638832 wait complete. 2024-09-19T08:16:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.110+0000 7fd95e733740 20 client.15681 trim_cache size 0 max 16384 2024-09-19T08:16:13.113 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:16:13.113 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-19T08:16:13.113 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "3ef1d6f2-994a-4c75-a231-78b37b1662d9", 2024-09-19T08:16:13.113 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-19T08:16:13.113 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:16:13.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.110+0000 7fd95c35a640 1 -- 172.21.15.22:0/3903971696 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd9400a35d0 msgr2=0x7fd9400a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:13.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.110+0000 7fd95c35a640 1 --2- 172.21.15.22:0/3903971696 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd9400a35d0 0x7fd9400a5a90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009370 tx=0x7fd94c009760 comp rx=0 tx=0).stop 2024-09-19T08:16:13.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.110+0000 7fd95c35a640 1 -- 172.21.15.22:0/3903971696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd954100e20 msgr2=0x7fd9541963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:13.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.110+0000 7fd95c35a640 1 --2- 172.21.15.22:0/3903971696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd954100e20 0x7fd9541963f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd95000d720 tx=0x7fd95000dbe0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.111+0000 7fd95c35a640 1 -- 172.21.15.22:0/3903971696 shutdown_connections 2024-09-19T08:16:13.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.111+0000 7fd95c35a640 1 --2- 172.21.15.22:0/3903971696 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd9400a35d0 0x7fd9400a5a90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.111+0000 7fd95c35a640 1 --2- 172.21.15.22:0/3903971696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd954102ef0 0x7fd95419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.111+0000 7fd95c35a640 1 --2- 172.21.15.22:0/3903971696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd954102570 0x7fd954196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.111+0000 7fd95c35a640 1 --2- 172.21.15.22:0/3903971696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd954100e20 0x7fd9541963f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:13.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.111+0000 7fd95c35a640 1 -- 172.21.15.22:0/3903971696 >> 172.21.15.22:0/3903971696 conn(0x7fd9540fc7d0 msgr2=0x7fd9540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:13.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.111+0000 7fd95c35a640 1 -- 172.21.15.22:0/3903971696 shutdown_connections 2024-09-19T08:16:13.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:13.111+0000 7fd95c35a640 1 -- 172.21.15.22:0/3903971696 wait complete. 2024-09-19T08:16:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:12 smithi157 ceph-mon[35143]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T08:16:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:12 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:16:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:12 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:16:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:12 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:16:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:12 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:16:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:12 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:16:13.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:16:14.189 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:14.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.810+0000 7fb1062ca640 1 -- 172.21.15.22:0/2125938638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb100100aa0 msgr2=0x7fb10010ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:14.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.810+0000 7fb1062ca640 1 --2- 172.21.15.22:0/2125938638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb100100aa0 0x7fb10010ea20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009ca0 tx=0x7fb0f802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.810+0000 7fb1062ca640 1 -- 172.21.15.22:0/2125938638 shutdown_connections 2024-09-19T08:16:14.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.810+0000 7fb1062ca640 1 --2- 172.21.15.22:0/2125938638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb100100aa0 0x7fb10010ea20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.810+0000 7fb1062ca640 1 --2- 172.21.15.22:0/2125938638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100100100 0x7fb100100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.810+0000 7fb1062ca640 1 --2- 172.21.15.22:0/2125938638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb100106b80 0x7fb100106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.810+0000 7fb1062ca640 1 -- 172.21.15.22:0/2125938638 >> 172.21.15.22:0/2125938638 conn(0x7fb100077230 msgr2=0x7fb1000fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:14.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.810+0000 7fb1062ca640 1 -- 172.21.15.22:0/2125938638 shutdown_connections 2024-09-19T08:16:14.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.810+0000 7fb1062ca640 1 -- 172.21.15.22:0/2125938638 wait complete. 2024-09-19T08:16:14.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.811+0000 7fb1062ca640 1 Processor -- start 2024-09-19T08:16:14.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.811+0000 7fb1062ca640 1 -- start start 2024-09-19T08:16:14.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.811+0000 7fb1062ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100100100 0x7fb100198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.811+0000 7fb1062ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb100100aa0 0x7fb100198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.811+0000 7fb1062ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb100106b80 0x7fb10019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.811+0000 7fb1062ca640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb100112050 con 0x7fb100100aa0 2024-09-19T08:16:14.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.812+0000 7fb1062ca640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb100111ed0 con 0x7fb100106b80 2024-09-19T08:16:14.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.812+0000 7fb1062ca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001121d0 con 0x7fb100100100 2024-09-19T08:16:14.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.812+0000 7fb0ff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb100100aa0 0x7fb100198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.812+0000 7fb0ff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb100100aa0 0x7fb100198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33166/0 (socket says 172.21.15.22:33166) 2024-09-19T08:16:14.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.812+0000 7fb0fffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100100100 0x7fb100198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.812+0000 7fb0ff7fe640 1 -- 172.21.15.22:0/571325448 learned_addr learned my addr 172.21.15.22:0/571325448 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:14.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.812+0000 7fb104867640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb100106b80 0x7fb10019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.812+0000 7fb0fffff640 1 -- 172.21.15.22:0/571325448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb100106b80 msgr2=0x7fb10019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:14.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.812+0000 7fb0fffff640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb100106b80 0x7fb10019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.812+0000 7fb0fffff640 1 -- 172.21.15.22:0/571325448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb100100aa0 msgr2=0x7fb100198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:14.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.812+0000 7fb0fffff640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb100100aa0 0x7fb100198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.812+0000 7fb0fffff640 1 -- 172.21.15.22:0/571325448 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10019d3e0 con 0x7fb100100100 2024-09-19T08:16:14.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.812+0000 7fb104867640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb100106b80 0x7fb10019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:14.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.812+0000 7fb0ff7fe640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb100100aa0 0x7fb100198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:14.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.813+0000 7fb0fffff640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100100100 0x7fb100198430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000e9f0 tx=0x7fb0f000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:14.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.813+0000 7fb0fd7fa640 1 -- 172.21.15.22:0/571325448 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000ccc0 con 0x7fb100100100 2024-09-19T08:16:14.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.813+0000 7fb1062ca640 1 -- 172.21.15.22:0/571325448 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10019d6d0 con 0x7fb100100100 2024-09-19T08:16:14.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.813+0000 7fb0fd7fa640 1 -- 172.21.15.22:0/571325448 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f0010910 con 0x7fb100100100 2024-09-19T08:16:14.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.813+0000 7fb0fd7fa640 1 -- 172.21.15.22:0/571325448 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0018880 con 0x7fb100100100 2024-09-19T08:16:14.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.813+0000 7fb1062ca640 1 -- 172.21.15.22:0/571325448 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1001a4fb0 con 0x7fb100100100 2024-09-19T08:16:14.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.816+0000 7fb1086a3740 1 Processor -- start 2024-09-19T08:16:14.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.816+0000 7fb1086a3740 1 -- start start 2024-09-19T08:16:14.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.816+0000 7fb1086a3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413ab360 0x55f6413ab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.816+0000 7fb1086a3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413abc80 0x55f6413ba550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.816+0000 7fb1086a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413baa90 0x55f6413bce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.816+0000 7fb1086a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f6413112e0 con 0x55f6413baa90 2024-09-19T08:16:14.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.816+0000 7fb1086a3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f6412b8ce0 con 0x55f6413ab360 2024-09-19T08:16:14.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.816+0000 7fb1086a3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f64130d2a0 con 0x55f6413abc80 2024-09-19T08:16:14.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.816+0000 7fb0ff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413baa90 0x55f6413bce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.816+0000 7fb0ff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413baa90 0x55f6413bce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33170/0 (socket says 172.21.15.22:33170) 2024-09-19T08:16:14.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.816+0000 7fb0ff7fe640 1 -- 172.21.15.22:0/3687826865 learned_addr learned my addr 172.21.15.22:0/3687826865 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:14.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.816+0000 7fb104867640 1 --2- 172.21.15.22:0/3687826865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413ab360 0x55f6413ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.816+0000 7fb0fd7fa640 1 -- 172.21.15.22:0/571325448 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fb0f0010ac0 con 0x7fb100100100 2024-09-19T08:16:14.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.817+0000 7fb104867640 1 -- 172.21.15.22:0/3687826865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413abc80 msgr2=0x55f6413ba550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:14.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.817+0000 7fb0fffff640 1 --2- 172.21.15.22:0/3687826865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413abc80 0x55f6413ba550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.817+0000 7fb104867640 1 --2- 172.21.15.22:0/3687826865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413abc80 0x55f6413ba550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.817+0000 7fb104867640 1 -- 172.21.15.22:0/3687826865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413baa90 msgr2=0x55f6413bce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:14.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.817+0000 7fb104867640 1 --2- 172.21.15.22:0/3687826865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413baa90 0x55f6413bce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.817+0000 7fb104867640 1 -- 172.21.15.22:0/3687826865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f64129d440 con 0x55f6413ab360 2024-09-19T08:16:14.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.817+0000 7fb0ff7fe640 1 --2- 172.21.15.22:0/3687826865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413baa90 0x55f6413bce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:14.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.817+0000 7fb104867640 1 --2- 172.21.15.22:0/3687826865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413ab360 0x55f6413ab740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009bf0 tx=0x7fb0f8009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8786d89f2e0124d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:14.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.817+0000 7fb0fffff640 1 --2- 172.21.15.22:0/3687826865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413abc80 0x55f6413ba550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:16:14.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.818+0000 7fb0e67fc640 1 -- 172.21.15.22:0/3687826865 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8003b70 con 0x55f6413ab360 2024-09-19T08:16:14.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.818+0000 7fb0fd7fa640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb0e80a3560 0x7fb0e80a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.818+0000 7fb0e67fc640 1 -- 172.21.15.22:0/3687826865 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f8036040 con 0x55f6413ab360 2024-09-19T08:16:14.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.818+0000 7fb0e67fc640 1 -- 172.21.15.22:0/3687826865 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80045d0 con 0x55f6413ab360 2024-09-19T08:16:14.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.818+0000 7fb0fffff640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb0e80a3560 0x7fb0e80a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.818+0000 7fb0fd7fa640 1 -- 172.21.15.22:0/571325448 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb0f00d02c0 con 0x7fb100100100 2024-09-19T08:16:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.819+0000 7fb0fffff640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb0e80a3560 0x7fb0e80a5a20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00cd840 tx=0x7fb0f0000bc0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.819+0000 7fb1086a3740 1 -- 172.21.15.22:0/3687826865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413ab360 msgr2=0x55f6413ab740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.819+0000 7fb1086a3740 1 --2- 172.21.15.22:0/3687826865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413ab360 0x55f6413ab740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009bf0 tx=0x7fb0f8009800 comp rx=0 tx=0).stop 2024-09-19T08:16:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.819+0000 7fb1086a3740 1 -- 172.21.15.22:0/3687826865 shutdown_connections 2024-09-19T08:16:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.819+0000 7fb1086a3740 1 --2- 172.21.15.22:0/3687826865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413baa90 0x55f6413bce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.819+0000 7fb1086a3740 1 --2- 172.21.15.22:0/3687826865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413abc80 0x55f6413ba550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.819+0000 7fb1086a3740 1 --2- 172.21.15.22:0/3687826865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413ab360 0x55f6413ab740 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.819+0000 7fb1086a3740 1 -- 172.21.15.22:0/3687826865 >> 172.21.15.22:0/3687826865 conn(0x55f6413a7a30 msgr2=0x55f6413a7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.819+0000 7fb1086a3740 1 -- 172.21.15.22:0/3687826865 shutdown_connections 2024-09-19T08:16:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.819+0000 7fb1086a3740 1 -- 172.21.15.22:0/3687826865 wait complete. 2024-09-19T08:16:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.820+0000 7fb1086a3740 1 -- start start 2024-09-19T08:16:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.820+0000 7fb1086a3740 1 Processor -- start 2024-09-19T08:16:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.820+0000 7fb1086a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413ab360 0x55f6413ba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.820+0000 7fb1086a3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413abc80 0x55f6413b7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.820+0000 7fb1086a3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413baa90 0x55f6413b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.820+0000 7fb1086a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f6413bd480 con 0x55f6413ab360 2024-09-19T08:16:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.820+0000 7fb1086a3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f6412cc050 con 0x55f6413abc80 2024-09-19T08:16:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.820+0000 7fb1086a3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f6413bd600 con 0x55f6413baa90 2024-09-19T08:16:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.820+0000 7fb1086a3740 10 client.0 _ll_register_callbacks cb 0x55f64130c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.820+0000 7fb104867640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413ab360 0x55f6413ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.821+0000 7fb104867640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413ab360 0x55f6413ba450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33186/0 (socket says 172.21.15.22:33186) 2024-09-19T08:16:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.821+0000 7fb104867640 1 -- 172.21.15.22:0/512361797 learned_addr learned my addr 172.21.15.22:0/512361797 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.821+0000 7fb0ff7fe640 1 --2- 172.21.15.22:0/512361797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413abc80 0x55f6413b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:14 smithi022 ceph-mon[30963]: scrub queued for path: / 2024-09-19T08:16:14.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:14 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:16:14.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:14 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:16:14.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:14 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:16:14.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:14 smithi022 ceph-mon[30963]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T08:16:14.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.821+0000 7fb104867640 1 -- 172.21.15.22:0/512361797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413baa90 msgr2=0x55f6413b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:14.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.821+0000 7fb104867640 1 --2- 172.21.15.22:0/512361797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413baa90 0x55f6413b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.821+0000 7fb104867640 1 -- 172.21.15.22:0/512361797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413abc80 msgr2=0x55f6413b7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:14.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.821+0000 7fb104867640 1 --2- 172.21.15.22:0/512361797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413abc80 0x55f6413b7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.821+0000 7fb104867640 1 -- 172.21.15.22:0/512361797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6410823c0 con 0x55f6413ab360 2024-09-19T08:16:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.821+0000 7fb0ff7fe640 1 --2- 172.21.15.22:0/512361797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413abc80 0x55f6413b7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.821+0000 7fb104867640 1 --2- 172.21.15.22:0/512361797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413ab360 0x55f6413ba450 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb0f802ade0 tx=0x7fb0f8005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f497b05353177faa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.821+0000 7fb0df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.821+0000 7fb0df7fe640 1 -- 172.21.15.22:0/512361797 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80317a0 con 0x55f6413ab360 2024-09-19T08:16:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.822+0000 7fb0df7fe640 1 -- 172.21.15.22:0/512361797 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb0f8031940 con 0x55f6413ab360 2024-09-19T08:16:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.822+0000 7fb1086a3740 1 -- 172.21.15.22:0/512361797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0f8036040 con 0x55f6413ab360 2024-09-19T08:16:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.822+0000 7fb0df7fe640 1 -- 172.21.15.22:0/512361797 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb0f8003970 con 0x55f6413ab360 2024-09-19T08:16:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.822+0000 7fb1086a3740 10 client.15693 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.822+0000 7fb1086a3740 1 -- 172.21.15.22:0/512361797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f6413bd480 con 0x55f6413ab360 2024-09-19T08:16:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.822+0000 7fb0df7fe640 1 -- 172.21.15.22:0/512361797 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f8036040 con 0x55f6413ab360 2024-09-19T08:16:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.823+0000 7fb0df7fe640 1 -- 172.21.15.22:0/512361797 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb0f8057030 con 0x55f6413ab360 2024-09-19T08:16:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.823+0000 7fb1086a3740 10 client.15693 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:14.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.823+0000 7fb1086a3740 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:16:14.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.823+0000 7fb1086a3740 20 client.15693 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:14.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.823+0000 7fb1086a3740 1 --2- 172.21.15.22:0/512361797 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f6413c9fd0 0x55f6413cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.823+0000 7fb1086a3740 4 client.15693 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:16:14.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.823+0000 7fb1086a3740 1 -- 172.21.15.22:0/512361797 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f64131e850 con 0x55f6413c9fd0 2024-09-19T08:16:14.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.823+0000 7fb0ff7fe640 1 --2- 172.21.15.22:0/512361797 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f6413c9fd0 0x55f6413cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.824+0000 7fb0ff7fe640 1 --2- 172.21.15.22:0/512361797 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f6413c9fd0 0x55f6413cc3f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=866c82bbe2461220 server_cookie=9ad7e5823cdc8999 in_seq=0 out_seq=0 2024-09-19T08:16:14.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.824+0000 7fb0df7fe640 10 client.15693 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:14.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb0df7fe640 1 -- 172.21.15.22:0/512361797 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f64131e850 con 0x55f6413c9fd0 2024-09-19T08:16:14.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb0df7fe640 10 client.15693 handle_command_reply: tid=0 2024-09-19T08:16:14.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb1086a3740 1 client.15693 shutdown 2024-09-19T08:16:14.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb1086a3740 1 -- 172.21.15.22:0/512361797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413ab360 msgr2=0x55f6413ba450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:14.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb1086a3740 1 --2- 172.21.15.22:0/512361797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413ab360 0x55f6413ba450 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb0f802ade0 tx=0x7fb0f8005fa0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb1086a3740 1 -- 172.21.15.22:0/512361797 shutdown_connections 2024-09-19T08:16:14.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb1086a3740 1 --2- 172.21.15.22:0/512361797 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f6413c9fd0 0x55f6413cc3f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb1086a3740 1 --2- 172.21.15.22:0/512361797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413baa90 0x55f6413b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb1086a3740 1 --2- 172.21.15.22:0/512361797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413abc80 0x55f6413b7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb1086a3740 1 --2- 172.21.15.22:0/512361797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413ab360 0x55f6413ba450 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb1086a3740 1 -- 172.21.15.22:0/512361797 >> 172.21.15.22:0/512361797 conn(0x55f6413a7a30 msgr2=0x55f6413a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:14.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb0df7fe640 0 client.15693 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:14.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb1086a3740 1 -- 172.21.15.22:0/512361797 shutdown_connections 2024-09-19T08:16:14.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb1086a3740 1 -- 172.21.15.22:0/512361797 wait complete. 2024-09-19T08:16:14.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.825+0000 7fb1086a3740 20 client.15693 trim_cache size 0 max 16384 2024-09-19T08:16:14.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.858+0000 7fb1086a3740 1 Processor -- start 2024-09-19T08:16:14.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.858+0000 7fb1086a3740 1 -- start start 2024-09-19T08:16:14.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.858+0000 7fb1086a3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413ab360 0x55f6413c6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.858+0000 7fb1086a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413abc80 0x55f6413d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.858+0000 7fb1086a3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413baa90 0x55f6413d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.858+0000 7fb1086a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001ade20 con 0x55f6413abc80 2024-09-19T08:16:14.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.858+0000 7fb1086a3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001ab6f0 con 0x55f6413baa90 2024-09-19T08:16:14.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.858+0000 7fb1086a3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001a9540 con 0x55f6413ab360 2024-09-19T08:16:14.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.859+0000 7fb0ff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413abc80 0x55f6413d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.859+0000 7fb0ff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413abc80 0x55f6413d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33200/0 (socket says 172.21.15.22:33200) 2024-09-19T08:16:14.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.859+0000 7fb0ff7fe640 1 -- 172.21.15.22:0/3898868829 learned_addr learned my addr 172.21.15.22:0/3898868829 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:14.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.859+0000 7fb104867640 1 --2- 172.21.15.22:0/3898868829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413ab360 0x55f6413c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.859+0000 7fb104867640 1 --2- 172.21.15.22:0/3898868829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413baa90 0x55f6413d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.859+0000 7fb0ff7fe640 1 -- 172.21.15.22:0/3898868829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413ab360 msgr2=0x55f6413c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:14.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.859+0000 7fb0ff7fe640 1 --2- 172.21.15.22:0/3898868829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413ab360 0x55f6413c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.859+0000 7fb0ff7fe640 1 -- 172.21.15.22:0/3898868829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413baa90 msgr2=0x55f6413d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:14.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.859+0000 7fb0ff7fe640 1 --2- 172.21.15.22:0/3898868829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413baa90 0x55f6413d21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.859+0000 7fb0ff7fe640 1 -- 172.21.15.22:0/3898868829 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6413abab0 con 0x55f6413abc80 2024-09-19T08:16:14.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.859+0000 7fb0ff7fe640 1 --2- 172.21.15.22:0/3898868829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413abc80 0x55f6413d1c80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x55f6413a8b20 tx=0x7fb0f400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be05baa55c2d7472 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:14.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.860+0000 7fb0e5ffb640 1 -- 172.21.15.22:0/3898868829 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4005220 con 0x55f6413abc80 2024-09-19T08:16:14.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.860+0000 7fb0e5ffb640 1 -- 172.21.15.22:0/3898868829 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f4004c80 con 0x55f6413abc80 2024-09-19T08:16:14.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.860+0000 7fb0e5ffb640 1 -- 172.21.15.22:0/3898868829 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4026860 con 0x55f6413abc80 2024-09-19T08:16:14.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.861+0000 7fb1086a3740 1 -- 172.21.15.22:0/3898868829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413abc80 msgr2=0x55f6413d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:14.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.861+0000 7fb1086a3740 1 --2- 172.21.15.22:0/3898868829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413abc80 0x55f6413d1c80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x55f6413a8b20 tx=0x7fb0f400b4a0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.861+0000 7fb1086a3740 1 -- 172.21.15.22:0/3898868829 shutdown_connections 2024-09-19T08:16:14.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.861+0000 7fb1086a3740 1 --2- 172.21.15.22:0/3898868829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413baa90 0x55f6413d21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.861+0000 7fb1086a3740 1 --2- 172.21.15.22:0/3898868829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413abc80 0x55f6413d1c80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.861+0000 7fb1086a3740 1 --2- 172.21.15.22:0/3898868829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413ab360 0x55f6413c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.861+0000 7fb1086a3740 1 -- 172.21.15.22:0/3898868829 >> 172.21.15.22:0/3898868829 conn(0x55f6413a7a30 msgr2=0x55f6413b67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:14.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.861+0000 7fb1086a3740 1 -- 172.21.15.22:0/3898868829 shutdown_connections 2024-09-19T08:16:14.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.861+0000 7fb1086a3740 1 -- 172.21.15.22:0/3898868829 wait complete. 2024-09-19T08:16:14.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.861+0000 7fb1086a3740 1 -- start start 2024-09-19T08:16:14.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.862+0000 7fb1086a3740 1 Processor -- start 2024-09-19T08:16:14.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.862+0000 7fb1086a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413ab360 0x55f6413bdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.862+0000 7fb1086a3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413abc80 0x55f6413be510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.862+0000 7fb1086a3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413baa90 0x55f641432c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.862+0000 7fb1086a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001a9bb0 con 0x55f6413ab360 2024-09-19T08:16:14.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.862+0000 7fb1086a3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001ab020 con 0x55f6413baa90 2024-09-19T08:16:14.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.862+0000 7fb1086a3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001aa280 con 0x55f6413abc80 2024-09-19T08:16:14.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.862+0000 7fb1086a3740 10 client.0 _ll_register_callbacks cb 0x7fb0f400bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:14.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.862+0000 7fb104867640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413ab360 0x55f6413bdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.862+0000 7fb104867640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413ab360 0x55f6413bdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33206/0 (socket says 172.21.15.22:33206) 2024-09-19T08:16:14.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.862+0000 7fb104867640 1 -- 172.21.15.22:0/1000514663 learned_addr learned my addr 172.21.15.22:0/1000514663 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:14.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.862+0000 7fb0ff7fe640 1 --2- 172.21.15.22:0/1000514663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413abc80 0x55f6413be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.863+0000 7fb104867640 1 --2- 172.21.15.22:0/1000514663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413baa90 0x55f641432c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.863+0000 7fb0ff7fe640 1 -- 172.21.15.22:0/1000514663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413baa90 msgr2=0x55f641432c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:14.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.863+0000 7fb0ff7fe640 1 --2- 172.21.15.22:0/1000514663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413baa90 0x55f641432c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.863+0000 7fb0ff7fe640 1 -- 172.21.15.22:0/1000514663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413ab360 msgr2=0x55f6413bdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:14.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.863+0000 7fb0ff7fe640 1 --2- 172.21.15.22:0/1000514663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413ab360 0x55f6413bdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.863+0000 7fb0ff7fe640 1 -- 172.21.15.22:0/1000514663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f641433330 con 0x55f6413abc80 2024-09-19T08:16:14.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.863+0000 7fb0ff7fe640 1 --2- 172.21.15.22:0/1000514663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413abc80 0x55f6413be510 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55f6413a3580 tx=0x7fb0f4004ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8de882630028dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:14.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.863+0000 7fb0df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:16:14.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.863+0000 7fb0df7fe640 1 -- 172.21.15.22:0/1000514663 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4034060 con 0x55f6413abc80 2024-09-19T08:16:14.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.863+0000 7fb0df7fe640 1 -- 172.21.15.22:0/1000514663 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb0f4004730 con 0x55f6413abc80 2024-09-19T08:16:14.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.863+0000 7fb1086a3740 1 -- 172.21.15.22:0/1000514663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0f4004c80 con 0x55f6413abc80 2024-09-19T08:16:14.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.864+0000 7fb0df7fe640 1 -- 172.21.15.22:0/1000514663 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb0f4004930 con 0x55f6413abc80 2024-09-19T08:16:14.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.864+0000 7fb1086a3740 10 client.25291 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:14.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.864+0000 7fb1086a3740 1 -- 172.21.15.22:0/1000514663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f641434620 con 0x55f6413abc80 2024-09-19T08:16:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.864+0000 7fb0df7fe640 1 -- 172.21.15.22:0/1000514663 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f4004c80 con 0x55f6413abc80 2024-09-19T08:16:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.864+0000 7fb0df7fe640 1 -- 172.21.15.22:0/1000514663 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb0f4026c40 con 0x55f6413abc80 2024-09-19T08:16:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.864+0000 7fb1086a3740 10 client.25291 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.864+0000 7fb1086a3740 10 client.25291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:16:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.864+0000 7fb1086a3740 20 client.25291 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.864+0000 7fb1086a3740 1 --2- 172.21.15.22:0/1000514663 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f641434bf0 0x55f641437010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.864+0000 7fb1086a3740 4 client.25291 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:16:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.864+0000 7fb1086a3740 1 -- 172.21.15.22:0/1000514663 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f641318d60 con 0x55f641434bf0 2024-09-19T08:16:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.865+0000 7fb104867640 1 --2- 172.21.15.22:0/1000514663 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f641434bf0 0x55f641437010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.866+0000 7fb104867640 1 --2- 172.21.15.22:0/1000514663 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f641434bf0 0x55f641437010 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b5241c2fda1b47 server_cookie=1701b47d85dabe07 in_seq=0 out_seq=0 2024-09-19T08:16:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.866+0000 7fb0df7fe640 10 client.25291 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.866+0000 7fb0df7fe640 1 -- 172.21.15.22:0/1000514663 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f641318d60 con 0x55f641434bf0 2024-09-19T08:16:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.866+0000 7fb0df7fe640 10 client.25291 handle_command_reply: tid=0 2024-09-19T08:16:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.866+0000 7fb1086a3740 1 client.25291 shutdown 2024-09-19T08:16:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.866+0000 7fb1086a3740 1 -- 172.21.15.22:0/1000514663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413abc80 msgr2=0x55f6413be510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.866+0000 7fb1086a3740 1 --2- 172.21.15.22:0/1000514663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413abc80 0x55f6413be510 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55f6413a3580 tx=0x7fb0f4004ea0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.866+0000 7fb1086a3740 1 -- 172.21.15.22:0/1000514663 shutdown_connections 2024-09-19T08:16:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.866+0000 7fb1086a3740 1 --2- 172.21.15.22:0/1000514663 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f641434bf0 0x55f641437010 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.866+0000 7fb1086a3740 1 --2- 172.21.15.22:0/1000514663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6413baa90 0x55f641432c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.866+0000 7fb1086a3740 1 --2- 172.21.15.22:0/1000514663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f6413abc80 0x55f6413be510 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.866+0000 7fb1086a3740 1 --2- 172.21.15.22:0/1000514663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f6413ab360 0x55f6413bdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.866+0000 7fb1086a3740 1 -- 172.21.15.22:0/1000514663 >> 172.21.15.22:0/1000514663 conn(0x55f6413a7a30 msgr2=0x55f6413b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:14.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.866+0000 7fb0df7fe640 0 client.25291 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:14.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.867+0000 7fb1086a3740 1 -- 172.21.15.22:0/1000514663 shutdown_connections 2024-09-19T08:16:14.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.867+0000 7fb1086a3740 1 -- 172.21.15.22:0/1000514663 wait complete. 2024-09-19T08:16:14.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.867+0000 7fb1086a3740 20 client.25291 trim_cache size 0 max 16384 2024-09-19T08:16:14.872 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:16:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:16:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-19T08:16:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: "3ef1d6f2-994a-4c75-a231-78b37b1662d9": { 2024-09-19T08:16:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-19T08:16:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "3ef1d6f2-994a-4c75-a231-78b37b1662d9", 2024-09-19T08:16:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-09-19T08:16:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:14.873 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:16:14.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.867+0000 7fb1062ca640 1 -- 172.21.15.22:0/571325448 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb0e80a3560 msgr2=0x7fb0e80a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:14.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.867+0000 7fb1062ca640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb0e80a3560 0x7fb0e80a5a20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00cd840 tx=0x7fb0f0000bc0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.867+0000 7fb1062ca640 1 -- 172.21.15.22:0/571325448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100100100 msgr2=0x7fb100198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:14.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.867+0000 7fb1062ca640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100100100 0x7fb100198430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000e9f0 tx=0x7fb0f000eeb0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.868+0000 7fb1062ca640 1 -- 172.21.15.22:0/571325448 shutdown_connections 2024-09-19T08:16:14.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.868+0000 7fb1062ca640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb0e80a3560 0x7fb0e80a5a20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.868+0000 7fb1062ca640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb100106b80 0x7fb10019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.868+0000 7fb1062ca640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb100100aa0 0x7fb100198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.868+0000 7fb1062ca640 1 --2- 172.21.15.22:0/571325448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100100100 0x7fb100198430 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:14.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.868+0000 7fb1062ca640 1 -- 172.21.15.22:0/571325448 >> 172.21.15.22:0/571325448 conn(0x7fb100077230 msgr2=0x7fb1000fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:14.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.868+0000 7fb1062ca640 1 -- 172.21.15.22:0/571325448 shutdown_connections 2024-09-19T08:16:14.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:14.868+0000 7fb1062ca640 1 -- 172.21.15.22:0/571325448 wait complete. 2024-09-19T08:16:15.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:14 smithi064 ceph-mon[34900]: scrub queued for path: / 2024-09-19T08:16:15.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:14 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:16:15.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:14 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:16:15.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:14 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:16:15.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:14 smithi064 ceph-mon[34900]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T08:16:15.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:14 smithi157 ceph-mon[35143]: scrub queued for path: / 2024-09-19T08:16:15.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:14 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:16:15.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:14 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:16:15.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:14 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:16:15.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:14 smithi157 ceph-mon[35143]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T08:16:15.699 INFO:tasks.cephfs.filesystem:scrub status for tag:3ef1d6f2-994a-4c75-a231-78b37b1662d9 - {'path': '/', 'tag': '3ef1d6f2-994a-4c75-a231-78b37b1662d9', 'options': 'recursive,force'} 2024-09-19T08:16:17.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:16 smithi064 ceph-mon[34900]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-19T08:16:17.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:16 smithi157 ceph-mon[35143]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-19T08:16:17.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:16 smithi022 ceph-mon[30963]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-19T08:16:18.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:17 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:16:18.154 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:17 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:16:18.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:17 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:16:19.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:18 smithi064 ceph-mon[34900]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-19T08:16:19.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:18 smithi157 ceph-mon[35143]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-19T08:16:19.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:18 smithi022 ceph-mon[30963]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-19T08:16:21.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:20 smithi064 ceph-mon[34900]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:16:21.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:20 smithi157 ceph-mon[35143]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:16:21.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:20 smithi022 ceph-mon[30963]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:16:22.512 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T08:16:22.512 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T08:16:22.513 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:16:22.784 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:22.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:22 smithi022 ceph-mon[30963]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:16:23.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:22 smithi064 ceph-mon[34900]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:16:23.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:22 smithi157 ceph-mon[35143]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:16:23.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.282+0000 7ff866dbc640 1 -- 172.21.15.22:0/1106355586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff860100e20 msgr2=0x7ff860101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:23.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.282+0000 7ff866dbc640 1 --2- 172.21.15.22:0/1106355586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff860100e20 0x7ff860101200 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff848009900 tx=0x7ff84802a480 comp rx=0 tx=0).stop 2024-09-19T08:16:23.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.282+0000 7ff866dbc640 1 -- 172.21.15.22:0/1106355586 shutdown_connections 2024-09-19T08:16:23.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.282+0000 7ff866dbc640 1 --2- 172.21.15.22:0/1106355586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff860102ef0 0x7ff86010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:23.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.282+0000 7ff866dbc640 1 --2- 172.21.15.22:0/1106355586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff860102570 0x7ff8601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:23.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.282+0000 7ff866dbc640 1 --2- 172.21.15.22:0/1106355586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff860100e20 0x7ff860101200 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:23.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.282+0000 7ff866dbc640 1 -- 172.21.15.22:0/1106355586 >> 172.21.15.22:0/1106355586 conn(0x7ff8600fc7d0 msgr2=0x7ff8600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:23.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.282+0000 7ff866dbc640 1 -- 172.21.15.22:0/1106355586 shutdown_connections 2024-09-19T08:16:23.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.282+0000 7ff866dbc640 1 -- 172.21.15.22:0/1106355586 wait complete. 2024-09-19T08:16:23.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.283+0000 7ff866dbc640 1 Processor -- start 2024-09-19T08:16:23.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.283+0000 7ff866dbc640 1 -- start start 2024-09-19T08:16:23.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.283+0000 7ff866dbc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff860100e20 0x7ff86019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:23.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.283+0000 7ff866dbc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff860102570 0x7ff86019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:23.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.283+0000 7ff866dbc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff860102ef0 0x7ff86019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:23.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.283+0000 7ff866dbc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff860110f80 con 0x7ff860100e20 2024-09-19T08:16:23.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.283+0000 7ff866dbc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff860110e00 con 0x7ff860102ef0 2024-09-19T08:16:23.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.283+0000 7ff866dbc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff860111100 con 0x7ff860102570 2024-09-19T08:16:23.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.283+0000 7ff864b58640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff860100e20 0x7ff86019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:23.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.283+0000 7ff864b58640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff860100e20 0x7ff86019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32990/0 (socket says 172.21.15.22:32990) 2024-09-19T08:16:23.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.283+0000 7ff864b58640 1 -- 172.21.15.22:0/2719142659 learned_addr learned my addr 172.21.15.22:0/2719142659 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:23.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.284+0000 7ff865359640 1 --2- 172.21.15.22:0/2719142659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff860102ef0 0x7ff86019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:23.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.284+0000 7ff85ffff640 1 --2- 172.21.15.22:0/2719142659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff860102570 0x7ff86019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:23.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.284+0000 7ff864b58640 1 -- 172.21.15.22:0/2719142659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff860102570 msgr2=0x7ff86019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:23.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.284+0000 7ff864b58640 1 --2- 172.21.15.22:0/2719142659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff860102570 0x7ff86019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:23.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.284+0000 7ff864b58640 1 -- 172.21.15.22:0/2719142659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff860102ef0 msgr2=0x7ff86019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:23.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.284+0000 7ff864b58640 1 --2- 172.21.15.22:0/2719142659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff860102ef0 0x7ff86019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:23.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.284+0000 7ff864b58640 1 -- 172.21.15.22:0/2719142659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86019f6a0 con 0x7ff860100e20 2024-09-19T08:16:23.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.284+0000 7ff864b58640 1 --2- 172.21.15.22:0/2719142659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff860100e20 0x7ff86019a6f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff848006010 tx=0x7ff848003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:23.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.284+0000 7ff85dffb640 1 -- 172.21.15.22:0/2719142659 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84803f070 con 0x7ff860100e20 2024-09-19T08:16:23.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.284+0000 7ff866dbc640 1 -- 172.21.15.22:0/2719142659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86019f930 con 0x7ff860100e20 2024-09-19T08:16:23.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.284+0000 7ff85dffb640 1 -- 172.21.15.22:0/2719142659 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff848031c30 con 0x7ff860100e20 2024-09-19T08:16:23.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.284+0000 7ff85dffb640 1 -- 172.21.15.22:0/2719142659 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848035030 con 0x7ff860100e20 2024-09-19T08:16:23.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.284+0000 7ff866dbc640 1 -- 172.21.15.22:0/2719142659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8601a7270 con 0x7ff860100e20 2024-09-19T08:16:23.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.285+0000 7ff866dbc640 1 -- 172.21.15.22:0/2719142659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff860101200 con 0x7ff860100e20 2024-09-19T08:16:23.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.287+0000 7ff85dffb640 1 -- 172.21.15.22:0/2719142659 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7ff848042070 con 0x7ff860100e20 2024-09-19T08:16:23.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.287+0000 7ff85dffb640 1 --2- 172.21.15.22:0/2719142659 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff84c0a3560 0x7ff84c0a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:23.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.288+0000 7ff85ffff640 1 --2- 172.21.15.22:0/2719142659 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff84c0a3560 0x7ff84c0a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:23.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.288+0000 7ff85dffb640 1 -- 172.21.15.22:0/2719142659 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff8480f0bb0 con 0x7ff860100e20 2024-09-19T08:16:23.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.288+0000 7ff85ffff640 1 --2- 172.21.15.22:0/2719142659 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff84c0a3560 0x7ff84c0a5a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff86010de80 tx=0x7ff850009870 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:23.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.291+0000 7ff85dffb640 1 -- 172.21.15.22:0/2719142659 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff8480bd360 con 0x7ff860100e20 2024-09-19T08:16:23.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.503+0000 7ff866dbc640 1 -- 172.21.15.22:0/2719142659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8601a7440 con 0x7ff860100e20 2024-09-19T08:16:23.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.504+0000 7ff85dffb640 1 -- 172.21.15.22:0/2719142659 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11906 (secure 0 0 0) 0x7ff8480bcc60 con 0x7ff860100e20 2024-09-19T08:16:23.504 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:16:23.504 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"btime":"2024-09-19T08:15:00:109390+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:15:00.059208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:16:23.505 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-09-19T08:16:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.506+0000 7ff866dbc640 1 -- 172.21.15.22:0/2719142659 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff84c0a3560 msgr2=0x7ff84c0a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.506+0000 7ff866dbc640 1 --2- 172.21.15.22:0/2719142659 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff84c0a3560 0x7ff84c0a5a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff86010de80 tx=0x7ff850009870 comp rx=0 tx=0).stop 2024-09-19T08:16:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.506+0000 7ff866dbc640 1 -- 172.21.15.22:0/2719142659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff860100e20 msgr2=0x7ff86019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.506+0000 7ff866dbc640 1 --2- 172.21.15.22:0/2719142659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff860100e20 0x7ff86019a6f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff848006010 tx=0x7ff848003c10 comp rx=0 tx=0).stop 2024-09-19T08:16:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.506+0000 7ff866dbc640 1 -- 172.21.15.22:0/2719142659 shutdown_connections 2024-09-19T08:16:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.506+0000 7ff866dbc640 1 --2- 172.21.15.22:0/2719142659 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff84c0a3560 0x7ff84c0a5a20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.506+0000 7ff866dbc640 1 --2- 172.21.15.22:0/2719142659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff860102ef0 0x7ff86019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.506+0000 7ff866dbc640 1 --2- 172.21.15.22:0/2719142659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff860102570 0x7ff86019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.506+0000 7ff866dbc640 1 --2- 172.21.15.22:0/2719142659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff860100e20 0x7ff86019a6f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:23.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.506+0000 7ff866dbc640 1 -- 172.21.15.22:0/2719142659 >> 172.21.15.22:0/2719142659 conn(0x7ff8600fc7d0 msgr2=0x7ff86006a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:23.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.506+0000 7ff866dbc640 1 -- 172.21.15.22:0/2719142659 shutdown_connections 2024-09-19T08:16:23.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:23.506+0000 7ff866dbc640 1 -- 172.21.15.22:0/2719142659 wait complete. 2024-09-19T08:16:24.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:23 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2719142659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:24.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:23 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2719142659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:24.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:23 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2719142659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:24.497 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T08:16:24.497 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T08:16:24.758 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:25.084 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:24 smithi022 ceph-mon[30963]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-19T08:16:25.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:24 smithi064 ceph-mon[34900]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-19T08:16:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:24 smithi157 ceph-mon[35143]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-19T08:16:25.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.318+0000 7fbd663f6640 1 -- 172.21.15.22:0/34032355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd60102570 msgr2=0x7fbd601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:25.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.318+0000 7fbd663f6640 1 --2- 172.21.15.22:0/34032355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd60102570 0x7fbd601029b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009900 tx=0x7fbd4802a480 comp rx=0 tx=0).stop 2024-09-19T08:16:25.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.318+0000 7fbd663f6640 1 -- 172.21.15.22:0/34032355 shutdown_connections 2024-09-19T08:16:25.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.318+0000 7fbd663f6640 1 --2- 172.21.15.22:0/34032355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd60102ef0 0x7fbd6010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.318+0000 7fbd663f6640 1 --2- 172.21.15.22:0/34032355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd60102570 0x7fbd601029b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.318+0000 7fbd663f6640 1 --2- 172.21.15.22:0/34032355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd60100e20 0x7fbd60101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.318+0000 7fbd663f6640 1 -- 172.21.15.22:0/34032355 >> 172.21.15.22:0/34032355 conn(0x7fbd600fc7d0 msgr2=0x7fbd600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:25.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.318+0000 7fbd663f6640 1 -- 172.21.15.22:0/34032355 shutdown_connections 2024-09-19T08:16:25.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.318+0000 7fbd663f6640 1 -- 172.21.15.22:0/34032355 wait complete. 2024-09-19T08:16:25.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.319+0000 7fbd663f6640 1 Processor -- start 2024-09-19T08:16:25.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.319+0000 7fbd663f6640 1 -- start start 2024-09-19T08:16:25.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.319+0000 7fbd663f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd60100e20 0x7fbd60199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.319+0000 7fbd663f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd60102570 0x7fbd601997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.319+0000 7fbd663f6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd60102ef0 0x7fbd601a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.319+0000 7fbd663f6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbd60110f80 con 0x7fbd60100e20 2024-09-19T08:16:25.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.319+0000 7fbd663f6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbd60110e00 con 0x7fbd60102570 2024-09-19T08:16:25.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.319+0000 7fbd663f6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd60111100 con 0x7fbd60102ef0 2024-09-19T08:16:25.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.319+0000 7fbd5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd60100e20 0x7fbd60199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.319+0000 7fbd5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd60100e20 0x7fbd60199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33006/0 (socket says 172.21.15.22:33006) 2024-09-19T08:16:25.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.319+0000 7fbd5ffff640 1 -- 172.21.15.22:0/115148314 learned_addr learned my addr 172.21.15.22:0/115148314 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:25.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd57fff640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd60102570 0x7fbd601997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd64993640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd60102ef0 0x7fbd601a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd5ffff640 1 -- 172.21.15.22:0/115148314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd60102ef0 msgr2=0x7fbd601a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:25.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd5ffff640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd60102ef0 0x7fbd601a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd5ffff640 1 -- 172.21.15.22:0/115148314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd60102570 msgr2=0x7fbd601997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:25.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd5ffff640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd60102570 0x7fbd601997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd5ffff640 1 -- 172.21.15.22:0/115148314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd601a4850 con 0x7fbd60100e20 2024-09-19T08:16:25.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd64993640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd60102ef0 0x7fbd601a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:25.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd57fff640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd60102570 0x7fbd601997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:25.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd5ffff640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd60100e20 0x7fbd60199290 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000d950 tx=0x7fbd5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:25.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd5dffb640 1 -- 172.21.15.22:0/115148314 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50017070 con 0x7fbd60100e20 2024-09-19T08:16:25.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd5dffb640 1 -- 172.21.15.22:0/115148314 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd50009c30 con 0x7fbd60100e20 2024-09-19T08:16:25.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd5dffb640 1 -- 172.21.15.22:0/115148314 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5000f6b0 con 0x7fbd60100e20 2024-09-19T08:16:25.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd663f6640 1 -- 172.21.15.22:0/115148314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd601a4a20 con 0x7fbd60100e20 2024-09-19T08:16:25.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.320+0000 7fbd663f6640 1 -- 172.21.15.22:0/115148314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd601a4f60 con 0x7fbd60100e20 2024-09-19T08:16:25.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd5dffb640 1 -- 172.21.15.22:0/115148314 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fbd500109d0 con 0x7fbd60100e20 2024-09-19T08:16:25.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd687cf740 1 Processor -- start 2024-09-19T08:16:25.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd687cf740 1 -- start start 2024-09-19T08:16:25.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd687cf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc7c60 0x558928cc8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd687cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc8580 0x558928cd2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd687cf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cd2e40 0x558928cd5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd687cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558928c28160 con 0x558928cc8580 2024-09-19T08:16:25.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd687cf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558928bcf720 con 0x558928cd2e40 2024-09-19T08:16:25.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd687cf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558928c24120 con 0x558928cc7c60 2024-09-19T08:16:25.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc8580 0x558928cd2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc8580 0x558928cd2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33022/0 (socket says 172.21.15.22:33022) 2024-09-19T08:16:25.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd5ffff640 1 -- 172.21.15.22:0/2215297321 learned_addr learned my addr 172.21.15.22:0/2215297321 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:25.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd64993640 1 --2- 172.21.15.22:0/2215297321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc7c60 0x558928cc8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd57fff640 1 --2- 172.21.15.22:0/2215297321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cd2e40 0x558928cd5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.323+0000 7fbd5dffb640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbd4c0a35e0 0x7fbd4c0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.324+0000 7fbd5ffff640 1 -- 172.21.15.22:0/2215297321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc7c60 msgr2=0x558928cc8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:25.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.324+0000 7fbd5ffff640 1 --2- 172.21.15.22:0/2215297321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc7c60 0x558928cc8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.324+0000 7fbd5ffff640 1 -- 172.21.15.22:0/2215297321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cd2e40 msgr2=0x558928cd5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:25.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.324+0000 7fbd5ffff640 1 --2- 172.21.15.22:0/2215297321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cd2e40 0x558928cd5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.324+0000 7fbd5ffff640 1 -- 172.21.15.22:0/2215297321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558928999ba0 con 0x558928cc8580 2024-09-19T08:16:25.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.324+0000 7fbd57fff640 1 --2- 172.21.15.22:0/2215297321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cd2e40 0x558928cd5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:25.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.324+0000 7fbd64993640 1 --2- 172.21.15.22:0/2215297321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc7c60 0x558928cc8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:25.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.324+0000 7fbd64993640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbd4c0a35e0 0x7fbd4c0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.324+0000 7fbd5dffb640 1 -- 172.21.15.22:0/115148314 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fbd500cfc80 con 0x7fbd60100e20 2024-09-19T08:16:25.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.324+0000 7fbd5ffff640 1 --2- 172.21.15.22:0/2215297321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc8580 0x558928cd2900 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbd500cd140 tx=0x7fbd50010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51f1f7cc81fdeb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:25.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.324+0000 7fbd567fc640 1 -- 172.21.15.22:0/2215297321 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50020070 con 0x558928cc8580 2024-09-19T08:16:25.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.324+0000 7fbd64993640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbd4c0a35e0 0x7fbd4c0a5aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbd58007f50 tx=0x7fbd58004a40 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:25.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.324+0000 7fbd567fc640 1 -- 172.21.15.22:0/2215297321 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd50009de0 con 0x558928cc8580 2024-09-19T08:16:25.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.324+0000 7fbd567fc640 1 -- 172.21.15.22:0/2215297321 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5009bd30 con 0x558928cc8580 2024-09-19T08:16:25.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.325+0000 7fbd687cf740 1 -- 172.21.15.22:0/2215297321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc8580 msgr2=0x558928cd2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:25.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.325+0000 7fbd687cf740 1 --2- 172.21.15.22:0/2215297321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc8580 0x558928cd2900 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbd500cd140 tx=0x7fbd50010d70 comp rx=0 tx=0).stop 2024-09-19T08:16:25.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.325+0000 7fbd687cf740 1 -- 172.21.15.22:0/2215297321 shutdown_connections 2024-09-19T08:16:25.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.325+0000 7fbd687cf740 1 --2- 172.21.15.22:0/2215297321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cd2e40 0x558928cd5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.325+0000 7fbd687cf740 1 --2- 172.21.15.22:0/2215297321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc8580 0x558928cd2900 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.325+0000 7fbd687cf740 1 --2- 172.21.15.22:0/2215297321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc7c60 0x558928cc8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.325+0000 7fbd687cf740 1 -- 172.21.15.22:0/2215297321 >> 172.21.15.22:0/2215297321 conn(0x558928cc41f0 msgr2=0x558928cc45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:25.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.325+0000 7fbd687cf740 1 -- 172.21.15.22:0/2215297321 shutdown_connections 2024-09-19T08:16:25.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.325+0000 7fbd687cf740 1 -- 172.21.15.22:0/2215297321 wait complete. 2024-09-19T08:16:25.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.326+0000 7fbd687cf740 1 -- start start 2024-09-19T08:16:25.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.326+0000 7fbd687cf740 1 Processor -- start 2024-09-19T08:16:25.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.326+0000 7fbd687cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc7c60 0x558928ccfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.326+0000 7fbd687cf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc8580 0x558928cd0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.326+0000 7fbd687cf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cd2e40 0x558928cd0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.326+0000 7fbd687cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558928cd5a10 con 0x558928cc7c60 2024-09-19T08:16:25.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.326+0000 7fbd687cf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558928bb0900 con 0x558928cd2e40 2024-09-19T08:16:25.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.326+0000 7fbd687cf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558928cd5b90 con 0x558928cc8580 2024-09-19T08:16:25.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.326+0000 7fbd687cf740 10 client.0 _ll_register_callbacks cb 0x558928bae780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:25.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.326+0000 7fbd5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc7c60 0x558928ccfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.326+0000 7fbd5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc7c60 0x558928ccfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33028/0 (socket says 172.21.15.22:33028) 2024-09-19T08:16:25.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.326+0000 7fbd5ffff640 1 -- 172.21.15.22:0/984335674 learned_addr learned my addr 172.21.15.22:0/984335674 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:25.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.326+0000 7fbd64993640 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cd2e40 0x558928cd0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.326+0000 7fbd57fff640 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc8580 0x558928cd0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd5ffff640 1 -- 172.21.15.22:0/984335674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc8580 msgr2=0x558928cd0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:25.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd5ffff640 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc8580 0x558928cd0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd5ffff640 1 -- 172.21.15.22:0/984335674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cd2e40 msgr2=0x558928cd0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:25.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd5ffff640 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cd2e40 0x558928cd0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd5ffff640 1 -- 172.21.15.22:0/984335674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558928bf0c50 con 0x558928cc7c60 2024-09-19T08:16:25.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd57fff640 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc8580 0x558928cd0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:16:25.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd64993640 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cd2e40 0x558928cd0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:16:25.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd5ffff640 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc7c60 0x558928ccfd00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbd50009f90 tx=0x7fbd50009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e0ae3eb4206da9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:25.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:25.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd3f7fe640 1 -- 172.21.15.22:0/984335674 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50020040 con 0x558928cc7c60 2024-09-19T08:16:25.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd3f7fe640 1 -- 172.21.15.22:0/984335674 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fbd5009a030 con 0x558928cc7c60 2024-09-19T08:16:25.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd687cf740 1 -- 172.21.15.22:0/984335674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd50009de0 con 0x558928cc7c60 2024-09-19T08:16:25.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd3f7fe640 1 -- 172.21.15.22:0/984335674 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbd5009b690 con 0x558928cc7c60 2024-09-19T08:16:25.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd687cf740 10 client.15717 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:25.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd687cf740 1 -- 172.21.15.22:0/984335674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558928ce1690 con 0x558928cc7c60 2024-09-19T08:16:25.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.327+0000 7fbd3f7fe640 1 -- 172.21.15.22:0/984335674 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd50009de0 con 0x558928cc7c60 2024-09-19T08:16:25.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.328+0000 7fbd3f7fe640 1 -- 172.21.15.22:0/984335674 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbd500d5060 con 0x558928cc7c60 2024-09-19T08:16:25.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.328+0000 7fbd687cf740 10 client.15717 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:25.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.328+0000 7fbd687cf740 10 client.15717 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:16:25.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.328+0000 7fbd687cf740 20 client.15717 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:25.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.328+0000 7fbd687cf740 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558928ce29e0 0x558928ce4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.328+0000 7fbd687cf740 4 client.15717 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:16:25.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.328+0000 7fbd687cf740 1 -- 172.21.15.22:0/984335674 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558928c35890 con 0x558928ce29e0 2024-09-19T08:16:25.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.328+0000 7fbd57fff640 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558928ce29e0 0x558928ce4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.329+0000 7fbd3f7fe640 10 client.15717 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:25.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.329+0000 7fbd57fff640 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558928ce29e0 0x558928ce4dd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e1f94713e42e5b server_cookie=7d2af1aa2815e36c in_seq=0 out_seq=0 2024-09-19T08:16:25.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.329+0000 7fbd3f7fe640 1 -- 172.21.15.22:0/984335674 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558928c35890 con 0x558928ce29e0 2024-09-19T08:16:25.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.329+0000 7fbd3f7fe640 10 client.15717 handle_command_reply: tid=0 2024-09-19T08:16:25.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.330+0000 7fbd687cf740 1 client.15717 shutdown 2024-09-19T08:16:25.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.330+0000 7fbd687cf740 1 -- 172.21.15.22:0/984335674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc7c60 msgr2=0x558928ccfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:25.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.330+0000 7fbd687cf740 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc7c60 0x558928ccfd00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbd50009f90 tx=0x7fbd50009740 comp rx=0 tx=0).stop 2024-09-19T08:16:25.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.330+0000 7fbd687cf740 1 -- 172.21.15.22:0/984335674 shutdown_connections 2024-09-19T08:16:25.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.330+0000 7fbd687cf740 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558928ce29e0 0x558928ce4dd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.330+0000 7fbd3f7fe640 0 client.15717 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:25.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.330+0000 7fbd687cf740 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cd2e40 0x558928cd0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.330+0000 7fbd687cf740 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc8580 0x558928cd0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.330+0000 7fbd687cf740 1 --2- 172.21.15.22:0/984335674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc7c60 0x558928ccfd00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.330+0000 7fbd687cf740 1 -- 172.21.15.22:0/984335674 >> 172.21.15.22:0/984335674 conn(0x558928cc41f0 msgr2=0x558928cbeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:25.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.330+0000 7fbd687cf740 1 -- 172.21.15.22:0/984335674 shutdown_connections 2024-09-19T08:16:25.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.330+0000 7fbd687cf740 1 -- 172.21.15.22:0/984335674 wait complete. 2024-09-19T08:16:25.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.330+0000 7fbd687cf740 20 client.15717 trim_cache size 0 max 16384 2024-09-19T08:16:25.345 INFO:tasks.workunit.client.0.smithi022.stderr:Updating files: 91% (11852/12896) Updating files: 92% (11865/12896) Updating files: 93% (11994/12896) Updating files: 94% (12123/12896) Updating files: 95% (12252/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T08:16:25.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.347+0000 7fbd687cf740 1 Processor -- start 2024-09-19T08:16:25.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.347+0000 7fbd687cf740 1 -- start start 2024-09-19T08:16:25.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.347+0000 7fbd687cf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cc7c60 0x558928ccec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.347+0000 7fbd687cf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc8580 0x558928cca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.347+0000 7fbd687cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cd2e40 0x558928cea3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.347+0000 7fbd687cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbd601adfa0 con 0x558928cd2e40 2024-09-19T08:16:25.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.347+0000 7fbd687cf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbd601ab870 con 0x558928cc7c60 2024-09-19T08:16:25.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.347+0000 7fbd687cf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd601a96c0 con 0x558928cc8580 2024-09-19T08:16:25.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.348+0000 7fbd64993640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cd2e40 0x558928cea3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.348+0000 7fbd64993640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cd2e40 0x558928cea3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33038/0 (socket says 172.21.15.22:33038) 2024-09-19T08:16:25.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.348+0000 7fbd64993640 1 -- 172.21.15.22:0/1218059736 learned_addr learned my addr 172.21.15.22:0/1218059736 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:25.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.348+0000 7fbd57fff640 1 --2- 172.21.15.22:0/1218059736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc8580 0x558928cca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.348+0000 7fbd5ffff640 1 --2- 172.21.15.22:0/1218059736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cc7c60 0x558928ccec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.348+0000 7fbd64993640 1 -- 172.21.15.22:0/1218059736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc8580 msgr2=0x558928cca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:25.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.348+0000 7fbd64993640 1 --2- 172.21.15.22:0/1218059736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc8580 0x558928cca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.348+0000 7fbd64993640 1 -- 172.21.15.22:0/1218059736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cc7c60 msgr2=0x558928ccec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:25.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.348+0000 7fbd64993640 1 --2- 172.21.15.22:0/1218059736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cc7c60 0x558928ccec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.348+0000 7fbd64993640 1 -- 172.21.15.22:0/1218059736 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558928cc83b0 con 0x558928cd2e40 2024-09-19T08:16:25.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.348+0000 7fbd64993640 1 --2- 172.21.15.22:0/1218059736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cd2e40 0x558928cea3a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbd5801e760 tx=0x7fbd58013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf00d2cf221fef98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:25.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.349+0000 7fbd55ffb640 1 -- 172.21.15.22:0/1218059736 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd580129a0 con 0x558928cd2e40 2024-09-19T08:16:25.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.349+0000 7fbd55ffb640 1 -- 172.21.15.22:0/1218059736 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd58012b40 con 0x558928cd2e40 2024-09-19T08:16:25.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.349+0000 7fbd55ffb640 1 -- 172.21.15.22:0/1218059736 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5801a6b0 con 0x558928cd2e40 2024-09-19T08:16:25.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.349+0000 7fbd687cf740 1 -- 172.21.15.22:0/1218059736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cd2e40 msgr2=0x558928cea3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:25.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.349+0000 7fbd687cf740 1 --2- 172.21.15.22:0/1218059736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cd2e40 0x558928cea3a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbd5801e760 tx=0x7fbd58013000 comp rx=0 tx=0).stop 2024-09-19T08:16:25.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.349+0000 7fbd687cf740 1 -- 172.21.15.22:0/1218059736 shutdown_connections 2024-09-19T08:16:25.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.349+0000 7fbd687cf740 1 --2- 172.21.15.22:0/1218059736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cd2e40 0x558928cea3a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.349+0000 7fbd687cf740 1 --2- 172.21.15.22:0/1218059736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cc8580 0x558928cca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.349+0000 7fbd687cf740 1 --2- 172.21.15.22:0/1218059736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cc7c60 0x558928ccec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.349+0000 7fbd687cf740 1 -- 172.21.15.22:0/1218059736 >> 172.21.15.22:0/1218059736 conn(0x558928cc41f0 msgr2=0x558928cd1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:25.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.350+0000 7fbd687cf740 1 -- 172.21.15.22:0/1218059736 shutdown_connections 2024-09-19T08:16:25.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.350+0000 7fbd687cf740 1 -- 172.21.15.22:0/1218059736 wait complete. 2024-09-19T08:16:25.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.350+0000 7fbd687cf740 1 -- start start 2024-09-19T08:16:25.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.350+0000 7fbd687cf740 1 Processor -- start 2024-09-19T08:16:25.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd687cf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cc7c60 0x558928ccce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd687cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc8580 0x558928ccd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd687cf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cd2e40 0x558928ccd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd687cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbd601a9d30 con 0x558928cc8580 2024-09-19T08:16:25.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd687cf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbd601ab1a0 con 0x558928cc7c60 2024-09-19T08:16:25.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd687cf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd601aa400 con 0x558928cd2e40 2024-09-19T08:16:25.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd687cf740 10 client.0 _ll_register_callbacks cb 0x7fbd50097e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:25.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd57fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc8580 0x558928ccd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd5ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cc7c60 0x558928ccce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd57fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc8580 0x558928ccd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33054/0 (socket says 172.21.15.22:33054) 2024-09-19T08:16:25.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd57fff640 1 -- 172.21.15.22:0/2016481319 learned_addr learned my addr 172.21.15.22:0/2016481319 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:25.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd64993640 1 --2- 172.21.15.22:0/2016481319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cd2e40 0x558928ccd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd57fff640 1 -- 172.21.15.22:0/2016481319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cd2e40 msgr2=0x558928ccd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:25.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd57fff640 1 --2- 172.21.15.22:0/2016481319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cd2e40 0x558928ccd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd57fff640 1 -- 172.21.15.22:0/2016481319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cc7c60 msgr2=0x558928ccce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:25.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd57fff640 1 --2- 172.21.15.22:0/2016481319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cc7c60 0x558928ccce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd57fff640 1 -- 172.21.15.22:0/2016481319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558928d4c4a0 con 0x558928cc8580 2024-09-19T08:16:25.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd5ffff640 1 --2- 172.21.15.22:0/2016481319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cc7c60 0x558928ccce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:25.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.351+0000 7fbd64993640 1 --2- 172.21.15.22:0/2016481319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cd2e40 0x558928ccd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:25.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.352+0000 7fbd57fff640 1 --2- 172.21.15.22:0/2016481319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc8580 0x558928ccd3b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbd48005940 tx=0x7fbd480045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b1185ffcabe89d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:25.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.352+0000 7fbd3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:25.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.352+0000 7fbd3f7fe640 1 -- 172.21.15.22:0/2016481319 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48035040 con 0x558928cc8580 2024-09-19T08:16:25.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.352+0000 7fbd3f7fe640 1 -- 172.21.15.22:0/2016481319 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fbd480046c0 con 0x558928cc8580 2024-09-19T08:16:25.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.352+0000 7fbd687cf740 1 -- 172.21.15.22:0/2016481319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd58012b40 con 0x558928cc8580 2024-09-19T08:16:25.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.352+0000 7fbd3f7fe640 1 -- 172.21.15.22:0/2016481319 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbd48005610 con 0x558928cc8580 2024-09-19T08:16:25.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.352+0000 7fbd687cf740 10 client.15723 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:25.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.352+0000 7fbd687cf740 1 -- 172.21.15.22:0/2016481319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558928d4d7c0 con 0x558928cc8580 2024-09-19T08:16:25.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.352+0000 7fbd3f7fe640 1 -- 172.21.15.22:0/2016481319 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd48046070 con 0x558928cc8580 2024-09-19T08:16:25.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.353+0000 7fbd3f7fe640 1 -- 172.21.15.22:0/2016481319 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbd48003cf0 con 0x558928cc8580 2024-09-19T08:16:25.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.353+0000 7fbd687cf740 10 client.15723 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:25.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.353+0000 7fbd687cf740 10 client.15723 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:16:25.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.353+0000 7fbd687cf740 20 client.15723 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:25.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.353+0000 7fbd687cf740 1 --2- 172.21.15.22:0/2016481319 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558928ce29e0 0x558928d4fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:25.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.353+0000 7fbd687cf740 4 client.15723 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T08:16:25.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.353+0000 7fbd687cf740 1 -- 172.21.15.22:0/2016481319 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x7fbd58012b40 con 0x558928ce29e0 2024-09-19T08:16:25.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.353+0000 7fbd5ffff640 1 --2- 172.21.15.22:0/2016481319 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558928ce29e0 0x558928d4fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:25.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.353+0000 7fbd3f7fe640 10 client.15723 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:25.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.353+0000 7fbd5ffff640 1 --2- 172.21.15.22:0/2016481319 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558928ce29e0 0x558928d4fff0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e45730ea9f50493 server_cookie=7b5dd31dac292776 in_seq=0 out_seq=0 2024-09-19T08:16:25.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd3f7fe640 1 -- 172.21.15.22:0/2016481319 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7fbd58012b40 con 0x558928ce29e0 2024-09-19T08:16:25.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd3f7fe640 10 client.15723 handle_command_reply: tid=0 2024-09-19T08:16:25.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd687cf740 1 client.15723 shutdown 2024-09-19T08:16:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd687cf740 1 -- 172.21.15.22:0/2016481319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc8580 msgr2=0x558928ccd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd687cf740 1 --2- 172.21.15.22:0/2016481319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc8580 0x558928ccd3b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbd48005940 tx=0x7fbd480045a0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd687cf740 1 -- 172.21.15.22:0/2016481319 shutdown_connections 2024-09-19T08:16:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd687cf740 1 --2- 172.21.15.22:0/2016481319 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558928ce29e0 0x558928d4fff0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd687cf740 1 --2- 172.21.15.22:0/2016481319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558928cd2e40 0x558928ccd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd687cf740 1 --2- 172.21.15.22:0/2016481319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558928cc8580 0x558928ccd3b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd687cf740 1 --2- 172.21.15.22:0/2016481319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558928cc7c60 0x558928ccce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd687cf740 1 -- 172.21.15.22:0/2016481319 >> 172.21.15.22:0/2016481319 conn(0x558928cc41f0 msgr2=0x558928ccfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd3f7fe640 0 client.15723 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:25.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd687cf740 1 -- 172.21.15.22:0/2016481319 shutdown_connections 2024-09-19T08:16:25.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd687cf740 1 -- 172.21.15.22:0/2016481319 wait complete. 2024-09-19T08:16:25.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.360+0000 7fbd687cf740 20 client.15723 trim_cache size 0 max 16384 2024-09-19T08:16:25.362 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:16:25.362 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 25, 2024-09-19T08:16:25.362 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:16:25.362 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 6, 2024-09-19T08:16:25.362 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:16:25.362 INFO:teuthology.orchestra.run.smithi022.stdout: "efa99971": { 2024-09-19T08:16:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 6, 2024-09-19T08:16:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:16:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:16:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:16:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:16:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:16:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:16:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 140.3, 2024-09-19T08:16:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:16:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:16:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:16:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:16:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:16:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:16:25.364 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:25.364 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:25.364 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:25.364 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:25.364 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:25.364 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:16:25.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.361+0000 7fbd663f6640 1 -- 172.21.15.22:0/115148314 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbd4c0a35e0 msgr2=0x7fbd4c0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:25.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.361+0000 7fbd663f6640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbd4c0a35e0 0x7fbd4c0a5aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbd58007f50 tx=0x7fbd58004a40 comp rx=0 tx=0).stop 2024-09-19T08:16:25.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.361+0000 7fbd663f6640 1 -- 172.21.15.22:0/115148314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd60100e20 msgr2=0x7fbd60199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:25.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.361+0000 7fbd663f6640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd60100e20 0x7fbd60199290 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000d950 tx=0x7fbd5000de10 comp rx=0 tx=0).stop 2024-09-19T08:16:25.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.361+0000 7fbd663f6640 1 -- 172.21.15.22:0/115148314 shutdown_connections 2024-09-19T08:16:25.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.361+0000 7fbd663f6640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbd4c0a35e0 0x7fbd4c0a5aa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.361+0000 7fbd663f6640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd60102ef0 0x7fbd601a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.361+0000 7fbd663f6640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd60102570 0x7fbd601997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.361+0000 7fbd663f6640 1 --2- 172.21.15.22:0/115148314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd60100e20 0x7fbd60199290 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:25.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.361+0000 7fbd663f6640 1 -- 172.21.15.22:0/115148314 >> 172.21.15.22:0/115148314 conn(0x7fbd600fc7d0 msgr2=0x7fbd600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:25.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.361+0000 7fbd663f6640 1 -- 172.21.15.22:0/115148314 shutdown_connections 2024-09-19T08:16:25.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:25.361+0000 7fbd663f6640 1 -- 172.21.15.22:0/115148314 wait complete. 2024-09-19T08:16:25.990 INFO:tasks.workunit.client.0.smithi022.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T08:16:25.990 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-09-19T08:16:25.990 INFO:tasks.workunit.client.0.smithi022.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T08:16:25.990 INFO:tasks.workunit.client.0.smithi022.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T08:16:25.990 INFO:tasks.workunit.client.0.smithi022.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T08:16:25.990 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-09-19T08:16:25.991 INFO:tasks.workunit.client.0.smithi022.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T08:16:25.991 INFO:tasks.workunit.client.0.smithi022.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T08:16:25.991 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-09-19T08:16:25.991 INFO:tasks.workunit.client.0.smithi022.stderr: git switch -c 2024-09-19T08:16:25.991 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-09-19T08:16:25.991 INFO:tasks.workunit.client.0.smithi022.stderr:Or undo this operation with: 2024-09-19T08:16:25.991 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-09-19T08:16:25.991 INFO:tasks.workunit.client.0.smithi022.stderr: git switch - 2024-09-19T08:16:25.991 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-09-19T08:16:25.991 INFO:tasks.workunit.client.0.smithi022.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T08:16:25.991 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-09-19T08:16:25.991 INFO:tasks.workunit.client.0.smithi022.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T08:16:25.995 DEBUG:teuthology.orchestra.run.smithi022:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T08:16:26.068 INFO:tasks.workunit.client.0.smithi022.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T08:16:26.071 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T08:16:26.071 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T08:16:26.279 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'efa99971', quiesce took 3.8 seconds. Will release after: 16.55 2024-09-19T08:16:26.974 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T08:16:27.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:26 smithi022 ceph-mon[30963]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:16:27.042 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T08:16:27.084 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T08:16:27.086 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T08:16:27.086 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T08:16:27.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:26 smithi064 ceph-mon[34900]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:16:27.127 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T08:16:27.131 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:16:27.131 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T08:16:27.147 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-09-19T08:16:27.148 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-09-19T08:16:27.149 DEBUG:teuthology.orchestra.run.smithi022:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-09-19T08:16:27.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:26 smithi157 ceph-mon[35143]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T08:16:29.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:28 smithi064 ceph-mon[34900]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T08:16:29.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:28 smithi157 ceph-mon[35143]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T08:16:29.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:28 smithi022 ceph-mon[30963]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T08:16:31.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:30 smithi064 ceph-mon[34900]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:16:31.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:30 smithi157 ceph-mon[35143]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:16:31.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:30 smithi022 ceph-mon[30963]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:16:33.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:32 smithi064 ceph-mon[34900]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:16:33.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:32 smithi157 ceph-mon[35143]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:16:33.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:32 smithi022 ceph-mon[30963]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:16:35.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:34 smithi064 ceph-mon[34900]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-19T08:16:35.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:34 smithi064 ceph-mon[34900]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T08:16:35.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:34 smithi157 ceph-mon[35143]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-19T08:16:35.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:34 smithi157 ceph-mon[35143]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T08:16:35.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:34 smithi022 ceph-mon[30963]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-19T08:16:35.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:34 smithi022 ceph-mon[30963]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T08:16:37.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:36 smithi064 ceph-mon[34900]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-19T08:16:37.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:36 smithi157 ceph-mon[35143]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-19T08:16:37.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:36 smithi022 ceph-mon[30963]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-19T08:16:39.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:38 smithi064 ceph-mon[34900]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T08:16:39.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:38 smithi157 ceph-mon[35143]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T08:16:39.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:38 smithi022 ceph-mon[30963]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T08:16:41.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:40 smithi064 ceph-mon[34900]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-19T08:16:41.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:40 smithi157 ceph-mon[35143]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-19T08:16:41.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:40 smithi022 ceph-mon[30963]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-19T08:16:42.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:16:43.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:42 smithi064 ceph-mon[34900]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-19T08:16:43.103 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:43.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:42 smithi022 ceph-mon[30963]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-19T08:16:43.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:42 smithi157 ceph-mon[35143]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-19T08:16:43.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.598+0000 7fe40ebd5640 1 -- 172.21.15.22:0/2013979390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe408106a80 msgr2=0x7fe408106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:43.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.598+0000 7fe40ebd5640 1 --2- 172.21.15.22:0/2013979390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe408106a80 0x7fe408106e60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009930 tx=0x7fe3f002a420 comp rx=0 tx=0).stop 2024-09-19T08:16:43.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.598+0000 7fe40ebd5640 1 -- 172.21.15.22:0/2013979390 shutdown_connections 2024-09-19T08:16:43.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.598+0000 7fe40ebd5640 1 --2- 172.21.15.22:0/2013979390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe408101420 0x7fe40810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:43.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.598+0000 7fe40ebd5640 1 --2- 172.21.15.22:0/2013979390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408100a80 0x7fe408100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:43.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.598+0000 7fe40ebd5640 1 --2- 172.21.15.22:0/2013979390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe408106a80 0x7fe408106e60 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:43.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.598+0000 7fe40ebd5640 1 -- 172.21.15.22:0/2013979390 >> 172.21.15.22:0/2013979390 conn(0x7fe4080fc7a0 msgr2=0x7fe4080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:43.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.598+0000 7fe40ebd5640 1 -- 172.21.15.22:0/2013979390 shutdown_connections 2024-09-19T08:16:43.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.598+0000 7fe40ebd5640 1 -- 172.21.15.22:0/2013979390 wait complete. 2024-09-19T08:16:43.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.599+0000 7fe40ebd5640 1 Processor -- start 2024-09-19T08:16:43.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.599+0000 7fe40ebd5640 1 -- start start 2024-09-19T08:16:43.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.599+0000 7fe40ebd5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe408100a80 0x7fe4081992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:43.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.599+0000 7fe40ebd5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408101420 0x7fe4081997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:43.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.599+0000 7fe40ebd5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe408106a80 0x7fe4081a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:43.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.599+0000 7fe40ebd5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe408110ea0 con 0x7fe408106a80 2024-09-19T08:16:43.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.599+0000 7fe40ebd5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe408110d20 con 0x7fe408101420 2024-09-19T08:16:43.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.599+0000 7fe40ebd5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe408111020 con 0x7fe408100a80 2024-09-19T08:16:43.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.599+0000 7fe40d172640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe408106a80 0x7fe4081a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:43.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe40d172640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe408106a80 0x7fe4081a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57082/0 (socket says 172.21.15.22:57082) 2024-09-19T08:16:43.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe40d172640 1 -- 172.21.15.22:0/637012305 learned_addr learned my addr 172.21.15.22:0/637012305 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:43.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe407fff640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408101420 0x7fe4081997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:43.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe40c971640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe408100a80 0x7fe4081992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:43.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe40d172640 1 -- 172.21.15.22:0/637012305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe408100a80 msgr2=0x7fe4081992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:43.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe40d172640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe408100a80 0x7fe4081992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:43.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe40d172640 1 -- 172.21.15.22:0/637012305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408101420 msgr2=0x7fe4081997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:43.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe40d172640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408101420 0x7fe4081997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:43.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe40d172640 1 -- 172.21.15.22:0/637012305 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4081a4870 con 0x7fe408106a80 2024-09-19T08:16:43.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe407fff640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408101420 0x7fe4081997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:43.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe40d172640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe408106a80 0x7fe4081a4330 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe400009fd0 tx=0x7fe40000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:43.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe405ffb640 1 -- 172.21.15.22:0/637012305 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400011830 con 0x7fe408106a80 2024-09-19T08:16:43.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe405ffb640 1 -- 172.21.15.22:0/637012305 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe400016070 con 0x7fe408106a80 2024-09-19T08:16:43.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe405ffb640 1 -- 172.21.15.22:0/637012305 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40000f2e0 con 0x7fe408106a80 2024-09-19T08:16:43.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe40ebd5640 1 -- 172.21.15.22:0/637012305 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4081a4a40 con 0x7fe408106a80 2024-09-19T08:16:43.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.600+0000 7fe40ebd5640 1 -- 172.21.15.22:0/637012305 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4081a4f00 con 0x7fe408106a80 2024-09-19T08:16:43.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.601+0000 7fe40ebd5640 1 -- 172.21.15.22:0/637012305 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe408100ee0 con 0x7fe408106a80 2024-09-19T08:16:43.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.603+0000 7fe405ffb640 1 -- 172.21.15.22:0/637012305 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe4000119d0 con 0x7fe408106a80 2024-09-19T08:16:43.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.603+0000 7fe405ffb640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe3f40a3430 0x7fe3f40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:43.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.603+0000 7fe40c971640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe3f40a3430 0x7fe3f40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:43.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.603+0000 7fe405ffb640 1 -- 172.21.15.22:0/637012305 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe4000cf2b0 con 0x7fe408106a80 2024-09-19T08:16:43.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.604+0000 7fe40c971640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe3f40a3430 0x7fe3f40a58f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0000c00 tx=0x7fe3f0031490 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:43.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.608+0000 7fe405ffb640 1 -- 172.21.15.22:0/637012305 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe40009bdb0 con 0x7fe408106a80 2024-09-19T08:16:43.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.822+0000 7fe40ebd5640 1 -- 172.21.15.22:0/637012305 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4081a51f0 con 0x7fe408106a80 2024-09-19T08:16:43.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.822+0000 7fe405ffb640 1 -- 172.21.15.22:0/637012305 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11906 (secure 0 0 0) 0x7fe40009b6b0 con 0x7fe408106a80 2024-09-19T08:16:43.823 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:16:43.823 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"btime":"2024-09-19T08:15:00:109390+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:15:00.059208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:16:43.823 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-09-19T08:16:43.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.825+0000 7fe40ebd5640 1 -- 172.21.15.22:0/637012305 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe3f40a3430 msgr2=0x7fe3f40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:43.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.825+0000 7fe40ebd5640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe3f40a3430 0x7fe3f40a58f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0000c00 tx=0x7fe3f0031490 comp rx=0 tx=0).stop 2024-09-19T08:16:43.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.825+0000 7fe40ebd5640 1 -- 172.21.15.22:0/637012305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe408106a80 msgr2=0x7fe4081a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:43.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.825+0000 7fe40ebd5640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe408106a80 0x7fe4081a4330 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe400009fd0 tx=0x7fe40000b8e0 comp rx=0 tx=0).stop 2024-09-19T08:16:43.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.825+0000 7fe40ebd5640 1 -- 172.21.15.22:0/637012305 shutdown_connections 2024-09-19T08:16:43.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.825+0000 7fe40ebd5640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe3f40a3430 0x7fe3f40a58f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:43.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.825+0000 7fe40ebd5640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe408106a80 0x7fe4081a4330 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:43.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.825+0000 7fe40ebd5640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408101420 0x7fe4081997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:43.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.825+0000 7fe40ebd5640 1 --2- 172.21.15.22:0/637012305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe408100a80 0x7fe4081992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:43.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.825+0000 7fe40ebd5640 1 -- 172.21.15.22:0/637012305 >> 172.21.15.22:0/637012305 conn(0x7fe4080fc7a0 msgr2=0x7fe4080fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:43.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.825+0000 7fe40ebd5640 1 -- 172.21.15.22:0/637012305 shutdown_connections 2024-09-19T08:16:43.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:43.825+0000 7fe40ebd5640 1 -- 172.21.15.22:0/637012305 wait complete. 2024-09-19T08:16:44.481 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id efa99971 --release --await' 2024-09-19T08:16:44.481 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id efa99971 --release --await 2024-09-19T08:16:44.746 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:45.069 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:44 smithi022 ceph-mon[30963]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T08:16:45.069 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:44 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/637012305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:45.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:44 smithi064 ceph-mon[34900]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T08:16:45.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:44 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/637012305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:45.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:44 smithi157 ceph-mon[35143]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T08:16:45.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:44 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/637012305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:45.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.285+0000 7f8148423640 1 -- 172.21.15.22:0/191850579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140106a70 msgr2=0x7f8140106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:45.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.285+0000 7f8148423640 1 --2- 172.21.15.22:0/191850579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140106a70 0x7f8140106e50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f812c009cd0 tx=0x7f812c02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.285+0000 7f8148423640 1 -- 172.21.15.22:0/191850579 shutdown_connections 2024-09-19T08:16:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.285+0000 7f8148423640 1 --2- 172.21.15.22:0/191850579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8140101410 0x7f814010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.285+0000 7f8148423640 1 --2- 172.21.15.22:0/191850579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8140100a70 0x7f8140100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.285+0000 7f8148423640 1 --2- 172.21.15.22:0/191850579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140106a70 0x7f8140106e50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.285+0000 7f8148423640 1 -- 172.21.15.22:0/191850579 >> 172.21.15.22:0/191850579 conn(0x7f81400fc790 msgr2=0x7f81400febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.285+0000 7f8148423640 1 -- 172.21.15.22:0/191850579 shutdown_connections 2024-09-19T08:16:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.285+0000 7f8148423640 1 -- 172.21.15.22:0/191850579 wait complete. 2024-09-19T08:16:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.285+0000 7f8148423640 1 Processor -- start 2024-09-19T08:16:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f8148423640 1 -- start start 2024-09-19T08:16:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f8148423640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8140100a70 0x7f81401992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f8148423640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101410 0x7f8140199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f8148423640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8140106a70 0x7f81401a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f8148423640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8140110ec0 con 0x7f8140101410 2024-09-19T08:16:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f8148423640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8140110d40 con 0x7f8140100a70 2024-09-19T08:16:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f8148423640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8140111040 con 0x7f8140106a70 2024-09-19T08:16:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f81459be640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101410 0x7f8140199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f81459be640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101410 0x7f8140199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57098/0 (socket says 172.21.15.22:57098) 2024-09-19T08:16:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f81459be640 1 -- 172.21.15.22:0/2249248672 learned_addr learned my addr 172.21.15.22:0/2249248672 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f81461bf640 1 --2- 172.21.15.22:0/2249248672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8140100a70 0x7f81401992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f81469c0640 1 --2- 172.21.15.22:0/2249248672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8140106a70 0x7f81401a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f81459be640 1 -- 172.21.15.22:0/2249248672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8140106a70 msgr2=0x7f81401a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f81459be640 1 --2- 172.21.15.22:0/2249248672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8140106a70 0x7f81401a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f81459be640 1 -- 172.21.15.22:0/2249248672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8140100a70 msgr2=0x7f81401992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f81459be640 1 --2- 172.21.15.22:0/2249248672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8140100a70 0x7f81401992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.286+0000 7f81459be640 1 -- 172.21.15.22:0/2249248672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81401a48a0 con 0x7f8140101410 2024-09-19T08:16:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.287+0000 7f81459be640 1 --2- 172.21.15.22:0/2249248672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101410 0x7f8140199820 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f813400d950 tx=0x7f813400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.287+0000 7f813f7fe640 1 -- 172.21.15.22:0/2249248672 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134017030 con 0x7f8140101410 2024-09-19T08:16:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.287+0000 7f8148423640 1 -- 172.21.15.22:0/2249248672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81401a4a70 con 0x7f8140101410 2024-09-19T08:16:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.287+0000 7f813f7fe640 1 -- 172.21.15.22:0/2249248672 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8134004020 con 0x7f8140101410 2024-09-19T08:16:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.287+0000 7f813f7fe640 1 -- 172.21.15.22:0/2249248672 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81340047a0 con 0x7f8140101410 2024-09-19T08:16:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.287+0000 7f8148423640 1 -- 172.21.15.22:0/2249248672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81401a4fb0 con 0x7f8140101410 2024-09-19T08:16:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.289+0000 7f813f7fe640 1 -- 172.21.15.22:0/2249248672 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f81340041d0 con 0x7f8140101410 2024-09-19T08:16:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.289+0000 7f814a7fc740 1 Processor -- start 2024-09-19T08:16:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f814a7fc740 1 -- start start 2024-09-19T08:16:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f814a7fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb7af40 0x555bdeb7b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f814a7fc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 0x555bdeb8a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f814a7fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb8a650 0x555bdeb8ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f814a7fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555bdeae0620 con 0x555bdeb7af40 2024-09-19T08:16:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f814a7fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555bdea87e50 con 0x555bdeb8a650 2024-09-19T08:16:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f814a7fc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555bdeadc5e0 con 0x555bdeb7b860 2024-09-19T08:16:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f813f7fe640 1 --2- 172.21.15.22:0/2249248672 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f81300a33b0 0x7f81300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f81469c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb7af40 0x555bdeb7b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f81469c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb7af40 0x555bdeb7b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57110/0 (socket says 172.21.15.22:57110) 2024-09-19T08:16:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f81469c0640 1 -- 172.21.15.22:0/1892310113 learned_addr learned my addr 172.21.15.22:0/1892310113 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f81459be640 1 --2- 172.21.15.22:0/1892310113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb8a650 0x555bdeb8ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f81461bf640 1 --2- 172.21.15.22:0/1892310113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 0x555bdeb8a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f81469c0640 1 --2- 172.21.15.22:0/2249248672 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f81300a33b0 0x7f81300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.290+0000 7f813f7fe640 1 -- 172.21.15.22:0/2249248672 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f81340cf6f0 con 0x7f8140101410 2024-09-19T08:16:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.291+0000 7f81469c0640 1 -- 172.21.15.22:0/1892310113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 msgr2=0x555bdeb8a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.291+0000 7f81469c0640 1 --2- 172.21.15.22:0/1892310113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 0x555bdeb8a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.291+0000 7f81469c0640 1 -- 172.21.15.22:0/1892310113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb8a650 msgr2=0x555bdeb8ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.291+0000 7f81469c0640 1 --2- 172.21.15.22:0/1892310113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb8a650 0x555bdeb8ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.291+0000 7f81469c0640 1 -- 172.21.15.22:0/1892310113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bdea78d40 con 0x555bdeb7af40 2024-09-19T08:16:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.291+0000 7f81461bf640 1 --2- 172.21.15.22:0/1892310113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 0x555bdeb8a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:16:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.291+0000 7f81459be640 1 --2- 172.21.15.22:0/1892310113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb8a650 0x555bdeb8ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.291+0000 7f81469c0640 1 --2- 172.21.15.22:0/1892310113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb7af40 0x555bdeb7b320 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f81380137c0 tx=0x7f8138013c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee9ce96144d1c5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.291+0000 7f813cdea640 1 -- 172.21.15.22:0/1892310113 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813801d7c0 con 0x555bdeb7af40 2024-09-19T08:16:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.291+0000 7f813cdea640 1 -- 172.21.15.22:0/1892310113 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8138022070 con 0x555bdeb7af40 2024-09-19T08:16:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.291+0000 7f813cdea640 1 -- 172.21.15.22:0/1892310113 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813801ca00 con 0x555bdeb7af40 2024-09-19T08:16:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.291+0000 7f81469c0640 1 --2- 172.21.15.22:0/2249248672 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f81300a33b0 0x7f81300a5870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8138021000 tx=0x7f813801f000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.292+0000 7f814a7fc740 1 -- 172.21.15.22:0/1892310113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb7af40 msgr2=0x555bdeb7b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.292+0000 7f814a7fc740 1 --2- 172.21.15.22:0/1892310113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb7af40 0x555bdeb7b320 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f81380137c0 tx=0x7f8138013c80 comp rx=0 tx=0).stop 2024-09-19T08:16:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.292+0000 7f814a7fc740 1 -- 172.21.15.22:0/1892310113 shutdown_connections 2024-09-19T08:16:45.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.292+0000 7f814a7fc740 1 --2- 172.21.15.22:0/1892310113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb8a650 0x555bdeb8ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.292+0000 7f814a7fc740 1 --2- 172.21.15.22:0/1892310113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 0x555bdeb8a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.292+0000 7f814a7fc740 1 --2- 172.21.15.22:0/1892310113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb7af40 0x555bdeb7b320 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.292+0000 7f814a7fc740 1 -- 172.21.15.22:0/1892310113 >> 172.21.15.22:0/1892310113 conn(0x555bdeb775f0 msgr2=0x555bdeb779d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:45.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.292+0000 7f814a7fc740 1 -- 172.21.15.22:0/1892310113 shutdown_connections 2024-09-19T08:16:45.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.292+0000 7f814a7fc740 1 -- 172.21.15.22:0/1892310113 wait complete. 2024-09-19T08:16:45.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.292+0000 7f814a7fc740 1 -- start start 2024-09-19T08:16:45.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.292+0000 7f814a7fc740 1 Processor -- start 2024-09-19T08:16:45.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f814a7fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 0x555bdeb87580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f814a7fc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 0x555bdeb87ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f814a7fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb8a650 0x555bdeb88000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f814a7fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555bdeb8d220 con 0x555bdeb8a650 2024-09-19T08:16:45.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f814a7fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555bdea68f10 con 0x555bdeb7af40 2024-09-19T08:16:45.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f814a7fc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555bdeb8d3a0 con 0x555bdeb7b860 2024-09-19T08:16:45.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f814a7fc740 10 client.0 _ll_register_callbacks cb 0x555bdea8cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:45.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f81469c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb8a650 0x555bdeb88000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f81469c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb8a650 0x555bdeb88000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57122/0 (socket says 172.21.15.22:57122) 2024-09-19T08:16:45.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f81469c0640 1 -- 172.21.15.22:0/150821026 learned_addr learned my addr 172.21.15.22:0/150821026 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:45.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f81461bf640 1 --2- 172.21.15.22:0/150821026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 0x555bdeb87580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f81459be640 1 --2- 172.21.15.22:0/150821026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 0x555bdeb87ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f81469c0640 1 -- 172.21.15.22:0/150821026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 msgr2=0x555bdeb87ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:45.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f81469c0640 1 --2- 172.21.15.22:0/150821026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 0x555bdeb87ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f81469c0640 1 -- 172.21.15.22:0/150821026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 msgr2=0x555bdeb87580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f81469c0640 1 --2- 172.21.15.22:0/150821026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 0x555bdeb87580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.293+0000 7f81469c0640 1 -- 172.21.15.22:0/150821026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bde7ef580 con 0x555bdeb8a650 2024-09-19T08:16:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.294+0000 7f81469c0640 1 --2- 172.21.15.22:0/150821026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb8a650 0x555bdeb88000 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8138005690 tx=0x7f81380056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804db050d7b077eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.294+0000 7f8129ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.294+0000 7f8129ffb640 1 -- 172.21.15.22:0/150821026 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813801ced0 con 0x555bdeb8a650 2024-09-19T08:16:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.294+0000 7f8129ffb640 1 -- 172.21.15.22:0/150821026 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f813801c7a0 con 0x555bdeb8a650 2024-09-19T08:16:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.294+0000 7f814a7fc740 1 -- 172.21.15.22:0/150821026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8138022070 con 0x555bdeb8a650 2024-09-19T08:16:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.294+0000 7f8129ffb640 1 -- 172.21.15.22:0/150821026 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f813803d050 con 0x555bdeb8a650 2024-09-19T08:16:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.294+0000 7f814a7fc740 10 client.15741 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.294+0000 7f814a7fc740 1 -- 172.21.15.22:0/150821026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555bde8b9960 con 0x555bdeb8a650 2024-09-19T08:16:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.294+0000 7f8129ffb640 1 -- 172.21.15.22:0/150821026 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8138022070 con 0x555bdeb8a650 2024-09-19T08:16:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.294+0000 7f8129ffb640 1 -- 172.21.15.22:0/150821026 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f813801da00 con 0x555bdeb8a650 2024-09-19T08:16:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.294+0000 7f814a7fc740 10 client.15741 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.294+0000 7f814a7fc740 10 client.15741 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:16:45.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.294+0000 7f814a7fc740 20 client.15741 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:45.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.295+0000 7f814a7fc740 1 --2- 172.21.15.22:0/150821026 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555bdeb9a160 0x555bdeb9c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.295+0000 7f814a7fc740 4 client.15741 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:16:45.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.295+0000 7f814a7fc740 1 -- 172.21.15.22:0/150821026 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bdeaedb90 con 0x555bdeb9a160 2024-09-19T08:16:45.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.295+0000 7f81461bf640 1 --2- 172.21.15.22:0/150821026 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555bdeb9a160 0x555bdeb9c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.295+0000 7f81461bf640 1 --2- 172.21.15.22:0/150821026 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555bdeb9a160 0x555bdeb9c550 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe915eebf9c7621 server_cookie=8ab66ba371d22197 in_seq=0 out_seq=0 2024-09-19T08:16:45.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.295+0000 7f8129ffb640 10 client.15741 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:45.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f8129ffb640 1 -- 172.21.15.22:0/150821026 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555bdeaedb90 con 0x555bdeb9a160 2024-09-19T08:16:45.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f8129ffb640 10 client.15741 handle_command_reply: tid=0 2024-09-19T08:16:45.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f814a7fc740 1 client.15741 shutdown 2024-09-19T08:16:45.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f814a7fc740 1 -- 172.21.15.22:0/150821026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb8a650 msgr2=0x555bdeb88000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:45.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f814a7fc740 1 --2- 172.21.15.22:0/150821026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb8a650 0x555bdeb88000 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8138005690 tx=0x7f81380056c0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f814a7fc740 1 -- 172.21.15.22:0/150821026 shutdown_connections 2024-09-19T08:16:45.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f814a7fc740 1 --2- 172.21.15.22:0/150821026 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555bdeb9a160 0x555bdeb9c550 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f8129ffb640 0 client.15741 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:45.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f814a7fc740 1 --2- 172.21.15.22:0/150821026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb8a650 0x555bdeb88000 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f814a7fc740 1 --2- 172.21.15.22:0/150821026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 0x555bdeb87ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f814a7fc740 1 --2- 172.21.15.22:0/150821026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 0x555bdeb87580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f814a7fc740 1 -- 172.21.15.22:0/150821026 >> 172.21.15.22:0/150821026 conn(0x555bdeb775f0 msgr2=0x555bdeb7d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:45.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f814a7fc740 1 -- 172.21.15.22:0/150821026 shutdown_connections 2024-09-19T08:16:45.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f814a7fc740 1 -- 172.21.15.22:0/150821026 wait complete. 2024-09-19T08:16:45.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.296+0000 7f814a7fc740 20 client.15741 trim_cache size 0 max 16384 2024-09-19T08:16:45.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f814a7fc740 1 Processor -- start 2024-09-19T08:16:45.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f814a7fc740 1 -- start start 2024-09-19T08:16:45.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f814a7fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 0x555bdeb85e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f814a7fc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 0x555bdeb7d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f814a7fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb8a650 0x555bdeba1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f814a7fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f81401ae080 con 0x555bdeb8a650 2024-09-19T08:16:45.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f814a7fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f81401ab950 con 0x555bdeb7af40 2024-09-19T08:16:45.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f814a7fc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81401a97a0 con 0x555bdeb7b860 2024-09-19T08:16:45.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f81469c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb8a650 0x555bdeba1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f81469c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb8a650 0x555bdeba1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57124/0 (socket says 172.21.15.22:57124) 2024-09-19T08:16:45.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f81469c0640 1 -- 172.21.15.22:0/930515066 learned_addr learned my addr 172.21.15.22:0/930515066 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:45.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f81459be640 1 --2- 172.21.15.22:0/930515066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 0x555bdeb7d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f81461bf640 1 --2- 172.21.15.22:0/930515066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 0x555bdeb85e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f81469c0640 1 -- 172.21.15.22:0/930515066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 msgr2=0x555bdeb7d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:45.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.314+0000 7f81469c0640 1 --2- 172.21.15.22:0/930515066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 0x555bdeb7d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.315+0000 7f81469c0640 1 -- 172.21.15.22:0/930515066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 msgr2=0x555bdeb85e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:45.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.315+0000 7f81469c0640 1 --2- 172.21.15.22:0/930515066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 0x555bdeb85e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.315+0000 7f81469c0640 1 -- 172.21.15.22:0/930515066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bdeb8d070 con 0x555bdeb8a650 2024-09-19T08:16:45.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.315+0000 7f81469c0640 1 --2- 172.21.15.22:0/930515066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb8a650 0x555bdeba1be0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8138011930 tx=0x7f813800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e92722c66c8e2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:45.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.315+0000 7f812bfff640 1 -- 172.21.15.22:0/930515066 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813800bcb0 con 0x555bdeb8a650 2024-09-19T08:16:45.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.315+0000 7f812bfff640 1 -- 172.21.15.22:0/930515066 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8138022040 con 0x555bdeb8a650 2024-09-19T08:16:45.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.315+0000 7f812bfff640 1 -- 172.21.15.22:0/930515066 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813801c990 con 0x555bdeb8a650 2024-09-19T08:16:45.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.316+0000 7f814a7fc740 1 -- 172.21.15.22:0/930515066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb8a650 msgr2=0x555bdeba1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:45.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.316+0000 7f814a7fc740 1 --2- 172.21.15.22:0/930515066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb8a650 0x555bdeba1be0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8138011930 tx=0x7f813800b420 comp rx=0 tx=0).stop 2024-09-19T08:16:45.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.316+0000 7f814a7fc740 1 -- 172.21.15.22:0/930515066 shutdown_connections 2024-09-19T08:16:45.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.316+0000 7f814a7fc740 1 --2- 172.21.15.22:0/930515066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb8a650 0x555bdeba1be0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.316+0000 7f814a7fc740 1 --2- 172.21.15.22:0/930515066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb7b860 0x555bdeb7d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.316+0000 7f814a7fc740 1 --2- 172.21.15.22:0/930515066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 0x555bdeb85e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.316+0000 7f814a7fc740 1 -- 172.21.15.22:0/930515066 >> 172.21.15.22:0/930515066 conn(0x555bdeb775f0 msgr2=0x555bdeb7ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:45.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.316+0000 7f814a7fc740 1 -- 172.21.15.22:0/930515066 shutdown_connections 2024-09-19T08:16:45.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.316+0000 7f814a7fc740 1 -- 172.21.15.22:0/930515066 wait complete. 2024-09-19T08:16:45.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.316+0000 7f814a7fc740 1 -- start start 2024-09-19T08:16:45.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.316+0000 7f814a7fc740 1 Processor -- start 2024-09-19T08:16:45.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.316+0000 7f814a7fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 0x555bdeb8ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.316+0000 7f814a7fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb7b860 0x555bdeb8f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f814a7fc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb8a650 0x555bdebfdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f814a7fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f81401a9e10 con 0x555bdeb7b860 2024-09-19T08:16:45.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f814a7fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f81401ab280 con 0x555bdeb7af40 2024-09-19T08:16:45.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f814a7fc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81401aa4e0 con 0x555bdeb8a650 2024-09-19T08:16:45.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f814a7fc740 10 client.0 _ll_register_callbacks cb 0x7f8138011800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:45.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f81459be640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb7b860 0x555bdeb8f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f81459be640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb7b860 0x555bdeb8f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57132/0 (socket says 172.21.15.22:57132) 2024-09-19T08:16:45.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f81459be640 1 -- 172.21.15.22:0/3642581930 learned_addr learned my addr 172.21.15.22:0/3642581930 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:45.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f81461bf640 1 --2- 172.21.15.22:0/3642581930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 0x555bdeb8ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f81469c0640 1 --2- 172.21.15.22:0/3642581930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb8a650 0x555bdebfdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f81459be640 1 -- 172.21.15.22:0/3642581930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb8a650 msgr2=0x555bdebfdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:45.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f81459be640 1 --2- 172.21.15.22:0/3642581930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb8a650 0x555bdebfdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f81459be640 1 -- 172.21.15.22:0/3642581930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 msgr2=0x555bdeb8ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:45.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f81459be640 1 --2- 172.21.15.22:0/3642581930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 0x555bdeb8ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f81459be640 1 -- 172.21.15.22:0/3642581930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bdebfe1c0 con 0x555bdeb7b860 2024-09-19T08:16:45.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f81469c0640 1 --2- 172.21.15.22:0/3642581930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb8a650 0x555bdebfdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:45.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f81459be640 1 --2- 172.21.15.22:0/3642581930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb7b860 0x555bdeb8f160 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f813409b6e0 tx=0x7f813406a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6a65a8b2c8a6be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:45.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f8129ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:45.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f8129ffb640 1 -- 172.21.15.22:0/3642581930 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813409fd90 con 0x555bdeb7b860 2024-09-19T08:16:45.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f8129ffb640 1 -- 172.21.15.22:0/3642581930 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f81340cd0d0 con 0x555bdeb7b860 2024-09-19T08:16:45.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.317+0000 7f814a7fc740 1 -- 172.21.15.22:0/3642581930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8138022040 con 0x555bdeb7b860 2024-09-19T08:16:45.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.318+0000 7f8129ffb640 1 -- 172.21.15.22:0/3642581930 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f81340a4650 con 0x555bdeb7b860 2024-09-19T08:16:45.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.318+0000 7f814a7fc740 10 client.15747 fetch_fsmap learned FSMap version 25 2024-09-19T08:16:45.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.318+0000 7f814a7fc740 1 -- 172.21.15.22:0/3642581930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555bdec05590 con 0x555bdeb7b860 2024-09-19T08:16:45.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.319+0000 7f8129ffb640 1 -- 172.21.15.22:0/3642581930 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8138022040 con 0x555bdeb7b860 2024-09-19T08:16:45.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.319+0000 7f8129ffb640 1 -- 172.21.15.22:0/3642581930 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8134084ce0 con 0x555bdeb7b860 2024-09-19T08:16:45.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.319+0000 7f814a7fc740 10 client.15747 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T08:16:45.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.319+0000 7f814a7fc740 10 client.15747 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:16:45.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.319+0000 7f814a7fc740 20 client.15747 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:45.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.319+0000 7f814a7fc740 1 --2- 172.21.15.22:0/3642581930 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555bdec05d40 0x555bdec08160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:45.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.319+0000 7f814a7fc740 4 client.15747 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "efa99971", "release": true, "await": true}] 2024-09-19T08:16:45.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.319+0000 7f814a7fc740 1 -- 172.21.15.22:0/3642581930 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "set_id": "efa99971", "release": true, "await": true}) -- 0x555bdeae80a0 con 0x555bdec05d40 2024-09-19T08:16:45.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.319+0000 7f81461bf640 1 --2- 172.21.15.22:0/3642581930 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555bdec05d40 0x555bdec08160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:45.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.320+0000 7f8129ffb640 10 client.15747 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:45.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.320+0000 7f81461bf640 1 --2- 172.21.15.22:0/3642581930 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555bdec05d40 0x555bdec08160 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e86b856e35b46c server_cookie=e2265163fabd6d04 in_seq=0 out_seq=0 2024-09-19T08:16:45.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.323+0000 7f8129ffb640 1 -- 172.21.15.22:0/3642581930 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x555bdeae80a0 con 0x555bdec05d40 2024-09-19T08:16:45.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.323+0000 7f8129ffb640 10 client.15747 handle_command_reply: tid=0 2024-09-19T08:16:45.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.323+0000 7f814a7fc740 1 client.15747 shutdown 2024-09-19T08:16:45.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.323+0000 7f814a7fc740 1 -- 172.21.15.22:0/3642581930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb7b860 msgr2=0x555bdeb8f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:45.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.324+0000 7f814a7fc740 1 --2- 172.21.15.22:0/3642581930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb7b860 0x555bdeb8f160 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f813409b6e0 tx=0x7f813406a6d0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.324+0000 7f814a7fc740 1 -- 172.21.15.22:0/3642581930 shutdown_connections 2024-09-19T08:16:45.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.324+0000 7f814a7fc740 1 --2- 172.21.15.22:0/3642581930 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555bdec05d40 0x555bdec08160 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.324+0000 7f8129ffb640 0 client.15747 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:45.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.324+0000 7f814a7fc740 1 --2- 172.21.15.22:0/3642581930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555bdeb8a650 0x555bdebfdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.324+0000 7f814a7fc740 1 --2- 172.21.15.22:0/3642581930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555bdeb7b860 0x555bdeb8f160 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.324+0000 7f814a7fc740 1 --2- 172.21.15.22:0/3642581930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bdeb7af40 0x555bdeb8ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.324+0000 7f814a7fc740 1 -- 172.21.15.22:0/3642581930 >> 172.21.15.22:0/3642581930 conn(0x555bdeb775f0 msgr2=0x555bdeb76750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:45.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.324+0000 7f814a7fc740 1 -- 172.21.15.22:0/3642581930 shutdown_connections 2024-09-19T08:16:45.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.324+0000 7f814a7fc740 1 -- 172.21.15.22:0/3642581930 wait complete. 2024-09-19T08:16:45.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.324+0000 7f814a7fc740 20 client.15747 trim_cache size 0 max 16384 2024-09-19T08:16:45.326 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:16:45.326 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 25, 2024-09-19T08:16:45.326 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:16:45.326 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 8, 2024-09-19T08:16:45.326 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:16:45.326 INFO:teuthology.orchestra.run.smithi022.stdout: "efa99971": { 2024-09-19T08:16:45.326 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 8, 2024-09-19T08:16:45.327 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:16:45.327 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:16:45.327 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:16:45.327 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:16:45.327 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:16:45.327 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:16:45.327 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 140.3, 2024-09-19T08:16:45.327 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:16:45.327 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:16:45.327 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:16:45.327 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:16:45.327 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:16:45.327 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:16:45.327 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:45.328 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:45.328 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:45.328 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:45.328 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:45.328 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:16:45.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.325+0000 7f8148423640 1 -- 172.21.15.22:0/2249248672 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f81300a33b0 msgr2=0x7f81300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:45.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.325+0000 7f8148423640 1 --2- 172.21.15.22:0/2249248672 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f81300a33b0 0x7f81300a5870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8138021000 tx=0x7f813801f000 comp rx=0 tx=0).stop 2024-09-19T08:16:45.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.325+0000 7f8148423640 1 -- 172.21.15.22:0/2249248672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101410 msgr2=0x7f8140199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:45.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.325+0000 7f8148423640 1 --2- 172.21.15.22:0/2249248672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101410 0x7f8140199820 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f813400d950 tx=0x7f813400de10 comp rx=0 tx=0).stop 2024-09-19T08:16:45.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.326+0000 7f8148423640 1 -- 172.21.15.22:0/2249248672 shutdown_connections 2024-09-19T08:16:45.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.326+0000 7f8148423640 1 --2- 172.21.15.22:0/2249248672 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f81300a33b0 0x7f81300a5870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.326+0000 7f8148423640 1 --2- 172.21.15.22:0/2249248672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8140106a70 0x7f81401a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.326+0000 7f8148423640 1 --2- 172.21.15.22:0/2249248672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101410 0x7f8140199820 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.326+0000 7f8148423640 1 --2- 172.21.15.22:0/2249248672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8140100a70 0x7f81401992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:45.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.326+0000 7f8148423640 1 -- 172.21.15.22:0/2249248672 >> 172.21.15.22:0/2249248672 conn(0x7f81400fc790 msgr2=0x7f81400fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:45.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.326+0000 7f8148423640 1 -- 172.21.15.22:0/2249248672 shutdown_connections 2024-09-19T08:16:45.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:45.326+0000 7f8148423640 1 -- 172.21.15.22:0/2249248672 wait complete. 2024-09-19T08:16:45.330 INFO:tasks.workunit.client.0.smithi022.stdout:getting blogbench 2024-09-19T08:16:45.330 INFO:tasks.workunit.client.0.smithi022.stderr:+ echo 'getting blogbench' 2024-09-19T08:16:45.330 INFO:tasks.workunit.client.0.smithi022.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-19T08:16:45.449 INFO:tasks.workunit.client.0.smithi022.stderr:--2024-09-19 08:16:45-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-19T08:16:45.449 INFO:tasks.workunit.client.0.smithi022.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-19T08:16:45.518 INFO:tasks.workunit.client.0.smithi022.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-19T08:16:45.579 INFO:tasks.workunit.client.0.smithi022.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-19T08:16:45.579 INFO:tasks.workunit.client.0.smithi022.stderr:Length: 103355 (101K) [application/octet-stream] 2024-09-19T08:16:45.580 INFO:tasks.workunit.client.0.smithi022.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-09-19T08:16:45.582 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-09-19T08:16:45.700 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:16:45.714 INFO:tasks.workunit.client.0.smithi022.stderr: 0K .......... .......... .......... .......... .......... 49% 376K 0s 2024-09-19T08:16:45.781 INFO:tasks.workunit.client.0.smithi022.stderr: 50K .......... .......... .......... .......... .......... 99% 735K 0s 2024-09-19T08:16:45.782 INFO:tasks.workunit.client.0.smithi022.stderr: 100K 100% 1.74T=0.2s 2024-09-19T08:16:45.782 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-09-19T08:16:45.782 INFO:tasks.workunit.client.0.smithi022.stderr:2024-09-19 08:16:45 (502 KB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-09-19T08:16:45.782 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-09-19T08:16:45.782 INFO:tasks.workunit.client.0.smithi022.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-09-19T08:16:45.847 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/ 2024-09-19T08:16:45.848 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/m4/ 2024-09-19T08:16:45.849 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-09-19T08:16:45.850 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/man/ 2024-09-19T08:16:45.851 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/man/blogbench.8 2024-09-19T08:16:45.852 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/man/Makefile.am 2024-09-19T08:16:45.853 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/man/Makefile.in 2024-09-19T08:16:45.854 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/ 2024-09-19T08:16:45.854 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/mysnprintf.c 2024-09-19T08:16:45.856 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/mysnprintf.h 2024-09-19T08:16:45.856 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/blogbench.c 2024-09-19T08:16:45.857 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/blogbench.h 2024-09-19T08:16:45.858 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/fakesnprintf.c 2024-09-19T08:16:45.859 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/fakesnprintf.h 2024-09-19T08:16:45.860 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/blogbench_p.h 2024-09-19T08:16:45.861 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-09-19T08:16:45.862 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-09-19T08:16:45.862 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/Makefile.am 2024-09-19T08:16:45.863 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/Makefile.in 2024-09-19T08:16:45.864 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/process.c 2024-09-19T08:16:46.076 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/globals.h 2024-09-19T08:16:46.076 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/commenter.c 2024-09-19T08:16:46.077 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/rewriter.c 2024-09-19T08:16:46.077 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/reader.c 2024-09-19T08:16:46.077 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/writer.c 2024-09-19T08:16:46.077 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/src/helpers.c 2024-09-19T08:16:46.077 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/NEWS 2024-09-19T08:16:46.077 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/compile 2024-09-19T08:16:46.077 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/depcomp 2024-09-19T08:16:46.077 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/aclocal.m4 2024-09-19T08:16:46.077 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/README 2024-09-19T08:16:46.077 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/configure 2024-09-19T08:16:46.077 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/configure.ac 2024-09-19T08:16:46.077 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/config.guess 2024-09-19T08:16:46.077 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/install-sh 2024-09-19T08:16:46.077 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/config.sub 2024-09-19T08:16:46.078 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/missing 2024-09-19T08:16:46.078 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/mkinstalldirs 2024-09-19T08:16:46.078 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/Makefile.am 2024-09-19T08:16:46.078 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/Makefile.in 2024-09-19T08:16:46.078 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/config.h.in 2024-09-19T08:16:46.078 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/AUTHORS 2024-09-19T08:16:46.078 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/INSTALL 2024-09-19T08:16:46.078 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/ChangeLog 2024-09-19T08:16:46.078 INFO:tasks.workunit.client.0.smithi022.stdout:blogbench-1.0/COPYING 2024-09-19T08:16:46.078 INFO:tasks.workunit.client.0.smithi022.stderr:+ cd blogbench-1.0/ 2024-09-19T08:16:46.079 INFO:tasks.workunit.client.0.smithi022.stderr:+ echo 'making blogbench' 2024-09-19T08:16:46.079 INFO:tasks.workunit.client.0.smithi022.stdout:making blogbench 2024-09-19T08:16:46.079 INFO:tasks.workunit.client.0.smithi022.stderr:+ ./configure 2024-09-19T08:16:46.080 INFO:tasks.workunit.client.0.smithi022.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-19T08:16:46.543 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'efa99971', total seconds elapsed: 24.0 2024-09-19T08:16:46.543 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T08:16:46.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:16:46.603 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:47.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:46 smithi022 ceph-mon[30963]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T08:16:47.040 INFO:tasks.workunit.client.0.smithi022.stdout:checking whether build environment is sane... yes 2024-09-19T08:16:47.049 INFO:tasks.workunit.client.0.smithi022.stdout:checking for gawk... gawk 2024-09-19T08:16:47.065 INFO:tasks.workunit.client.0.smithi022.stdout:checking whether make sets $(MAKE)... yes 2024-09-19T08:16:47.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:46 smithi064 ceph-mon[34900]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T08:16:47.198 INFO:tasks.workunit.client.0.smithi022.stdout:checking for gcc... gcc 2024-09-19T08:16:47.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:46 smithi157 ceph-mon[35143]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T08:16:47.342 INFO:tasks.workunit.client.0.smithi022.stdout:checking for C compiler default output file name... a.out 2024-09-19T08:16:47.345 INFO:tasks.workunit.client.0.smithi022.stdout:checking whether the C compiler works... yes 2024-09-19T08:16:47.347 INFO:tasks.workunit.client.0.smithi022.stdout:checking whether we are cross compiling... no 2024-09-19T08:16:47.585 INFO:tasks.workunit.client.0.smithi022.stdout:checking for suffix of executables... 2024-09-19T08:16:47.725 INFO:tasks.workunit.client.0.smithi022.stdout:checking for suffix of object files... o 2024-09-19T08:16:47.863 INFO:tasks.workunit.client.0.smithi022.stdout:checking whether we are using the GNU C compiler... yes 2024-09-19T08:16:47.948 INFO:tasks.workunit.client.0.smithi022.stdout:checking whether gcc accepts -g... yes 2024-09-19T08:16:48.075 INFO:tasks.workunit.client.0.smithi022.stdout:checking for gcc option to accept ANSI C... none needed 2024-09-19T08:16:48.135 INFO:tasks.workunit.client.0.smithi022.stdout:checking for style of include used by make... GNU 2024-09-19T08:16:48.236 INFO:tasks.workunit.client.0.smithi022.stdout:checking dependency style of gcc... gcc3 2024-09-19T08:16:48.236 INFO:tasks.workunit.client.0.smithi022.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-19T08:16:48.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.287+0000 7f188950f640 1 -- 172.21.15.22:0/2380573691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188410c180 msgr2=0x7f188410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:48.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.287+0000 7f188950f640 1 --2- 172.21.15.22:0/2380573691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188410c180 0x7f188410e610 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f187c009ca0 tx=0x7f187c02a950 comp rx=0 tx=0).stop 2024-09-19T08:16:48.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.288+0000 7f188950f640 1 -- 172.21.15.22:0/2380573691 shutdown_connections 2024-09-19T08:16:48.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.288+0000 7f188950f640 1 --2- 172.21.15.22:0/2380573691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188410c180 0x7f188410e610 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.288+0000 7f188950f640 1 --2- 172.21.15.22:0/2380573691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1884103700 0x7f188410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.288+0000 7f188950f640 1 --2- 172.21.15.22:0/2380573691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1884102de0 0x7f18841031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.288+0000 7f188950f640 1 -- 172.21.15.22:0/2380573691 >> 172.21.15.22:0/2380573691 conn(0x7f18840fc7a0 msgr2=0x7f18840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:48.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.288+0000 7f188950f640 1 -- 172.21.15.22:0/2380573691 shutdown_connections 2024-09-19T08:16:48.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.288+0000 7f188950f640 1 -- 172.21.15.22:0/2380573691 wait complete. 2024-09-19T08:16:48.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.288+0000 7f188950f640 1 Processor -- start 2024-09-19T08:16:48.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f188950f640 1 -- start start 2024-09-19T08:16:48.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f188950f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1884102de0 0x7f1884198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f188950f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1884103700 0x7f1884198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f188950f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188410c180 0x7f188419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f188950f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1884111f50 con 0x7f188410c180 2024-09-19T08:16:48.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f188950f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1884111dd0 con 0x7f1884102de0 2024-09-19T08:16:48.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f188950f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f18841120d0 con 0x7f1884103700 2024-09-19T08:16:48.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f18837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188410c180 0x7f188419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f18837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188410c180 0x7f188419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49104/0 (socket says 172.21.15.22:49104) 2024-09-19T08:16:48.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f18837fe640 1 -- 172.21.15.22:0/3286205121 learned_addr learned my addr 172.21.15.22:0/3286205121 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:48.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f187a7fc640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1884103700 0x7f1884198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f1882ffd640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1884102de0 0x7f1884198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f18837fe640 1 -- 172.21.15.22:0/3286205121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1884103700 msgr2=0x7f1884198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:48.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f18837fe640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1884103700 0x7f1884198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f18837fe640 1 -- 172.21.15.22:0/3286205121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1884102de0 msgr2=0x7f1884198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:48.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f18837fe640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1884102de0 0x7f1884198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f18837fe640 1 -- 172.21.15.22:0/3286205121 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188419d3c0 con 0x7f188410c180 2024-09-19T08:16:48.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.289+0000 7f1882ffd640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1884102de0 0x7f1884198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:16:48.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.290+0000 7f18837fe640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188410c180 0x7f188419cce0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f187c004af0 tx=0x7f187c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:48.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.290+0000 7f1880ff9640 1 -- 172.21.15.22:0/3286205121 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c034040 con 0x7f188410c180 2024-09-19T08:16:48.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.290+0000 7f1880ff9640 1 -- 172.21.15.22:0/3286205121 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f187c030920 con 0x7f188410c180 2024-09-19T08:16:48.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.290+0000 7f188950f640 1 -- 172.21.15.22:0/3286205121 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188419d650 con 0x7f188410c180 2024-09-19T08:16:48.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.290+0000 7f1880ff9640 1 -- 172.21.15.22:0/3286205121 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c0047b0 con 0x7f188410c180 2024-09-19T08:16:48.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.290+0000 7f188950f640 1 -- 172.21.15.22:0/3286205121 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18841a4f90 con 0x7f188410c180 2024-09-19T08:16:48.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.292+0000 7f1880ff9640 1 -- 172.21.15.22:0/3286205121 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f187c037020 con 0x7f188410c180 2024-09-19T08:16:48.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.292+0000 7f188b8e8740 1 Processor -- start 2024-09-19T08:16:48.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f188b8e8740 1 -- start start 2024-09-19T08:16:48.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f188b8e8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339b4a0 0x5626c339b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f1880ff9640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f18700a33b0 0x7f18700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f188b8e8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339bdc0 0x5626c33a6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f188b8e8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c33a6680 0x5626c33a8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f188b8e8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5626c33012e0 con 0x5626c339b4a0 2024-09-19T08:16:48.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f188b8e8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5626c32a8ce0 con 0x5626c339bdc0 2024-09-19T08:16:48.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f188b8e8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5626c32fd2a0 con 0x5626c33a6680 2024-09-19T08:16:48.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f187a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339b4a0 0x5626c339b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f1882ffd640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f18700a33b0 0x7f18700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f187a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339b4a0 0x5626c339b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49114/0 (socket says 172.21.15.22:49114) 2024-09-19T08:16:48.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f187a7fc640 1 -- 172.21.15.22:0/3488248889 learned_addr learned my addr 172.21.15.22:0/3488248889 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:48.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f1880ff9640 1 -- 172.21.15.22:0/3286205121 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f187c0f0a40 con 0x7f188410c180 2024-09-19T08:16:48.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f18837fe640 1 --2- 172.21.15.22:0/3488248889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c33a6680 0x5626c33a8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f187a7fc640 1 --2- 172.21.15.22:0/3488248889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339bdc0 0x5626c33a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f187a7fc640 1 -- 172.21.15.22:0/3488248889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c33a6680 msgr2=0x5626c33a8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:48.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.293+0000 7f187a7fc640 1 --2- 172.21.15.22:0/3488248889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c33a6680 0x5626c33a8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.294+0000 7f187a7fc640 1 -- 172.21.15.22:0/3488248889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339bdc0 msgr2=0x5626c33a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:48.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.294+0000 7f187a7fc640 1 --2- 172.21.15.22:0/3488248889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339bdc0 0x5626c33a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.294+0000 7f187a7fc640 1 -- 172.21.15.22:0/3488248889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c328d440 con 0x5626c339b4a0 2024-09-19T08:16:48.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.294+0000 7f1882ffd640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f18700a33b0 0x7f18700a5870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1874009b40 tx=0x7f1874005e50 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:48.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.294+0000 7f187a7fc640 1 --2- 172.21.15.22:0/3488248889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339b4a0 0x5626c339b880 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f186c0058c0 tx=0x7f186c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42857ffe0b174441 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:48.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.294+0000 7f18797fa640 1 -- 172.21.15.22:0/3488248889 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c004050 con 0x5626c339b4a0 2024-09-19T08:16:48.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.294+0000 7f18797fa640 1 -- 172.21.15.22:0/3488248889 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f186c0041f0 con 0x5626c339b4a0 2024-09-19T08:16:48.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.294+0000 7f18797fa640 1 -- 172.21.15.22:0/3488248889 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c010750 con 0x5626c339b4a0 2024-09-19T08:16:48.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.295+0000 7f188b8e8740 1 -- 172.21.15.22:0/3488248889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339b4a0 msgr2=0x5626c339b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:48.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.295+0000 7f188b8e8740 1 --2- 172.21.15.22:0/3488248889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339b4a0 0x5626c339b880 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f186c0058c0 tx=0x7f186c005850 comp rx=0 tx=0).stop 2024-09-19T08:16:48.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.295+0000 7f188b8e8740 1 -- 172.21.15.22:0/3488248889 shutdown_connections 2024-09-19T08:16:48.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.295+0000 7f188b8e8740 1 --2- 172.21.15.22:0/3488248889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c33a6680 0x5626c33a8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.295+0000 7f188b8e8740 1 --2- 172.21.15.22:0/3488248889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339bdc0 0x5626c33a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.295+0000 7f188b8e8740 1 --2- 172.21.15.22:0/3488248889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339b4a0 0x5626c339b880 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.295+0000 7f188b8e8740 1 -- 172.21.15.22:0/3488248889 >> 172.21.15.22:0/3488248889 conn(0x5626c3397a30 msgr2=0x5626c3397e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:48.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.295+0000 7f188b8e8740 1 -- 172.21.15.22:0/3488248889 shutdown_connections 2024-09-19T08:16:48.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.295+0000 7f188b8e8740 1 -- 172.21.15.22:0/3488248889 wait complete. 2024-09-19T08:16:48.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.295+0000 7f188b8e8740 1 -- start start 2024-09-19T08:16:48.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.295+0000 7f188b8e8740 1 Processor -- start 2024-09-19T08:16:48.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f188b8e8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339b4a0 0x5626c33a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f188b8e8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c339bdc0 0x5626c33a3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f188b8e8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c33a6680 0x5626c33a3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f188b8e8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5626c33a9250 con 0x5626c339b4a0 2024-09-19T08:16:48.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f1882ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339b4a0 0x5626c33a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f188b8e8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5626c32bc050 con 0x5626c33a6680 2024-09-19T08:16:48.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f188b8e8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5626c33a93d0 con 0x5626c339bdc0 2024-09-19T08:16:48.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f1882ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339b4a0 0x5626c33a5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49130/0 (socket says 172.21.15.22:49130) 2024-09-19T08:16:48.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f1882ffd640 1 -- 172.21.15.22:0/768054301 learned_addr learned my addr 172.21.15.22:0/768054301 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:48.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f187a7fc640 1 --2- 172.21.15.22:0/768054301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c339bdc0 0x5626c33a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f18837fe640 1 --2- 172.21.15.22:0/768054301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c33a6680 0x5626c33a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f188b8e8740 10 client.0 _ll_register_callbacks cb 0x5626c32fc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:48.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f1882ffd640 1 -- 172.21.15.22:0/768054301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c339bdc0 msgr2=0x5626c33a3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:48.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f1882ffd640 1 --2- 172.21.15.22:0/768054301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c339bdc0 0x5626c33a3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f1882ffd640 1 -- 172.21.15.22:0/768054301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c33a6680 msgr2=0x5626c33a3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:48.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f1882ffd640 1 --2- 172.21.15.22:0/768054301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c33a6680 0x5626c33a3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f1882ffd640 1 -- 172.21.15.22:0/768054301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c30723c0 con 0x5626c339b4a0 2024-09-19T08:16:48.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f187a7fc640 1 --2- 172.21.15.22:0/768054301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c339bdc0 0x5626c33a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:48.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f1882ffd640 1 --2- 172.21.15.22:0/768054301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339b4a0 0x5626c33a5fa0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f187401ca00 tx=0x7f18740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e795109e3c4a55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:48.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.296+0000 7f186a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:48.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f186a7fc640 1 -- 172.21.15.22:0/768054301 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187401c410 con 0x5626c339b4a0 2024-09-19T08:16:48.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f186a7fc640 1 -- 172.21.15.22:0/768054301 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1874011d80 con 0x5626c339b4a0 2024-09-19T08:16:48.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f188b8e8740 1 -- 172.21.15.22:0/768054301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f186c0041f0 con 0x5626c339b4a0 2024-09-19T08:16:48.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f186a7fc640 1 -- 172.21.15.22:0/768054301 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1874025420 con 0x5626c339b4a0 2024-09-19T08:16:48.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f188b8e8740 10 client.15759 fetch_fsmap learned FSMap version 26 2024-09-19T08:16:48.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f188b8e8740 1 -- 172.21.15.22:0/768054301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626c33bd420 con 0x5626c339b4a0 2024-09-19T08:16:48.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f186a7fc640 1 -- 172.21.15.22:0/768054301 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f186c0041f0 con 0x5626c339b4a0 2024-09-19T08:16:48.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f186a7fc640 1 -- 172.21.15.22:0/768054301 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f187400f040 con 0x5626c339b4a0 2024-09-19T08:16:48.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f188b8e8740 10 client.15759 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:16:48.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f188b8e8740 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:16:48.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f188b8e8740 20 client.15759 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:48.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f188b8e8740 1 --2- 172.21.15.22:0/768054301 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5626c33be8f0 0x5626c33c0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f188b8e8740 4 client.15759 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:16:48.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f188b8e8740 1 -- 172.21.15.22:0/768054301 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626c330e850 con 0x5626c33be8f0 2024-09-19T08:16:48.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.297+0000 7f187a7fc640 1 --2- 172.21.15.22:0/768054301 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5626c33be8f0 0x5626c33c0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.298+0000 7f186a7fc640 10 client.15759 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:48.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.298+0000 7f187a7fc640 1 --2- 172.21.15.22:0/768054301 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5626c33be8f0 0x5626c33c0d70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5dcc3f00a173b1c server_cookie=5e79a5a60eef7d23 in_seq=0 out_seq=0 2024-09-19T08:16:48.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.298+0000 7f186a7fc640 1 -- 172.21.15.22:0/768054301 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626c330e850 con 0x5626c33be8f0 2024-09-19T08:16:48.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.298+0000 7f186a7fc640 10 client.15759 handle_command_reply: tid=0 2024-09-19T08:16:48.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.298+0000 7f188b8e8740 1 client.15759 shutdown 2024-09-19T08:16:48.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.299+0000 7f188b8e8740 1 -- 172.21.15.22:0/768054301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339b4a0 msgr2=0x5626c33a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:48.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.299+0000 7f188b8e8740 1 --2- 172.21.15.22:0/768054301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339b4a0 0x5626c33a5fa0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f187401ca00 tx=0x7f18740046c0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.299+0000 7f188b8e8740 1 -- 172.21.15.22:0/768054301 shutdown_connections 2024-09-19T08:16:48.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.299+0000 7f188b8e8740 1 --2- 172.21.15.22:0/768054301 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5626c33be8f0 0x5626c33c0d70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.299+0000 7f188b8e8740 1 --2- 172.21.15.22:0/768054301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c33a6680 0x5626c33a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.299+0000 7f188b8e8740 1 --2- 172.21.15.22:0/768054301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c339bdc0 0x5626c33a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.299+0000 7f186a7fc640 0 client.15759 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:48.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.299+0000 7f188b8e8740 1 --2- 172.21.15.22:0/768054301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339b4a0 0x5626c33a5fa0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.299+0000 7f188b8e8740 1 -- 172.21.15.22:0/768054301 >> 172.21.15.22:0/768054301 conn(0x5626c3397a30 msgr2=0x5626c339d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:48.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.299+0000 7f188b8e8740 1 -- 172.21.15.22:0/768054301 shutdown_connections 2024-09-19T08:16:48.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.299+0000 7f188b8e8740 1 -- 172.21.15.22:0/768054301 wait complete. 2024-09-19T08:16:48.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.299+0000 7f188b8e8740 20 client.15759 trim_cache size 0 max 16384 2024-09-19T08:16:48.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.319+0000 7f188b8e8740 1 Processor -- start 2024-09-19T08:16:48.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.319+0000 7f188b8e8740 1 -- start start 2024-09-19T08:16:48.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f188b8e8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339b4a0 0x5626c33b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f188b8e8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339bdc0 0x5626c339e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f188b8e8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c33a6680 0x5626c33c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f188b8e8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f18841adf20 con 0x5626c339bdc0 2024-09-19T08:16:48.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f188b8e8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f18841ab7f0 con 0x5626c339b4a0 2024-09-19T08:16:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f188b8e8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f18841a9640 con 0x5626c33a6680 2024-09-19T08:16:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f187a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339bdc0 0x5626c339e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f187a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339bdc0 0x5626c339e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49146/0 (socket says 172.21.15.22:49146) 2024-09-19T08:16:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f187a7fc640 1 -- 172.21.15.22:0/78928830 learned_addr learned my addr 172.21.15.22:0/78928830 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f1882ffd640 1 --2- 172.21.15.22:0/78928830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339b4a0 0x5626c33b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f18837fe640 1 --2- 172.21.15.22:0/78928830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c33a6680 0x5626c33c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f187a7fc640 1 -- 172.21.15.22:0/78928830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c33a6680 msgr2=0x5626c33c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f187a7fc640 1 --2- 172.21.15.22:0/78928830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c33a6680 0x5626c33c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f187a7fc640 1 -- 172.21.15.22:0/78928830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339b4a0 msgr2=0x5626c33b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f187a7fc640 1 --2- 172.21.15.22:0/78928830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339b4a0 0x5626c33b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f187a7fc640 1 -- 172.21.15.22:0/78928830 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c33974d0 con 0x5626c339bdc0 2024-09-19T08:16:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.320+0000 7f18837fe640 1 --2- 172.21.15.22:0/78928830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c33a6680 0x5626c33c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:48.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.321+0000 7f187a7fc640 1 --2- 172.21.15.22:0/78928830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339bdc0 0x5626c339e7f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f186c003e00 tx=0x7f186c009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e0e6aaeb0e3e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:48.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.321+0000 7f1878ff9640 1 -- 172.21.15.22:0/78928830 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c003900 con 0x5626c339bdc0 2024-09-19T08:16:48.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.321+0000 7f1878ff9640 1 -- 172.21.15.22:0/78928830 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f186c009ca0 con 0x5626c339bdc0 2024-09-19T08:16:48.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.321+0000 7f1878ff9640 1 -- 172.21.15.22:0/78928830 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c010be0 con 0x5626c339bdc0 2024-09-19T08:16:48.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.322+0000 7f188b8e8740 1 -- 172.21.15.22:0/78928830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339bdc0 msgr2=0x5626c339e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:48.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.322+0000 7f188b8e8740 1 --2- 172.21.15.22:0/78928830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339bdc0 0x5626c339e7f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f186c003e00 tx=0x7f186c009f20 comp rx=0 tx=0).stop 2024-09-19T08:16:48.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.322+0000 7f188b8e8740 1 -- 172.21.15.22:0/78928830 shutdown_connections 2024-09-19T08:16:48.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.322+0000 7f188b8e8740 1 --2- 172.21.15.22:0/78928830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c33a6680 0x5626c33c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.322+0000 7f188b8e8740 1 --2- 172.21.15.22:0/78928830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c339bdc0 0x5626c339e7f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.322+0000 7f188b8e8740 1 --2- 172.21.15.22:0/78928830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339b4a0 0x5626c33b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.322+0000 7f188b8e8740 1 -- 172.21.15.22:0/78928830 >> 172.21.15.22:0/78928830 conn(0x5626c3397a30 msgr2=0x5626c339e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:48.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.322+0000 7f188b8e8740 1 -- 172.21.15.22:0/78928830 shutdown_connections 2024-09-19T08:16:48.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.322+0000 7f188b8e8740 1 -- 172.21.15.22:0/78928830 wait complete. 2024-09-19T08:16:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.322+0000 7f188b8e8740 1 -- start start 2024-09-19T08:16:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.322+0000 7f188b8e8740 1 Processor -- start 2024-09-19T08:16:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f188b8e8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339b4a0 0x5626c33aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f188b8e8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c339bdc0 0x5626c33a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f188b8e8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c33a6680 0x5626c33aa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f18837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c33a6680 0x5626c33aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f18837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c33a6680 0x5626c33aa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49162/0 (socket says 172.21.15.22:49162) 2024-09-19T08:16:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f188b8e8740 1 -- 172.21.15.22:0/1050108933 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f18841a9cb0 con 0x5626c33a6680 2024-09-19T08:16:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f188b8e8740 1 -- 172.21.15.22:0/1050108933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f18841ab120 con 0x5626c339b4a0 2024-09-19T08:16:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f18837fe640 1 -- 172.21.15.22:0/1050108933 learned_addr learned my addr 172.21.15.22:0/1050108933 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f188b8e8740 1 -- 172.21.15.22:0/1050108933 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f18841aa380 con 0x5626c339bdc0 2024-09-19T08:16:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f187a7fc640 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c339bdc0 0x5626c33a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f1882ffd640 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339b4a0 0x5626c33aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f188b8e8740 10 client.0 _ll_register_callbacks cb 0x7f18740097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:48.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f18837fe640 1 -- 172.21.15.22:0/1050108933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c339bdc0 msgr2=0x5626c33a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:48.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f18837fe640 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c339bdc0 0x5626c33a9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f18837fe640 1 -- 172.21.15.22:0/1050108933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339b4a0 msgr2=0x5626c33aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:48.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f18837fe640 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339b4a0 0x5626c33aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f18837fe640 1 -- 172.21.15.22:0/1050108933 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c33aa920 con 0x5626c33a6680 2024-09-19T08:16:48.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f187a7fc640 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c339bdc0 0x5626c33a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:48.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.323+0000 7f1882ffd640 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339b4a0 0x5626c33aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:48.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.324+0000 7f18837fe640 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c33a6680 0x5626c33aa360 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f187c0bcd70 tx=0x7f187c0bc800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadf133c8b564173 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:48.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.324+0000 7f186a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:48.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.324+0000 7f186a7fc640 1 -- 172.21.15.22:0/1050108933 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c02c040 con 0x5626c33a6680 2024-09-19T08:16:48.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.324+0000 7f186a7fc640 1 -- 172.21.15.22:0/1050108933 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f187c0ee3c0 con 0x5626c33a6680 2024-09-19T08:16:48.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.324+0000 7f188b8e8740 1 -- 172.21.15.22:0/1050108933 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f186c009ca0 con 0x5626c33a6680 2024-09-19T08:16:48.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.324+0000 7f186a7fc640 1 -- 172.21.15.22:0/1050108933 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f187c0ee5c0 con 0x5626c33a6680 2024-09-19T08:16:48.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.324+0000 7f188b8e8740 10 client.15765 fetch_fsmap learned FSMap version 26 2024-09-19T08:16:48.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.324+0000 7f188b8e8740 1 -- 172.21.15.22:0/1050108933 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626c3429370 con 0x5626c33a6680 2024-09-19T08:16:48.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.324+0000 7f186a7fc640 1 -- 172.21.15.22:0/1050108933 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f186c009ca0 con 0x5626c33a6680 2024-09-19T08:16:48.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.324+0000 7f186a7fc640 1 -- 172.21.15.22:0/1050108933 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f187c04d030 con 0x5626c33a6680 2024-09-19T08:16:48.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.325+0000 7f188b8e8740 10 client.15765 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:16:48.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.325+0000 7f188b8e8740 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:16:48.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.325+0000 7f188b8e8740 20 client.15765 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:48.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.325+0000 7f188b8e8740 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5626c3429ae0 0x5626c342bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:48.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.325+0000 7f188b8e8740 4 client.15765 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:16:48.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.325+0000 7f188b8e8740 1 -- 172.21.15.22:0/1050108933 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5626c3308d60 con 0x5626c3429ae0 2024-09-19T08:16:48.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.325+0000 7f1882ffd640 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5626c3429ae0 0x5626c342bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:48.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.325+0000 7f1882ffd640 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5626c3429ae0 0x5626c342bf00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42c10aee14663dde server_cookie=7340ef8e8b5ce928 in_seq=0 out_seq=0 2024-09-19T08:16:48.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.325+0000 7f186a7fc640 10 client.15765 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:48.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.325+0000 7f186a7fc640 1 -- 172.21.15.22:0/1050108933 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5626c3308d60 con 0x5626c3429ae0 2024-09-19T08:16:48.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.325+0000 7f186a7fc640 10 client.15765 handle_command_reply: tid=0 2024-09-19T08:16:48.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.325+0000 7f188b8e8740 1 client.15765 shutdown 2024-09-19T08:16:48.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.326+0000 7f188b8e8740 1 -- 172.21.15.22:0/1050108933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c33a6680 msgr2=0x5626c33aa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:48.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.326+0000 7f188b8e8740 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c33a6680 0x5626c33aa360 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f187c0bcd70 tx=0x7f187c0bc800 comp rx=0 tx=0).stop 2024-09-19T08:16:48.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.326+0000 7f188b8e8740 1 -- 172.21.15.22:0/1050108933 shutdown_connections 2024-09-19T08:16:48.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.326+0000 7f188b8e8740 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5626c3429ae0 0x5626c342bf00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.326+0000 7f188b8e8740 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5626c33a6680 0x5626c33aa360 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.326+0000 7f188b8e8740 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c339bdc0 0x5626c33a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.326+0000 7f188b8e8740 1 --2- 172.21.15.22:0/1050108933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626c339b4a0 0x5626c33aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.326+0000 7f188b8e8740 1 -- 172.21.15.22:0/1050108933 >> 172.21.15.22:0/1050108933 conn(0x5626c3397a30 msgr2=0x5626c339e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:48.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.326+0000 7f186a7fc640 0 client.15765 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:48.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.326+0000 7f188b8e8740 1 -- 172.21.15.22:0/1050108933 shutdown_connections 2024-09-19T08:16:48.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.326+0000 7f188b8e8740 1 -- 172.21.15.22:0/1050108933 wait complete. 2024-09-19T08:16:48.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.326+0000 7f188b8e8740 20 client.15765 trim_cache size 0 max 16384 2024-09-19T08:16:48.333 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:16:48.333 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-19T08:16:48.333 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-09-19T08:16:48.333 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:16:48.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.327+0000 7f188950f640 1 -- 172.21.15.22:0/3286205121 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f18700a33b0 msgr2=0x7f18700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:48.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.327+0000 7f188950f640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f18700a33b0 0x7f18700a5870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1874009b40 tx=0x7f1874005e50 comp rx=0 tx=0).stop 2024-09-19T08:16:48.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.327+0000 7f188950f640 1 -- 172.21.15.22:0/3286205121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188410c180 msgr2=0x7f188419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:48.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.327+0000 7f188950f640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188410c180 0x7f188419cce0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f187c004af0 tx=0x7f187c02f5b0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.327+0000 7f188950f640 1 -- 172.21.15.22:0/3286205121 shutdown_connections 2024-09-19T08:16:48.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.327+0000 7f188950f640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f18700a33b0 0x7f18700a5870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.327+0000 7f188950f640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f188410c180 0x7f188419cce0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.327+0000 7f188950f640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1884103700 0x7f1884198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.327+0000 7f188950f640 1 --2- 172.21.15.22:0/3286205121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1884102de0 0x7f1884198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:48.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.327+0000 7f188950f640 1 -- 172.21.15.22:0/3286205121 >> 172.21.15.22:0/3286205121 conn(0x7f18840fc7a0 msgr2=0x7f18840fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:48.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.327+0000 7f188950f640 1 -- 172.21.15.22:0/3286205121 shutdown_connections 2024-09-19T08:16:48.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:48.327+0000 7f188950f640 1 -- 172.21.15.22:0/3286205121 wait complete. 2024-09-19T08:16:48.352 INFO:tasks.workunit.client.0.smithi022.stdout:checking how to run the C preprocessor... gcc -E 2024-09-19T08:16:48.396 INFO:tasks.workunit.client.0.smithi022.stdout:checking for egrep... grep -E 2024-09-19T08:16:48.403 INFO:tasks.workunit.client.0.smithi022.stdout:checking for AIX... no 2024-09-19T08:16:48.470 INFO:tasks.workunit.client.0.smithi022.stdout:checking for library containing strerror... none required 2024-09-19T08:16:48.581 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:48.640 INFO:tasks.workunit.client.0.smithi022.stdout:checking for ANSI C header files... yes 2024-09-19T08:16:48.686 INFO:tasks.workunit.client.0.smithi022.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-19T08:16:48.744 INFO:tasks.workunit.client.0.smithi022.stdout:checking for sys/types.h... yes 2024-09-19T08:16:48.795 INFO:tasks.workunit.client.0.smithi022.stdout:checking for sys/stat.h... yes 2024-09-19T08:16:48.845 INFO:tasks.workunit.client.0.smithi022.stdout:checking for stdlib.h... yes 2024-09-19T08:16:48.897 INFO:tasks.workunit.client.0.smithi022.stdout:checking for string.h... yes 2024-09-19T08:16:48.967 INFO:tasks.workunit.client.0.smithi022.stdout:checking for memory.h... yes 2024-09-19T08:16:49.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:48 smithi022 ceph-mon[30963]: pgmap v56: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-09-19T08:16:49.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:48 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:16:49.018 INFO:tasks.workunit.client.0.smithi022.stdout:checking for strings.h... yes 2024-09-19T08:16:49.073 INFO:tasks.workunit.client.0.smithi022.stdout:checking for inttypes.h... yes 2024-09-19T08:16:49.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:48 smithi064 ceph-mon[34900]: pgmap v56: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-09-19T08:16:49.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:48 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:16:49.123 INFO:tasks.workunit.client.0.smithi022.stdout:checking for stdint.h... yes 2024-09-19T08:16:49.159 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:16:49.160 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:16:49.176 INFO:tasks.workunit.client.0.smithi022.stdout:checking for unistd.h... yes 2024-09-19T08:16:49.188 INFO:tasks.workunit.client.0.smithi022.stdout:checking for memory.h... (cached) yes 2024-09-19T08:16:49.193 INFO:tasks.workunit.client.0.smithi022.stdout:checking for strings.h... (cached) yes 2024-09-19T08:16:49.198 INFO:tasks.workunit.client.0.smithi022.stdout:checking for unistd.h... (cached) yes 2024-09-19T08:16:49.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:48 smithi157 ceph-mon[35143]: pgmap v56: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-09-19T08:16:49.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:48 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:16:49.273 INFO:tasks.workunit.client.0.smithi022.stdout:checking locale.h usability... yes 2024-09-19T08:16:49.296 INFO:tasks.workunit.client.0.smithi022.stdout:checking locale.h presence... yes 2024-09-19T08:16:49.297 INFO:tasks.workunit.client.0.smithi022.stdout:checking for locale.h... yes 2024-09-19T08:16:49.312 INFO:tasks.workunit.client.0.smithi022.stdout:checking for stdlib.h... (cached) yes 2024-09-19T08:16:49.317 INFO:tasks.workunit.client.0.smithi022.stdout:checking for string.h... (cached) yes 2024-09-19T08:16:49.426 INFO:tasks.workunit.client.0.smithi022.stdout:checking sys/socket.h usability... yes 2024-09-19T08:16:49.451 INFO:tasks.workunit.client.0.smithi022.stdout:checking sys/socket.h presence... yes 2024-09-19T08:16:49.452 INFO:tasks.workunit.client.0.smithi022.stdout:checking for sys/socket.h... yes 2024-09-19T08:16:49.505 INFO:tasks.workunit.client.0.smithi022.stdout:checking sys/time.h usability... yes 2024-09-19T08:16:49.525 INFO:tasks.workunit.client.0.smithi022.stdout:checking sys/time.h presence... yes 2024-09-19T08:16:49.525 INFO:tasks.workunit.client.0.smithi022.stdout:checking for sys/time.h... yes 2024-09-19T08:16:49.578 INFO:tasks.workunit.client.0.smithi022.stdout:checking fcntl.h usability... yes 2024-09-19T08:16:49.598 INFO:tasks.workunit.client.0.smithi022.stdout:checking fcntl.h presence... yes 2024-09-19T08:16:49.599 INFO:tasks.workunit.client.0.smithi022.stdout:checking for fcntl.h... yes 2024-09-19T08:16:49.695 INFO:tasks.workunit.client.0.smithi022.stdout:checking sys/fcntl.h usability... yes 2024-09-19T08:16:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.648+0000 7fe7809d7640 1 -- 172.21.15.22:0/64639436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe77c10e480 msgr2=0x7fe77c110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.648+0000 7fe7809d7640 1 --2- 172.21.15.22:0/64639436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe77c10e480 0x7fe77c110870 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe774009ca0 tx=0x7fe77402a8c0 comp rx=0 tx=0).stop 2024-09-19T08:16:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.648+0000 7fe7809d7640 1 -- 172.21.15.22:0/64639436 shutdown_connections 2024-09-19T08:16:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.648+0000 7fe7809d7640 1 --2- 172.21.15.22:0/64639436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe77c10e480 0x7fe77c110870 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.648+0000 7fe7809d7640 1 --2- 172.21.15.22:0/64639436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77c101280 0x7fe77c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.648+0000 7fe7809d7640 1 --2- 172.21.15.22:0/64639436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c1008d0 0x7fe77c100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.648+0000 7fe7809d7640 1 -- 172.21.15.22:0/64639436 >> 172.21.15.22:0/64639436 conn(0x7fe77c0fc7a0 msgr2=0x7fe77c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.648+0000 7fe7809d7640 1 -- 172.21.15.22:0/64639436 shutdown_connections 2024-09-19T08:16:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.648+0000 7fe7809d7640 1 -- 172.21.15.22:0/64639436 wait complete. 2024-09-19T08:16:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.649+0000 7fe7809d7640 1 Processor -- start 2024-09-19T08:16:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.649+0000 7fe7809d7640 1 -- start start 2024-09-19T08:16:49.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.649+0000 7fe7809d7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77c1008d0 0x7fe77c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:49.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.649+0000 7fe7809d7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c101280 0x7fe77c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:49.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.649+0000 7fe7809d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe77c10e480 0x7fe77c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:49.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.649+0000 7fe7809d7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c114180 con 0x7fe77c10e480 2024-09-19T08:16:49.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.649+0000 7fe7809d7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c114000 con 0x7fe77c101280 2024-09-19T08:16:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.649+0000 7fe7809d7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c114300 con 0x7fe77c1008d0 2024-09-19T08:16:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe77a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77c1008d0 0x7fe77c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe77ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe77c10e480 0x7fe77c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe77a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77c1008d0 0x7fe77c19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:53598/0 (socket says 172.21.15.22:53598) 2024-09-19T08:16:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe77a59c640 1 -- 172.21.15.22:0/91087768 learned_addr learned my addr 172.21.15.22:0/91087768 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe773fff640 1 --2- 172.21.15.22:0/91087768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c101280 0x7fe77c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:49.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe77a59c640 1 -- 172.21.15.22:0/91087768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c101280 msgr2=0x7fe77c19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:49.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe77a59c640 1 --2- 172.21.15.22:0/91087768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c101280 0x7fe77c19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:49.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe77a59c640 1 -- 172.21.15.22:0/91087768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe77c10e480 msgr2=0x7fe77c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:49.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe77a59c640 1 --2- 172.21.15.22:0/91087768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe77c10e480 0x7fe77c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:49.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe77a59c640 1 -- 172.21.15.22:0/91087768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77c19f650 con 0x7fe77c1008d0 2024-09-19T08:16:49.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe77ad9d640 1 --2- 172.21.15.22:0/91087768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe77c10e480 0x7fe77c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:49.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe77a59c640 1 --2- 172.21.15.22:0/91087768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77c1008d0 0x7fe77c19a6a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00e9f0 tx=0x7fe76c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:49.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe7737fe640 1 -- 172.21.15.22:0/91087768 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00cd80 con 0x7fe77c1008d0 2024-09-19T08:16:49.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe7809d7640 1 -- 172.21.15.22:0/91087768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe77c19f940 con 0x7fe77c1008d0 2024-09-19T08:16:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe7737fe640 1 -- 172.21.15.22:0/91087768 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe76c010910 con 0x7fe77c1008d0 2024-09-19T08:16:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe7737fe640 1 -- 172.21.15.22:0/91087768 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c0188f0 con 0x7fe77c1008d0 2024-09-19T08:16:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.650+0000 7fe7809d7640 1 -- 172.21.15.22:0/91087768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe77c1a7220 con 0x7fe77c1008d0 2024-09-19T08:16:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.651+0000 7fe7809d7640 1 -- 172.21.15.22:0/91087768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe77c100cb0 con 0x7fe77c1008d0 2024-09-19T08:16:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.652+0000 7fe7737fe640 1 -- 172.21.15.22:0/91087768 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe76c010ac0 con 0x7fe77c1008d0 2024-09-19T08:16:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.653+0000 7fe7737fe640 1 --2- 172.21.15.22:0/91087768 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe7680a33b0 0x7fe7680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.653+0000 7fe773fff640 1 --2- 172.21.15.22:0/91087768 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe7680a33b0 0x7fe7680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.653+0000 7fe7737fe640 1 -- 172.21.15.22:0/91087768 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe76c0d0100 con 0x7fe77c1008d0 2024-09-19T08:16:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.654+0000 7fe773fff640 1 --2- 172.21.15.22:0/91087768 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe7680a33b0 0x7fe7680a5870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe77c1102c0 tx=0x7fe764005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.658+0000 7fe7737fe640 1 -- 172.21.15.22:0/91087768 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe76c09c440 con 0x7fe77c1008d0 2024-09-19T08:16:49.717 INFO:tasks.workunit.client.0.smithi022.stdout:checking sys/fcntl.h presence... yes 2024-09-19T08:16:49.717 INFO:tasks.workunit.client.0.smithi022.stdout:checking for sys/fcntl.h... yes 2024-09-19T08:16:49.742 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:49.838 INFO:tasks.workunit.client.0.smithi022.stdout:checking sys/param.h usability... yes 2024-09-19T08:16:49.860 INFO:tasks.workunit.client.0.smithi022.stdout:checking sys/param.h presence... yes 2024-09-19T08:16:49.861 INFO:tasks.workunit.client.0.smithi022.stdout:checking for sys/param.h... yes 2024-09-19T08:16:49.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.889+0000 7fe7809d7640 1 -- 172.21.15.22:0/91087768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe77c109640 con 0x7fe77c1008d0 2024-09-19T08:16:49.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.890+0000 7fe7737fe640 1 -- 172.21.15.22:0/91087768 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11907 (secure 0 0 0) 0x7fe76c0a0740 con 0x7fe77c1008d0 2024-09-19T08:16:49.890 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:16:49.890 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"btime":"2024-09-19T08:16:47:822061+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:16:46.867495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:16:49.891 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-09-19T08:16:49.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.892+0000 7fe7809d7640 1 -- 172.21.15.22:0/91087768 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe7680a33b0 msgr2=0x7fe7680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:49.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.892+0000 7fe7809d7640 1 --2- 172.21.15.22:0/91087768 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe7680a33b0 0x7fe7680a5870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe77c1102c0 tx=0x7fe764005f90 comp rx=0 tx=0).stop 2024-09-19T08:16:49.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.892+0000 7fe7809d7640 1 -- 172.21.15.22:0/91087768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77c1008d0 msgr2=0x7fe77c19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:49.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.892+0000 7fe7809d7640 1 --2- 172.21.15.22:0/91087768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77c1008d0 0x7fe77c19a6a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00e9f0 tx=0x7fe76c00eeb0 comp rx=0 tx=0).stop 2024-09-19T08:16:49.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.892+0000 7fe7809d7640 1 -- 172.21.15.22:0/91087768 shutdown_connections 2024-09-19T08:16:49.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.892+0000 7fe7809d7640 1 --2- 172.21.15.22:0/91087768 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe7680a33b0 0x7fe7680a5870 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:49.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.892+0000 7fe7809d7640 1 --2- 172.21.15.22:0/91087768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe77c10e480 0x7fe77c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:49.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.892+0000 7fe7809d7640 1 --2- 172.21.15.22:0/91087768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe77c101280 0x7fe77c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:49.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.892+0000 7fe7809d7640 1 --2- 172.21.15.22:0/91087768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77c1008d0 0x7fe77c19a6a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:49.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.892+0000 7fe7809d7640 1 -- 172.21.15.22:0/91087768 >> 172.21.15.22:0/91087768 conn(0x7fe77c0fc7a0 msgr2=0x7fe77c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:49.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.892+0000 7fe7809d7640 1 -- 172.21.15.22:0/91087768 shutdown_connections 2024-09-19T08:16:49.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:49.892+0000 7fe7809d7640 1 -- 172.21.15.22:0/91087768 wait complete. 2024-09-19T08:16:49.913 INFO:tasks.workunit.client.0.smithi022.stdout:checking ioctl.h usability... no 2024-09-19T08:16:49.934 INFO:tasks.workunit.client.0.smithi022.stdout:checking ioctl.h presence... no 2024-09-19T08:16:49.934 INFO:tasks.workunit.client.0.smithi022.stdout:checking for ioctl.h... no 2024-09-19T08:16:50.001 INFO:tasks.workunit.client.0.smithi022.stdout:checking sys/ioctl.h usability... yes 2024-09-19T08:16:50.022 INFO:tasks.workunit.client.0.smithi022.stdout:checking sys/ioctl.h presence... yes 2024-09-19T08:16:50.022 INFO:tasks.workunit.client.0.smithi022.stdout:checking for sys/ioctl.h... yes 2024-09-19T08:16:50.107 INFO:tasks.workunit.client.0.smithi022.stdout:checking return type of signal handlers... void 2024-09-19T08:16:50.152 INFO:tasks.workunit.client.0.smithi022.stdout:checking for size_t... yes 2024-09-19T08:16:50.193 INFO:tasks.workunit.client.0.smithi022.stdout:checking for dev_t... yes 2024-09-19T08:16:50.243 INFO:tasks.workunit.client.0.smithi022.stdout:checking for short... yes 2024-09-19T08:16:50.322 INFO:tasks.workunit.client.0.smithi022.stdout:checking size of short... 2 2024-09-19T08:16:50.372 INFO:tasks.workunit.client.0.smithi022.stdout:checking for int... yes 2024-09-19T08:16:50.457 INFO:tasks.workunit.client.0.smithi022.stdout:checking size of int... 4 2024-09-19T08:16:50.521 INFO:tasks.workunit.client.0.smithi022.stdout:checking for long... yes 2024-09-19T08:16:50.595 INFO:tasks.workunit.client.0.smithi022.stdout:checking size of long... 8 2024-09-19T08:16:50.642 INFO:tasks.workunit.client.0.smithi022.stdout:checking for long long... yes 2024-09-19T08:16:50.730 INFO:tasks.workunit.client.0.smithi022.stdout:checking size of long long... 8 2024-09-19T08:16:50.772 INFO:tasks.workunit.client.0.smithi022.stdout:checking for ssize_t... yes 2024-09-19T08:16:50.782 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T08:16:50.783 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T08:16:50.813 INFO:tasks.workunit.client.0.smithi022.stdout:checking for sig_atomic_t... yes 2024-09-19T08:16:50.896 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:50 smithi022 ceph-mon[30963]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 5 op/s 2024-09-19T08:16:50.896 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:50 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/91087768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:50.905 INFO:tasks.workunit.client.0.smithi022.stdout:checking whether gcc needs -traditional... no 2024-09-19T08:16:50.941 INFO:tasks.workunit.client.0.smithi022.stdout:checking for an ANSI C-conforming const... yes 2024-09-19T08:16:50.966 INFO:tasks.workunit.client.0.smithi022.stdout:checking for inline... inline 2024-09-19T08:16:51.027 INFO:tasks.workunit.client.0.smithi022.stdout:checking for working alloca.h... yes 2024-09-19T08:16:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.994+0000 7fbed4b0b640 1 -- 172.21.15.22:0/308643400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbed0101e10 msgr2=0x7fbed01021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.994+0000 7fbed4b0b640 1 --2- 172.21.15.22:0/308643400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbed0101e10 0x7fbed01021f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009900 tx=0x7fbec002a410 comp rx=0 tx=0).stop 2024-09-19T08:16:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.995+0000 7fbed4b0b640 1 -- 172.21.15.22:0/308643400 shutdown_connections 2024-09-19T08:16:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.995+0000 7fbed4b0b640 1 --2- 172.21.15.22:0/308643400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbed0106f40 0x7fbed010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.995+0000 7fbed4b0b640 1 --2- 172.21.15.22:0/308643400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed01027c0 0x7fbed0106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.995+0000 7fbed4b0b640 1 --2- 172.21.15.22:0/308643400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbed0101e10 0x7fbed01021f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.995+0000 7fbed4b0b640 1 -- 172.21.15.22:0/308643400 >> 172.21.15.22:0/308643400 conn(0x7fbed00fd8d0 msgr2=0x7fbed00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.995+0000 7fbed4b0b640 1 -- 172.21.15.22:0/308643400 shutdown_connections 2024-09-19T08:16:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.995+0000 7fbed4b0b640 1 -- 172.21.15.22:0/308643400 wait complete. 2024-09-19T08:16:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.995+0000 7fbed4b0b640 1 Processor -- start 2024-09-19T08:16:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbed4b0b640 1 -- start start 2024-09-19T08:16:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbed4b0b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbed0101e10 0x7fbed0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbed4b0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed01027c0 0x7fbed0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbed4b0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbed0106f40 0x7fbed019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbed4b0b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0111fa0 con 0x7fbed0106f40 2024-09-19T08:16:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbed4b0b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0111e20 con 0x7fbed0101e10 2024-09-19T08:16:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbed4b0b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0112120 con 0x7fbed01027c0 2024-09-19T08:16:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbeced9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbed0106f40 0x7fbed019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbeced9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbed0106f40 0x7fbed019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49182/0 (socket says 172.21.15.22:49182) 2024-09-19T08:16:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbeced9d640 1 -- 172.21.15.22:0/4128202109 learned_addr learned my addr 172.21.15.22:0/4128202109 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbece59c640 1 --2- 172.21.15.22:0/4128202109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbed0101e10 0x7fbed0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbecdd9b640 1 --2- 172.21.15.22:0/4128202109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed01027c0 0x7fbed0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbeced9d640 1 -- 172.21.15.22:0/4128202109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed01027c0 msgr2=0x7fbed0198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbeced9d640 1 --2- 172.21.15.22:0/4128202109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed01027c0 0x7fbed0198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbeced9d640 1 -- 172.21.15.22:0/4128202109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbed0101e10 msgr2=0x7fbed0198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbeced9d640 1 --2- 172.21.15.22:0/4128202109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbed0101e10 0x7fbed0198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.996+0000 7fbeced9d640 1 -- 172.21.15.22:0/4128202109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed019d3d0 con 0x7fbed0106f40 2024-09-19T08:16:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.997+0000 7fbeced9d640 1 --2- 172.21.15.22:0/4128202109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbed0106f40 0x7fbed019ccf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbec800eb10 tx=0x7fbec800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.997+0000 7fbebf7fe640 1 -- 172.21.15.22:0/4128202109 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec8004b20 con 0x7fbed0106f40 2024-09-19T08:16:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.997+0000 7fbed4b0b640 1 -- 172.21.15.22:0/4128202109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed019d6c0 con 0x7fbed0106f40 2024-09-19T08:16:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.997+0000 7fbebf7fe640 1 -- 172.21.15.22:0/4128202109 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbec8004cc0 con 0x7fbed0106f40 2024-09-19T08:16:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.997+0000 7fbebf7fe640 1 -- 172.21.15.22:0/4128202109 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec8010760 con 0x7fbed0106f40 2024-09-19T08:16:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.997+0000 7fbed4b0b640 1 -- 172.21.15.22:0/4128202109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed01a4fa0 con 0x7fbed0106f40 2024-09-19T08:16:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.998+0000 7fbed4b0b640 1 -- 172.21.15.22:0/4128202109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed01021f0 con 0x7fbed0106f40 2024-09-19T08:16:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:50.999+0000 7fbebf7fe640 1 -- 172.21.15.22:0/4128202109 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbec8005020 con 0x7fbed0106f40 2024-09-19T08:16:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.000+0000 7fbebf7fe640 1 --2- 172.21.15.22:0/4128202109 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbeb80a33b0 0x7fbeb80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.000+0000 7fbebf7fe640 1 -- 172.21.15.22:0/4128202109 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fbec80d0200 con 0x7fbed0106f40 2024-09-19T08:16:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.000+0000 7fbece59c640 1 --2- 172.21.15.22:0/4128202109 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbeb80a33b0 0x7fbeb80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.001+0000 7fbece59c640 1 --2- 172.21.15.22:0/4128202109 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbeb80a33b0 0x7fbeb80a5870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009230 tx=0x7fbec00058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.005+0000 7fbebf7fe640 1 -- 172.21.15.22:0/4128202109 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbec809cbf0 con 0x7fbed0106f40 2024-09-19T08:16:51.078 INFO:tasks.workunit.client.0.smithi022.stdout:checking for alloca... yes 2024-09-19T08:16:51.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:50 smithi064 ceph-mon[34900]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 5 op/s 2024-09-19T08:16:51.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:50 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/91087768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:51.119 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:51.148 INFO:tasks.workunit.client.0.smithi022.stdout:checking for working memcmp... yes 2024-09-19T08:16:51.202 INFO:tasks.workunit.client.0.smithi022.stdout:checking for strftime... yes 2024-09-19T08:16:51.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.251+0000 7fbed4b0b640 1 -- 172.21.15.22:0/4128202109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbed010a180 con 0x7fbed0106f40 2024-09-19T08:16:51.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.252+0000 7fbebf7fe640 1 -- 172.21.15.22:0/4128202109 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11907 (secure 0 0 0) 0x7fbec809c4f0 con 0x7fbed0106f40 2024-09-19T08:16:51.252 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:16:51.252 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"btime":"2024-09-19T08:16:47:822061+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:16:46.867495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:16:51.252 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-09-19T08:16:51.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:50 smithi157 ceph-mon[35143]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 5 op/s 2024-09-19T08:16:51.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:50 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/91087768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:51.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.255+0000 7fbed4b0b640 1 -- 172.21.15.22:0/4128202109 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbeb80a33b0 msgr2=0x7fbeb80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:51.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.255+0000 7fbed4b0b640 1 --2- 172.21.15.22:0/4128202109 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbeb80a33b0 0x7fbeb80a5870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009230 tx=0x7fbec00058d0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.255+0000 7fbed4b0b640 1 -- 172.21.15.22:0/4128202109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbed0106f40 msgr2=0x7fbed019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:51.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.255+0000 7fbed4b0b640 1 --2- 172.21.15.22:0/4128202109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbed0106f40 0x7fbed019ccf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbec800eb10 tx=0x7fbec800c2d0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.255+0000 7fbed4b0b640 1 -- 172.21.15.22:0/4128202109 shutdown_connections 2024-09-19T08:16:51.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.255+0000 7fbed4b0b640 1 --2- 172.21.15.22:0/4128202109 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbeb80a33b0 0x7fbeb80a5870 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.255+0000 7fbed4b0b640 1 --2- 172.21.15.22:0/4128202109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbed0106f40 0x7fbed019ccf0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.255+0000 7fbed4b0b640 1 --2- 172.21.15.22:0/4128202109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed01027c0 0x7fbed0198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.255+0000 7fbed4b0b640 1 --2- 172.21.15.22:0/4128202109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbed0101e10 0x7fbed0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.255+0000 7fbed4b0b640 1 -- 172.21.15.22:0/4128202109 >> 172.21.15.22:0/4128202109 conn(0x7fbed00fd8d0 msgr2=0x7fbed00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:51.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.255+0000 7fbed4b0b640 1 -- 172.21.15.22:0/4128202109 shutdown_connections 2024-09-19T08:16:51.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.255+0000 7fbed4b0b640 1 -- 172.21.15.22:0/4128202109 wait complete. 2024-09-19T08:16:51.288 INFO:tasks.workunit.client.0.smithi022.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-09-19T08:16:51.348 INFO:tasks.workunit.client.0.smithi022.stdout:checking whether stat accepts an empty string... no 2024-09-19T08:16:51.404 INFO:tasks.workunit.client.0.smithi022.stdout:checking for vprintf... yes 2024-09-19T08:16:51.465 INFO:tasks.workunit.client.0.smithi022.stdout:checking for _doprnt... no 2024-09-19T08:16:51.486 INFO:tasks.workunit.client.0.smithi022.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-19T08:16:51.498 INFO:tasks.workunit.client.0.smithi022.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-19T08:16:51.647 INFO:tasks.workunit.client.0.smithi022.stdout:checking for the pthreads library -lpthreads... no 2024-09-19T08:16:51.702 INFO:tasks.workunit.client.0.smithi022.stdout:checking whether pthreads work without any flags... yes 2024-09-19T08:16:51.748 INFO:tasks.workunit.client.0.smithi022.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-09-19T08:16:51.748 INFO:tasks.workunit.client.0.smithi022.stdout:checking if more special flags are required for pthreads... no 2024-09-19T08:16:51.748 INFO:tasks.workunit.client.0.smithi022.stdout:checking for cc_r... gcc 2024-09-19T08:16:51.821 INFO:tasks.workunit.client.0.smithi022.stdout:checking for setrlimit... yes 2024-09-19T08:16:51.885 INFO:tasks.workunit.client.0.smithi022.stdout:checking for setproctitle... no 2024-09-19T08:16:51.946 INFO:tasks.workunit.client.0.smithi022.stdout:checking for getopt_long... yes 2024-09-19T08:16:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.934+0000 7f62793ce640 1 -- 172.21.15.22:0/3291632856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274105560 msgr2=0x7f627410d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.934+0000 7f62793ce640 1 --2- 172.21.15.22:0/3291632856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274105560 0x7f627410d0e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f626c009ca0 tx=0x7f626c02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.934+0000 7f62793ce640 1 -- 172.21.15.22:0/3291632856 shutdown_connections 2024-09-19T08:16:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.934+0000 7f62793ce640 1 --2- 172.21.15.22:0/3291632856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274105560 0x7f627410d0e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.934+0000 7f62793ce640 1 --2- 172.21.15.22:0/3291632856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6274100de0 0x7f6274104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.934+0000 7f62793ce640 1 --2- 172.21.15.22:0/3291632856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6274100430 0x7f6274100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.934+0000 7f62793ce640 1 -- 172.21.15.22:0/3291632856 >> 172.21.15.22:0/3291632856 conn(0x7f6274077230 msgr2=0x7f62740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.934+0000 7f62793ce640 1 -- 172.21.15.22:0/3291632856 shutdown_connections 2024-09-19T08:16:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.934+0000 7f62793ce640 1 -- 172.21.15.22:0/3291632856 wait complete. 2024-09-19T08:16:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.935+0000 7f62793ce640 1 Processor -- start 2024-09-19T08:16:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.935+0000 7f62793ce640 1 -- start start 2024-09-19T08:16:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f62793ce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274100430 0x7f62741992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f62793ce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6274100de0 0x7f6274199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f62793ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6274105560 0x7f62741a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f62793ce640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6274110570 con 0x7f6274100430 2024-09-19T08:16:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f62793ce640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62741103f0 con 0x7f6274105560 2024-09-19T08:16:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f62793ce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f62741106f0 con 0x7f6274100de0 2024-09-19T08:16:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f6272ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274100430 0x7f62741992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f6272ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274100430 0x7f62741992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49202/0 (socket says 172.21.15.22:49202) 2024-09-19T08:16:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f626a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6274100de0 0x7f6274199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f6272ffd640 1 -- 172.21.15.22:0/369378952 learned_addr learned my addr 172.21.15.22:0/369378952 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f62737fe640 1 --2- 172.21.15.22:0/369378952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6274105560 0x7f62741a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f6272ffd640 1 -- 172.21.15.22:0/369378952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6274100de0 msgr2=0x7f6274199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f6272ffd640 1 --2- 172.21.15.22:0/369378952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6274100de0 0x7f6274199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f6272ffd640 1 -- 172.21.15.22:0/369378952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6274105560 msgr2=0x7f62741a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f6272ffd640 1 --2- 172.21.15.22:0/369378952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6274105560 0x7f62741a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f6272ffd640 1 -- 172.21.15.22:0/369378952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62741a4890 con 0x7f6274100430 2024-09-19T08:16:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f626a7fc640 1 --2- 172.21.15.22:0/369378952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6274100de0 0x7f6274199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f6272ffd640 1 --2- 172.21.15.22:0/369378952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274100430 0x7f62741992d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f626400d950 tx=0x7f626400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f6270ff9640 1 -- 172.21.15.22:0/369378952 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264017070 con 0x7f6274100430 2024-09-19T08:16:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f6270ff9640 1 -- 172.21.15.22:0/369378952 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6264009c30 con 0x7f6274100430 2024-09-19T08:16:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.936+0000 7f62793ce640 1 -- 172.21.15.22:0/369378952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62741a4a60 con 0x7f6274100430 2024-09-19T08:16:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.937+0000 7f6270ff9640 1 -- 172.21.15.22:0/369378952 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626400f6b0 con 0x7f6274100430 2024-09-19T08:16:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.937+0000 7f62793ce640 1 -- 172.21.15.22:0/369378952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62741a4fa0 con 0x7f6274100430 2024-09-19T08:16:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.939+0000 7f6270ff9640 1 -- 172.21.15.22:0/369378952 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f62640109d0 con 0x7f6274100430 2024-09-19T08:16:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.939+0000 7f627b7a7740 1 Processor -- start 2024-09-19T08:16:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.939+0000 7f627b7a7740 1 -- start start 2024-09-19T08:16:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f627b7a7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106bf8850 0x557106bf8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f627b7a7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf9170 0x557106c034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f627b7a7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106c03a30 0x557106c05e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f627b7a7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557106b5d160 con 0x557106bf9170 2024-09-19T08:16:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f627b7a7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557106b04720 con 0x557106bf8850 2024-09-19T08:16:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f627b7a7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557106b59120 con 0x557106c03a30 2024-09-19T08:16:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f6272ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf9170 0x557106c034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f626a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106c03a30 0x557106c05e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f626a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106c03a30 0x557106c05e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:53614/0 (socket says 172.21.15.22:53614) 2024-09-19T08:16:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f626a7fc640 1 -- 172.21.15.22:0/4017516928 learned_addr learned my addr 172.21.15.22:0/4017516928 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f62737fe640 1 --2- 172.21.15.22:0/4017516928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106bf8850 0x557106bf8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f6270ff9640 1 --2- 172.21.15.22:0/369378952 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f62600a3430 0x7f62600a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f626a7fc640 1 -- 172.21.15.22:0/4017516928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106bf8850 msgr2=0x557106bf8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f626a7fc640 1 --2- 172.21.15.22:0/4017516928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106bf8850 0x557106bf8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f626a7fc640 1 -- 172.21.15.22:0/4017516928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf9170 msgr2=0x557106c034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f626a7fc640 1 --2- 172.21.15.22:0/4017516928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf9170 0x557106c034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f626a7fc640 1 -- 172.21.15.22:0/4017516928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571068ceba0 con 0x557106c03a30 2024-09-19T08:16:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f62737fe640 1 --2- 172.21.15.22:0/4017516928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106bf8850 0x557106bf8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f62737fe640 1 --2- 172.21.15.22:0/369378952 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f62600a3430 0x7f62600a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f6270ff9640 1 -- 172.21.15.22:0/369378952 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f62640cf8b0 con 0x7f6274100430 2024-09-19T08:16:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.940+0000 7f626a7fc640 1 --2- 172.21.15.22:0/4017516928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106c03a30 0x557106c05e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f625c011470 tx=0x7f625c011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cb1a51f12d9ed3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.941+0000 7f62697fa640 1 -- 172.21.15.22:0/4017516928 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c01e070 con 0x557106c03a30 2024-09-19T08:16:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.941+0000 7f62697fa640 1 -- 172.21.15.22:0/4017516928 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c0035f0 con 0x557106c03a30 2024-09-19T08:16:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.941+0000 7f62697fa640 1 -- 172.21.15.22:0/4017516928 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c00f860 con 0x557106c03a30 2024-09-19T08:16:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.941+0000 7f627b7a7740 1 -- 172.21.15.22:0/4017516928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106c03a30 msgr2=0x557106c05e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.941+0000 7f62737fe640 1 --2- 172.21.15.22:0/369378952 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f62600a3430 0x7f62600a58f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f626c005e30 tx=0x7f626c043040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.941+0000 7f627b7a7740 1 --2- 172.21.15.22:0/4017516928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106c03a30 0x557106c05e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f625c011470 tx=0x7f625c011930 comp rx=0 tx=0).stop 2024-09-19T08:16:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.941+0000 7f627b7a7740 1 -- 172.21.15.22:0/4017516928 shutdown_connections 2024-09-19T08:16:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.941+0000 7f627b7a7740 1 --2- 172.21.15.22:0/4017516928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106c03a30 0x557106c05e20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.941+0000 7f627b7a7740 1 --2- 172.21.15.22:0/4017516928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf9170 0x557106c034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.941+0000 7f627b7a7740 1 --2- 172.21.15.22:0/4017516928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106bf8850 0x557106bf8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f627b7a7740 1 -- 172.21.15.22:0/4017516928 >> 172.21.15.22:0/4017516928 conn(0x557106bf4de0 msgr2=0x557106bf51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f627b7a7740 1 -- 172.21.15.22:0/4017516928 shutdown_connections 2024-09-19T08:16:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f627b7a7740 1 -- 172.21.15.22:0/4017516928 wait complete. 2024-09-19T08:16:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f627b7a7740 1 -- start start 2024-09-19T08:16:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f627b7a7740 1 Processor -- start 2024-09-19T08:16:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f627b7a7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf8850 0x557106c008d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f627b7a7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106bf9170 0x557106c00e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f627b7a7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106c03a30 0x557106c01350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f627b7a7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557106c06600 con 0x557106bf8850 2024-09-19T08:16:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f627b7a7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557106ae5900 con 0x557106c03a30 2024-09-19T08:16:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f627b7a7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557106c06780 con 0x557106bf9170 2024-09-19T08:16:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f627b7a7740 10 client.0 _ll_register_callbacks cb 0x557106ae3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f6272ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf8850 0x557106c008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f6272ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf8850 0x557106c008d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49230/0 (socket says 172.21.15.22:49230) 2024-09-19T08:16:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f6272ffd640 1 -- 172.21.15.22:0/1681520115 learned_addr learned my addr 172.21.15.22:0/1681520115 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.942+0000 7f626a7fc640 1 --2- 172.21.15.22:0/1681520115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106bf9170 0x557106c00e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f62737fe640 1 --2- 172.21.15.22:0/1681520115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106c03a30 0x557106c01350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f6272ffd640 1 -- 172.21.15.22:0/1681520115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106bf9170 msgr2=0x557106c00e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f6272ffd640 1 --2- 172.21.15.22:0/1681520115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106bf9170 0x557106c00e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f6272ffd640 1 -- 172.21.15.22:0/1681520115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106c03a30 msgr2=0x557106c01350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f6272ffd640 1 --2- 172.21.15.22:0/1681520115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106c03a30 0x557106c01350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f6272ffd640 1 -- 172.21.15.22:0/1681520115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557106b25c50 con 0x557106bf8850 2024-09-19T08:16:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f62737fe640 1 --2- 172.21.15.22:0/1681520115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106c03a30 0x557106c01350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:16:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f6272ffd640 1 --2- 172.21.15.22:0/1681520115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf8850 0x557106c008d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f626409bb80 tx=0x7f62640d3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2396372427cc88b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f625a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f625a7fc640 1 -- 172.21.15.22:0/1681520115 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264020040 con 0x557106bf8850 2024-09-19T08:16:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f625a7fc640 1 -- 172.21.15.22:0/1681520115 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f626409b040 con 0x557106bf8850 2024-09-19T08:16:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f627b7a7740 1 -- 172.21.15.22:0/1681520115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f625c0035f0 con 0x557106bf8850 2024-09-19T08:16:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f625a7fc640 1 -- 172.21.15.22:0/1681520115 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f62640cd430 con 0x557106bf8850 2024-09-19T08:16:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f627b7a7740 10 client.15789 fetch_fsmap learned FSMap version 26 2024-09-19T08:16:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f627b7a7740 1 -- 172.21.15.22:0/1681520115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557106c12290 con 0x557106bf8850 2024-09-19T08:16:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.943+0000 7f625a7fc640 1 -- 172.21.15.22:0/1681520115 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c0035f0 con 0x557106bf8850 2024-09-19T08:16:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.944+0000 7f625a7fc640 1 -- 172.21.15.22:0/1681520115 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f6264097940 con 0x557106bf8850 2024-09-19T08:16:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.944+0000 7f627b7a7740 10 client.15789 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:16:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.944+0000 7f627b7a7740 10 client.15789 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:16:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.944+0000 7f627b7a7740 20 client.15789 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.944+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1681520115 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557106c136a0 0x557106c15ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.944+0000 7f627b7a7740 4 client.15789 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:16:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.944+0000 7f627b7a7740 1 -- 172.21.15.22:0/1681520115 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557106b6a890 con 0x557106c136a0 2024-09-19T08:16:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.944+0000 7f626a7fc640 1 --2- 172.21.15.22:0/1681520115 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557106c136a0 0x557106c15ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.944+0000 7f625a7fc640 10 client.15789 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.944+0000 7f626a7fc640 1 --2- 172.21.15.22:0/1681520115 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557106c136a0 0x557106c15ab0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17a5967e11e05454 server_cookie=4abd940d1d90c605 in_seq=0 out_seq=0 2024-09-19T08:16:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f625a7fc640 1 -- 172.21.15.22:0/1681520115 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557106b6a890 con 0x557106c136a0 2024-09-19T08:16:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f625a7fc640 10 client.15789 handle_command_reply: tid=0 2024-09-19T08:16:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f627b7a7740 1 client.15789 shutdown 2024-09-19T08:16:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f627b7a7740 1 -- 172.21.15.22:0/1681520115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf8850 msgr2=0x557106c008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1681520115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf8850 0x557106c008d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f626409bb80 tx=0x7f62640d3040 comp rx=0 tx=0).stop 2024-09-19T08:16:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f627b7a7740 1 -- 172.21.15.22:0/1681520115 shutdown_connections 2024-09-19T08:16:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1681520115 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557106c136a0 0x557106c15ab0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1681520115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106c03a30 0x557106c01350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f625a7fc640 0 client.15789 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1681520115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106bf9170 0x557106c00e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1681520115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf8850 0x557106c008d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f627b7a7740 1 -- 172.21.15.22:0/1681520115 >> 172.21.15.22:0/1681520115 conn(0x557106bf4de0 msgr2=0x557106bfab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f627b7a7740 1 -- 172.21.15.22:0/1681520115 shutdown_connections 2024-09-19T08:16:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f627b7a7740 1 -- 172.21.15.22:0/1681520115 wait complete. 2024-09-19T08:16:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.947+0000 7f627b7a7740 20 client.15789 trim_cache size 0 max 16384 2024-09-19T08:16:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.966+0000 7f627b7a7740 1 Processor -- start 2024-09-19T08:16:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.966+0000 7f627b7a7740 1 -- start start 2024-09-19T08:16:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.966+0000 7f627b7a7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf8850 0x557106bfb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.966+0000 7f627b7a7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106bf9170 0x557106bf3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.966+0000 7f627b7a7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106c03a30 0x557106bf4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.966+0000 7f627b7a7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f62741adef0 con 0x557106bf8850 2024-09-19T08:16:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.966+0000 7f627b7a7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62741ab7c0 con 0x557106c03a30 2024-09-19T08:16:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.966+0000 7f627b7a7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f62741a9630 con 0x557106bf9170 2024-09-19T08:16:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f6272ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf8850 0x557106bfb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f626a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106bf9170 0x557106bf3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f626a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106bf9170 0x557106bf3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:53632/0 (socket says 172.21.15.22:53632) 2024-09-19T08:16:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f626a7fc640 1 -- 172.21.15.22:0/1585642647 learned_addr learned my addr 172.21.15.22:0/1585642647 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:51.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f6272ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf8850 0x557106bfb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49246/0 (socket says 172.21.15.22:49246) 2024-09-19T08:16:51.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f62737fe640 1 --2- 172.21.15.22:0/1585642647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106c03a30 0x557106bf4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f626a7fc640 1 -- 172.21.15.22:0/1585642647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106c03a30 msgr2=0x557106bf4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:51.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f626a7fc640 1 --2- 172.21.15.22:0/1585642647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106c03a30 0x557106bf4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f626a7fc640 1 -- 172.21.15.22:0/1585642647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf8850 msgr2=0x557106bfb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:51.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f626a7fc640 1 --2- 172.21.15.22:0/1585642647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf8850 0x557106bfb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f626a7fc640 1 -- 172.21.15.22:0/1585642647 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557106bf8fa0 con 0x557106bf9170 2024-09-19T08:16:51.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f62737fe640 1 --2- 172.21.15.22:0/1585642647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106c03a30 0x557106bf4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:51.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f626a7fc640 1 --2- 172.21.15.22:0/1585642647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106bf9170 0x557106bf3d20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f625c011470 tx=0x7f625c009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fcc7ce895081b18 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:51.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f6268ff9640 1 -- 172.21.15.22:0/1585642647 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c01e040 con 0x557106bf9170 2024-09-19T08:16:51.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f6268ff9640 1 -- 172.21.15.22:0/1585642647 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c02a070 con 0x557106bf9170 2024-09-19T08:16:51.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.967+0000 7f6268ff9640 1 -- 172.21.15.22:0/1585642647 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c00f930 con 0x557106bf9170 2024-09-19T08:16:51.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.968+0000 7f627b7a7740 1 -- 172.21.15.22:0/1585642647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106bf9170 msgr2=0x557106bf3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:51.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.968+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1585642647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106bf9170 0x557106bf3d20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f625c011470 tx=0x7f625c009f10 comp rx=0 tx=0).stop 2024-09-19T08:16:51.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.968+0000 7f627b7a7740 1 -- 172.21.15.22:0/1585642647 shutdown_connections 2024-09-19T08:16:51.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.968+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1585642647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106c03a30 0x557106bf4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.968+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1585642647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106bf9170 0x557106bf3d20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.968+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1585642647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf8850 0x557106bfb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.968+0000 7f627b7a7740 1 -- 172.21.15.22:0/1585642647 >> 172.21.15.22:0/1585642647 conn(0x557106bf4de0 msgr2=0x557106bf3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:51.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.968+0000 7f627b7a7740 1 -- 172.21.15.22:0/1585642647 shutdown_connections 2024-09-19T08:16:51.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.968+0000 7f627b7a7740 1 -- 172.21.15.22:0/1585642647 wait complete. 2024-09-19T08:16:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.968+0000 7f627b7a7740 1 -- start start 2024-09-19T08:16:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f627b7a7740 1 Processor -- start 2024-09-19T08:16:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f627b7a7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106bf8850 0x557106c078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f627b7a7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf9170 0x557106c07e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f627b7a7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106c03a30 0x557106c08350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f627b7a7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f62741a9c80 con 0x557106bf9170 2024-09-19T08:16:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f627b7a7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62741ab0f0 con 0x557106bf8850 2024-09-19T08:16:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f627b7a7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f62741aa350 con 0x557106c03a30 2024-09-19T08:16:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f627b7a7740 10 client.0 _ll_register_callbacks cb 0x7f626400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f626a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf9170 0x557106c07e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f626a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf9170 0x557106c07e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49260/0 (socket says 172.21.15.22:49260) 2024-09-19T08:16:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f626a7fc640 1 -- 172.21.15.22:0/1848901377 learned_addr learned my addr 172.21.15.22:0/1848901377 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f62737fe640 1 --2- 172.21.15.22:0/1848901377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106c03a30 0x557106c08350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f6272ffd640 1 --2- 172.21.15.22:0/1848901377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106bf8850 0x557106c078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f626a7fc640 1 -- 172.21.15.22:0/1848901377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106c03a30 msgr2=0x557106c08350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f626a7fc640 1 --2- 172.21.15.22:0/1848901377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106c03a30 0x557106c08350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f626a7fc640 1 -- 172.21.15.22:0/1848901377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106bf8850 msgr2=0x557106c078d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f626a7fc640 1 --2- 172.21.15.22:0/1848901377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106bf8850 0x557106c078d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.969+0000 7f626a7fc640 1 -- 172.21.15.22:0/1848901377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557106c7d4f0 con 0x557106bf9170 2024-09-19T08:16:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f626a7fc640 1 --2- 172.21.15.22:0/1848901377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf9170 0x557106c07e10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f625c011e30 tx=0x7f625c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e9b6e5a629a63c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f625a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f625a7fc640 1 -- 172.21.15.22:0/1848901377 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c01e070 con 0x557106bf9170 2024-09-19T08:16:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f625a7fc640 1 -- 172.21.15.22:0/1848901377 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f625c0039a0 con 0x557106bf9170 2024-09-19T08:16:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f627b7a7740 1 -- 172.21.15.22:0/1848901377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f625c02a070 con 0x557106bf9170 2024-09-19T08:16:51.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f625a7fc640 1 -- 172.21.15.22:0/1848901377 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f625c003bc0 con 0x557106bf9170 2024-09-19T08:16:51.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f627b7a7740 10 client.15795 fetch_fsmap learned FSMap version 26 2024-09-19T08:16:51.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f627b7a7740 1 -- 172.21.15.22:0/1848901377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557106c7e770 con 0x557106bf9170 2024-09-19T08:16:51.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f625a7fc640 1 -- 172.21.15.22:0/1848901377 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c02a070 con 0x557106bf9170 2024-09-19T08:16:51.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f625a7fc640 1 -- 172.21.15.22:0/1848901377 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f625c03a4a0 con 0x557106bf9170 2024-09-19T08:16:51.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f627b7a7740 10 client.15795 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:16:51.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f627b7a7740 10 client.15795 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:16:51.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f627b7a7740 20 client.15795 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:51.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1848901377 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557106c7ece0 0x557106c81100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f627b7a7740 4 client.15795 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T08:16:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.970+0000 7f627b7a7740 1 -- 172.21.15.22:0/1848901377 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x557106b64be0 con 0x557106c7ece0 2024-09-19T08:16:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.971+0000 7f6272ffd640 1 --2- 172.21.15.22:0/1848901377 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557106c7ece0 0x557106c81100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.971+0000 7f6272ffd640 1 --2- 172.21.15.22:0/1848901377 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557106c7ece0 0x557106c81100 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=397c758d4e2221ab server_cookie=695f03b6945f2a95 in_seq=0 out_seq=0 2024-09-19T08:16:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:51.971+0000 7f625a7fc640 10 client.15795 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.029+0000 7f625a7fc640 1 -- 172.21.15.22:0/1848901377 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557106b64be0 con 0x557106c7ece0 2024-09-19T08:16:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.029+0000 7f625a7fc640 10 client.15795 handle_command_reply: tid=0 2024-09-19T08:16:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.029+0000 7f627b7a7740 1 client.15795 shutdown 2024-09-19T08:16:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.030+0000 7f627b7a7740 1 -- 172.21.15.22:0/1848901377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf9170 msgr2=0x557106c07e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.030+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1848901377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf9170 0x557106c07e10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f625c011e30 tx=0x7f625c005850 comp rx=0 tx=0).stop 2024-09-19T08:16:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.030+0000 7f627b7a7740 1 -- 172.21.15.22:0/1848901377 shutdown_connections 2024-09-19T08:16:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.030+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1848901377 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557106c7ece0 0x557106c81100 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.030+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1848901377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557106c03a30 0x557106c08350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.030+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1848901377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557106bf9170 0x557106c07e10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.030+0000 7f625a7fc640 0 client.15795 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.030+0000 7f627b7a7740 1 --2- 172.21.15.22:0/1848901377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557106bf8850 0x557106c078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.030+0000 7f627b7a7740 1 -- 172.21.15.22:0/1848901377 >> 172.21.15.22:0/1848901377 conn(0x557106bf4de0 msgr2=0x557106bf3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.030+0000 7f627b7a7740 1 -- 172.21.15.22:0/1848901377 shutdown_connections 2024-09-19T08:16:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.030+0000 7f627b7a7740 1 -- 172.21.15.22:0/1848901377 wait complete. 2024-09-19T08:16:52.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.030+0000 7f627b7a7740 20 client.15795 trim_cache size 0 max 16384 2024-09-19T08:16:52.031 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:16:52.031 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 26, 2024-09-19T08:16:52.032 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:16:52.032 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 10, 2024-09-19T08:16:52.032 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:16:52.032 INFO:teuthology.orchestra.run.smithi022.stdout: "f364c789": { 2024-09-19T08:16:52.032 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 10, 2024-09-19T08:16:52.032 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:16:52.032 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:16:52.032 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:16:52.032 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:16:52.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:16:52.032 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:16:52.032 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 140.3, 2024-09-19T08:16:52.032 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:16:52.033 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:16:52.033 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:16:52.033 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:16:52.033 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:16:52.033 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:16:52.033 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:52.033 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:52.033 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:52.033 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:52.033 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:16:52.033 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:16:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.031+0000 7f62793ce640 1 -- 172.21.15.22:0/369378952 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f62600a3430 msgr2=0x7f62600a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.031+0000 7f62793ce640 1 --2- 172.21.15.22:0/369378952 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f62600a3430 0x7f62600a58f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f626c005e30 tx=0x7f626c043040 comp rx=0 tx=0).stop 2024-09-19T08:16:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.031+0000 7f62793ce640 1 -- 172.21.15.22:0/369378952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274100430 msgr2=0x7f62741992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.031+0000 7f62793ce640 1 --2- 172.21.15.22:0/369378952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274100430 0x7f62741992d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f626400d950 tx=0x7f626400de10 comp rx=0 tx=0).stop 2024-09-19T08:16:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.031+0000 7f62793ce640 1 -- 172.21.15.22:0/369378952 shutdown_connections 2024-09-19T08:16:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.031+0000 7f62793ce640 1 --2- 172.21.15.22:0/369378952 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f62600a3430 0x7f62600a58f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.031+0000 7f62793ce640 1 --2- 172.21.15.22:0/369378952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6274105560 0x7f62741a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:52.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.031+0000 7f62793ce640 1 --2- 172.21.15.22:0/369378952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6274100de0 0x7f6274199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:52.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.031+0000 7f62793ce640 1 --2- 172.21.15.22:0/369378952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6274100430 0x7f62741992d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:52.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.031+0000 7f62793ce640 1 -- 172.21.15.22:0/369378952 >> 172.21.15.22:0/369378952 conn(0x7f6274077230 msgr2=0x7f62740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:52.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.031+0000 7f62793ce640 1 -- 172.21.15.22:0/369378952 shutdown_connections 2024-09-19T08:16:52.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:52.031+0000 7f62793ce640 1 -- 172.21.15.22:0/369378952 wait complete. 2024-09-19T08:16:52.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:51 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/4128202109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:52.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:51 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/4128202109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:52.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:51 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/4128202109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:16:52.794 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f364c789', quiesce took 6.3 seconds. Will release after: 14.05 2024-09-19T08:16:53.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:52 smithi064 ceph-mon[34900]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 5 op/s 2024-09-19T08:16:53.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:52 smithi022 ceph-mon[30963]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 5 op/s 2024-09-19T08:16:53.219 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:16:53.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:52 smithi157 ceph-mon[35143]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 5 op/s 2024-09-19T08:16:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:53.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.962+0000 7f4956c41640 1 -- 172.21.15.22:0/100356829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4950102790 msgr2=0x7f49501067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.962+0000 7f4956c41640 1 --2- 172.21.15.22:0/100356829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4950102790 0x7f49501067e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4940009900 tx=0x7f494002a410 comp rx=0 tx=0).stop 2024-09-19T08:16:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.962+0000 7f4956c41640 1 -- 172.21.15.22:0/100356829 shutdown_connections 2024-09-19T08:16:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.962+0000 7f4956c41640 1 --2- 172.21.15.22:0/100356829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4950106f10 0x7f495010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.962+0000 7f4956c41640 1 --2- 172.21.15.22:0/100356829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4950102790 0x7f49501067e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.962+0000 7f4956c41640 1 --2- 172.21.15.22:0/100356829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950101de0 0x7f49501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.962+0000 7f4956c41640 1 -- 172.21.15.22:0/100356829 >> 172.21.15.22:0/100356829 conn(0x7f49500fd8c0 msgr2=0x7f49500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.962+0000 7f4956c41640 1 -- 172.21.15.22:0/100356829 shutdown_connections 2024-09-19T08:16:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.962+0000 7f4956c41640 1 -- 172.21.15.22:0/100356829 wait complete. 2024-09-19T08:16:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.963+0000 7f4956c41640 1 Processor -- start 2024-09-19T08:16:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.963+0000 7f4956c41640 1 -- start start 2024-09-19T08:16:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.963+0000 7f4956c41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950101de0 0x7f495019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.963+0000 7f4956c41640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4950102790 0x7f495019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.963+0000 7f4956c41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4950106f10 0x7f495019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.963+0000 7f4956c41640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49500747b0 con 0x7f4950106f10 2024-09-19T08:16:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.963+0000 7f4956c41640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4950074630 con 0x7f4950101de0 2024-09-19T08:16:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.963+0000 7f4956c41640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4950074930 con 0x7f4950102790 2024-09-19T08:16:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.963+0000 7f49551de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4950106f10 0x7f495019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.963+0000 7f49551de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4950106f10 0x7f495019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49272/0 (socket says 172.21.15.22:49272) 2024-09-19T08:16:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.963+0000 7f49551de640 1 -- 172.21.15.22:0/2862574763 learned_addr learned my addr 172.21.15.22:0/2862574763 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.963+0000 7f49549dd640 1 --2- 172.21.15.22:0/2862574763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950101de0 0x7f495019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.963+0000 7f4947fff640 1 --2- 172.21.15.22:0/2862574763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4950102790 0x7f495019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.963+0000 7f49551de640 1 -- 172.21.15.22:0/2862574763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4950102790 msgr2=0x7f495019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.964+0000 7f49551de640 1 --2- 172.21.15.22:0/2862574763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4950102790 0x7f495019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.964+0000 7f49551de640 1 -- 172.21.15.22:0/2862574763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950101de0 msgr2=0x7f495019aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.964+0000 7f49551de640 1 --2- 172.21.15.22:0/2862574763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950101de0 0x7f495019aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.964+0000 7f49551de640 1 -- 172.21.15.22:0/2862574763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495019f5f0 con 0x7f4950106f10 2024-09-19T08:16:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.964+0000 7f49551de640 1 --2- 172.21.15.22:0/2862574763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4950106f10 0x7f495019eec0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f494800cce0 tx=0x7f494800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.964+0000 7f4945ffb640 1 -- 172.21.15.22:0/2862574763 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494800f450 con 0x7f4950106f10 2024-09-19T08:16:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.964+0000 7f4945ffb640 1 -- 172.21.15.22:0/2862574763 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f494800fad0 con 0x7f4950106f10 2024-09-19T08:16:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.964+0000 7f4956c41640 1 -- 172.21.15.22:0/2862574763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495019f8e0 con 0x7f4950106f10 2024-09-19T08:16:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.964+0000 7f4945ffb640 1 -- 172.21.15.22:0/2862574763 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948018c10 con 0x7f4950106f10 2024-09-19T08:16:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.964+0000 7f4956c41640 1 -- 172.21.15.22:0/2862574763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49501a7170 con 0x7f4950106f10 2024-09-19T08:16:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.966+0000 7f4945ffb640 1 -- 172.21.15.22:0/2862574763 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f494800f5f0 con 0x7f4950106f10 2024-09-19T08:16:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f495901a740 1 Processor -- start 2024-09-19T08:16:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f495901a740 1 -- start start 2024-09-19T08:16:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f4945ffb640 1 --2- 172.21.15.22:0/2862574763 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f49380a33b0 0x7f49380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f495901a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14e7b40 0x55a8c14e6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f495901a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e65d0 0x55a8c14e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f495901a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14f1370 0x55a8c14f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f495901a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a8c144c2f0 con 0x55a8c14e65d0 2024-09-19T08:16:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f495901a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a8c13f3cf0 con 0x55a8c14e7b40 2024-09-19T08:16:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f495901a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a8c14482b0 con 0x55a8c14f1370 2024-09-19T08:16:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f4947fff640 1 --2- 172.21.15.22:0/2862574763 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f49380a33b0 0x7f49380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f4945ffb640 1 -- 172.21.15.22:0/2862574763 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f49480d02f0 con 0x7f4950106f10 2024-09-19T08:16:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f4947fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e65d0 0x55a8c14e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f4947fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e65d0 0x55a8c14e6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49274/0 (socket says 172.21.15.22:49274) 2024-09-19T08:16:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f4947fff640 1 -- 172.21.15.22:0/3041318307 learned_addr learned my addr 172.21.15.22:0/3041318307 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f49549dd640 1 --2- 172.21.15.22:0/3041318307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14e7b40 0x55a8c14e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.967+0000 7f49551de640 1 --2- 172.21.15.22:0/3041318307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14f1370 0x55a8c14f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.968+0000 7f4947fff640 1 -- 172.21.15.22:0/3041318307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14f1370 msgr2=0x55a8c14f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.968+0000 7f4947fff640 1 --2- 172.21.15.22:0/3041318307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14f1370 0x55a8c14f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.968+0000 7f4947fff640 1 -- 172.21.15.22:0/3041318307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14e7b40 msgr2=0x55a8c14e6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:53.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.968+0000 7f4947fff640 1 --2- 172.21.15.22:0/3041318307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14e7b40 0x55a8c14e6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.968+0000 7f4947fff640 1 -- 172.21.15.22:0/3041318307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c13d83a0 con 0x55a8c14e65d0 2024-09-19T08:16:53.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.968+0000 7f4947fff640 1 --2- 172.21.15.22:0/2862574763 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f49380a33b0 0x7f49380a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4940032fd0 tx=0x7f4940005e60 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:53.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.968+0000 7f4947fff640 1 --2- 172.21.15.22:0/3041318307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e65d0 0x55a8c14e6a10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f494002a930 tx=0x7f4940046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2543a7046405fcad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:53.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.968+0000 7f4936ffd640 1 -- 172.21.15.22:0/3041318307 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49400035f0 con 0x55a8c14e65d0 2024-09-19T08:16:53.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.968+0000 7f4936ffd640 1 -- 172.21.15.22:0/3041318307 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4940003c70 con 0x55a8c14e65d0 2024-09-19T08:16:53.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.968+0000 7f4936ffd640 1 -- 172.21.15.22:0/3041318307 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494003dbe0 con 0x55a8c14e65d0 2024-09-19T08:16:53.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.969+0000 7f495901a740 1 -- 172.21.15.22:0/3041318307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e65d0 msgr2=0x55a8c14e6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:53.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.969+0000 7f495901a740 1 --2- 172.21.15.22:0/3041318307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e65d0 0x55a8c14e6a10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f494002a930 tx=0x7f4940046000 comp rx=0 tx=0).stop 2024-09-19T08:16:53.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.969+0000 7f495901a740 1 -- 172.21.15.22:0/3041318307 shutdown_connections 2024-09-19T08:16:53.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.969+0000 7f495901a740 1 --2- 172.21.15.22:0/3041318307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14f1370 0x55a8c14f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.969+0000 7f495901a740 1 --2- 172.21.15.22:0/3041318307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e65d0 0x55a8c14e6a10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.969+0000 7f495901a740 1 --2- 172.21.15.22:0/3041318307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14e7b40 0x55a8c14e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.969+0000 7f495901a740 1 -- 172.21.15.22:0/3041318307 >> 172.21.15.22:0/3041318307 conn(0x55a8c14af310 msgr2=0x55a8c14e2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:53.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.969+0000 7f495901a740 1 -- 172.21.15.22:0/3041318307 shutdown_connections 2024-09-19T08:16:53.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.969+0000 7f495901a740 1 -- 172.21.15.22:0/3041318307 wait complete. 2024-09-19T08:16:53.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.969+0000 7f495901a740 1 -- start start 2024-09-19T08:16:53.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.969+0000 7f495901a740 1 Processor -- start 2024-09-19T08:16:53.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f495901a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e65d0 0x55a8c14f0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f495901a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e7b40 0x55a8c14ee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f495901a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14f1370 0x55a8c14ee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f495901a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a8c14f4010 con 0x55a8c14e65d0 2024-09-19T08:16:53.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f495901a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a8c1407060 con 0x55a8c14f1370 2024-09-19T08:16:53.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f495901a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a8c14f4190 con 0x55a8c14e7b40 2024-09-19T08:16:53.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f495901a740 10 client.0 _ll_register_callbacks cb 0x55a8c115a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:53.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f49549dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e65d0 0x55a8c14f0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f49549dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e65d0 0x55a8c14f0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49286/0 (socket says 172.21.15.22:49286) 2024-09-19T08:16:53.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f49549dd640 1 -- 172.21.15.22:0/4276868330 learned_addr learned my addr 172.21.15.22:0/4276868330 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:53.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f49551de640 1 --2- 172.21.15.22:0/4276868330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14f1370 0x55a8c14ee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f4947fff640 1 --2- 172.21.15.22:0/4276868330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e7b40 0x55a8c14ee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f49549dd640 1 -- 172.21.15.22:0/4276868330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e7b40 msgr2=0x55a8c14ee060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:53.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f49549dd640 1 --2- 172.21.15.22:0/4276868330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e7b40 0x55a8c14ee060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f49549dd640 1 -- 172.21.15.22:0/4276868330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14f1370 msgr2=0x55a8c14ee7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:53.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f49549dd640 1 --2- 172.21.15.22:0/4276868330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14f1370 0x55a8c14ee7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f49549dd640 1 -- 172.21.15.22:0/4276868330 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c11bd3c0 con 0x55a8c14e65d0 2024-09-19T08:16:53.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f49549dd640 1 --2- 172.21.15.22:0/4276868330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e65d0 0x55a8c14f0db0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f494c00ba80 tx=0x7f494c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b8975c32e2c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:53.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f492ffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:53.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f492ffff640 1 -- 172.21.15.22:0/4276868330 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c011440 con 0x55a8c14e65d0 2024-09-19T08:16:53.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f492ffff640 1 -- 172.21.15.22:0/4276868330 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f494c0115e0 con 0x55a8c14e65d0 2024-09-19T08:16:53.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.970+0000 7f495901a740 1 -- 172.21.15.22:0/4276868330 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4940003c70 con 0x55a8c14e65d0 2024-09-19T08:16:53.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.971+0000 7f492ffff640 1 -- 172.21.15.22:0/4276868330 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f494c011820 con 0x55a8c14e65d0 2024-09-19T08:16:53.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.971+0000 7f495901a740 10 client.15807 fetch_fsmap learned FSMap version 26 2024-09-19T08:16:53.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.971+0000 7f495901a740 1 -- 172.21.15.22:0/4276868330 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a8c14ffbb0 con 0x55a8c14e65d0 2024-09-19T08:16:53.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.971+0000 7f492ffff640 1 -- 172.21.15.22:0/4276868330 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4940003c70 con 0x55a8c14e65d0 2024-09-19T08:16:53.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.971+0000 7f492ffff640 1 -- 172.21.15.22:0/4276868330 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f494c01bab0 con 0x55a8c14e65d0 2024-09-19T08:16:53.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.971+0000 7f495901a740 10 client.15807 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:16:53.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.971+0000 7f495901a740 10 client.15807 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:16:53.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.971+0000 7f495901a740 20 client.15807 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:53.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.971+0000 7f495901a740 1 --2- 172.21.15.22:0/4276868330 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a8c1501070 0x55a8c15034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.971+0000 7f495901a740 4 client.15807 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:16:53.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.971+0000 7f495901a740 1 -- 172.21.15.22:0/4276868330 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8c1459860 con 0x55a8c1501070 2024-09-19T08:16:53.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.971+0000 7f4947fff640 1 --2- 172.21.15.22:0/4276868330 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a8c1501070 0x55a8c15034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.972+0000 7f4947fff640 1 --2- 172.21.15.22:0/4276868330 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a8c1501070 0x55a8c15034f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab4b258677ef6946 server_cookie=bf4a01cfaa54c788 in_seq=0 out_seq=0 2024-09-19T08:16:53.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.972+0000 7f492ffff640 10 client.15807 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:16:53.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f492ffff640 1 -- 172.21.15.22:0/4276868330 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8c1459860 con 0x55a8c1501070 2024-09-19T08:16:53.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f492ffff640 10 client.15807 handle_command_reply: tid=0 2024-09-19T08:16:53.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f495901a740 1 client.15807 shutdown 2024-09-19T08:16:53.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f495901a740 1 -- 172.21.15.22:0/4276868330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e65d0 msgr2=0x55a8c14f0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:53.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f495901a740 1 --2- 172.21.15.22:0/4276868330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e65d0 0x55a8c14f0db0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f494c00ba80 tx=0x7f494c00bf40 comp rx=0 tx=0).stop 2024-09-19T08:16:53.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f495901a740 1 -- 172.21.15.22:0/4276868330 shutdown_connections 2024-09-19T08:16:53.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f495901a740 1 --2- 172.21.15.22:0/4276868330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14f1370 0x55a8c14ee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f495901a740 1 --2- 172.21.15.22:0/4276868330 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a8c1501070 0x55a8c15034f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f495901a740 1 --2- 172.21.15.22:0/4276868330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e7b40 0x55a8c14ee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f495901a740 1 --2- 172.21.15.22:0/4276868330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e65d0 0x55a8c14f0db0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f492ffff640 0 client.15807 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:16:53.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f495901a740 1 -- 172.21.15.22:0/4276868330 >> 172.21.15.22:0/4276868330 conn(0x55a8c14af310 msgr2=0x55a8c14e8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:53.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f495901a740 1 -- 172.21.15.22:0/4276868330 shutdown_connections 2024-09-19T08:16:53.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f495901a740 1 -- 172.21.15.22:0/4276868330 wait complete. 2024-09-19T08:16:53.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.973+0000 7f495901a740 20 client.15807 trim_cache size 0 max 16384 2024-09-19T08:16:53.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.986+0000 7f495901a740 1 Processor -- start 2024-09-19T08:16:53.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.986+0000 7f495901a740 1 -- start start 2024-09-19T08:16:53.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.986+0000 7f495901a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e65d0 0x55a8c14e2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.986+0000 7f495901a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e7b40 0x55a8c14e9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.986+0000 7f495901a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14f1370 0x55a8c14e9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.986+0000 7f495901a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49501b2300 con 0x55a8c14e7b40 2024-09-19T08:16:53.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.986+0000 7f495901a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f49501afbd0 con 0x55a8c14f1370 2024-09-19T08:16:53.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.986+0000 7f495901a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f49501ada20 con 0x55a8c14e65d0 2024-09-19T08:16:53.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.986+0000 7f4947fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e7b40 0x55a8c14e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.986+0000 7f4947fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e7b40 0x55a8c14e9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49300/0 (socket says 172.21.15.22:49300) 2024-09-19T08:16:53.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.986+0000 7f4947fff640 1 -- 172.21.15.22:0/2595379442 learned_addr learned my addr 172.21.15.22:0/2595379442 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:53.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.986+0000 7f49551de640 1 --2- 172.21.15.22:0/2595379442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14f1370 0x55a8c14e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.987+0000 7f49549dd640 1 --2- 172.21.15.22:0/2595379442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e65d0 0x55a8c14e2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.987+0000 7f4947fff640 1 -- 172.21.15.22:0/2595379442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e65d0 msgr2=0x55a8c14e2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:53.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.987+0000 7f4947fff640 1 --2- 172.21.15.22:0/2595379442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e65d0 0x55a8c14e2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.987+0000 7f4947fff640 1 -- 172.21.15.22:0/2595379442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14f1370 msgr2=0x55a8c14e9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:53.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.987+0000 7f4947fff640 1 --2- 172.21.15.22:0/2595379442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14f1370 0x55a8c14e9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.987+0000 7f4947fff640 1 -- 172.21.15.22:0/2595379442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c14f3f50 con 0x55a8c14e7b40 2024-09-19T08:16:53.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.987+0000 7f4947fff640 1 --2- 172.21.15.22:0/2595379442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e7b40 0x55a8c14e9120 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4940031ba0 tx=0x7f494003d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6de1b67c446490 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:53.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.987+0000 7f49367fc640 1 -- 172.21.15.22:0/2595379442 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494003d7b0 con 0x55a8c14e7b40 2024-09-19T08:16:53.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.987+0000 7f49367fc640 1 -- 172.21.15.22:0/2595379442 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f494003de30 con 0x55a8c14e7b40 2024-09-19T08:16:53.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.987+0000 7f49367fc640 1 -- 172.21.15.22:0/2595379442 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940003cb0 con 0x55a8c14e7b40 2024-09-19T08:16:53.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.988+0000 7f495901a740 1 -- 172.21.15.22:0/2595379442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e7b40 msgr2=0x55a8c14e9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:53.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.988+0000 7f495901a740 1 --2- 172.21.15.22:0/2595379442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e7b40 0x55a8c14e9120 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4940031ba0 tx=0x7f494003d740 comp rx=0 tx=0).stop 2024-09-19T08:16:53.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.988+0000 7f495901a740 1 -- 172.21.15.22:0/2595379442 shutdown_connections 2024-09-19T08:16:53.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.988+0000 7f495901a740 1 --2- 172.21.15.22:0/2595379442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14f1370 0x55a8c14e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.988+0000 7f495901a740 1 --2- 172.21.15.22:0/2595379442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14e7b40 0x55a8c14e9120 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.988+0000 7f495901a740 1 --2- 172.21.15.22:0/2595379442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e65d0 0x55a8c14e2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.988+0000 7f495901a740 1 -- 172.21.15.22:0/2595379442 >> 172.21.15.22:0/2595379442 conn(0x55a8c14af310 msgr2=0x55a8c14e2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:53.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.988+0000 7f495901a740 1 -- 172.21.15.22:0/2595379442 shutdown_connections 2024-09-19T08:16:53.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.988+0000 7f495901a740 1 -- 172.21.15.22:0/2595379442 wait complete. 2024-09-19T08:16:53.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.988+0000 7f495901a740 1 -- start start 2024-09-19T08:16:53.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.988+0000 7f495901a740 1 Processor -- start 2024-09-19T08:16:53.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f495901a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14e65d0 0x55a8c14f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f495901a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e7b40 0x55a8c14f9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f495901a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14f1370 0x55a8c14eac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f495901a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49501ae090 con 0x55a8c14f1370 2024-09-19T08:16:53.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f495901a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f49501af500 con 0x55a8c14e65d0 2024-09-19T08:16:53.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f495901a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f49501ae760 con 0x55a8c14e7b40 2024-09-19T08:16:53.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f49551de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14f1370 0x55a8c14eac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f495901a740 10 client.0 _ll_register_callbacks cb 0x7f494c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:53.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f49551de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14f1370 0x55a8c14eac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49314/0 (socket says 172.21.15.22:49314) 2024-09-19T08:16:53.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f49551de640 1 -- 172.21.15.22:0/1959151575 learned_addr learned my addr 172.21.15.22:0/1959151575 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:53.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f4947fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e7b40 0x55a8c14f9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f49549dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14e65d0 0x55a8c14f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f49551de640 1 -- 172.21.15.22:0/1959151575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e7b40 msgr2=0x55a8c14f9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:53.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f49551de640 1 --2- 172.21.15.22:0/1959151575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e7b40 0x55a8c14f9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f49551de640 1 -- 172.21.15.22:0/1959151575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14e65d0 msgr2=0x55a8c14f97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f49551de640 1 --2- 172.21.15.22:0/1959151575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14e65d0 0x55a8c14f97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f49551de640 1 -- 172.21.15.22:0/1959151575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c14eb440 con 0x55a8c14f1370 2024-09-19T08:16:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f49549dd640 1 --2- 172.21.15.22:0/1959151575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14e65d0 0x55a8c14f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f49551de640 1 --2- 172.21.15.22:0/1959151575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14f1370 0x55a8c14eac40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f494809c280 tx=0x7f494809cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f50b7971b529e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.989+0000 7f492ffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f492ffff640 1 -- 172.21.15.22:0/1959151575 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49480a51f0 con 0x55a8c14f1370 2024-09-19T08:16:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f492ffff640 1 -- 172.21.15.22:0/1959151575 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4948048400 con 0x55a8c14f1370 2024-09-19T08:16:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f495901a740 1 -- 172.21.15.22:0/1959151575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f494003de30 con 0x55a8c14f1370 2024-09-19T08:16:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f492ffff640 1 -- 172.21.15.22:0/1959151575 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f494800f8e0 con 0x55a8c14f1370 2024-09-19T08:16:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f495901a740 10 client.15813 fetch_fsmap learned FSMap version 26 2024-09-19T08:16:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f495901a740 1 -- 172.21.15.22:0/1959151575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a8c156bab0 con 0x55a8c14f1370 2024-09-19T08:16:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f492ffff640 1 -- 172.21.15.22:0/1959151575 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f494003de30 con 0x55a8c14f1370 2024-09-19T08:16:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f492ffff640 1 -- 172.21.15.22:0/1959151575 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f49480a4d60 con 0x55a8c14f1370 2024-09-19T08:16:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f495901a740 10 client.15813 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:16:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f495901a740 10 client.15813 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:16:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f495901a740 20 client.15813 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f495901a740 1 --2- 172.21.15.22:0/1959151575 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a8c156c220 0x55a8c156e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f495901a740 4 client.15813 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:16:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f495901a740 1 -- 172.21.15.22:0/1959151575 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8c1453d70 con 0x55a8c156c220 2024-09-19T08:16:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.990+0000 7f49549dd640 1 --2- 172.21.15.22:0/1959151575 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a8c156c220 0x55a8c156e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f492ffff640 10 client.15813 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:16:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f49549dd640 1 --2- 172.21.15.22:0/1959151575 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a8c156c220 0x55a8c156e640 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=132136b18696f915 server_cookie=1a6117334ef06c19 in_seq=0 out_seq=0 2024-09-19T08:16:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f492ffff640 1 -- 172.21.15.22:0/1959151575 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8c1453d70 con 0x55a8c156c220 2024-09-19T08:16:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f492ffff640 10 client.15813 handle_command_reply: tid=0 2024-09-19T08:16:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f495901a740 1 client.15813 shutdown 2024-09-19T08:16:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f495901a740 1 -- 172.21.15.22:0/1959151575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14f1370 msgr2=0x55a8c14eac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:53.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f495901a740 1 --2- 172.21.15.22:0/1959151575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14f1370 0x55a8c14eac40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f494809c280 tx=0x7f494809cd40 comp rx=0 tx=0).stop 2024-09-19T08:16:53.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f495901a740 1 -- 172.21.15.22:0/1959151575 shutdown_connections 2024-09-19T08:16:53.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f495901a740 1 --2- 172.21.15.22:0/1959151575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a8c14f1370 0x55a8c14eac40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f495901a740 1 --2- 172.21.15.22:0/1959151575 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a8c156c220 0x55a8c156e640 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f495901a740 1 --2- 172.21.15.22:0/1959151575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a8c14e7b40 0x55a8c14f9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f492ffff640 0 client.15813 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:16:53.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f495901a740 1 --2- 172.21.15.22:0/1959151575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c14e65d0 0x55a8c14f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:53.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f495901a740 1 -- 172.21.15.22:0/1959151575 >> 172.21.15.22:0/1959151575 conn(0x55a8c14af310 msgr2=0x55a8c14e14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:53.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.991+0000 7f495901a740 1 -- 172.21.15.22:0/1959151575 shutdown_connections 2024-09-19T08:16:53.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.992+0000 7f495901a740 1 -- 172.21.15.22:0/1959151575 wait complete. 2024-09-19T08:16:53.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.992+0000 7f495901a740 20 client.15813 trim_cache size 0 max 16384 2024-09-19T08:16:54.000 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:16:54.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.992+0000 7f4956c41640 1 -- 172.21.15.22:0/2862574763 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f49380a33b0 msgr2=0x7f49380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:54.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.992+0000 7f4956c41640 1 --2- 172.21.15.22:0/2862574763 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f49380a33b0 0x7f49380a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4940032fd0 tx=0x7f4940005e60 comp rx=0 tx=0).stop 2024-09-19T08:16:54.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.992+0000 7f4956c41640 1 -- 172.21.15.22:0/2862574763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4950106f10 msgr2=0x7f495019eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:54.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.992+0000 7f4956c41640 1 --2- 172.21.15.22:0/2862574763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4950106f10 0x7f495019eec0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f494800cce0 tx=0x7f494800a620 comp rx=0 tx=0).stop 2024-09-19T08:16:54.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.992+0000 7f4956c41640 1 -- 172.21.15.22:0/2862574763 shutdown_connections 2024-09-19T08:16:54.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.992+0000 7f4956c41640 1 --2- 172.21.15.22:0/2862574763 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f49380a33b0 0x7f49380a5870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:54.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.992+0000 7f4956c41640 1 --2- 172.21.15.22:0/2862574763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4950106f10 0x7f495019eec0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:54.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.992+0000 7f4956c41640 1 --2- 172.21.15.22:0/2862574763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4950102790 0x7f495019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:54.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.992+0000 7f4956c41640 1 --2- 172.21.15.22:0/2862574763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950101de0 0x7f495019aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:54.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.992+0000 7f4956c41640 1 -- 172.21.15.22:0/2862574763 >> 172.21.15.22:0/2862574763 conn(0x7f49500fd8c0 msgr2=0x7f49501057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:54.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.992+0000 7f4956c41640 1 -- 172.21.15.22:0/2862574763 shutdown_connections 2024-09-19T08:16:54.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:53.992+0000 7f4956c41640 1 -- 172.21.15.22:0/2862574763 wait complete. 2024-09-19T08:16:54.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:16:55.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:54 smithi022 ceph-mon[30963]: pgmap v59: 97 pgs: 97 active+clean; 4.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 291 KiB/s wr, 32 op/s 2024-09-19T08:16:55.007 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:55.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:54 smithi157 ceph-mon[35143]: pgmap v59: 97 pgs: 97 active+clean; 4.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 291 KiB/s wr, 32 op/s 2024-09-19T08:16:55.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:54 smithi064 ceph-mon[34900]: pgmap v59: 97 pgs: 97 active+clean; 4.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 291 KiB/s wr, 32 op/s 2024-09-19T08:16:55.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.505+0000 7f89bc5c0640 1 -- 172.21.15.22:0/2902323207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b4103740 msgr2=0x7f89b4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:55.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.505+0000 7f89bc5c0640 1 --2- 172.21.15.22:0/2902323207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b4103740 0x7f89b4107b80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009930 tx=0x7f89a402a420 comp rx=0 tx=0).stop 2024-09-19T08:16:55.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.505+0000 7f89bc5c0640 1 -- 172.21.15.22:0/2902323207 shutdown_connections 2024-09-19T08:16:55.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.505+0000 7f89bc5c0640 1 --2- 172.21.15.22:0/2902323207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4108290 0x7f89b410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.505+0000 7f89bc5c0640 1 --2- 172.21.15.22:0/2902323207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b4103740 0x7f89b4107b80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.505+0000 7f89bc5c0640 1 --2- 172.21.15.22:0/2902323207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b4102d90 0x7f89b4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.505+0000 7f89bc5c0640 1 -- 172.21.15.22:0/2902323207 >> 172.21.15.22:0/2902323207 conn(0x7f89b40fc7d0 msgr2=0x7f89b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:55.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.505+0000 7f89bc5c0640 1 -- 172.21.15.22:0/2902323207 shutdown_connections 2024-09-19T08:16:55.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.505+0000 7f89bc5c0640 1 -- 172.21.15.22:0/2902323207 wait complete. 2024-09-19T08:16:55.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89bc5c0640 1 Processor -- start 2024-09-19T08:16:55.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89bc5c0640 1 -- start start 2024-09-19T08:16:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89bc5c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b4102d90 0x7f89b4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89bc5c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4103740 0x7f89b4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89bc5c0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b4108290 0x7f89b419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89bc5c0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f89b4112090 con 0x7f89b4102d90 2024-09-19T08:16:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89bc5c0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f89b4111f10 con 0x7f89b4103740 2024-09-19T08:16:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89bc5c0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89b4112210 con 0x7f89b4108290 2024-09-19T08:16:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89ba35c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b4102d90 0x7f89b4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89ba35c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b4102d90 0x7f89b4198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49330/0 (socket says 172.21.15.22:49330) 2024-09-19T08:16:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89ba35c640 1 -- 172.21.15.22:0/774989494 learned_addr learned my addr 172.21.15.22:0/774989494 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89bab5d640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b4108290 0x7f89b419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89b9b5b640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4103740 0x7f89b4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89ba35c640 1 -- 172.21.15.22:0/774989494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b4108290 msgr2=0x7f89b419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89ba35c640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b4108290 0x7f89b419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89ba35c640 1 -- 172.21.15.22:0/774989494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4103740 msgr2=0x7f89b4198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89ba35c640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4103740 0x7f89b4198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89ba35c640 1 -- 172.21.15.22:0/774989494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b419d520 con 0x7f89b4102d90 2024-09-19T08:16:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.506+0000 7f89bab5d640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b4108290 0x7f89b419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.507+0000 7f89ba35c640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b4102d90 0x7f89b4198570 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f89b000d770 tx=0x7f89b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.507+0000 7f89ab7fe640 1 -- 172.21.15.22:0/774989494 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0017070 con 0x7f89b4102d90 2024-09-19T08:16:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.507+0000 7f89ab7fe640 1 -- 172.21.15.22:0/774989494 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b0005500 con 0x7f89b4102d90 2024-09-19T08:16:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.507+0000 7f89bc5c0640 1 -- 172.21.15.22:0/774989494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b419d810 con 0x7f89b4102d90 2024-09-19T08:16:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.507+0000 7f89ab7fe640 1 -- 172.21.15.22:0/774989494 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0004e00 con 0x7f89b4102d90 2024-09-19T08:16:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.507+0000 7f89bc5c0640 1 -- 172.21.15.22:0/774989494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b41a50f0 con 0x7f89b4102d90 2024-09-19T08:16:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.509+0000 7f89ab7fe640 1 -- 172.21.15.22:0/774989494 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f89b0004fa0 con 0x7f89b4102d90 2024-09-19T08:16:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.509+0000 7f89be999740 1 Processor -- start 2024-09-19T08:16:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89be999740 1 -- start start 2024-09-19T08:16:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89be999740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fca2b40 0x562a9fca1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89be999740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca15d0 0x562a9fca1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89be999740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fcac370 0x562a9fcae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89be999740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562a9fc072f0 con 0x562a9fca2b40 2024-09-19T08:16:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89be999740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562a9fbaecf0 con 0x562a9fcac370 2024-09-19T08:16:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89be999740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562a9fc032b0 con 0x562a9fca15d0 2024-09-19T08:16:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89ab7fe640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f89a00a33b0 0x7f89a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89bab5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fca2b40 0x562a9fca1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89bab5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fca2b40 0x562a9fca1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49344/0 (socket says 172.21.15.22:49344) 2024-09-19T08:16:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89bab5d640 1 -- 172.21.15.22:0/401527660 learned_addr learned my addr 172.21.15.22:0/401527660 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89ba35c640 1 --2- 172.21.15.22:0/401527660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca15d0 0x562a9fca1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89bab5d640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f89a00a33b0 0x7f89a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89b9b5b640 1 --2- 172.21.15.22:0/401527660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fcac370 0x562a9fcae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89ab7fe640 1 -- 172.21.15.22:0/774989494 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f89b00cf8a0 con 0x7f89b4102d90 2024-09-19T08:16:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89bab5d640 1 -- 172.21.15.22:0/401527660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca15d0 msgr2=0x562a9fca1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.510+0000 7f89bab5d640 1 --2- 172.21.15.22:0/401527660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca15d0 0x562a9fca1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.511+0000 7f89bab5d640 1 -- 172.21.15.22:0/401527660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fcac370 msgr2=0x562a9fcae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.511+0000 7f89bab5d640 1 --2- 172.21.15.22:0/401527660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fcac370 0x562a9fcae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.511+0000 7f89bab5d640 1 -- 172.21.15.22:0/401527660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a9fb933a0 con 0x562a9fca2b40 2024-09-19T08:16:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.511+0000 7f89bab5d640 1 --2- 172.21.15.22:0/401527660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fca2b40 0x562a9fca1090 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f89ac013e30 tx=0x7f89ac010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537d47aa3de45306 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.511+0000 7f89bab5d640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f89a00a33b0 0x7f89a00a5870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f89ac007fd0 tx=0x7f89ac01b000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.511+0000 7f89a8ff9640 1 -- 172.21.15.22:0/401527660 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac00add0 con 0x562a9fca2b40 2024-09-19T08:16:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.511+0000 7f89a8ff9640 1 -- 172.21.15.22:0/401527660 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ac007c00 con 0x562a9fca2b40 2024-09-19T08:16:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.511+0000 7f89a8ff9640 1 -- 172.21.15.22:0/401527660 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac003df0 con 0x562a9fca2b40 2024-09-19T08:16:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 -- 172.21.15.22:0/401527660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fca2b40 msgr2=0x562a9fca1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 --2- 172.21.15.22:0/401527660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fca2b40 0x562a9fca1090 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f89ac013e30 tx=0x7f89ac010740 comp rx=0 tx=0).stop 2024-09-19T08:16:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 -- 172.21.15.22:0/401527660 shutdown_connections 2024-09-19T08:16:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 --2- 172.21.15.22:0/401527660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fcac370 0x562a9fcae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 --2- 172.21.15.22:0/401527660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca15d0 0x562a9fca1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 --2- 172.21.15.22:0/401527660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fca2b40 0x562a9fca1090 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 -- 172.21.15.22:0/401527660 >> 172.21.15.22:0/401527660 conn(0x562a9fc6a310 msgr2=0x562a9fc9dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 -- 172.21.15.22:0/401527660 shutdown_connections 2024-09-19T08:16:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 -- 172.21.15.22:0/401527660 wait complete. 2024-09-19T08:16:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 -- start start 2024-09-19T08:16:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 Processor -- start 2024-09-19T08:16:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fca15d0 0x562a9fcabda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca2b40 0x562a9fca9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fcac370 0x562a9fca97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562a9fcaf010 con 0x562a9fcac370 2024-09-19T08:16:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562a9fbc2060 con 0x562a9fca15d0 2024-09-19T08:16:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.512+0000 7f89be999740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562a9fcaf190 con 0x562a9fca2b40 2024-09-19T08:16:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f89be999740 10 client.0 _ll_register_callbacks cb 0x562a9f9158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f89bab5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fcac370 0x562a9fca97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f89bab5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fcac370 0x562a9fca97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49358/0 (socket says 172.21.15.22:49358) 2024-09-19T08:16:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f89bab5d640 1 -- 172.21.15.22:0/844620598 learned_addr learned my addr 172.21.15.22:0/844620598 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f89ba35c640 1 --2- 172.21.15.22:0/844620598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fca15d0 0x562a9fcabda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f89b9b5b640 1 --2- 172.21.15.22:0/844620598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca2b40 0x562a9fca9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f89bab5d640 1 -- 172.21.15.22:0/844620598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca2b40 msgr2=0x562a9fca9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f89bab5d640 1 --2- 172.21.15.22:0/844620598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca2b40 0x562a9fca9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f89bab5d640 1 -- 172.21.15.22:0/844620598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fca15d0 msgr2=0x562a9fcabda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f89bab5d640 1 --2- 172.21.15.22:0/844620598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fca15d0 0x562a9fcabda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f89bab5d640 1 -- 172.21.15.22:0/844620598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a9f9783c0 con 0x562a9fcac370 2024-09-19T08:16:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f89b9b5b640 1 --2- 172.21.15.22:0/844620598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca2b40 0x562a9fca9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:16:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f89bab5d640 1 --2- 172.21.15.22:0/844620598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fcac370 0x562a9fca97e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f89ac007310 tx=0x7f89ac003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d8b4d39d2e8683 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f899dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f899dffb640 1 -- 172.21.15.22:0/844620598 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0052c0 con 0x562a9fcac370 2024-09-19T08:16:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f899dffb640 1 -- 172.21.15.22:0/844620598 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f89ac005460 con 0x562a9fcac370 2024-09-19T08:16:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f89be999740 1 -- 172.21.15.22:0/844620598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89ac007c00 con 0x562a9fcac370 2024-09-19T08:16:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.513+0000 7f899dffb640 1 -- 172.21.15.22:0/844620598 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f89ac013480 con 0x562a9fcac370 2024-09-19T08:16:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.514+0000 7f89be999740 10 client.15825 fetch_fsmap learned FSMap version 26 2024-09-19T08:16:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.514+0000 7f89be999740 1 -- 172.21.15.22:0/844620598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a9fcbaaf0 con 0x562a9fcac370 2024-09-19T08:16:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.514+0000 7f899dffb640 1 -- 172.21.15.22:0/844620598 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ac007c00 con 0x562a9fcac370 2024-09-19T08:16:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.514+0000 7f899dffb640 1 -- 172.21.15.22:0/844620598 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f89ac01c070 con 0x562a9fcac370 2024-09-19T08:16:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.514+0000 7f89be999740 10 client.15825 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:16:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.514+0000 7f89be999740 10 client.15825 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:16:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.514+0000 7f89be999740 20 client.15825 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.514+0000 7f89be999740 1 --2- 172.21.15.22:0/844620598 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x562a9fcbbfb0 0x562a9fcbe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.514+0000 7f89be999740 4 client.15825 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:16:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.514+0000 7f89be999740 1 -- 172.21.15.22:0/844620598 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a9fc14860 con 0x562a9fcbbfb0 2024-09-19T08:16:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.514+0000 7f89ba35c640 1 --2- 172.21.15.22:0/844620598 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x562a9fcbbfb0 0x562a9fcbe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.515+0000 7f899dffb640 10 client.15825 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:16:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.515+0000 7f89ba35c640 1 --2- 172.21.15.22:0/844620598 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x562a9fcbbfb0 0x562a9fcbe3a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86bcea061e5f0d7c server_cookie=9d2fa7c6d2a3c14d in_seq=0 out_seq=0 2024-09-19T08:16:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f899dffb640 1 -- 172.21.15.22:0/844620598 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a9fc14860 con 0x562a9fcbbfb0 2024-09-19T08:16:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f899dffb640 10 client.15825 handle_command_reply: tid=0 2024-09-19T08:16:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f89be999740 1 client.15825 shutdown 2024-09-19T08:16:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f89be999740 1 -- 172.21.15.22:0/844620598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fcac370 msgr2=0x562a9fca97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f89be999740 1 --2- 172.21.15.22:0/844620598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fcac370 0x562a9fca97e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f89ac007310 tx=0x7f89ac003da0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f89be999740 1 -- 172.21.15.22:0/844620598 shutdown_connections 2024-09-19T08:16:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f89be999740 1 --2- 172.21.15.22:0/844620598 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x562a9fcbbfb0 0x562a9fcbe3a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f89be999740 1 --2- 172.21.15.22:0/844620598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fcac370 0x562a9fca97e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f89be999740 1 --2- 172.21.15.22:0/844620598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca2b40 0x562a9fca9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f89be999740 1 --2- 172.21.15.22:0/844620598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fca15d0 0x562a9fcabda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f89be999740 1 -- 172.21.15.22:0/844620598 >> 172.21.15.22:0/844620598 conn(0x562a9fc6a310 msgr2=0x562a9fca3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f899dffb640 0 client.15825 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:16:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f89be999740 1 -- 172.21.15.22:0/844620598 shutdown_connections 2024-09-19T08:16:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f89be999740 1 -- 172.21.15.22:0/844620598 wait complete. 2024-09-19T08:16:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.516+0000 7f89be999740 20 client.15825 trim_cache size 0 max 16384 2024-09-19T08:16:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.529+0000 7f89be999740 1 Processor -- start 2024-09-19T08:16:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.529+0000 7f89be999740 1 -- start start 2024-09-19T08:16:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.529+0000 7f89be999740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca15d0 0x562a9fc9dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.529+0000 7f89be999740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fca2b40 0x562a9fca4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.529+0000 7f89be999740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fcac370 0x562a9fca4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.529+0000 7f89be999740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41adf10 con 0x562a9fcac370 2024-09-19T08:16:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.529+0000 7f89be999740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41ab7e0 con 0x562a9fca2b40 2024-09-19T08:16:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.529+0000 7f89be999740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41a9630 con 0x562a9fca15d0 2024-09-19T08:16:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.529+0000 7f89bab5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fcac370 0x562a9fca4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.529+0000 7f89bab5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fcac370 0x562a9fca4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49360/0 (socket says 172.21.15.22:49360) 2024-09-19T08:16:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.529+0000 7f89bab5d640 1 -- 172.21.15.22:0/609678644 learned_addr learned my addr 172.21.15.22:0/609678644 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.529+0000 7f89ba35c640 1 --2- 172.21.15.22:0/609678644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca15d0 0x562a9fc9dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.530+0000 7f89bab5d640 1 -- 172.21.15.22:0/609678644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca15d0 msgr2=0x562a9fc9dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.530+0000 7f89bab5d640 1 --2- 172.21.15.22:0/609678644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca15d0 0x562a9fc9dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.530+0000 7f89bab5d640 1 -- 172.21.15.22:0/609678644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fca2b40 msgr2=0x562a9fca4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.530+0000 7f89bab5d640 1 --2- 172.21.15.22:0/609678644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fca2b40 0x562a9fca4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.530+0000 7f89bab5d640 1 -- 172.21.15.22:0/609678644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a9fc9c310 con 0x562a9fcac370 2024-09-19T08:16:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.530+0000 7f89bab5d640 1 --2- 172.21.15.22:0/609678644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fcac370 0x562a9fca4660 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f89ac002230 tx=0x7f89ac010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8900de8a8c238b9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.530+0000 7f899ffff640 1 -- 172.21.15.22:0/609678644 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0111f0 con 0x562a9fcac370 2024-09-19T08:16:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.531+0000 7f899ffff640 1 -- 172.21.15.22:0/609678644 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ac02e910 con 0x562a9fcac370 2024-09-19T08:16:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.531+0000 7f899ffff640 1 -- 172.21.15.22:0/609678644 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0056d0 con 0x562a9fcac370 2024-09-19T08:16:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.531+0000 7f89be999740 1 -- 172.21.15.22:0/609678644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fcac370 msgr2=0x562a9fca4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.531+0000 7f89be999740 1 --2- 172.21.15.22:0/609678644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fcac370 0x562a9fca4660 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f89ac002230 tx=0x7f89ac010740 comp rx=0 tx=0).stop 2024-09-19T08:16:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.531+0000 7f89be999740 1 -- 172.21.15.22:0/609678644 shutdown_connections 2024-09-19T08:16:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.531+0000 7f89be999740 1 --2- 172.21.15.22:0/609678644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fcac370 0x562a9fca4660 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.531+0000 7f89be999740 1 --2- 172.21.15.22:0/609678644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fca2b40 0x562a9fca4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.531+0000 7f89be999740 1 --2- 172.21.15.22:0/609678644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fca15d0 0x562a9fc9dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.531+0000 7f89be999740 1 -- 172.21.15.22:0/609678644 >> 172.21.15.22:0/609678644 conn(0x562a9fc6a310 msgr2=0x562a9fca81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:55.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.531+0000 7f89be999740 1 -- 172.21.15.22:0/609678644 shutdown_connections 2024-09-19T08:16:55.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.531+0000 7f89be999740 1 -- 172.21.15.22:0/609678644 wait complete. 2024-09-19T08:16:55.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.532+0000 7f89be999740 1 -- start start 2024-09-19T08:16:55.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.532+0000 7f89be999740 1 Processor -- start 2024-09-19T08:16:55.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.532+0000 7f89be999740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fca15d0 0x562a9fcb47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.532+0000 7f89be999740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fca2b40 0x562a9fcb4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.532+0000 7f89be999740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fcac370 0x562a9fca5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.532+0000 7f89be999740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41a9ca0 con 0x562a9fca15d0 2024-09-19T08:16:55.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.532+0000 7f89be999740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41ab110 con 0x562a9fca2b40 2024-09-19T08:16:55.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.532+0000 7f89be999740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41aa370 con 0x562a9fcac370 2024-09-19T08:16:55.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.532+0000 7f89be999740 10 client.0 _ll_register_callbacks cb 0x7f89a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:55.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.532+0000 7f89ba35c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fca15d0 0x562a9fcb47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f89ba35c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fca15d0 0x562a9fcb47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49376/0 (socket says 172.21.15.22:49376) 2024-09-19T08:16:55.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f89ba35c640 1 -- 172.21.15.22:0/2174623100 learned_addr learned my addr 172.21.15.22:0/2174623100 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:55.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f89bab5d640 1 --2- 172.21.15.22:0/2174623100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fcac370 0x562a9fca5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f89b9b5b640 1 --2- 172.21.15.22:0/2174623100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fca2b40 0x562a9fcb4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f89ba35c640 1 -- 172.21.15.22:0/2174623100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fcac370 msgr2=0x562a9fca5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:55.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f89ba35c640 1 --2- 172.21.15.22:0/2174623100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fcac370 0x562a9fca5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f89ba35c640 1 -- 172.21.15.22:0/2174623100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fca2b40 msgr2=0x562a9fcb4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:55.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f89ba35c640 1 --2- 172.21.15.22:0/2174623100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fca2b40 0x562a9fcb4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f89ba35c640 1 -- 172.21.15.22:0/2174623100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a9fc9e960 con 0x562a9fca15d0 2024-09-19T08:16:55.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f89bab5d640 1 --2- 172.21.15.22:0/2174623100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fcac370 0x562a9fca5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:55.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f89ba35c640 1 --2- 172.21.15.22:0/2174623100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fca15d0 0x562a9fcb47d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009610 tx=0x7f89b006b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469c78c2fb080065 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:55.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f899dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:55.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f899dffb640 1 -- 172.21.15.22:0/2174623100 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b009ce60 con 0x562a9fca15d0 2024-09-19T08:16:55.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f899dffb640 1 -- 172.21.15.22:0/2174623100 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f89b0009cc0 con 0x562a9fca15d0 2024-09-19T08:16:55.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f89be999740 1 -- 172.21.15.22:0/2174623100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89ac02e910 con 0x562a9fca15d0 2024-09-19T08:16:55.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f899dffb640 1 -- 172.21.15.22:0/2174623100 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f89b0020040 con 0x562a9fca15d0 2024-09-19T08:16:55.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f89be999740 10 client.15831 fetch_fsmap learned FSMap version 26 2024-09-19T08:16:55.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.533+0000 7f89be999740 1 -- 172.21.15.22:0/2174623100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a9fd266b0 con 0x562a9fca15d0 2024-09-19T08:16:55.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.534+0000 7f899dffb640 1 -- 172.21.15.22:0/2174623100 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ac02e910 con 0x562a9fca15d0 2024-09-19T08:16:55.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.534+0000 7f899dffb640 1 -- 172.21.15.22:0/2174623100 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f89b00981a0 con 0x562a9fca15d0 2024-09-19T08:16:55.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.534+0000 7f89be999740 10 client.15831 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:16:55.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.534+0000 7f89be999740 10 client.15831 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:16:55.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.534+0000 7f89be999740 20 client.15831 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:55.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.534+0000 7f89be999740 1 --2- 172.21.15.22:0/2174623100 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x562a9fd26e50 0x562a9fd29240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:55.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.534+0000 7f89be999740 4 client.15831 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:16:55.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.534+0000 7f89be999740 1 -- 172.21.15.22:0/2174623100 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a9fc0ed70 con 0x562a9fd26e50 2024-09-19T08:16:55.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.534+0000 7f89b9b5b640 1 --2- 172.21.15.22:0/2174623100 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x562a9fd26e50 0x562a9fd29240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:55.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f899dffb640 10 client.15831 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:16:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f89b9b5b640 1 --2- 172.21.15.22:0/2174623100 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x562a9fd26e50 0x562a9fd29240 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee15cceafa2d4348 server_cookie=1aa814e2b7468f9d in_seq=0 out_seq=0 2024-09-19T08:16:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f899dffb640 1 -- 172.21.15.22:0/2174623100 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a9fc0ed70 con 0x562a9fd26e50 2024-09-19T08:16:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f899dffb640 10 client.15831 handle_command_reply: tid=0 2024-09-19T08:16:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f89be999740 1 client.15831 shutdown 2024-09-19T08:16:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f89be999740 1 -- 172.21.15.22:0/2174623100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fca15d0 msgr2=0x562a9fcb47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f89be999740 1 --2- 172.21.15.22:0/2174623100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fca15d0 0x562a9fcb47d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009610 tx=0x7f89b006b750 comp rx=0 tx=0).stop 2024-09-19T08:16:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f89be999740 1 -- 172.21.15.22:0/2174623100 shutdown_connections 2024-09-19T08:16:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f89be999740 1 --2- 172.21.15.22:0/2174623100 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x562a9fd26e50 0x562a9fd29240 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f89be999740 1 --2- 172.21.15.22:0/2174623100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562a9fcac370 0x562a9fca5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f89be999740 1 --2- 172.21.15.22:0/2174623100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a9fca2b40 0x562a9fcb4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f899dffb640 0 client.15831 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:16:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f89be999740 1 --2- 172.21.15.22:0/2174623100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562a9fca15d0 0x562a9fcb47d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f89be999740 1 -- 172.21.15.22:0/2174623100 >> 172.21.15.22:0/2174623100 conn(0x562a9fc6a310 msgr2=0x562a9fcaa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:55.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f89be999740 1 -- 172.21.15.22:0/2174623100 shutdown_connections 2024-09-19T08:16:55.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f89be999740 1 -- 172.21.15.22:0/2174623100 wait complete. 2024-09-19T08:16:55.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.535+0000 7f89be999740 20 client.15831 trim_cache size 0 max 16384 2024-09-19T08:16:55.554 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:16:55.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.536+0000 7f89bc5c0640 1 -- 172.21.15.22:0/774989494 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f89a00a33b0 msgr2=0x7f89a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:55.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.536+0000 7f89bc5c0640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f89a00a33b0 0x7f89a00a5870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f89ac007fd0 tx=0x7f89ac01b000 comp rx=0 tx=0).stop 2024-09-19T08:16:55.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.536+0000 7f89bc5c0640 1 -- 172.21.15.22:0/774989494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b4102d90 msgr2=0x7f89b4198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:55.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.536+0000 7f89bc5c0640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b4102d90 0x7f89b4198570 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f89b000d770 tx=0x7f89b000dc30 comp rx=0 tx=0).stop 2024-09-19T08:16:55.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.536+0000 7f89bc5c0640 1 -- 172.21.15.22:0/774989494 shutdown_connections 2024-09-19T08:16:55.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.536+0000 7f89bc5c0640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f89a00a33b0 0x7f89a00a5870 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.536+0000 7f89bc5c0640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b4108290 0x7f89b419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.536+0000 7f89bc5c0640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4103740 0x7f89b4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.536+0000 7f89bc5c0640 1 --2- 172.21.15.22:0/774989494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b4102d90 0x7f89b4198570 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:55.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.536+0000 7f89bc5c0640 1 -- 172.21.15.22:0/774989494 >> 172.21.15.22:0/774989494 conn(0x7f89b40fc7d0 msgr2=0x7f89b4105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:55.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.536+0000 7f89bc5c0640 1 -- 172.21.15.22:0/774989494 shutdown_connections 2024-09-19T08:16:55.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:55.536+0000 7f89bc5c0640 1 -- 172.21.15.22:0/774989494 wait complete. 2024-09-19T08:16:56.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:16:56.521 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:56 smithi022 ceph-mon[30963]: pgmap v60: 97 pgs: 97 active+clean; 4.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 291 KiB/s wr, 30 op/s 2024-09-19T08:16:57.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.008+0000 7fe551a10640 1 -- 172.21.15.22:0/112123139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe54c1082a0 msgr2=0x7fe54c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:57.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.008+0000 7fe551a10640 1 --2- 172.21.15.22:0/112123139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe54c1082a0 0x7fe54c10ec40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009cd0 tx=0x7fe53c02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.008+0000 7fe551a10640 1 -- 172.21.15.22:0/112123139 shutdown_connections 2024-09-19T08:16:57.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.008+0000 7fe551a10640 1 --2- 172.21.15.22:0/112123139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe54c1082a0 0x7fe54c10ec40 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.008+0000 7fe551a10640 1 --2- 172.21.15.22:0/112123139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe54c1036e0 0x7fe54c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.008+0000 7fe551a10640 1 --2- 172.21.15.22:0/112123139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c102d30 0x7fe54c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.008+0000 7fe551a10640 1 -- 172.21.15.22:0/112123139 >> 172.21.15.22:0/112123139 conn(0x7fe54c0fc7d0 msgr2=0x7fe54c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:57.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.008+0000 7fe551a10640 1 -- 172.21.15.22:0/112123139 shutdown_connections 2024-09-19T08:16:57.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.008+0000 7fe551a10640 1 -- 172.21.15.22:0/112123139 wait complete. 2024-09-19T08:16:57.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.008+0000 7fe551a10640 1 Processor -- start 2024-09-19T08:16:57.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe551a10640 1 -- start start 2024-09-19T08:16:57.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe551a10640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe54c102d30 0x7fe54c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe551a10640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe54c1036e0 0x7fe54c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe551a10640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c1082a0 0x7fe54c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe551a10640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1120a0 con 0x7fe54c102d30 2024-09-19T08:16:57.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe551a10640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c111f20 con 0x7fe54c1082a0 2024-09-19T08:16:57.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe551a10640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c112220 con 0x7fe54c1036e0 2024-09-19T08:16:57.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe54affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe54c102d30 0x7fe54c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe54affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe54c102d30 0x7fe54c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49406/0 (socket says 172.21.15.22:49406) 2024-09-19T08:16:57.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe54affd640 1 -- 172.21.15.22:0/1194579283 learned_addr learned my addr 172.21.15.22:0/1194579283 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:57.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe54b7fe640 1 --2- 172.21.15.22:0/1194579283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c1082a0 0x7fe54c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe54a7fc640 1 --2- 172.21.15.22:0/1194579283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe54c1036e0 0x7fe54c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe54affd640 1 -- 172.21.15.22:0/1194579283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe54c1036e0 msgr2=0x7fe54c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:57.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe54affd640 1 --2- 172.21.15.22:0/1194579283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe54c1036e0 0x7fe54c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe54affd640 1 -- 172.21.15.22:0/1194579283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c1082a0 msgr2=0x7fe54c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:57.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe54affd640 1 --2- 172.21.15.22:0/1194579283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c1082a0 0x7fe54c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.009+0000 7fe54affd640 1 -- 172.21.15.22:0/1194579283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c1a6a20 con 0x7fe54c102d30 2024-09-19T08:16:57.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.010+0000 7fe54affd640 1 --2- 172.21.15.22:0/1194579283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe54c102d30 0x7fe54c100990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe54400ea60 tx=0x7fe54400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:57.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.010+0000 7fe550a0e640 1 -- 172.21.15.22:0/1194579283 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54400cde0 con 0x7fe54c102d30 2024-09-19T08:16:57.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.010+0000 7fe550a0e640 1 -- 172.21.15.22:0/1194579283 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5440035f0 con 0x7fe54c102d30 2024-09-19T08:16:57.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.010+0000 7fe551a10640 1 -- 172.21.15.22:0/1194579283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54c1a6d10 con 0x7fe54c102d30 2024-09-19T08:16:57.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.010+0000 7fe550a0e640 1 -- 172.21.15.22:0/1194579283 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544010670 con 0x7fe54c102d30 2024-09-19T08:16:57.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.010+0000 7fe551a10640 1 -- 172.21.15.22:0/1194579283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54c1a71f0 con 0x7fe54c102d30 2024-09-19T08:16:57.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.012+0000 7fe550a0e640 1 -- 172.21.15.22:0/1194579283 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe5440037a0 con 0x7fe54c102d30 2024-09-19T08:16:57.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.012+0000 7fe553de9740 1 Processor -- start 2024-09-19T08:16:57.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.012+0000 7fe553de9740 1 -- start start 2024-09-19T08:16:57.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.012+0000 7fe553de9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 0x561f9f274090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.012+0000 7fe553de9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f2745d0 0x561f9f274a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.012+0000 7fe553de9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f27f370 0x561f9f281760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe553de9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561f9f1da2f0 con 0x561f9f2745d0 2024-09-19T08:16:57.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe553de9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561f9f181cf0 con 0x561f9f275b40 2024-09-19T08:16:57.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe553de9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561f9f1d62b0 con 0x561f9f27f370 2024-09-19T08:16:57.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe54affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f2745d0 0x561f9f274a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe54affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f2745d0 0x561f9f274a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49412/0 (socket says 172.21.15.22:49412) 2024-09-19T08:16:57.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe54affd640 1 -- 172.21.15.22:0/3513819745 learned_addr learned my addr 172.21.15.22:0/3513819745 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:57.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe54a7fc640 1 --2- 172.21.15.22:0/3513819745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f27f370 0x561f9f281760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe54b7fe640 1 --2- 172.21.15.22:0/3513819745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 0x561f9f274090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe550a0e640 1 --2- 172.21.15.22:0/1194579283 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe5380a33b0 0x7fe5380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe54affd640 1 -- 172.21.15.22:0/3513819745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f27f370 msgr2=0x561f9f281760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:57.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe54affd640 1 --2- 172.21.15.22:0/3513819745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f27f370 0x561f9f281760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe54affd640 1 -- 172.21.15.22:0/3513819745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 msgr2=0x561f9f274090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:57.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe54affd640 1 --2- 172.21.15.22:0/3513819745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 0x561f9f274090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe54affd640 1 -- 172.21.15.22:0/3513819745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f9f1663a0 con 0x561f9f2745d0 2024-09-19T08:16:57.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe54b7fe640 1 --2- 172.21.15.22:0/1194579283 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe5380a33b0 0x7fe5380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe550a0e640 1 -- 172.21.15.22:0/1194579283 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe5440d0070 con 0x7fe54c102d30 2024-09-19T08:16:57.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.013+0000 7fe54affd640 1 --2- 172.21.15.22:0/3513819745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f2745d0 0x561f9f274a10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe5440cd740 tx=0x7fe5440b04c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e72b5de8045174 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:57.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.014+0000 7fe541ffb640 1 -- 172.21.15.22:0/3513819745 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54409bea0 con 0x561f9f2745d0 2024-09-19T08:16:57.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.014+0000 7fe541ffb640 1 -- 172.21.15.22:0/3513819745 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54409ca60 con 0x561f9f2745d0 2024-09-19T08:16:57.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.014+0000 7fe541ffb640 1 -- 172.21.15.22:0/3513819745 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54409bcc0 con 0x561f9f2745d0 2024-09-19T08:16:57.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.014+0000 7fe553de9740 1 -- 172.21.15.22:0/3513819745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f2745d0 msgr2=0x561f9f274a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:57.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.014+0000 7fe54b7fe640 1 --2- 172.21.15.22:0/1194579283 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe5380a33b0 0x7fe5380a5870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe53c004ab0 tx=0x7fe53c0095c0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:57.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.014+0000 7fe553de9740 1 --2- 172.21.15.22:0/3513819745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f2745d0 0x561f9f274a10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe5440cd740 tx=0x7fe5440b04c0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.014+0000 7fe553de9740 1 -- 172.21.15.22:0/3513819745 shutdown_connections 2024-09-19T08:16:57.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.014+0000 7fe553de9740 1 --2- 172.21.15.22:0/3513819745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f27f370 0x561f9f281760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.014+0000 7fe553de9740 1 --2- 172.21.15.22:0/3513819745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f2745d0 0x561f9f274a10 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.014+0000 7fe553de9740 1 --2- 172.21.15.22:0/3513819745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 0x561f9f274090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.014+0000 7fe553de9740 1 -- 172.21.15.22:0/3513819745 >> 172.21.15.22:0/3513819745 conn(0x561f9f23d310 msgr2=0x561f9f270c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:57.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.014+0000 7fe553de9740 1 -- 172.21.15.22:0/3513819745 shutdown_connections 2024-09-19T08:16:57.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.014+0000 7fe553de9740 1 -- 172.21.15.22:0/3513819745 wait complete. 2024-09-19T08:16:57.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.015+0000 7fe553de9740 1 -- start start 2024-09-19T08:16:57.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.015+0000 7fe553de9740 1 Processor -- start 2024-09-19T08:16:57.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.015+0000 7fe553de9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f2745d0 0x561f9f27ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.015+0000 7fe553de9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 0x561f9f27bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.015+0000 7fe553de9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f27c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.015+0000 7fe553de9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561f9f282070 con 0x561f9f27f370 2024-09-19T08:16:57.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.015+0000 7fe553de9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561f9f195060 con 0x561f9f275b40 2024-09-19T08:16:57.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.015+0000 7fe553de9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561f9f2821f0 con 0x561f9f2745d0 2024-09-19T08:16:57.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.015+0000 7fe553de9740 10 client.0 _ll_register_callbacks cb 0x561f9eee88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:57.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.015+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f27c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.015+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f27c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49422/0 (socket says 172.21.15.22:49422) 2024-09-19T08:16:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.015+0000 7fe54b7fe640 1 -- 172.21.15.22:0/2243656239 learned_addr learned my addr 172.21.15.22:0/2243656239 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.015+0000 7fe54affd640 1 --2- 172.21.15.22:0/2243656239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f2745d0 0x561f9f27ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.016+0000 7fe54a7fc640 1 --2- 172.21.15.22:0/2243656239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 0x561f9f27bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.016+0000 7fe54b7fe640 1 -- 172.21.15.22:0/2243656239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f2745d0 msgr2=0x561f9f27ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.016+0000 7fe54b7fe640 1 --2- 172.21.15.22:0/2243656239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f2745d0 0x561f9f27ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.016+0000 7fe54b7fe640 1 -- 172.21.15.22:0/2243656239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 msgr2=0x561f9f27bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.016+0000 7fe54b7fe640 1 --2- 172.21.15.22:0/2243656239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 0x561f9f27bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.016+0000 7fe54b7fe640 1 -- 172.21.15.22:0/2243656239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f9ef4b3c0 con 0x561f9f27f370 2024-09-19T08:16:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.016+0000 7fe54b7fe640 1 --2- 172.21.15.22:0/2243656239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f27c570 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009cd0 tx=0x7fe53c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be34a0ffe9a48f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:57.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.016+0000 7fe52affd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:57.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.016+0000 7fe52affd640 1 -- 172.21.15.22:0/2243656239 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c004720 con 0x561f9f27f370 2024-09-19T08:16:57.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.016+0000 7fe52affd640 1 -- 172.21.15.22:0/2243656239 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe53c0305a0 con 0x561f9f27f370 2024-09-19T08:16:57.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.016+0000 7fe553de9740 1 -- 172.21.15.22:0/2243656239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe54409ca60 con 0x561f9f27f370 2024-09-19T08:16:57.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.016+0000 7fe52affd640 1 -- 172.21.15.22:0/2243656239 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe53c0307a0 con 0x561f9f27f370 2024-09-19T08:16:57.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.016+0000 7fe553de9740 10 client.15843 fetch_fsmap learned FSMap version 26 2024-09-19T08:16:57.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.016+0000 7fe553de9740 1 -- 172.21.15.22:0/2243656239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561f9f28db20 con 0x561f9f27f370 2024-09-19T08:16:57.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.017+0000 7fe52affd640 1 -- 172.21.15.22:0/2243656239 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54409ca60 con 0x561f9f27f370 2024-09-19T08:16:57.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.017+0000 7fe52affd640 1 -- 172.21.15.22:0/2243656239 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fe53c030070 con 0x561f9f27f370 2024-09-19T08:16:57.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.017+0000 7fe553de9740 10 client.15843 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:16:57.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.017+0000 7fe553de9740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:16:57.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.017+0000 7fe553de9740 20 client.15843 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:57.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.017+0000 7fe553de9740 1 --2- 172.21.15.22:0/2243656239 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f9f28f080 0x561f9f2914e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.017+0000 7fe553de9740 4 client.15843 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:16:57.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.017+0000 7fe553de9740 1 -- 172.21.15.22:0/2243656239 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f9f1e7860 con 0x561f9f28f080 2024-09-19T08:16:57.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.017+0000 7fe54affd640 1 --2- 172.21.15.22:0/2243656239 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f9f28f080 0x561f9f2914e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.018+0000 7fe54affd640 1 --2- 172.21.15.22:0/2243656239 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f9f28f080 0x561f9f2914e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ef9823b01f835c server_cookie=89cdc98da3a2044c in_seq=0 out_seq=0 2024-09-19T08:16:57.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.018+0000 7fe52affd640 10 client.15843 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:57.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.018+0000 7fe52affd640 1 -- 172.21.15.22:0/2243656239 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f9f1e7860 con 0x561f9f28f080 2024-09-19T08:16:57.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.018+0000 7fe52affd640 10 client.15843 handle_command_reply: tid=0 2024-09-19T08:16:57.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.019+0000 7fe553de9740 1 client.15843 shutdown 2024-09-19T08:16:57.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.019+0000 7fe553de9740 1 -- 172.21.15.22:0/2243656239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 msgr2=0x561f9f27c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:57.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.019+0000 7fe553de9740 1 --2- 172.21.15.22:0/2243656239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f27c570 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009cd0 tx=0x7fe53c02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.019+0000 7fe553de9740 1 -- 172.21.15.22:0/2243656239 shutdown_connections 2024-09-19T08:16:57.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.019+0000 7fe553de9740 1 --2- 172.21.15.22:0/2243656239 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f9f28f080 0x561f9f2914e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.019+0000 7fe553de9740 1 --2- 172.21.15.22:0/2243656239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f27c570 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.019+0000 7fe553de9740 1 --2- 172.21.15.22:0/2243656239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 0x561f9f27bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.019+0000 7fe553de9740 1 --2- 172.21.15.22:0/2243656239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f2745d0 0x561f9f27ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.019+0000 7fe52affd640 0 client.15843 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:57.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.019+0000 7fe553de9740 1 -- 172.21.15.22:0/2243656239 >> 172.21.15.22:0/2243656239 conn(0x561f9f23d310 msgr2=0x561f9f276930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:57.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.019+0000 7fe553de9740 1 -- 172.21.15.22:0/2243656239 shutdown_connections 2024-09-19T08:16:57.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.019+0000 7fe553de9740 1 -- 172.21.15.22:0/2243656239 wait complete. 2024-09-19T08:16:57.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.019+0000 7fe553de9740 20 client.15843 trim_cache size 0 max 16384 2024-09-19T08:16:57.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.032+0000 7fe553de9740 1 Processor -- start 2024-09-19T08:16:57.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.032+0000 7fe553de9740 1 -- start start 2024-09-19T08:16:57.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.032+0000 7fe553de9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f2745d0 0x561f9f270e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.032+0000 7fe553de9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 0x561f9f28b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.032+0000 7fe553de9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f296df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.032+0000 7fe553de9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1b2410 con 0x561f9f27f370 2024-09-19T08:16:57.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.032+0000 7fe553de9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1afce0 con 0x561f9f275b40 2024-09-19T08:16:57.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.032+0000 7fe553de9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1adb80 con 0x561f9f2745d0 2024-09-19T08:16:57.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.032+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f296df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.032+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f296df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49428/0 (socket says 172.21.15.22:49428) 2024-09-19T08:16:57.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.033+0000 7fe54b7fe640 1 -- 172.21.15.22:0/1784257289 learned_addr learned my addr 172.21.15.22:0/1784257289 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.033+0000 7fe54affd640 1 --2- 172.21.15.22:0/1784257289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f2745d0 0x561f9f270e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.033+0000 7fe54a7fc640 1 --2- 172.21.15.22:0/1784257289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 0x561f9f28b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.033+0000 7fe54b7fe640 1 -- 172.21.15.22:0/1784257289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f2745d0 msgr2=0x561f9f270e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.033+0000 7fe54b7fe640 1 --2- 172.21.15.22:0/1784257289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f2745d0 0x561f9f270e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.033+0000 7fe54b7fe640 1 -- 172.21.15.22:0/1784257289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 msgr2=0x561f9f28b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.033+0000 7fe54b7fe640 1 --2- 172.21.15.22:0/1784257289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 0x561f9f28b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.033+0000 7fe54b7fe640 1 -- 172.21.15.22:0/1784257289 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f9f27c7b0 con 0x561f9f27f370 2024-09-19T08:16:57.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.033+0000 7fe54b7fe640 1 --2- 172.21.15.22:0/1784257289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f296df0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe53c02ae00 tx=0x7fe53c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a6692587e952bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:57.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.033+0000 7fe5417fa640 1 -- 172.21.15.22:0/1784257289 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c0305a0 con 0x561f9f27f370 2024-09-19T08:16:57.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.033+0000 7fe5417fa640 1 -- 172.21.15.22:0/1784257289 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe53c036040 con 0x561f9f27f370 2024-09-19T08:16:57.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.033+0000 7fe5417fa640 1 -- 172.21.15.22:0/1784257289 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c058660 con 0x561f9f27f370 2024-09-19T08:16:57.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.034+0000 7fe553de9740 1 -- 172.21.15.22:0/1784257289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 msgr2=0x561f9f296df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:57.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.034+0000 7fe553de9740 1 --2- 172.21.15.22:0/1784257289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f296df0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe53c02ae00 tx=0x7fe53c004a40 comp rx=0 tx=0).stop 2024-09-19T08:16:57.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.034+0000 7fe553de9740 1 -- 172.21.15.22:0/1784257289 shutdown_connections 2024-09-19T08:16:57.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.034+0000 7fe553de9740 1 --2- 172.21.15.22:0/1784257289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f296df0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.034+0000 7fe553de9740 1 --2- 172.21.15.22:0/1784257289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f275b40 0x561f9f28b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.034+0000 7fe553de9740 1 --2- 172.21.15.22:0/1784257289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f2745d0 0x561f9f270e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.034+0000 7fe553de9740 1 -- 172.21.15.22:0/1784257289 >> 172.21.15.22:0/1784257289 conn(0x561f9f23d310 msgr2=0x561f9f270700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:57.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.034+0000 7fe553de9740 1 -- 172.21.15.22:0/1784257289 shutdown_connections 2024-09-19T08:16:57.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.034+0000 7fe553de9740 1 -- 172.21.15.22:0/1784257289 wait complete. 2024-09-19T08:16:57.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.034+0000 7fe553de9740 1 -- start start 2024-09-19T08:16:57.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.034+0000 7fe553de9740 1 Processor -- start 2024-09-19T08:16:57.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe553de9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f2745d0 0x561f9f2790e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe553de9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f275b40 0x561f9f279620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe553de9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f279b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe553de9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1ae1a0 con 0x561f9f27f370 2024-09-19T08:16:57.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe553de9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1af610 con 0x561f9f2745d0 2024-09-19T08:16:57.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe553de9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1ae870 con 0x561f9f275b40 2024-09-19T08:16:57.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe553de9740 10 client.0 _ll_register_callbacks cb 0x7fe53c009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:57.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f279b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f279b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49438/0 (socket says 172.21.15.22:49438) 2024-09-19T08:16:57.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe54b7fe640 1 -- 172.21.15.22:0/2528937513 learned_addr learned my addr 172.21.15.22:0/2528937513 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:57.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe54affd640 1 --2- 172.21.15.22:0/2528937513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f2745d0 0x561f9f2790e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe54a7fc640 1 --2- 172.21.15.22:0/2528937513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f275b40 0x561f9f279620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe54b7fe640 1 -- 172.21.15.22:0/2528937513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f275b40 msgr2=0x561f9f279620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:57.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe54b7fe640 1 --2- 172.21.15.22:0/2528937513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f275b40 0x561f9f279620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe54b7fe640 1 -- 172.21.15.22:0/2528937513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f2745d0 msgr2=0x561f9f2790e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:57.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe54b7fe640 1 --2- 172.21.15.22:0/2528937513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f2745d0 0x561f9f2790e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe54b7fe640 1 -- 172.21.15.22:0/2528937513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f9f2f8e90 con 0x561f9f27f370 2024-09-19T08:16:57.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe54b7fe640 1 --2- 172.21.15.22:0/2528937513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f279b60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe53c045a70 tx=0x7fe53c004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0db5eade0e0317e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:57.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.035+0000 7fe52affd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:57.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe52affd640 1 -- 172.21.15.22:0/2528937513 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c0305a0 con 0x561f9f27f370 2024-09-19T08:16:57.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe52affd640 1 -- 172.21.15.22:0/2528937513 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe53c030b50 con 0x561f9f27f370 2024-09-19T08:16:57.064 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:16:57.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe553de9740 1 -- 172.21.15.22:0/2528937513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe53c036040 con 0x561f9f27f370 2024-09-19T08:16:57.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe52affd640 1 -- 172.21.15.22:0/2528937513 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe53c070050 con 0x561f9f27f370 2024-09-19T08:16:57.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe553de9740 10 client.15849 fetch_fsmap learned FSMap version 26 2024-09-19T08:16:57.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe553de9740 1 -- 172.21.15.22:0/2528937513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561f9f2fa170 con 0x561f9f27f370 2024-09-19T08:16:57.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe52affd640 1 -- 172.21.15.22:0/2528937513 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe53c036040 con 0x561f9f27f370 2024-09-19T08:16:57.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe52affd640 1 -- 172.21.15.22:0/2528937513 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fe53c06b030 con 0x561f9f27f370 2024-09-19T08:16:57.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe553de9740 10 client.15849 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:16:57.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe553de9740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:16:57.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe553de9740 20 client.15849 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:57.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe553de9740 1 --2- 172.21.15.22:0/2528937513 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f9f2fa7d0 0x561f9f2fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:57.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe553de9740 4 client.15849 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:16:57.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe553de9740 1 -- 172.21.15.22:0/2528937513 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561f9f1e1d70 con 0x561f9f2fa7d0 2024-09-19T08:16:57.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.036+0000 7fe54affd640 1 --2- 172.21.15.22:0/2528937513 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f9f2fa7d0 0x561f9f2fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:57.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe52affd640 10 client.15849 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:57.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe54affd640 1 --2- 172.21.15.22:0/2528937513 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f9f2fa7d0 0x561f9f2fcbc0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25947d60d915493d server_cookie=b7c2dc0954ef40ee in_seq=0 out_seq=0 2024-09-19T08:16:57.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe52affd640 1 -- 172.21.15.22:0/2528937513 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561f9f1e1d70 con 0x561f9f2fa7d0 2024-09-19T08:16:57.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe52affd640 10 client.15849 handle_command_reply: tid=0 2024-09-19T08:16:57.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe553de9740 1 client.15849 shutdown 2024-09-19T08:16:57.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe553de9740 1 -- 172.21.15.22:0/2528937513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 msgr2=0x561f9f279b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:57.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe553de9740 1 --2- 172.21.15.22:0/2528937513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f279b60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe53c045a70 tx=0x7fe53c004480 comp rx=0 tx=0).stop 2024-09-19T08:16:57.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe553de9740 1 -- 172.21.15.22:0/2528937513 shutdown_connections 2024-09-19T08:16:57.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe553de9740 1 --2- 172.21.15.22:0/2528937513 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f9f2fa7d0 0x561f9f2fcbc0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe553de9740 1 --2- 172.21.15.22:0/2528937513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f9f27f370 0x561f9f279b60 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe553de9740 1 --2- 172.21.15.22:0/2528937513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f9f275b40 0x561f9f279620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe553de9740 1 --2- 172.21.15.22:0/2528937513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f9f2745d0 0x561f9f2790e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe52affd640 0 client.15849 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:57.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe553de9740 1 -- 172.21.15.22:0/2528937513 >> 172.21.15.22:0/2528937513 conn(0x561f9f23d310 msgr2=0x561f9f27a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:57.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe553de9740 1 -- 172.21.15.22:0/2528937513 shutdown_connections 2024-09-19T08:16:57.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.037+0000 7fe553de9740 1 -- 172.21.15.22:0/2528937513 wait complete. 2024-09-19T08:16:57.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.038+0000 7fe553de9740 20 client.15849 trim_cache size 0 max 16384 2024-09-19T08:16:57.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.038+0000 7fe551a10640 1 -- 172.21.15.22:0/1194579283 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe5380a33b0 msgr2=0x7fe5380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:57.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.038+0000 7fe551a10640 1 --2- 172.21.15.22:0/1194579283 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe5380a33b0 0x7fe5380a5870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe53c004ab0 tx=0x7fe53c0095c0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.038+0000 7fe551a10640 1 -- 172.21.15.22:0/1194579283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe54c102d30 msgr2=0x7fe54c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:57.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.038+0000 7fe551a10640 1 --2- 172.21.15.22:0/1194579283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe54c102d30 0x7fe54c100990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe54400ea60 tx=0x7fe54400ef20 comp rx=0 tx=0).stop 2024-09-19T08:16:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.038+0000 7fe551a10640 1 -- 172.21.15.22:0/1194579283 shutdown_connections 2024-09-19T08:16:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.038+0000 7fe551a10640 1 --2- 172.21.15.22:0/1194579283 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe5380a33b0 0x7fe5380a5870 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.038+0000 7fe551a10640 1 --2- 172.21.15.22:0/1194579283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c1082a0 0x7fe54c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.038+0000 7fe551a10640 1 --2- 172.21.15.22:0/1194579283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe54c1036e0 0x7fe54c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.038+0000 7fe551a10640 1 --2- 172.21.15.22:0/1194579283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe54c102d30 0x7fe54c100990 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:57.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.038+0000 7fe551a10640 1 -- 172.21.15.22:0/1194579283 >> 172.21.15.22:0/1194579283 conn(0x7fe54c0fc7d0 msgr2=0x7fe54c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:57.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.038+0000 7fe551a10640 1 -- 172.21.15.22:0/1194579283 shutdown_connections 2024-09-19T08:16:57.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:57.038+0000 7fe551a10640 1 -- 172.21.15.22:0/1194579283 wait complete. 2024-09-19T08:16:57.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:56 smithi157 ceph-mon[35143]: pgmap v60: 97 pgs: 97 active+clean; 4.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 291 KiB/s wr, 30 op/s 2024-09-19T08:16:57.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:56 smithi064 ceph-mon[34900]: pgmap v60: 97 pgs: 97 active+clean; 4.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 291 KiB/s wr, 30 op/s 2024-09-19T08:16:58.761 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:16:58.761 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:16:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:16:59.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:58 smithi022 ceph-mon[30963]: pgmap v61: 97 pgs: 97 active+clean; 4.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 302 KiB/s wr, 43 op/s 2024-09-19T08:16:59.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:58 smithi157 ceph-mon[35143]: pgmap v61: 97 pgs: 97 active+clean; 4.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 302 KiB/s wr, 43 op/s 2024-09-19T08:16:59.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:58 smithi064 ceph-mon[34900]: pgmap v61: 97 pgs: 97 active+clean; 4.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 302 KiB/s wr, 43 op/s 2024-09-19T08:16:59.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.509+0000 7fe068860640 1 -- 172.21.15.22:0/2636004265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe064103660 msgr2=0x7fe06410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:59.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.509+0000 7fe068860640 1 --2- 172.21.15.22:0/2636004265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe064103660 0x7fe06410bb50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe058009900 tx=0x7fe05802a480 comp rx=0 tx=0).stop 2024-09-19T08:16:59.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.510+0000 7fe068860640 1 -- 172.21.15.22:0/2636004265 shutdown_connections 2024-09-19T08:16:59.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.510+0000 7fe068860640 1 --2- 172.21.15.22:0/2636004265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe06410c170 0x7fe06410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.510+0000 7fe068860640 1 --2- 172.21.15.22:0/2636004265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe064103660 0x7fe06410bb50 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.510+0000 7fe068860640 1 --2- 172.21.15.22:0/2636004265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe064102d40 0x7fe064103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.510+0000 7fe068860640 1 -- 172.21.15.22:0/2636004265 >> 172.21.15.22:0/2636004265 conn(0x7fe0640fc7a0 msgr2=0x7fe0640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.510+0000 7fe068860640 1 -- 172.21.15.22:0/2636004265 shutdown_connections 2024-09-19T08:16:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.510+0000 7fe068860640 1 -- 172.21.15.22:0/2636004265 wait complete. 2024-09-19T08:16:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.510+0000 7fe068860640 1 Processor -- start 2024-09-19T08:16:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.510+0000 7fe068860640 1 -- start start 2024-09-19T08:16:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe068860640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe064102d40 0x7fe0641985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe068860640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe064103660 0x7fe064198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe068860640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe06410c170 0x7fe06419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe068860640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe064111fb0 con 0x7fe06410c170 2024-09-19T08:16:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe068860640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe064111e30 con 0x7fe064102d40 2024-09-19T08:16:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe068860640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe064112130 con 0x7fe064103660 2024-09-19T08:16:59.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe062d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe06410c170 0x7fe06419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe062d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe06410c170 0x7fe06419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41556/0 (socket says 172.21.15.22:41556) 2024-09-19T08:16:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe062d9d640 1 -- 172.21.15.22:0/3642112481 learned_addr learned my addr 172.21.15.22:0/3642112481 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe06259c640 1 --2- 172.21.15.22:0/3642112481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe064102d40 0x7fe0641985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe061d9b640 1 --2- 172.21.15.22:0/3642112481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe064103660 0x7fe064198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe062d9d640 1 -- 172.21.15.22:0/3642112481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe064103660 msgr2=0x7fe064198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe062d9d640 1 --2- 172.21.15.22:0/3642112481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe064103660 0x7fe064198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe062d9d640 1 -- 172.21.15.22:0/3642112481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe064102d40 msgr2=0x7fe0641985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe062d9d640 1 --2- 172.21.15.22:0/3642112481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe064102d40 0x7fe0641985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe062d9d640 1 -- 172.21.15.22:0/3642112481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06419d560 con 0x7fe06410c170 2024-09-19T08:16:59.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe062d9d640 1 --2- 172.21.15.22:0/3642112481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe06410c170 0x7fe06419ce80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00eaa0 tx=0x7fe05c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:59.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe0537fe640 1 -- 172.21.15.22:0/3642112481 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c004d20 con 0x7fe06410c170 2024-09-19T08:16:59.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe0537fe640 1 -- 172.21.15.22:0/3642112481 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe05c005020 con 0x7fe06410c170 2024-09-19T08:16:59.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe068860640 1 -- 172.21.15.22:0/3642112481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06419d850 con 0x7fe06410c170 2024-09-19T08:16:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.511+0000 7fe0537fe640 1 -- 172.21.15.22:0/3642112481 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c0116d0 con 0x7fe06410c170 2024-09-19T08:16:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.512+0000 7fe068860640 1 -- 172.21.15.22:0/3642112481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0641a5130 con 0x7fe06410c170 2024-09-19T08:16:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.514+0000 7fe0537fe640 1 -- 172.21.15.22:0/3642112481 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe05c0051d0 con 0x7fe06410c170 2024-09-19T08:16:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.514+0000 7fe06ac39740 1 Processor -- start 2024-09-19T08:16:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.514+0000 7fe06ac39740 1 -- start start 2024-09-19T08:16:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.514+0000 7fe06ac39740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 0x561f5ae2df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.514+0000 7fe06ac39740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2e480 0x561f5ae38800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe06ac39740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae38d40 0x561f5ae3b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe06ac39740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561f5ad93280 con 0x561f5ae38d40 2024-09-19T08:16:59.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe06ac39740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561f5ad3ab10 con 0x561f5ae2e480 2024-09-19T08:16:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe06ac39740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561f5ad8f240 con 0x561f5ae2db60 2024-09-19T08:16:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe0537fe640 1 --2- 172.21.15.22:0/3642112481 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe04c0a3430 0x7fe04c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe061d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae38d40 0x561f5ae3b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe061d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae38d40 0x561f5ae3b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41568/0 (socket says 172.21.15.22:41568) 2024-09-19T08:16:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe061d9b640 1 -- 172.21.15.22:0/908309401 learned_addr learned my addr 172.21.15.22:0/908309401 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe061d9b640 1 --2- 172.21.15.22:0/908309401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 0x561f5ae2df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe06259c640 1 --2- 172.21.15.22:0/908309401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2e480 0x561f5ae38800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe062d9d640 1 --2- 172.21.15.22:0/3642112481 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe04c0a3430 0x7fe04c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe0537fe640 1 -- 172.21.15.22:0/3642112481 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe05c0cfed0 con 0x7fe06410c170 2024-09-19T08:16:59.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe061d9b640 1 -- 172.21.15.22:0/908309401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 msgr2=0x561f5ae2df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:59.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe061d9b640 1 --2- 172.21.15.22:0/908309401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 0x561f5ae2df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe061d9b640 1 -- 172.21.15.22:0/908309401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2e480 msgr2=0x561f5ae38800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:59.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe061d9b640 1 --2- 172.21.15.22:0/908309401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2e480 0x561f5ae38800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe061d9b640 1 -- 172.21.15.22:0/908309401 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5ad471f0 con 0x561f5ae38d40 2024-09-19T08:16:59.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe06259c640 1 --2- 172.21.15.22:0/908309401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2e480 0x561f5ae38800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:16:59.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe062d9d640 1 --2- 172.21.15.22:0/3642112481 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe04c0a3430 0x7fe04c0a58f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe05c098b90 tx=0x7fe05c0bc040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:59.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.515+0000 7fe061d9b640 1 --2- 172.21.15.22:0/908309401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae38d40 0x561f5ae3b130 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe058009900 tx=0x7fe0580056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e140ba553f424e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:59.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.516+0000 7fe050ff9640 1 -- 172.21.15.22:0/908309401 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058035040 con 0x561f5ae38d40 2024-09-19T08:16:59.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.516+0000 7fe050ff9640 1 -- 172.21.15.22:0/908309401 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0580035f0 con 0x561f5ae38d40 2024-09-19T08:16:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.516+0000 7fe050ff9640 1 -- 172.21.15.22:0/908309401 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05803d6a0 con 0x561f5ae38d40 2024-09-19T08:16:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.516+0000 7fe06ac39740 1 -- 172.21.15.22:0/908309401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae38d40 msgr2=0x561f5ae3b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.516+0000 7fe06ac39740 1 --2- 172.21.15.22:0/908309401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae38d40 0x561f5ae3b130 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe058009900 tx=0x7fe0580056d0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.516+0000 7fe06ac39740 1 -- 172.21.15.22:0/908309401 shutdown_connections 2024-09-19T08:16:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.516+0000 7fe06ac39740 1 --2- 172.21.15.22:0/908309401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae38d40 0x561f5ae3b130 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.516+0000 7fe06ac39740 1 --2- 172.21.15.22:0/908309401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2e480 0x561f5ae38800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.516+0000 7fe06ac39740 1 --2- 172.21.15.22:0/908309401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 0x561f5ae2df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.516+0000 7fe06ac39740 1 -- 172.21.15.22:0/908309401 >> 172.21.15.22:0/908309401 conn(0x561f5ae2a0f0 msgr2=0x561f5ae2a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.516+0000 7fe06ac39740 1 -- 172.21.15.22:0/908309401 shutdown_connections 2024-09-19T08:16:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.516+0000 7fe06ac39740 1 -- 172.21.15.22:0/908309401 wait complete. 2024-09-19T08:16:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.517+0000 7fe06ac39740 1 -- start start 2024-09-19T08:16:59.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.517+0000 7fe06ac39740 1 Processor -- start 2024-09-19T08:16:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.517+0000 7fe06ac39740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 0x561f5ae35c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.517+0000 7fe06ac39740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae2e480 0x561f5ae36150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.517+0000 7fe06ac39740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae38d40 0x561f5ae36690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.517+0000 7fe06ac39740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561f5ae3b910 con 0x561f5ae2e480 2024-09-19T08:16:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.517+0000 7fe06ac39740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561f5ad4df10 con 0x561f5ae38d40 2024-09-19T08:16:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.517+0000 7fe06ac39740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561f5ae3ba90 con 0x561f5ae2db60 2024-09-19T08:16:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.517+0000 7fe06ac39740 10 client.0 _ll_register_callbacks cb 0x561f5ad16a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.517+0000 7fe06259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae2e480 0x561f5ae36150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.517+0000 7fe06259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae2e480 0x561f5ae36150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41582/0 (socket says 172.21.15.22:41582) 2024-09-19T08:16:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.517+0000 7fe06259c640 1 -- 172.21.15.22:0/3343420628 learned_addr learned my addr 172.21.15.22:0/3343420628 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.517+0000 7fe061d9b640 1 --2- 172.21.15.22:0/3343420628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 0x561f5ae35c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe061d9b640 1 --2- 172.21.15.22:0/3343420628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae38d40 0x561f5ae36690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe06259c640 1 -- 172.21.15.22:0/3343420628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 msgr2=0x561f5ae35c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe06259c640 1 --2- 172.21.15.22:0/3343420628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 0x561f5ae35c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe06259c640 1 -- 172.21.15.22:0/3343420628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae38d40 msgr2=0x561f5ae36690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe06259c640 1 --2- 172.21.15.22:0/3343420628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae38d40 0x561f5ae36690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe06259c640 1 -- 172.21.15.22:0/3343420628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5ab046a0 con 0x561f5ae2e480 2024-09-19T08:16:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe061d9b640 1 --2- 172.21.15.22:0/3343420628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 0x561f5ae35c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe06259c640 1 --2- 172.21.15.22:0/3343420628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae2e480 0x561f5ae36150 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe054011570 tx=0x7fe054011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ba12c901679425 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe049ffb640 1 -- 172.21.15.22:0/3343420628 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05400f6d0 con 0x561f5ae2e480 2024-09-19T08:16:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe049ffb640 1 -- 172.21.15.22:0/3343420628 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe05400f870 con 0x561f5ae2e480 2024-09-19T08:16:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe06ac39740 1 -- 172.21.15.22:0/3343420628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0580035f0 con 0x561f5ae2e480 2024-09-19T08:16:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe049ffb640 1 -- 172.21.15.22:0/3343420628 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe05400fa70 con 0x561f5ae2e480 2024-09-19T08:16:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe06ac39740 10 client.15861 fetch_fsmap learned FSMap version 26 2024-09-19T08:16:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe06ac39740 1 -- 172.21.15.22:0/3343420628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561f5ae4b980 con 0x561f5ae2e480 2024-09-19T08:16:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.518+0000 7fe049ffb640 1 -- 172.21.15.22:0/3343420628 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0580035f0 con 0x561f5ae2e480 2024-09-19T08:16:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.519+0000 7fe049ffb640 1 -- 172.21.15.22:0/3343420628 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fe054014440 con 0x561f5ae2e480 2024-09-19T08:16:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.519+0000 7fe06ac39740 10 client.15861 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:16:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.519+0000 7fe06ac39740 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:16:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.519+0000 7fe06ac39740 20 client.15861 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.519+0000 7fe06ac39740 1 --2- 172.21.15.22:0/3343420628 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f5ae4ce30 0x561f5ae4f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.519+0000 7fe06ac39740 4 client.15861 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:16:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.519+0000 7fe06ac39740 1 -- 172.21.15.22:0/3343420628 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f5ada07f0 con 0x561f5ae4ce30 2024-09-19T08:16:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.519+0000 7fe061d9b640 1 --2- 172.21.15.22:0/3343420628 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f5ae4ce30 0x561f5ae4f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.519+0000 7fe061d9b640 1 --2- 172.21.15.22:0/3343420628 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f5ae4ce30 0x561f5ae4f220 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ec267c25f4bed85 server_cookie=1885ef37a02128ff in_seq=0 out_seq=0 2024-09-19T08:16:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.519+0000 7fe049ffb640 10 client.15861 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.520+0000 7fe049ffb640 1 -- 172.21.15.22:0/3343420628 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f5ada07f0 con 0x561f5ae4ce30 2024-09-19T08:16:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.520+0000 7fe049ffb640 10 client.15861 handle_command_reply: tid=0 2024-09-19T08:16:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.520+0000 7fe06ac39740 1 client.15861 shutdown 2024-09-19T08:16:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.520+0000 7fe06ac39740 1 -- 172.21.15.22:0/3343420628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae2e480 msgr2=0x561f5ae36150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.520+0000 7fe06ac39740 1 --2- 172.21.15.22:0/3343420628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae2e480 0x561f5ae36150 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe054011570 tx=0x7fe054011a30 comp rx=0 tx=0).stop 2024-09-19T08:16:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.520+0000 7fe06ac39740 1 -- 172.21.15.22:0/3343420628 shutdown_connections 2024-09-19T08:16:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.520+0000 7fe06ac39740 1 --2- 172.21.15.22:0/3343420628 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f5ae4ce30 0x561f5ae4f220 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.520+0000 7fe06ac39740 1 --2- 172.21.15.22:0/3343420628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae38d40 0x561f5ae36690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.520+0000 7fe06ac39740 1 --2- 172.21.15.22:0/3343420628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae2e480 0x561f5ae36150 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.520+0000 7fe049ffb640 0 client.15861 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.520+0000 7fe06ac39740 1 --2- 172.21.15.22:0/3343420628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 0x561f5ae35c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.520+0000 7fe06ac39740 1 -- 172.21.15.22:0/3343420628 >> 172.21.15.22:0/3343420628 conn(0x561f5ae2a0f0 msgr2=0x561f5ae2fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:59.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.520+0000 7fe06ac39740 1 -- 172.21.15.22:0/3343420628 shutdown_connections 2024-09-19T08:16:59.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.521+0000 7fe06ac39740 1 -- 172.21.15.22:0/3343420628 wait complete. 2024-09-19T08:16:59.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.521+0000 7fe06ac39740 20 client.15861 trim_cache size 0 max 16384 2024-09-19T08:16:59.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.538+0000 7fe06ac39740 1 Processor -- start 2024-09-19T08:16:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.538+0000 7fe06ac39740 1 -- start start 2024-09-19T08:16:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.538+0000 7fe06ac39740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 0x561f5ae54af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.538+0000 7fe06ac39740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2e480 0x561f5ae55030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.538+0000 7fe06ac39740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae38d40 0x561f5ae28a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.538+0000 7fe06ac39740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641adf80 con 0x561f5ae38d40 2024-09-19T08:16:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.538+0000 7fe06ac39740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641ab850 con 0x561f5ae2e480 2024-09-19T08:16:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.538+0000 7fe06ac39740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641a96a0 con 0x561f5ae2db60 2024-09-19T08:16:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.538+0000 7fe061d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae38d40 0x561f5ae28a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.538+0000 7fe061d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae38d40 0x561f5ae28a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41590/0 (socket says 172.21.15.22:41590) 2024-09-19T08:16:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.538+0000 7fe061d9b640 1 -- 172.21.15.22:0/4054702957 learned_addr learned my addr 172.21.15.22:0/4054702957 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.538+0000 7fe061d9b640 1 --2- 172.21.15.22:0/4054702957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 0x561f5ae54af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.538+0000 7fe06259c640 1 --2- 172.21.15.22:0/4054702957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2e480 0x561f5ae55030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.539+0000 7fe061d9b640 1 -- 172.21.15.22:0/4054702957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 msgr2=0x561f5ae54af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.539+0000 7fe061d9b640 1 --2- 172.21.15.22:0/4054702957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 0x561f5ae54af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.539+0000 7fe061d9b640 1 -- 172.21.15.22:0/4054702957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2e480 msgr2=0x561f5ae55030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.539+0000 7fe061d9b640 1 --2- 172.21.15.22:0/4054702957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2e480 0x561f5ae55030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.539+0000 7fe061d9b640 1 -- 172.21.15.22:0/4054702957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5ae2e2b0 con 0x561f5ae38d40 2024-09-19T08:16:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.539+0000 7fe061d9b640 1 --2- 172.21.15.22:0/4054702957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae38d40 0x561f5ae28a50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe058009560 tx=0x7fe058000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5599004889608cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.539+0000 7fe04bfff640 1 -- 172.21.15.22:0/4054702957 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05805a070 con 0x561f5ae38d40 2024-09-19T08:16:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.539+0000 7fe04bfff640 1 -- 172.21.15.22:0/4054702957 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0580044d0 con 0x561f5ae38d40 2024-09-19T08:16:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.539+0000 7fe04bfff640 1 -- 172.21.15.22:0/4054702957 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058035040 con 0x561f5ae38d40 2024-09-19T08:16:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.539+0000 7fe06ac39740 1 -- 172.21.15.22:0/4054702957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae38d40 msgr2=0x561f5ae28a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.539+0000 7fe06ac39740 1 --2- 172.21.15.22:0/4054702957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae38d40 0x561f5ae28a50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe058009560 tx=0x7fe058000bc0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 -- 172.21.15.22:0/4054702957 shutdown_connections 2024-09-19T08:16:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 --2- 172.21.15.22:0/4054702957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae38d40 0x561f5ae28a50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 --2- 172.21.15.22:0/4054702957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2e480 0x561f5ae55030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 --2- 172.21.15.22:0/4054702957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae2db60 0x561f5ae54af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 -- 172.21.15.22:0/4054702957 >> 172.21.15.22:0/4054702957 conn(0x561f5ae2a0f0 msgr2=0x561f5ae34d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 -- 172.21.15.22:0/4054702957 shutdown_connections 2024-09-19T08:16:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 -- 172.21.15.22:0/4054702957 wait complete. 2024-09-19T08:16:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 -- start start 2024-09-19T08:16:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 Processor -- start 2024-09-19T08:16:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2db60 0x561f5ae40ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae2e480 0x561f5ae41430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae38d40 0x561f5ae41970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641a9d10 con 0x561f5ae2e480 2024-09-19T08:16:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641ab180 con 0x561f5ae2db60 2024-09-19T08:16:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641aa3e0 con 0x561f5ae38d40 2024-09-19T08:16:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06ac39740 10 client.0 _ll_register_callbacks cb 0x7fe0540095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:16:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae2e480 0x561f5ae41430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae2e480 0x561f5ae41430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41592/0 (socket says 172.21.15.22:41592) 2024-09-19T08:16:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe06259c640 1 -- 172.21.15.22:0/4214056547 learned_addr learned my addr 172.21.15.22:0/4214056547 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:16:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.540+0000 7fe061d9b640 1 --2- 172.21.15.22:0/4214056547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2db60 0x561f5ae40ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe061d9b640 1 --2- 172.21.15.22:0/4214056547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae38d40 0x561f5ae41970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe06259c640 1 -- 172.21.15.22:0/4214056547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae38d40 msgr2=0x561f5ae41970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:59.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe06259c640 1 --2- 172.21.15.22:0/4214056547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae38d40 0x561f5ae41970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe06259c640 1 -- 172.21.15.22:0/4214056547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2db60 msgr2=0x561f5ae40ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:16:59.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe06259c640 1 --2- 172.21.15.22:0/4214056547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2db60 0x561f5ae40ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe06259c640 1 -- 172.21.15.22:0/4214056547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5aeb6af0 con 0x561f5ae2e480 2024-09-19T08:16:59.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe061d9b640 1 --2- 172.21.15.22:0/4214056547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae38d40 0x561f5ae41970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:16:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe06259c640 1 --2- 172.21.15.22:0/4214056547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae2e480 0x561f5ae41430 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe054011540 tx=0x7fe05400f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62afa9de6aeb7d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:16:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:16:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe049ffb640 1 -- 172.21.15.22:0/4214056547 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05400fe10 con 0x561f5ae2e480 2024-09-19T08:16:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe049ffb640 1 -- 172.21.15.22:0/4214056547 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe054026470 con 0x561f5ae2e480 2024-09-19T08:16:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe06ac39740 1 -- 172.21.15.22:0/4214056547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0580044d0 con 0x561f5ae2e480 2024-09-19T08:16:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe049ffb640 1 -- 172.21.15.22:0/4214056547 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe054014c90 con 0x561f5ae2e480 2024-09-19T08:16:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe06ac39740 10 client.15867 fetch_fsmap learned FSMap version 26 2024-09-19T08:16:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.541+0000 7fe06ac39740 1 -- 172.21.15.22:0/4214056547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561f5aeb7df0 con 0x561f5ae2e480 2024-09-19T08:16:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.542+0000 7fe049ffb640 1 -- 172.21.15.22:0/4214056547 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0580044d0 con 0x561f5ae2e480 2024-09-19T08:16:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.542+0000 7fe049ffb640 1 -- 172.21.15.22:0/4214056547 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fe05402f530 con 0x561f5ae2e480 2024-09-19T08:16:59.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.542+0000 7fe06ac39740 10 client.15867 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:16:59.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.542+0000 7fe06ac39740 10 client.15867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:16:59.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.542+0000 7fe06ac39740 20 client.15867 populate_metadata read hostname 'smithi022' 2024-09-19T08:16:59.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.542+0000 7fe06ac39740 1 --2- 172.21.15.22:0/4214056547 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f5ae4ce30 0x561f5aeba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:16:59.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.542+0000 7fe06ac39740 4 client.15867 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:16:59.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.542+0000 7fe06ac39740 1 -- 172.21.15.22:0/4214056547 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561f5ad9ad00 con 0x561f5ae4ce30 2024-09-19T08:16:59.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.542+0000 7fe061d9b640 1 --2- 172.21.15.22:0/4214056547 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f5ae4ce30 0x561f5aeba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:16:59.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.542+0000 7fe061d9b640 1 --2- 172.21.15.22:0/4214056547 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f5ae4ce30 0x561f5aeba580 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a444b8b9a66707cf server_cookie=ca2ce0e1e8100733 in_seq=0 out_seq=0 2024-09-19T08:16:59.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.542+0000 7fe049ffb640 10 client.15867 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:59.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.543+0000 7fe049ffb640 1 -- 172.21.15.22:0/4214056547 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561f5ad9ad00 con 0x561f5ae4ce30 2024-09-19T08:16:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.543+0000 7fe049ffb640 10 client.15867 handle_command_reply: tid=0 2024-09-19T08:16:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.543+0000 7fe06ac39740 1 client.15867 shutdown 2024-09-19T08:16:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe06ac39740 1 -- 172.21.15.22:0/4214056547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae2e480 msgr2=0x561f5ae41430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe06ac39740 1 --2- 172.21.15.22:0/4214056547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae2e480 0x561f5ae41430 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe054011540 tx=0x7fe05400f900 comp rx=0 tx=0).stop 2024-09-19T08:16:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe06ac39740 1 -- 172.21.15.22:0/4214056547 shutdown_connections 2024-09-19T08:16:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe06ac39740 1 --2- 172.21.15.22:0/4214056547 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x561f5ae4ce30 0x561f5aeba580 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe06ac39740 1 --2- 172.21.15.22:0/4214056547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561f5ae38d40 0x561f5ae41970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe06ac39740 1 --2- 172.21.15.22:0/4214056547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f5ae2e480 0x561f5ae41430 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe06ac39740 1 --2- 172.21.15.22:0/4214056547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f5ae2db60 0x561f5ae40ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe049ffb640 0 client.15867 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:16:59.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe06ac39740 1 -- 172.21.15.22:0/4214056547 >> 172.21.15.22:0/4214056547 conn(0x561f5ae2a0f0 msgr2=0x561f5ae47ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:59.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe06ac39740 1 -- 172.21.15.22:0/4214056547 shutdown_connections 2024-09-19T08:16:59.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe06ac39740 1 -- 172.21.15.22:0/4214056547 wait complete. 2024-09-19T08:16:59.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe06ac39740 20 client.15867 trim_cache size 0 max 16384 2024-09-19T08:16:59.565 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:16:59.565 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-19T08:16:59.566 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "bde903ce-e8c6-40f4-a8e8-25b5ef351986", 2024-09-19T08:16:59.566 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-19T08:16:59.566 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:16:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe068860640 1 -- 172.21.15.22:0/3642112481 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe04c0a3430 msgr2=0x7fe04c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe068860640 1 --2- 172.21.15.22:0/3642112481 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe04c0a3430 0x7fe04c0a58f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe05c098b90 tx=0x7fe05c0bc040 comp rx=0 tx=0).stop 2024-09-19T08:16:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe068860640 1 -- 172.21.15.22:0/3642112481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe06410c170 msgr2=0x7fe06419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:16:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe068860640 1 --2- 172.21.15.22:0/3642112481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe06410c170 0x7fe06419ce80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00eaa0 tx=0x7fe05c00eea0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe068860640 1 -- 172.21.15.22:0/3642112481 shutdown_connections 2024-09-19T08:16:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe068860640 1 --2- 172.21.15.22:0/3642112481 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe04c0a3430 0x7fe04c0a58f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe068860640 1 --2- 172.21.15.22:0/3642112481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe06410c170 0x7fe06419ce80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.544+0000 7fe068860640 1 --2- 172.21.15.22:0/3642112481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe064103660 0x7fe064198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.545+0000 7fe068860640 1 --2- 172.21.15.22:0/3642112481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe064102d40 0x7fe0641985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:16:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.545+0000 7fe068860640 1 -- 172.21.15.22:0/3642112481 >> 172.21.15.22:0/3642112481 conn(0x7fe0640fc7a0 msgr2=0x7fe0640fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:16:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.545+0000 7fe068860640 1 -- 172.21.15.22:0/3642112481 shutdown_connections 2024-09-19T08:16:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:16:59.545+0000 7fe068860640 1 -- 172.21.15.22:0/3642112481 wait complete. 2024-09-19T08:17:00.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:59 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:17:00.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:16:59 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:00.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:59 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:17:00.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:16:59 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:00.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:17:00.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:59 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:17:00.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:16:59 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:17:00.924 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:00 smithi022 ceph-mon[30963]: pgmap v62: 97 pgs: 97 active+clean; 4.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 289 KiB/s wr, 40 op/s 2024-09-19T08:17:00.924 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:00 smithi022 ceph-mon[30963]: scrub queued for path: / 2024-09-19T08:17:00.924 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:00 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:00.924 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:00 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:17:00.925 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:00 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.007+0000 7f0760d98640 1 -- 172.21.15.22:0/1213547667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c102d40 msgr2=0x7f075c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.007+0000 7f0760d98640 1 --2- 172.21.15.22:0/1213547667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c102d40 0x7f075c103120 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f074c009900 tx=0x7f074c02a480 comp rx=0 tx=0).stop 2024-09-19T08:17:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.007+0000 7f0760d98640 1 -- 172.21.15.22:0/1213547667 shutdown_connections 2024-09-19T08:17:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.007+0000 7f0760d98640 1 --2- 172.21.15.22:0/1213547667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c10c170 0x7f075c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.007+0000 7f0760d98640 1 --2- 172.21.15.22:0/1213547667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f075c103660 0x7f075c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.007+0000 7f0760d98640 1 --2- 172.21.15.22:0/1213547667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c102d40 0x7f075c103120 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.007+0000 7f0760d98640 1 -- 172.21.15.22:0/1213547667 >> 172.21.15.22:0/1213547667 conn(0x7f075c0fc7a0 msgr2=0x7f075c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.007+0000 7f0760d98640 1 -- 172.21.15.22:0/1213547667 shutdown_connections 2024-09-19T08:17:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.007+0000 7f0760d98640 1 -- 172.21.15.22:0/1213547667 wait complete. 2024-09-19T08:17:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.008+0000 7f0760d98640 1 Processor -- start 2024-09-19T08:17:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.008+0000 7f0760d98640 1 -- start start 2024-09-19T08:17:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.008+0000 7f0760d98640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c102d40 0x7f075c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.008+0000 7f0760d98640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c103660 0x7f075c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.008+0000 7f0760d98640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f075c10c170 0x7f075c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.008+0000 7f0760d98640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f075c111f70 con 0x7f075c102d40 2024-09-19T08:17:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.008+0000 7f0760d98640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f075c111df0 con 0x7f075c10c170 2024-09-19T08:17:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.008+0000 7f0760d98640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1120f0 con 0x7f075c103660 2024-09-19T08:17:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.008+0000 7f075a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c102d40 0x7f075c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.008+0000 7f075a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c102d40 0x7f075c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41610/0 (socket says 172.21.15.22:41610) 2024-09-19T08:17:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.008+0000 7f075a59c640 1 -- 172.21.15.22:0/835229325 learned_addr learned my addr 172.21.15.22:0/835229325 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.008+0000 7f075ad9d640 1 --2- 172.21.15.22:0/835229325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f075c10c170 0x7f075c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.008+0000 7f0751d9b640 1 --2- 172.21.15.22:0/835229325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c103660 0x7f075c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.009+0000 7f075a59c640 1 -- 172.21.15.22:0/835229325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c103660 msgr2=0x7f075c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.009+0000 7f075a59c640 1 --2- 172.21.15.22:0/835229325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c103660 0x7f075c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.009+0000 7f075a59c640 1 -- 172.21.15.22:0/835229325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f075c10c170 msgr2=0x7f075c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.009+0000 7f075a59c640 1 --2- 172.21.15.22:0/835229325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f075c10c170 0x7f075c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.009+0000 7f075a59c640 1 -- 172.21.15.22:0/835229325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c1a68c0 con 0x7f075c102d40 2024-09-19T08:17:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.009+0000 7f075a59c640 1 --2- 172.21.15.22:0/835229325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c102d40 0x7f075c1009a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f074c006010 tx=0x7f074c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.009+0000 7f0753fff640 1 -- 172.21.15.22:0/835229325 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c03f030 con 0x7f075c102d40 2024-09-19T08:17:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.009+0000 7f0753fff640 1 -- 172.21.15.22:0/835229325 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074c003ce0 con 0x7f075c102d40 2024-09-19T08:17:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.009+0000 7f0760d98640 1 -- 172.21.15.22:0/835229325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c1a6b50 con 0x7f075c102d40 2024-09-19T08:17:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.009+0000 7f0753fff640 1 -- 172.21.15.22:0/835229325 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c035070 con 0x7f075c102d40 2024-09-19T08:17:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.009+0000 7f0760d98640 1 -- 172.21.15.22:0/835229325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c1a7030 con 0x7f075c102d40 2024-09-19T08:17:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.011+0000 7f0753fff640 1 -- 172.21.15.22:0/835229325 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f074c042030 con 0x7f075c102d40 2024-09-19T08:17:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.012+0000 7f0763171740 1 Processor -- start 2024-09-19T08:17:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.012+0000 7f0763171740 1 -- start start 2024-09-19T08:17:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.012+0000 7f0763171740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6653880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.012+0000 7f0763171740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a6653dc0 0x5581a665e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.012+0000 7f0763171740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a665e680 0x5581a6660a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.012+0000 7f0763171740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5581a65b92e0 con 0x5581a66534a0 2024-09-19T08:17:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.012+0000 7f0763171740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5581a6560ce0 con 0x5581a665e680 2024-09-19T08:17:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.012+0000 7f0763171740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5581a65b52a0 con 0x5581a6653dc0 2024-09-19T08:17:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.012+0000 7f075ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6653880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.012+0000 7f075ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6653880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41612/0 (socket says 172.21.15.22:41612) 2024-09-19T08:17:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.012+0000 7f075ad9d640 1 -- 172.21.15.22:0/1543609375 learned_addr learned my addr 172.21.15.22:0/1543609375 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.012+0000 7f0753fff640 1 --2- 172.21.15.22:0/835229325 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f07480a33b0 0x7f07480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.012+0000 7f0751d9b640 1 --2- 172.21.15.22:0/1543609375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a665e680 0x5581a6660a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.012+0000 7f075ad9d640 1 --2- 172.21.15.22:0/835229325 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f07480a33b0 0x7f07480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.013+0000 7f075ad9d640 1 -- 172.21.15.22:0/1543609375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a6653dc0 msgr2=0x5581a665e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.013+0000 7f075a59c640 1 --2- 172.21.15.22:0/1543609375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a6653dc0 0x5581a665e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.013+0000 7f0753fff640 1 -- 172.21.15.22:0/835229325 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f074c0f0170 con 0x7f075c102d40 2024-09-19T08:17:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.013+0000 7f075ad9d640 1 --2- 172.21.15.22:0/1543609375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a6653dc0 0x5581a665e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.013+0000 7f075ad9d640 1 -- 172.21.15.22:0/1543609375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a665e680 msgr2=0x5581a6660a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.013+0000 7f075ad9d640 1 --2- 172.21.15.22:0/1543609375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a665e680 0x5581a6660a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.013+0000 7f075ad9d640 1 -- 172.21.15.22:0/1543609375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581a6545440 con 0x5581a66534a0 2024-09-19T08:17:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.013+0000 7f0751d9b640 1 --2- 172.21.15.22:0/1543609375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a665e680 0x5581a6660a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:17:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.013+0000 7f075a59c640 1 --2- 172.21.15.22:0/1543609375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a6653dc0 0x5581a665e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:17:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.013+0000 7f075ad9d640 1 --2- 172.21.15.22:0/1543609375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6653880 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0754005f40 tx=0x7f0754012f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b0294ef61212ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.013+0000 7f0750d99640 1 -- 172.21.15.22:0/1543609375 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07540124b0 con 0x5581a66534a0 2024-09-19T08:17:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.013+0000 7f0750d99640 1 -- 172.21.15.22:0/1543609375 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0754022070 con 0x5581a66534a0 2024-09-19T08:17:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.013+0000 7f0750d99640 1 -- 172.21.15.22:0/1543609375 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075400fdc0 con 0x5581a66534a0 2024-09-19T08:17:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.013+0000 7f075ad9d640 1 --2- 172.21.15.22:0/835229325 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f07480a33b0 0x7f07480a5870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0754021000 tx=0x7f075401f000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.014+0000 7f0763171740 1 -- 172.21.15.22:0/1543609375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 msgr2=0x5581a6653880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.014+0000 7f0763171740 1 --2- 172.21.15.22:0/1543609375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6653880 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0754005f40 tx=0x7f0754012f00 comp rx=0 tx=0).stop 2024-09-19T08:17:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.014+0000 7f0763171740 1 -- 172.21.15.22:0/1543609375 shutdown_connections 2024-09-19T08:17:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.014+0000 7f0763171740 1 --2- 172.21.15.22:0/1543609375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a665e680 0x5581a6660a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.014+0000 7f0763171740 1 --2- 172.21.15.22:0/1543609375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a6653dc0 0x5581a665e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.014+0000 7f0763171740 1 --2- 172.21.15.22:0/1543609375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6653880 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.014+0000 7f0763171740 1 -- 172.21.15.22:0/1543609375 >> 172.21.15.22:0/1543609375 conn(0x5581a664fa30 msgr2=0x5581a664fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.014+0000 7f0763171740 1 -- 172.21.15.22:0/1543609375 shutdown_connections 2024-09-19T08:17:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.014+0000 7f0763171740 1 -- 172.21.15.22:0/1543609375 wait complete. 2024-09-19T08:17:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.014+0000 7f0763171740 1 -- start start 2024-09-19T08:17:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.014+0000 7f0763171740 1 Processor -- start 2024-09-19T08:17:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0763171740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a66534a0 0x5581a665df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0763171740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a6653dc0 0x5581a665b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0763171740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a665e680 0x5581a665b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0763171740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5581a6661250 con 0x5581a6653dc0 2024-09-19T08:17:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0763171740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5581a6574050 con 0x5581a665e680 2024-09-19T08:17:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0763171740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5581a66613d0 con 0x5581a66534a0 2024-09-19T08:17:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0763171740 10 client.0 _ll_register_callbacks cb 0x5581a65b4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0751d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a6653dc0 0x5581a665b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0751d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a6653dc0 0x5581a665b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41628/0 (socket says 172.21.15.22:41628) 2024-09-19T08:17:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0751d9b640 1 -- 172.21.15.22:0/2973284634 learned_addr learned my addr 172.21.15.22:0/2973284634 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f075ad9d640 1 --2- 172.21.15.22:0/2973284634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a665e680 0x5581a665b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f075a59c640 1 --2- 172.21.15.22:0/2973284634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a66534a0 0x5581a665df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0751d9b640 1 -- 172.21.15.22:0/2973284634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a66534a0 msgr2=0x5581a665df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0751d9b640 1 --2- 172.21.15.22:0/2973284634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a66534a0 0x5581a665df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0751d9b640 1 -- 172.21.15.22:0/2973284634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a665e680 msgr2=0x5581a665b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0751d9b640 1 --2- 172.21.15.22:0/2973284634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a665e680 0x5581a665b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0751d9b640 1 -- 172.21.15.22:0/2973284634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581a632a3c0 con 0x5581a6653dc0 2024-09-19T08:17:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f075a59c640 1 --2- 172.21.15.22:0/2973284634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a66534a0 0x5581a665df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:17:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0751d9b640 1 --2- 172.21.15.22:0/2973284634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a6653dc0 0x5581a665b230 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0744012700 tx=0x7f0744012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac7d5121fce8992 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:17:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0741ffb640 1 -- 172.21.15.22:0/2973284634 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074401f060 con 0x5581a6653dc0 2024-09-19T08:17:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0741ffb640 1 -- 172.21.15.22:0/2973284634 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0744010dc0 con 0x5581a6653dc0 2024-09-19T08:17:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.015+0000 7f0763171740 1 -- 172.21.15.22:0/2973284634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0754022070 con 0x5581a6653dc0 2024-09-19T08:17:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.016+0000 7f0741ffb640 1 -- 172.21.15.22:0/2973284634 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f074401a430 con 0x5581a6653dc0 2024-09-19T08:17:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.016+0000 7f0763171740 10 client.15879 fetch_fsmap learned FSMap version 26 2024-09-19T08:17:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.016+0000 7f0763171740 1 -- 172.21.15.22:0/2973284634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5581a666cbc0 con 0x5581a6653dc0 2024-09-19T08:17:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.016+0000 7f0741ffb640 1 -- 172.21.15.22:0/2973284634 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0754022070 con 0x5581a6653dc0 2024-09-19T08:17:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.016+0000 7f0741ffb640 1 -- 172.21.15.22:0/2973284634 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f0744022600 con 0x5581a6653dc0 2024-09-19T08:17:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.016+0000 7f0763171740 10 client.15879 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:17:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.016+0000 7f0763171740 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:17:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.016+0000 7f0763171740 20 client.15879 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.016+0000 7f0763171740 1 --2- 172.21.15.22:0/2973284634 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5581a666e0d0 0x5581a66704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.016+0000 7f0763171740 4 client.15879 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:17:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.016+0000 7f0763171740 1 -- 172.21.15.22:0/2973284634 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581a65c6850 con 0x5581a666e0d0 2024-09-19T08:17:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.016+0000 7f075a59c640 1 --2- 172.21.15.22:0/2973284634 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5581a666e0d0 0x5581a66704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.017+0000 7f0741ffb640 10 client.15879 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.017+0000 7f075a59c640 1 --2- 172.21.15.22:0/2973284634 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5581a666e0d0 0x5581a66704e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47762b7ea06b9df7 server_cookie=50a9da9e3bf9651 in_seq=0 out_seq=0 2024-09-19T08:17:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.017+0000 7f0741ffb640 1 -- 172.21.15.22:0/2973284634 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581a65c6850 con 0x5581a666e0d0 2024-09-19T08:17:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.017+0000 7f0741ffb640 10 client.15879 handle_command_reply: tid=0 2024-09-19T08:17:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.017+0000 7f0763171740 1 client.15879 shutdown 2024-09-19T08:17:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.018+0000 7f0763171740 1 -- 172.21.15.22:0/2973284634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a6653dc0 msgr2=0x5581a665b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.018+0000 7f0763171740 1 --2- 172.21.15.22:0/2973284634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a6653dc0 0x5581a665b230 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0744012700 tx=0x7f0744012bc0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.018+0000 7f0763171740 1 -- 172.21.15.22:0/2973284634 shutdown_connections 2024-09-19T08:17:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.018+0000 7f0763171740 1 --2- 172.21.15.22:0/2973284634 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5581a666e0d0 0x5581a66704e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.018+0000 7f0763171740 1 --2- 172.21.15.22:0/2973284634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a665e680 0x5581a665b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.018+0000 7f0763171740 1 --2- 172.21.15.22:0/2973284634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a6653dc0 0x5581a665b230 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.018+0000 7f0763171740 1 --2- 172.21.15.22:0/2973284634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a66534a0 0x5581a665df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.018+0000 7f0763171740 1 -- 172.21.15.22:0/2973284634 >> 172.21.15.22:0/2973284634 conn(0x5581a664fa30 msgr2=0x5581a6655710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:01.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.018+0000 7f0741ffb640 0 client.15879 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:01.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.018+0000 7f0763171740 1 -- 172.21.15.22:0/2973284634 shutdown_connections 2024-09-19T08:17:01.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.018+0000 7f0763171740 1 -- 172.21.15.22:0/2973284634 wait complete. 2024-09-19T08:17:01.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.018+0000 7f0763171740 20 client.15879 trim_cache size 0 max 16384 2024-09-19T08:17:01.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.035+0000 7f0763171740 1 Processor -- start 2024-09-19T08:17:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.035+0000 7f0763171740 1 -- start start 2024-09-19T08:17:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.035+0000 7f0763171740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a664fa30 0x5581a665b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.035+0000 7f0763171740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6675df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.035+0000 7f0763171740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a6653dc0 0x5581a6676330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.035+0000 7f0763171740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1b22a0 con 0x5581a66534a0 2024-09-19T08:17:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.035+0000 7f0763171740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1afb70 con 0x5581a6653dc0 2024-09-19T08:17:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.035+0000 7f0763171740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1ad9c0 con 0x5581a664fa30 2024-09-19T08:17:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.035+0000 7f0751d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6675df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.035+0000 7f0751d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6675df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41630/0 (socket says 172.21.15.22:41630) 2024-09-19T08:17:01.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.035+0000 7f0751d9b640 1 -- 172.21.15.22:0/1158466297 learned_addr learned my addr 172.21.15.22:0/1158466297 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:01.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.036+0000 7f075ad9d640 1 --2- 172.21.15.22:0/1158466297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a6653dc0 0x5581a6676330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.036+0000 7f0751d9b640 1 -- 172.21.15.22:0/1158466297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a664fa30 msgr2=0x5581a665b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:01.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.036+0000 7f0751d9b640 1 --2- 172.21.15.22:0/1158466297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a664fa30 0x5581a665b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.036+0000 7f0751d9b640 1 -- 172.21.15.22:0/1158466297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a6653dc0 msgr2=0x5581a6676330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:01.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.036+0000 7f0751d9b640 1 --2- 172.21.15.22:0/1158466297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a6653dc0 0x5581a6676330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.036+0000 7f0751d9b640 1 -- 172.21.15.22:0/1158466297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581a666af10 con 0x5581a66534a0 2024-09-19T08:17:01.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.036+0000 7f0751d9b640 1 --2- 172.21.15.22:0/1158466297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6675df0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f07440126d0 tx=0x7f0744005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637c110f3e9949a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:01.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.036+0000 7f0743fff640 1 -- 172.21.15.22:0/1158466297 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744010bc0 con 0x5581a66534a0 2024-09-19T08:17:01.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.036+0000 7f0743fff640 1 -- 172.21.15.22:0/1158466297 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0744030070 con 0x5581a66534a0 2024-09-19T08:17:01.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.036+0000 7f0743fff640 1 -- 172.21.15.22:0/1158466297 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074401ac40 con 0x5581a66534a0 2024-09-19T08:17:01.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 -- 172.21.15.22:0/1158466297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 msgr2=0x5581a6675df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:01.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 --2- 172.21.15.22:0/1158466297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6675df0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f07440126d0 tx=0x7f0744005670 comp rx=0 tx=0).stop 2024-09-19T08:17:01.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 -- 172.21.15.22:0/1158466297 shutdown_connections 2024-09-19T08:17:01.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 --2- 172.21.15.22:0/1158466297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a6653dc0 0x5581a6676330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 --2- 172.21.15.22:0/1158466297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6675df0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 --2- 172.21.15.22:0/1158466297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a664fa30 0x5581a665b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 -- 172.21.15.22:0/1158466297 >> 172.21.15.22:0/1158466297 conn(0x5581a665b8d0 msgr2=0x5581a665d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:01.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 -- 172.21.15.22:0/1158466297 shutdown_connections 2024-09-19T08:17:01.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 -- 172.21.15.22:0/1158466297 wait complete. 2024-09-19T08:17:01.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 -- start start 2024-09-19T08:17:01.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 Processor -- start 2024-09-19T08:17:01.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a664fa30 0x5581a6664790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6664cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a6653dc0 0x5581a66d7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1ae030 con 0x5581a66534a0 2024-09-19T08:17:01.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1af4a0 con 0x5581a664fa30 2024-09-19T08:17:01.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.037+0000 7f0763171740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1ae700 con 0x5581a6653dc0 2024-09-19T08:17:01.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f0763171740 10 client.0 _ll_register_callbacks cb 0x7f074400ae40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:01.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f0751d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6664cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f0751d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6664cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41632/0 (socket says 172.21.15.22:41632) 2024-09-19T08:17:01.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f0751d9b640 1 -- 172.21.15.22:0/3809427403 learned_addr learned my addr 172.21.15.22:0/3809427403 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:01.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f075ad9d640 1 --2- 172.21.15.22:0/3809427403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a6653dc0 0x5581a66d7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f075a59c640 1 --2- 172.21.15.22:0/3809427403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a664fa30 0x5581a6664790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f0751d9b640 1 -- 172.21.15.22:0/3809427403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a6653dc0 msgr2=0x5581a66d7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:01.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f0751d9b640 1 --2- 172.21.15.22:0/3809427403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a6653dc0 0x5581a66d7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f0751d9b640 1 -- 172.21.15.22:0/3809427403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a664fa30 msgr2=0x5581a6664790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:01.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f0751d9b640 1 --2- 172.21.15.22:0/3809427403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a664fa30 0x5581a6664790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f0751d9b640 1 -- 172.21.15.22:0/3809427403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581a66d8310 con 0x5581a66534a0 2024-09-19T08:17:01.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f0751d9b640 1 --2- 172.21.15.22:0/3809427403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6664cd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f074402d040 tx=0x7f074401a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a7b440f3b8abd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:01.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f0741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:17:01.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f0741ffb640 1 -- 172.21.15.22:0/3809427403 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074403a020 con 0x5581a66534a0 2024-09-19T08:17:01.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f0741ffb640 1 -- 172.21.15.22:0/3809427403 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f074401a980 con 0x5581a66534a0 2024-09-19T08:17:01.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.038+0000 7f0763171740 1 -- 172.21.15.22:0/3809427403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0744030070 con 0x5581a66534a0 2024-09-19T08:17:01.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.039+0000 7f0741ffb640 1 -- 172.21.15.22:0/3809427403 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0744045050 con 0x5581a66534a0 2024-09-19T08:17:01.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.039+0000 7f0763171740 10 client.15885 fetch_fsmap learned FSMap version 26 2024-09-19T08:17:01.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.039+0000 7f0763171740 1 -- 172.21.15.22:0/3809427403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5581a66d9480 con 0x5581a66534a0 2024-09-19T08:17:01.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.039+0000 7f0741ffb640 1 -- 172.21.15.22:0/3809427403 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0744030070 con 0x5581a66534a0 2024-09-19T08:17:01.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.039+0000 7f0741ffb640 1 -- 172.21.15.22:0/3809427403 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f0744034930 con 0x5581a66534a0 2024-09-19T08:17:01.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.039+0000 7f0763171740 10 client.15885 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:17:01.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.039+0000 7f0763171740 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:17:01.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.039+0000 7f0763171740 20 client.15885 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:01.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.039+0000 7f0763171740 1 --2- 172.21.15.22:0/3809427403 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5581a665e680 0x5581a66dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:01.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.039+0000 7f0763171740 4 client.15885 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:17:01.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.039+0000 7f0763171740 1 -- 172.21.15.22:0/3809427403 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5581a65c0d60 con 0x5581a665e680 2024-09-19T08:17:01.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.039+0000 7f075a59c640 1 --2- 172.21.15.22:0/3809427403 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5581a665e680 0x5581a66dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:01.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f075a59c640 1 --2- 172.21.15.22:0/3809427403 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5581a665e680 0x5581a66dbb80 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33d685e54287dd50 server_cookie=60430f52a43b5055 in_seq=0 out_seq=0 2024-09-19T08:17:01.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0741ffb640 10 client.15885 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:01.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0741ffb640 1 -- 172.21.15.22:0/3809427403 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5581a65c0d60 con 0x5581a665e680 2024-09-19T08:17:01.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0741ffb640 10 client.15885 handle_command_reply: tid=0 2024-09-19T08:17:01.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0763171740 1 client.15885 shutdown 2024-09-19T08:17:01.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0763171740 1 -- 172.21.15.22:0/3809427403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 msgr2=0x5581a6664cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:01.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0763171740 1 --2- 172.21.15.22:0/3809427403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6664cd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f074402d040 tx=0x7f074401a8a0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0763171740 1 -- 172.21.15.22:0/3809427403 shutdown_connections 2024-09-19T08:17:01.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0763171740 1 --2- 172.21.15.22:0/3809427403 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5581a665e680 0x5581a66dbb80 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0763171740 1 --2- 172.21.15.22:0/3809427403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581a6653dc0 0x5581a66d7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0763171740 1 --2- 172.21.15.22:0/3809427403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581a66534a0 0x5581a6664cd0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0763171740 1 --2- 172.21.15.22:0/3809427403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581a664fa30 0x5581a6664790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0741ffb640 0 client.15885 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:01.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0763171740 1 -- 172.21.15.22:0/3809427403 >> 172.21.15.22:0/3809427403 conn(0x5581a665b8d0 msgr2=0x5581a665d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:01.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0763171740 1 -- 172.21.15.22:0/3809427403 shutdown_connections 2024-09-19T08:17:01.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0763171740 1 -- 172.21.15.22:0/3809427403 wait complete. 2024-09-19T08:17:01.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.040+0000 7f0763171740 20 client.15885 trim_cache size 0 max 16384 2024-09-19T08:17:01.049 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:17:01.049 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:17:01.049 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-19T08:17:01.049 INFO:teuthology.orchestra.run.smithi022.stdout: "bde903ce-e8c6-40f4-a8e8-25b5ef351986": { 2024-09-19T08:17:01.049 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-19T08:17:01.049 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "bde903ce-e8c6-40f4-a8e8-25b5ef351986", 2024-09-19T08:17:01.049 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-09-19T08:17:01.050 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:17:01.050 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:17:01.050 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:17:01.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.041+0000 7f0760d98640 1 -- 172.21.15.22:0/835229325 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f07480a33b0 msgr2=0x7f07480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:01.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.041+0000 7f0760d98640 1 --2- 172.21.15.22:0/835229325 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f07480a33b0 0x7f07480a5870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0754021000 tx=0x7f075401f000 comp rx=0 tx=0).stop 2024-09-19T08:17:01.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.041+0000 7f0760d98640 1 -- 172.21.15.22:0/835229325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c102d40 msgr2=0x7f075c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:01.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.041+0000 7f0760d98640 1 --2- 172.21.15.22:0/835229325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c102d40 0x7f075c1009a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f074c006010 tx=0x7f074c0042f0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.041+0000 7f0760d98640 1 -- 172.21.15.22:0/835229325 shutdown_connections 2024-09-19T08:17:01.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.041+0000 7f0760d98640 1 --2- 172.21.15.22:0/835229325 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f07480a33b0 0x7f07480a5870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.041+0000 7f0760d98640 1 --2- 172.21.15.22:0/835229325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f075c10c170 0x7f075c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.041+0000 7f0760d98640 1 --2- 172.21.15.22:0/835229325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f075c103660 0x7f075c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.041+0000 7f0760d98640 1 --2- 172.21.15.22:0/835229325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c102d40 0x7f075c1009a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:01.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.041+0000 7f0760d98640 1 -- 172.21.15.22:0/835229325 >> 172.21.15.22:0/835229325 conn(0x7f075c0fc7a0 msgr2=0x7f075c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:01.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.041+0000 7f0760d98640 1 -- 172.21.15.22:0/835229325 shutdown_connections 2024-09-19T08:17:01.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:01.041+0000 7f0760d98640 1 -- 172.21.15.22:0/835229325 wait complete. 2024-09-19T08:17:01.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:00 smithi157 ceph-mon[35143]: pgmap v62: 97 pgs: 97 active+clean; 4.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 289 KiB/s wr, 40 op/s 2024-09-19T08:17:01.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:00 smithi157 ceph-mon[35143]: scrub queued for path: / 2024-09-19T08:17:01.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:00 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:01.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:00 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:17:01.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:00 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:01.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:00 smithi064 ceph-mon[34900]: pgmap v62: 97 pgs: 97 active+clean; 4.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 289 KiB/s wr, 40 op/s 2024-09-19T08:17:01.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:00 smithi064 ceph-mon[34900]: scrub queued for path: / 2024-09-19T08:17:01.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:00 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:01.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:00 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:17:01.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:00 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:01.798 INFO:tasks.cephfs.filesystem:scrub status for tag:bde903ce-e8c6-40f4-a8e8-25b5ef351986 - {'path': '/', 'tag': 'bde903ce-e8c6-40f4-a8e8-25b5ef351986', 'options': 'recursive,force'} 2024-09-19T08:17:03.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:02 smithi157 ceph-mon[35143]: pgmap v63: 97 pgs: 97 active+clean; 4.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 289 KiB/s wr, 40 op/s 2024-09-19T08:17:03.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:02 smithi022 ceph-mon[30963]: pgmap v63: 97 pgs: 97 active+clean; 4.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 289 KiB/s wr, 40 op/s 2024-09-19T08:17:03.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:02 smithi064 ceph-mon[34900]: pgmap v63: 97 pgs: 97 active+clean; 4.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 289 KiB/s wr, 40 op/s 2024-09-19T08:17:05.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:04 smithi157 ceph-mon[35143]: pgmap v64: 97 pgs: 97 active+clean; 4.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 358 KiB/s wr, 63 op/s 2024-09-19T08:17:05.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:04 smithi022 ceph-mon[30963]: pgmap v64: 97 pgs: 97 active+clean; 4.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 358 KiB/s wr, 63 op/s 2024-09-19T08:17:05.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:04 smithi064 ceph-mon[34900]: pgmap v64: 97 pgs: 97 active+clean; 4.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 358 KiB/s wr, 63 op/s 2024-09-19T08:17:06.846 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:17:07.116 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:17:07.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:06 smithi022 ceph-mon[30963]: pgmap v65: 97 pgs: 97 active+clean; 4.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 81 KiB/s wr, 36 op/s 2024-09-19T08:17:07.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:06 smithi157 ceph-mon[35143]: pgmap v65: 97 pgs: 97 active+clean; 4.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 81 KiB/s wr, 36 op/s 2024-09-19T08:17:07.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:06 smithi064 ceph-mon[34900]: pgmap v65: 97 pgs: 97 active+clean; 4.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 81 KiB/s wr, 36 op/s 2024-09-19T08:17:07.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.593+0000 7f7e7141a640 1 -- 172.21.15.22:0/3597049279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e6c106a80 msgr2=0x7f7e6c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:07.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.593+0000 7f7e7141a640 1 --2- 172.21.15.22:0/3597049279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e6c106a80 0x7f7e6c106e60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009930 tx=0x7f7e5c02a420 comp rx=0 tx=0).stop 2024-09-19T08:17:07.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.593+0000 7f7e7141a640 1 -- 172.21.15.22:0/3597049279 shutdown_connections 2024-09-19T08:17:07.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.593+0000 7f7e7141a640 1 --2- 172.21.15.22:0/3597049279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e6c101420 0x7f7e6c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:07.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.593+0000 7f7e7141a640 1 --2- 172.21.15.22:0/3597049279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c100a80 0x7f7e6c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:07.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.593+0000 7f7e7141a640 1 --2- 172.21.15.22:0/3597049279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e6c106a80 0x7f7e6c106e60 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:07.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.593+0000 7f7e7141a640 1 -- 172.21.15.22:0/3597049279 >> 172.21.15.22:0/3597049279 conn(0x7f7e6c0fc7a0 msgr2=0x7f7e6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:07.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.593+0000 7f7e7141a640 1 -- 172.21.15.22:0/3597049279 shutdown_connections 2024-09-19T08:17:07.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.593+0000 7f7e7141a640 1 -- 172.21.15.22:0/3597049279 wait complete. 2024-09-19T08:17:07.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e7141a640 1 Processor -- start 2024-09-19T08:17:07.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e7141a640 1 -- start start 2024-09-19T08:17:07.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e7141a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c100a80 0x7f7e6c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:07.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e7141a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e6c101420 0x7f7e6c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:07.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e7141a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e6c106a80 0x7f7e6c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:07.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e7141a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c110e80 con 0x7f7e6c106a80 2024-09-19T08:17:07.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e7141a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c110d00 con 0x7f7e6c100a80 2024-09-19T08:17:07.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e7141a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c111000 con 0x7f7e6c101420 2024-09-19T08:17:07.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e6b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e6c106a80 0x7f7e6c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:07.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e6b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e6c106a80 0x7f7e6c19cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38090/0 (socket says 172.21.15.22:38090) 2024-09-19T08:17:07.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e6b7fe640 1 -- 172.21.15.22:0/2883948351 learned_addr learned my addr 172.21.15.22:0/2883948351 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:07.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e627fc640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e6c101420 0x7f7e6c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:07.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e6affd640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c100a80 0x7f7e6c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:07.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e6b7fe640 1 -- 172.21.15.22:0/2883948351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e6c101420 msgr2=0x7f7e6c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:07.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e6b7fe640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e6c101420 0x7f7e6c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:07.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e6b7fe640 1 -- 172.21.15.22:0/2883948351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c100a80 msgr2=0x7f7e6c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:07.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e6b7fe640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c100a80 0x7f7e6c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:07.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e6b7fe640 1 -- 172.21.15.22:0/2883948351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6c19d3f0 con 0x7f7e6c106a80 2024-09-19T08:17:07.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.594+0000 7f7e6affd640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c100a80 0x7f7e6c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:17:07.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.595+0000 7f7e6b7fe640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e6c106a80 0x7f7e6c19cd10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400eaa0 tx=0x7f7e6400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:07.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.595+0000 7f7e68ff9640 1 -- 172.21.15.22:0/2883948351 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400cdb0 con 0x7f7e6c106a80 2024-09-19T08:17:07.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.595+0000 7f7e7141a640 1 -- 172.21.15.22:0/2883948351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6c19d6e0 con 0x7f7e6c106a80 2024-09-19T08:17:07.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.595+0000 7f7e68ff9640 1 -- 172.21.15.22:0/2883948351 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e64005020 con 0x7f7e6c106a80 2024-09-19T08:17:07.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.595+0000 7f7e68ff9640 1 -- 172.21.15.22:0/2883948351 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e640116d0 con 0x7f7e6c106a80 2024-09-19T08:17:07.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.595+0000 7f7e7141a640 1 -- 172.21.15.22:0/2883948351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e6c1a4fc0 con 0x7f7e6c106a80 2024-09-19T08:17:07.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.596+0000 7f7e7141a640 1 -- 172.21.15.22:0/2883948351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e6c100ee0 con 0x7f7e6c106a80 2024-09-19T08:17:07.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.597+0000 7f7e68ff9640 1 -- 172.21.15.22:0/2883948351 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7e640054d0 con 0x7f7e6c106a80 2024-09-19T08:17:07.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.598+0000 7f7e68ff9640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7e580a33b0 0x7f7e580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:07.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.598+0000 7f7e6affd640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7e580a33b0 0x7f7e580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:07.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.598+0000 7f7e68ff9640 1 -- 172.21.15.22:0/2883948351 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7e640d0330 con 0x7f7e6c106a80 2024-09-19T08:17:07.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.598+0000 7f7e6affd640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7e580a33b0 0x7f7e580a5870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c000c00 tx=0x7f7e5c032c70 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:07.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.601+0000 7f7e68ff9640 1 -- 172.21.15.22:0/2883948351 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7e64016030 con 0x7f7e6c106a80 2024-09-19T08:17:07.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.812+0000 7f7e7141a640 1 -- 172.21.15.22:0/2883948351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e6c1a52c0 con 0x7f7e6c106a80 2024-09-19T08:17:07.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.813+0000 7f7e68ff9640 1 -- 172.21.15.22:0/2883948351 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11907 (secure 0 0 0) 0x7f7e6409ccc0 con 0x7f7e6c106a80 2024-09-19T08:17:07.813 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:17:07.813 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"btime":"2024-09-19T08:16:47:822061+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:16:46.867495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:17:07.813 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-09-19T08:17:07.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.815+0000 7f7e7141a640 1 -- 172.21.15.22:0/2883948351 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7e580a33b0 msgr2=0x7f7e580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:07.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.815+0000 7f7e7141a640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7e580a33b0 0x7f7e580a5870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c000c00 tx=0x7f7e5c032c70 comp rx=0 tx=0).stop 2024-09-19T08:17:07.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.815+0000 7f7e7141a640 1 -- 172.21.15.22:0/2883948351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e6c106a80 msgr2=0x7f7e6c19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:07.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.815+0000 7f7e7141a640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e6c106a80 0x7f7e6c19cd10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400eaa0 tx=0x7f7e6400ef60 comp rx=0 tx=0).stop 2024-09-19T08:17:07.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.815+0000 7f7e7141a640 1 -- 172.21.15.22:0/2883948351 shutdown_connections 2024-09-19T08:17:07.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.815+0000 7f7e7141a640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7e580a33b0 0x7f7e580a5870 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:07.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.815+0000 7f7e7141a640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e6c106a80 0x7f7e6c19cd10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:07.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.815+0000 7f7e7141a640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e6c101420 0x7f7e6c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:07.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.815+0000 7f7e7141a640 1 --2- 172.21.15.22:0/2883948351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e6c100a80 0x7f7e6c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:07.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.815+0000 7f7e7141a640 1 -- 172.21.15.22:0/2883948351 >> 172.21.15.22:0/2883948351 conn(0x7f7e6c0fc7a0 msgr2=0x7f7e6c104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:07.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.815+0000 7f7e7141a640 1 -- 172.21.15.22:0/2883948351 shutdown_connections 2024-09-19T08:17:07.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:07.815+0000 7f7e7141a640 1 -- 172.21.15.22:0/2883948351 wait complete. 2024-09-19T08:17:08.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:07 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:17:08.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:07 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:17:08.191 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:07 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2883948351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:17:08.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:07 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:17:08.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:07 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:17:08.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:07 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2883948351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:17:08.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:07 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:17:08.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:07 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:17:08.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:07 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2883948351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:17:08.644 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id f364c789 --release --await' 2024-09-19T08:17:08.644 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id f364c789 --release --await 2024-09-19T08:17:08.909 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:17:09.175 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:08 smithi022 ceph-mon[30963]: pgmap v66: 97 pgs: 97 active+clean; 4.7 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 43 op/s 2024-09-19T08:17:09.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:08 smithi157 ceph-mon[35143]: pgmap v66: 97 pgs: 97 active+clean; 4.7 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 43 op/s 2024-09-19T08:17:09.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:08 smithi064 ceph-mon[34900]: pgmap v66: 97 pgs: 97 active+clean; 4.7 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 43 op/s 2024-09-19T08:17:09.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.418+0000 7fc8e7915640 1 -- 172.21.15.22:0/3398662017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8e0100a80 msgr2=0x7fc8e0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:09.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.418+0000 7fc8e7915640 1 --2- 172.21.15.22:0/3398662017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8e0100a80 0x7fc8e0100ee0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009930 tx=0x7fc8d002a420 comp rx=0 tx=0).stop 2024-09-19T08:17:09.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.419+0000 7fc8e7915640 1 -- 172.21.15.22:0/3398662017 shutdown_connections 2024-09-19T08:17:09.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.419+0000 7fc8e7915640 1 --2- 172.21.15.22:0/3398662017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e0101420 0x7fc8e010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.419+0000 7fc8e7915640 1 --2- 172.21.15.22:0/3398662017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8e0100a80 0x7fc8e0100ee0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.419+0000 7fc8e7915640 1 --2- 172.21.15.22:0/3398662017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e0106a80 0x7fc8e0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.419+0000 7fc8e7915640 1 -- 172.21.15.22:0/3398662017 >> 172.21.15.22:0/3398662017 conn(0x7fc8e00fc7a0 msgr2=0x7fc8e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:09.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.419+0000 7fc8e7915640 1 -- 172.21.15.22:0/3398662017 shutdown_connections 2024-09-19T08:17:09.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.419+0000 7fc8e7915640 1 -- 172.21.15.22:0/3398662017 wait complete. 2024-09-19T08:17:09.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.419+0000 7fc8e7915640 1 Processor -- start 2024-09-19T08:17:09.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.419+0000 7fc8e7915640 1 -- start start 2024-09-19T08:17:09.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.419+0000 7fc8e7915640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e0100a80 0x7fc8e00fce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e7915640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8e0101420 0x7fc8e00fd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e7915640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e0106a80 0x7fc8e00fd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e7915640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e0110e40 con 0x7fc8e0101420 2024-09-19T08:17:09.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e7915640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e0110cc0 con 0x7fc8e0100a80 2024-09-19T08:17:09.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e7915640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e0110fc0 con 0x7fc8e0106a80 2024-09-19T08:17:09.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e4eb0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8e0101420 0x7fc8e00fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e4eb0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8e0101420 0x7fc8e00fd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38118/0 (socket says 172.21.15.22:38118) 2024-09-19T08:17:09.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e4eb0640 1 -- 172.21.15.22:0/114303392 learned_addr learned my addr 172.21.15.22:0/114303392 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:09.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e56b1640 1 --2- 172.21.15.22:0/114303392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e0100a80 0x7fc8e00fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e5eb2640 1 --2- 172.21.15.22:0/114303392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e0106a80 0x7fc8e00fd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e4eb0640 1 -- 172.21.15.22:0/114303392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e0106a80 msgr2=0x7fc8e00fd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:09.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e4eb0640 1 --2- 172.21.15.22:0/114303392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e0106a80 0x7fc8e00fd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e4eb0640 1 -- 172.21.15.22:0/114303392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e0100a80 msgr2=0x7fc8e00fce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:09.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e4eb0640 1 --2- 172.21.15.22:0/114303392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e0100a80 0x7fc8e00fce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e4eb0640 1 -- 172.21.15.22:0/114303392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e01a68f0 con 0x7fc8e0101420 2024-09-19T08:17:09.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e4eb0640 1 --2- 172.21.15.22:0/114303392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8e0101420 0x7fc8e00fd340 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009230 tx=0x7fc8d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:09.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8d67fc640 1 -- 172.21.15.22:0/114303392 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d003f030 con 0x7fc8e0101420 2024-09-19T08:17:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8d67fc640 1 -- 172.21.15.22:0/114303392 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d0003ad0 con 0x7fc8e0101420 2024-09-19T08:17:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e7915640 1 -- 172.21.15.22:0/114303392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e01a6b80 con 0x7fc8e0101420 2024-09-19T08:17:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8d67fc640 1 -- 172.21.15.22:0/114303392 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0035070 con 0x7fc8e0101420 2024-09-19T08:17:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.420+0000 7fc8e7915640 1 -- 172.21.15.22:0/114303392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e01a70b0 con 0x7fc8e0101420 2024-09-19T08:17:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.423+0000 7fc8d67fc640 1 -- 172.21.15.22:0/114303392 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc8d0042030 con 0x7fc8e0101420 2024-09-19T08:17:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.423+0000 7fc8e9cee740 1 Processor -- start 2024-09-19T08:17:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.423+0000 7fc8e9cee740 1 -- start start 2024-09-19T08:17:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.423+0000 7fc8e9cee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac85060 0x55ec3ac85440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.423+0000 7fc8d67fc640 1 --2- 172.21.15.22:0/114303392 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc8cc0a3430 0x7fc8cc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.423+0000 7fc8e9cee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85980 0x55ec3ac8fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e9cee740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac90240 0x55ec3ac92630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e9cee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ec3abea620 con 0x55ec3ac85980 2024-09-19T08:17:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e9cee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ec3ab91e50 con 0x55ec3ac85060 2024-09-19T08:17:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e9cee740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ec3abe65e0 con 0x55ec3ac90240 2024-09-19T08:17:09.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e4eb0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85980 0x55ec3ac8fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e56b1640 1 --2- 172.21.15.22:0/114303392 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc8cc0a3430 0x7fc8cc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e5eb2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac85060 0x55ec3ac85440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e4eb0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85980 0x55ec3ac8fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38126/0 (socket says 172.21.15.22:38126) 2024-09-19T08:17:09.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e4eb0640 1 -- 172.21.15.22:0/3375243914 learned_addr learned my addr 172.21.15.22:0/3375243914 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:09.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e5eb2640 1 --2- 172.21.15.22:0/3375243914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac90240 0x55ec3ac92630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8d67fc640 1 -- 172.21.15.22:0/114303392 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc8d00efe70 con 0x7fc8e0101420 2024-09-19T08:17:09.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e5eb2640 1 -- 172.21.15.22:0/3375243914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac90240 msgr2=0x55ec3ac92630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:09.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e5eb2640 1 --2- 172.21.15.22:0/3375243914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac90240 0x55ec3ac92630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e5eb2640 1 -- 172.21.15.22:0/3375243914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85980 msgr2=0x55ec3ac8fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:09.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e5eb2640 1 --2- 172.21.15.22:0/3375243914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85980 0x55ec3ac8fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e5eb2640 1 -- 172.21.15.22:0/3375243914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec3ab82d40 con 0x55ec3ac85060 2024-09-19T08:17:09.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e4eb0640 1 --2- 172.21.15.22:0/3375243914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85980 0x55ec3ac8fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:09.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e5eb2640 1 --2- 172.21.15.22:0/3375243914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac85060 0x55ec3ac85440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8005fd0 tx=0x7fc8d8009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d75613fd4b29ed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:09.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.424+0000 7fc8e56b1640 1 --2- 172.21.15.22:0/114303392 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc8cc0a3430 0x7fc8cc0a58f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00bb60 tx=0x7fc8dc00b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:09.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.425+0000 7fc8cb7fe640 1 -- 172.21.15.22:0/3375243914 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8010770 con 0x55ec3ac85060 2024-09-19T08:17:09.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.425+0000 7fc8cb7fe640 1 -- 172.21.15.22:0/3375243914 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d8010df0 con 0x55ec3ac85060 2024-09-19T08:17:09.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.425+0000 7fc8cb7fe640 1 -- 172.21.15.22:0/3375243914 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8011da0 con 0x55ec3ac85060 2024-09-19T08:17:09.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.425+0000 7fc8e9cee740 1 -- 172.21.15.22:0/3375243914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac85060 msgr2=0x55ec3ac85440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:09.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.425+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/3375243914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac85060 0x55ec3ac85440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8005fd0 tx=0x7fc8d8009710 comp rx=0 tx=0).stop 2024-09-19T08:17:09.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.425+0000 7fc8e9cee740 1 -- 172.21.15.22:0/3375243914 shutdown_connections 2024-09-19T08:17:09.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.425+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/3375243914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac90240 0x55ec3ac92630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.425+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/3375243914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85980 0x55ec3ac8fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.425+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/3375243914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac85060 0x55ec3ac85440 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.425+0000 7fc8e9cee740 1 -- 172.21.15.22:0/3375243914 >> 172.21.15.22:0/3375243914 conn(0x55ec3ac815f0 msgr2=0x55ec3ac819d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:09.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.426+0000 7fc8e9cee740 1 -- 172.21.15.22:0/3375243914 shutdown_connections 2024-09-19T08:17:09.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.426+0000 7fc8e9cee740 1 -- 172.21.15.22:0/3375243914 wait complete. 2024-09-19T08:17:09.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.426+0000 7fc8e9cee740 1 -- start start 2024-09-19T08:17:09.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.426+0000 7fc8e9cee740 1 Processor -- start 2024-09-19T08:17:09.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.426+0000 7fc8e9cee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac85060 0x55ec3ac8d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.426+0000 7fc8e9cee740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85980 0x55ec3ac8d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.426+0000 7fc8e9cee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac90240 0x55ec3ac8dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.426+0000 7fc8e9cee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ec3ac92e10 con 0x55ec3ac90240 2024-09-19T08:17:09.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.426+0000 7fc8e9cee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ec3ab72f10 con 0x55ec3ac85060 2024-09-19T08:17:09.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.426+0000 7fc8e9cee740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ec3ac92f90 con 0x55ec3ac85980 2024-09-19T08:17:09.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.426+0000 7fc8e9cee740 10 client.0 _ll_register_callbacks cb 0x55ec3ab96db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:09.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8e56b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac85060 0x55ec3ac8d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8e5eb2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac90240 0x55ec3ac8dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8e56b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac85060 0x55ec3ac8d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:56258/0 (socket says 172.21.15.22:56258) 2024-09-19T08:17:09.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8e56b1640 1 -- 172.21.15.22:0/1842495992 learned_addr learned my addr 172.21.15.22:0/1842495992 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:09.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8e4eb0640 1 --2- 172.21.15.22:0/1842495992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85980 0x55ec3ac8d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8e56b1640 1 -- 172.21.15.22:0/1842495992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85980 msgr2=0x55ec3ac8d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:09.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8e56b1640 1 --2- 172.21.15.22:0/1842495992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85980 0x55ec3ac8d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8e56b1640 1 -- 172.21.15.22:0/1842495992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac90240 msgr2=0x55ec3ac8dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:09.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8e56b1640 1 --2- 172.21.15.22:0/1842495992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac90240 0x55ec3ac8dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8e56b1640 1 -- 172.21.15.22:0/1842495992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec3a8f9580 con 0x55ec3ac85060 2024-09-19T08:17:09.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8e5eb2640 1 --2- 172.21.15.22:0/1842495992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac90240 0x55ec3ac8dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:09.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8e56b1640 1 --2- 172.21.15.22:0/1842495992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac85060 0x55ec3ac8d170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0116c0 tx=0x7fc8dc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b09b693a93d94b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:09.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:17:09.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8c8ff9640 1 -- 172.21.15.22:0/1842495992 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc01e8f0 con 0x55ec3ac85060 2024-09-19T08:17:09.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8e9cee740 1 -- 172.21.15.22:0/1842495992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d8010df0 con 0x55ec3ac85060 2024-09-19T08:17:09.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.427+0000 7fc8c8ff9640 1 -- 172.21.15.22:0/1842495992 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc8dc01ea90 con 0x55ec3ac85060 2024-09-19T08:17:09.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.428+0000 7fc8c8ff9640 1 -- 172.21.15.22:0/1842495992 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc8dc01ec90 con 0x55ec3ac85060 2024-09-19T08:17:09.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.428+0000 7fc8e9cee740 10 client.25535 fetch_fsmap learned FSMap version 26 2024-09-19T08:17:09.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.428+0000 7fc8e9cee740 1 -- 172.21.15.22:0/1842495992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ec3a9c3960 con 0x55ec3ac85060 2024-09-19T08:17:09.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.428+0000 7fc8c8ff9640 1 -- 172.21.15.22:0/1842495992 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d8010df0 con 0x55ec3ac85060 2024-09-19T08:17:09.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.428+0000 7fc8c8ff9640 1 -- 172.21.15.22:0/1842495992 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fc8dc026500 con 0x55ec3ac85060 2024-09-19T08:17:09.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.428+0000 7fc8e9cee740 10 client.25535 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:17:09.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.428+0000 7fc8e9cee740 10 client.25535 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:17:09.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.428+0000 7fc8e9cee740 20 client.25535 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:09.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.428+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/1842495992 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ec3ac9fe30 0x55ec3aca2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.428+0000 7fc8e9cee740 4 client.25535 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:17:09.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.428+0000 7fc8e9cee740 1 -- 172.21.15.22:0/1842495992 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec3abf7b90 con 0x55ec3ac9fe30 2024-09-19T08:17:09.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.428+0000 7fc8e4eb0640 1 --2- 172.21.15.22:0/1842495992 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ec3ac9fe30 0x55ec3aca2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.429+0000 7fc8e4eb0640 1 --2- 172.21.15.22:0/1842495992 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ec3ac9fe30 0x55ec3aca2240 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc18c1608a201c1e server_cookie=3a466879dfd16057 in_seq=0 out_seq=0 2024-09-19T08:17:09.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.429+0000 7fc8c8ff9640 10 client.25535 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:09.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.429+0000 7fc8c8ff9640 1 -- 172.21.15.22:0/1842495992 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec3abf7b90 con 0x55ec3ac9fe30 2024-09-19T08:17:09.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.429+0000 7fc8c8ff9640 10 client.25535 handle_command_reply: tid=0 2024-09-19T08:17:09.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.429+0000 7fc8e9cee740 1 client.25535 shutdown 2024-09-19T08:17:09.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.430+0000 7fc8e9cee740 1 -- 172.21.15.22:0/1842495992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac85060 msgr2=0x55ec3ac8d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:09.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.430+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/1842495992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac85060 0x55ec3ac8d170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0116c0 tx=0x7fc8dc004730 comp rx=0 tx=0).stop 2024-09-19T08:17:09.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.430+0000 7fc8e9cee740 1 -- 172.21.15.22:0/1842495992 shutdown_connections 2024-09-19T08:17:09.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.430+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/1842495992 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ec3ac9fe30 0x55ec3aca2240 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.430+0000 7fc8c8ff9640 0 client.25535 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:09.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.430+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/1842495992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac90240 0x55ec3ac8dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.430+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/1842495992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85980 0x55ec3ac8d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.430+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/1842495992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac85060 0x55ec3ac8d170 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.430+0000 7fc8e9cee740 1 -- 172.21.15.22:0/1842495992 >> 172.21.15.22:0/1842495992 conn(0x55ec3ac815f0 msgr2=0x55ec3ac87320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:09.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.430+0000 7fc8e9cee740 1 -- 172.21.15.22:0/1842495992 shutdown_connections 2024-09-19T08:17:09.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.430+0000 7fc8e9cee740 1 -- 172.21.15.22:0/1842495992 wait complete. 2024-09-19T08:17:09.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.430+0000 7fc8e9cee740 20 client.25535 trim_cache size 0 max 16384 2024-09-19T08:17:09.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.447+0000 7fc8e9cee740 1 Processor -- start 2024-09-19T08:17:09.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.447+0000 7fc8e9cee740 1 -- start start 2024-09-19T08:17:09.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.447+0000 7fc8e9cee740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85060 0x55ec3ac884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.447+0000 7fc8e9cee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85980 0x55ec3ac8f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.447+0000 7fc8e9cee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac90240 0x55ec3aca7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.447+0000 7fc8e9cee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01b22b0 con 0x55ec3ac85980 2024-09-19T08:17:09.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.447+0000 7fc8e9cee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01afb80 con 0x55ec3ac90240 2024-09-19T08:17:09.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.447+0000 7fc8e9cee740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01ad9d0 con 0x55ec3ac85060 2024-09-19T08:17:09.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.447+0000 7fc8e4eb0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85980 0x55ec3ac8f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.447+0000 7fc8e4eb0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85980 0x55ec3ac8f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38144/0 (socket says 172.21.15.22:38144) 2024-09-19T08:17:09.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.447+0000 7fc8e4eb0640 1 -- 172.21.15.22:0/5438566 learned_addr learned my addr 172.21.15.22:0/5438566 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:09.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.448+0000 7fc8e5eb2640 1 --2- 172.21.15.22:0/5438566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac90240 0x55ec3aca7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.448+0000 7fc8e4eb0640 1 -- 172.21.15.22:0/5438566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85060 msgr2=0x55ec3ac884c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:09.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.448+0000 7fc8e56b1640 1 --2- 172.21.15.22:0/5438566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85060 0x55ec3ac884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.448+0000 7fc8e4eb0640 1 --2- 172.21.15.22:0/5438566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85060 0x55ec3ac884c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.448+0000 7fc8e4eb0640 1 -- 172.21.15.22:0/5438566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac90240 msgr2=0x55ec3aca7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:09.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.448+0000 7fc8e4eb0640 1 --2- 172.21.15.22:0/5438566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac90240 0x55ec3aca7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.448+0000 7fc8e4eb0640 1 -- 172.21.15.22:0/5438566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec3ac80610 con 0x55ec3ac85980 2024-09-19T08:17:09.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.448+0000 7fc8e56b1640 1 --2- 172.21.15.22:0/5438566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85060 0x55ec3ac884c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:17:09.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.448+0000 7fc8e5eb2640 1 --2- 172.21.15.22:0/5438566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac90240 0x55ec3aca7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:17:09.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.448+0000 7fc8e4eb0640 1 --2- 172.21.15.22:0/5438566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85980 0x55ec3ac8f0e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00b8560 tx=0x7fc8d0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9ede068f8c42b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:09.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.448+0000 7fc8caffd640 1 -- 172.21.15.22:0/5438566 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0049040 con 0x55ec3ac85980 2024-09-19T08:17:09.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.448+0000 7fc8caffd640 1 -- 172.21.15.22:0/5438566 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d00c04b0 con 0x55ec3ac85980 2024-09-19T08:17:09.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.448+0000 7fc8caffd640 1 -- 172.21.15.22:0/5438566 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00c3620 con 0x55ec3ac85980 2024-09-19T08:17:09.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.449+0000 7fc8e9cee740 1 -- 172.21.15.22:0/5438566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85980 msgr2=0x55ec3ac8f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:09.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.449+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/5438566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85980 0x55ec3ac8f0e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00b8560 tx=0x7fc8d0005c00 comp rx=0 tx=0).stop 2024-09-19T08:17:09.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.449+0000 7fc8e9cee740 1 -- 172.21.15.22:0/5438566 shutdown_connections 2024-09-19T08:17:09.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.449+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/5438566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac90240 0x55ec3aca7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.449+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/5438566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85980 0x55ec3ac8f0e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.449+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/5438566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85060 0x55ec3ac884c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.449+0000 7fc8e9cee740 1 -- 172.21.15.22:0/5438566 >> 172.21.15.22:0/5438566 conn(0x55ec3ac815f0 msgr2=0x55ec3ac8e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:09.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.449+0000 7fc8e9cee740 1 -- 172.21.15.22:0/5438566 shutdown_connections 2024-09-19T08:17:09.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.449+0000 7fc8e9cee740 1 -- 172.21.15.22:0/5438566 wait complete. 2024-09-19T08:17:09.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.449+0000 7fc8e9cee740 1 -- start start 2024-09-19T08:17:09.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.449+0000 7fc8e9cee740 1 Processor -- start 2024-09-19T08:17:09.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e9cee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85060 0x55ec3ac983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e9cee740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85980 0x55ec3ac98950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e9cee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac90240 0x55ec3ac95240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e9cee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01ae040 con 0x55ec3ac85060 2024-09-19T08:17:09.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e9cee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01af4b0 con 0x55ec3ac90240 2024-09-19T08:17:09.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e9cee740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01ae710 con 0x55ec3ac85980 2024-09-19T08:17:09.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e9cee740 10 client.0 _ll_register_callbacks cb 0x7fc8dc010e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:09.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e56b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85060 0x55ec3ac983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e56b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85060 0x55ec3ac983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38146/0 (socket says 172.21.15.22:38146) 2024-09-19T08:17:09.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e56b1640 1 -- 172.21.15.22:0/4087731494 learned_addr learned my addr 172.21.15.22:0/4087731494 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:09.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e4eb0640 1 --2- 172.21.15.22:0/4087731494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85980 0x55ec3ac98950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e5eb2640 1 --2- 172.21.15.22:0/4087731494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac90240 0x55ec3ac95240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e56b1640 1 -- 172.21.15.22:0/4087731494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85980 msgr2=0x55ec3ac98950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:09.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e56b1640 1 --2- 172.21.15.22:0/4087731494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85980 0x55ec3ac98950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e56b1640 1 -- 172.21.15.22:0/4087731494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac90240 msgr2=0x55ec3ac95240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:09.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e56b1640 1 --2- 172.21.15.22:0/4087731494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac90240 0x55ec3ac95240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e56b1640 1 -- 172.21.15.22:0/4087731494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec3ac95a70 con 0x55ec3ac85060 2024-09-19T08:17:09.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8e56b1640 1 --2- 172.21.15.22:0/4087731494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85060 0x55ec3ac983f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc010e40 tx=0x7fc8dc0266b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9701186e35d401e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:09.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:17:09.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.450+0000 7fc8c8ff9640 1 -- 172.21.15.22:0/4087731494 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc035040 con 0x55ec3ac85060 2024-09-19T08:17:09.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.451+0000 7fc8c8ff9640 1 -- 172.21.15.22:0/4087731494 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc8dc01e8f0 con 0x55ec3ac85060 2024-09-19T08:17:09.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.451+0000 7fc8e9cee740 1 -- 172.21.15.22:0/4087731494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d00c04b0 con 0x55ec3ac85060 2024-09-19T08:17:09.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.451+0000 7fc8c8ff9640 1 -- 172.21.15.22:0/4087731494 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc8dc01eaf0 con 0x55ec3ac85060 2024-09-19T08:17:09.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.451+0000 7fc8e9cee740 10 client.15909 fetch_fsmap learned FSMap version 26 2024-09-19T08:17:09.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.451+0000 7fc8e9cee740 1 -- 172.21.15.22:0/4087731494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ec3ad0b620 con 0x55ec3ac85060 2024-09-19T08:17:09.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.451+0000 7fc8c8ff9640 1 -- 172.21.15.22:0/4087731494 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d00c04b0 con 0x55ec3ac85060 2024-09-19T08:17:09.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.451+0000 7fc8c8ff9640 1 -- 172.21.15.22:0/4087731494 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fc8dc0048c0 con 0x55ec3ac85060 2024-09-19T08:17:09.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.451+0000 7fc8e9cee740 10 client.15909 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T08:17:09.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.451+0000 7fc8e9cee740 10 client.15909 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:17:09.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.451+0000 7fc8e9cee740 20 client.15909 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:09.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.451+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/4087731494 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ec3ac9fe30 0x55ec3ad0de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:09.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.451+0000 7fc8e9cee740 4 client.15909 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f364c789", "release": true, "await": true}] 2024-09-19T08:17:09.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.451+0000 7fc8e9cee740 1 -- 172.21.15.22:0/4087731494 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f364c789", "release": true, "await": true}) -- 0x55ec3abf20a0 con 0x55ec3ac9fe30 2024-09-19T08:17:09.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.451+0000 7fc8e4eb0640 1 --2- 172.21.15.22:0/4087731494 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ec3ac9fe30 0x55ec3ad0de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:09.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.452+0000 7fc8e4eb0640 1 --2- 172.21.15.22:0/4087731494 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ec3ac9fe30 0x55ec3ad0de10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fde5915e5dc33763 server_cookie=afd6a1f528edf160 in_seq=0 out_seq=0 2024-09-19T08:17:09.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.452+0000 7fc8c8ff9640 10 client.15909 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:09.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8c8ff9640 1 -- 172.21.15.22:0/4087731494 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ec3abf20a0 con 0x55ec3ac9fe30 2024-09-19T08:17:09.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8c8ff9640 10 client.15909 handle_command_reply: tid=0 2024-09-19T08:17:09.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8e9cee740 1 client.15909 shutdown 2024-09-19T08:17:09.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8e9cee740 1 -- 172.21.15.22:0/4087731494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85060 msgr2=0x55ec3ac983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:09.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/4087731494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85060 0x55ec3ac983f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc010e40 tx=0x7fc8dc0266b0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8e9cee740 1 -- 172.21.15.22:0/4087731494 shutdown_connections 2024-09-19T08:17:09.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/4087731494 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ec3ac9fe30 0x55ec3ad0de10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/4087731494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec3ac90240 0x55ec3ac95240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/4087731494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3ac85980 0x55ec3ac98950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8c8ff9640 0 client.15909 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:09.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8e9cee740 1 --2- 172.21.15.22:0/4087731494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec3ac85060 0x55ec3ac983f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8e9cee740 1 -- 172.21.15.22:0/4087731494 >> 172.21.15.22:0/4087731494 conn(0x55ec3ac815f0 msgr2=0x55ec3ac8f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:09.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8e9cee740 1 -- 172.21.15.22:0/4087731494 shutdown_connections 2024-09-19T08:17:09.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8e9cee740 1 -- 172.21.15.22:0/4087731494 wait complete. 2024-09-19T08:17:09.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.459+0000 7fc8e9cee740 20 client.15909 trim_cache size 0 max 16384 2024-09-19T08:17:09.461 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:17:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 26, 2024-09-19T08:17:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:17:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 12, 2024-09-19T08:17:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:17:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: "f364c789": { 2024-09-19T08:17:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 12, 2024-09-19T08:17:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:17:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:17:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:17:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:17:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:17:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:17:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 140.3, 2024-09-19T08:17:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:17:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:17:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:17:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:17:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:17:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:17:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:17:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:17:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:17:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:17:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:17:09.463 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:17:09.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.460+0000 7fc8e7915640 1 -- 172.21.15.22:0/114303392 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc8cc0a3430 msgr2=0x7fc8cc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:09.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.460+0000 7fc8e7915640 1 --2- 172.21.15.22:0/114303392 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc8cc0a3430 0x7fc8cc0a58f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00bb60 tx=0x7fc8dc00b390 comp rx=0 tx=0).stop 2024-09-19T08:17:09.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.460+0000 7fc8e7915640 1 -- 172.21.15.22:0/114303392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8e0101420 msgr2=0x7fc8e00fd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:09.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.460+0000 7fc8e7915640 1 --2- 172.21.15.22:0/114303392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8e0101420 0x7fc8e00fd340 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009230 tx=0x7fc8d0004270 comp rx=0 tx=0).stop 2024-09-19T08:17:09.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.460+0000 7fc8e7915640 1 -- 172.21.15.22:0/114303392 shutdown_connections 2024-09-19T08:17:09.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.460+0000 7fc8e7915640 1 --2- 172.21.15.22:0/114303392 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc8cc0a3430 0x7fc8cc0a58f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.460+0000 7fc8e7915640 1 --2- 172.21.15.22:0/114303392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8e0106a80 0x7fc8e00fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.460+0000 7fc8e7915640 1 --2- 172.21.15.22:0/114303392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8e0101420 0x7fc8e00fd340 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.460+0000 7fc8e7915640 1 --2- 172.21.15.22:0/114303392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e0100a80 0x7fc8e00fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:09.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.460+0000 7fc8e7915640 1 -- 172.21.15.22:0/114303392 >> 172.21.15.22:0/114303392 conn(0x7fc8e00fc7a0 msgr2=0x7fc8e006aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:09.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.460+0000 7fc8e7915640 1 -- 172.21.15.22:0/114303392 shutdown_connections 2024-09-19T08:17:09.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:09.460+0000 7fc8e7915640 1 -- 172.21.15.22:0/114303392 wait complete. 2024-09-19T08:17:09.480 INFO:tasks.workunit.client.0.smithi022.stdout:checking for strtoull... yes 2024-09-19T08:17:09.553 INFO:tasks.workunit.client.0.smithi022.stdout:checking for strtoq... yes 2024-09-19T08:17:09.615 INFO:tasks.workunit.client.0.smithi022.stdout:checking for snprintf... yes 2024-09-19T08:17:09.679 INFO:tasks.workunit.client.0.smithi022.stdout:checking for vsnprintf... yes 2024-09-19T08:17:09.741 INFO:tasks.workunit.client.0.smithi022.stdout:checking for vfprintf... yes 2024-09-19T08:17:09.801 INFO:tasks.workunit.client.0.smithi022.stdout:checking for gethostname... yes 2024-09-19T08:17:09.865 INFO:tasks.workunit.client.0.smithi022.stdout:checking for setlocale... yes 2024-09-19T08:17:09.926 INFO:tasks.workunit.client.0.smithi022.stdout:checking for sync... yes 2024-09-19T08:17:09.992 INFO:tasks.workunit.client.0.smithi022.stdout:checking whether snprintf is C99 conformant... done 2024-09-19T08:17:10.003 INFO:tasks.workunit.client.0.smithi022.stdout:configure: creating ./config.status 2024-09-19T08:17:10.157 INFO:tasks.workunit.client.0.smithi022.stdout:config.status: creating Makefile 2024-09-19T08:17:10.175 INFO:tasks.workunit.client.0.smithi022.stdout:config.status: creating src/Makefile 2024-09-19T08:17:10.366 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f364c789', total seconds elapsed: 23.8 2024-09-19T08:17:11.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:10 smithi157 ceph-mon[35143]: pgmap v67: 97 pgs: 97 active+clean; 4.7 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 KiB/s wr, 30 op/s 2024-09-19T08:17:11.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:10 smithi022 ceph-mon[30963]: pgmap v67: 97 pgs: 97 active+clean; 4.7 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 KiB/s wr, 30 op/s 2024-09-19T08:17:11.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:10 smithi064 ceph-mon[34900]: pgmap v67: 97 pgs: 97 active+clean; 4.7 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 KiB/s wr, 30 op/s 2024-09-19T08:17:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:12 smithi157 ceph-mon[35143]: pgmap v68: 97 pgs: 97 active+clean; 4.7 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 KiB/s wr, 30 op/s 2024-09-19T08:17:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:12 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:17:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:12 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:12 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:12 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:12 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:13.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:12 smithi022 ceph-mon[30963]: pgmap v68: 97 pgs: 97 active+clean; 4.7 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 KiB/s wr, 30 op/s 2024-09-19T08:17:13.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:12 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:17:13.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:12 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:13.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:12 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:13.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:12 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:13.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:12 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:13.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:12 smithi064 ceph-mon[34900]: pgmap v68: 97 pgs: 97 active+clean; 4.7 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 KiB/s wr, 30 op/s 2024-09-19T08:17:13.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:12 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:17:13.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:12 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:13.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:12 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:13.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:12 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:13.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:12 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:13 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:17:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:13 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:17:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:13 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:14.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:13 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:17:14.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:13 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:17:14.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:13 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:14.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:13 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:17:14.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:13 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:17:14.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:13 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:17:14.752 INFO:tasks.workunit.client.0.smithi022.stdout:config.status: creating man/Makefile 2024-09-19T08:17:14.770 INFO:tasks.workunit.client.0.smithi022.stdout:config.status: creating config.h 2024-09-19T08:17:14.840 INFO:tasks.workunit.client.0.smithi022.stdout:config.status: executing depfiles commands 2024-09-19T08:17:14.981 INFO:tasks.workunit.client.0.smithi022.stderr:+ make 2024-09-19T08:17:14.990 INFO:tasks.workunit.client.0.smithi022.stdout:make all-recursive 2024-09-19T08:17:14.992 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-19T08:17:14.995 INFO:tasks.workunit.client.0.smithi022.stdout:Making all in src 2024-09-19T08:17:15.007 INFO:tasks.workunit.client.0.smithi022.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-19T08:17:15.007 INFO:tasks.workunit.client.0.smithi022.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-09-19T08:17:15.008 INFO:tasks.workunit.client.0.smithi022.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-09-19T08:17:15.137 INFO:tasks.workunit.client.0.smithi022.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-09-19T08:17:15.138 INFO:tasks.workunit.client.0.smithi022.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-09-19T08:17:15.233 INFO:tasks.workunit.client.0.smithi022.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-09-19T08:17:15.233 INFO:tasks.workunit.client.0.smithi022.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-09-19T08:17:15.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:14 smithi157 ceph-mon[35143]: pgmap v69: 97 pgs: 97 active+clean; 5.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 113 KiB/s wr, 35 op/s 2024-09-19T08:17:15.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:14 smithi022 ceph-mon[30963]: pgmap v69: 97 pgs: 97 active+clean; 5.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 113 KiB/s wr, 35 op/s 2024-09-19T08:17:15.314 INFO:tasks.workunit.client.0.smithi022.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-09-19T08:17:15.314 INFO:tasks.workunit.client.0.smithi022.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-09-19T08:17:15.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:14 smithi064 ceph-mon[34900]: pgmap v69: 97 pgs: 97 active+clean; 5.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 113 KiB/s wr, 35 op/s 2024-09-19T08:17:15.396 INFO:tasks.workunit.client.0.smithi022.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-09-19T08:17:15.396 INFO:tasks.workunit.client.0.smithi022.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-09-19T08:17:15.476 INFO:tasks.workunit.client.0.smithi022.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-09-19T08:17:15.476 INFO:tasks.workunit.client.0.smithi022.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-09-19T08:17:15.552 INFO:tasks.workunit.client.0.smithi022.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-09-19T08:17:15.552 INFO:tasks.workunit.client.0.smithi022.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-09-19T08:17:15.654 INFO:tasks.workunit.client.0.smithi022.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-09-19T08:17:15.655 INFO:tasks.workunit.client.0.smithi022.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-09-19T08:17:15.717 INFO:tasks.workunit.client.0.smithi022.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-09-19T08:17:15.717 INFO:tasks.workunit.client.0.smithi022.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-09-19T08:17:15.783 INFO:tasks.workunit.client.0.smithi022.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-09-19T08:17:15.784 INFO:tasks.workunit.client.0.smithi022.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-09-19T08:17:15.840 INFO:tasks.workunit.client.0.smithi022.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-09-19T08:17:15.860 INFO:tasks.workunit.client.0.smithi022.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-19T08:17:15.860 INFO:tasks.workunit.client.0.smithi022.stdout:Making all in man 2024-09-19T08:17:15.862 INFO:tasks.workunit.client.0.smithi022.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-19T08:17:15.862 INFO:tasks.workunit.client.0.smithi022.stdout:make[2]: Nothing to be done for 'all'. 2024-09-19T08:17:15.862 INFO:tasks.workunit.client.0.smithi022.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-19T08:17:15.864 INFO:tasks.workunit.client.0.smithi022.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-19T08:17:15.865 INFO:tasks.workunit.client.0.smithi022.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-19T08:17:15.865 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-19T08:17:15.865 INFO:tasks.workunit.client.0.smithi022.stderr:+ cd src 2024-09-19T08:17:15.872 INFO:tasks.workunit.client.0.smithi022.stderr:+ mkdir blogtest_in 2024-09-19T08:17:15.875 INFO:tasks.workunit.client.0.smithi022.stdout:running blogbench 2024-09-19T08:17:15.875 INFO:tasks.workunit.client.0.smithi022.stderr:+ echo 'running blogbench' 2024-09-19T08:17:15.875 INFO:tasks.workunit.client.0.smithi022.stderr:+ ./blogbench -d blogtest_in 2024-09-19T08:17:15.875 INFO:tasks.workunit.client.0.smithi022.stdout: 2024-09-19T08:17:15.875 INFO:tasks.workunit.client.0.smithi022.stdout:Frequency = 10 secs 2024-09-19T08:17:15.876 INFO:tasks.workunit.client.0.smithi022.stdout:Scratch dir = [blogtest_in] 2024-09-19T08:17:16.151 INFO:tasks.workunit.client.0.smithi022.stdout:Spawning 3 writers... 2024-09-19T08:17:16.152 INFO:tasks.workunit.client.0.smithi022.stdout:Spawning 1 rewriters... 2024-09-19T08:17:16.152 INFO:tasks.workunit.client.0.smithi022.stdout:Spawning 5 commenters... 2024-09-19T08:17:16.152 INFO:tasks.workunit.client.0.smithi022.stdout:Spawning 100 readers... 2024-09-19T08:17:16.153 INFO:tasks.workunit.client.0.smithi022.stdout:Benchmarking for 30 iterations. 2024-09-19T08:17:16.154 INFO:tasks.workunit.client.0.smithi022.stdout:The test will run during 5 minutes. 2024-09-19T08:17:16.154 INFO:tasks.workunit.client.0.smithi022.stdout: 2024-09-19T08:17:16.154 INFO:tasks.workunit.client.0.smithi022.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-09-19T08:17:17.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:16 smithi157 ceph-mon[35143]: pgmap v70: 97 pgs: 97 active+clean; 5.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-19T08:17:17.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:16 smithi022 ceph-mon[30963]: pgmap v70: 97 pgs: 97 active+clean; 5.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-19T08:17:17.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:16 smithi064 ceph-mon[34900]: pgmap v70: 97 pgs: 97 active+clean; 5.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-19T08:17:19.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:18 smithi157 ceph-mon[35143]: pgmap v71: 97 pgs: 97 active+clean; 8.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 323 KiB/s wr, 32 op/s 2024-09-19T08:17:19.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:18 smithi022 ceph-mon[30963]: pgmap v71: 97 pgs: 97 active+clean; 8.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 323 KiB/s wr, 32 op/s 2024-09-19T08:17:19.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:18 smithi064 ceph-mon[34900]: pgmap v71: 97 pgs: 97 active+clean; 8.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 323 KiB/s wr, 32 op/s 2024-09-19T08:17:21.138 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:20 smithi064 ceph-mon[34900]: pgmap v72: 97 pgs: 97 active+clean; 8.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 320 KiB/s wr, 25 op/s 2024-09-19T08:17:21.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:20 smithi157 ceph-mon[35143]: pgmap v72: 97 pgs: 97 active+clean; 8.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 320 KiB/s wr, 25 op/s 2024-09-19T08:17:21.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:20 smithi022 ceph-mon[30963]: pgmap v72: 97 pgs: 97 active+clean; 8.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 320 KiB/s wr, 25 op/s 2024-09-19T08:17:23.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:22 smithi157 ceph-mon[35143]: pgmap v73: 97 pgs: 97 active+clean; 8.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 320 KiB/s wr, 25 op/s 2024-09-19T08:17:23.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:22 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:17:23.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:22 smithi022 ceph-mon[30963]: pgmap v73: 97 pgs: 97 active+clean; 8.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 320 KiB/s wr, 25 op/s 2024-09-19T08:17:23.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:22 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:17:23.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:22 smithi064 ceph-mon[34900]: pgmap v73: 97 pgs: 97 active+clean; 8.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 320 KiB/s wr, 25 op/s 2024-09-19T08:17:23.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:22 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:17:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:24 smithi157 ceph-mon[35143]: pgmap v74: 97 pgs: 97 active+clean; 60 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 4.8 MiB/s wr, 208 op/s 2024-09-19T08:17:25.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:24 smithi022 ceph-mon[30963]: pgmap v74: 97 pgs: 97 active+clean; 60 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 4.8 MiB/s wr, 208 op/s 2024-09-19T08:17:25.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:24 smithi064 ceph-mon[34900]: pgmap v74: 97 pgs: 97 active+clean; 60 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 4.8 MiB/s wr, 208 op/s 2024-09-19T08:17:26.154 INFO:tasks.workunit.client.0.smithi022.stdout: 21 168771 1149 119184 1073 47000 1000 2024-09-19T08:17:27.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:26 smithi157 ceph-mon[35143]: pgmap v75: 97 pgs: 97 active+clean; 60 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 4.7 MiB/s wr, 202 op/s 2024-09-19T08:17:27.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:26 smithi022 ceph-mon[30963]: pgmap v75: 97 pgs: 97 active+clean; 60 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 4.7 MiB/s wr, 202 op/s 2024-09-19T08:17:27.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:26 smithi064 ceph-mon[34900]: pgmap v75: 97 pgs: 97 active+clean; 60 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 4.7 MiB/s wr, 202 op/s 2024-09-19T08:17:29.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:28 smithi157 ceph-mon[35143]: pgmap v76: 97 pgs: 97 active+clean; 68 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 248 op/s 2024-09-19T08:17:29.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:28 smithi022 ceph-mon[30963]: pgmap v76: 97 pgs: 97 active+clean; 68 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 248 op/s 2024-09-19T08:17:29.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:28 smithi064 ceph-mon[34900]: pgmap v76: 97 pgs: 97 active+clean; 68 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 248 op/s 2024-09-19T08:17:31.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:30 smithi157 ceph-mon[35143]: pgmap v77: 97 pgs: 97 active+clean; 68 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 228 op/s 2024-09-19T08:17:31.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:30 smithi022 ceph-mon[30963]: pgmap v77: 97 pgs: 97 active+clean; 68 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 228 op/s 2024-09-19T08:17:31.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:30 smithi064 ceph-mon[34900]: pgmap v77: 97 pgs: 97 active+clean; 68 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 228 op/s 2024-09-19T08:17:31.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:17:32.140 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:17:32.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.659+0000 7f16c971f640 1 -- 172.21.15.22:0/2961165528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4103780 msgr2=0x7f16c4107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:32.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.659+0000 7f16c971f640 1 --2- 172.21.15.22:0/2961165528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4103780 0x7f16c4107bc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009930 tx=0x7f16b802a420 comp rx=0 tx=0).stop 2024-09-19T08:17:32.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.659+0000 7f16c971f640 1 -- 172.21.15.22:0/2961165528 shutdown_connections 2024-09-19T08:17:32.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.659+0000 7f16c971f640 1 --2- 172.21.15.22:0/2961165528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c4108300 0x7f16c410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.659+0000 7f16c971f640 1 --2- 172.21.15.22:0/2961165528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4103780 0x7f16c4107bc0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.659+0000 7f16c971f640 1 --2- 172.21.15.22:0/2961165528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c4102dd0 0x7f16c41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.659+0000 7f16c971f640 1 -- 172.21.15.22:0/2961165528 >> 172.21.15.22:0/2961165528 conn(0x7f16c40fc7d0 msgr2=0x7f16c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:32.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.659+0000 7f16c971f640 1 -- 172.21.15.22:0/2961165528 shutdown_connections 2024-09-19T08:17:32.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.659+0000 7f16c971f640 1 -- 172.21.15.22:0/2961165528 wait complete. 2024-09-19T08:17:32.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.660+0000 7f16c971f640 1 Processor -- start 2024-09-19T08:17:32.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.660+0000 7f16c971f640 1 -- start start 2024-09-19T08:17:32.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.660+0000 7f16c971f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c4102dd0 0x7f16c4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.660+0000 7f16c971f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c4103780 0x7f16c4198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.660+0000 7f16c971f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4108300 0x7f16c419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.660+0000 7f16c971f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41120c0 con 0x7f16c4108300 2024-09-19T08:17:32.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.660+0000 7f16c971f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4111f40 con 0x7f16c4103780 2024-09-19T08:17:32.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.660+0000 7f16c971f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4112240 con 0x7f16c4102dd0 2024-09-19T08:17:32.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.660+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c4102dd0 0x7f16c4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.660+0000 7f16c27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c4103780 0x7f16c4198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.660+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4108300 0x7f16c419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.660+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c4102dd0 0x7f16c4198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:56946/0 (socket says 172.21.15.22:56946) 2024-09-19T08:17:32.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.660+0000 7f16c2ffd640 1 -- 172.21.15.22:0/1029719645 learned_addr learned my addr 172.21.15.22:0/1029719645 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:32.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.661+0000 7f16c2ffd640 1 -- 172.21.15.22:0/1029719645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c4103780 msgr2=0x7f16c4198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:32.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.661+0000 7f16c2ffd640 1 --2- 172.21.15.22:0/1029719645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c4103780 0x7f16c4198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.661+0000 7f16c2ffd640 1 -- 172.21.15.22:0/1029719645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4108300 msgr2=0x7f16c419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:32.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.661+0000 7f16c2ffd640 1 --2- 172.21.15.22:0/1029719645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4108300 0x7f16c419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.661+0000 7f16c2ffd640 1 -- 172.21.15.22:0/1029719645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c419d5b0 con 0x7f16c4102dd0 2024-09-19T08:17:32.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.661+0000 7f16c27fc640 1 --2- 172.21.15.22:0/1029719645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c4103780 0x7f16c4198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:32.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.661+0000 7f16c37fe640 1 --2- 172.21.15.22:0/1029719645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4108300 0x7f16c419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:32.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.661+0000 7f16c2ffd640 1 --2- 172.21.15.22:0/1029719645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c4102dd0 0x7f16c4198600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f16b400e9f0 tx=0x7f16b400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:32.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.661+0000 7f16abfff640 1 -- 172.21.15.22:0/1029719645 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b400cd80 con 0x7f16c4102dd0 2024-09-19T08:17:32.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.661+0000 7f16abfff640 1 -- 172.21.15.22:0/1029719645 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b4005500 con 0x7f16c4102dd0 2024-09-19T08:17:32.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.661+0000 7f16c971f640 1 -- 172.21.15.22:0/1029719645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c419d8a0 con 0x7f16c4102dd0 2024-09-19T08:17:32.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.661+0000 7f16abfff640 1 -- 172.21.15.22:0/1029719645 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4010690 con 0x7f16c4102dd0 2024-09-19T08:17:32.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.661+0000 7f16c971f640 1 -- 172.21.15.22:0/1029719645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c41a5180 con 0x7f16c4102dd0 2024-09-19T08:17:32.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.664+0000 7f16abfff640 1 -- 172.21.15.22:0/1029719645 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f16b40046c0 con 0x7f16c4102dd0 2024-09-19T08:17:32.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.664+0000 7f16cbaf8740 1 Processor -- start 2024-09-19T08:17:32.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.664+0000 7f16cbaf8740 1 -- start start 2024-09-19T08:17:32.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.664+0000 7f16cbaf8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96b44a0 0x55dca96b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.664+0000 7f16cbaf8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b4dc0 0x55dca96bf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.664+0000 7f16cbaf8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96bf680 0x55dca96c1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.664+0000 7f16cbaf8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dca961a2e0 con 0x55dca96b44a0 2024-09-19T08:17:32.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.664+0000 7f16cbaf8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dca95c1ce0 con 0x55dca96b4dc0 2024-09-19T08:17:32.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.664+0000 7f16cbaf8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dca96162a0 con 0x55dca96bf680 2024-09-19T08:17:32.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.664+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96b44a0 0x55dca96b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.664+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b4dc0 0x55dca96bf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.664+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96b44a0 0x55dca96b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53906/0 (socket says 172.21.15.22:53906) 2024-09-19T08:17:32.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.664+0000 7f16c37fe640 1 -- 172.21.15.22:0/1197411836 learned_addr learned my addr 172.21.15.22:0/1197411836 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:32.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.665+0000 7f16c27fc640 1 --2- 172.21.15.22:0/1197411836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96bf680 0x55dca96c1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.665+0000 7f16abfff640 1 --2- 172.21.15.22:0/1029719645 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16ac0a33b0 0x7f16ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.665+0000 7f16c37fe640 1 -- 172.21.15.22:0/1197411836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96bf680 msgr2=0x55dca96c1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:32.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.665+0000 7f16c37fe640 1 --2- 172.21.15.22:0/1197411836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96bf680 0x55dca96c1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.665+0000 7f16c37fe640 1 -- 172.21.15.22:0/1197411836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b4dc0 msgr2=0x55dca96bf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:32.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.665+0000 7f16c37fe640 1 --2- 172.21.15.22:0/1197411836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b4dc0 0x55dca96bf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.665+0000 7f16c37fe640 1 -- 172.21.15.22:0/1197411836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca95a6440 con 0x55dca96b44a0 2024-09-19T08:17:32.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.665+0000 7f16c37fe640 1 --2- 172.21.15.22:0/1029719645 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16ac0a33b0 0x7f16ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.665+0000 7f16abfff640 1 -- 172.21.15.22:0/1029719645 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f16b40d0150 con 0x7f16c4102dd0 2024-09-19T08:17:32.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.665+0000 7f16c37fe640 1 --2- 172.21.15.22:0/1197411836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96b44a0 0x55dca96b4880 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0118d0 tx=0x7f16bc011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d84d05dbf061e2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:32.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.665+0000 7f16a97fa640 1 -- 172.21.15.22:0/1197411836 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc017050 con 0x55dca96b44a0 2024-09-19T08:17:32.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.665+0000 7f16a97fa640 1 -- 172.21.15.22:0/1197411836 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16bc026440 con 0x55dca96b44a0 2024-09-19T08:17:32.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.665+0000 7f16a97fa640 1 -- 172.21.15.22:0/1197411836 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00e5b0 con 0x55dca96b44a0 2024-09-19T08:17:32.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.666+0000 7f16cbaf8740 1 -- 172.21.15.22:0/1197411836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96b44a0 msgr2=0x55dca96b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:32.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.666+0000 7f16c37fe640 1 --2- 172.21.15.22:0/1029719645 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16ac0a33b0 0x7f16ac0a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f16bc023000 tx=0x7f16bc024000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:32.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.666+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/1197411836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96b44a0 0x55dca96b4880 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0118d0 tx=0x7f16bc011d90 comp rx=0 tx=0).stop 2024-09-19T08:17:32.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.666+0000 7f16cbaf8740 1 -- 172.21.15.22:0/1197411836 shutdown_connections 2024-09-19T08:17:32.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.666+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/1197411836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96bf680 0x55dca96c1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.666+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/1197411836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b4dc0 0x55dca96bf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.666+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/1197411836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96b44a0 0x55dca96b4880 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.666+0000 7f16cbaf8740 1 -- 172.21.15.22:0/1197411836 >> 172.21.15.22:0/1197411836 conn(0x55dca96b0a30 msgr2=0x55dca96b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:32.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.666+0000 7f16cbaf8740 1 -- 172.21.15.22:0/1197411836 shutdown_connections 2024-09-19T08:17:32.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.666+0000 7f16cbaf8740 1 -- 172.21.15.22:0/1197411836 wait complete. 2024-09-19T08:17:32.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.667+0000 7f16cbaf8740 1 -- start start 2024-09-19T08:17:32.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.667+0000 7f16cbaf8740 1 Processor -- start 2024-09-19T08:17:32.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.667+0000 7f16cbaf8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96bef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.667+0000 7f16cbaf8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96b4dc0 0x55dca96bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.667+0000 7f16cbaf8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96bf680 0x55dca96bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.667+0000 7f16cbaf8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dca96c22f0 con 0x55dca96b4dc0 2024-09-19T08:17:32.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.667+0000 7f16cbaf8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dca95d5050 con 0x55dca96b44a0 2024-09-19T08:17:32.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.667+0000 7f16cbaf8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dca96c2470 con 0x55dca96bf680 2024-09-19T08:17:32.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.667+0000 7f16cbaf8740 10 client.0 _ll_register_callbacks cb 0x55dca9615160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:32.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.667+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96bf680 0x55dca96bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.667+0000 7f16c27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96b4dc0 0x55dca96bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.667+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96bf680 0x55dca96bc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:56958/0 (socket says 172.21.15.22:56958) 2024-09-19T08:17:32.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.667+0000 7f16c37fe640 1 -- 172.21.15.22:0/2974954124 learned_addr learned my addr 172.21.15.22:0/2974954124 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:32.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.667+0000 7f16c2ffd640 1 --2- 172.21.15.22:0/2974954124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96bef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16c37fe640 1 -- 172.21.15.22:0/2974954124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 msgr2=0x55dca96bef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:32.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16c37fe640 1 --2- 172.21.15.22:0/2974954124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96bef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16c37fe640 1 -- 172.21.15.22:0/2974954124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96b4dc0 msgr2=0x55dca96bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:32.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16c37fe640 1 --2- 172.21.15.22:0/2974954124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96b4dc0 0x55dca96bc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16c37fe640 1 -- 172.21.15.22:0/2974954124 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dca938b3c0 con 0x55dca96bf680 2024-09-19T08:17:32.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16c2ffd640 1 --2- 172.21.15.22:0/2974954124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96bef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:32.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16c27fc640 1 --2- 172.21.15.22:0/2974954124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96b4dc0 0x55dca96bc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:32.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16c37fe640 1 --2- 172.21.15.22:0/2974954124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96bf680 0x55dca96bc760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16bc012fd0 tx=0x7f16bc004f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dba279a2a13903e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:32.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:17:32.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16a6ff5640 1 -- 172.21.15.22:0/2974954124 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc034020 con 0x55dca96bf680 2024-09-19T08:17:32.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16a6ff5640 1 -- 172.21.15.22:0/2974954124 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f16bc01bc60 con 0x55dca96bf680 2024-09-19T08:17:32.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2974954124 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16bc026440 con 0x55dca96bf680 2024-09-19T08:17:32.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16a6ff5640 1 -- 172.21.15.22:0/2974954124 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16bc03f050 con 0x55dca96bf680 2024-09-19T08:17:32.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16cbaf8740 10 client.25483 fetch_fsmap learned FSMap version 27 2024-09-19T08:17:32.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.668+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2974954124 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dca96d2070 con 0x55dca96bf680 2024-09-19T08:17:32.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.669+0000 7f16a6ff5640 1 -- 172.21.15.22:0/2974954124 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16bc026440 con 0x55dca96bf680 2024-09-19T08:17:32.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.669+0000 7f16a6ff5640 1 -- 172.21.15.22:0/2974954124 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f16bc005190 con 0x55dca96bf680 2024-09-19T08:17:32.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.669+0000 7f16cbaf8740 10 client.25483 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:17:32.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.669+0000 7f16cbaf8740 10 client.25483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:17:32.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.669+0000 7f16cbaf8740 20 client.25483 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:32.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.669+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/2974954124 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55dca96d36c0 0x55dca96d5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.669+0000 7f16cbaf8740 4 client.25483 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:17:32.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.669+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2974954124 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dca9627850 con 0x55dca96d36c0 2024-09-19T08:17:32.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.669+0000 7f16c2ffd640 1 --2- 172.21.15.22:0/2974954124 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55dca96d36c0 0x55dca96d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.670+0000 7f16c2ffd640 1 --2- 172.21.15.22:0/2974954124 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55dca96d36c0 0x55dca96d5ab0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c97e6e87f4d548 server_cookie=fb1e91b8bb6b9e33 in_seq=0 out_seq=0 2024-09-19T08:17:32.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.670+0000 7f16a6ff5640 10 client.25483 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:32.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.670+0000 7f16a6ff5640 1 -- 172.21.15.22:0/2974954124 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dca9627850 con 0x55dca96d36c0 2024-09-19T08:17:32.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.670+0000 7f16a6ff5640 10 client.25483 handle_command_reply: tid=0 2024-09-19T08:17:32.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.670+0000 7f16cbaf8740 1 client.25483 shutdown 2024-09-19T08:17:32.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.670+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2974954124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96bf680 msgr2=0x55dca96bc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:32.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.670+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/2974954124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96bf680 0x55dca96bc760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16bc012fd0 tx=0x7f16bc004f60 comp rx=0 tx=0).stop 2024-09-19T08:17:32.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.670+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2974954124 shutdown_connections 2024-09-19T08:17:32.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.670+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/2974954124 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55dca96d36c0 0x55dca96d5ab0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.670+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/2974954124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96bf680 0x55dca96bc760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.670+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/2974954124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96b4dc0 0x55dca96bc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.670+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/2974954124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96bef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.670+0000 7f16a6ff5640 0 client.25483 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:32.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.671+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2974954124 >> 172.21.15.22:0/2974954124 conn(0x55dca96b0a30 msgr2=0x55dca96b6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:32.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.671+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2974954124 shutdown_connections 2024-09-19T08:17:32.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.671+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2974954124 wait complete. 2024-09-19T08:17:32.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.671+0000 7f16cbaf8740 20 client.25483 trim_cache size 0 max 16384 2024-09-19T08:17:32.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.690+0000 7f16cbaf8740 1 Processor -- start 2024-09-19T08:17:32.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.690+0000 7f16cbaf8740 1 -- start start 2024-09-19T08:17:32.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.690+0000 7f16cbaf8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96db420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.690+0000 7f16cbaf8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96b4dc0 0x55dca96be190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.690+0000 7f16cbaf8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96bf680 0x55dca96d85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.690+0000 7f16cbaf8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41ae040 con 0x55dca96bf680 2024-09-19T08:17:32.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.690+0000 7f16cbaf8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41ab8e0 con 0x55dca96b44a0 2024-09-19T08:17:32.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.690+0000 7f16cbaf8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41a9730 con 0x55dca96b4dc0 2024-09-19T08:17:32.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.690+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96bf680 0x55dca96d85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16c27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96b4dc0 0x55dca96be190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96db420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96bf680 0x55dca96d85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53928/0 (socket says 172.21.15.22:53928) 2024-09-19T08:17:32.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16c27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96b4dc0 0x55dca96be190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:56968/0 (socket says 172.21.15.22:56968) 2024-09-19T08:17:32.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16c37fe640 1 -- 172.21.15.22:0/734545174 learned_addr learned my addr 172.21.15.22:0/734545174 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:32.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16c37fe640 1 -- 172.21.15.22:0/734545174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96b4dc0 msgr2=0x55dca96be190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:32.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16c37fe640 1 --2- 172.21.15.22:0/734545174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96b4dc0 0x55dca96be190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16c37fe640 1 -- 172.21.15.22:0/734545174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 msgr2=0x55dca96db420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:32.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16c37fe640 1 --2- 172.21.15.22:0/734545174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96db420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16c37fe640 1 -- 172.21.15.22:0/734545174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca96bb1d0 con 0x55dca96bf680 2024-09-19T08:17:32.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16c27fc640 1 --2- 172.21.15.22:0/734545174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96b4dc0 0x55dca96be190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:32.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16c37fe640 1 --2- 172.21.15.22:0/734545174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96bf680 0x55dca96d85a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f16bc005630 tx=0x7f16bc011780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8f68e1a4205647 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:32.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16a8ff9640 1 -- 172.21.15.22:0/734545174 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc017070 con 0x55dca96bf680 2024-09-19T08:17:32.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16a8ff9640 1 -- 172.21.15.22:0/734545174 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16bc02c040 con 0x55dca96bf680 2024-09-19T08:17:32.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.691+0000 7f16a8ff9640 1 -- 172.21.15.22:0/734545174 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc034040 con 0x55dca96bf680 2024-09-19T08:17:32.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.692+0000 7f16cbaf8740 1 -- 172.21.15.22:0/734545174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96bf680 msgr2=0x55dca96d85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:32.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.692+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/734545174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96bf680 0x55dca96d85a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f16bc005630 tx=0x7f16bc011780 comp rx=0 tx=0).stop 2024-09-19T08:17:32.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.692+0000 7f16cbaf8740 1 -- 172.21.15.22:0/734545174 shutdown_connections 2024-09-19T08:17:32.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.692+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/734545174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96bf680 0x55dca96d85a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.692+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/734545174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96b4dc0 0x55dca96be190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.692+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/734545174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96db420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.692+0000 7f16cbaf8740 1 -- 172.21.15.22:0/734545174 >> 172.21.15.22:0/734545174 conn(0x55dca96b0a30 msgr2=0x55dca96bda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:32.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.692+0000 7f16cbaf8740 1 -- 172.21.15.22:0/734545174 shutdown_connections 2024-09-19T08:17:32.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.692+0000 7f16cbaf8740 1 -- 172.21.15.22:0/734545174 wait complete. 2024-09-19T08:17:32.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.693+0000 7f16cbaf8740 1 -- start start 2024-09-19T08:17:32.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.693+0000 7f16cbaf8740 1 Processor -- start 2024-09-19T08:17:32.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.693+0000 7f16cbaf8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96d7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.693+0000 7f16cbaf8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96b4dc0 0x55dca96d7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.693+0000 7f16cbaf8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96bf680 0x55dca973c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.693+0000 7f16cbaf8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41a9da0 con 0x55dca96bf680 2024-09-19T08:17:32.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.693+0000 7f16cbaf8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41ab210 con 0x55dca96b44a0 2024-09-19T08:17:32.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.693+0000 7f16cbaf8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41aa470 con 0x55dca96b4dc0 2024-09-19T08:17:32.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.693+0000 7f16cbaf8740 10 client.0 _ll_register_callbacks cb 0x7f16b4098700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:32.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.693+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96d7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.693+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96bf680 0x55dca973c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.693+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96d7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:50500/0 (socket says 172.21.15.22:50500) 2024-09-19T08:17:32.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.693+0000 7f16c27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96b4dc0 0x55dca96d7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.693+0000 7f16c2ffd640 1 -- 172.21.15.22:0/2863550936 learned_addr learned my addr 172.21.15.22:0/2863550936 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:32.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.694+0000 7f16c2ffd640 1 -- 172.21.15.22:0/2863550936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96b4dc0 msgr2=0x55dca96d7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:32.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.694+0000 7f16c2ffd640 1 --2- 172.21.15.22:0/2863550936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96b4dc0 0x55dca96d7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.694+0000 7f16c2ffd640 1 -- 172.21.15.22:0/2863550936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96bf680 msgr2=0x55dca973c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:32.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.694+0000 7f16c2ffd640 1 --2- 172.21.15.22:0/2863550936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96bf680 0x55dca973c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.694+0000 7f16c2ffd640 1 -- 172.21.15.22:0/2863550936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dca96bf2e0 con 0x55dca96b44a0 2024-09-19T08:17:32.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.694+0000 7f16c2ffd640 1 --2- 172.21.15.22:0/2863550936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96d7720 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f16b409c370 tx=0x7f16b40b0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ccfa7d956eb930c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:32.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.694+0000 7f16a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:17:32.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.694+0000 7f16a6ff5640 1 -- 172.21.15.22:0/2863550936 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b409bd40 con 0x55dca96b44a0 2024-09-19T08:17:32.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.694+0000 7f16a6ff5640 1 -- 172.21.15.22:0/2863550936 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f16b40a4bc0 con 0x55dca96b44a0 2024-09-19T08:17:32.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.694+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2863550936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16bc02c040 con 0x55dca96b44a0 2024-09-19T08:17:32.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.694+0000 7f16a6ff5640 1 -- 172.21.15.22:0/2863550936 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16b40cdcd0 con 0x55dca96b44a0 2024-09-19T08:17:32.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.694+0000 7f16cbaf8740 10 client.25559 fetch_fsmap learned FSMap version 27 2024-09-19T08:17:32.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.694+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2863550936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dca973d5e0 con 0x55dca96b44a0 2024-09-19T08:17:32.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.695+0000 7f16a6ff5640 1 -- 172.21.15.22:0/2863550936 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16bc02c040 con 0x55dca96b44a0 2024-09-19T08:17:32.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.695+0000 7f16a6ff5640 1 -- 172.21.15.22:0/2863550936 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f16b40d20b0 con 0x55dca96b44a0 2024-09-19T08:17:32.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.695+0000 7f16cbaf8740 10 client.25559 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:17:32.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.695+0000 7f16cbaf8740 10 client.25559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:17:32.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.695+0000 7f16cbaf8740 20 client.25559 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:32.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.695+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/2863550936 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55dca973dd00 0x55dca9740120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:32.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.695+0000 7f16cbaf8740 4 client.25559 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:17:32.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.695+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2863550936 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dca9621d60 con 0x55dca973dd00 2024-09-19T08:17:32.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.695+0000 7f16c27fc640 1 --2- 172.21.15.22:0/2863550936 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55dca973dd00 0x55dca9740120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:32.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.696+0000 7f16c27fc640 1 --2- 172.21.15.22:0/2863550936 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55dca973dd00 0x55dca9740120 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=960ef0eb859c8c0a server_cookie=c3c24e9384638a6d in_seq=0 out_seq=0 2024-09-19T08:17:32.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.696+0000 7f16a6ff5640 10 client.25559 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:32.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.696+0000 7f16a6ff5640 1 -- 172.21.15.22:0/2863550936 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dca9621d60 con 0x55dca973dd00 2024-09-19T08:17:32.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.696+0000 7f16a6ff5640 10 client.25559 handle_command_reply: tid=0 2024-09-19T08:17:32.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.696+0000 7f16cbaf8740 1 client.25559 shutdown 2024-09-19T08:17:32.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.696+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2863550936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 msgr2=0x55dca96d7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:32.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.696+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/2863550936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96d7720 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f16b409c370 tx=0x7f16b40b0040 comp rx=0 tx=0).stop 2024-09-19T08:17:32.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.696+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2863550936 shutdown_connections 2024-09-19T08:17:32.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.696+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/2863550936 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55dca973dd00 0x55dca9740120 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.696+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/2863550936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca96bf680 0x55dca973c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.696+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/2863550936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dca96b4dc0 0x55dca96d7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.696+0000 7f16cbaf8740 1 --2- 172.21.15.22:0/2863550936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dca96b44a0 0x55dca96d7720 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.696+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2863550936 >> 172.21.15.22:0/2863550936 conn(0x55dca96b0a30 msgr2=0x55dca96be2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:32.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.696+0000 7f16a6ff5640 0 client.25559 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:32.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.697+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2863550936 shutdown_connections 2024-09-19T08:17:32.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.697+0000 7f16cbaf8740 1 -- 172.21.15.22:0/2863550936 wait complete. 2024-09-19T08:17:32.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.697+0000 7f16cbaf8740 20 client.25559 trim_cache size 0 max 16384 2024-09-19T08:17:32.703 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:17:32.703 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-19T08:17:32.704 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-09-19T08:17:32.704 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:17:32.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.697+0000 7f16c971f640 1 -- 172.21.15.22:0/1029719645 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16ac0a33b0 msgr2=0x7f16ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:32.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.697+0000 7f16c971f640 1 --2- 172.21.15.22:0/1029719645 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16ac0a33b0 0x7f16ac0a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f16bc023000 tx=0x7f16bc024000 comp rx=0 tx=0).stop 2024-09-19T08:17:32.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.697+0000 7f16c971f640 1 -- 172.21.15.22:0/1029719645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c4102dd0 msgr2=0x7f16c4198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:32.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.697+0000 7f16c971f640 1 --2- 172.21.15.22:0/1029719645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c4102dd0 0x7f16c4198600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f16b400e9f0 tx=0x7f16b400edf0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.697+0000 7f16c971f640 1 -- 172.21.15.22:0/1029719645 shutdown_connections 2024-09-19T08:17:32.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.697+0000 7f16c971f640 1 --2- 172.21.15.22:0/1029719645 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16ac0a33b0 0x7f16ac0a5870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.697+0000 7f16c971f640 1 --2- 172.21.15.22:0/1029719645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4108300 0x7f16c419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.697+0000 7f16c971f640 1 --2- 172.21.15.22:0/1029719645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16c4103780 0x7f16c4198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.697+0000 7f16c971f640 1 --2- 172.21.15.22:0/1029719645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c4102dd0 0x7f16c4198600 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:32.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.697+0000 7f16c971f640 1 -- 172.21.15.22:0/1029719645 >> 172.21.15.22:0/1029719645 conn(0x7f16c40fc7d0 msgr2=0x7f16c4105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:32.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.698+0000 7f16c971f640 1 -- 172.21.15.22:0/1029719645 shutdown_connections 2024-09-19T08:17:32.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:32.698+0000 7f16c971f640 1 -- 172.21.15.22:0/1029719645 wait complete. 2024-09-19T08:17:33.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:32 smithi157 ceph-mon[35143]: pgmap v78: 97 pgs: 97 active+clean; 68 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 228 op/s 2024-09-19T08:17:33.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:32 smithi022 ceph-mon[30963]: pgmap v78: 97 pgs: 97 active+clean; 68 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 228 op/s 2024-09-19T08:17:33.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:32 smithi064 ceph-mon[34900]: pgmap v78: 97 pgs: 97 active+clean; 68 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 228 op/s 2024-09-19T08:17:33.544 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:17:33.544 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:17:33.868 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:17:34.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.444+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1059390623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa924102d40 msgr2=0x7fa924103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:34.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.444+0000 7fa92b9f7640 1 --2- 172.21.15.22:0/1059390623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa924102d40 0x7fa924103120 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa920009900 tx=0x7fa92002a480 comp rx=0 tx=0).stop 2024-09-19T08:17:34.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.444+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1059390623 shutdown_connections 2024-09-19T08:17:34.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.444+0000 7fa92b9f7640 1 --2- 172.21.15.22:0/1059390623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa92410c170 0x7fa92410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.444+0000 7fa92b9f7640 1 --2- 172.21.15.22:0/1059390623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa924103660 0x7fa92410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.444+0000 7fa92b9f7640 1 --2- 172.21.15.22:0/1059390623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa924102d40 0x7fa924103120 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.444+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1059390623 >> 172.21.15.22:0/1059390623 conn(0x7fa9240fc7a0 msgr2=0x7fa9240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.444+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1059390623 shutdown_connections 2024-09-19T08:17:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.444+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1059390623 wait complete. 2024-09-19T08:17:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa92b9f7640 1 Processor -- start 2024-09-19T08:17:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa92b9f7640 1 -- start start 2024-09-19T08:17:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa92b9f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa924102d40 0x7fa924198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa92b9f7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa924103660 0x7fa924198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa92b9f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa92410c170 0x7fa92419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa92b9f7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa924111fc0 con 0x7fa924102d40 2024-09-19T08:17:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa92b9f7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa924111e40 con 0x7fa92410c170 2024-09-19T08:17:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa92b9f7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa924112140 con 0x7fa924103660 2024-09-19T08:17:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa929793640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa924102d40 0x7fa924198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa929793640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa924102d40 0x7fa924198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53964/0 (socket says 172.21.15.22:53964) 2024-09-19T08:17:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa929793640 1 -- 172.21.15.22:0/1001283065 learned_addr learned my addr 172.21.15.22:0/1001283065 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa928f92640 1 --2- 172.21.15.22:0/1001283065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa924103660 0x7fa924198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa929f94640 1 --2- 172.21.15.22:0/1001283065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa92410c170 0x7fa92419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa929793640 1 -- 172.21.15.22:0/1001283065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa924103660 msgr2=0x7fa924198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa929793640 1 --2- 172.21.15.22:0/1001283065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa924103660 0x7fa924198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa929793640 1 -- 172.21.15.22:0/1001283065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa92410c170 msgr2=0x7fa92419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa929793640 1 --2- 172.21.15.22:0/1001283065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa92410c170 0x7fa92419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.445+0000 7fa929793640 1 -- 172.21.15.22:0/1001283065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92419d3f0 con 0x7fa924102d40 2024-09-19T08:17:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.446+0000 7fa929793640 1 --2- 172.21.15.22:0/1001283065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa924102d40 0x7fa924198440 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa920004770 tx=0x7fa920003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.446+0000 7fa91a7fc640 1 -- 172.21.15.22:0/1001283065 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92003f070 con 0x7fa924102d40 2024-09-19T08:17:34.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.446+0000 7fa91a7fc640 1 -- 172.21.15.22:0/1001283065 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa920031ca0 con 0x7fa924102d40 2024-09-19T08:17:34.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.446+0000 7fa91a7fc640 1 -- 172.21.15.22:0/1001283065 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa920035030 con 0x7fa924102d40 2024-09-19T08:17:34.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.446+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1001283065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92419d680 con 0x7fa924102d40 2024-09-19T08:17:34.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.446+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1001283065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9241a4fc0 con 0x7fa924102d40 2024-09-19T08:17:34.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.447+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1001283065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa92406b070 con 0x7fa924102d40 2024-09-19T08:17:34.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.448+0000 7fa91a7fc640 1 -- 172.21.15.22:0/1001283065 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa920042070 con 0x7fa924102d40 2024-09-19T08:17:34.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.449+0000 7fa91a7fc640 1 --2- 172.21.15.22:0/1001283065 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa9100a33b0 0x7fa9100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:34.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.449+0000 7fa928f92640 1 --2- 172.21.15.22:0/1001283065 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa9100a33b0 0x7fa9100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:34.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.449+0000 7fa91a7fc640 1 -- 172.21.15.22:0/1001283065 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa9200f08b0 con 0x7fa924102d40 2024-09-19T08:17:34.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.449+0000 7fa928f92640 1 --2- 172.21.15.22:0/1001283065 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa9100a33b0 0x7fa9100a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa92410d540 tx=0x7fa914005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:34.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.452+0000 7fa91a7fc640 1 -- 172.21.15.22:0/1001283065 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa9200bcd10 con 0x7fa924102d40 2024-09-19T08:17:34.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.727+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1001283065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9241a5190 con 0x7fa924102d40 2024-09-19T08:17:34.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.728+0000 7fa91a7fc640 1 -- 172.21.15.22:0/1001283065 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11908 (secure 0 0 0) 0x7fa9200c1010 con 0x7fa924102d40 2024-09-19T08:17:34.728 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:17:34.728 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":27,"btime":"2024-09-19T08:17:21:880541+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:17:21.773867+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:17:34.729 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 27 2024-09-19T08:17:34.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.730+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1001283065 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa9100a33b0 msgr2=0x7fa9100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:34.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.730+0000 7fa92b9f7640 1 --2- 172.21.15.22:0/1001283065 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa9100a33b0 0x7fa9100a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa92410d540 tx=0x7fa914005f90 comp rx=0 tx=0).stop 2024-09-19T08:17:34.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.730+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1001283065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa924102d40 msgr2=0x7fa924198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:34.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.730+0000 7fa92b9f7640 1 --2- 172.21.15.22:0/1001283065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa924102d40 0x7fa924198440 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa920004770 tx=0x7fa920003c10 comp rx=0 tx=0).stop 2024-09-19T08:17:34.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.730+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1001283065 shutdown_connections 2024-09-19T08:17:34.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.730+0000 7fa92b9f7640 1 --2- 172.21.15.22:0/1001283065 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa9100a33b0 0x7fa9100a5870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:34.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.730+0000 7fa92b9f7640 1 --2- 172.21.15.22:0/1001283065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa92410c170 0x7fa92419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:34.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.730+0000 7fa92b9f7640 1 --2- 172.21.15.22:0/1001283065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa924103660 0x7fa924198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:34.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.730+0000 7fa92b9f7640 1 --2- 172.21.15.22:0/1001283065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa924102d40 0x7fa924198440 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:34.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.730+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1001283065 >> 172.21.15.22:0/1001283065 conn(0x7fa9240fc7a0 msgr2=0x7fa9240fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:34.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.730+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1001283065 shutdown_connections 2024-09-19T08:17:34.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:34.731+0000 7fa92b9f7640 1 -- 172.21.15.22:0/1001283065 wait complete. 2024-09-19T08:17:35.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:34 smithi022 ceph-mon[30963]: pgmap v79: 97 pgs: 97 active+clean; 77 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 276 op/s 2024-09-19T08:17:35.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:34 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1001283065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:17:35.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:34 smithi157 ceph-mon[35143]: pgmap v79: 97 pgs: 97 active+clean; 77 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 276 op/s 2024-09-19T08:17:35.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:34 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1001283065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:17:35.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:34 smithi064 ceph-mon[34900]: pgmap v79: 97 pgs: 97 active+clean; 77 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 276 op/s 2024-09-19T08:17:35.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:34 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1001283065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:17:35.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:17:36.154 INFO:tasks.workunit.client.0.smithi022.stdout: 37 681458 1113 380372 602 204134 1695 2024-09-19T08:17:36.158 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:17:36.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.688+0000 7f0f8e582640 1 -- 172.21.15.22:0/2070864819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f8810c170 msgr2=0x7f0f8810e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:36.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.688+0000 7f0f8e582640 1 --2- 172.21.15.22:0/2070864819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f8810c170 0x7f0f8810e5f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009ca0 tx=0x7f0f8002a950 comp rx=0 tx=0).stop 2024-09-19T08:17:36.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.688+0000 7f0f8e582640 1 -- 172.21.15.22:0/2070864819 shutdown_connections 2024-09-19T08:17:36.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.688+0000 7f0f8e582640 1 --2- 172.21.15.22:0/2070864819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f8810c170 0x7f0f8810e5f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.688+0000 7f0f8e582640 1 --2- 172.21.15.22:0/2070864819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f881036b0 0x7f0f8810bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.688+0000 7f0f8e582640 1 --2- 172.21.15.22:0/2070864819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f88102d90 0x7f0f88103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.688+0000 7f0f8e582640 1 -- 172.21.15.22:0/2070864819 >> 172.21.15.22:0/2070864819 conn(0x7f0f880fc790 msgr2=0x7f0f880febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:36.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.688+0000 7f0f8e582640 1 -- 172.21.15.22:0/2070864819 shutdown_connections 2024-09-19T08:17:36.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.688+0000 7f0f8e582640 1 -- 172.21.15.22:0/2070864819 wait complete. 2024-09-19T08:17:36.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.689+0000 7f0f8e582640 1 Processor -- start 2024-09-19T08:17:36.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.689+0000 7f0f8e582640 1 -- start start 2024-09-19T08:17:36.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.689+0000 7f0f8e582640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f88102d90 0x7f0f881983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.689+0000 7f0f8e582640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f881036b0 0x7f0f88198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.689+0000 7f0f8e582640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f8810c170 0x7f0f8819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f8e582640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0f88111f30 con 0x7f0f8810c170 2024-09-19T08:17:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f8e582640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0f88111db0 con 0x7f0f88102d90 2024-09-19T08:17:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f8e582640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881120b0 con 0x7f0f881036b0 2024-09-19T08:17:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f8cb1f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f8810c170 0x7f0f8819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f877fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f881036b0 0x7f0f88198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f87fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f88102d90 0x7f0f881983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f8cb1f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f8810c170 0x7f0f8819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53982/0 (socket says 172.21.15.22:53982) 2024-09-19T08:17:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f8cb1f640 1 -- 172.21.15.22:0/1011884896 learned_addr learned my addr 172.21.15.22:0/1011884896 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:36.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f877fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f881036b0 0x7f0f88198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:57016/0 (socket says 172.21.15.22:57016) 2024-09-19T08:17:36.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f8cb1f640 1 -- 172.21.15.22:0/1011884896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f881036b0 msgr2=0x7f0f88198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:36.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f8cb1f640 1 --2- 172.21.15.22:0/1011884896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f881036b0 0x7f0f88198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f8cb1f640 1 -- 172.21.15.22:0/1011884896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f88102d90 msgr2=0x7f0f881983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:36.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f8cb1f640 1 --2- 172.21.15.22:0/1011884896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f88102d90 0x7f0f881983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f8cb1f640 1 -- 172.21.15.22:0/1011884896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8819d380 con 0x7f0f8810c170 2024-09-19T08:17:36.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f877fe640 1 --2- 172.21.15.22:0/1011884896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f881036b0 0x7f0f88198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:36.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f87fff640 1 --2- 172.21.15.22:0/1011884896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f88102d90 0x7f0f881983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:36.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f8cb1f640 1 --2- 172.21.15.22:0/1011884896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f8810c170 0x7f0f8819cca0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0f80004af0 tx=0x7f0f8002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:36.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.690+0000 7f0f857fa640 1 -- 172.21.15.22:0/1011884896 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80034040 con 0x7f0f8810c170 2024-09-19T08:17:36.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.691+0000 7f0f8e582640 1 -- 172.21.15.22:0/1011884896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8819d610 con 0x7f0f8810c170 2024-09-19T08:17:36.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.691+0000 7f0f857fa640 1 -- 172.21.15.22:0/1011884896 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f80030920 con 0x7f0f8810c170 2024-09-19T08:17:36.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.691+0000 7f0f857fa640 1 -- 172.21.15.22:0/1011884896 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80004670 con 0x7f0f8810c170 2024-09-19T08:17:36.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.691+0000 7f0f8e582640 1 -- 172.21.15.22:0/1011884896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f88100920 con 0x7f0f8810c170 2024-09-19T08:17:36.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.693+0000 7f0f857fa640 1 -- 172.21.15.22:0/1011884896 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0f80037020 con 0x7f0f8810c170 2024-09-19T08:17:36.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.693+0000 7f0f9095b740 1 Processor -- start 2024-09-19T08:17:36.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f9095b740 1 -- start start 2024-09-19T08:17:36.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f9095b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c3b40 0x5597ef3c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f9095b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c25d0 0x5597ef3c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f9095b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 0x5597ef3cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f9095b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5597ef3282f0 con 0x5597ef3c25d0 2024-09-19T08:17:36.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f9095b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5597ef2cfcf0 con 0x5597ef3c3b40 2024-09-19T08:17:36.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f9095b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5597ef3242b0 con 0x5597ef3cd370 2024-09-19T08:17:36.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f87fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c25d0 0x5597ef3c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f857fa640 1 --2- 172.21.15.22:0/1011884896 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0f700a33b0 0x7f0f700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f87fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c25d0 0x5597ef3c2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53988/0 (socket says 172.21.15.22:53988) 2024-09-19T08:17:36.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f87fff640 1 -- 172.21.15.22:0/2824466635 learned_addr learned my addr 172.21.15.22:0/2824466635 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:36.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f877fe640 1 --2- 172.21.15.22:0/2824466635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 0x5597ef3cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f857fa640 1 -- 172.21.15.22:0/1011884896 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0f800f08f0 con 0x7f0f8810c170 2024-09-19T08:17:36.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f87fff640 1 -- 172.21.15.22:0/2824466635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 msgr2=0x5597ef3cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:36.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f8cb1f640 1 --2- 172.21.15.22:0/1011884896 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0f700a33b0 0x7f0f700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f87fff640 1 --2- 172.21.15.22:0/2824466635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 0x5597ef3cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f87fff640 1 -- 172.21.15.22:0/2824466635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c3b40 msgr2=0x5597ef3c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:36.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f87fff640 1 --2- 172.21.15.22:0/2824466635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c3b40 0x5597ef3c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.694+0000 7f0f87fff640 1 -- 172.21.15.22:0/2824466635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ef2b43a0 con 0x5597ef3c25d0 2024-09-19T08:17:36.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.695+0000 7f0f87fff640 1 --2- 172.21.15.22:0/2824466635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c25d0 0x5597ef3c2a10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009c00 tx=0x7f0f78005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f6b0df57d1e49f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:36.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.695+0000 7f0f8cb1f640 1 --2- 172.21.15.22:0/1011884896 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0f700a33b0 0x7f0f700a5870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0f800ee210 tx=0x7f0f800095c0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:36.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.695+0000 7f0f6e7fc640 1 -- 172.21.15.22:0/2824466635 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78022020 con 0x5597ef3c25d0 2024-09-19T08:17:36.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.695+0000 7f0f6e7fc640 1 -- 172.21.15.22:0/2824466635 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f7800bc80 con 0x5597ef3c25d0 2024-09-19T08:17:36.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.695+0000 7f0f6e7fc640 1 -- 172.21.15.22:0/2824466635 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7801d690 con 0x5597ef3c25d0 2024-09-19T08:17:36.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.696+0000 7f0f9095b740 1 -- 172.21.15.22:0/2824466635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c25d0 msgr2=0x5597ef3c2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:36.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.696+0000 7f0f9095b740 1 --2- 172.21.15.22:0/2824466635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c25d0 0x5597ef3c2a10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009c00 tx=0x7f0f78005f90 comp rx=0 tx=0).stop 2024-09-19T08:17:36.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.696+0000 7f0f9095b740 1 -- 172.21.15.22:0/2824466635 shutdown_connections 2024-09-19T08:17:36.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.696+0000 7f0f9095b740 1 --2- 172.21.15.22:0/2824466635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 0x5597ef3cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.696+0000 7f0f9095b740 1 --2- 172.21.15.22:0/2824466635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c25d0 0x5597ef3c2a10 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.696+0000 7f0f9095b740 1 --2- 172.21.15.22:0/2824466635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c3b40 0x5597ef3c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.696+0000 7f0f9095b740 1 -- 172.21.15.22:0/2824466635 >> 172.21.15.22:0/2824466635 conn(0x5597ef38b310 msgr2=0x5597ef3bec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:36.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.696+0000 7f0f9095b740 1 -- 172.21.15.22:0/2824466635 shutdown_connections 2024-09-19T08:17:36.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.696+0000 7f0f9095b740 1 -- 172.21.15.22:0/2824466635 wait complete. 2024-09-19T08:17:36.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.696+0000 7f0f9095b740 1 -- start start 2024-09-19T08:17:36.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.696+0000 7f0f9095b740 1 Processor -- start 2024-09-19T08:17:36.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f9095b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c25d0 0x5597ef3ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f9095b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f9095b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 0x5597ef3ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f9095b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5597ef3d0070 con 0x5597ef3c3b40 2024-09-19T08:17:36.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f9095b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5597ef2e3060 con 0x5597ef3c25d0 2024-09-19T08:17:36.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f9095b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5597ef3d01f0 con 0x5597ef3cd370 2024-09-19T08:17:36.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f9095b740 10 client.0 _ll_register_callbacks cb 0x5597ef0368c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:36.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f87fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f87fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53990/0 (socket says 172.21.15.22:53990) 2024-09-19T08:17:36.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f87fff640 1 -- 172.21.15.22:0/3936454322 learned_addr learned my addr 172.21.15.22:0/3936454322 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:36.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f877fe640 1 --2- 172.21.15.22:0/3936454322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 0x5597ef3ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f877fe640 1 --2- 172.21.15.22:0/3936454322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c25d0 0x5597ef3ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f87fff640 1 -- 172.21.15.22:0/3936454322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 msgr2=0x5597ef3ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:36.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f87fff640 1 --2- 172.21.15.22:0/3936454322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 0x5597ef3ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f87fff640 1 -- 172.21.15.22:0/3936454322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c25d0 msgr2=0x5597ef3ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:36.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f87fff640 1 --2- 172.21.15.22:0/3936454322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c25d0 0x5597ef3ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f87fff640 1 -- 172.21.15.22:0/3936454322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ef0993c0 con 0x5597ef3c3b40 2024-09-19T08:17:36.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.697+0000 7f0f877fe640 1 --2- 172.21.15.22:0/3936454322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c25d0 0x5597ef3ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:17:36.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.698+0000 7f0f87fff640 1 --2- 172.21.15.22:0/3936454322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3c9f90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0f7801eb90 tx=0x7f0f780056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1040c503578a0fd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:36.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.698+0000 7f0f677fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:17:36.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.698+0000 7f0f677fe640 1 -- 172.21.15.22:0/3936454322 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78022050 con 0x5597ef3c3b40 2024-09-19T08:17:36.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.698+0000 7f0f677fe640 1 -- 172.21.15.22:0/3936454322 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0f78004240 con 0x5597ef3c3b40 2024-09-19T08:17:36.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.698+0000 7f0f9095b740 1 -- 172.21.15.22:0/3936454322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f7800bc80 con 0x5597ef3c3b40 2024-09-19T08:17:36.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.698+0000 7f0f677fe640 1 -- 172.21.15.22:0/3936454322 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f78004440 con 0x5597ef3c3b40 2024-09-19T08:17:36.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.698+0000 7f0f9095b740 10 client.15945 fetch_fsmap learned FSMap version 27 2024-09-19T08:17:36.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.698+0000 7f0f9095b740 1 -- 172.21.15.22:0/3936454322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597ef3dba80 con 0x5597ef3c3b40 2024-09-19T08:17:36.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.698+0000 7f0f677fe640 1 -- 172.21.15.22:0/3936454322 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f7800bc80 con 0x5597ef3c3b40 2024-09-19T08:17:36.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.698+0000 7f0f677fe640 1 -- 172.21.15.22:0/3936454322 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0f7802f020 con 0x5597ef3c3b40 2024-09-19T08:17:36.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.698+0000 7f0f9095b740 10 client.15945 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:17:36.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.698+0000 7f0f9095b740 10 client.15945 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:17:36.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.698+0000 7f0f9095b740 20 client.15945 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:36.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.699+0000 7f0f9095b740 1 --2- 172.21.15.22:0/3936454322 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5597ef3dce80 0x5597ef3df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.699+0000 7f0f9095b740 4 client.15945 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:17:36.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.699+0000 7f0f9095b740 1 -- 172.21.15.22:0/3936454322 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597ef335860 con 0x5597ef3dce80 2024-09-19T08:17:36.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.699+0000 7f0f877fe640 1 --2- 172.21.15.22:0/3936454322 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5597ef3dce80 0x5597ef3df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.699+0000 7f0f677fe640 10 client.15945 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:17:36.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.699+0000 7f0f877fe640 1 --2- 172.21.15.22:0/3936454322 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5597ef3dce80 0x5597ef3df270 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b9c3334a95cf51 server_cookie=c041755d11fdd994 in_seq=0 out_seq=0 2024-09-19T08:17:36.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.701+0000 7f0f677fe640 1 -- 172.21.15.22:0/3936454322 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597ef335860 con 0x5597ef3dce80 2024-09-19T08:17:36.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.701+0000 7f0f677fe640 10 client.15945 handle_command_reply: tid=0 2024-09-19T08:17:36.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.701+0000 7f0f9095b740 1 client.15945 shutdown 2024-09-19T08:17:36.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.702+0000 7f0f9095b740 1 -- 172.21.15.22:0/3936454322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 msgr2=0x5597ef3c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:36.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.702+0000 7f0f9095b740 1 --2- 172.21.15.22:0/3936454322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3c9f90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0f7801eb90 tx=0x7f0f780056d0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.702+0000 7f0f9095b740 1 -- 172.21.15.22:0/3936454322 shutdown_connections 2024-09-19T08:17:36.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.702+0000 7f0f9095b740 1 --2- 172.21.15.22:0/3936454322 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5597ef3dce80 0x5597ef3df270 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.702+0000 7f0f677fe640 0 client.15945 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:17:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.702+0000 7f0f9095b740 1 --2- 172.21.15.22:0/3936454322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 0x5597ef3ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.702+0000 7f0f9095b740 1 --2- 172.21.15.22:0/3936454322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3c9f90 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.702+0000 7f0f9095b740 1 --2- 172.21.15.22:0/3936454322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c25d0 0x5597ef3ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.702+0000 7f0f9095b740 1 -- 172.21.15.22:0/3936454322 >> 172.21.15.22:0/3936454322 conn(0x5597ef38b310 msgr2=0x5597ef3c4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.702+0000 7f0f9095b740 1 -- 172.21.15.22:0/3936454322 shutdown_connections 2024-09-19T08:17:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.702+0000 7f0f9095b740 1 -- 172.21.15.22:0/3936454322 wait complete. 2024-09-19T08:17:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.702+0000 7f0f9095b740 20 client.15945 trim_cache size 0 max 16384 2024-09-19T08:17:36.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.718+0000 7f0f9095b740 1 Processor -- start 2024-09-19T08:17:36.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.718+0000 7f0f9095b740 1 -- start start 2024-09-19T08:17:36.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.718+0000 7f0f9095b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3c25d0 0x5597ef3d9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.718+0000 7f0f9095b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.718+0000 7f0f9095b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3cd370 0x5597ef3ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f9095b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881adf20 con 0x5597ef3c3b40 2024-09-19T08:17:36.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f9095b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881ab7c0 con 0x5597ef3cd370 2024-09-19T08:17:36.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f9095b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881a9610 con 0x5597ef3c25d0 2024-09-19T08:17:36.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f877fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3c25d0 0x5597ef3d9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f87fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f877fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3c25d0 0x5597ef3d9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:57038/0 (socket says 172.21.15.22:57038) 2024-09-19T08:17:36.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f87fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3c1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53994/0 (socket says 172.21.15.22:53994) 2024-09-19T08:17:36.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f877fe640 1 -- 172.21.15.22:0/4228787292 learned_addr learned my addr 172.21.15.22:0/4228787292 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:36.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f877fe640 1 --2- 172.21.15.22:0/4228787292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3cd370 0x5597ef3ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f877fe640 1 -- 172.21.15.22:0/4228787292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3cd370 msgr2=0x5597ef3ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:36.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f877fe640 1 --2- 172.21.15.22:0/4228787292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3cd370 0x5597ef3ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f877fe640 1 -- 172.21.15.22:0/4228787292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 msgr2=0x5597ef3c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f877fe640 1 --2- 172.21.15.22:0/4228787292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f877fe640 1 -- 172.21.15.22:0/4228787292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ef3be510 con 0x5597ef3c25d0 2024-09-19T08:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f87fff640 1 --2- 172.21.15.22:0/4228787292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.719+0000 7f0f877fe640 1 --2- 172.21.15.22:0/4228787292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3c25d0 0x5597ef3d9310 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0f880fed20 tx=0x7f0f7c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc96879e8aec1df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.720+0000 7f0f6dffb640 1 -- 172.21.15.22:0/4228787292 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c01a040 con 0x5597ef3c25d0 2024-09-19T08:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.720+0000 7f0f6dffb640 1 -- 172.21.15.22:0/4228787292 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f7c025040 con 0x5597ef3c25d0 2024-09-19T08:17:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.720+0000 7f0f6dffb640 1 -- 172.21.15.22:0/4228787292 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c02e480 con 0x5597ef3c25d0 2024-09-19T08:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.720+0000 7f0f9095b740 1 -- 172.21.15.22:0/4228787292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3c25d0 msgr2=0x5597ef3d9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.720+0000 7f0f9095b740 1 --2- 172.21.15.22:0/4228787292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3c25d0 0x5597ef3d9310 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0f880fed20 tx=0x7f0f7c0046c0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.720+0000 7f0f9095b740 1 -- 172.21.15.22:0/4228787292 shutdown_connections 2024-09-19T08:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.720+0000 7f0f9095b740 1 --2- 172.21.15.22:0/4228787292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3cd370 0x5597ef3ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.720+0000 7f0f9095b740 1 --2- 172.21.15.22:0/4228787292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.720+0000 7f0f9095b740 1 --2- 172.21.15.22:0/4228787292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3c25d0 0x5597ef3d9310 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.720+0000 7f0f9095b740 1 -- 172.21.15.22:0/4228787292 >> 172.21.15.22:0/4228787292 conn(0x5597ef38b310 msgr2=0x5597ef3c9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.721+0000 7f0f9095b740 1 -- 172.21.15.22:0/4228787292 shutdown_connections 2024-09-19T08:17:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.721+0000 7f0f9095b740 1 -- 172.21.15.22:0/4228787292 wait complete. 2024-09-19T08:17:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.721+0000 7f0f9095b740 1 -- start start 2024-09-19T08:17:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.721+0000 7f0f9095b740 1 Processor -- start 2024-09-19T08:17:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.722+0000 7f0f9095b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c25d0 0x5597ef3d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.722+0000 7f0f9095b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3d5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.722+0000 7f0f9095b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 0x5597ef3c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.722+0000 7f0f9095b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881a9c80 con 0x5597ef3c3b40 2024-09-19T08:17:36.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.722+0000 7f0f9095b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881ab0f0 con 0x5597ef3c25d0 2024-09-19T08:17:36.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.722+0000 7f0f9095b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881aa350 con 0x5597ef3cd370 2024-09-19T08:17:36.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.722+0000 7f0f9095b740 10 client.0 _ll_register_callbacks cb 0x7f0f78009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:36.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.722+0000 7f0f87fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3d5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.722+0000 7f0f87fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3d5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54004/0 (socket says 172.21.15.22:54004) 2024-09-19T08:17:36.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.722+0000 7f0f877fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 0x5597ef3c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.722+0000 7f0f87fff640 1 -- 172.21.15.22:0/4196748939 learned_addr learned my addr 172.21.15.22:0/4196748939 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.722+0000 7f0f877fe640 1 --2- 172.21.15.22:0/4196748939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c25d0 0x5597ef3d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.723+0000 7f0f877fe640 1 -- 172.21.15.22:0/4196748939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c25d0 msgr2=0x5597ef3d5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.723+0000 7f0f877fe640 1 --2- 172.21.15.22:0/4196748939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c25d0 0x5597ef3d5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.723+0000 7f0f877fe640 1 -- 172.21.15.22:0/4196748939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 msgr2=0x5597ef3d5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.723+0000 7f0f877fe640 1 --2- 172.21.15.22:0/4196748939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3d5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.723+0000 7f0f877fe640 1 -- 172.21.15.22:0/4196748939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ef3c7920 con 0x5597ef3cd370 2024-09-19T08:17:36.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.723+0000 7f0f877fe640 1 --2- 172.21.15.22:0/4196748939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 0x5597ef3c7210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0059f0 tx=0x7f0f7c003be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54a7438e8fecd01f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:36.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.723+0000 7f0f677fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:17:36.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.723+0000 7f0f677fe640 1 -- 172.21.15.22:0/4196748939 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0039e0 con 0x5597ef3cd370 2024-09-19T08:17:36.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.723+0000 7f0f677fe640 1 -- 172.21.15.22:0/4196748939 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0f7c011030 con 0x5597ef3cd370 2024-09-19T08:17:36.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.723+0000 7f0f9095b740 1 -- 172.21.15.22:0/4196748939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f7c025040 con 0x5597ef3cd370 2024-09-19T08:17:36.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.723+0000 7f0f677fe640 1 -- 172.21.15.22:0/4196748939 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f7c011230 con 0x5597ef3cd370 2024-09-19T08:17:36.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.723+0000 7f0f9095b740 10 client.25513 fetch_fsmap learned FSMap version 27 2024-09-19T08:17:36.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.723+0000 7f0f9095b740 1 -- 172.21.15.22:0/4196748939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597ef3e47b0 con 0x5597ef3cd370 2024-09-19T08:17:36.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.724+0000 7f0f677fe640 1 -- 172.21.15.22:0/4196748939 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f7c025040 con 0x5597ef3cd370 2024-09-19T08:17:36.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.725+0000 7f0f677fe640 1 -- 172.21.15.22:0/4196748939 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0f7c02e5c0 con 0x5597ef3cd370 2024-09-19T08:17:36.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.725+0000 7f0f9095b740 10 client.25513 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:17:36.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.725+0000 7f0f9095b740 10 client.25513 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:17:36.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.725+0000 7f0f9095b740 20 client.25513 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:36.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.725+0000 7f0f9095b740 1 --2- 172.21.15.22:0/4196748939 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5597ef3dce80 0x5597ef449b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:36.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.725+0000 7f0f9095b740 4 client.25513 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:17:36.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.725+0000 7f0f9095b740 1 -- 172.21.15.22:0/4196748939 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597ef32fd70 con 0x5597ef3dce80 2024-09-19T08:17:36.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.725+0000 7f0f87fff640 1 --2- 172.21.15.22:0/4196748939 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5597ef3dce80 0x5597ef449b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:36.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.725+0000 7f0f87fff640 1 --2- 172.21.15.22:0/4196748939 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5597ef3dce80 0x5597ef449b00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af5b16064c9f44a server_cookie=ecad51491ab59f02 in_seq=0 out_seq=0 2024-09-19T08:17:36.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.726+0000 7f0f677fe640 10 client.25513 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:17:36.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.730+0000 7f0f677fe640 1 -- 172.21.15.22:0/4196748939 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597ef32fd70 con 0x5597ef3dce80 2024-09-19T08:17:36.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.730+0000 7f0f677fe640 10 client.25513 handle_command_reply: tid=0 2024-09-19T08:17:36.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.730+0000 7f0f9095b740 1 client.25513 shutdown 2024-09-19T08:17:36.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.731+0000 7f0f9095b740 1 -- 172.21.15.22:0/4196748939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 msgr2=0x5597ef3c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:36.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.731+0000 7f0f9095b740 1 --2- 172.21.15.22:0/4196748939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 0x5597ef3c7210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0059f0 tx=0x7f0f7c003be0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.731+0000 7f0f9095b740 1 -- 172.21.15.22:0/4196748939 shutdown_connections 2024-09-19T08:17:36.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.731+0000 7f0f9095b740 1 --2- 172.21.15.22:0/4196748939 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5597ef3dce80 0x5597ef449b00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.731+0000 7f0f9095b740 1 --2- 172.21.15.22:0/4196748939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5597ef3cd370 0x5597ef3c7210 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.731+0000 7f0f9095b740 1 --2- 172.21.15.22:0/4196748939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597ef3c3b40 0x5597ef3d5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.731+0000 7f0f9095b740 1 --2- 172.21.15.22:0/4196748939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597ef3c25d0 0x5597ef3d5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.731+0000 7f0f9095b740 1 -- 172.21.15.22:0/4196748939 >> 172.21.15.22:0/4196748939 conn(0x5597ef38b310 msgr2=0x5597ef3bd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:36.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.731+0000 7f0f677fe640 0 client.25513 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:17:36.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.731+0000 7f0f9095b740 1 -- 172.21.15.22:0/4196748939 shutdown_connections 2024-09-19T08:17:36.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.731+0000 7f0f9095b740 1 -- 172.21.15.22:0/4196748939 wait complete. 2024-09-19T08:17:36.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.731+0000 7f0f9095b740 20 client.25513 trim_cache size 0 max 16384 2024-09-19T08:17:36.741 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:17:36.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.732+0000 7f0f8e582640 1 -- 172.21.15.22:0/1011884896 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0f700a33b0 msgr2=0x7f0f700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:36.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.732+0000 7f0f8e582640 1 --2- 172.21.15.22:0/1011884896 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0f700a33b0 0x7f0f700a5870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0f800ee210 tx=0x7f0f800095c0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.732+0000 7f0f8e582640 1 -- 172.21.15.22:0/1011884896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f8810c170 msgr2=0x7f0f8819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:36.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.732+0000 7f0f8e582640 1 --2- 172.21.15.22:0/1011884896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f8810c170 0x7f0f8819cca0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0f80004af0 tx=0x7f0f8002f3a0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.732+0000 7f0f8e582640 1 -- 172.21.15.22:0/1011884896 shutdown_connections 2024-09-19T08:17:36.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.732+0000 7f0f8e582640 1 --2- 172.21.15.22:0/1011884896 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0f700a33b0 0x7f0f700a5870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.732+0000 7f0f8e582640 1 --2- 172.21.15.22:0/1011884896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f8810c170 0x7f0f8819cca0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.732+0000 7f0f8e582640 1 --2- 172.21.15.22:0/1011884896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f881036b0 0x7f0f88198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.732+0000 7f0f8e582640 1 --2- 172.21.15.22:0/1011884896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f88102d90 0x7f0f881983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:36.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.732+0000 7f0f8e582640 1 -- 172.21.15.22:0/1011884896 >> 172.21.15.22:0/1011884896 conn(0x7f0f880fc790 msgr2=0x7f0f880fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:36.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.732+0000 7f0f8e582640 1 -- 172.21.15.22:0/1011884896 shutdown_connections 2024-09-19T08:17:36.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:36.732+0000 7f0f8e582640 1 -- 172.21.15.22:0/1011884896 wait complete. 2024-09-19T08:17:37.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:36 smithi157 ceph-mon[35143]: pgmap v80: 97 pgs: 97 active+clean; 77 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 94 op/s 2024-09-19T08:17:37.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:36 smithi022 ceph-mon[30963]: pgmap v80: 97 pgs: 97 active+clean; 77 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 94 op/s 2024-09-19T08:17:37.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:36 smithi064 ceph-mon[34900]: pgmap v80: 97 pgs: 97 active+clean; 77 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 94 op/s 2024-09-19T08:17:38.125 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:17:38.535 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:17:38.966 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:38 smithi022 ceph-mon[30963]: pgmap v81: 97 pgs: 97 active+clean; 85 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 145 op/s 2024-09-19T08:17:39.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.091+0000 7f865bfff640 1 -- 172.21.15.22:0/4464571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f865c1019c0 msgr2=0x7f865c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:39.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.091+0000 7f865bfff640 1 --2- 172.21.15.22:0/4464571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f865c1019c0 0x7f865c101e20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8650009930 tx=0x7f865002a420 comp rx=0 tx=0).stop 2024-09-19T08:17:39.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.091+0000 7f865bfff640 1 -- 172.21.15.22:0/4464571 shutdown_connections 2024-09-19T08:17:39.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.091+0000 7f865bfff640 1 --2- 172.21.15.22:0/4464571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865c102360 0x7f865c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.091+0000 7f865bfff640 1 --2- 172.21.15.22:0/4464571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f865c1019c0 0x7f865c101e20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.091+0000 7f865bfff640 1 --2- 172.21.15.22:0/4464571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c1079c0 0x7f865c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.091+0000 7f865bfff640 1 -- 172.21.15.22:0/4464571 >> 172.21.15.22:0/4464571 conn(0x7f865c0fd720 msgr2=0x7f865c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:39.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.091+0000 7f865bfff640 1 -- 172.21.15.22:0/4464571 shutdown_connections 2024-09-19T08:17:39.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.091+0000 7f865bfff640 1 -- 172.21.15.22:0/4464571 wait complete. 2024-09-19T08:17:39.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.092+0000 7f865bfff640 1 Processor -- start 2024-09-19T08:17:39.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.092+0000 7f865bfff640 1 -- start start 2024-09-19T08:17:39.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.092+0000 7f865bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c1019c0 0x7f865c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.092+0000 7f865bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f865c102360 0x7f865c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.092+0000 7f865bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865c1079c0 0x7f865c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.092+0000 7f865bfff640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f865c111dc0 con 0x7f865c102360 2024-09-19T08:17:39.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.092+0000 7f865bfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f865c111c40 con 0x7f865c1019c0 2024-09-19T08:17:39.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.092+0000 7f865bfff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f865c111f40 con 0x7f865c1079c0 2024-09-19T08:17:39.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.092+0000 7f865a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f865c102360 0x7f865c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.092+0000 7f865affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c1019c0 0x7f865c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.092+0000 7f865a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f865c102360 0x7f865c198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52498/0 (socket says 172.21.15.22:52498) 2024-09-19T08:17:39.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.092+0000 7f865a7fc640 1 -- 172.21.15.22:0/1832474169 learned_addr learned my addr 172.21.15.22:0/1832474169 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:39.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.092+0000 7f865b7fe640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865c1079c0 0x7f865c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.093+0000 7f865b7fe640 1 -- 172.21.15.22:0/1832474169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c1019c0 msgr2=0x7f865c1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:39.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.093+0000 7f865b7fe640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c1019c0 0x7f865c1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.093+0000 7f865b7fe640 1 -- 172.21.15.22:0/1832474169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f865c102360 msgr2=0x7f865c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:39.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.093+0000 7f865b7fe640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f865c102360 0x7f865c198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.093+0000 7f865b7fe640 1 -- 172.21.15.22:0/1832474169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c19d270 con 0x7f865c1079c0 2024-09-19T08:17:39.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.093+0000 7f865affd640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c1019c0 0x7f865c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:39.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.093+0000 7f865a7fc640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f865c102360 0x7f865c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:39.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.093+0000 7f865b7fe640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865c1079c0 0x7f865c19cb90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8654009fd0 tx=0x7f865400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:39.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.093+0000 7f8643fff640 1 -- 172.21.15.22:0/1832474169 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865400ce20 con 0x7f865c1079c0 2024-09-19T08:17:39.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.093+0000 7f865bfff640 1 -- 172.21.15.22:0/1832474169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c19d560 con 0x7f865c1079c0 2024-09-19T08:17:39.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.093+0000 7f8643fff640 1 -- 172.21.15.22:0/1832474169 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8654005020 con 0x7f865c1079c0 2024-09-19T08:17:39.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.093+0000 7f8643fff640 1 -- 172.21.15.22:0/1832474169 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654010720 con 0x7f865c1079c0 2024-09-19T08:17:39.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.093+0000 7f865bfff640 1 -- 172.21.15.22:0/1832474169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c1a4e40 con 0x7f865c1079c0 2024-09-19T08:17:39.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.096+0000 7f86638e7740 1 Processor -- start 2024-09-19T08:17:39.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.096+0000 7f8643fff640 1 -- 172.21.15.22:0/1832474169 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8654005540 con 0x7f865c1079c0 2024-09-19T08:17:39.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.096+0000 7f86638e7740 1 -- start start 2024-09-19T08:17:39.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.096+0000 7f86638e7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c233270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.096+0000 7f86638e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c233bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.096+0000 7f86638e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c242960 0x55cd1c244d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.096+0000 7f86638e7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cd1c1992f0 con 0x55cd1c2337b0 2024-09-19T08:17:39.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.096+0000 7f86638e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cd1c140cf0 con 0x55cd1c242960 2024-09-19T08:17:39.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.096+0000 7f86638e7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cd1c1952b0 con 0x55cd1c234d20 2024-09-19T08:17:39.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f865affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c233bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f865a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c233270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f865a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c233270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:50340/0 (socket says 172.21.15.22:50340) 2024-09-19T08:17:39.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f865affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c233bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52506/0 (socket says 172.21.15.22:52506) 2024-09-19T08:17:39.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f865a7fc640 1 -- 172.21.15.22:0/3525913909 learned_addr learned my addr 172.21.15.22:0/3525913909 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:39.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f865a7fc640 1 --2- 172.21.15.22:0/3525913909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c242960 0x55cd1c244d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f865affd640 1 -- 172.21.15.22:0/3525913909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 msgr2=0x55cd1c233270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:39.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f865affd640 1 --2- 172.21.15.22:0/3525913909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c233270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f865affd640 1 -- 172.21.15.22:0/3525913909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c242960 msgr2=0x55cd1c244d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:39.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f865affd640 1 --2- 172.21.15.22:0/3525913909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c242960 0x55cd1c244d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f865affd640 1 -- 172.21.15.22:0/3525913909 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd1c1253a0 con 0x55cd1c2337b0 2024-09-19T08:17:39.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f865a7fc640 1 --2- 172.21.15.22:0/3525913909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c242960 0x55cd1c244d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:17:39.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f8643fff640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f86440a33b0 0x7f86440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f865affd640 1 --2- 172.21.15.22:0/3525913909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c233bf0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f864c009f50 tx=0x7f864c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe1658e8e21f7e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:39.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f865a7fc640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f86440a33b0 0x7f86440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f8643fff640 1 -- 172.21.15.22:0/1832474169 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f86540d0310 con 0x7f865c1079c0 2024-09-19T08:17:39.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f86417fa640 1 -- 172.21.15.22:0/3525913909 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c004030 con 0x55cd1c2337b0 2024-09-19T08:17:39.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f86417fa640 1 -- 172.21.15.22:0/3525913909 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f864c005500 con 0x55cd1c2337b0 2024-09-19T08:17:39.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.097+0000 7f86417fa640 1 -- 172.21.15.22:0/3525913909 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c004cc0 con 0x55cd1c2337b0 2024-09-19T08:17:39.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.098+0000 7f86638e7740 1 -- 172.21.15.22:0/3525913909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 msgr2=0x55cd1c233bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:39.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.098+0000 7f865a7fc640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f86440a33b0 0x7f86440a5870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55cd1c234250 tx=0x7f8650005d50 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:39.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.098+0000 7f86638e7740 1 --2- 172.21.15.22:0/3525913909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c233bf0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f864c009f50 tx=0x7f864c0118d0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.098+0000 7f86638e7740 1 -- 172.21.15.22:0/3525913909 shutdown_connections 2024-09-19T08:17:39.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.098+0000 7f86638e7740 1 --2- 172.21.15.22:0/3525913909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c242960 0x55cd1c244d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.098+0000 7f86638e7740 1 --2- 172.21.15.22:0/3525913909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c233bf0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.098+0000 7f86638e7740 1 --2- 172.21.15.22:0/3525913909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c233270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.098+0000 7f86638e7740 1 -- 172.21.15.22:0/3525913909 >> 172.21.15.22:0/3525913909 conn(0x55cd1c1fc310 msgr2=0x55cd1c22fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:39.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.098+0000 7f86638e7740 1 -- 172.21.15.22:0/3525913909 shutdown_connections 2024-09-19T08:17:39.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.098+0000 7f86638e7740 1 -- 172.21.15.22:0/3525913909 wait complete. 2024-09-19T08:17:39.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.099+0000 7f86638e7740 1 -- start start 2024-09-19T08:17:39.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.099+0000 7f86638e7740 1 Processor -- start 2024-09-19T08:17:39.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.099+0000 7f86638e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c242300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.099+0000 7f86638e7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c23f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.099+0000 7f86638e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c242960 0x55cd1c23fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.099+0000 7f865a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c23f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.099+0000 7f865a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c23f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:50350/0 (socket says 172.21.15.22:50350) 2024-09-19T08:17:39.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.099+0000 7f865affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c242300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.099+0000 7f865affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c242300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52512/0 (socket says 172.21.15.22:52512) 2024-09-19T08:17:39.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.099+0000 7f865a7fc640 1 -- 172.21.15.22:0/4096999871 learned_addr learned my addr 172.21.15.22:0/4096999871 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:39.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.099+0000 7f86638e7740 1 -- 172.21.15.22:0/4096999871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cd1c245660 con 0x55cd1c2337b0 2024-09-19T08:17:39.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.099+0000 7f86638e7740 1 -- 172.21.15.22:0/4096999871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cd1c154060 con 0x55cd1c242960 2024-09-19T08:17:39.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.099+0000 7f86638e7740 1 -- 172.21.15.22:0/4096999871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cd1c2457e0 con 0x55cd1c234d20 2024-09-19T08:17:39.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.100+0000 7f865b7fe640 1 --2- 172.21.15.22:0/4096999871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c242960 0x55cd1c23fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.100+0000 7f86638e7740 10 client.0 _ll_register_callbacks cb 0x55cd1bea78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:39.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.100+0000 7f865a7fc640 1 -- 172.21.15.22:0/4096999871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c242960 msgr2=0x55cd1c23fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:39.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.100+0000 7f865a7fc640 1 --2- 172.21.15.22:0/4096999871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c242960 0x55cd1c23fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.100+0000 7f865a7fc640 1 -- 172.21.15.22:0/4096999871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 msgr2=0x55cd1c242300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:39.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.100+0000 7f865a7fc640 1 --2- 172.21.15.22:0/4096999871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c242300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.100+0000 7f865a7fc640 1 -- 172.21.15.22:0/4096999871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd1bf0a3c0 con 0x55cd1c234d20 2024-09-19T08:17:39.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.100+0000 7f865a7fc640 1 --2- 172.21.15.22:0/4096999871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c23f5b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8650005780 tx=0x7f8650009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e9613fa43a5260 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:39.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.100+0000 7f863eff5640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:17:39.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.100+0000 7f863eff5640 1 -- 172.21.15.22:0/4096999871 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865004e030 con 0x55cd1c234d20 2024-09-19T08:17:39.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.100+0000 7f863eff5640 1 -- 172.21.15.22:0/4096999871 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f86500035f0 con 0x55cd1c234d20 2024-09-19T08:17:39.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.100+0000 7f86638e7740 1 -- 172.21.15.22:0/4096999871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f864c005500 con 0x55cd1c234d20 2024-09-19T08:17:39.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.101+0000 7f863eff5640 1 -- 172.21.15.22:0/4096999871 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8650035040 con 0x55cd1c234d20 2024-09-19T08:17:39.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.101+0000 7f86638e7740 10 client.25525 fetch_fsmap learned FSMap version 27 2024-09-19T08:17:39.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.101+0000 7f86638e7740 1 -- 172.21.15.22:0/4096999871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd1c2510e0 con 0x55cd1c234d20 2024-09-19T08:17:39.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.101+0000 7f863eff5640 1 -- 172.21.15.22:0/4096999871 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f864c005500 con 0x55cd1c234d20 2024-09-19T08:17:39.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.101+0000 7f863eff5640 1 -- 172.21.15.22:0/4096999871 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8650047050 con 0x55cd1c234d20 2024-09-19T08:17:39.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.101+0000 7f86638e7740 10 client.25525 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:17:39.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.101+0000 7f86638e7740 10 client.25525 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:17:39.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.101+0000 7f86638e7740 20 client.25525 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:39.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.101+0000 7f86638e7740 1 --2- 172.21.15.22:0/4096999871 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55cd1c2525b0 0x55cd1c254a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.101+0000 7f86638e7740 4 client.25525 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:17:39.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.101+0000 7f86638e7740 1 -- 172.21.15.22:0/4096999871 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd1c1a6860 con 0x55cd1c2525b0 2024-09-19T08:17:39.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.101+0000 7f865affd640 1 --2- 172.21.15.22:0/4096999871 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55cd1c2525b0 0x55cd1c254a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.102+0000 7f865affd640 1 --2- 172.21.15.22:0/4096999871 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55cd1c2525b0 0x55cd1c254a30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8dcaf3d0577088c2 server_cookie=61d9cdacbd3665fa in_seq=0 out_seq=0 2024-09-19T08:17:39.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.102+0000 7f863eff5640 10 client.25525 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:17:39.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f863eff5640 1 -- 172.21.15.22:0/4096999871 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd1c1a6860 con 0x55cd1c2525b0 2024-09-19T08:17:39.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f863eff5640 10 client.25525 handle_command_reply: tid=0 2024-09-19T08:17:39.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f86638e7740 1 client.25525 shutdown 2024-09-19T08:17:39.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f86638e7740 1 -- 172.21.15.22:0/4096999871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 msgr2=0x55cd1c23f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:39.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f86638e7740 1 --2- 172.21.15.22:0/4096999871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c23f5b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8650005780 tx=0x7f8650009260 comp rx=0 tx=0).stop 2024-09-19T08:17:39.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f86638e7740 1 -- 172.21.15.22:0/4096999871 shutdown_connections 2024-09-19T08:17:39.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f86638e7740 1 --2- 172.21.15.22:0/4096999871 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55cd1c2525b0 0x55cd1c254a30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f86638e7740 1 --2- 172.21.15.22:0/4096999871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c242960 0x55cd1c23fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f86638e7740 1 --2- 172.21.15.22:0/4096999871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c23f5b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f863eff5640 0 client.25525 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:17:39.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f86638e7740 1 --2- 172.21.15.22:0/4096999871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c242300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f86638e7740 1 -- 172.21.15.22:0/4096999871 >> 172.21.15.22:0/4096999871 conn(0x55cd1c1fc310 msgr2=0x55cd1c235b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:39.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f86638e7740 1 -- 172.21.15.22:0/4096999871 shutdown_connections 2024-09-19T08:17:39.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f86638e7740 1 -- 172.21.15.22:0/4096999871 wait complete. 2024-09-19T08:17:39.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.103+0000 7f86638e7740 20 client.25525 trim_cache size 0 max 16384 2024-09-19T08:17:39.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.116+0000 7f86638e7740 1 Processor -- start 2024-09-19T08:17:39.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.116+0000 7f86638e7740 1 -- start start 2024-09-19T08:17:39.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f86638e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c24e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f86638e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c234d20 0x55cd1c241520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f86638e7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c242960 0x55cd1c25a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f86638e7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1ade20 con 0x55cd1c2337b0 2024-09-19T08:17:39.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f86638e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1ab6c0 con 0x55cd1c234d20 2024-09-19T08:17:39.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f86638e7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1a9510 con 0x55cd1c242960 2024-09-19T08:17:39.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f865affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c24e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f865affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c24e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52518/0 (socket says 172.21.15.22:52518) 2024-09-19T08:17:39.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f865affd640 1 -- 172.21.15.22:0/2777015490 learned_addr learned my addr 172.21.15.22:0/2777015490 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:39.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f865b7fe640 1 --2- 172.21.15.22:0/2777015490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c242960 0x55cd1c25a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f865a7fc640 1 --2- 172.21.15.22:0/2777015490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c234d20 0x55cd1c241520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f865affd640 1 -- 172.21.15.22:0/2777015490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c242960 msgr2=0x55cd1c25a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:39.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f865affd640 1 --2- 172.21.15.22:0/2777015490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c242960 0x55cd1c25a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f865affd640 1 -- 172.21.15.22:0/2777015490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c234d20 msgr2=0x55cd1c241520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:39.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f865affd640 1 --2- 172.21.15.22:0/2777015490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c234d20 0x55cd1c241520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f865affd640 1 -- 172.21.15.22:0/2777015490 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd1c23b1b0 con 0x55cd1c2337b0 2024-09-19T08:17:39.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.117+0000 7f865affd640 1 --2- 172.21.15.22:0/2777015490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c24e930 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f864c01b040 tx=0x7f864c004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03502d06fc5f1a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:39.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.118+0000 7f8640ff9640 1 -- 172.21.15.22:0/2777015490 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c00f540 con 0x55cd1c2337b0 2024-09-19T08:17:39.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.118+0000 7f8640ff9640 1 -- 172.21.15.22:0/2777015490 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f864c02f030 con 0x55cd1c2337b0 2024-09-19T08:17:39.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.118+0000 7f8640ff9640 1 -- 172.21.15.22:0/2777015490 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c02b5e0 con 0x55cd1c2337b0 2024-09-19T08:17:39.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.118+0000 7f86638e7740 1 -- 172.21.15.22:0/2777015490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 msgr2=0x55cd1c24e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:39.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.118+0000 7f86638e7740 1 --2- 172.21.15.22:0/2777015490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c24e930 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f864c01b040 tx=0x7f864c004840 comp rx=0 tx=0).stop 2024-09-19T08:17:39.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 -- 172.21.15.22:0/2777015490 shutdown_connections 2024-09-19T08:17:39.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 --2- 172.21.15.22:0/2777015490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c242960 0x55cd1c25a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 --2- 172.21.15.22:0/2777015490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c234d20 0x55cd1c241520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 --2- 172.21.15.22:0/2777015490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c2337b0 0x55cd1c24e930 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 -- 172.21.15.22:0/2777015490 >> 172.21.15.22:0/2777015490 conn(0x55cd1c1fc310 msgr2=0x55cd1c240e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:39.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 -- 172.21.15.22:0/2777015490 shutdown_connections 2024-09-19T08:17:39.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 -- 172.21.15.22:0/2777015490 wait complete. 2024-09-19T08:17:39.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 -- start start 2024-09-19T08:17:39.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 Processor -- start 2024-09-19T08:17:39.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c2337b0 0x55cd1c23c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c23cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c242960 0x55cd1c23d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1a9b80 con 0x55cd1c242960 2024-09-19T08:17:39.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1aaff0 con 0x55cd1c2337b0 2024-09-19T08:17:39.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1aa250 con 0x55cd1c234d20 2024-09-19T08:17:39.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.119+0000 7f86638e7740 10 client.0 _ll_register_callbacks cb 0x7f8650009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:39.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f865b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c242960 0x55cd1c23d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f865a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c23cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f865b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c242960 0x55cd1c23d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52522/0 (socket says 172.21.15.22:52522) 2024-09-19T08:17:39.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f865b7fe640 1 -- 172.21.15.22:0/1038466324 learned_addr learned my addr 172.21.15.22:0/1038466324 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:39.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f865affd640 1 --2- 172.21.15.22:0/1038466324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c2337b0 0x55cd1c23c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f865b7fe640 1 -- 172.21.15.22:0/1038466324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 msgr2=0x55cd1c23cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:39.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f865b7fe640 1 --2- 172.21.15.22:0/1038466324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c23cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f865b7fe640 1 -- 172.21.15.22:0/1038466324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c2337b0 msgr2=0x55cd1c23c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:39.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f865b7fe640 1 --2- 172.21.15.22:0/1038466324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c2337b0 0x55cd1c23c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f865b7fe640 1 -- 172.21.15.22:0/1038466324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd1c2bc320 con 0x55cd1c242960 2024-09-19T08:17:39.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f865a7fc640 1 --2- 172.21.15.22:0/1038466324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c23cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:39.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f865b7fe640 1 --2- 172.21.15.22:0/1038466324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c242960 0x55cd1c23d070 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8654098fd0 tx=0x7f865409cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d118133942cf6217 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:39.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f863eff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:17:39.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f863eff5640 1 -- 172.21.15.22:0/1038466324 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86540ba060 con 0x55cd1c242960 2024-09-19T08:17:39.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f863eff5640 1 -- 172.21.15.22:0/1038466324 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f86540cdcf0 con 0x55cd1c242960 2024-09-19T08:17:39.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f86638e7740 1 -- 172.21.15.22:0/1038466324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f864c02f030 con 0x55cd1c242960 2024-09-19T08:17:39.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f863eff5640 1 -- 172.21.15.22:0/1038466324 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8654098040 con 0x55cd1c242960 2024-09-19T08:17:39.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f86638e7740 10 client.15969 fetch_fsmap learned FSMap version 27 2024-09-19T08:17:39.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.120+0000 7f86638e7740 1 -- 172.21.15.22:0/1038466324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd1c2bd660 con 0x55cd1c242960 2024-09-19T08:17:39.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.121+0000 7f863eff5640 1 -- 172.21.15.22:0/1038466324 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f864c02f030 con 0x55cd1c242960 2024-09-19T08:17:39.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.121+0000 7f863eff5640 1 -- 172.21.15.22:0/1038466324 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f865409ba50 con 0x55cd1c242960 2024-09-19T08:17:39.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.121+0000 7f86638e7740 10 client.15969 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:17:39.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.121+0000 7f86638e7740 10 client.15969 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:17:39.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.121+0000 7f86638e7740 20 client.15969 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:39.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.121+0000 7f86638e7740 1 --2- 172.21.15.22:0/1038466324 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55cd1c2bddb0 0x55cd1c2c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:39.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.121+0000 7f86638e7740 4 client.15969 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:17:39.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.121+0000 7f86638e7740 1 -- 172.21.15.22:0/1038466324 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd1c1a0d70 con 0x55cd1c2bddb0 2024-09-19T08:17:39.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.121+0000 7f865affd640 1 --2- 172.21.15.22:0/1038466324 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55cd1c2bddb0 0x55cd1c2c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:39.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f865affd640 1 --2- 172.21.15.22:0/1038466324 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55cd1c2bddb0 0x55cd1c2c01d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1deea87fc772d8c server_cookie=b9d9b11b9e9151ec in_seq=0 out_seq=0 2024-09-19T08:17:39.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f863eff5640 10 client.15969 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:17:39.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f863eff5640 1 -- 172.21.15.22:0/1038466324 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd1c1a0d70 con 0x55cd1c2bddb0 2024-09-19T08:17:39.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f863eff5640 10 client.15969 handle_command_reply: tid=0 2024-09-19T08:17:39.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f86638e7740 1 client.15969 shutdown 2024-09-19T08:17:39.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f86638e7740 1 -- 172.21.15.22:0/1038466324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c242960 msgr2=0x55cd1c23d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:39.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f86638e7740 1 --2- 172.21.15.22:0/1038466324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c242960 0x55cd1c23d070 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8654098fd0 tx=0x7f865409cdc0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f86638e7740 1 -- 172.21.15.22:0/1038466324 shutdown_connections 2024-09-19T08:17:39.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f86638e7740 1 --2- 172.21.15.22:0/1038466324 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55cd1c2bddb0 0x55cd1c2c01d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f86638e7740 1 --2- 172.21.15.22:0/1038466324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd1c242960 0x55cd1c23d070 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f86638e7740 1 --2- 172.21.15.22:0/1038466324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd1c234d20 0x55cd1c23cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f863eff5640 0 client.15969 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:17:39.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f86638e7740 1 --2- 172.21.15.22:0/1038466324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1c2337b0 0x55cd1c23c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f86638e7740 1 -- 172.21.15.22:0/1038466324 >> 172.21.15.22:0/1038466324 conn(0x55cd1c1fc310 msgr2=0x55cd1c23dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:39.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f86638e7740 1 -- 172.21.15.22:0/1038466324 shutdown_connections 2024-09-19T08:17:39.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f86638e7740 1 -- 172.21.15.22:0/1038466324 wait complete. 2024-09-19T08:17:39.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.122+0000 7f86638e7740 20 client.15969 trim_cache size 0 max 16384 2024-09-19T08:17:39.128 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:17:39.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.123+0000 7f865bfff640 1 -- 172.21.15.22:0/1832474169 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f86440a33b0 msgr2=0x7f86440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:39.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.123+0000 7f865bfff640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f86440a33b0 0x7f86440a5870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55cd1c234250 tx=0x7f8650005d50 comp rx=0 tx=0).stop 2024-09-19T08:17:39.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.123+0000 7f865bfff640 1 -- 172.21.15.22:0/1832474169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865c1079c0 msgr2=0x7f865c19cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:39.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.123+0000 7f865bfff640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865c1079c0 0x7f865c19cb90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8654009fd0 tx=0x7f865400ef10 comp rx=0 tx=0).stop 2024-09-19T08:17:39.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.123+0000 7f865bfff640 1 -- 172.21.15.22:0/1832474169 shutdown_connections 2024-09-19T08:17:39.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.123+0000 7f865bfff640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f86440a33b0 0x7f86440a5870 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.123+0000 7f865bfff640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865c1079c0 0x7f865c19cb90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.123+0000 7f865bfff640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f865c102360 0x7f865c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.123+0000 7f865bfff640 1 --2- 172.21.15.22:0/1832474169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f865c1019c0 0x7f865c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:39.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.123+0000 7f865bfff640 1 -- 172.21.15.22:0/1832474169 >> 172.21.15.22:0/1832474169 conn(0x7f865c0fd720 msgr2=0x7f865c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:39.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.123+0000 7f865bfff640 1 -- 172.21.15.22:0/1832474169 shutdown_connections 2024-09-19T08:17:39.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:39.123+0000 7f865bfff640 1 -- 172.21.15.22:0/1832474169 wait complete. 2024-09-19T08:17:39.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:38 smithi157 ceph-mon[35143]: pgmap v81: 97 pgs: 97 active+clean; 85 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 145 op/s 2024-09-19T08:17:39.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:38 smithi064 ceph-mon[34900]: pgmap v81: 97 pgs: 97 active+clean; 85 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 145 op/s 2024-09-19T08:17:39.891 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:17:40.201 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:17:40.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.762+0000 7f9b1600f640 1 -- 172.21.15.22:0/2807603446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b10106f50 msgr2=0x7f9b1010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:40.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.762+0000 7f9b1600f640 1 --2- 172.21.15.22:0/2807603446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b10106f50 0x7f9b1010eaf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009cd0 tx=0x7f9b0802a8e0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.762+0000 7f9b1600f640 1 -- 172.21.15.22:0/2807603446 shutdown_connections 2024-09-19T08:17:40.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.762+0000 7f9b1600f640 1 --2- 172.21.15.22:0/2807603446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b10106f50 0x7f9b1010eaf0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.762+0000 7f9b1600f640 1 --2- 172.21.15.22:0/2807603446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b10100ca0 0x7f9b10106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.762+0000 7f9b1600f640 1 --2- 172.21.15.22:0/2807603446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b101002f0 0x7f9b101006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.762+0000 7f9b1600f640 1 -- 172.21.15.22:0/2807603446 >> 172.21.15.22:0/2807603446 conn(0x7f9b10077210 msgr2=0x7f9b100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:40.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.762+0000 7f9b1600f640 1 -- 172.21.15.22:0/2807603446 shutdown_connections 2024-09-19T08:17:40.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.762+0000 7f9b1600f640 1 -- 172.21.15.22:0/2807603446 wait complete. 2024-09-19T08:17:40.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.764+0000 7f9b1600f640 1 Processor -- start 2024-09-19T08:17:40.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.764+0000 7f9b1600f640 1 -- start start 2024-09-19T08:17:40.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.764+0000 7f9b1600f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b101002f0 0x7f9b10198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.764+0000 7f9b1600f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b10100ca0 0x7f9b10198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.764+0000 7f9b1600f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b10106f50 0x7f9b1019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.764+0000 7f9b1600f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10111ff0 con 0x7f9b101002f0 2024-09-19T08:17:40.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.764+0000 7f9b1600f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10111e70 con 0x7f9b10106f50 2024-09-19T08:17:40.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.764+0000 7f9b1600f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10112170 con 0x7f9b10100ca0 2024-09-19T08:17:40.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.765+0000 7f9b0f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b101002f0 0x7f9b10198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.765+0000 7f9b0effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b10100ca0 0x7f9b10198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.765+0000 7f9b0f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b101002f0 0x7f9b10198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52538/0 (socket says 172.21.15.22:52538) 2024-09-19T08:17:40.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.765+0000 7f9b0f7fe640 1 -- 172.21.15.22:0/1787224074 learned_addr learned my addr 172.21.15.22:0/1787224074 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:40.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.765+0000 7f9b0ffff640 1 --2- 172.21.15.22:0/1787224074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b10106f50 0x7f9b1019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.765+0000 7f9b0effd640 1 -- 172.21.15.22:0/1787224074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b10106f50 msgr2=0x7f9b1019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:40.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.765+0000 7f9b0effd640 1 --2- 172.21.15.22:0/1787224074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b10106f50 0x7f9b1019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.765+0000 7f9b0effd640 1 -- 172.21.15.22:0/1787224074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b101002f0 msgr2=0x7f9b10198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:40.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.765+0000 7f9b0effd640 1 --2- 172.21.15.22:0/1787224074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b101002f0 0x7f9b10198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.765+0000 7f9b0effd640 1 -- 172.21.15.22:0/1787224074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1019d400 con 0x7f9b10100ca0 2024-09-19T08:17:40.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.765+0000 7f9b0effd640 1 --2- 172.21.15.22:0/1787224074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b10100ca0 0x7f9b10198990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400e9c0 tx=0x7f9b0400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:40.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.766+0000 7f9b0cff9640 1 -- 172.21.15.22:0/1787224074 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0400ccc0 con 0x7f9b10100ca0 2024-09-19T08:17:40.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.766+0000 7f9b0cff9640 1 -- 172.21.15.22:0/1787224074 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b04010910 con 0x7f9b10100ca0 2024-09-19T08:17:40.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.766+0000 7f9b1600f640 1 -- 172.21.15.22:0/1787224074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1019d6f0 con 0x7f9b10100ca0 2024-09-19T08:17:40.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.766+0000 7f9b0cff9640 1 -- 172.21.15.22:0/1787224074 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b040188c0 con 0x7f9b10100ca0 2024-09-19T08:17:40.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.766+0000 7f9b1600f640 1 -- 172.21.15.22:0/1787224074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b101a4fd0 con 0x7f9b10100ca0 2024-09-19T08:17:40.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.768+0000 7f9b0cff9640 1 -- 172.21.15.22:0/1787224074 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9b04010ac0 con 0x7f9b10100ca0 2024-09-19T08:17:40.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.769+0000 7f9b0cff9640 1 --2- 172.21.15.22:0/1787224074 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9af80a33b0 0x7f9af80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.769+0000 7f9b0cff9640 1 -- 172.21.15.22:0/1787224074 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9b040d0060 con 0x7f9b10100ca0 2024-09-19T08:17:40.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.769+0000 7f9b0f7fe640 1 --2- 172.21.15.22:0/1787224074 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9af80a33b0 0x7f9af80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.770+0000 7f9b0f7fe640 1 --2- 172.21.15.22:0/1787224074 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9af80a33b0 0x7f9af80a5870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009c90 tx=0x7f9b00005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:40.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.770+0000 7f9b183e8740 1 Processor -- start 2024-09-19T08:17:40.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.770+0000 7f9b183e8740 1 -- start start 2024-09-19T08:17:40.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b183e8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5cfb40 0x55db9a5ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b183e8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5ce5d0 0x55db9a5cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b183e8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5dd780 0x55db9a5dfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b183e8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55db9a5342f0 con 0x55db9a5ce5d0 2024-09-19T08:17:40.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b183e8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55db9a4dbcf0 con 0x55db9a5cfb40 2024-09-19T08:17:40.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b183e8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55db9a5302b0 con 0x55db9a5dd780 2024-09-19T08:17:40.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b0ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5dd780 0x55db9a5dfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b0ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5dd780 0x55db9a5dfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:50400/0 (socket says 172.21.15.22:50400) 2024-09-19T08:17:40.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b0ffff640 1 -- 172.21.15.22:0/3460322862 learned_addr learned my addr 172.21.15.22:0/3460322862 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:40.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b0effd640 1 --2- 172.21.15.22:0/3460322862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5ce5d0 0x55db9a5cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b0ffff640 1 -- 172.21.15.22:0/3460322862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5cfb40 msgr2=0x55db9a5ce090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:17:40.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b0ffff640 1 --2- 172.21.15.22:0/3460322862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5cfb40 0x55db9a5ce090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b0ffff640 1 -- 172.21.15.22:0/3460322862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5ce5d0 msgr2=0x55db9a5cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:40.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b0ffff640 1 --2- 172.21.15.22:0/3460322862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5ce5d0 0x55db9a5cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b0ffff640 1 -- 172.21.15.22:0/3460322862 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db9a4c03a0 con 0x55db9a5dd780 2024-09-19T08:17:40.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.771+0000 7f9b0effd640 1 --2- 172.21.15.22:0/3460322862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5ce5d0 0x55db9a5cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:17:40.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.772+0000 7f9b0ffff640 1 --2- 172.21.15.22:0/3460322862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5dd780 0x55db9a5dfb70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009cd0 tx=0x7f9b08009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ea6571b5aa997b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:40.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.772+0000 7f9af5ffb640 1 -- 172.21.15.22:0/3460322862 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08049070 con 0x55db9a5dd780 2024-09-19T08:17:40.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.772+0000 7f9af5ffb640 1 -- 172.21.15.22:0/3460322862 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b08036040 con 0x55db9a5dd780 2024-09-19T08:17:40.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.772+0000 7f9af5ffb640 1 -- 172.21.15.22:0/3460322862 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08004040 con 0x55db9a5dd780 2024-09-19T08:17:40.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.773+0000 7f9b183e8740 1 -- 172.21.15.22:0/3460322862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5dd780 msgr2=0x55db9a5dfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:40.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.773+0000 7f9b183e8740 1 --2- 172.21.15.22:0/3460322862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5dd780 0x55db9a5dfb70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009cd0 tx=0x7f9b08009a20 comp rx=0 tx=0).stop 2024-09-19T08:17:40.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.773+0000 7f9b183e8740 1 -- 172.21.15.22:0/3460322862 shutdown_connections 2024-09-19T08:17:40.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.773+0000 7f9b183e8740 1 --2- 172.21.15.22:0/3460322862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5dd780 0x55db9a5dfb70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.773+0000 7f9b183e8740 1 --2- 172.21.15.22:0/3460322862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5ce5d0 0x55db9a5cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.773+0000 7f9b183e8740 1 --2- 172.21.15.22:0/3460322862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5cfb40 0x55db9a5ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.773+0000 7f9b183e8740 1 -- 172.21.15.22:0/3460322862 >> 172.21.15.22:0/3460322862 conn(0x55db9a597310 msgr2=0x55db9a5cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:40.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.775+0000 7f9b183e8740 1 -- 172.21.15.22:0/3460322862 shutdown_connections 2024-09-19T08:17:40.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.775+0000 7f9b183e8740 1 -- 172.21.15.22:0/3460322862 wait complete. 2024-09-19T08:17:40.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.776+0000 7f9b183e8740 1 -- start start 2024-09-19T08:17:40.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.776+0000 7f9b183e8740 1 Processor -- start 2024-09-19T08:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.777+0000 7f9b183e8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5ce5d0 0x55db9a5dd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.777+0000 7f9b183e8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5cfb40 0x55db9a5da480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.777+0000 7f9b183e8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5dd780 0x55db9a5dac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.777+0000 7f9b183e8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55db9a5e0420 con 0x55db9a5cfb40 2024-09-19T08:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.777+0000 7f9b183e8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55db9a4ef060 con 0x55db9a5ce5d0 2024-09-19T08:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.777+0000 7f9b183e8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55db9a5e05a0 con 0x55db9a5dd780 2024-09-19T08:17:40.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.777+0000 7f9b183e8740 10 client.0 _ll_register_callbacks cb 0x55db9a2428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:40.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.777+0000 7f9b0effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5cfb40 0x55db9a5da480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.777+0000 7f9b0effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5cfb40 0x55db9a5da480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52556/0 (socket says 172.21.15.22:52556) 2024-09-19T08:17:40.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.777+0000 7f9b0effd640 1 -- 172.21.15.22:0/1564851923 learned_addr learned my addr 172.21.15.22:0/1564851923 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:40.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.777+0000 7f9b0ffff640 1 --2- 172.21.15.22:0/1564851923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5dd780 0x55db9a5dac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.777+0000 7f9b0f7fe640 1 --2- 172.21.15.22:0/1564851923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5ce5d0 0x55db9a5dd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.778+0000 7f9b0ffff640 1 -- 172.21.15.22:0/1564851923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5ce5d0 msgr2=0x55db9a5dd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:40.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.778+0000 7f9b0ffff640 1 --2- 172.21.15.22:0/1564851923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5ce5d0 0x55db9a5dd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.778+0000 7f9b0ffff640 1 -- 172.21.15.22:0/1564851923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5cfb40 msgr2=0x55db9a5da480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:40.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.778+0000 7f9b0ffff640 1 --2- 172.21.15.22:0/1564851923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5cfb40 0x55db9a5da480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.778+0000 7f9b0ffff640 1 -- 172.21.15.22:0/1564851923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9a2a53c0 con 0x55db9a5dd780 2024-09-19T08:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.779+0000 7f9b0ffff640 1 --2- 172.21.15.22:0/1564851923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5dd780 0x55db9a5dac10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9b08033040 tx=0x7f9b08003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e0cc75359b20b2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.779+0000 7f9aeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.779+0000 7f9aeeffd640 1 -- 172.21.15.22:0/1564851923 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08049040 con 0x55db9a5dd780 2024-09-19T08:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.779+0000 7f9aeeffd640 1 -- 172.21.15.22:0/1564851923 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9b08031c70 con 0x55db9a5dd780 2024-09-19T08:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.779+0000 7f9b183e8740 1 -- 172.21.15.22:0/1564851923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b08036040 con 0x55db9a5dd780 2024-09-19T08:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.780+0000 7f9aeeffd640 1 -- 172.21.15.22:0/1564851923 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9b08003970 con 0x55db9a5dd780 2024-09-19T08:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.780+0000 7f9b0effd640 1 --2- 172.21.15.22:0/1564851923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5cfb40 0x55db9a5da480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.780+0000 7f9b183e8740 10 client.25543 fetch_fsmap learned FSMap version 27 2024-09-19T08:17:40.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.780+0000 7f9b183e8740 1 -- 172.21.15.22:0/1564851923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55db9a5e0420 con 0x55db9a5dd780 2024-09-19T08:17:40.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.780+0000 7f9aeeffd640 1 -- 172.21.15.22:0/1564851923 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b08036040 con 0x55db9a5dd780 2024-09-19T08:17:40.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.782+0000 7f9aeeffd640 1 -- 172.21.15.22:0/1564851923 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9b08030070 con 0x55db9a5dd780 2024-09-19T08:17:40.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.782+0000 7f9b183e8740 10 client.25543 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:17:40.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.782+0000 7f9b183e8740 10 client.25543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:17:40.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.782+0000 7f9b183e8740 20 client.25543 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:40.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.782+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1564851923 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55db9a5ed0b0 0x55db9a5ef4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.782+0000 7f9b183e8740 4 client.25543 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:17:40.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.782+0000 7f9b183e8740 1 -- 172.21.15.22:0/1564851923 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db9a541860 con 0x55db9a5ed0b0 2024-09-19T08:17:40.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.782+0000 7f9b0f7fe640 1 --2- 172.21.15.22:0/1564851923 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55db9a5ed0b0 0x55db9a5ef4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.782+0000 7f9aeeffd640 10 client.25543 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.782+0000 7f9b0f7fe640 1 --2- 172.21.15.22:0/1564851923 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55db9a5ed0b0 0x55db9a5ef4a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e577bce81a4b1093 server_cookie=af71385826921296 in_seq=0 out_seq=0 2024-09-19T08:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.783+0000 7f9aeeffd640 1 -- 172.21.15.22:0/1564851923 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db9a541860 con 0x55db9a5ed0b0 2024-09-19T08:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.783+0000 7f9aeeffd640 10 client.25543 handle_command_reply: tid=0 2024-09-19T08:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.783+0000 7f9b183e8740 1 client.25543 shutdown 2024-09-19T08:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.783+0000 7f9b183e8740 1 -- 172.21.15.22:0/1564851923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5dd780 msgr2=0x55db9a5dac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.783+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1564851923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5dd780 0x55db9a5dac10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9b08033040 tx=0x7f9b08003b80 comp rx=0 tx=0).stop 2024-09-19T08:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.783+0000 7f9b183e8740 1 -- 172.21.15.22:0/1564851923 shutdown_connections 2024-09-19T08:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.784+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1564851923 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55db9a5ed0b0 0x55db9a5ef4a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.784+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1564851923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5dd780 0x55db9a5dac10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.784+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1564851923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5cfb40 0x55db9a5da480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.784+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1564851923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5ce5d0 0x55db9a5dd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.784+0000 7f9b183e8740 1 -- 172.21.15.22:0/1564851923 >> 172.21.15.22:0/1564851923 conn(0x55db9a597310 msgr2=0x55db9a5d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.784+0000 7f9aeeffd640 0 client.25543 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.784+0000 7f9b183e8740 1 -- 172.21.15.22:0/1564851923 shutdown_connections 2024-09-19T08:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.784+0000 7f9b183e8740 1 -- 172.21.15.22:0/1564851923 wait complete. 2024-09-19T08:17:40.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.784+0000 7f9b183e8740 20 client.25543 trim_cache size 0 max 16384 2024-09-19T08:17:40.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.812+0000 7f9b183e8740 1 Processor -- start 2024-09-19T08:17:40.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.812+0000 7f9b183e8740 1 -- start start 2024-09-19T08:17:40.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.813+0000 7f9b183e8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5ce5d0 0x55db9a5d9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.813+0000 7f9b183e8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5cfb40 0x55db9a5d9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.813+0000 7f9b183e8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5dd780 0x55db9a5f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.813+0000 7f9b183e8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101adfe0 con 0x55db9a5ce5d0 2024-09-19T08:17:40.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.813+0000 7f9b0f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5ce5d0 0x55db9a5d9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.813+0000 7f9b0effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5cfb40 0x55db9a5d9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.813+0000 7f9b0f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5ce5d0 0x55db9a5d9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52566/0 (socket says 172.21.15.22:52566) 2024-09-19T08:17:40.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.813+0000 7f9b0f7fe640 1 -- 172.21.15.22:0/1905801153 learned_addr learned my addr 172.21.15.22:0/1905801153 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:40.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.813+0000 7f9b183e8740 1 -- 172.21.15.22:0/1905801153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101ab880 con 0x55db9a5dd780 2024-09-19T08:17:40.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.813+0000 7f9b183e8740 1 -- 172.21.15.22:0/1905801153 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101a9690 con 0x55db9a5cfb40 2024-09-19T08:17:40.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.814+0000 7f9b0ffff640 1 --2- 172.21.15.22:0/1905801153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5dd780 0x55db9a5f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.814+0000 7f9b0effd640 1 -- 172.21.15.22:0/1905801153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5dd780 msgr2=0x55db9a5f4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:40.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.814+0000 7f9b0effd640 1 --2- 172.21.15.22:0/1905801153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5dd780 0x55db9a5f4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.814+0000 7f9b0effd640 1 -- 172.21.15.22:0/1905801153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5ce5d0 msgr2=0x55db9a5d9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:40.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.814+0000 7f9b0effd640 1 --2- 172.21.15.22:0/1905801153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5ce5d0 0x55db9a5d9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.814+0000 7f9b0effd640 1 -- 172.21.15.22:0/1905801153 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db9a5d1120 con 0x55db9a5cfb40 2024-09-19T08:17:40.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.814+0000 7f9b0f7fe640 1 --2- 172.21.15.22:0/1905801153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5ce5d0 0x55db9a5d9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:40.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.814+0000 7f9b0effd640 1 --2- 172.21.15.22:0/1905801153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5cfb40 0x55db9a5d9460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b0409c330 tx=0x7f9b04098040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a186c56327f96a7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:40.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.814+0000 7f9af57fa640 1 -- 172.21.15.22:0/1905801153 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04098a10 con 0x55db9a5cfb40 2024-09-19T08:17:40.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.814+0000 7f9af57fa640 1 -- 172.21.15.22:0/1905801153 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b040a0170 con 0x55db9a5cfb40 2024-09-19T08:17:40.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.814+0000 7f9af57fa640 1 -- 172.21.15.22:0/1905801153 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04010430 con 0x55db9a5cfb40 2024-09-19T08:17:40.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.815+0000 7f9b183e8740 1 -- 172.21.15.22:0/1905801153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5cfb40 msgr2=0x55db9a5d9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:40.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.815+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1905801153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5cfb40 0x55db9a5d9460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b0409c330 tx=0x7f9b04098040 comp rx=0 tx=0).stop 2024-09-19T08:17:40.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.816+0000 7f9b183e8740 1 -- 172.21.15.22:0/1905801153 shutdown_connections 2024-09-19T08:17:40.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.816+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1905801153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5dd780 0x55db9a5f4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.816+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1905801153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5cfb40 0x55db9a5d9460 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.816+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1905801153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5ce5d0 0x55db9a5d9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.816+0000 7f9b183e8740 1 -- 172.21.15.22:0/1905801153 >> 172.21.15.22:0/1905801153 conn(0x55db9a597310 msgr2=0x55db9a5d8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:40.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.816+0000 7f9b183e8740 1 -- 172.21.15.22:0/1905801153 shutdown_connections 2024-09-19T08:17:40.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.816+0000 7f9b183e8740 1 -- 172.21.15.22:0/1905801153 wait complete. 2024-09-19T08:17:40.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.816+0000 7f9b183e8740 1 -- start start 2024-09-19T08:17:40.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.817+0000 7f9b183e8740 1 Processor -- start 2024-09-19T08:17:40.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.817+0000 7f9b183e8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5ce5d0 0x55db9a5d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.817+0000 7f9b183e8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5cfb40 0x55db9a5d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.817+0000 7f9b183e8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5dd780 0x55db9a5e4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.817+0000 7f9b183e8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101a9d40 con 0x55db9a5dd780 2024-09-19T08:17:40.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.817+0000 7f9b183e8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101ab1b0 con 0x55db9a5cfb40 2024-09-19T08:17:40.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.817+0000 7f9b183e8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101aa410 con 0x55db9a5ce5d0 2024-09-19T08:17:40.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.817+0000 7f9b183e8740 10 client.0 _ll_register_callbacks cb 0x7f9b08047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:40.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.817+0000 7f9b0ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5dd780 0x55db9a5e4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.817+0000 7f9b0ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5dd780 0x55db9a5e4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52578/0 (socket says 172.21.15.22:52578) 2024-09-19T08:17:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.817+0000 7f9b0ffff640 1 -- 172.21.15.22:0/1543179363 learned_addr learned my addr 172.21.15.22:0/1543179363 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.817+0000 7f9b0f7fe640 1 --2- 172.21.15.22:0/1543179363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5ce5d0 0x55db9a5d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.817+0000 7f9b0effd640 1 --2- 172.21.15.22:0/1543179363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5cfb40 0x55db9a5d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.818+0000 7f9b0f7fe640 1 -- 172.21.15.22:0/1543179363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5cfb40 msgr2=0x55db9a5d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.818+0000 7f9b0f7fe640 1 --2- 172.21.15.22:0/1543179363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5cfb40 0x55db9a5d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.818+0000 7f9b0f7fe640 1 -- 172.21.15.22:0/1543179363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5dd780 msgr2=0x55db9a5e4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.818+0000 7f9b0f7fe640 1 --2- 172.21.15.22:0/1543179363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5dd780 0x55db9a5e4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.818+0000 7f9b0f7fe640 1 -- 172.21.15.22:0/1543179363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9a5e4d60 con 0x55db9a5ce5d0 2024-09-19T08:17:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.818+0000 7f9b0f7fe640 1 --2- 172.21.15.22:0/1543179363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5ce5d0 0x55db9a5d1a60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55db9a4dbd10 tx=0x7f9b00004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=456f5a3c0d5f1dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.818+0000 7f9aeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:17:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.818+0000 7f9aeeffd640 1 -- 172.21.15.22:0/1543179363 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0002d030 con 0x55db9a5ce5d0 2024-09-19T08:17:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.818+0000 7f9aeeffd640 1 -- 172.21.15.22:0/1543179363 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9b0002a4f0 con 0x55db9a5ce5d0 2024-09-19T08:17:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.818+0000 7f9b183e8740 1 -- 172.21.15.22:0/1543179363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b040a0170 con 0x55db9a5ce5d0 2024-09-19T08:17:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.819+0000 7f9aeeffd640 1 -- 172.21.15.22:0/1543179363 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9b00039050 con 0x55db9a5ce5d0 2024-09-19T08:17:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.819+0000 7f9b183e8740 10 client.25549 fetch_fsmap learned FSMap version 27 2024-09-19T08:17:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.819+0000 7f9b183e8740 1 -- 172.21.15.22:0/1543179363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55db9a5e60c0 con 0x55db9a5ce5d0 2024-09-19T08:17:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.819+0000 7f9aeeffd640 1 -- 172.21.15.22:0/1543179363 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b0002abd0 con 0x55db9a5ce5d0 2024-09-19T08:17:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.819+0000 7f9aeeffd640 1 -- 172.21.15.22:0/1543179363 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9b00028c90 con 0x55db9a5ce5d0 2024-09-19T08:17:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.819+0000 7f9b183e8740 10 client.25549 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:17:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.819+0000 7f9b183e8740 10 client.25549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:17:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.819+0000 7f9b183e8740 20 client.25549 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.819+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1543179363 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55db9a65c2b0 0x55db9a65e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.819+0000 7f9b183e8740 4 client.25549 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:17:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.819+0000 7f9b183e8740 1 -- 172.21.15.22:0/1543179363 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9b040a0170 con 0x55db9a65c2b0 2024-09-19T08:17:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.820+0000 7f9b0effd640 1 --2- 172.21.15.22:0/1543179363 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55db9a65c2b0 0x55db9a65e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.820+0000 7f9aeeffd640 10 client.25549 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.820+0000 7f9b0effd640 1 --2- 172.21.15.22:0/1543179363 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55db9a65c2b0 0x55db9a65e6a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d2c100a3af3cb2d server_cookie=172b410954c902aa in_seq=0 out_seq=0 2024-09-19T08:17:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.820+0000 7f9aeeffd640 1 -- 172.21.15.22:0/1543179363 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9b040a0170 con 0x55db9a65c2b0 2024-09-19T08:17:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.820+0000 7f9aeeffd640 10 client.25549 handle_command_reply: tid=0 2024-09-19T08:17:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.820+0000 7f9b183e8740 1 client.25549 shutdown 2024-09-19T08:17:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.820+0000 7f9b183e8740 1 -- 172.21.15.22:0/1543179363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5ce5d0 msgr2=0x55db9a5d1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.820+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1543179363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5ce5d0 0x55db9a5d1a60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55db9a4dbd10 tx=0x7f9b00004730 comp rx=0 tx=0).stop 2024-09-19T08:17:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.821+0000 7f9b183e8740 1 -- 172.21.15.22:0/1543179363 shutdown_connections 2024-09-19T08:17:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.821+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1543179363 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55db9a65c2b0 0x55db9a65e6a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.821+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1543179363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db9a5dd780 0x55db9a5e4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.821+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1543179363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db9a5cfb40 0x55db9a5d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.821+0000 7f9b183e8740 1 --2- 172.21.15.22:0/1543179363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db9a5ce5d0 0x55db9a5d1a60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.821+0000 7f9b183e8740 1 -- 172.21.15.22:0/1543179363 >> 172.21.15.22:0/1543179363 conn(0x55db9a597310 msgr2=0x55db9a5d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.821+0000 7f9aeeffd640 0 client.25549 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.821+0000 7f9b183e8740 1 -- 172.21.15.22:0/1543179363 shutdown_connections 2024-09-19T08:17:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.821+0000 7f9b183e8740 1 -- 172.21.15.22:0/1543179363 wait complete. 2024-09-19T08:17:40.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.821+0000 7f9b183e8740 20 client.25549 trim_cache size 0 max 16384 2024-09-19T08:17:40.834 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:17:40.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.822+0000 7f9b1600f640 1 -- 172.21.15.22:0/1787224074 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9af80a33b0 msgr2=0x7f9af80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:40.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.822+0000 7f9b1600f640 1 --2- 172.21.15.22:0/1787224074 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9af80a33b0 0x7f9af80a5870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009c90 tx=0x7f9b00005f90 comp rx=0 tx=0).stop 2024-09-19T08:17:40.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.822+0000 7f9b1600f640 1 -- 172.21.15.22:0/1787224074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b10100ca0 msgr2=0x7f9b10198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:40.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.822+0000 7f9b1600f640 1 --2- 172.21.15.22:0/1787224074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b10100ca0 0x7f9b10198990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400e9c0 tx=0x7f9b0400ee80 comp rx=0 tx=0).stop 2024-09-19T08:17:40.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.822+0000 7f9b1600f640 1 -- 172.21.15.22:0/1787224074 shutdown_connections 2024-09-19T08:17:40.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.822+0000 7f9b1600f640 1 --2- 172.21.15.22:0/1787224074 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9af80a33b0 0x7f9af80a5870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.822+0000 7f9b1600f640 1 --2- 172.21.15.22:0/1787224074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b10106f50 0x7f9b1019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.822+0000 7f9b1600f640 1 --2- 172.21.15.22:0/1787224074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b10100ca0 0x7f9b10198990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.822+0000 7f9b1600f640 1 --2- 172.21.15.22:0/1787224074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b101002f0 0x7f9b10198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:40.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.822+0000 7f9b1600f640 1 -- 172.21.15.22:0/1787224074 >> 172.21.15.22:0/1787224074 conn(0x7f9b10077210 msgr2=0x7f9b101057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:40.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.823+0000 7f9b1600f640 1 -- 172.21.15.22:0/1787224074 shutdown_connections 2024-09-19T08:17:40.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:40.823+0000 7f9b1600f640 1 -- 172.21.15.22:0/1787224074 wait complete. 2024-09-19T08:17:41.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:40 smithi157 ceph-mon[35143]: pgmap v82: 97 pgs: 97 active+clean; 85 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-09-19T08:17:41.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:40 smithi022 ceph-mon[30963]: pgmap v82: 97 pgs: 97 active+clean; 85 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-09-19T08:17:41.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:40 smithi064 ceph-mon[34900]: pgmap v82: 97 pgs: 97 active+clean; 85 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-09-19T08:17:42.746 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:17:42.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:17:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:17:43.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:42 smithi157 ceph-mon[35143]: pgmap v83: 97 pgs: 97 active+clean; 85 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-09-19T08:17:43.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:42 smithi022 ceph-mon[30963]: pgmap v83: 97 pgs: 97 active+clean; 85 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-09-19T08:17:43.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:42 smithi064 ceph-mon[34900]: pgmap v83: 97 pgs: 97 active+clean; 85 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 98 op/s 2024-09-19T08:17:43.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.982+0000 7fcfe361a640 1 -- 172.21.15.22:0/1777402618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfdc101e20 msgr2=0x7fcfdc1056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.982+0000 7fcfe361a640 1 --2- 172.21.15.22:0/1777402618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfdc101e20 0x7fcfdc1056b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009860 tx=0x7fcfcc02a380 comp rx=0 tx=0).stop 2024-09-19T08:17:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.983+0000 7fcfe361a640 1 -- 172.21.15.22:0/1777402618 shutdown_connections 2024-09-19T08:17:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.983+0000 7fcfe361a640 1 --2- 172.21.15.22:0/1777402618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfdc105de0 0x7fcfdc10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.983+0000 7fcfe361a640 1 --2- 172.21.15.22:0/1777402618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfdc101e20 0x7fcfdc1056b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.983+0000 7fcfe361a640 1 --2- 172.21.15.22:0/1777402618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfdc101470 0x7fcfdc101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.983+0000 7fcfe361a640 1 -- 172.21.15.22:0/1777402618 >> 172.21.15.22:0/1777402618 conn(0x7fcfdc077230 msgr2=0x7fcfdc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.983+0000 7fcfe361a640 1 -- 172.21.15.22:0/1777402618 shutdown_connections 2024-09-19T08:17:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.983+0000 7fcfe361a640 1 -- 172.21.15.22:0/1777402618 wait complete. 2024-09-19T08:17:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.984+0000 7fcfe361a640 1 Processor -- start 2024-09-19T08:17:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.985+0000 7fcfe361a640 1 -- start start 2024-09-19T08:17:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.985+0000 7fcfe361a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfdc101470 0x7fcfdc10d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.985+0000 7fcfe361a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfdc101e20 0x7fcfdc10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.985+0000 7fcfe361a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfdc105de0 0x7fcfdc108640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.985+0000 7fcfe361a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc06e6d0 con 0x7fcfdc101e20 2024-09-19T08:17:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.985+0000 7fcfe361a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc06e550 con 0x7fcfdc101470 2024-09-19T08:17:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.985+0000 7fcfe361a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc06e850 con 0x7fcfdc105de0 2024-09-19T08:17:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.985+0000 7fcfe0bb5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfdc101e20 0x7fcfdc10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.985+0000 7fcfe1bb7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfdc105de0 0x7fcfdc108640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.985+0000 7fcfe1bb7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfdc105de0 0x7fcfdc108640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:50460/0 (socket says 172.21.15.22:50460) 2024-09-19T08:17:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.986+0000 7fcfe1bb7640 1 -- 172.21.15.22:0/1223010321 learned_addr learned my addr 172.21.15.22:0/1223010321 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.986+0000 7fcfe13b6640 1 --2- 172.21.15.22:0/1223010321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfdc101470 0x7fcfdc10d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.986+0000 7fcfe0bb5640 1 -- 172.21.15.22:0/1223010321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfdc105de0 msgr2=0x7fcfdc108640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.986+0000 7fcfe0bb5640 1 --2- 172.21.15.22:0/1223010321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfdc105de0 0x7fcfdc108640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.986+0000 7fcfe0bb5640 1 -- 172.21.15.22:0/1223010321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfdc101470 msgr2=0x7fcfdc10d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.986+0000 7fcfe0bb5640 1 --2- 172.21.15.22:0/1223010321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfdc101470 0x7fcfdc10d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.986+0000 7fcfe0bb5640 1 -- 172.21.15.22:0/1223010321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc108f00 con 0x7fcfdc101e20 2024-09-19T08:17:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.986+0000 7fcfe1bb7640 1 --2- 172.21.15.22:0/1223010321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfdc105de0 0x7fcfdc108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.987+0000 7fcfe0bb5640 1 --2- 172.21.15.22:0/1223010321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfdc101e20 0x7fcfdc10d910 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc02a970 tx=0x7fcfcc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.988+0000 7fcfd27fc640 1 -- 172.21.15.22:0/1223010321 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc03f020 con 0x7fcfdc101e20 2024-09-19T08:17:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.989+0000 7fcfe361a640 1 -- 172.21.15.22:0/1223010321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc109190 con 0x7fcfdc101e20 2024-09-19T08:17:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.989+0000 7fcfe361a640 1 -- 172.21.15.22:0/1223010321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc1bc3e0 con 0x7fcfdc101e20 2024-09-19T08:17:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.997+0000 7fcfe59f3740 1 Processor -- start 2024-09-19T08:17:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.999+0000 7fcfe59f3740 1 -- start start 2024-09-19T08:17:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.999+0000 7fcfe59f3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 0x555b33c30f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:43.999+0000 7fcfe59f3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c31480 0x555b33c3b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.000+0000 7fcfe1bb7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 0x555b33c30f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.000+0000 7fcfe1bb7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 0x555b33c30f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:50474/0 (socket says 172.21.15.22:50474) 2024-09-19T08:17:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.000+0000 7fcfe13b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c31480 0x555b33c3b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.000+0000 7fcfe13b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c31480 0x555b33c3b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52608/0 (socket says 172.21.15.22:52608) 2024-09-19T08:17:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.000+0000 7fcfe59f3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c3bd40 0x555b33c3e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.000+0000 7fcfe59f3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555b33b96280 con 0x555b33c31480 2024-09-19T08:17:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.000+0000 7fcfe59f3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555b33b3db10 con 0x555b33c3bd40 2024-09-19T08:17:44.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.000+0000 7fcfe59f3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555b33b92240 con 0x555b33c30b60 2024-09-19T08:17:44.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.000+0000 7fcfd27fc640 1 -- 172.21.15.22:0/1223010321 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfcc0035f0 con 0x7fcfdc101e20 2024-09-19T08:17:44.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.000+0000 7fcfd27fc640 1 -- 172.21.15.22:0/1223010321 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc037040 con 0x7fcfdc101e20 2024-09-19T08:17:44.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.000+0000 7fcfd27fc640 1 -- 172.21.15.22:0/1223010321 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcfcc042020 con 0x7fcfdc101e20 2024-09-19T08:17:44.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.000+0000 7fcfe0bb5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c3bd40 0x555b33c3e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:44.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.001+0000 7fcfe0bb5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c3bd40 0x555b33c3e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:45676/0 (socket says 172.21.15.22:45676) 2024-09-19T08:17:44.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.001+0000 7fcfe0bb5640 1 -- 172.21.15.22:0/3676889608 learned_addr learned my addr 172.21.15.22:0/3676889608 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:44.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.001+0000 7fcfe0bb5640 1 -- 172.21.15.22:0/3676889608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 msgr2=0x555b33c30f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:44.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.001+0000 7fcfe0bb5640 1 --2- 172.21.15.22:0/3676889608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 0x555b33c30f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.001+0000 7fcfe0bb5640 1 -- 172.21.15.22:0/3676889608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c31480 msgr2=0x555b33c3b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.001+0000 7fcfe0bb5640 1 --2- 172.21.15.22:0/3676889608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c31480 0x555b33c3b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.002+0000 7fcfd27fc640 1 --2- 172.21.15.22:0/1223010321 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcfc80a3600 0x7fcfc80a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.002+0000 7fcfd27fc640 1 -- 172.21.15.22:0/1223010321 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcfcc0f0630 con 0x7fcfdc101e20 2024-09-19T08:17:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.003+0000 7fcfe1bb7640 1 --2- 172.21.15.22:0/3676889608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 0x555b33c30f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-19T08:17:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.003+0000 7fcfe13b6640 1 --2- 172.21.15.22:0/3676889608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c31480 0x555b33c3b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-19T08:17:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.004+0000 7fcfe0bb5640 1 -- 172.21.15.22:0/3676889608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b33b4a1f0 con 0x555b33c3bd40 2024-09-19T08:17:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.004+0000 7fcfe1bb7640 1 --2- 172.21.15.22:0/1223010321 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcfc80a3600 0x7fcfc80a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.005+0000 7fcfe1bb7640 1 --2- 172.21.15.22:0/1223010321 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcfc80a3600 0x7fcfc80a5ac0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40078b0 tx=0x7fcfd400b040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.012+0000 7fcfe0bb5640 1 --2- 172.21.15.22:0/3676889608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c3bd40 0x555b33c3e130 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0ede30 tx=0x7fcfcc033f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8597a08634fb4d4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.016+0000 7fcfc77fe640 1 -- 172.21.15.22:0/3676889608 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0b8cf0 con 0x555b33c3bd40 2024-09-19T08:17:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.016+0000 7fcfc77fe640 1 -- 172.21.15.22:0/3676889608 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfcc0bd150 con 0x555b33c3bd40 2024-09-19T08:17:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.016+0000 7fcfc77fe640 1 -- 172.21.15.22:0/3676889608 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00b0c0 con 0x555b33c3bd40 2024-09-19T08:17:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.019+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3676889608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c3bd40 msgr2=0x555b33c3e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:44.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.019+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3676889608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c3bd40 0x555b33c3e130 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0ede30 tx=0x7fcfcc033f10 comp rx=0 tx=0).stop 2024-09-19T08:17:44.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.019+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3676889608 shutdown_connections 2024-09-19T08:17:44.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.019+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3676889608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c3bd40 0x555b33c3e130 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.019+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3676889608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c31480 0x555b33c3b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.019+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3676889608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 0x555b33c30f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.019+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3676889608 >> 172.21.15.22:0/3676889608 conn(0x555b33c2d0f0 msgr2=0x555b33c2d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:44.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.019+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3676889608 shutdown_connections 2024-09-19T08:17:44.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.021+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3676889608 wait complete. 2024-09-19T08:17:44.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.022+0000 7fcfe59f3740 1 -- start start 2024-09-19T08:17:44.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.022+0000 7fcfe59f3740 1 Processor -- start 2024-09-19T08:17:44.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.023+0000 7fcfe59f3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 0x555b33c38bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.023+0000 7fcfe59f3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c31480 0x555b33c39100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.023+0000 7fcfe59f3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c3bd40 0x555b33c39640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.023+0000 7fcfe59f3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555b33c3e6d0 con 0x555b33c31480 2024-09-19T08:17:44.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.023+0000 7fcfe59f3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555b33b50f10 con 0x555b33c3bd40 2024-09-19T08:17:44.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.023+0000 7fcfe59f3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555b33c3e850 con 0x555b33c30b60 2024-09-19T08:17:44.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.023+0000 7fcfe59f3740 10 client.0 _ll_register_callbacks cb 0x555b33b19a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:44.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.024+0000 7fcfe0bb5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c31480 0x555b33c39100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:44.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.024+0000 7fcfe0bb5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c31480 0x555b33c39100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52620/0 (socket says 172.21.15.22:52620) 2024-09-19T08:17:44.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.024+0000 7fcfe0bb5640 1 -- 172.21.15.22:0/2857156065 learned_addr learned my addr 172.21.15.22:0/2857156065 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:44.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.025+0000 7fcfe0bb5640 1 -- 172.21.15.22:0/2857156065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 msgr2=0x555b33c38bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:17:44.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.025+0000 7fcfe0bb5640 1 --2- 172.21.15.22:0/2857156065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 0x555b33c38bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.025+0000 7fcfe0bb5640 1 -- 172.21.15.22:0/2857156065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c3bd40 msgr2=0x555b33c39640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:17:44.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.025+0000 7fcfe0bb5640 1 --2- 172.21.15.22:0/2857156065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c3bd40 0x555b33c39640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.025+0000 7fcfe0bb5640 1 -- 172.21.15.22:0/2857156065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b339076a0 con 0x555b33c31480 2024-09-19T08:17:44.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.025+0000 7fcfe0bb5640 1 --2- 172.21.15.22:0/2857156065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c31480 0x555b33c39100 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0b8eb0 tx=0x7fcfcc0f3f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff66043507dd43b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:44.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.026+0000 7fcfc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:17:44.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.026+0000 7fcfc4ff9640 1 -- 172.21.15.22:0/2857156065 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0bc3b0 con 0x555b33c31480 2024-09-19T08:17:44.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.026+0000 7fcfe59f3740 1 -- 172.21.15.22:0/2857156065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfcc0bd150 con 0x555b33c31480 2024-09-19T08:17:44.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.026+0000 7fcfc4ff9640 1 -- 172.21.15.22:0/2857156065 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcfcc0ee030 con 0x555b33c31480 2024-09-19T08:17:44.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.027+0000 7fcfc4ff9640 1 -- 172.21.15.22:0/2857156065 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcfcc0bc590 con 0x555b33c31480 2024-09-19T08:17:44.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.029+0000 7fcfe59f3740 10 client.15996 fetch_fsmap learned FSMap version 27 2024-09-19T08:17:44.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.029+0000 7fcfe59f3740 1 -- 172.21.15.22:0/2857156065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555b33c3e6d0 con 0x555b33c31480 2024-09-19T08:17:44.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.033+0000 7fcfc4ff9640 1 -- 172.21.15.22:0/2857156065 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfcc0bd150 con 0x555b33c31480 2024-09-19T08:17:44.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.033+0000 7fcfc4ff9640 1 -- 172.21.15.22:0/2857156065 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcfcc03a030 con 0x555b33c31480 2024-09-19T08:17:44.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.034+0000 7fcfe59f3740 10 client.15996 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:17:44.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.034+0000 7fcfe59f3740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:17:44.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.034+0000 7fcfe59f3740 20 client.15996 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:44.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.034+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/2857156065 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555b33c4b410 0x555b33c4d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.034+0000 7fcfe59f3740 4 client.15996 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:17:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.034+0000 7fcfe59f3740 1 -- 172.21.15.22:0/2857156065 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b33ba37f0 con 0x555b33c4b410 2024-09-19T08:17:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.036+0000 7fcfe13b6640 1 --2- 172.21.15.22:0/2857156065 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555b33c4b410 0x555b33c4d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.041+0000 7fcfe13b6640 1 --2- 172.21.15.22:0/2857156065 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555b33c4b410 0x555b33c4d890 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8569c055ae3587d server_cookie=8304895db19cdf76 in_seq=0 out_seq=0 2024-09-19T08:17:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.042+0000 7fcfc4ff9640 10 client.15996 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.047+0000 7fcfc4ff9640 1 -- 172.21.15.22:0/2857156065 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b33ba37f0 con 0x555b33c4b410 2024-09-19T08:17:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.047+0000 7fcfc4ff9640 10 client.15996 handle_command_reply: tid=0 2024-09-19T08:17:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.049+0000 7fcfe59f3740 1 client.15996 shutdown 2024-09-19T08:17:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.061+0000 7fcfe59f3740 1 -- 172.21.15.22:0/2857156065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c31480 msgr2=0x555b33c39100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.061+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/2857156065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c31480 0x555b33c39100 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0b8eb0 tx=0x7fcfcc0f3f30 comp rx=0 tx=0).stop 2024-09-19T08:17:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.061+0000 7fcfe59f3740 1 -- 172.21.15.22:0/2857156065 shutdown_connections 2024-09-19T08:17:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.061+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/2857156065 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555b33c4b410 0x555b33c4d890 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.061+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/2857156065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c3bd40 0x555b33c39640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.061+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/2857156065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c31480 0x555b33c39100 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.061+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/2857156065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 0x555b33c38bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.061+0000 7fcfe59f3740 1 -- 172.21.15.22:0/2857156065 >> 172.21.15.22:0/2857156065 conn(0x555b33c2d0f0 msgr2=0x555b33c32e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.062+0000 7fcfc4ff9640 0 client.15996 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.064+0000 7fcfe59f3740 1 -- 172.21.15.22:0/2857156065 shutdown_connections 2024-09-19T08:17:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.065+0000 7fcfe59f3740 1 -- 172.21.15.22:0/2857156065 wait complete. 2024-09-19T08:17:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.065+0000 7fcfe59f3740 20 client.15996 trim_cache size 0 max 16384 2024-09-19T08:17:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.113+0000 7fcfe59f3740 1 Processor -- start 2024-09-19T08:17:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.113+0000 7fcfe59f3740 1 -- start start 2024-09-19T08:17:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.113+0000 7fcfe59f3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 0x555b33c39490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.113+0000 7fcfe59f3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c31480 0x555b33c3aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.113+0000 7fcfe59f3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c3bd40 0x555b33c53160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.113+0000 7fcfe1bb7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c3bd40 0x555b33c53160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.113+0000 7fcfe1bb7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c3bd40 0x555b33c53160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52632/0 (socket says 172.21.15.22:52632) 2024-09-19T08:17:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.113+0000 7fcfe1bb7640 1 -- 172.21.15.22:0/3317019679 learned_addr learned my addr 172.21.15.22:0/3317019679 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.114+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3317019679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc00111d0 con 0x555b33c3bd40 2024-09-19T08:17:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.114+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3317019679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc000ea70 con 0x555b33c31480 2024-09-19T08:17:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.114+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3317019679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc000c8c0 con 0x555b33c30b60 2024-09-19T08:17:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.114+0000 7fcfe1bb7640 1 -- 172.21.15.22:0/3317019679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 msgr2=0x555b33c39490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:17:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.114+0000 7fcfe1bb7640 1 --2- 172.21.15.22:0/3317019679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 0x555b33c39490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.114+0000 7fcfe1bb7640 1 -- 172.21.15.22:0/3317019679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c31480 msgr2=0x555b33c3aa20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:17:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.114+0000 7fcfe1bb7640 1 --2- 172.21.15.22:0/3317019679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c31480 0x555b33c3aa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.114+0000 7fcfe1bb7640 1 -- 172.21.15.22:0/3317019679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b33c2cf20 con 0x555b33c3bd40 2024-09-19T08:17:44.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.115+0000 7fcfe1bb7640 1 --2- 172.21.15.22:0/3317019679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c3bd40 0x555b33c53160 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400fca0 tx=0x7fcfd4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8718988ca365b85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:44.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.115+0000 7fcfc6ffd640 1 -- 172.21.15.22:0/3317019679 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4026070 con 0x555b33c3bd40 2024-09-19T08:17:44.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.116+0000 7fcfc6ffd640 1 -- 172.21.15.22:0/3317019679 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd400fcd0 con 0x555b33c3bd40 2024-09-19T08:17:44.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.116+0000 7fcfc6ffd640 1 -- 172.21.15.22:0/3317019679 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd40146e0 con 0x555b33c3bd40 2024-09-19T08:17:44.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.117+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3317019679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c3bd40 msgr2=0x555b33c53160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:44.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.117+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3317019679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c3bd40 0x555b33c53160 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400fca0 tx=0x7fcfd4012000 comp rx=0 tx=0).stop 2024-09-19T08:17:44.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.117+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3317019679 shutdown_connections 2024-09-19T08:17:44.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.118+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3317019679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c3bd40 0x555b33c53160 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.118+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3317019679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c31480 0x555b33c3aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.118+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3317019679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c30b60 0x555b33c39490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.118+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3317019679 >> 172.21.15.22:0/3317019679 conn(0x555b33c2d0f0 msgr2=0x555b33c3a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:44.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.127+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3317019679 shutdown_connections 2024-09-19T08:17:44.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.134+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3317019679 wait complete. 2024-09-19T08:17:44.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.135+0000 7fcfe59f3740 1 -- start start 2024-09-19T08:17:44.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.135+0000 7fcfe59f3740 1 Processor -- start 2024-09-19T08:17:44.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.136+0000 7fcfe59f3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c30b60 0x555b33c3fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.136+0000 7fcfe13b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c30b60 0x555b33c3fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:44.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.137+0000 7fcfe13b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c30b60 0x555b33c3fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52648/0 (socket says 172.21.15.22:52648) 2024-09-19T08:17:44.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.137+0000 7fcfe13b6640 1 -- 172.21.15.22:0/3773334641 learned_addr learned my addr 172.21.15.22:0/3773334641 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:44.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.137+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c31480 0x555b33c403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.137+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c3bd40 0x555b33cb5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.137+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3773334641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc000cf30 con 0x555b33c30b60 2024-09-19T08:17:44.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.137+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3773334641 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc000e3a0 con 0x555b33c31480 2024-09-19T08:17:44.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.137+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3773334641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc000d600 con 0x555b33c3bd40 2024-09-19T08:17:44.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.137+0000 7fcfe1bb7640 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c3bd40 0x555b33cb5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:44.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.138+0000 7fcfe13b6640 1 -- 172.21.15.22:0/3773334641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c3bd40 msgr2=0x555b33cb5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:44.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.138+0000 7fcfe13b6640 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c3bd40 0x555b33cb5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.138+0000 7fcfe13b6640 1 -- 172.21.15.22:0/3773334641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c31480 msgr2=0x555b33c403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.138+0000 7fcfe13b6640 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c31480 0x555b33c403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.138+0000 7fcfe13b6640 1 -- 172.21.15.22:0/3773334641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b33cb5670 con 0x555b33c30b60 2024-09-19T08:17:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.139+0000 7fcfe1bb7640 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c3bd40 0x555b33cb5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:17:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.139+0000 7fcfe13b6640 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c30b60 0x555b33c3fe70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8005ab0 tx=0x7fcfd8012930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0acfd92f7c6fc71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.140+0000 7fcfe59f3740 10 client.0 _ll_register_callbacks cb 0x7fcfcc0f6340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.140+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3773334641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd400fcd0 con 0x555b33c30b60 2024-09-19T08:17:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.140+0000 7fcfc4ff9640 10 client.16002 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:17:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.140+0000 7fcfc4ff9640 1 -- 172.21.15.22:0/3773334641 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8004250 con 0x555b33c30b60 2024-09-19T08:17:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.140+0000 7fcfc4ff9640 1 -- 172.21.15.22:0/3773334641 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcfd80043f0 con 0x555b33c30b60 2024-09-19T08:17:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.140+0000 7fcfc4ff9640 1 -- 172.21.15.22:0/3773334641 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcfd8028050 con 0x555b33c30b60 2024-09-19T08:17:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.141+0000 7fcfe59f3740 10 client.16002 fetch_fsmap learned FSMap version 27 2024-09-19T08:17:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.142+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3773334641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=27}) -- 0x555b33cb6820 con 0x555b33c30b60 2024-09-19T08:17:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.142+0000 7fcfc4ff9640 1 -- 172.21.15.22:0/3773334641 <== mon.0 v2:172.21.15.22:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcfd800f040 con 0x555b33c30b60 2024-09-19T08:17:44.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.142+0000 7fcfe59f3740 10 client.16002 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:17:44.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.142+0000 7fcfe59f3740 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:17:44.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.142+0000 7fcfe59f3740 20 client.16002 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:44.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.142+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555b33cb6e40 0x555b33cb9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:44.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.142+0000 7fcfe59f3740 4 client.16002 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:17:44.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.142+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3773334641 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fcfd400fcd0 con 0x555b33cb6e40 2024-09-19T08:17:44.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.143+0000 7fcfe0bb5640 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555b33cb6e40 0x555b33cb9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:44.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.143+0000 7fcfe0bb5640 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555b33cb6e40 0x555b33cb9300 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d95fc582d7c4906 server_cookie=527c07a84fc6cb6a in_seq=0 out_seq=0 2024-09-19T08:17:44.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.143+0000 7fcfc4ff9640 10 client.16002 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:44.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.145+0000 7fcfc4ff9640 1 -- 172.21.15.22:0/3773334641 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fcfd400fcd0 con 0x555b33cb6e40 2024-09-19T08:17:44.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.145+0000 7fcfc4ff9640 10 client.16002 handle_command_reply: tid=0 2024-09-19T08:17:44.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.145+0000 7fcfe59f3740 1 client.16002 shutdown 2024-09-19T08:17:44.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.146+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3773334641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c30b60 msgr2=0x555b33c3fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:44.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.146+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c30b60 0x555b33c3fe70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8005ab0 tx=0x7fcfd8012930 comp rx=0 tx=0).stop 2024-09-19T08:17:44.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.146+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3773334641 shutdown_connections 2024-09-19T08:17:44.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.146+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555b33cb6e40 0x555b33cb9300 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.146+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555b33c3bd40 0x555b33cb5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.146+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b33c31480 0x555b33c403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.146+0000 7fcfe59f3740 1 --2- 172.21.15.22:0/3773334641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b33c30b60 0x555b33c3fe70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.146+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3773334641 >> 172.21.15.22:0/3773334641 conn(0x555b33c2d0f0 msgr2=0x555b33c47140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:44.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.147+0000 7fcfc4ff9640 0 client.16002 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:44.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.147+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3773334641 shutdown_connections 2024-09-19T08:17:44.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.148+0000 7fcfe59f3740 1 -- 172.21.15.22:0/3773334641 wait complete. 2024-09-19T08:17:44.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.148+0000 7fcfe59f3740 20 client.16002 trim_cache size 0 max 16384 2024-09-19T08:17:44.150 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:17:44.150 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-19T08:17:44.150 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "9746ab43-7a0d-474d-97f8-1f013de7566e", 2024-09-19T08:17:44.150 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-19T08:17:44.150 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:17:44.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.150+0000 7fcfe361a640 1 -- 172.21.15.22:0/1223010321 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcfc80a3600 msgr2=0x7fcfc80a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:44.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.150+0000 7fcfe361a640 1 --2- 172.21.15.22:0/1223010321 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcfc80a3600 0x7fcfc80a5ac0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40078b0 tx=0x7fcfd400b040 comp rx=0 tx=0).stop 2024-09-19T08:17:44.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.150+0000 7fcfe361a640 1 -- 172.21.15.22:0/1223010321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfdc101e20 msgr2=0x7fcfdc10d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:44.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.150+0000 7fcfe361a640 1 --2- 172.21.15.22:0/1223010321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfdc101e20 0x7fcfdc10d910 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc02a970 tx=0x7fcfcc004270 comp rx=0 tx=0).stop 2024-09-19T08:17:44.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.151+0000 7fcfe361a640 1 -- 172.21.15.22:0/1223010321 shutdown_connections 2024-09-19T08:17:44.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.151+0000 7fcfe361a640 1 --2- 172.21.15.22:0/1223010321 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcfc80a3600 0x7fcfc80a5ac0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.151+0000 7fcfe361a640 1 --2- 172.21.15.22:0/1223010321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfdc105de0 0x7fcfdc108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.151+0000 7fcfe361a640 1 --2- 172.21.15.22:0/1223010321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfdc101e20 0x7fcfdc10d910 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.151+0000 7fcfe361a640 1 --2- 172.21.15.22:0/1223010321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfdc101470 0x7fcfdc10d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:44.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.151+0000 7fcfe361a640 1 -- 172.21.15.22:0/1223010321 >> 172.21.15.22:0/1223010321 conn(0x7fcfdc077230 msgr2=0x7fcfdc103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:44.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.153+0000 7fcfe361a640 1 -- 172.21.15.22:0/1223010321 shutdown_connections 2024-09-19T08:17:44.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:44.153+0000 7fcfe361a640 1 -- 172.21.15.22:0/1223010321 wait complete. 2024-09-19T08:17:45.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:44 smithi022 ceph-mon[30963]: pgmap v84: 97 pgs: 97 active+clean; 94 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 134 op/s 2024-09-19T08:17:45.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:44 smithi022 ceph-mon[30963]: scrub queued for path: / 2024-09-19T08:17:45.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:44 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:45.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:44 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:17:45.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:44 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:17:45.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:44 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:45.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:44 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:17:45.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:17:45.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:44 smithi157 ceph-mon[35143]: pgmap v84: 97 pgs: 97 active+clean; 94 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 134 op/s 2024-09-19T08:17:45.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:44 smithi157 ceph-mon[35143]: scrub queued for path: / 2024-09-19T08:17:45.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:44 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:45.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:44 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:17:45.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:44 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:17:45.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:44 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:45.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:44 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:17:45.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:44 smithi064 ceph-mon[34900]: pgmap v84: 97 pgs: 97 active+clean; 94 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 134 op/s 2024-09-19T08:17:45.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:44 smithi064 ceph-mon[34900]: scrub queued for path: / 2024-09-19T08:17:45.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:44 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:45.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:44 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:17:45.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:44 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:17:45.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:44 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:45.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:44 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:17:45.537 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:17:46.877 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:45 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:46.878 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:45 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:46.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.112+0000 7fdc44cb3640 1 -- 172.21.15.22:0/2409287346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc40100b20 msgr2=0x7fdc40100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.112+0000 7fdc44cb3640 1 --2- 172.21.15.22:0/2409287346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc40100b20 0x7fdc40100f00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009900 tx=0x7fdc3002a410 comp rx=0 tx=0).stop 2024-09-19T08:17:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.112+0000 7fdc44cb3640 1 -- 172.21.15.22:0/2409287346 shutdown_connections 2024-09-19T08:17:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.112+0000 7fdc44cb3640 1 --2- 172.21.15.22:0/2409287346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc4010e500 0x7fdc40110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.112+0000 7fdc44cb3640 1 --2- 172.21.15.22:0/2409287346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc40101440 0x7fdc4010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.112+0000 7fdc44cb3640 1 --2- 172.21.15.22:0/2409287346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc40100b20 0x7fdc40100f00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.112+0000 7fdc44cb3640 1 -- 172.21.15.22:0/2409287346 >> 172.21.15.22:0/2409287346 conn(0x7fdc400fc7d0 msgr2=0x7fdc400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.112+0000 7fdc44cb3640 1 -- 172.21.15.22:0/2409287346 shutdown_connections 2024-09-19T08:17:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.112+0000 7fdc44cb3640 1 -- 172.21.15.22:0/2409287346 wait complete. 2024-09-19T08:17:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.113+0000 7fdc44cb3640 1 Processor -- start 2024-09-19T08:17:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.113+0000 7fdc44cb3640 1 -- start start 2024-09-19T08:17:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.113+0000 7fdc44cb3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40100b20 0x7fdc4019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.113+0000 7fdc44cb3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc40101440 0x7fdc4019ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.113+0000 7fdc44cb3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4010e500 0x7fdc4019e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.113+0000 7fdc44cb3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc40114280 con 0x7fdc40101440 2024-09-19T08:17:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.113+0000 7fdc44cb3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc40114100 con 0x7fdc40100b20 2024-09-19T08:17:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.113+0000 7fdc44cb3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc40114400 con 0x7fdc4010e500 2024-09-19T08:17:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.114+0000 7fdc3dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc40101440 0x7fdc4019ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.114+0000 7fdc3dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc40101440 0x7fdc4019ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52666/0 (socket says 172.21.15.22:52666) 2024-09-19T08:17:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.114+0000 7fdc3dd9b640 1 -- 172.21.15.22:0/3771565380 learned_addr learned my addr 172.21.15.22:0/3771565380 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.114+0000 7fdc3e59c640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40100b20 0x7fdc4019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.114+0000 7fdc3ed9d640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4010e500 0x7fdc4019e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.114+0000 7fdc3dd9b640 1 -- 172.21.15.22:0/3771565380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4010e500 msgr2=0x7fdc4019e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.114+0000 7fdc3dd9b640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4010e500 0x7fdc4019e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.114+0000 7fdc3dd9b640 1 -- 172.21.15.22:0/3771565380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40100b20 msgr2=0x7fdc4019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.114+0000 7fdc3dd9b640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40100b20 0x7fdc4019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.114+0000 7fdc3dd9b640 1 -- 172.21.15.22:0/3771565380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4019e730 con 0x7fdc40101440 2024-09-19T08:17:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.114+0000 7fdc3e59c640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40100b20 0x7fdc4019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.114+0000 7fdc3dd9b640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc40101440 0x7fdc4019ad20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400e9c0 tx=0x7fdc3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.114+0000 7fdc2f7fe640 1 -- 172.21.15.22:0/3771565380 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3400cd80 con 0x7fdc40101440 2024-09-19T08:17:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.114+0000 7fdc44cb3640 1 -- 172.21.15.22:0/3771565380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4019ea20 con 0x7fdc40101440 2024-09-19T08:17:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.115+0000 7fdc2f7fe640 1 -- 172.21.15.22:0/3771565380 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc340035f0 con 0x7fdc40101440 2024-09-19T08:17:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.115+0000 7fdc2f7fe640 1 -- 172.21.15.22:0/3771565380 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34010690 con 0x7fdc40101440 2024-09-19T08:17:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.115+0000 7fdc44cb3640 1 -- 172.21.15.22:0/3771565380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc401a7390 con 0x7fdc40101440 2024-09-19T08:17:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.117+0000 7fdc2f7fe640 1 -- 172.21.15.22:0/3771565380 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdc340037a0 con 0x7fdc40101440 2024-09-19T08:17:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc4708c740 1 Processor -- start 2024-09-19T08:17:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc4708c740 1 -- start start 2024-09-19T08:17:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc2f7fe640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdc280a33b0 0x7fdc280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc3ed9d640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdc280a33b0 0x7fdc280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc2f7fe640 1 -- 172.21.15.22:0/3771565380 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fdc340d0060 con 0x7fdc40101440 2024-09-19T08:17:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc4708c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55878611d4a0 0x55878611d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc4708c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x558786128140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc4708c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558786128680 0x55878612aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc4708c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5587860832e0 con 0x558786128680 2024-09-19T08:17:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc4708c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55878602ace0 con 0x55878611ddc0 2024-09-19T08:17:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc4708c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55878607f2a0 con 0x55878611d4a0 2024-09-19T08:17:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55878611d4a0 0x55878611d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc3ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558786128680 0x55878612aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc3dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x558786128140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.118+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55878611d4a0 0x55878611d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:50514/0 (socket says 172.21.15.22:50514) 2024-09-19T08:17:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.119+0000 7fdc3e59c640 1 -- 172.21.15.22:0/2581997527 learned_addr learned my addr 172.21.15.22:0/2581997527 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.119+0000 7fdc3e59c640 1 -- 172.21.15.22:0/2581997527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 msgr2=0x558786128140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.119+0000 7fdc3e59c640 1 --2- 172.21.15.22:0/2581997527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x558786128140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.119+0000 7fdc3e59c640 1 -- 172.21.15.22:0/2581997527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558786128680 msgr2=0x55878612aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.119+0000 7fdc3e59c640 1 --2- 172.21.15.22:0/2581997527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558786128680 0x55878612aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.119+0000 7fdc3e59c640 1 -- 172.21.15.22:0/2581997527 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55878600f440 con 0x55878611d4a0 2024-09-19T08:17:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.119+0000 7fdc3dd9b640 1 --2- 172.21.15.22:0/2581997527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x558786128140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.119+0000 7fdc3ed9d640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdc280a33b0 0x7fdc280a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdc40069760 tx=0x7fdc38013ae0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.119+0000 7fdc3e59c640 1 --2- 172.21.15.22:0/2581997527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55878611d4a0 0x55878611d880 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009900 tx=0x7fdc30005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51c259da962a231f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.119+0000 7fdc2cff9640 1 -- 172.21.15.22:0/2581997527 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30035040 con 0x55878611d4a0 2024-09-19T08:17:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.119+0000 7fdc2cff9640 1 -- 172.21.15.22:0/2581997527 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc30031830 con 0x55878611d4a0 2024-09-19T08:17:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.119+0000 7fdc2cff9640 1 -- 172.21.15.22:0/2581997527 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc300047a0 con 0x55878611d4a0 2024-09-19T08:17:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.120+0000 7fdc4708c740 1 -- 172.21.15.22:0/2581997527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55878611d4a0 msgr2=0x55878611d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.120+0000 7fdc4708c740 1 --2- 172.21.15.22:0/2581997527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55878611d4a0 0x55878611d880 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009900 tx=0x7fdc30005d50 comp rx=0 tx=0).stop 2024-09-19T08:17:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.120+0000 7fdc4708c740 1 -- 172.21.15.22:0/2581997527 shutdown_connections 2024-09-19T08:17:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.120+0000 7fdc4708c740 1 --2- 172.21.15.22:0/2581997527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558786128680 0x55878612aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.120+0000 7fdc4708c740 1 --2- 172.21.15.22:0/2581997527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x558786128140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.120+0000 7fdc4708c740 1 --2- 172.21.15.22:0/2581997527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55878611d4a0 0x55878611d880 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.120+0000 7fdc4708c740 1 -- 172.21.15.22:0/2581997527 >> 172.21.15.22:0/2581997527 conn(0x558786119a30 msgr2=0x558786119e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.120+0000 7fdc4708c740 1 -- 172.21.15.22:0/2581997527 shutdown_connections 2024-09-19T08:17:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.120+0000 7fdc4708c740 1 -- 172.21.15.22:0/2581997527 wait complete. 2024-09-19T08:17:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.121+0000 7fdc4708c740 1 -- start start 2024-09-19T08:17:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.121+0000 7fdc4708c740 1 Processor -- start 2024-09-19T08:17:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.121+0000 7fdc4708c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x558786127e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.121+0000 7fdc4708c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x558786125120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.121+0000 7fdc4708c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558786128680 0x558786125660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.121+0000 7fdc4708c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55878612b250 con 0x55878611d4a0 2024-09-19T08:17:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.121+0000 7fdc4708c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55878603e050 con 0x55878611ddc0 2024-09-19T08:17:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.121+0000 7fdc4708c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55878612b3d0 con 0x558786128680 2024-09-19T08:17:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.121+0000 7fdc4708c740 10 client.0 _ll_register_callbacks cb 0x55878607e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.121+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x558786127e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.121+0000 7fdc3dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x558786125120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.121+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x558786127e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52678/0 (socket says 172.21.15.22:52678) 2024-09-19T08:17:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.121+0000 7fdc3e59c640 1 -- 172.21.15.22:0/490378201 learned_addr learned my addr 172.21.15.22:0/490378201 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:46.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.121+0000 7fdc3ed9d640 1 --2- 172.21.15.22:0/490378201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558786128680 0x558786125660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.122+0000 7fdc3dd9b640 1 -- 172.21.15.22:0/490378201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558786128680 msgr2=0x558786125660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:46.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.122+0000 7fdc3dd9b640 1 --2- 172.21.15.22:0/490378201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558786128680 0x558786125660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.122+0000 7fdc3dd9b640 1 -- 172.21.15.22:0/490378201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 msgr2=0x558786127e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:46.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.122+0000 7fdc3dd9b640 1 --2- 172.21.15.22:0/490378201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x558786127e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.122+0000 7fdc3dd9b640 1 -- 172.21.15.22:0/490378201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558785df43c0 con 0x55878611ddc0 2024-09-19T08:17:46.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.122+0000 7fdc3dd9b640 1 --2- 172.21.15.22:0/490378201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x558786125120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc340cd7c0 tx=0x7fdc3409cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f6302fe9d2977b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:46.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.122+0000 7fdc25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:17:46.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.122+0000 7fdc25ffb640 1 -- 172.21.15.22:0/490378201 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc340a4fc0 con 0x55878611ddc0 2024-09-19T08:17:46.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.122+0000 7fdc25ffb640 1 -- 172.21.15.22:0/490378201 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fdc340b52a0 con 0x55878611ddc0 2024-09-19T08:17:46.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.122+0000 7fdc4708c740 1 -- 172.21.15.22:0/490378201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc30031830 con 0x55878611ddc0 2024-09-19T08:17:46.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.122+0000 7fdc25ffb640 1 -- 172.21.15.22:0/490378201 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc340a4b30 con 0x55878611ddc0 2024-09-19T08:17:46.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.122+0000 7fdc4708c740 10 client.25631 fetch_fsmap learned FSMap version 27 2024-09-19T08:17:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.122+0000 7fdc4708c740 1 -- 172.21.15.22:0/490378201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558786136b50 con 0x55878611ddc0 2024-09-19T08:17:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.122+0000 7fdc25ffb640 1 -- 172.21.15.22:0/490378201 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc30031830 con 0x55878611ddc0 2024-09-19T08:17:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.123+0000 7fdc25ffb640 1 -- 172.21.15.22:0/490378201 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdc34004000 con 0x55878611ddc0 2024-09-19T08:17:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.123+0000 7fdc4708c740 10 client.25631 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:17:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.123+0000 7fdc4708c740 10 client.25631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:17:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.123+0000 7fdc4708c740 20 client.25631 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.123+0000 7fdc4708c740 1 --2- 172.21.15.22:0/490378201 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558786138010 0x55878613a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.123+0000 7fdc4708c740 4 client.25631 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:17:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.123+0000 7fdc4708c740 1 -- 172.21.15.22:0/490378201 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558786090850 con 0x558786138010 2024-09-19T08:17:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.123+0000 7fdc3e59c640 1 --2- 172.21.15.22:0/490378201 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558786138010 0x55878613a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.123+0000 7fdc3e59c640 1 --2- 172.21.15.22:0/490378201 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558786138010 0x55878613a4d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab6dddc2463f1d1c server_cookie=a97deff6f0357d72 in_seq=0 out_seq=0 2024-09-19T08:17:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.123+0000 7fdc25ffb640 10 client.25631 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.124+0000 7fdc25ffb640 1 -- 172.21.15.22:0/490378201 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558786090850 con 0x558786138010 2024-09-19T08:17:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.124+0000 7fdc25ffb640 10 client.25631 handle_command_reply: tid=0 2024-09-19T08:17:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.124+0000 7fdc4708c740 1 client.25631 shutdown 2024-09-19T08:17:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.125+0000 7fdc4708c740 1 -- 172.21.15.22:0/490378201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 msgr2=0x558786125120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.125+0000 7fdc4708c740 1 --2- 172.21.15.22:0/490378201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x558786125120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc340cd7c0 tx=0x7fdc3409cb80 comp rx=0 tx=0).stop 2024-09-19T08:17:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.125+0000 7fdc4708c740 1 -- 172.21.15.22:0/490378201 shutdown_connections 2024-09-19T08:17:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.125+0000 7fdc4708c740 1 --2- 172.21.15.22:0/490378201 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558786138010 0x55878613a4d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.125+0000 7fdc4708c740 1 --2- 172.21.15.22:0/490378201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558786128680 0x558786125660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.125+0000 7fdc4708c740 1 --2- 172.21.15.22:0/490378201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x558786125120 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.125+0000 7fdc4708c740 1 --2- 172.21.15.22:0/490378201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x558786127e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.125+0000 7fdc4708c740 1 -- 172.21.15.22:0/490378201 >> 172.21.15.22:0/490378201 conn(0x558786119a30 msgr2=0x558786118b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.125+0000 7fdc25ffb640 0 client.25631 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.125+0000 7fdc4708c740 1 -- 172.21.15.22:0/490378201 shutdown_connections 2024-09-19T08:17:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.125+0000 7fdc4708c740 1 -- 172.21.15.22:0/490378201 wait complete. 2024-09-19T08:17:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.125+0000 7fdc4708c740 20 client.25631 trim_cache size 0 max 16384 2024-09-19T08:17:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.144+0000 7fdc4708c740 1 Processor -- start 2024-09-19T08:17:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.144+0000 7fdc4708c740 1 -- start start 2024-09-19T08:17:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.144+0000 7fdc4708c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x558786125310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.144+0000 7fdc4708c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x5587861207f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.144+0000 7fdc4708c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558786128680 0x55878613fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.144+0000 7fdc4708c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc401b23d0 con 0x55878611d4a0 2024-09-19T08:17:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.144+0000 7fdc4708c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc401afc70 con 0x55878611ddc0 2024-09-19T08:17:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.144+0000 7fdc4708c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc401adac0 con 0x558786128680 2024-09-19T08:17:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.144+0000 7fdc3dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x5587861207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.144+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x558786125310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.144+0000 7fdc3dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x5587861207f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:45722/0 (socket says 172.21.15.22:45722) 2024-09-19T08:17:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.145+0000 7fdc3dd9b640 1 -- 172.21.15.22:0/4263938852 learned_addr learned my addr 172.21.15.22:0/4263938852 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.145+0000 7fdc3ed9d640 1 --2- 172.21.15.22:0/4263938852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558786128680 0x55878613fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.145+0000 7fdc3dd9b640 1 -- 172.21.15.22:0/4263938852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558786128680 msgr2=0x55878613fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.145+0000 7fdc3dd9b640 1 --2- 172.21.15.22:0/4263938852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558786128680 0x55878613fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.145+0000 7fdc3dd9b640 1 -- 172.21.15.22:0/4263938852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 msgr2=0x558786125310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.145+0000 7fdc3dd9b640 1 --2- 172.21.15.22:0/4263938852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x558786125310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.145+0000 7fdc3dd9b640 1 -- 172.21.15.22:0/4263938852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587861240d0 con 0x55878611ddc0 2024-09-19T08:17:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.145+0000 7fdc3e59c640 1 --2- 172.21.15.22:0/4263938852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x558786125310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.145+0000 7fdc3dd9b640 1 --2- 172.21.15.22:0/4263938852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x5587861207f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc34098d20 tx=0x7fdc34098720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d79265543a39792 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.145+0000 7fdc27fff640 1 -- 172.21.15.22:0/4263938852 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc340a4fc0 con 0x55878611ddc0 2024-09-19T08:17:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.145+0000 7fdc27fff640 1 -- 172.21.15.22:0/4263938852 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc340a4cd0 con 0x55878611ddc0 2024-09-19T08:17:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.145+0000 7fdc27fff640 1 -- 172.21.15.22:0/4263938852 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3409ba00 con 0x55878611ddc0 2024-09-19T08:17:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.146+0000 7fdc4708c740 1 -- 172.21.15.22:0/4263938852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 msgr2=0x5587861207f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.146+0000 7fdc4708c740 1 --2- 172.21.15.22:0/4263938852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x5587861207f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc34098d20 tx=0x7fdc34098720 comp rx=0 tx=0).stop 2024-09-19T08:17:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.146+0000 7fdc4708c740 1 -- 172.21.15.22:0/4263938852 shutdown_connections 2024-09-19T08:17:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.146+0000 7fdc4708c740 1 --2- 172.21.15.22:0/4263938852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558786128680 0x55878613fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.146+0000 7fdc4708c740 1 --2- 172.21.15.22:0/4263938852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55878611ddc0 0x5587861207f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.146+0000 7fdc4708c740 1 --2- 172.21.15.22:0/4263938852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x558786125310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.146+0000 7fdc4708c740 1 -- 172.21.15.22:0/4263938852 >> 172.21.15.22:0/4263938852 conn(0x558786119a30 msgr2=0x5587861200e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.146+0000 7fdc4708c740 1 -- 172.21.15.22:0/4263938852 shutdown_connections 2024-09-19T08:17:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.146+0000 7fdc4708c740 1 -- 172.21.15.22:0/4263938852 wait complete. 2024-09-19T08:17:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.146+0000 7fdc4708c740 1 -- start start 2024-09-19T08:17:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.147+0000 7fdc4708c740 1 Processor -- start 2024-09-19T08:17:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.147+0000 7fdc4708c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x55878611fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.147+0000 7fdc4708c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55878611ddc0 0x558786120300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.147+0000 7fdc4708c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558786128680 0x558786120840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.147+0000 7fdc4708c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc401ae130 con 0x55878611d4a0 2024-09-19T08:17:46.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.147+0000 7fdc4708c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc401af5a0 con 0x558786128680 2024-09-19T08:17:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.147+0000 7fdc4708c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc401ae800 con 0x55878611ddc0 2024-09-19T08:17:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.147+0000 7fdc4708c740 10 client.0 _ll_register_callbacks cb 0x7fdc3409c120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:17:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.147+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x55878611fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.147+0000 7fdc3ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558786128680 0x558786120840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.147+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x55878611fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52700/0 (socket says 172.21.15.22:52700) 2024-09-19T08:17:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.147+0000 7fdc3e59c640 1 -- 172.21.15.22:0/3905336750 learned_addr learned my addr 172.21.15.22:0/3905336750 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:17:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.147+0000 7fdc3e59c640 1 -- 172.21.15.22:0/3905336750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55878611ddc0 msgr2=0x558786120300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:17:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc3dd9b640 1 --2- 172.21.15.22:0/3905336750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55878611ddc0 0x558786120300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc3e59c640 1 --2- 172.21.15.22:0/3905336750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55878611ddc0 0x558786120300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc3e59c640 1 -- 172.21.15.22:0/3905336750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558786128680 msgr2=0x558786120840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:17:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc3e59c640 1 --2- 172.21.15.22:0/3905336750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558786128680 0x558786120840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc3e59c640 1 -- 172.21.15.22:0/3905336750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55878612edf0 con 0x55878611d4a0 2024-09-19T08:17:46.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc3ed9d640 1 --2- 172.21.15.22:0/3905336750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558786128680 0x558786120840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:17:46.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc3e59c640 1 --2- 172.21.15.22:0/3905336750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x55878611fdc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdc30033000 tx=0x7fdc30005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bab8fb50a21e9d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:17:46.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:17:46.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc25ffb640 1 -- 172.21.15.22:0/3905336750 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30035040 con 0x55878611d4a0 2024-09-19T08:17:46.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc4708c740 1 -- 172.21.15.22:0/3905336750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc340a4cd0 con 0x55878611d4a0 2024-09-19T08:17:46.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc25ffb640 1 -- 172.21.15.22:0/3905336750 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fdc30004040 con 0x55878611d4a0 2024-09-19T08:17:46.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc25ffb640 1 -- 172.21.15.22:0/3905336750 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc30062050 con 0x55878611d4a0 2024-09-19T08:17:46.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc4708c740 10 client.16020 fetch_fsmap learned FSMap version 27 2024-09-19T08:17:46.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc4708c740 1 -- 172.21.15.22:0/3905336750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5587861a3130 con 0x55878611d4a0 2024-09-19T08:17:46.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.148+0000 7fdc25ffb640 1 -- 172.21.15.22:0/3905336750 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc340a4cd0 con 0x55878611d4a0 2024-09-19T08:17:46.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.149+0000 7fdc25ffb640 1 -- 172.21.15.22:0/3905336750 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdc3005d030 con 0x55878611d4a0 2024-09-19T08:17:46.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.149+0000 7fdc4708c740 10 client.16020 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:17:46.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.149+0000 7fdc4708c740 10 client.16020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:17:46.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.149+0000 7fdc4708c740 20 client.16020 populate_metadata read hostname 'smithi022' 2024-09-19T08:17:46.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.149+0000 7fdc4708c740 1 --2- 172.21.15.22:0/3905336750 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5587861a3490 0x5587861a5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:17:46.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.149+0000 7fdc4708c740 4 client.16020 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:17:46.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.149+0000 7fdc4708c740 1 -- 172.21.15.22:0/3905336750 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587861259f0 con 0x5587861a3490 2024-09-19T08:17:46.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.149+0000 7fdc3dd9b640 1 --2- 172.21.15.22:0/3905336750 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5587861a3490 0x5587861a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:17:46.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.149+0000 7fdc3dd9b640 1 --2- 172.21.15.22:0/3905336750 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5587861a3490 0x5587861a5880 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47bc3049091ca048 server_cookie=4dd807258c1b97c9 in_seq=0 out_seq=0 2024-09-19T08:17:46.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.149+0000 7fdc25ffb640 10 client.16020 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:46.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc25ffb640 1 -- 172.21.15.22:0/3905336750 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5587861259f0 con 0x5587861a3490 2024-09-19T08:17:46.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc25ffb640 10 client.16020 handle_command_reply: tid=0 2024-09-19T08:17:46.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc4708c740 1 client.16020 shutdown 2024-09-19T08:17:46.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc4708c740 1 -- 172.21.15.22:0/3905336750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 msgr2=0x55878611fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:46.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc4708c740 1 --2- 172.21.15.22:0/3905336750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x55878611fdc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdc30033000 tx=0x7fdc30005dc0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc4708c740 1 -- 172.21.15.22:0/3905336750 shutdown_connections 2024-09-19T08:17:46.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc4708c740 1 --2- 172.21.15.22:0/3905336750 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5587861a3490 0x5587861a5880 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc4708c740 1 --2- 172.21.15.22:0/3905336750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558786128680 0x558786120840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc4708c740 1 --2- 172.21.15.22:0/3905336750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55878611ddc0 0x558786120300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc25ffb640 0 client.16020 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:17:46.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc4708c740 1 --2- 172.21.15.22:0/3905336750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55878611d4a0 0x55878611fdc0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc4708c740 1 -- 172.21.15.22:0/3905336750 >> 172.21.15.22:0/3905336750 conn(0x558786119a30 msgr2=0x558786118b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:46.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc4708c740 1 -- 172.21.15.22:0/3905336750 shutdown_connections 2024-09-19T08:17:46.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc4708c740 1 -- 172.21.15.22:0/3905336750 wait complete. 2024-09-19T08:17:46.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.150+0000 7fdc4708c740 20 client.16020 trim_cache size 0 max 16384 2024-09-19T08:17:46.907 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:17:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:17:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-19T08:17:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: "9746ab43-7a0d-474d-97f8-1f013de7566e": { 2024-09-19T08:17:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-19T08:17:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "9746ab43-7a0d-474d-97f8-1f013de7566e", 2024-09-19T08:17:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-09-19T08:17:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:17:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:17:46.908 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:17:46.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.151+0000 7fdc44cb3640 1 -- 172.21.15.22:0/3771565380 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdc280a33b0 msgr2=0x7fdc280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:46.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.151+0000 7fdc44cb3640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdc280a33b0 0x7fdc280a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdc40069760 tx=0x7fdc38013ae0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.151+0000 7fdc44cb3640 1 -- 172.21.15.22:0/3771565380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc40101440 msgr2=0x7fdc4019ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:17:46.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.151+0000 7fdc44cb3640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc40101440 0x7fdc4019ad20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400e9c0 tx=0x7fdc3400ee80 comp rx=0 tx=0).stop 2024-09-19T08:17:46.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.151+0000 7fdc44cb3640 1 -- 172.21.15.22:0/3771565380 shutdown_connections 2024-09-19T08:17:46.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.151+0000 7fdc44cb3640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdc280a33b0 0x7fdc280a5870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.151+0000 7fdc44cb3640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc4010e500 0x7fdc4019e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.151+0000 7fdc44cb3640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc40101440 0x7fdc4019ad20 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.151+0000 7fdc44cb3640 1 --2- 172.21.15.22:0/3771565380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40100b20 0x7fdc4019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:17:46.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.151+0000 7fdc44cb3640 1 -- 172.21.15.22:0/3771565380 >> 172.21.15.22:0/3771565380 conn(0x7fdc400fc7d0 msgr2=0x7fdc400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:17:46.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.151+0000 7fdc44cb3640 1 -- 172.21.15.22:0/3771565380 shutdown_connections 2024-09-19T08:17:46.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:17:46.151+0000 7fdc44cb3640 1 -- 172.21.15.22:0/3771565380 wait complete. 2024-09-19T08:17:46.910 INFO:tasks.workunit.client.0.smithi022.stdout: 44 1291474 503 723089 672 528647 1595 2024-09-19T08:17:46.910 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:45 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:47.024 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:46 smithi022 ceph-mon[30963]: pgmap v85: 97 pgs: 97 active+clean; 94 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 86 op/s 2024-09-19T08:17:47.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:46 smithi157 ceph-mon[35143]: pgmap v85: 97 pgs: 97 active+clean; 94 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 86 op/s 2024-09-19T08:17:47.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:46 smithi064 ceph-mon[34900]: pgmap v85: 97 pgs: 97 active+clean; 94 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 86 op/s 2024-09-19T08:17:47.385 INFO:tasks.cephfs.filesystem:scrub status for tag:9746ab43-7a0d-474d-97f8-1f013de7566e - {'path': '/', 'tag': '9746ab43-7a0d-474d-97f8-1f013de7566e', 'options': 'recursive,force'} 2024-09-19T08:17:49.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:48 smithi157 ceph-mon[35143]: pgmap v86: 97 pgs: 97 active+clean; 96 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 106 op/s 2024-09-19T08:17:49.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:48 smithi022 ceph-mon[30963]: pgmap v86: 97 pgs: 97 active+clean; 96 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 106 op/s 2024-09-19T08:17:49.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:48 smithi064 ceph-mon[34900]: pgmap v86: 97 pgs: 97 active+clean; 96 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 106 op/s 2024-09-19T08:17:51.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:50 smithi157 ceph-mon[35143]: pgmap v87: 97 pgs: 97 active+clean; 135 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 221 op/s 2024-09-19T08:17:51.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:50 smithi022 ceph-mon[30963]: pgmap v87: 97 pgs: 97 active+clean; 135 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 221 op/s 2024-09-19T08:17:51.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:50 smithi064 ceph-mon[34900]: pgmap v87: 97 pgs: 97 active+clean; 135 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 221 op/s 2024-09-19T08:17:53.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:52 smithi157 ceph-mon[35143]: pgmap v88: 97 pgs: 97 active+clean; 135 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 221 op/s 2024-09-19T08:17:53.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:52 smithi022 ceph-mon[30963]: pgmap v88: 97 pgs: 97 active+clean; 135 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 221 op/s 2024-09-19T08:17:53.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:52 smithi064 ceph-mon[34900]: pgmap v88: 97 pgs: 97 active+clean; 135 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 221 op/s 2024-09-19T08:17:55.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:54 smithi157 ceph-mon[35143]: pgmap v89: 97 pgs: 97 active+clean; 172 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.5 MiB/s wr, 550 op/s 2024-09-19T08:17:55.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:54 smithi022 ceph-mon[30963]: pgmap v89: 97 pgs: 97 active+clean; 172 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.5 MiB/s wr, 550 op/s 2024-09-19T08:17:55.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:54 smithi064 ceph-mon[34900]: pgmap v89: 97 pgs: 97 active+clean; 172 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.5 MiB/s wr, 550 op/s 2024-09-19T08:17:56.156 INFO:tasks.workunit.client.0.smithi022.stdout: 48 823639 347 492119 123 370848 691 2024-09-19T08:17:56.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:55 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:56.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:55 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:56.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:55 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:17:57.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:56 smithi157 ceph-mon[35143]: pgmap v90: 97 pgs: 97 active+clean; 172 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 514 op/s 2024-09-19T08:17:57.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:56 smithi022 ceph-mon[30963]: pgmap v90: 97 pgs: 97 active+clean; 172 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 514 op/s 2024-09-19T08:17:57.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:56 smithi064 ceph-mon[34900]: pgmap v90: 97 pgs: 97 active+clean; 172 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 514 op/s 2024-09-19T08:17:59.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:17:58 smithi157 ceph-mon[35143]: pgmap v91: 97 pgs: 97 active+clean; 176 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.0 MiB/s wr, 554 op/s 2024-09-19T08:17:59.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:17:58 smithi022 ceph-mon[30963]: pgmap v91: 97 pgs: 97 active+clean; 176 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.0 MiB/s wr, 554 op/s 2024-09-19T08:17:59.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:17:58 smithi064 ceph-mon[34900]: pgmap v91: 97 pgs: 97 active+clean; 176 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.0 MiB/s wr, 554 op/s 2024-09-19T08:18:01.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:00 smithi157 ceph-mon[35143]: pgmap v92: 97 pgs: 97 active+clean; 228 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 863 op/s 2024-09-19T08:18:01.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:00 smithi022 ceph-mon[30963]: pgmap v92: 97 pgs: 97 active+clean; 228 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 863 op/s 2024-09-19T08:18:01.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:00 smithi064 ceph-mon[34900]: pgmap v92: 97 pgs: 97 active+clean; 228 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 863 op/s 2024-09-19T08:18:02.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:01 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:18:02.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:01 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:18:02.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:01 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:18:03.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:02 smithi157 ceph-mon[35143]: pgmap v93: 97 pgs: 97 active+clean; 228 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.3 MiB/s wr, 697 op/s 2024-09-19T08:18:03.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:02 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:18:03.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:02 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:18:03.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:02 smithi022 ceph-mon[30963]: pgmap v93: 97 pgs: 97 active+clean; 228 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.3 MiB/s wr, 697 op/s 2024-09-19T08:18:03.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:02 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:18:03.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:02 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:18:03.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:02 smithi064 ceph-mon[34900]: pgmap v93: 97 pgs: 97 active+clean; 228 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.3 MiB/s wr, 697 op/s 2024-09-19T08:18:03.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:02 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:18:03.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:02 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:18:05.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:04 smithi157 ceph-mon[35143]: pgmap v94: 97 pgs: 97 active+clean; 236 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 10 MiB/s wr, 766 op/s 2024-09-19T08:18:05.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:04 smithi022 ceph-mon[30963]: pgmap v94: 97 pgs: 97 active+clean; 236 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 10 MiB/s wr, 766 op/s 2024-09-19T08:18:05.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:04 smithi064 ceph-mon[34900]: pgmap v94: 97 pgs: 97 active+clean; 236 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 10 MiB/s wr, 766 op/s 2024-09-19T08:18:06.154 INFO:tasks.workunit.client.0.smithi022.stdout: 62 670426 616 383695 561 310272 1730 2024-09-19T08:18:07.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:06 smithi157 ceph-mon[35143]: pgmap v95: 97 pgs: 97 active+clean; 236 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 437 op/s 2024-09-19T08:18:07.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:06 smithi022 ceph-mon[30963]: pgmap v95: 97 pgs: 97 active+clean; 236 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 437 op/s 2024-09-19T08:18:07.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:06 smithi064 ceph-mon[34900]: pgmap v95: 97 pgs: 97 active+clean; 236 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 437 op/s 2024-09-19T08:18:08.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:07 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:18:08.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:07 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:18:08.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:07 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:18:09.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:08 smithi157 ceph-mon[35143]: pgmap v96: 97 pgs: 97 active+clean; 236 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 445 op/s 2024-09-19T08:18:09.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:08 smithi022 ceph-mon[30963]: pgmap v96: 97 pgs: 97 active+clean; 236 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 445 op/s 2024-09-19T08:18:09.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:08 smithi064 ceph-mon[34900]: pgmap v96: 97 pgs: 97 active+clean; 236 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 445 op/s 2024-09-19T08:18:11.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:10 smithi157 ceph-mon[35143]: pgmap v97: 97 pgs: 97 active+clean; 247 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.8 MiB/s wr, 449 op/s 2024-09-19T08:18:11.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:10 smithi022 ceph-mon[30963]: pgmap v97: 97 pgs: 97 active+clean; 247 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.8 MiB/s wr, 449 op/s 2024-09-19T08:18:11.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:10 smithi064 ceph-mon[34900]: pgmap v97: 97 pgs: 97 active+clean; 247 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.8 MiB/s wr, 449 op/s 2024-09-19T08:18:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:12 smithi157 ceph-mon[35143]: pgmap v98: 97 pgs: 97 active+clean; 247 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 120 op/s 2024-09-19T08:18:13.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:12 smithi022 ceph-mon[30963]: pgmap v98: 97 pgs: 97 active+clean; 247 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 120 op/s 2024-09-19T08:18:13.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:12 smithi064 ceph-mon[34900]: pgmap v98: 97 pgs: 97 active+clean; 247 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 120 op/s 2024-09-19T08:18:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:13 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:18:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:13 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:18:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:13 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:18:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:13 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:18:14.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:14 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:18:14.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:14 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:18:14.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:14 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:18:14.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:14 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:18:14.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:13 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:18:14.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:13 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:18:14.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:13 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:18:14.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:13 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:18:15.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:14 smithi157 ceph-mon[35143]: pgmap v99: 97 pgs: 97 active+clean; 258 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.9 MiB/s wr, 358 op/s 2024-09-19T08:18:15.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:14 smithi022 ceph-mon[30963]: pgmap v99: 97 pgs: 97 active+clean; 258 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.9 MiB/s wr, 358 op/s 2024-09-19T08:18:15.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:14 smithi064 ceph-mon[34900]: pgmap v99: 97 pgs: 97 active+clean; 258 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.9 MiB/s wr, 358 op/s 2024-09-19T08:18:16.154 INFO:tasks.workunit.client.0.smithi022.stdout: 74 936338 560 543653 553 453027 955 2024-09-19T08:18:17.257 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:17 smithi157 ceph-mon[35143]: pgmap v100: 97 pgs: 97 active+clean; 258 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.9 MiB/s wr, 289 op/s 2024-09-19T08:18:17.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:17 smithi022 ceph-mon[30963]: pgmap v100: 97 pgs: 97 active+clean; 258 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.9 MiB/s wr, 289 op/s 2024-09-19T08:18:17.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:17 smithi064 ceph-mon[34900]: pgmap v100: 97 pgs: 97 active+clean; 258 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.9 MiB/s wr, 289 op/s 2024-09-19T08:18:17.386 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:18:17.741 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:18:18.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.347+0000 7fe3044a1640 1 -- 172.21.15.22:0/3918252629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2fc074590 msgr2=0x7fe2fc074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:18.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.347+0000 7fe3044a1640 1 --2- 172.21.15.22:0/3918252629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2fc074590 0x7fe2fc074970 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009ca0 tx=0x7fe2ec02a950 comp rx=0 tx=0).stop 2024-09-19T08:18:18.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.347+0000 7fe3044a1640 1 -- 172.21.15.22:0/3918252629 shutdown_connections 2024-09-19T08:18:18.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.347+0000 7fe3044a1640 1 --2- 172.21.15.22:0/3918252629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2fc112b90 0x7fe2fc114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.347+0000 7fe3044a1640 1 --2- 172.21.15.22:0/3918252629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2fc074eb0 0x7fe2fc112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.347+0000 7fe3044a1640 1 --2- 172.21.15.22:0/3918252629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2fc074590 0x7fe2fc074970 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.347+0000 7fe3044a1640 1 -- 172.21.15.22:0/3918252629 >> 172.21.15.22:0/3918252629 conn(0x7fe2fc0fd8c0 msgr2=0x7fe2fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:18.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.347+0000 7fe3044a1640 1 -- 172.21.15.22:0/3918252629 shutdown_connections 2024-09-19T08:18:18.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.347+0000 7fe3044a1640 1 -- 172.21.15.22:0/3918252629 wait complete. 2024-09-19T08:18:18.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.347+0000 7fe3044a1640 1 Processor -- start 2024-09-19T08:18:18.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.347+0000 7fe3044a1640 1 -- start start 2024-09-19T08:18:18.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe3044a1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2fc074590 0x7fe2fc1a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe3044a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2fc074eb0 0x7fe2fc1a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe3044a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2fc112b90 0x7fe2fc19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe3044a1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc118570 con 0x7fe2fc112b90 2024-09-19T08:18:18.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe3044a1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1183f0 con 0x7fe2fc074590 2024-09-19T08:18:18.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe3044a1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1186f0 con 0x7fe2fc074eb0 2024-09-19T08:18:18.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe302a3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2fc112b90 0x7fe2fc19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe30223d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2fc074590 0x7fe2fc1a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe302a3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2fc112b90 0x7fe2fc19e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60934/0 (socket says 172.21.15.22:60934) 2024-09-19T08:18:18.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe301a3c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2fc074eb0 0x7fe2fc1a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe302a3e640 1 -- 172.21.15.22:0/4054113558 learned_addr learned my addr 172.21.15.22:0/4054113558 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:18.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe302a3e640 1 -- 172.21.15.22:0/4054113558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2fc074eb0 msgr2=0x7fe2fc1a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:18.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe302a3e640 1 --2- 172.21.15.22:0/4054113558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2fc074eb0 0x7fe2fc1a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe302a3e640 1 -- 172.21.15.22:0/4054113558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2fc074590 msgr2=0x7fe2fc1a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:18.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe302a3e640 1 --2- 172.21.15.22:0/4054113558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2fc074590 0x7fe2fc1a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe302a3e640 1 -- 172.21.15.22:0/4054113558 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc19f010 con 0x7fe2fc112b90 2024-09-19T08:18:18.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe30223d640 1 --2- 172.21.15.22:0/4054113558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2fc074590 0x7fe2fc1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:18.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe302a3e640 1 --2- 172.21.15.22:0/4054113558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2fc112b90 0x7fe2fc19e8e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400e970 tx=0x7fe2f400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:18.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.348+0000 7fe2fb7fe640 1 -- 172.21.15.22:0/4054113558 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f400ccc0 con 0x7fe2fc112b90 2024-09-19T08:18:18.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.349+0000 7fe2fb7fe640 1 -- 172.21.15.22:0/4054113558 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f4005500 con 0x7fe2fc112b90 2024-09-19T08:18:18.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.349+0000 7fe3044a1640 1 -- 172.21.15.22:0/4054113558 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2fc19f300 con 0x7fe2fc112b90 2024-09-19T08:18:18.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.349+0000 7fe2fb7fe640 1 -- 172.21.15.22:0/4054113558 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4010690 con 0x7fe2fc112b90 2024-09-19T08:18:18.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.349+0000 7fe3044a1640 1 -- 172.21.15.22:0/4054113558 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2fc1ab5f0 con 0x7fe2fc112b90 2024-09-19T08:18:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.351+0000 7fe2fb7fe640 1 -- 172.21.15.22:0/4054113558 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe2f40046c0 con 0x7fe2fc112b90 2024-09-19T08:18:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe30687a740 1 Processor -- start 2024-09-19T08:18:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe30687a740 1 -- start start 2024-09-19T08:18:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe30687a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de54a0 0x55c1c7de5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe30687a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de5dc0 0x55c1c7df0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe30687a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7df0680 0x55c1c7df2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe30687a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c1c7d4b2e0 con 0x55c1c7de54a0 2024-09-19T08:18:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe30687a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c1c7cf2ce0 con 0x55c1c7df0680 2024-09-19T08:18:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe30687a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c1c7d472a0 con 0x55c1c7de5dc0 2024-09-19T08:18:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe2fb7fe640 1 --2- 172.21.15.22:0/4054113558 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe2e40a33b0 0x7fe2e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe301a3c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de54a0 0x55c1c7de5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe301a3c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de54a0 0x55c1c7de5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60950/0 (socket says 172.21.15.22:60950) 2024-09-19T08:18:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe301a3c640 1 -- 172.21.15.22:0/135345702 learned_addr learned my addr 172.21.15.22:0/135345702 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe30223d640 1 --2- 172.21.15.22:0/135345702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de5dc0 0x55c1c7df0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe302a3e640 1 --2- 172.21.15.22:0/4054113558 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe2e40a33b0 0x7fe2e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.352+0000 7fe2fb7fe640 1 -- 172.21.15.22:0/4054113558 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe2f40d0080 con 0x7fe2fc112b90 2024-09-19T08:18:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.353+0000 7fe301a3c640 1 -- 172.21.15.22:0/135345702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de5dc0 msgr2=0x55c1c7df0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.353+0000 7fe301a3c640 1 --2- 172.21.15.22:0/135345702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de5dc0 0x55c1c7df0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.353+0000 7fe301a3c640 1 -- 172.21.15.22:0/135345702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7df0680 msgr2=0x55c1c7df2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.353+0000 7fe301a3c640 1 --2- 172.21.15.22:0/135345702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7df0680 0x55c1c7df2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.353+0000 7fe301a3c640 1 -- 172.21.15.22:0/135345702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1c7cd7440 con 0x55c1c7de54a0 2024-09-19T08:18:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.353+0000 7fe301a3c640 1 --2- 172.21.15.22:0/135345702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de54a0 0x55c1c7de5880 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009d00 tx=0x7fe2f0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f1c98ff20b4890 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.353+0000 7fe302a3e640 1 --2- 172.21.15.22:0/4054113558 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe2e40a33b0 0x7fe2e40a5870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40987d0 tx=0x7fe2f40bc040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.353+0000 7fe2f8ff9640 1 -- 172.21.15.22:0/135345702 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0004250 con 0x55c1c7de54a0 2024-09-19T08:18:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.353+0000 7fe2f8ff9640 1 -- 172.21.15.22:0/135345702 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f00043f0 con 0x55c1c7de54a0 2024-09-19T08:18:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.353+0000 7fe2f8ff9640 1 -- 172.21.15.22:0/135345702 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0004e60 con 0x55c1c7de54a0 2024-09-19T08:18:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.354+0000 7fe30687a740 1 -- 172.21.15.22:0/135345702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de54a0 msgr2=0x55c1c7de5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.354+0000 7fe30687a740 1 --2- 172.21.15.22:0/135345702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de54a0 0x55c1c7de5880 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009d00 tx=0x7fe2f0011810 comp rx=0 tx=0).stop 2024-09-19T08:18:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.354+0000 7fe30687a740 1 -- 172.21.15.22:0/135345702 shutdown_connections 2024-09-19T08:18:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.354+0000 7fe30687a740 1 --2- 172.21.15.22:0/135345702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7df0680 0x55c1c7df2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.354+0000 7fe30687a740 1 --2- 172.21.15.22:0/135345702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de5dc0 0x55c1c7df0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.354+0000 7fe30687a740 1 --2- 172.21.15.22:0/135345702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de54a0 0x55c1c7de5880 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.354+0000 7fe30687a740 1 -- 172.21.15.22:0/135345702 >> 172.21.15.22:0/135345702 conn(0x55c1c7de1a30 msgr2=0x55c1c7de1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.354+0000 7fe30687a740 1 -- 172.21.15.22:0/135345702 shutdown_connections 2024-09-19T08:18:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.354+0000 7fe30687a740 1 -- 172.21.15.22:0/135345702 wait complete. 2024-09-19T08:18:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.354+0000 7fe30687a740 1 -- start start 2024-09-19T08:18:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.354+0000 7fe30687a740 1 Processor -- start 2024-09-19T08:18:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe30687a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de54a0 0x55c1c7deff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe30687a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7de5dc0 0x55c1c7ded240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe30687a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7df0680 0x55c1c7ded780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe30687a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c1c7df3250 con 0x55c1c7df0680 2024-09-19T08:18:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe30687a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c1c7d06050 con 0x55c1c7de5dc0 2024-09-19T08:18:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe30687a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c1c7df33d0 con 0x55c1c7de54a0 2024-09-19T08:18:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe30687a740 10 client.0 _ll_register_callbacks cb 0x55c1c7d46160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe301a3c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de54a0 0x55c1c7deff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe30223d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7de5dc0 0x55c1c7ded240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe301a3c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de54a0 0x55c1c7deff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42810/0 (socket says 172.21.15.22:42810) 2024-09-19T08:18:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe301a3c640 1 -- 172.21.15.22:0/3054748963 learned_addr learned my addr 172.21.15.22:0/3054748963 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe301a3c640 1 --2- 172.21.15.22:0/3054748963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7df0680 0x55c1c7ded780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe30223d640 1 -- 172.21.15.22:0/3054748963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de54a0 msgr2=0x55c1c7deff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe30223d640 1 --2- 172.21.15.22:0/3054748963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de54a0 0x55c1c7deff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe30223d640 1 -- 172.21.15.22:0/3054748963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7df0680 msgr2=0x55c1c7ded780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe30223d640 1 --2- 172.21.15.22:0/3054748963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7df0680 0x55c1c7ded780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe30223d640 1 -- 172.21.15.22:0/3054748963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1c7abc3c0 con 0x55c1c7de5dc0 2024-09-19T08:18:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe301a3c640 1 --2- 172.21.15.22:0/3054748963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de54a0 0x55c1c7deff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe30223d640 1 --2- 172.21.15.22:0/3054748963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7de5dc0 0x55c1c7ded240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc19fa20 tx=0x7fe2ec009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c5652721d263462 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.355+0000 7fe2e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:18:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.356+0000 7fe2e1ffb640 1 -- 172.21.15.22:0/3054748963 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec049060 con 0x55c1c7de5dc0 2024-09-19T08:18:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.356+0000 7fe30687a740 1 -- 172.21.15.22:0/3054748963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2f00043f0 con 0x55c1c7de5dc0 2024-09-19T08:18:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.356+0000 7fe2e1ffb640 1 -- 172.21.15.22:0/3054748963 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe2ec003d80 con 0x55c1c7de5dc0 2024-09-19T08:18:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.356+0000 7fe2e1ffb640 1 -- 172.21.15.22:0/3054748963 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe2ec034040 con 0x55c1c7de5dc0 2024-09-19T08:18:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.356+0000 7fe30687a740 10 client.25646 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.356+0000 7fe30687a740 1 -- 172.21.15.22:0/3054748963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c1c7dfec20 con 0x55c1c7de5dc0 2024-09-19T08:18:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.356+0000 7fe2e1ffb640 1 -- 172.21.15.22:0/3054748963 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f00043f0 con 0x55c1c7de5dc0 2024-09-19T08:18:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.356+0000 7fe2e1ffb640 1 -- 172.21.15.22:0/3054748963 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fe2ec0306a0 con 0x55c1c7de5dc0 2024-09-19T08:18:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.356+0000 7fe30687a740 10 client.25646 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.356+0000 7fe30687a740 10 client.25646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:18:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.356+0000 7fe30687a740 20 client.25646 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.356+0000 7fe30687a740 1 --2- 172.21.15.22:0/3054748963 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c1c7e00030 0x55c1c7e02450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.356+0000 7fe30687a740 4 client.25646 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:18:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.356+0000 7fe30687a740 1 -- 172.21.15.22:0/3054748963 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1c7d58850 con 0x55c1c7e00030 2024-09-19T08:18:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.357+0000 7fe301a3c640 1 --2- 172.21.15.22:0/3054748963 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c1c7e00030 0x55c1c7e02450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.357+0000 7fe301a3c640 1 --2- 172.21.15.22:0/3054748963 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c1c7e00030 0x55c1c7e02450 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14943fe3cfba61c server_cookie=3ad95ea73e45d295 in_seq=0 out_seq=0 2024-09-19T08:18:18.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.357+0000 7fe2e1ffb640 10 client.25646 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:18.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe2e1ffb640 1 -- 172.21.15.22:0/3054748963 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1c7d58850 con 0x55c1c7e00030 2024-09-19T08:18:18.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe2e1ffb640 10 client.25646 handle_command_reply: tid=0 2024-09-19T08:18:18.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe30687a740 1 client.25646 shutdown 2024-09-19T08:18:18.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe30687a740 1 -- 172.21.15.22:0/3054748963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7de5dc0 msgr2=0x55c1c7ded240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:18.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe30687a740 1 --2- 172.21.15.22:0/3054748963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7de5dc0 0x55c1c7ded240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc19fa20 tx=0x7fe2ec009670 comp rx=0 tx=0).stop 2024-09-19T08:18:18.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe30687a740 1 -- 172.21.15.22:0/3054748963 shutdown_connections 2024-09-19T08:18:18.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe30687a740 1 --2- 172.21.15.22:0/3054748963 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c1c7e00030 0x55c1c7e02450 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe30687a740 1 --2- 172.21.15.22:0/3054748963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7df0680 0x55c1c7ded780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe30687a740 1 --2- 172.21.15.22:0/3054748963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7de5dc0 0x55c1c7ded240 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe30687a740 1 --2- 172.21.15.22:0/3054748963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de54a0 0x55c1c7deff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe30687a740 1 -- 172.21.15.22:0/3054748963 >> 172.21.15.22:0/3054748963 conn(0x55c1c7de1a30 msgr2=0x55c1c7de7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:18.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe2e1ffb640 0 client.25646 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe30687a740 1 -- 172.21.15.22:0/3054748963 shutdown_connections 2024-09-19T08:18:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe30687a740 1 -- 172.21.15.22:0/3054748963 wait complete. 2024-09-19T08:18:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.358+0000 7fe30687a740 20 client.25646 trim_cache size 0 max 16384 2024-09-19T08:18:18.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.376+0000 7fe30687a740 1 Processor -- start 2024-09-19T08:18:18.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.376+0000 7fe30687a740 1 -- start start 2024-09-19T08:18:18.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.376+0000 7fe30687a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de54a0 0x55c1c7dfc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.376+0000 7fe30687a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de5dc0 0x55c1c7e07d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.376+0000 7fe30687a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7df0680 0x55c1c7e08240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.376+0000 7fe30687a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1b67f0 con 0x55c1c7de5dc0 2024-09-19T08:18:18.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.376+0000 7fe30687a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1b4060 con 0x55c1c7df0680 2024-09-19T08:18:18.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.376+0000 7fe30687a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1b1eb0 con 0x55c1c7de54a0 2024-09-19T08:18:18.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.376+0000 7fe301a3c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de54a0 0x55c1c7dfc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.376+0000 7fe30223d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de5dc0 0x55c1c7e07d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.376+0000 7fe301a3c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de54a0 0x55c1c7dfc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42824/0 (socket says 172.21.15.22:42824) 2024-09-19T08:18:18.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.376+0000 7fe301a3c640 1 -- 172.21.15.22:0/4034964376 learned_addr learned my addr 172.21.15.22:0/4034964376 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:18.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.377+0000 7fe301a3c640 1 --2- 172.21.15.22:0/4034964376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7df0680 0x55c1c7e08240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.377+0000 7fe30223d640 1 -- 172.21.15.22:0/4034964376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de54a0 msgr2=0x55c1c7dfc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:18.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.377+0000 7fe30223d640 1 --2- 172.21.15.22:0/4034964376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de54a0 0x55c1c7dfc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.377+0000 7fe30223d640 1 -- 172.21.15.22:0/4034964376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7df0680 msgr2=0x55c1c7e08240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:18.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.377+0000 7fe30223d640 1 --2- 172.21.15.22:0/4034964376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7df0680 0x55c1c7e08240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.377+0000 7fe30223d640 1 -- 172.21.15.22:0/4034964376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1c7defbb0 con 0x55c1c7de5dc0 2024-09-19T08:18:18.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.377+0000 7fe30223d640 1 --2- 172.21.15.22:0/4034964376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de5dc0 0x55c1c7e07d00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec002230 tx=0x7fe2ec02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9d689ae80aef27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:18.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.377+0000 7fe2e3fff640 1 -- 172.21.15.22:0/4034964376 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec049040 con 0x55c1c7de5dc0 2024-09-19T08:18:18.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.377+0000 7fe2e3fff640 1 -- 172.21.15.22:0/4034964376 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2ec030640 con 0x55c1c7de5dc0 2024-09-19T08:18:18.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.377+0000 7fe2e3fff640 1 -- 172.21.15.22:0/4034964376 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec03aca0 con 0x55c1c7de5dc0 2024-09-19T08:18:18.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.378+0000 7fe30687a740 1 -- 172.21.15.22:0/4034964376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de5dc0 msgr2=0x55c1c7e07d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:18.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.378+0000 7fe30687a740 1 --2- 172.21.15.22:0/4034964376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de5dc0 0x55c1c7e07d00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec002230 tx=0x7fe2ec02a950 comp rx=0 tx=0).stop 2024-09-19T08:18:18.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.378+0000 7fe30687a740 1 -- 172.21.15.22:0/4034964376 shutdown_connections 2024-09-19T08:18:18.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.378+0000 7fe30687a740 1 --2- 172.21.15.22:0/4034964376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7df0680 0x55c1c7e08240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.378+0000 7fe30687a740 1 --2- 172.21.15.22:0/4034964376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de5dc0 0x55c1c7e07d00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.378+0000 7fe30687a740 1 --2- 172.21.15.22:0/4034964376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7de54a0 0x55c1c7dfc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.378+0000 7fe30687a740 1 -- 172.21.15.22:0/4034964376 >> 172.21.15.22:0/4034964376 conn(0x55c1c7de1a30 msgr2=0x55c1c7de8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:18.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.378+0000 7fe30687a740 1 -- 172.21.15.22:0/4034964376 shutdown_connections 2024-09-19T08:18:18.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.378+0000 7fe30687a740 1 -- 172.21.15.22:0/4034964376 wait complete. 2024-09-19T08:18:18.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe30687a740 1 -- start start 2024-09-19T08:18:18.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe30687a740 1 Processor -- start 2024-09-19T08:18:18.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe30687a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de54a0 0x55c1c7df8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe30687a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7de5dc0 0x55c1c7df3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe30687a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7df0680 0x55c1c7df4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe30687a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1b2520 con 0x55c1c7de54a0 2024-09-19T08:18:18.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe30687a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1b3990 con 0x55c1c7de5dc0 2024-09-19T08:18:18.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe30687a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1b2bf0 con 0x55c1c7df0680 2024-09-19T08:18:18.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe30687a740 10 client.0 _ll_register_callbacks cb 0x7fe2ec009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:18.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe301a3c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de54a0 0x55c1c7df8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe301a3c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de54a0 0x55c1c7df8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60984/0 (socket says 172.21.15.22:60984) 2024-09-19T08:18:18.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe301a3c640 1 -- 172.21.15.22:0/1712827853 learned_addr learned my addr 172.21.15.22:0/1712827853 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:18.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe30223d640 1 --2- 172.21.15.22:0/1712827853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7de5dc0 0x55c1c7df3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe301a3c640 1 --2- 172.21.15.22:0/1712827853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7df0680 0x55c1c7df4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe301a3c640 1 -- 172.21.15.22:0/1712827853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7df0680 msgr2=0x55c1c7df4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:18.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.379+0000 7fe301a3c640 1 --2- 172.21.15.22:0/1712827853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7df0680 0x55c1c7df4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.380+0000 7fe301a3c640 1 -- 172.21.15.22:0/1712827853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7de5dc0 msgr2=0x55c1c7df3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:18.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.380+0000 7fe301a3c640 1 --2- 172.21.15.22:0/1712827853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7de5dc0 0x55c1c7df3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.380+0000 7fe301a3c640 1 -- 172.21.15.22:0/1712827853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1c7df4b10 con 0x55c1c7de54a0 2024-09-19T08:18:18.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.380+0000 7fe30223d640 1 --2- 172.21.15.22:0/1712827853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7de5dc0 0x55c1c7df3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:18.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.380+0000 7fe301a3c640 1 --2- 172.21.15.22:0/1712827853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de54a0 0x55c1c7df8c00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00058c0 tx=0x7fe2f00049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10b3e2b6e140476 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:18.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.380+0000 7fe2e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:18:18.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.380+0000 7fe2e1ffb640 1 -- 172.21.15.22:0/1712827853 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f003b060 con 0x55c1c7de54a0 2024-09-19T08:18:18.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.380+0000 7fe2e1ffb640 1 -- 172.21.15.22:0/1712827853 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe2f00052d0 con 0x55c1c7de54a0 2024-09-19T08:18:18.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.380+0000 7fe30687a740 1 -- 172.21.15.22:0/1712827853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2ec030640 con 0x55c1c7de54a0 2024-09-19T08:18:18.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.380+0000 7fe2e1ffb640 1 -- 172.21.15.22:0/1712827853 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe2f00054d0 con 0x55c1c7de54a0 2024-09-19T08:18:18.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.380+0000 7fe30687a740 10 client.16038 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:18.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.380+0000 7fe30687a740 1 -- 172.21.15.22:0/1712827853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c1c7e6abd0 con 0x55c1c7de54a0 2024-09-19T08:18:18.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.380+0000 7fe2e1ffb640 1 -- 172.21.15.22:0/1712827853 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2ec030640 con 0x55c1c7de54a0 2024-09-19T08:18:18.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.381+0000 7fe2e1ffb640 1 -- 172.21.15.22:0/1712827853 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fe2f0019430 con 0x55c1c7de54a0 2024-09-19T08:18:18.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.381+0000 7fe30687a740 10 client.16038 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:18.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.381+0000 7fe30687a740 10 client.16038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:18:18.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.381+0000 7fe30687a740 20 client.16038 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:18.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.381+0000 7fe30687a740 1 --2- 172.21.15.22:0/1712827853 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c1c7e6b1c0 0x55c1c7e6d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:18.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.381+0000 7fe30687a740 4 client.16038 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:18:18.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.381+0000 7fe30687a740 1 -- 172.21.15.22:0/1712827853 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c1c7d52d60 con 0x55c1c7e6b1c0 2024-09-19T08:18:18.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.381+0000 7fe30223d640 1 --2- 172.21.15.22:0/1712827853 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c1c7e6b1c0 0x55c1c7e6d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:18.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.381+0000 7fe2e1ffb640 10 client.16038 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:18.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.381+0000 7fe30223d640 1 --2- 172.21.15.22:0/1712827853 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c1c7e6b1c0 0x55c1c7e6d5e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be675344e97d3171 server_cookie=74c56c0cc75da2c8 in_seq=0 out_seq=0 2024-09-19T08:18:18.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.381+0000 7fe2e1ffb640 1 -- 172.21.15.22:0/1712827853 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c1c7d52d60 con 0x55c1c7e6b1c0 2024-09-19T08:18:18.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.381+0000 7fe2e1ffb640 10 client.16038 handle_command_reply: tid=0 2024-09-19T08:18:18.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe30687a740 1 client.16038 shutdown 2024-09-19T08:18:18.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe30687a740 1 -- 172.21.15.22:0/1712827853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de54a0 msgr2=0x55c1c7df8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:18.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe30687a740 1 --2- 172.21.15.22:0/1712827853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de54a0 0x55c1c7df8c00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00058c0 tx=0x7fe2f00049c0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe30687a740 1 -- 172.21.15.22:0/1712827853 shutdown_connections 2024-09-19T08:18:18.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe30687a740 1 --2- 172.21.15.22:0/1712827853 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55c1c7e6b1c0 0x55c1c7e6d5e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe30687a740 1 --2- 172.21.15.22:0/1712827853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1c7df0680 0x55c1c7df4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe30687a740 1 --2- 172.21.15.22:0/1712827853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1c7de5dc0 0x55c1c7df3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe30687a740 1 --2- 172.21.15.22:0/1712827853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c1c7de54a0 0x55c1c7df8c00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe30687a740 1 -- 172.21.15.22:0/1712827853 >> 172.21.15.22:0/1712827853 conn(0x55c1c7de1a30 msgr2=0x55c1c7de0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:18.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe2e1ffb640 0 client.16038 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:18.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe30687a740 1 -- 172.21.15.22:0/1712827853 shutdown_connections 2024-09-19T08:18:18.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe30687a740 1 -- 172.21.15.22:0/1712827853 wait complete. 2024-09-19T08:18:18.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe30687a740 20 client.16038 trim_cache size 0 max 16384 2024-09-19T08:18:18.387 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:18:18.387 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-19T08:18:18.387 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-09-19T08:18:18.387 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:18:18.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe3044a1640 1 -- 172.21.15.22:0/4054113558 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe2e40a33b0 msgr2=0x7fe2e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:18.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe3044a1640 1 --2- 172.21.15.22:0/4054113558 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe2e40a33b0 0x7fe2e40a5870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40987d0 tx=0x7fe2f40bc040 comp rx=0 tx=0).stop 2024-09-19T08:18:18.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe3044a1640 1 -- 172.21.15.22:0/4054113558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2fc112b90 msgr2=0x7fe2fc19e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:18.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.382+0000 7fe3044a1640 1 --2- 172.21.15.22:0/4054113558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2fc112b90 0x7fe2fc19e8e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400e970 tx=0x7fe2f400ee30 comp rx=0 tx=0).stop 2024-09-19T08:18:18.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.383+0000 7fe3044a1640 1 -- 172.21.15.22:0/4054113558 shutdown_connections 2024-09-19T08:18:18.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.383+0000 7fe3044a1640 1 --2- 172.21.15.22:0/4054113558 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe2e40a33b0 0x7fe2e40a5870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.383+0000 7fe3044a1640 1 --2- 172.21.15.22:0/4054113558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2fc112b90 0x7fe2fc19e8e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.383+0000 7fe3044a1640 1 --2- 172.21.15.22:0/4054113558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2fc074eb0 0x7fe2fc1a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.383+0000 7fe3044a1640 1 --2- 172.21.15.22:0/4054113558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2fc074590 0x7fe2fc1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:18.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.383+0000 7fe3044a1640 1 -- 172.21.15.22:0/4054113558 >> 172.21.15.22:0/4054113558 conn(0x7fe2fc0fd8c0 msgr2=0x7fe2fc0ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:18.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.383+0000 7fe3044a1640 1 -- 172.21.15.22:0/4054113558 shutdown_connections 2024-09-19T08:18:18.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:18.383+0000 7fe3044a1640 1 -- 172.21.15.22:0/4054113558 wait complete. 2024-09-19T08:18:19.185 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:19 smithi022 ceph-mon[30963]: pgmap v101: 97 pgs: 97 active+clean; 261 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2024-09-19T08:18:19.245 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:18:19.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:18:19.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:19 smithi064 ceph-mon[34900]: pgmap v101: 97 pgs: 97 active+clean; 261 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2024-09-19T08:18:19.506 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:19 smithi157 ceph-mon[35143]: pgmap v101: 97 pgs: 97 active+clean; 261 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2024-09-19T08:18:19.790 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:18:20.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.303+0000 7f514f2f8640 1 -- 172.21.15.22:0/2471413981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5148100a80 msgr2=0x7f5148100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:20.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.303+0000 7f514f2f8640 1 --2- 172.21.15.22:0/2471413981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5148100a80 0x7f5148100ee0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5138009930 tx=0x7f513802a420 comp rx=0 tx=0).stop 2024-09-19T08:18:20.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.303+0000 7f514f2f8640 1 -- 172.21.15.22:0/2471413981 shutdown_connections 2024-09-19T08:18:20.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.303+0000 7f514f2f8640 1 --2- 172.21.15.22:0/2471413981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5148101420 0x7f514810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:20.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.303+0000 7f514f2f8640 1 --2- 172.21.15.22:0/2471413981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5148100a80 0x7f5148100ee0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:20.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.303+0000 7f514f2f8640 1 --2- 172.21.15.22:0/2471413981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5148106a80 0x7f5148106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:20.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.303+0000 7f514f2f8640 1 -- 172.21.15.22:0/2471413981 >> 172.21.15.22:0/2471413981 conn(0x7f51480fc7a0 msgr2=0x7f51480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:20.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.303+0000 7f514f2f8640 1 -- 172.21.15.22:0/2471413981 shutdown_connections 2024-09-19T08:18:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.303+0000 7f514f2f8640 1 -- 172.21.15.22:0/2471413981 wait complete. 2024-09-19T08:18:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.304+0000 7f514f2f8640 1 Processor -- start 2024-09-19T08:18:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.304+0000 7f514f2f8640 1 -- start start 2024-09-19T08:18:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.304+0000 7f514f2f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5148100a80 0x7f5148198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.304+0000 7f514f2f8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5148101420 0x7f5148198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.304+0000 7f514f2f8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5148106a80 0x7f514819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:20.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.304+0000 7f514f2f8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5148110e80 con 0x7f5148101420 2024-09-19T08:18:20.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.304+0000 7f514f2f8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5148110d00 con 0x7f5148106a80 2024-09-19T08:18:20.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.304+0000 7f514f2f8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5148111000 con 0x7f5148100a80 2024-09-19T08:18:20.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.304+0000 7f514c893640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5148101420 0x7f5148198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:20.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.304+0000 7f514d094640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5148100a80 0x7f5148198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:20.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.304+0000 7f514d094640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5148100a80 0x7f5148198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42870/0 (socket says 172.21.15.22:42870) 2024-09-19T08:18:20.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.304+0000 7f514d895640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5148106a80 0x7f514819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:20.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.304+0000 7f514d094640 1 -- 172.21.15.22:0/218135971 learned_addr learned my addr 172.21.15.22:0/218135971 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.305+0000 7f514d895640 1 -- 172.21.15.22:0/218135971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5148100a80 msgr2=0x7f5148198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.305+0000 7f514d895640 1 --2- 172.21.15.22:0/218135971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5148100a80 0x7f5148198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.305+0000 7f514d895640 1 -- 172.21.15.22:0/218135971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5148101420 msgr2=0x7f5148198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.305+0000 7f514d895640 1 --2- 172.21.15.22:0/218135971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5148101420 0x7f5148198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.305+0000 7f514d895640 1 -- 172.21.15.22:0/218135971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514819d3b0 con 0x7f5148106a80 2024-09-19T08:18:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.305+0000 7f514d094640 1 --2- 172.21.15.22:0/218135971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5148100a80 0x7f5148198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.305+0000 7f514d895640 1 --2- 172.21.15.22:0/218135971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5148106a80 0x7f514819ccd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5140009fd0 tx=0x7f514000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.305+0000 7f513e7fc640 1 -- 172.21.15.22:0/218135971 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514000ce20 con 0x7f5148106a80 2024-09-19T08:18:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.305+0000 7f513e7fc640 1 -- 172.21.15.22:0/218135971 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5140005020 con 0x7f5148106a80 2024-09-19T08:18:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.305+0000 7f514f2f8640 1 -- 172.21.15.22:0/218135971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514819d6a0 con 0x7f5148106a80 2024-09-19T08:18:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.305+0000 7f513e7fc640 1 -- 172.21.15.22:0/218135971 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5140010720 con 0x7f5148106a80 2024-09-19T08:18:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.305+0000 7f514f2f8640 1 -- 172.21.15.22:0/218135971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51481a4f80 con 0x7f5148106a80 2024-09-19T08:18:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.306+0000 7f514f2f8640 1 -- 172.21.15.22:0/218135971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5148100ee0 con 0x7f5148106a80 2024-09-19T08:18:20.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.307+0000 7f513e7fc640 1 -- 172.21.15.22:0/218135971 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5140005540 con 0x7f5148106a80 2024-09-19T08:18:20.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.308+0000 7f513e7fc640 1 --2- 172.21.15.22:0/218135971 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f51340a33b0 0x7f51340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:20.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.308+0000 7f514d094640 1 --2- 172.21.15.22:0/218135971 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f51340a33b0 0x7f51340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:20.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.308+0000 7f513e7fc640 1 -- 172.21.15.22:0/218135971 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f51400d0400 con 0x7f5148106a80 2024-09-19T08:18:20.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.309+0000 7f514d094640 1 --2- 172.21.15.22:0/218135971 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f51340a33b0 0x7f51340a5870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5144011440 tx=0x7f5144005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:20.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.311+0000 7f513e7fc640 1 -- 172.21.15.22:0/218135971 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f514009cd90 con 0x7f5148106a80 2024-09-19T08:18:20.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.529+0000 7f514f2f8640 1 -- 172.21.15.22:0/218135971 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5148199760 con 0x7f5148106a80 2024-09-19T08:18:20.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.530+0000 7f513e7fc640 1 -- 172.21.15.22:0/218135971 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11910 (secure 0 0 0) 0x7f5148199760 con 0x7f5148106a80 2024-09-19T08:18:20.530 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:18:20.531 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"btime":"2024-09-19T08:18:06:960387+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:18:06.784015+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:18:20.531 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 28 2024-09-19T08:18:20.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.533+0000 7f514f2f8640 1 -- 172.21.15.22:0/218135971 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f51340a33b0 msgr2=0x7f51340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:20.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.533+0000 7f514f2f8640 1 --2- 172.21.15.22:0/218135971 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f51340a33b0 0x7f51340a5870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5144011440 tx=0x7f5144005f90 comp rx=0 tx=0).stop 2024-09-19T08:18:20.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.533+0000 7f514f2f8640 1 -- 172.21.15.22:0/218135971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5148106a80 msgr2=0x7f514819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:20.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.533+0000 7f514f2f8640 1 --2- 172.21.15.22:0/218135971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5148106a80 0x7f514819ccd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5140009fd0 tx=0x7f514000ef10 comp rx=0 tx=0).stop 2024-09-19T08:18:20.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.533+0000 7f514f2f8640 1 -- 172.21.15.22:0/218135971 shutdown_connections 2024-09-19T08:18:20.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.533+0000 7f514f2f8640 1 --2- 172.21.15.22:0/218135971 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f51340a33b0 0x7f51340a5870 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:20.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.533+0000 7f514f2f8640 1 --2- 172.21.15.22:0/218135971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5148106a80 0x7f514819ccd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:20.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.533+0000 7f514f2f8640 1 --2- 172.21.15.22:0/218135971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5148101420 0x7f5148198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:20.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.533+0000 7f514f2f8640 1 --2- 172.21.15.22:0/218135971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5148100a80 0x7f5148198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:20.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.533+0000 7f514f2f8640 1 -- 172.21.15.22:0/218135971 >> 172.21.15.22:0/218135971 conn(0x7f51480fc7a0 msgr2=0x7f514806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:20.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.533+0000 7f514f2f8640 1 -- 172.21.15.22:0/218135971 shutdown_connections 2024-09-19T08:18:20.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:20.533+0000 7f514f2f8640 1 -- 172.21.15.22:0/218135971 wait complete. 2024-09-19T08:18:21.245 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:21 smithi022 ceph-mon[30963]: pgmap v102: 97 pgs: 97 active+clean; 309 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.5 MiB/s wr, 557 op/s 2024-09-19T08:18:21.245 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:21 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/218135971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:18:21.274 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:18:21.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:21 smithi064 ceph-mon[34900]: pgmap v102: 97 pgs: 97 active+clean; 309 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.5 MiB/s wr, 557 op/s 2024-09-19T08:18:21.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:21 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/218135971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:18:21.502 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:21 smithi157 ceph-mon[35143]: pgmap v102: 97 pgs: 97 active+clean; 309 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.5 MiB/s wr, 557 op/s 2024-09-19T08:18:21.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:21 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/218135971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:18:21.723 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:18:22.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.320+0000 7ff53d7ba640 1 -- 172.21.15.22:0/1304315603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff538100a70 msgr2=0x7ff538100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:22.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.320+0000 7ff53d7ba640 1 --2- 172.21.15.22:0/1304315603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff538100a70 0x7ff538100ed0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009930 tx=0x7ff52c02a420 comp rx=0 tx=0).stop 2024-09-19T08:18:22.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.320+0000 7ff53d7ba640 1 -- 172.21.15.22:0/1304315603 shutdown_connections 2024-09-19T08:18:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.320+0000 7ff53d7ba640 1 --2- 172.21.15.22:0/1304315603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff538101410 0x7ff53810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.320+0000 7ff53d7ba640 1 --2- 172.21.15.22:0/1304315603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff538100a70 0x7ff538100ed0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.320+0000 7ff53d7ba640 1 --2- 172.21.15.22:0/1304315603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff538106a70 0x7ff538106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.320+0000 7ff53d7ba640 1 -- 172.21.15.22:0/1304315603 >> 172.21.15.22:0/1304315603 conn(0x7ff5380fc7d0 msgr2=0x7ff5380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.320+0000 7ff53d7ba640 1 -- 172.21.15.22:0/1304315603 shutdown_connections 2024-09-19T08:18:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.320+0000 7ff53d7ba640 1 -- 172.21.15.22:0/1304315603 wait complete. 2024-09-19T08:18:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.321+0000 7ff53d7ba640 1 Processor -- start 2024-09-19T08:18:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.321+0000 7ff53d7ba640 1 -- start start 2024-09-19T08:18:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.321+0000 7ff53d7ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff538100a70 0x7ff538197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.321+0000 7ff53d7ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff538101410 0x7ff538197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.321+0000 7ff53d7ba640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff538106a70 0x7ff538197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.321+0000 7ff53d7ba640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff538110e70 con 0x7ff538106a70 2024-09-19T08:18:22.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.321+0000 7ff53d7ba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff538110cf0 con 0x7ff538100a70 2024-09-19T08:18:22.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.321+0000 7ff53d7ba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff538110ff0 con 0x7ff538101410 2024-09-19T08:18:22.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.321+0000 7ff5377fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff538106a70 0x7ff538197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.321+0000 7ff5367fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff538101410 0x7ff538197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.321+0000 7ff5377fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff538106a70 0x7ff538197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32790/0 (socket says 172.21.15.22:32790) 2024-09-19T08:18:22.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.321+0000 7ff5377fe640 1 -- 172.21.15.22:0/2393894974 learned_addr learned my addr 172.21.15.22:0/2393894974 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:22.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.321+0000 7ff536ffd640 1 --2- 172.21.15.22:0/2393894974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff538100a70 0x7ff538197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.322+0000 7ff5377fe640 1 -- 172.21.15.22:0/2393894974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff538101410 msgr2=0x7ff538197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:22.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.322+0000 7ff5377fe640 1 --2- 172.21.15.22:0/2393894974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff538101410 0x7ff538197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.322+0000 7ff5377fe640 1 -- 172.21.15.22:0/2393894974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff538100a70 msgr2=0x7ff538197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:22.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.322+0000 7ff5377fe640 1 --2- 172.21.15.22:0/2393894974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff538100a70 0x7ff538197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.322+0000 7ff5377fe640 1 -- 172.21.15.22:0/2393894974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5381a6980 con 0x7ff538106a70 2024-09-19T08:18:22.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.322+0000 7ff5377fe640 1 --2- 172.21.15.22:0/2393894974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff538106a70 0x7ff538197b90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff53000e970 tx=0x7ff53000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:22.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.322+0000 7ff51ffff640 1 -- 172.21.15.22:0/2393894974 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53000ccc0 con 0x7ff538106a70 2024-09-19T08:18:22.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.322+0000 7ff53d7ba640 1 -- 172.21.15.22:0/2393894974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5381a6c70 con 0x7ff538106a70 2024-09-19T08:18:22.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.322+0000 7ff51ffff640 1 -- 172.21.15.22:0/2393894974 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff53000ce60 con 0x7ff538106a70 2024-09-19T08:18:22.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.322+0000 7ff51ffff640 1 -- 172.21.15.22:0/2393894974 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530010450 con 0x7ff538106a70 2024-09-19T08:18:22.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.322+0000 7ff53d7ba640 1 -- 172.21.15.22:0/2393894974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5381a7180 con 0x7ff538106a70 2024-09-19T08:18:22.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.324+0000 7ff51ffff640 1 -- 172.21.15.22:0/2393894974 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff530005020 con 0x7ff538106a70 2024-09-19T08:18:22.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff53fb93740 1 Processor -- start 2024-09-19T08:18:22.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff53fb93740 1 -- start start 2024-09-19T08:18:22.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff53fb93740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 0x557741b12090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff53fb93740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b12a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff53fb93740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 0x557741b1f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff53fb93740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557741a782f0 con 0x557741b125d0 2024-09-19T08:18:22.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff53fb93740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557741a1fcf0 con 0x557741b13b40 2024-09-19T08:18:22.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff53fb93740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557741a742b0 con 0x557741b1d370 2024-09-19T08:18:22.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff51ffff640 1 --2- 172.21.15.22:0/2393894974 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff5200a33b0 0x7ff5200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff536ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b12a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff536ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b12a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32792/0 (socket says 172.21.15.22:32792) 2024-09-19T08:18:22.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff536ffd640 1 -- 172.21.15.22:0/898032945 learned_addr learned my addr 172.21.15.22:0/898032945 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:22.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff5367fc640 1 --2- 172.21.15.22:0/898032945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 0x557741b1f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff5377fe640 1 --2- 172.21.15.22:0/2393894974 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff5200a33b0 0x7ff5200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff5367fc640 1 --2- 172.21.15.22:0/898032945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 0x557741b12090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.325+0000 7ff51ffff640 1 -- 172.21.15.22:0/2393894974 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff5300d0100 con 0x7ff538106a70 2024-09-19T08:18:22.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.326+0000 7ff536ffd640 1 -- 172.21.15.22:0/898032945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 msgr2=0x557741b1f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:22.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.326+0000 7ff536ffd640 1 --2- 172.21.15.22:0/898032945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 0x557741b1f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.326+0000 7ff536ffd640 1 -- 172.21.15.22:0/898032945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 msgr2=0x557741b12090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:22.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.326+0000 7ff536ffd640 1 --2- 172.21.15.22:0/898032945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 0x557741b12090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.326+0000 7ff536ffd640 1 -- 172.21.15.22:0/898032945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557741a043a0 con 0x557741b125d0 2024-09-19T08:18:22.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.326+0000 7ff5367fc640 1 --2- 172.21.15.22:0/898032945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 0x557741b1f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:18:22.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.326+0000 7ff536ffd640 1 --2- 172.21.15.22:0/898032945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b12a10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff528010370 tx=0x7ff528010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d44da3ab195f4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:22.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.326+0000 7ff5377fe640 1 --2- 172.21.15.22:0/2393894974 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff5200a33b0 0x7ff5200a5870 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff5300cd910 tx=0x7ff53006e170 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:22.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.326+0000 7ff51d7fa640 1 -- 172.21.15.22:0/898032945 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528016070 con 0x557741b125d0 2024-09-19T08:18:22.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.326+0000 7ff51d7fa640 1 -- 172.21.15.22:0/898032945 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff528004d20 con 0x557741b125d0 2024-09-19T08:18:22.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.326+0000 7ff51d7fa640 1 -- 172.21.15.22:0/898032945 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5280054e0 con 0x557741b125d0 2024-09-19T08:18:22.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.327+0000 7ff53fb93740 1 -- 172.21.15.22:0/898032945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 msgr2=0x557741b12a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:22.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.327+0000 7ff53fb93740 1 --2- 172.21.15.22:0/898032945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b12a10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff528010370 tx=0x7ff528010830 comp rx=0 tx=0).stop 2024-09-19T08:18:22.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.327+0000 7ff53fb93740 1 -- 172.21.15.22:0/898032945 shutdown_connections 2024-09-19T08:18:22.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.327+0000 7ff53fb93740 1 --2- 172.21.15.22:0/898032945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 0x557741b1f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.327+0000 7ff53fb93740 1 --2- 172.21.15.22:0/898032945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b12a10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.327+0000 7ff53fb93740 1 --2- 172.21.15.22:0/898032945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 0x557741b12090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.327+0000 7ff53fb93740 1 -- 172.21.15.22:0/898032945 >> 172.21.15.22:0/898032945 conn(0x557741adb310 msgr2=0x557741b0ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:22.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.327+0000 7ff53fb93740 1 -- 172.21.15.22:0/898032945 shutdown_connections 2024-09-19T08:18:22.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.327+0000 7ff53fb93740 1 -- 172.21.15.22:0/898032945 wait complete. 2024-09-19T08:18:22.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.327+0000 7ff53fb93740 1 -- start start 2024-09-19T08:18:22.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.327+0000 7ff53fb93740 1 Processor -- start 2024-09-19T08:18:22.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff53fb93740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b1cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff53fb93740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 0x557741b19f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff53fb93740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 0x557741b1a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff53fb93740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557741b20070 con 0x557741b125d0 2024-09-19T08:18:22.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff53fb93740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557741a33060 con 0x557741b13b40 2024-09-19T08:18:22.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff53fb93740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557741b201f0 con 0x557741b1d370 2024-09-19T08:18:22.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff53fb93740 10 client.0 _ll_register_callbacks cb 0x5577417868c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:22.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff5367fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b1cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff5367fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b1cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32794/0 (socket says 172.21.15.22:32794) 2024-09-19T08:18:22.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff5367fc640 1 -- 172.21.15.22:0/2084031835 learned_addr learned my addr 172.21.15.22:0/2084031835 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:22.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff5367fc640 1 --2- 172.21.15.22:0/2084031835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 0x557741b1a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff536ffd640 1 --2- 172.21.15.22:0/2084031835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 0x557741b19f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff5367fc640 1 -- 172.21.15.22:0/2084031835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 msgr2=0x557741b1a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:22.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff5367fc640 1 --2- 172.21.15.22:0/2084031835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 0x557741b1a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff5367fc640 1 -- 172.21.15.22:0/2084031835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 msgr2=0x557741b19f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:22.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff5367fc640 1 --2- 172.21.15.22:0/2084031835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 0x557741b19f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff5367fc640 1 -- 172.21.15.22:0/2084031835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577417e93c0 con 0x557741b125d0 2024-09-19T08:18:22.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff536ffd640 1 --2- 172.21.15.22:0/2084031835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 0x557741b19f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:18:22.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff5367fc640 1 --2- 172.21.15.22:0/2084031835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b1cce0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff52c02a940 tx=0x7ff52c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ddb32a90c7e32f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:22.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff51aff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:18:22.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.328+0000 7ff51aff5640 1 -- 172.21.15.22:0/2084031835 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c035070 con 0x557741b125d0 2024-09-19T08:18:22.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.329+0000 7ff51aff5640 1 -- 172.21.15.22:0/2084031835 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff52c004300 con 0x557741b125d0 2024-09-19T08:18:22.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.329+0000 7ff53fb93740 1 -- 172.21.15.22:0/2084031835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff528004d20 con 0x557741b125d0 2024-09-19T08:18:22.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.329+0000 7ff51aff5640 1 -- 172.21.15.22:0/2084031835 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff52c050050 con 0x557741b125d0 2024-09-19T08:18:22.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.329+0000 7ff53fb93740 10 client.16056 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:22.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.329+0000 7ff53fb93740 1 -- 172.21.15.22:0/2084031835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557741b2bab0 con 0x557741b125d0 2024-09-19T08:18:22.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.329+0000 7ff51aff5640 1 -- 172.21.15.22:0/2084031835 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff528004d20 con 0x557741b125d0 2024-09-19T08:18:22.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.329+0000 7ff51aff5640 1 -- 172.21.15.22:0/2084031835 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7ff52c048020 con 0x557741b125d0 2024-09-19T08:18:22.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.329+0000 7ff53fb93740 10 client.16056 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:22.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.329+0000 7ff53fb93740 10 client.16056 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:18:22.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.329+0000 7ff53fb93740 20 client.16056 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:22.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.329+0000 7ff53fb93740 1 --2- 172.21.15.22:0/2084031835 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x557741b2d060 0x557741b2f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.329+0000 7ff53fb93740 4 client.16056 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:18:22.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.329+0000 7ff53fb93740 1 -- 172.21.15.22:0/2084031835 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557741a85860 con 0x557741b2d060 2024-09-19T08:18:22.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.329+0000 7ff536ffd640 1 --2- 172.21.15.22:0/2084031835 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x557741b2d060 0x557741b2f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.330+0000 7ff51aff5640 10 client.16056 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:18:22.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.330+0000 7ff536ffd640 1 --2- 172.21.15.22:0/2084031835 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x557741b2d060 0x557741b2f4b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ead286979bd12b5 server_cookie=537dc5e87ea28af2 in_seq=0 out_seq=0 2024-09-19T08:18:22.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.334+0000 7ff51aff5640 1 -- 172.21.15.22:0/2084031835 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557741a85860 con 0x557741b2d060 2024-09-19T08:18:22.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.334+0000 7ff51aff5640 10 client.16056 handle_command_reply: tid=0 2024-09-19T08:18:22.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.334+0000 7ff53fb93740 1 client.16056 shutdown 2024-09-19T08:18:22.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.334+0000 7ff53fb93740 1 -- 172.21.15.22:0/2084031835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 msgr2=0x557741b1cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:22.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.334+0000 7ff53fb93740 1 --2- 172.21.15.22:0/2084031835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b1cce0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff52c02a940 tx=0x7ff52c005fd0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.334+0000 7ff53fb93740 1 -- 172.21.15.22:0/2084031835 shutdown_connections 2024-09-19T08:18:22.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.334+0000 7ff53fb93740 1 --2- 172.21.15.22:0/2084031835 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x557741b2d060 0x557741b2f4b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.335+0000 7ff53fb93740 1 --2- 172.21.15.22:0/2084031835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 0x557741b1a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.335+0000 7ff53fb93740 1 --2- 172.21.15.22:0/2084031835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 0x557741b19f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.335+0000 7ff53fb93740 1 --2- 172.21.15.22:0/2084031835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b1cce0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.335+0000 7ff53fb93740 1 -- 172.21.15.22:0/2084031835 >> 172.21.15.22:0/2084031835 conn(0x557741adb310 msgr2=0x557741b14930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:22.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.335+0000 7ff51aff5640 0 client.16056 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:18:22.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.335+0000 7ff53fb93740 1 -- 172.21.15.22:0/2084031835 shutdown_connections 2024-09-19T08:18:22.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.335+0000 7ff53fb93740 1 -- 172.21.15.22:0/2084031835 wait complete. 2024-09-19T08:18:22.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.335+0000 7ff53fb93740 20 client.16056 trim_cache size 0 max 16384 2024-09-19T08:18:22.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.349+0000 7ff53fb93740 1 Processor -- start 2024-09-19T08:18:22.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.349+0000 7ff53fb93740 1 -- start start 2024-09-19T08:18:22.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.349+0000 7ff53fb93740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b29340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.349+0000 7ff53fb93740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 0x557741b0dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.349+0000 7ff53fb93740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 0x557741b34d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.349+0000 7ff53fb93740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381b2430 con 0x557741b125d0 2024-09-19T08:18:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.349+0000 7ff53fb93740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381afca0 con 0x557741b13b40 2024-09-19T08:18:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.349+0000 7ff53fb93740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381adaf0 con 0x557741b1d370 2024-09-19T08:18:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.349+0000 7ff5367fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b29340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.349+0000 7ff5367fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b29340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32800/0 (socket says 172.21.15.22:32800) 2024-09-19T08:18:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.349+0000 7ff5367fc640 1 -- 172.21.15.22:0/3331872558 learned_addr learned my addr 172.21.15.22:0/3331872558 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.350+0000 7ff536ffd640 1 --2- 172.21.15.22:0/3331872558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 0x557741b0dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.350+0000 7ff5367fc640 1 -- 172.21.15.22:0/3331872558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 msgr2=0x557741b34d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.350+0000 7ff5367fc640 1 --2- 172.21.15.22:0/3331872558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 0x557741b34d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.350+0000 7ff5367fc640 1 -- 172.21.15.22:0/3331872558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 msgr2=0x557741b0dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.350+0000 7ff5367fc640 1 --2- 172.21.15.22:0/3331872558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 0x557741b0dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.350+0000 7ff5367fc640 1 -- 172.21.15.22:0/3331872558 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557741b0d310 con 0x557741b125d0 2024-09-19T08:18:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.350+0000 7ff5367fc640 1 --2- 172.21.15.22:0/3331872558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b29340 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff52c000c00 tx=0x7ff52c031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b962ed7b38d74f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.350+0000 7ff51cff9640 1 -- 172.21.15.22:0/3331872558 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c058020 con 0x557741b125d0 2024-09-19T08:18:22.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.350+0000 7ff51cff9640 1 -- 172.21.15.22:0/3331872558 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff52c005850 con 0x557741b125d0 2024-09-19T08:18:22.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.350+0000 7ff51cff9640 1 -- 172.21.15.22:0/3331872558 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c035040 con 0x557741b125d0 2024-09-19T08:18:22.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.351+0000 7ff53fb93740 1 -- 172.21.15.22:0/3331872558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 msgr2=0x557741b29340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:22.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.351+0000 7ff53fb93740 1 --2- 172.21.15.22:0/3331872558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b29340 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff52c000c00 tx=0x7ff52c031890 comp rx=0 tx=0).stop 2024-09-19T08:18:22.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.351+0000 7ff53fb93740 1 -- 172.21.15.22:0/3331872558 shutdown_connections 2024-09-19T08:18:22.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.351+0000 7ff53fb93740 1 --2- 172.21.15.22:0/3331872558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b1d370 0x557741b34d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.351+0000 7ff53fb93740 1 --2- 172.21.15.22:0/3331872558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b13b40 0x557741b0dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.351+0000 7ff53fb93740 1 --2- 172.21.15.22:0/3331872558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b125d0 0x557741b29340 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.351+0000 7ff53fb93740 1 -- 172.21.15.22:0/3331872558 >> 172.21.15.22:0/3331872558 conn(0x557741adb310 msgr2=0x557741b0d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:22.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.351+0000 7ff53fb93740 1 -- 172.21.15.22:0/3331872558 shutdown_connections 2024-09-19T08:18:22.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.351+0000 7ff53fb93740 1 -- 172.21.15.22:0/3331872558 wait complete. 2024-09-19T08:18:22.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.352+0000 7ff53fb93740 1 -- start start 2024-09-19T08:18:22.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.352+0000 7ff53fb93740 1 Processor -- start 2024-09-19T08:18:22.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.352+0000 7ff53fb93740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b125d0 0x557741b25840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.352+0000 7ff53fb93740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b13b40 0x557741b25d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.352+0000 7ff53fb93740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b1d370 0x557741b16b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.352+0000 7ff53fb93740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381ae160 con 0x557741b13b40 2024-09-19T08:18:22.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.352+0000 7ff53fb93740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381af5d0 con 0x557741b1d370 2024-09-19T08:18:22.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.352+0000 7ff53fb93740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5381ae830 con 0x557741b125d0 2024-09-19T08:18:22.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.352+0000 7ff53fb93740 10 client.0 _ll_register_callbacks cb 0x7ff52c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:22.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.352+0000 7ff536ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b13b40 0x557741b25d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.352+0000 7ff5367fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b125d0 0x557741b25840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.352+0000 7ff536ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b13b40 0x557741b25d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32814/0 (socket says 172.21.15.22:32814) 2024-09-19T08:18:22.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.352+0000 7ff536ffd640 1 -- 172.21.15.22:0/309938408 learned_addr learned my addr 172.21.15.22:0/309938408 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:22.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.352+0000 7ff5367fc640 1 --2- 172.21.15.22:0/309938408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b1d370 0x557741b16b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.353+0000 7ff536ffd640 1 -- 172.21.15.22:0/309938408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b125d0 msgr2=0x557741b25840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:22.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.353+0000 7ff536ffd640 1 --2- 172.21.15.22:0/309938408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b125d0 0x557741b25840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.353+0000 7ff536ffd640 1 -- 172.21.15.22:0/309938408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b1d370 msgr2=0x557741b16b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:22.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.353+0000 7ff536ffd640 1 --2- 172.21.15.22:0/309938408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b1d370 0x557741b16b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.353+0000 7ff536ffd640 1 -- 172.21.15.22:0/309938408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557741b173c0 con 0x557741b13b40 2024-09-19T08:18:22.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.353+0000 7ff536ffd640 1 --2- 172.21.15.22:0/309938408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b13b40 0x557741b25d80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff528005090 tx=0x7ff528010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f311c0b05aa4ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:22.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.353+0000 7ff51aff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:18:22.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.353+0000 7ff51aff5640 1 -- 172.21.15.22:0/309938408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528016040 con 0x557741b13b40 2024-09-19T08:18:22.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.353+0000 7ff51aff5640 1 -- 172.21.15.22:0/309938408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff528004cd0 con 0x557741b13b40 2024-09-19T08:18:22.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.353+0000 7ff53fb93740 1 -- 172.21.15.22:0/309938408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff52c005850 con 0x557741b13b40 2024-09-19T08:18:22.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.353+0000 7ff51aff5640 1 -- 172.21.15.22:0/309938408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff528037050 con 0x557741b13b40 2024-09-19T08:18:22.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.353+0000 7ff53fb93740 10 client.16062 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:22.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.353+0000 7ff53fb93740 1 -- 172.21.15.22:0/309938408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557741b979b0 con 0x557741b13b40 2024-09-19T08:18:22.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.353+0000 7ff51aff5640 1 -- 172.21.15.22:0/309938408 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff52c005850 con 0x557741b13b40 2024-09-19T08:18:22.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.354+0000 7ff51aff5640 1 -- 172.21.15.22:0/309938408 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7ff528025040 con 0x557741b13b40 2024-09-19T08:18:22.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.354+0000 7ff53fb93740 10 client.16062 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:22.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.354+0000 7ff53fb93740 10 client.16062 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:18:22.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.354+0000 7ff53fb93740 20 client.16062 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:22.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.354+0000 7ff53fb93740 1 --2- 172.21.15.22:0/309938408 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x557741b97ea0 0x557741b9a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:22.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.354+0000 7ff53fb93740 4 client.16062 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:18:22.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.354+0000 7ff53fb93740 1 -- 172.21.15.22:0/309938408 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557741a7fd70 con 0x557741b97ea0 2024-09-19T08:18:22.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.354+0000 7ff5367fc640 1 --2- 172.21.15.22:0/309938408 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x557741b97ea0 0x557741b9a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:22.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.354+0000 7ff5367fc640 1 --2- 172.21.15.22:0/309938408 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x557741b97ea0 0x557741b9a2c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da3f2f7aeb931ae9 server_cookie=66b6e18f464b11f0 in_seq=0 out_seq=0 2024-09-19T08:18:22.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.354+0000 7ff51aff5640 10 client.16062 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:18:22.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff51aff5640 1 -- 172.21.15.22:0/309938408 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557741a7fd70 con 0x557741b97ea0 2024-09-19T08:18:22.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff51aff5640 10 client.16062 handle_command_reply: tid=0 2024-09-19T08:18:22.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff53fb93740 1 client.16062 shutdown 2024-09-19T08:18:22.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff53fb93740 1 -- 172.21.15.22:0/309938408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b13b40 msgr2=0x557741b25d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:22.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff53fb93740 1 --2- 172.21.15.22:0/309938408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b13b40 0x557741b25d80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff528005090 tx=0x7ff528010fa0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff53fb93740 1 -- 172.21.15.22:0/309938408 shutdown_connections 2024-09-19T08:18:22.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff53fb93740 1 --2- 172.21.15.22:0/309938408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557741b1d370 0x557741b16b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff53fb93740 1 --2- 172.21.15.22:0/309938408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557741b13b40 0x557741b25d80 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff53fb93740 1 --2- 172.21.15.22:0/309938408 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x557741b97ea0 0x557741b9a2c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff53fb93740 1 --2- 172.21.15.22:0/309938408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557741b125d0 0x557741b25840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff53fb93740 1 -- 172.21.15.22:0/309938408 >> 172.21.15.22:0/309938408 conn(0x557741adb310 msgr2=0x557741b0dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:22.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff51aff5640 0 client.16062 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:18:22.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff53fb93740 1 -- 172.21.15.22:0/309938408 shutdown_connections 2024-09-19T08:18:22.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff53fb93740 1 -- 172.21.15.22:0/309938408 wait complete. 2024-09-19T08:18:22.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.359+0000 7ff53fb93740 20 client.16062 trim_cache size 0 max 16384 2024-09-19T08:18:22.361 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:18:22.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.360+0000 7ff53d7ba640 1 -- 172.21.15.22:0/2393894974 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff5200a33b0 msgr2=0x7ff5200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:22.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.360+0000 7ff53d7ba640 1 --2- 172.21.15.22:0/2393894974 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff5200a33b0 0x7ff5200a5870 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff5300cd910 tx=0x7ff53006e170 comp rx=0 tx=0).stop 2024-09-19T08:18:22.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.360+0000 7ff53d7ba640 1 -- 172.21.15.22:0/2393894974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff538106a70 msgr2=0x7ff538197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:22.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.360+0000 7ff53d7ba640 1 --2- 172.21.15.22:0/2393894974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff538106a70 0x7ff538197b90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff53000e970 tx=0x7ff53000ed70 comp rx=0 tx=0).stop 2024-09-19T08:18:22.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.360+0000 7ff53d7ba640 1 -- 172.21.15.22:0/2393894974 shutdown_connections 2024-09-19T08:18:22.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.360+0000 7ff53d7ba640 1 --2- 172.21.15.22:0/2393894974 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff5200a33b0 0x7ff5200a5870 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.360+0000 7ff53d7ba640 1 --2- 172.21.15.22:0/2393894974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff538106a70 0x7ff538197b90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.360+0000 7ff53d7ba640 1 --2- 172.21.15.22:0/2393894974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff538101410 0x7ff538197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.360+0000 7ff53d7ba640 1 --2- 172.21.15.22:0/2393894974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff538100a70 0x7ff538197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:22.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.360+0000 7ff53d7ba640 1 -- 172.21.15.22:0/2393894974 >> 172.21.15.22:0/2393894974 conn(0x7ff5380fc7d0 msgr2=0x7ff5380fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:22.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.360+0000 7ff53d7ba640 1 -- 172.21.15.22:0/2393894974 shutdown_connections 2024-09-19T08:18:22.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:22.360+0000 7ff53d7ba640 1 -- 172.21.15.22:0/2393894974 wait complete. 2024-09-19T08:18:23.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:23 smithi064 ceph-mon[34900]: pgmap v103: 97 pgs: 97 active+clean; 309 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.5 MiB/s wr, 513 op/s 2024-09-19T08:18:23.502 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:23 smithi157 ceph-mon[35143]: pgmap v103: 97 pgs: 97 active+clean; 309 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.5 MiB/s wr, 513 op/s 2024-09-19T08:18:23.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:23 smithi022 ceph-mon[30963]: pgmap v103: 97 pgs: 97 active+clean; 309 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.5 MiB/s wr, 513 op/s 2024-09-19T08:18:23.609 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:18:24.264 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:18:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.270+0000 7f9f8c8af640 1 -- 172.21.15.22:0/466617819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f88109830 msgr2=0x7f9f88109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.270+0000 7f9f8c8af640 1 --2- 172.21.15.22:0/466617819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f88109830 0x7f9f88109c90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009860 tx=0x7f9f7802a340 comp rx=0 tx=0).stop 2024-09-19T08:18:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.271+0000 7f9f8c8af640 1 -- 172.21.15.22:0/466617819 shutdown_connections 2024-09-19T08:18:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.271+0000 7f9f8c8af640 1 --2- 172.21.15.22:0/466617819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f8811b140 0x7f9f8811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.271+0000 7f9f8c8af640 1 --2- 172.21.15.22:0/466617819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f88109830 0x7f9f88109c90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.271+0000 7f9f8c8af640 1 --2- 172.21.15.22:0/466617819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f88109450 0x7f9f88110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.271+0000 7f9f8c8af640 1 -- 172.21.15.22:0/466617819 >> 172.21.15.22:0/466617819 conn(0x7f9f8806e140 msgr2=0x7f9f8806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:25.273 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:25 smithi022 ceph-mon[30963]: pgmap v104: 97 pgs: 97 active+clean; 367 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 852 op/s 2024-09-19T08:18:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.274+0000 7f9f8c8af640 1 -- 172.21.15.22:0/466617819 shutdown_connections 2024-09-19T08:18:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.274+0000 7f9f8c8af640 1 -- 172.21.15.22:0/466617819 wait complete. 2024-09-19T08:18:25.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.275+0000 7f9f8c8af640 1 Processor -- start 2024-09-19T08:18:25.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.276+0000 7f9f8c8af640 1 -- start start 2024-09-19T08:18:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.276+0000 7f9f8c8af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f88109450 0x7f9f8811a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.276+0000 7f9f8c8af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f88109830 0x7f9f88114930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.276+0000 7f9f8c8af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8811b140 0x7f9f88114e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.276+0000 7f9f8c8af640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88111330 con 0x7f9f8811b140 2024-09-19T08:18:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.276+0000 7f9f8c8af640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881111b0 con 0x7f9f88109450 2024-09-19T08:18:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.276+0000 7f9f8c8af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881114b0 con 0x7f9f88109830 2024-09-19T08:18:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.277+0000 7f9f85d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f88109830 0x7f9f88114930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:25.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.277+0000 7f9f86d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8811b140 0x7f9f88114e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:25.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.277+0000 7f9f86d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8811b140 0x7f9f88114e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32838/0 (socket says 172.21.15.22:32838) 2024-09-19T08:18:25.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.277+0000 7f9f86d9d640 1 -- 172.21.15.22:0/430834040 learned_addr learned my addr 172.21.15.22:0/430834040 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:25.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.278+0000 7f9f8659c640 1 --2- 172.21.15.22:0/430834040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f88109450 0x7f9f8811a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:25.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.278+0000 7f9f86d9d640 1 -- 172.21.15.22:0/430834040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f88109830 msgr2=0x7f9f88114930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:25.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.278+0000 7f9f86d9d640 1 --2- 172.21.15.22:0/430834040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f88109830 0x7f9f88114930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.278+0000 7f9f86d9d640 1 -- 172.21.15.22:0/430834040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f88109450 msgr2=0x7f9f8811a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:25.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.278+0000 7f9f86d9d640 1 --2- 172.21.15.22:0/430834040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f88109450 0x7f9f8811a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.278+0000 7f9f86d9d640 1 -- 172.21.15.22:0/430834040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f881155c0 con 0x7f9f8811b140 2024-09-19T08:18:25.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.278+0000 7f9f86d9d640 1 --2- 172.21.15.22:0/430834040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8811b140 0x7f9f88114e70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000ebc0 tx=0x7f9f8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:25.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.282+0000 7f9f777fe640 1 -- 172.21.15.22:0/430834040 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80004b20 con 0x7f9f8811b140 2024-09-19T08:18:25.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.282+0000 7f9f777fe640 1 -- 172.21.15.22:0/430834040 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f80004cc0 con 0x7f9f8811b140 2024-09-19T08:18:25.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.282+0000 7f9f777fe640 1 -- 172.21.15.22:0/430834040 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80010760 con 0x7f9f8811b140 2024-09-19T08:18:25.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.283+0000 7f9f8c8af640 1 -- 172.21.15.22:0/430834040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f881bc130 con 0x7f9f8811b140 2024-09-19T08:18:25.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.283+0000 7f9f8c8af640 1 -- 172.21.15.22:0/430834040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f881bc590 con 0x7f9f8811b140 2024-09-19T08:18:25.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.294+0000 7f9f8ec88740 1 Processor -- start 2024-09-19T08:18:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.296+0000 7f9f8ec88740 1 -- start start 2024-09-19T08:18:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.297+0000 7f9f777fe640 1 -- 172.21.15.22:0/430834040 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9f80005000 con 0x7f9f8811b140 2024-09-19T08:18:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.297+0000 7f9f8ec88740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55580218ab40 0x555802189090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.297+0000 7f9f8ec88740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558021895d0 0x555802189a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.297+0000 7f9f8ec88740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555802194370 0x555802196760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.297+0000 7f9f8ec88740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5558020ef2f0 con 0x555802194370 2024-09-19T08:18:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.297+0000 7f9f8ec88740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555802096cf0 con 0x55580218ab40 2024-09-19T08:18:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.297+0000 7f9f8ec88740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5558020eb2b0 con 0x5558021895d0 2024-09-19T08:18:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.298+0000 7f9f85d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555802194370 0x555802196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.298+0000 7f9f85d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555802194370 0x555802196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32844/0 (socket says 172.21.15.22:32844) 2024-09-19T08:18:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.298+0000 7f9f85d9b640 1 -- 172.21.15.22:0/1794206274 learned_addr learned my addr 172.21.15.22:0/1794206274 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.298+0000 7f9f85d9b640 1 --2- 172.21.15.22:0/1794206274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55580218ab40 0x555802189090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.302+0000 7f9f777fe640 1 --2- 172.21.15.22:0/430834040 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9f700a35c0 0x7f9f700a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.302+0000 7f9f85d9b640 1 -- 172.21.15.22:0/1794206274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558021895d0 msgr2=0x555802189a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:18:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.302+0000 7f9f85d9b640 1 --2- 172.21.15.22:0/1794206274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558021895d0 0x555802189a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.302+0000 7f9f85d9b640 1 -- 172.21.15.22:0/1794206274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55580218ab40 msgr2=0x555802189090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.302+0000 7f9f85d9b640 1 --2- 172.21.15.22:0/1794206274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55580218ab40 0x555802189090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.302+0000 7f9f85d9b640 1 -- 172.21.15.22:0/1794206274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55580207b3a0 con 0x555802194370 2024-09-19T08:18:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.302+0000 7f9f777fe640 1 -- 172.21.15.22:0/430834040 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9f800d0320 con 0x7f9f8811b140 2024-09-19T08:18:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.303+0000 7f9f86d9d640 1 --2- 172.21.15.22:0/430834040 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9f700a35c0 0x7f9f700a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.303+0000 7f9f85d9b640 1 --2- 172.21.15.22:0/1794206274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555802194370 0x555802196760 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009780 tx=0x7f9f78000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ead6c789ca77fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:25.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.305+0000 7f9f74ff9640 1 -- 172.21.15.22:0/1794206274 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78068070 con 0x555802194370 2024-09-19T08:18:25.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.305+0000 7f9f74ff9640 1 -- 172.21.15.22:0/1794206274 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78004230 con 0x555802194370 2024-09-19T08:18:25.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.305+0000 7f9f74ff9640 1 -- 172.21.15.22:0/1794206274 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78037040 con 0x555802194370 2024-09-19T08:18:25.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.306+0000 7f9f86d9d640 1 --2- 172.21.15.22:0/430834040 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9f700a35c0 0x7f9f700a5a80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9f8009c2b0 tx=0x7f9f80005b40 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:25.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.307+0000 7f9f8ec88740 1 -- 172.21.15.22:0/1794206274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555802194370 msgr2=0x555802196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:25.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.307+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/1794206274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555802194370 0x555802196760 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009780 tx=0x7f9f78000bc0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.307+0000 7f9f8ec88740 1 -- 172.21.15.22:0/1794206274 shutdown_connections 2024-09-19T08:18:25.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.308+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/1794206274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555802194370 0x555802196760 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.308+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/1794206274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558021895d0 0x555802189a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.308+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/1794206274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55580218ab40 0x555802189090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.308+0000 7f9f8ec88740 1 -- 172.21.15.22:0/1794206274 >> 172.21.15.22:0/1794206274 conn(0x555802152310 msgr2=0x555802185c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:25.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.311+0000 7f9f8ec88740 1 -- 172.21.15.22:0/1794206274 shutdown_connections 2024-09-19T08:18:25.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.312+0000 7f9f8ec88740 1 -- 172.21.15.22:0/1794206274 wait complete. 2024-09-19T08:18:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.312+0000 7f9f8ec88740 1 -- start start 2024-09-19T08:18:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.312+0000 7f9f8ec88740 1 Processor -- start 2024-09-19T08:18:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.313+0000 7f9f8ec88740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 0x5558021910f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.313+0000 7f9f8ec88740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55580218ab40 0x555802191630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.313+0000 7f9f8ec88740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555802194370 0x555802191b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.313+0000 7f9f8ec88740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555802197010 con 0x5558021895d0 2024-09-19T08:18:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.313+0000 7f9f8ec88740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5558020aa060 con 0x555802194370 2024-09-19T08:18:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.313+0000 7f9f8ec88740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555802197190 con 0x55580218ab40 2024-09-19T08:18:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.313+0000 7f9f8ec88740 10 client.0 _ll_register_callbacks cb 0x555801dfd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.313+0000 7f9f8659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55580218ab40 0x555802191630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.313+0000 7f9f8659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55580218ab40 0x555802191630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42960/0 (socket says 172.21.15.22:42960) 2024-09-19T08:18:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.313+0000 7f9f8659c640 1 -- 172.21.15.22:0/1605351630 learned_addr learned my addr 172.21.15.22:0/1605351630 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.314+0000 7f9f8659c640 1 -- 172.21.15.22:0/1605351630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555802194370 msgr2=0x555802191b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.314+0000 7f9f85d9b640 1 --2- 172.21.15.22:0/1605351630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555802194370 0x555802191b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.314+0000 7f9f8659c640 1 --2- 172.21.15.22:0/1605351630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555802194370 0x555802191b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.314+0000 7f9f8659c640 1 -- 172.21.15.22:0/1605351630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 msgr2=0x5558021910f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.314+0000 7f9f8659c640 1 --2- 172.21.15.22:0/1605351630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 0x5558021910f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.314+0000 7f9f8659c640 1 -- 172.21.15.22:0/1605351630 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555801e603c0 con 0x55580218ab40 2024-09-19T08:18:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.314+0000 7f9f85d9b640 1 --2- 172.21.15.22:0/1605351630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555802194370 0x555802191b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:18:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.315+0000 7f9f8659c640 1 --2- 172.21.15.22:0/1605351630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55580218ab40 0x555802191630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00de00 tx=0x7f9f7c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e00b7d3aa4816b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.317+0000 7f9f6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:18:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.317+0000 7f9f6dffb640 1 -- 172.21.15.22:0/1605351630 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c011850 con 0x55580218ab40 2024-09-19T08:18:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.317+0000 7f9f8ec88740 1 -- 172.21.15.22:0/1605351630 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f78004230 con 0x55580218ab40 2024-09-19T08:18:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.318+0000 7f9f6dffb640 1 -- 172.21.15.22:0/1605351630 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9f7c0119f0 con 0x55580218ab40 2024-09-19T08:18:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.318+0000 7f9f6dffb640 1 -- 172.21.15.22:0/1605351630 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9f7c011bf0 con 0x55580218ab40 2024-09-19T08:18:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.318+0000 7f9f8ec88740 10 client.25618 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.318+0000 7f9f8ec88740 1 -- 172.21.15.22:0/1605351630 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5558021a2c30 con 0x55580218ab40 2024-09-19T08:18:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.319+0000 7f9f6dffb640 1 -- 172.21.15.22:0/1605351630 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78004230 con 0x55580218ab40 2024-09-19T08:18:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.319+0000 7f9f6dffb640 1 -- 172.21.15.22:0/1605351630 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f9f7c0233a0 con 0x55580218ab40 2024-09-19T08:18:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.319+0000 7f9f8ec88740 10 client.25618 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.319+0000 7f9f8ec88740 10 client.25618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:18:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.319+0000 7f9f8ec88740 20 client.25618 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.319+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/1605351630 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x5558021a4490 0x5558021a6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.319+0000 7f9f8ec88740 4 client.25618 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:18:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.319+0000 7f9f8ec88740 1 -- 172.21.15.22:0/1605351630 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558020fc860 con 0x5558021a4490 2024-09-19T08:18:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.319+0000 7f9f85d9b640 1 --2- 172.21.15.22:0/1605351630 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x5558021a4490 0x5558021a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.320+0000 7f9f85d9b640 1 --2- 172.21.15.22:0/1605351630 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x5558021a4490 0x5558021a6910 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40d4cbbf21c6eb4f server_cookie=f6d699242a69c162 in_seq=0 out_seq=0 2024-09-19T08:18:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.320+0000 7f9f6dffb640 10 client.25618 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:18:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.321+0000 7f9f6dffb640 1 -- 172.21.15.22:0/1605351630 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558020fc860 con 0x5558021a4490 2024-09-19T08:18:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.321+0000 7f9f6dffb640 10 client.25618 handle_command_reply: tid=0 2024-09-19T08:18:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.321+0000 7f9f8ec88740 1 client.25618 shutdown 2024-09-19T08:18:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.321+0000 7f9f8ec88740 1 -- 172.21.15.22:0/1605351630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55580218ab40 msgr2=0x555802191630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.321+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/1605351630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55580218ab40 0x555802191630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00de00 tx=0x7f9f7c00f730 comp rx=0 tx=0).stop 2024-09-19T08:18:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.322+0000 7f9f8ec88740 1 -- 172.21.15.22:0/1605351630 shutdown_connections 2024-09-19T08:18:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.322+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/1605351630 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x5558021a4490 0x5558021a6910 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.322+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/1605351630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555802194370 0x555802191b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.322+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/1605351630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55580218ab40 0x555802191630 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.322+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/1605351630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 0x5558021910f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.322+0000 7f9f8ec88740 1 -- 172.21.15.22:0/1605351630 >> 172.21.15.22:0/1605351630 conn(0x555802152310 msgr2=0x55580218b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.323+0000 7f9f6dffb640 0 client.25618 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:18:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.323+0000 7f9f8ec88740 1 -- 172.21.15.22:0/1605351630 shutdown_connections 2024-09-19T08:18:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.323+0000 7f9f8ec88740 1 -- 172.21.15.22:0/1605351630 wait complete. 2024-09-19T08:18:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.324+0000 7f9f8ec88740 20 client.25618 trim_cache size 0 max 16384 2024-09-19T08:18:25.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:25 smithi064 ceph-mon[34900]: pgmap v104: 97 pgs: 97 active+clean; 367 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 852 op/s 2024-09-19T08:18:25.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.373+0000 7f9f8ec88740 1 Processor -- start 2024-09-19T08:18:25.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.376+0000 7f9f8ec88740 1 -- start start 2024-09-19T08:18:25.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.376+0000 7f9f8ec88740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 0x555802195760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.376+0000 7f9f8ec88740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555802194370 0x555802195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.376+0000 7f9f8ec88740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5558021904b0 0x55580218c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.376+0000 7f9f8ec88740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8807f450 con 0x5558021895d0 2024-09-19T08:18:25.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.376+0000 7f9f8ec88740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8807ccc0 con 0x5558021904b0 2024-09-19T08:18:25.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.376+0000 7f9f8ec88740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8807ab10 con 0x555802194370 2024-09-19T08:18:25.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.377+0000 7f9f8659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555802194370 0x555802195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:25.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.377+0000 7f9f8659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555802194370 0x555802195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42962/0 (socket says 172.21.15.22:42962) 2024-09-19T08:18:25.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.377+0000 7f9f8659c640 1 -- 172.21.15.22:0/667466324 learned_addr learned my addr 172.21.15.22:0/667466324 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:25.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.377+0000 7f9f85d9b640 1 --2- 172.21.15.22:0/667466324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 0x555802195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:25.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.377+0000 7f9f85d9b640 1 --2- 172.21.15.22:0/667466324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5558021904b0 0x55580218c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:25.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.381+0000 7f9f8659c640 1 -- 172.21.15.22:0/667466324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5558021904b0 msgr2=0x55580218c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:25.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.381+0000 7f9f8659c640 1 --2- 172.21.15.22:0/667466324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5558021904b0 0x55580218c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.381+0000 7f9f8659c640 1 -- 172.21.15.22:0/667466324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 msgr2=0x555802195760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:25.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.381+0000 7f9f8659c640 1 --2- 172.21.15.22:0/667466324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 0x555802195760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.381+0000 7f9f8659c640 1 -- 172.21.15.22:0/667466324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555802197180 con 0x555802194370 2024-09-19T08:18:25.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.384+0000 7f9f8659c640 1 --2- 172.21.15.22:0/667466324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555802194370 0x555802195ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c001eb0 tx=0x7f9f7c00dfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c35e76fbd05a227 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:25.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.387+0000 7f9f6ffff640 1 -- 172.21.15.22:0/667466324 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c028070 con 0x555802194370 2024-09-19T08:18:25.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.387+0000 7f9f6ffff640 1 -- 172.21.15.22:0/667466324 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f7c00d4f0 con 0x555802194370 2024-09-19T08:18:25.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.387+0000 7f9f6ffff640 1 -- 172.21.15.22:0/667466324 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0119d0 con 0x555802194370 2024-09-19T08:18:25.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.390+0000 7f9f8ec88740 1 -- 172.21.15.22:0/667466324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555802194370 msgr2=0x555802195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:25.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.390+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/667466324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555802194370 0x555802195ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c001eb0 tx=0x7f9f7c00dfd0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.392+0000 7f9f8ec88740 1 -- 172.21.15.22:0/667466324 shutdown_connections 2024-09-19T08:18:25.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.392+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/667466324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5558021904b0 0x55580218c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.392+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/667466324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555802194370 0x555802195ca0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.392+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/667466324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 0x555802195760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.392+0000 7f9f8ec88740 1 -- 172.21.15.22:0/667466324 >> 172.21.15.22:0/667466324 conn(0x555802152310 msgr2=0x555802193170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:25.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.398+0000 7f9f8ec88740 1 -- 172.21.15.22:0/667466324 shutdown_connections 2024-09-19T08:18:25.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.401+0000 7f9f8ec88740 1 -- 172.21.15.22:0/667466324 wait complete. 2024-09-19T08:18:25.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.401+0000 7f9f8ec88740 1 -- start start 2024-09-19T08:18:25.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.401+0000 7f9f8ec88740 1 Processor -- start 2024-09-19T08:18:25.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.402+0000 7f9f8ec88740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 0x55580219c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.402+0000 7f9f85d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 0x55580219c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:25.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.402+0000 7f9f85d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 0x55580219c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32862/0 (socket says 172.21.15.22:32862) 2024-09-19T08:18:25.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.402+0000 7f9f85d9b640 1 -- 172.21.15.22:0/572782234 learned_addr learned my addr 172.21.15.22:0/572782234 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:25.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.402+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/572782234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5558021904b0 0x55580218e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.402+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/572782234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555802194370 0x55580218e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.402+0000 7f9f8ec88740 1 -- 172.21.15.22:0/572782234 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8807b180 con 0x5558021895d0 2024-09-19T08:18:25.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.402+0000 7f9f8ec88740 1 -- 172.21.15.22:0/572782234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8807c5f0 con 0x5558021904b0 2024-09-19T08:18:25.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.402+0000 7f9f8ec88740 1 -- 172.21.15.22:0/572782234 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8807b850 con 0x555802194370 2024-09-19T08:18:25.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.403+0000 7f9f85d9b640 1 -- 172.21.15.22:0/572782234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555802194370 msgr2=0x55580218e940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:18:25.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.403+0000 7f9f85d9b640 1 --2- 172.21.15.22:0/572782234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555802194370 0x55580218e940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.403+0000 7f9f85d9b640 1 -- 172.21.15.22:0/572782234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5558021904b0 msgr2=0x55580218e400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:18:25.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.403+0000 7f9f85d9b640 1 --2- 172.21.15.22:0/572782234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5558021904b0 0x55580218e400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.403+0000 7f9f85d9b640 1 -- 172.21.15.22:0/572782234 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55580218ee80 con 0x5558021895d0 2024-09-19T08:18:25.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.403+0000 7f9f85d9b640 1 --2- 172.21.15.22:0/572782234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 0x55580219c8b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009780 tx=0x7f9f78030780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7356e7504f7ecd53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:25.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.404+0000 7f9f8ec88740 10 client.0 _ll_register_callbacks cb 0x7f9f7c00d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:25.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.404+0000 7f9f8ec88740 1 -- 172.21.15.22:0/572782234 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f7c00d4f0 con 0x5558021895d0 2024-09-19T08:18:25.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.404+0000 7f9f6dffb640 10 client.16077 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:18:25.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.404+0000 7f9f6dffb640 1 -- 172.21.15.22:0/572782234 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78068070 con 0x5558021895d0 2024-09-19T08:18:25.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.404+0000 7f9f6dffb640 1 -- 172.21.15.22:0/572782234 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9f7806d2f0 con 0x5558021895d0 2024-09-19T08:18:25.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.404+0000 7f9f6dffb640 1 -- 172.21.15.22:0/572782234 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9f7807d050 con 0x5558021895d0 2024-09-19T08:18:25.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.408+0000 7f9f8ec88740 10 client.16077 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:25.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.408+0000 7f9f8ec88740 1 -- 172.21.15.22:0/572782234 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5558021ac500 con 0x5558021895d0 2024-09-19T08:18:25.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.408+0000 7f9f6dffb640 1 -- 172.21.15.22:0/572782234 <== mon.0 v2:172.21.15.22:3300/0 4 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f9f7806d4f0 con 0x5558021895d0 2024-09-19T08:18:25.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.409+0000 7f9f8ec88740 10 client.16077 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:25.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.409+0000 7f9f8ec88740 10 client.16077 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:18:25.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.409+0000 7f9f8ec88740 20 client.16077 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:25.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.409+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/572782234 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55580220f3f0 0x5558022118b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:25.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.409+0000 7f9f8ec88740 4 client.16077 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:18:25.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.409+0000 7f9f8ec88740 1 -- 172.21.15.22:0/572782234 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9f7c00d4f0 con 0x55580220f3f0 2024-09-19T08:18:25.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.411+0000 7f9f8659c640 1 --2- 172.21.15.22:0/572782234 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55580220f3f0 0x5558022118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:25.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.428+0000 7f9f8659c640 1 --2- 172.21.15.22:0/572782234 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55580220f3f0 0x5558022118b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1acbe08092d103c8 server_cookie=2259339b933223fb in_seq=0 out_seq=0 2024-09-19T08:18:25.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.429+0000 7f9f6dffb640 10 client.16077 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:18:25.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.431+0000 7f9f6dffb640 1 -- 172.21.15.22:0/572782234 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9f7c00d4f0 con 0x55580220f3f0 2024-09-19T08:18:25.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.431+0000 7f9f6dffb640 10 client.16077 handle_command_reply: tid=0 2024-09-19T08:18:25.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.431+0000 7f9f8ec88740 1 client.16077 shutdown 2024-09-19T08:18:25.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.440+0000 7f9f8ec88740 1 -- 172.21.15.22:0/572782234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 msgr2=0x55580219c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:25.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.440+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/572782234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 0x55580219c8b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009780 tx=0x7f9f78030780 comp rx=0 tx=0).stop 2024-09-19T08:18:25.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.440+0000 7f9f8ec88740 1 -- 172.21.15.22:0/572782234 shutdown_connections 2024-09-19T08:18:25.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.440+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/572782234 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55580220f3f0 0x5558022118b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.440+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/572782234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555802194370 0x55580218e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.440+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/572782234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5558021904b0 0x55580218e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.440+0000 7f9f8ec88740 1 --2- 172.21.15.22:0/572782234 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558021895d0 0x55580219c8b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.440+0000 7f9f8ec88740 1 -- 172.21.15.22:0/572782234 >> 172.21.15.22:0/572782234 conn(0x555802152310 msgr2=0x555802190f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:25.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.440+0000 7f9f6dffb640 0 client.16077 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:18:25.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.441+0000 7f9f8ec88740 1 -- 172.21.15.22:0/572782234 shutdown_connections 2024-09-19T08:18:25.443 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:18:25.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.442+0000 7f9f8ec88740 1 -- 172.21.15.22:0/572782234 wait complete. 2024-09-19T08:18:25.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.442+0000 7f9f8ec88740 20 client.16077 trim_cache size 0 max 16384 2024-09-19T08:18:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.444+0000 7f9f8c8af640 1 -- 172.21.15.22:0/430834040 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9f700a35c0 msgr2=0x7f9f700a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.444+0000 7f9f8c8af640 1 --2- 172.21.15.22:0/430834040 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9f700a35c0 0x7f9f700a5a80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9f8009c2b0 tx=0x7f9f80005b40 comp rx=0 tx=0).stop 2024-09-19T08:18:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.444+0000 7f9f8c8af640 1 -- 172.21.15.22:0/430834040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8811b140 msgr2=0x7f9f88114e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.444+0000 7f9f8c8af640 1 --2- 172.21.15.22:0/430834040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8811b140 0x7f9f88114e70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000ebc0 tx=0x7f9f8000c2d0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.444+0000 7f9f8c8af640 1 -- 172.21.15.22:0/430834040 shutdown_connections 2024-09-19T08:18:25.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.444+0000 7f9f8c8af640 1 --2- 172.21.15.22:0/430834040 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9f700a35c0 0x7f9f700a5a80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.444+0000 7f9f8c8af640 1 --2- 172.21.15.22:0/430834040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f8811b140 0x7f9f88114e70 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.444+0000 7f9f8c8af640 1 --2- 172.21.15.22:0/430834040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f88109830 0x7f9f88114930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.444+0000 7f9f8c8af640 1 --2- 172.21.15.22:0/430834040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f88109450 0x7f9f8811a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:25.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.444+0000 7f9f8c8af640 1 -- 172.21.15.22:0/430834040 >> 172.21.15.22:0/430834040 conn(0x7f9f8806e140 msgr2=0x7f9f88070ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:25.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.450+0000 7f9f8c8af640 1 -- 172.21.15.22:0/430834040 shutdown_connections 2024-09-19T08:18:25.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:25.450+0000 7f9f8c8af640 1 -- 172.21.15.22:0/430834040 wait complete. 2024-09-19T08:18:25.502 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:25 smithi157 ceph-mon[35143]: pgmap v104: 97 pgs: 97 active+clean; 367 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 852 op/s 2024-09-19T08:18:25.953 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-19T08:18:25.953 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:18:26.154 INFO:tasks.workunit.client.0.smithi022.stdout: 78 1044798 514 611692 305 518760 1233 2024-09-19T08:18:26.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:18:26.442 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:18:27.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:27 smithi022 ceph-mon[30963]: pgmap v105: 97 pgs: 97 active+clean; 367 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.8 MiB/s wr, 614 op/s 2024-09-19T08:18:27.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:27 smithi064 ceph-mon[34900]: pgmap v105: 97 pgs: 97 active+clean; 367 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.8 MiB/s wr, 614 op/s 2024-09-19T08:18:27.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.401+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/732868300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b18109830 msgr2=0x7f0b18109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:27.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.401+0000 7f0b1ff0f640 1 --2- 172.21.15.22:0/732868300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b18109830 0x7f0b18109c90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009900 tx=0x7f0b0802a440 comp rx=0 tx=0).stop 2024-09-19T08:18:27.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.403+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/732868300 shutdown_connections 2024-09-19T08:18:27.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.403+0000 7f0b1ff0f640 1 --2- 172.21.15.22:0/732868300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b1811b140 0x7f0b1811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:27.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.403+0000 7f0b1ff0f640 1 --2- 172.21.15.22:0/732868300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b18109830 0x7f0b18109c90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:27.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.403+0000 7f0b1ff0f640 1 --2- 172.21.15.22:0/732868300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b18109450 0x7f0b18110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:27.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.403+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/732868300 >> 172.21.15.22:0/732868300 conn(0x7f0b1806e140 msgr2=0x7f0b1806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.408+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/732868300 shutdown_connections 2024-09-19T08:18:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.410+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/732868300 wait complete. 2024-09-19T08:18:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.411+0000 7f0b1ff0f640 1 Processor -- start 2024-09-19T08:18:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.411+0000 7f0b1ff0f640 1 -- start start 2024-09-19T08:18:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.411+0000 7f0b1ff0f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b18109450 0x7f0b1811a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.411+0000 7f0b1ff0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b18109830 0x7f0b181137c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.411+0000 7f0b1ff0f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1811b140 0x7f0b18113d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.411+0000 7f0b1ff0f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0b18120ed0 con 0x7f0b1811b140 2024-09-19T08:18:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.411+0000 7f0b1ff0f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b18120d50 con 0x7f0b18109830 2024-09-19T08:18:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.411+0000 7f0b1ff0f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0b18121050 con 0x7f0b18109450 2024-09-19T08:18:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.412+0000 7f0b1dcab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b18109450 0x7f0b1811a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.412+0000 7f0b1e4ac640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1811b140 0x7f0b18113d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.412+0000 7f0b1e4ac640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1811b140 0x7f0b18113d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57014/0 (socket says 172.21.15.22:57014) 2024-09-19T08:18:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.412+0000 7f0b1e4ac640 1 -- 172.21.15.22:0/841755723 learned_addr learned my addr 172.21.15.22:0/841755723 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.412+0000 7f0b1d4aa640 1 --2- 172.21.15.22:0/841755723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b18109830 0x7f0b181137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.413+0000 7f0b1e4ac640 1 -- 172.21.15.22:0/841755723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b18109450 msgr2=0x7f0b1811a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.413+0000 7f0b1e4ac640 1 --2- 172.21.15.22:0/841755723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b18109450 0x7f0b1811a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:27.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.413+0000 7f0b1e4ac640 1 -- 172.21.15.22:0/841755723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b18109830 msgr2=0x7f0b181137c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:27.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.413+0000 7f0b1e4ac640 1 --2- 172.21.15.22:0/841755723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b18109830 0x7f0b181137c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:27.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.413+0000 7f0b1e4ac640 1 -- 172.21.15.22:0/841755723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b181142f0 con 0x7f0b1811b140 2024-09-19T08:18:27.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.413+0000 7f0b1d4aa640 1 --2- 172.21.15.22:0/841755723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b18109830 0x7f0b181137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:27.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.413+0000 7f0b1e4ac640 1 --2- 172.21.15.22:0/841755723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1811b140 0x7f0b18113d00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000bb20 tx=0x7f0b1000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:27.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.413+0000 7f0b0effd640 1 -- 172.21.15.22:0/841755723 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10019070 con 0x7f0b1811b140 2024-09-19T08:18:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.413+0000 7f0b0effd640 1 -- 172.21.15.22:0/841755723 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b10014440 con 0x7f0b1811b140 2024-09-19T08:18:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.413+0000 7f0b0effd640 1 -- 172.21.15.22:0/841755723 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b100075d0 con 0x7f0b1811b140 2024-09-19T08:18:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.414+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/841755723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b181145e0 con 0x7f0b1811b140 2024-09-19T08:18:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.414+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/841755723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b18118550 con 0x7f0b1811b140 2024-09-19T08:18:27.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.420+0000 7f0b0effd640 1 -- 172.21.15.22:0/841755723 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0b1001c070 con 0x7f0b1811b140 2024-09-19T08:18:27.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.420+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/841755723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b18073b00 con 0x7f0b1811b140 2024-09-19T08:18:27.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.422+0000 7f0b0effd640 1 --2- 172.21.15.22:0/841755723 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0b040a3500 0x7f0b040a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.422+0000 7f0b0effd640 1 -- 172.21.15.22:0/841755723 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0b100cf180 con 0x7f0b1811b140 2024-09-19T08:18:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.423+0000 7f0b1dcab640 1 --2- 172.21.15.22:0/841755723 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0b040a3500 0x7f0b040a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:27.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.423+0000 7f0b1dcab640 1 --2- 172.21.15.22:0/841755723 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0b040a3500 0x7f0b040a59c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0b14006fd0 tx=0x7f0b140090e0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:27.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.433+0000 7f0b0effd640 1 -- 172.21.15.22:0/841755723 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0b10006e20 con 0x7f0b1811b140 2024-09-19T08:18:27.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:27 smithi157 ceph-mon[35143]: pgmap v105: 97 pgs: 97 active+clean; 367 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.8 MiB/s wr, 614 op/s 2024-09-19T08:18:27.596 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:18:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.793+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/841755723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b18119000 con 0x7f0b1811b140 2024-09-19T08:18:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.793+0000 7f0b0effd640 1 -- 172.21.15.22:0/841755723 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11910 (secure 0 0 0) 0x7f0b1009b5d0 con 0x7f0b1811b140 2024-09-19T08:18:27.794 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:18:27.794 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"btime":"2024-09-19T08:18:06:960387+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:18:06.784015+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:18:27.794 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 28 2024-09-19T08:18:27.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.795+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/841755723 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0b040a3500 msgr2=0x7f0b040a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:27.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.795+0000 7f0b1ff0f640 1 --2- 172.21.15.22:0/841755723 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0b040a3500 0x7f0b040a59c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0b14006fd0 tx=0x7f0b140090e0 comp rx=0 tx=0).stop 2024-09-19T08:18:27.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.795+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/841755723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1811b140 msgr2=0x7f0b18113d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:27.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.795+0000 7f0b1ff0f640 1 --2- 172.21.15.22:0/841755723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1811b140 0x7f0b18113d00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000bb20 tx=0x7f0b1000c0c0 comp rx=0 tx=0).stop 2024-09-19T08:18:27.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.795+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/841755723 shutdown_connections 2024-09-19T08:18:27.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.795+0000 7f0b1ff0f640 1 --2- 172.21.15.22:0/841755723 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0b040a3500 0x7f0b040a59c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:27.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.795+0000 7f0b1ff0f640 1 --2- 172.21.15.22:0/841755723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1811b140 0x7f0b18113d00 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:27.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.795+0000 7f0b1ff0f640 1 --2- 172.21.15.22:0/841755723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b18109830 0x7f0b181137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:27.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.795+0000 7f0b1ff0f640 1 --2- 172.21.15.22:0/841755723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b18109450 0x7f0b1811a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:27.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.795+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/841755723 >> 172.21.15.22:0/841755723 conn(0x7f0b1806e140 msgr2=0x7f0b180723e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:27.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.796+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/841755723 shutdown_connections 2024-09-19T08:18:27.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:27.796+0000 7f0b1ff0f640 1 -- 172.21.15.22:0/841755723 wait complete. 2024-09-19T08:18:28.318 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:28 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/841755723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:18:28.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:28 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/841755723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:18:28.502 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:28 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/841755723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:18:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.317+0000 7f4779e90640 1 -- 172.21.15.22:0/3303350720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4774100430 msgr2=0x7f4774100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.317+0000 7f4779e90640 1 --2- 172.21.15.22:0/3303350720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4774100430 0x7f4774100810 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4764009900 tx=0x7f476402a410 comp rx=0 tx=0).stop 2024-09-19T08:18:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.318+0000 7f4779e90640 1 -- 172.21.15.22:0/3303350720 shutdown_connections 2024-09-19T08:18:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.318+0000 7f4779e90640 1 --2- 172.21.15.22:0/3303350720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4774105560 0x7f477410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.318+0000 7f4779e90640 1 --2- 172.21.15.22:0/3303350720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4774100de0 0x7f4774104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.318+0000 7f4779e90640 1 --2- 172.21.15.22:0/3303350720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4774100430 0x7f4774100810 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.318+0000 7f4779e90640 1 -- 172.21.15.22:0/3303350720 >> 172.21.15.22:0/3303350720 conn(0x7f4774077230 msgr2=0x7f47740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.318+0000 7f4779e90640 1 -- 172.21.15.22:0/3303350720 shutdown_connections 2024-09-19T08:18:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.318+0000 7f4779e90640 1 -- 172.21.15.22:0/3303350720 wait complete. 2024-09-19T08:18:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.319+0000 7f4779e90640 1 Processor -- start 2024-09-19T08:18:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.319+0000 7f4779e90640 1 -- start start 2024-09-19T08:18:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.319+0000 7f4779e90640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4774100430 0x7f477419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.319+0000 7f4779e90640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4774100de0 0x7f477419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.319+0000 7f4779e90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4774105560 0x7f477419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f4779e90640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f477410fd40 con 0x7f4774100430 2024-09-19T08:18:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f4779e90640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f477410fbc0 con 0x7f4774100de0 2024-09-19T08:18:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f4779e90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f477410fec0 con 0x7f4774105560 2024-09-19T08:18:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f47737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4774100430 0x7f477419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f47737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4774100430 0x7f477419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57032/0 (socket says 172.21.15.22:57032) 2024-09-19T08:18:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f47737fe640 1 -- 172.21.15.22:0/3907470486 learned_addr learned my addr 172.21.15.22:0/3907470486 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f4773fff640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4774105560 0x7f477419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f4772ffd640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4774100de0 0x7f477419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f4773fff640 1 -- 172.21.15.22:0/3907470486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4774100de0 msgr2=0x7f477419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f4773fff640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4774100de0 0x7f477419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f4773fff640 1 -- 172.21.15.22:0/3907470486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4774100430 msgr2=0x7f477419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f4773fff640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4774100430 0x7f477419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f4773fff640 1 -- 172.21.15.22:0/3907470486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477419f5c0 con 0x7f4774105560 2024-09-19T08:18:29.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f4772ffd640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4774100de0 0x7f477419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:18:29.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f4773fff640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4774105560 0x7f477419eee0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f476c005fd0 tx=0x7f476c00c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:29.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.320+0000 7f4770ff9640 1 -- 172.21.15.22:0/3907470486 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c004d20 con 0x7f4774105560 2024-09-19T08:18:29.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.321+0000 7f4770ff9640 1 -- 172.21.15.22:0/3907470486 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f476c00ba50 con 0x7f4774105560 2024-09-19T08:18:29.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.321+0000 7f4779e90640 1 -- 172.21.15.22:0/3907470486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477419f8b0 con 0x7f4774105560 2024-09-19T08:18:29.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.321+0000 7f4770ff9640 1 -- 172.21.15.22:0/3907470486 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c005780 con 0x7f4774105560 2024-09-19T08:18:29.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.321+0000 7f4779e90640 1 -- 172.21.15.22:0/3907470486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47741a7190 con 0x7f4774105560 2024-09-19T08:18:29.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.323+0000 7f4770ff9640 1 -- 172.21.15.22:0/3907470486 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f476c005920 con 0x7f4774105560 2024-09-19T08:18:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.324+0000 7f477c269740 1 Processor -- start 2024-09-19T08:18:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.324+0000 7f477c269740 1 -- start start 2024-09-19T08:18:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.324+0000 7f477c269740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3977b40 0x5600e3976090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.324+0000 7f477c269740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e39765d0 0x5600e3976a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.324+0000 7f477c269740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3981370 0x5600e3983760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.324+0000 7f477c269740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5600e38dc2f0 con 0x5600e39765d0 2024-09-19T08:18:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.324+0000 7f477c269740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5600e3883cf0 con 0x5600e3977b40 2024-09-19T08:18:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.324+0000 7f477c269740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5600e38d82b0 con 0x5600e3981370 2024-09-19T08:18:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.325+0000 7f4770ff9640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f475c0a33b0 0x7f475c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.325+0000 7f47737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e39765d0 0x5600e3976a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.325+0000 7f47737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e39765d0 0x5600e3976a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57046/0 (socket says 172.21.15.22:57046) 2024-09-19T08:18:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.325+0000 7f47737fe640 1 -- 172.21.15.22:0/1963427015 learned_addr learned my addr 172.21.15.22:0/1963427015 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.325+0000 7f4772ffd640 1 --2- 172.21.15.22:0/1963427015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3981370 0x5600e3983760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.325+0000 7f4773fff640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f475c0a33b0 0x7f475c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.325+0000 7f47737fe640 1 -- 172.21.15.22:0/1963427015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3981370 msgr2=0x5600e3983760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.325+0000 7f47737fe640 1 --2- 172.21.15.22:0/1963427015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3981370 0x5600e3983760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.325+0000 7f47737fe640 1 -- 172.21.15.22:0/1963427015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3977b40 msgr2=0x5600e3976090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.325+0000 7f47737fe640 1 --2- 172.21.15.22:0/1963427015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3977b40 0x5600e3976090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.325+0000 7f47737fe640 1 -- 172.21.15.22:0/1963427015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600e38683a0 con 0x5600e39765d0 2024-09-19T08:18:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.325+0000 7f4772ffd640 1 --2- 172.21.15.22:0/1963427015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3981370 0x5600e3983760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:18:29.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.326+0000 7f47737fe640 1 --2- 172.21.15.22:0/1963427015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e39765d0 0x5600e3976a10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4764009a20 tx=0x7f4764005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be5ba76bed33a14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:29.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.326+0000 7f4773fff640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f475c0a33b0 0x7f475c0a5870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f476c0cd930 tx=0x7f476c0d39c0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:29.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.326+0000 7f4770ff9640 1 -- 172.21.15.22:0/3907470486 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f476c0d0200 con 0x7f4774105560 2024-09-19T08:18:29.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.326+0000 7f4759ffb640 1 -- 172.21.15.22:0/1963427015 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764040070 con 0x5600e39765d0 2024-09-19T08:18:29.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.326+0000 7f4759ffb640 1 -- 172.21.15.22:0/1963427015 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47640317b0 con 0x5600e39765d0 2024-09-19T08:18:29.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.326+0000 7f4759ffb640 1 -- 172.21.15.22:0/1963427015 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764035040 con 0x5600e39765d0 2024-09-19T08:18:29.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.327+0000 7f477c269740 1 -- 172.21.15.22:0/1963427015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e39765d0 msgr2=0x5600e3976a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:29.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.327+0000 7f477c269740 1 --2- 172.21.15.22:0/1963427015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e39765d0 0x5600e3976a10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4764009a20 tx=0x7f4764005e80 comp rx=0 tx=0).stop 2024-09-19T08:18:29.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.327+0000 7f477c269740 1 -- 172.21.15.22:0/1963427015 shutdown_connections 2024-09-19T08:18:29.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.327+0000 7f477c269740 1 --2- 172.21.15.22:0/1963427015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3981370 0x5600e3983760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.327+0000 7f477c269740 1 --2- 172.21.15.22:0/1963427015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e39765d0 0x5600e3976a10 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.327+0000 7f477c269740 1 --2- 172.21.15.22:0/1963427015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3977b40 0x5600e3976090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.327+0000 7f477c269740 1 -- 172.21.15.22:0/1963427015 >> 172.21.15.22:0/1963427015 conn(0x5600e393f310 msgr2=0x5600e3972c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.327+0000 7f477c269740 1 -- 172.21.15.22:0/1963427015 shutdown_connections 2024-09-19T08:18:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.327+0000 7f477c269740 1 -- 172.21.15.22:0/1963427015 wait complete. 2024-09-19T08:18:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.327+0000 7f477c269740 1 -- start start 2024-09-19T08:18:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.328+0000 7f477c269740 1 Processor -- start 2024-09-19T08:18:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.328+0000 7f477c269740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e39765d0 0x5600e3980d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.328+0000 7f477c269740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3977b40 0x5600e397e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.328+0000 7f477c269740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3981370 0x5600e397e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.328+0000 7f477c269740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5600e3984010 con 0x5600e39765d0 2024-09-19T08:18:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.328+0000 7f477c269740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5600e3897060 con 0x5600e3981370 2024-09-19T08:18:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.328+0000 7f477c269740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5600e3984190 con 0x5600e3977b40 2024-09-19T08:18:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.328+0000 7f4772ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e39765d0 0x5600e3980d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.328+0000 7f47737fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3977b40 0x5600e397e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.328+0000 7f4772ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e39765d0 0x5600e3980d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57058/0 (socket says 172.21.15.22:57058) 2024-09-19T08:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.328+0000 7f4772ffd640 1 -- 172.21.15.22:0/4265482531 learned_addr learned my addr 172.21.15.22:0/4265482531 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.328+0000 7f477c269740 10 client.0 _ll_register_callbacks cb 0x5600e35ea8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.329+0000 7f4772ffd640 1 --2- 172.21.15.22:0/4265482531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3981370 0x5600e397e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.329+0000 7f47737fe640 1 -- 172.21.15.22:0/4265482531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3981370 msgr2=0x5600e397e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.329+0000 7f47737fe640 1 --2- 172.21.15.22:0/4265482531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3981370 0x5600e397e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.329+0000 7f47737fe640 1 -- 172.21.15.22:0/4265482531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e39765d0 msgr2=0x5600e3980d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.329+0000 7f47737fe640 1 --2- 172.21.15.22:0/4265482531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e39765d0 0x5600e3980d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.329+0000 7f47737fe640 1 -- 172.21.15.22:0/4265482531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600e364d3c0 con 0x5600e3977b40 2024-09-19T08:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.329+0000 7f4772ffd640 1 --2- 172.21.15.22:0/4265482531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3981370 0x5600e397e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.329+0000 7f47737fe640 1 --2- 172.21.15.22:0/4265482531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3977b40 0x5600e397e030 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4764031550 tx=0x7f47640051c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f518c716070d9be7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.329+0000 7f4752ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.329+0000 7f4752ffd640 1 -- 172.21.15.22:0/4265482531 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476404c020 con 0x5600e3977b40 2024-09-19T08:18:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.329+0000 7f4752ffd640 1 -- 172.21.15.22:0/4265482531 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f47640055f0 con 0x5600e3977b40 2024-09-19T08:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.329+0000 7f477c269740 1 -- 172.21.15.22:0/4265482531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47640317b0 con 0x5600e3977b40 2024-09-19T08:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.330+0000 7f4752ffd640 1 -- 172.21.15.22:0/4265482531 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4764057050 con 0x5600e3977b40 2024-09-19T08:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.330+0000 7f477c269740 10 client.25636 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.330+0000 7f477c269740 1 -- 172.21.15.22:0/4265482531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5600e3984010 con 0x5600e3977b40 2024-09-19T08:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.330+0000 7f4752ffd640 1 -- 172.21.15.22:0/4265482531 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47640317b0 con 0x5600e3977b40 2024-09-19T08:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.330+0000 7f4752ffd640 1 -- 172.21.15.22:0/4265482531 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f476403c760 con 0x5600e3977b40 2024-09-19T08:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.330+0000 7f477c269740 10 client.25636 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.330+0000 7f477c269740 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.330+0000 7f477c269740 20 client.25636 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.330+0000 7f477c269740 1 --2- 172.21.15.22:0/4265482531 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5600e3990f10 0x5600e3993300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.330+0000 7f477c269740 4 client.25636 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.330+0000 7f477c269740 1 -- 172.21.15.22:0/4265482531 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600e38e9860 con 0x5600e3990f10 2024-09-19T08:18:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.330+0000 7f4772ffd640 1 --2- 172.21.15.22:0/4265482531 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5600e3990f10 0x5600e3993300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.331+0000 7f4752ffd640 10 client.25636 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.331+0000 7f4772ffd640 1 --2- 172.21.15.22:0/4265482531 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5600e3990f10 0x5600e3993300 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=890b3270580e3c7d server_cookie=743241ef4e49710 in_seq=0 out_seq=0 2024-09-19T08:18:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f4752ffd640 1 -- 172.21.15.22:0/4265482531 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600e38e9860 con 0x5600e3990f10 2024-09-19T08:18:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f4752ffd640 10 client.25636 handle_command_reply: tid=0 2024-09-19T08:18:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f477c269740 1 client.25636 shutdown 2024-09-19T08:18:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f477c269740 1 -- 172.21.15.22:0/4265482531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3977b40 msgr2=0x5600e397e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f477c269740 1 --2- 172.21.15.22:0/4265482531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3977b40 0x5600e397e030 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4764031550 tx=0x7f47640051c0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f477c269740 1 -- 172.21.15.22:0/4265482531 shutdown_connections 2024-09-19T08:18:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f477c269740 1 --2- 172.21.15.22:0/4265482531 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5600e3990f10 0x5600e3993300 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f477c269740 1 --2- 172.21.15.22:0/4265482531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3981370 0x5600e397e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f477c269740 1 --2- 172.21.15.22:0/4265482531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3977b40 0x5600e397e030 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f477c269740 1 --2- 172.21.15.22:0/4265482531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e39765d0 0x5600e3980d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f477c269740 1 -- 172.21.15.22:0/4265482531 >> 172.21.15.22:0/4265482531 conn(0x5600e393f310 msgr2=0x5600e3978930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f4752ffd640 0 client.25636 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f477c269740 1 -- 172.21.15.22:0/4265482531 shutdown_connections 2024-09-19T08:18:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f477c269740 1 -- 172.21.15.22:0/4265482531 wait complete. 2024-09-19T08:18:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.332+0000 7f477c269740 20 client.25636 trim_cache size 0 max 16384 2024-09-19T08:18:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.346+0000 7f477c269740 1 Processor -- start 2024-09-19T08:18:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.346+0000 7f477c269740 1 -- start start 2024-09-19T08:18:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f477c269740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e39765d0 0x5600e398d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f477c269740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3977b40 0x5600e3972e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f477c269740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e3981370 0x5600e3996320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f477c269740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f47741b2410 con 0x5600e3981370 2024-09-19T08:18:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f477c269740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47741afc80 con 0x5600e3977b40 2024-09-19T08:18:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f477c269740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47741adad0 con 0x5600e39765d0 2024-09-19T08:18:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f4772ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e3981370 0x5600e3996320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f4772ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e3981370 0x5600e3996320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57062/0 (socket says 172.21.15.22:57062) 2024-09-19T08:18:29.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f4772ffd640 1 -- 172.21.15.22:0/185278317 learned_addr learned my addr 172.21.15.22:0/185278317 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:29.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f4772ffd640 1 --2- 172.21.15.22:0/185278317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e39765d0 0x5600e398d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f47737fe640 1 --2- 172.21.15.22:0/185278317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3977b40 0x5600e3972e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f4772ffd640 1 -- 172.21.15.22:0/185278317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e39765d0 msgr2=0x5600e398d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:29.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f4772ffd640 1 --2- 172.21.15.22:0/185278317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e39765d0 0x5600e398d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f4772ffd640 1 -- 172.21.15.22:0/185278317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3977b40 msgr2=0x5600e3972e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:29.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f4772ffd640 1 --2- 172.21.15.22:0/185278317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3977b40 0x5600e3972e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f4772ffd640 1 -- 172.21.15.22:0/185278317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600e39810f0 con 0x5600e3981370 2024-09-19T08:18:29.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.347+0000 7f4772ffd640 1 --2- 172.21.15.22:0/185278317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e3981370 0x5600e3996320 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f47680058c0 tx=0x7f4768004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea96032a12ff2ae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:29.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.348+0000 7f47597fa640 1 -- 172.21.15.22:0/185278317 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768013d90 con 0x5600e3981370 2024-09-19T08:18:29.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.348+0000 7f47597fa640 1 -- 172.21.15.22:0/185278317 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5600e38e9860 con 0x5600e3981370 2024-09-19T08:18:29.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.348+0000 7f47597fa640 1 -- 172.21.15.22:0/185278317 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476801c990 con 0x5600e3981370 2024-09-19T08:18:29.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.348+0000 7f477c269740 1 -- 172.21.15.22:0/185278317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e3981370 msgr2=0x5600e3996320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:29.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.348+0000 7f477c269740 1 --2- 172.21.15.22:0/185278317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e3981370 0x5600e3996320 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f47680058c0 tx=0x7f4768004730 comp rx=0 tx=0).stop 2024-09-19T08:18:29.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 -- 172.21.15.22:0/185278317 shutdown_connections 2024-09-19T08:18:29.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 --2- 172.21.15.22:0/185278317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e3981370 0x5600e3996320 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 --2- 172.21.15.22:0/185278317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e3977b40 0x5600e3972e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 --2- 172.21.15.22:0/185278317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e39765d0 0x5600e398d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 -- 172.21.15.22:0/185278317 >> 172.21.15.22:0/185278317 conn(0x5600e393f310 msgr2=0x5600e3972700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:29.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 -- 172.21.15.22:0/185278317 shutdown_connections 2024-09-19T08:18:29.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 -- 172.21.15.22:0/185278317 wait complete. 2024-09-19T08:18:29.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 -- start start 2024-09-19T08:18:29.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 Processor -- start 2024-09-19T08:18:29.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e39765d0 0x5600e3994220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3977b40 0x5600e3994760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e3981370 0x5600e3994ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f47741ae140 con 0x5600e3981370 2024-09-19T08:18:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47741af5b0 con 0x5600e39765d0 2024-09-19T08:18:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.349+0000 7f477c269740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47741ae810 con 0x5600e3977b40 2024-09-19T08:18:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.350+0000 7f477c269740 10 client.0 _ll_register_callbacks cb 0x7f4764005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.350+0000 7f47737fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3977b40 0x5600e3994760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.350+0000 7f4772ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e39765d0 0x5600e3994220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.350+0000 7f47737fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3977b40 0x5600e3994760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:50490/0 (socket says 172.21.15.22:50490) 2024-09-19T08:18:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.350+0000 7f47737fe640 1 -- 172.21.15.22:0/1410753267 learned_addr learned my addr 172.21.15.22:0/1410753267 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.350+0000 7f4772ffd640 1 --2- 172.21.15.22:0/1410753267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e3981370 0x5600e3994ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.350+0000 7f47737fe640 1 -- 172.21.15.22:0/1410753267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e39765d0 msgr2=0x5600e3994220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.350+0000 7f47737fe640 1 --2- 172.21.15.22:0/1410753267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e39765d0 0x5600e3994220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.350+0000 7f47737fe640 1 -- 172.21.15.22:0/1410753267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e3981370 msgr2=0x5600e3994ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.350+0000 7f47737fe640 1 --2- 172.21.15.22:0/1410753267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e3981370 0x5600e3994ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.350+0000 7f47737fe640 1 -- 172.21.15.22:0/1410753267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600e39fa9f0 con 0x5600e3977b40 2024-09-19T08:18:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.350+0000 7f47737fe640 1 --2- 172.21.15.22:0/1410753267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3977b40 0x5600e3994760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4764005ec0 tx=0x7f4764009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b738841b61f30b94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.350+0000 7f4752ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:18:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.351+0000 7f4752ffd640 1 -- 172.21.15.22:0/1410753267 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764060060 con 0x5600e3977b40 2024-09-19T08:18:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.351+0000 7f4752ffd640 1 -- 172.21.15.22:0/1410753267 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4764031b70 con 0x5600e3977b40 2024-09-19T08:18:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.351+0000 7f477c269740 1 -- 172.21.15.22:0/1410753267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5600e38e9860 con 0x5600e3977b40 2024-09-19T08:18:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.351+0000 7f4752ffd640 1 -- 172.21.15.22:0/1410753267 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f476404c040 con 0x5600e3977b40 2024-09-19T08:18:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.351+0000 7f477c269740 10 client.25642 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.351+0000 7f477c269740 1 -- 172.21.15.22:0/1410753267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5600e39fbc90 con 0x5600e3977b40 2024-09-19T08:18:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.351+0000 7f4752ffd640 1 -- 172.21.15.22:0/1410753267 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5600e38e9860 con 0x5600e3977b40 2024-09-19T08:18:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.351+0000 7f4752ffd640 1 -- 172.21.15.22:0/1410753267 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f47640048d0 con 0x5600e3977b40 2024-09-19T08:18:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.351+0000 7f477c269740 10 client.25642 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.351+0000 7f477c269740 10 client.25642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:18:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.351+0000 7f477c269740 20 client.25642 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.351+0000 7f477c269740 1 --2- 172.21.15.22:0/1410753267 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5600e39fc440 0x5600e39fe860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.351+0000 7f477c269740 4 client.25642 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:18:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.351+0000 7f477c269740 1 -- 172.21.15.22:0/1410753267 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600e38e3d70 con 0x5600e39fc440 2024-09-19T08:18:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.352+0000 7f4772ffd640 1 --2- 172.21.15.22:0/1410753267 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5600e39fc440 0x5600e39fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.352+0000 7f4752ffd640 10 client.25642 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.352+0000 7f4772ffd640 1 --2- 172.21.15.22:0/1410753267 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5600e39fc440 0x5600e39fe860 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9281eed956cab932 server_cookie=3ad0d9cb5fd2d6f4 in_seq=0 out_seq=0 2024-09-19T08:18:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.352+0000 7f4752ffd640 1 -- 172.21.15.22:0/1410753267 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600e38e3d70 con 0x5600e39fc440 2024-09-19T08:18:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.352+0000 7f4752ffd640 10 client.25642 handle_command_reply: tid=0 2024-09-19T08:18:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.352+0000 7f477c269740 1 client.25642 shutdown 2024-09-19T08:18:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.353+0000 7f477c269740 1 -- 172.21.15.22:0/1410753267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3977b40 msgr2=0x5600e3994760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.353+0000 7f477c269740 1 --2- 172.21.15.22:0/1410753267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3977b40 0x5600e3994760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4764005ec0 tx=0x7f4764009590 comp rx=0 tx=0).stop 2024-09-19T08:18:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.353+0000 7f477c269740 1 -- 172.21.15.22:0/1410753267 shutdown_connections 2024-09-19T08:18:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.353+0000 7f477c269740 1 --2- 172.21.15.22:0/1410753267 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5600e39fc440 0x5600e39fe860 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.353+0000 7f477c269740 1 --2- 172.21.15.22:0/1410753267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e3981370 0x5600e3994ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.353+0000 7f477c269740 1 --2- 172.21.15.22:0/1410753267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600e3977b40 0x5600e3994760 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.353+0000 7f477c269740 1 --2- 172.21.15.22:0/1410753267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600e39765d0 0x5600e3994220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.353+0000 7f477c269740 1 -- 172.21.15.22:0/1410753267 >> 172.21.15.22:0/1410753267 conn(0x5600e393f310 msgr2=0x5600e397f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.353+0000 7f4752ffd640 0 client.25642 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.353+0000 7f477c269740 1 -- 172.21.15.22:0/1410753267 shutdown_connections 2024-09-19T08:18:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.353+0000 7f477c269740 1 -- 172.21.15.22:0/1410753267 wait complete. 2024-09-19T08:18:29.087 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:18:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.353+0000 7f477c269740 20 client.25642 trim_cache size 0 max 16384 2024-09-19T08:18:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.354+0000 7f4779e90640 1 -- 172.21.15.22:0/3907470486 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f475c0a33b0 msgr2=0x7f475c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.354+0000 7f4779e90640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f475c0a33b0 0x7f475c0a5870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f476c0cd930 tx=0x7f476c0d39c0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.354+0000 7f4779e90640 1 -- 172.21.15.22:0/3907470486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4774105560 msgr2=0x7f477419eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.354+0000 7f4779e90640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4774105560 0x7f477419eee0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f476c005fd0 tx=0x7f476c00c7a0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.354+0000 7f4779e90640 1 -- 172.21.15.22:0/3907470486 shutdown_connections 2024-09-19T08:18:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.354+0000 7f4779e90640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f475c0a33b0 0x7f475c0a5870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.354+0000 7f4779e90640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4774105560 0x7f477419eee0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.354+0000 7f4779e90640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4774100de0 0x7f477419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.354+0000 7f4779e90640 1 --2- 172.21.15.22:0/3907470486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4774100430 0x7f477419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.354+0000 7f4779e90640 1 -- 172.21.15.22:0/3907470486 >> 172.21.15.22:0/3907470486 conn(0x7f4774077230 msgr2=0x7f47740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.354+0000 7f4779e90640 1 -- 172.21.15.22:0/3907470486 shutdown_connections 2024-09-19T08:18:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:28.354+0000 7f4779e90640 1 -- 172.21.15.22:0/3907470486 wait complete. 2024-09-19T08:18:29.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:29 smithi064 ceph-mon[34900]: pgmap v106: 97 pgs: 97 active+clean; 367 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.8 MiB/s wr, 622 op/s 2024-09-19T08:18:29.440 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:29 smithi022 ceph-mon[30963]: pgmap v106: 97 pgs: 97 active+clean; 367 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.8 MiB/s wr, 622 op/s 2024-09-19T08:18:29.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:29 smithi157 ceph-mon[35143]: pgmap v106: 97 pgs: 97 active+clean; 367 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.8 MiB/s wr, 622 op/s 2024-09-19T08:18:30.367 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-19T08:18:30.367 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-19T08:18:31.093 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:18:31.327 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:18:31.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:18:31.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:31 smithi064 ceph-mon[34900]: pgmap v107: 97 pgs: 97 active+clean; 378 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 657 op/s 2024-09-19T08:18:31.352 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:31 smithi022 ceph-mon[30963]: pgmap v107: 97 pgs: 97 active+clean; 378 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 657 op/s 2024-09-19T08:18:31.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:31 smithi157 ceph-mon[35143]: pgmap v107: 97 pgs: 97 active+clean; 378 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 657 op/s 2024-09-19T08:18:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.618+0000 7fd73a144640 1 -- 172.21.15.22:0/4040025567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7341027a0 msgr2=0x7fd7341067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.618+0000 7fd73a144640 1 --2- 172.21.15.22:0/4040025567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7341027a0 0x7fd7341067f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd728009900 tx=0x7fd72802a410 comp rx=0 tx=0).stop 2024-09-19T08:18:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.618+0000 7fd73a144640 1 -- 172.21.15.22:0/4040025567 shutdown_connections 2024-09-19T08:18:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.618+0000 7fd73a144640 1 --2- 172.21.15.22:0/4040025567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd734106f20 0x7fd73410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.618+0000 7fd73a144640 1 --2- 172.21.15.22:0/4040025567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7341027a0 0x7fd7341067f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.618+0000 7fd73a144640 1 --2- 172.21.15.22:0/4040025567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd734101df0 0x7fd7341021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.618+0000 7fd73a144640 1 -- 172.21.15.22:0/4040025567 >> 172.21.15.22:0/4040025567 conn(0x7fd7340fd8f0 msgr2=0x7fd7340ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.618+0000 7fd73a144640 1 -- 172.21.15.22:0/4040025567 shutdown_connections 2024-09-19T08:18:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.618+0000 7fd73a144640 1 -- 172.21.15.22:0/4040025567 wait complete. 2024-09-19T08:18:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.619+0000 7fd73a144640 1 Processor -- start 2024-09-19T08:18:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.619+0000 7fd73a144640 1 -- start start 2024-09-19T08:18:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.619+0000 7fd73a144640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd734101df0 0x7fd734198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.619+0000 7fd73a144640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7341027a0 0x7fd734198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.619+0000 7fd73a144640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd734106f20 0x7fd73419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.619+0000 7fd73a144640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd734111fd0 con 0x7fd734101df0 2024-09-19T08:18:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.619+0000 7fd73a144640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd734111e50 con 0x7fd734106f20 2024-09-19T08:18:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.619+0000 7fd73a144640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd734112150 con 0x7fd7341027a0 2024-09-19T08:18:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.620+0000 7fd732ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7341027a0 0x7fd734198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.620+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd734101df0 0x7fd734198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.620+0000 7fd733fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd734106f20 0x7fd73419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.620+0000 7fd732ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7341027a0 0x7fd734198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:50516/0 (socket says 172.21.15.22:50516) 2024-09-19T08:18:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.620+0000 7fd732ffd640 1 -- 172.21.15.22:0/347769800 learned_addr learned my addr 172.21.15.22:0/347769800 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.620+0000 7fd732ffd640 1 -- 172.21.15.22:0/347769800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd734106f20 msgr2=0x7fd73419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.620+0000 7fd732ffd640 1 --2- 172.21.15.22:0/347769800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd734106f20 0x7fd73419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.620+0000 7fd732ffd640 1 -- 172.21.15.22:0/347769800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd734101df0 msgr2=0x7fd734198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.620+0000 7fd732ffd640 1 --2- 172.21.15.22:0/347769800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd734101df0 0x7fd734198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.620+0000 7fd732ffd640 1 -- 172.21.15.22:0/347769800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73419d4e0 con 0x7fd7341027a0 2024-09-19T08:18:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.620+0000 7fd7337fe640 1 --2- 172.21.15.22:0/347769800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd734101df0 0x7fd734198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:31.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.620+0000 7fd732ffd640 1 --2- 172.21.15.22:0/347769800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7341027a0 0x7fd734198a70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd728004770 tx=0x7fd728003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:31.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.621+0000 7fd730ff9640 1 -- 172.21.15.22:0/347769800 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72803f070 con 0x7fd7341027a0 2024-09-19T08:18:31.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.621+0000 7fd730ff9640 1 -- 172.21.15.22:0/347769800 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd728031ca0 con 0x7fd7341027a0 2024-09-19T08:18:31.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.621+0000 7fd730ff9640 1 -- 172.21.15.22:0/347769800 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728035030 con 0x7fd7341027a0 2024-09-19T08:18:31.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.621+0000 7fd73a144640 1 -- 172.21.15.22:0/347769800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73419d770 con 0x7fd7341027a0 2024-09-19T08:18:31.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.621+0000 7fd73a144640 1 -- 172.21.15.22:0/347769800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73419dc00 con 0x7fd7341027a0 2024-09-19T08:18:31.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.624+0000 7fd73c51d740 1 Processor -- start 2024-09-19T08:18:31.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.624+0000 7fd73c51d740 1 -- start start 2024-09-19T08:18:31.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.624+0000 7fd730ff9640 1 -- 172.21.15.22:0/347769800 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd728042070 con 0x7fd7341027a0 2024-09-19T08:18:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.624+0000 7fd73c51d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db665b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.624+0000 7fd73c51d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 0x55b0db670400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd73c51d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db672d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd73c51d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b0db5cb2a0 con 0x55b0db670940 2024-09-19T08:18:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd73c51d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b0db572b10 con 0x55b0db665760 2024-09-19T08:18:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd73c51d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b0db5c7260 con 0x55b0db666080 2024-09-19T08:18:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd733fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db665b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd733fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db665b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:35550/0 (socket says 172.21.15.22:35550) 2024-09-19T08:18:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd733fff640 1 -- 172.21.15.22:0/964549300 learned_addr learned my addr 172.21.15.22:0/964549300 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd7337fe640 1 --2- 172.21.15.22:0/964549300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 0x55b0db670400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd732ffd640 1 --2- 172.21.15.22:0/964549300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db672d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd733fff640 1 -- 172.21.15.22:0/964549300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 msgr2=0x55b0db670400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd733fff640 1 --2- 172.21.15.22:0/964549300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 0x55b0db670400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd733fff640 1 -- 172.21.15.22:0/964549300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 msgr2=0x55b0db672d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd733fff640 1 --2- 172.21.15.22:0/964549300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db672d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd733fff640 1 -- 172.21.15.22:0/964549300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0db33cb90 con 0x55b0db665760 2024-09-19T08:18:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd732ffd640 1 --2- 172.21.15.22:0/964549300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db672d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:18:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.625+0000 7fd733fff640 1 --2- 172.21.15.22:0/964549300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db665b40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009f50 tx=0x7fd72c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81530f5a62dfce36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.626+0000 7fd730ff9640 1 --2- 172.21.15.22:0/347769800 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd71c0a33b0 0x7fd71c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.626+0000 7fd719ffb640 1 -- 172.21.15.22:0/964549300 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c004d20 con 0x55b0db665760 2024-09-19T08:18:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.626+0000 7fd719ffb640 1 -- 172.21.15.22:0/964549300 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd72c004050 con 0x55b0db665760 2024-09-19T08:18:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.626+0000 7fd719ffb640 1 -- 172.21.15.22:0/964549300 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c005280 con 0x55b0db665760 2024-09-19T08:18:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.626+0000 7fd7337fe640 1 --2- 172.21.15.22:0/347769800 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd71c0a33b0 0x7fd71c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.626+0000 7fd730ff9640 1 -- 172.21.15.22:0/347769800 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd7280f0330 con 0x7fd7341027a0 2024-09-19T08:18:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.627+0000 7fd73c51d740 1 -- 172.21.15.22:0/964549300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 msgr2=0x55b0db665b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.627+0000 7fd7337fe640 1 --2- 172.21.15.22:0/347769800 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd71c0a33b0 0x7fd71c0a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55b0db662b50 tx=0x7fd724012040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.627+0000 7fd73c51d740 1 --2- 172.21.15.22:0/964549300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db665b40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009f50 tx=0x7fd72c0118d0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.627+0000 7fd73c51d740 1 -- 172.21.15.22:0/964549300 shutdown_connections 2024-09-19T08:18:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.627+0000 7fd73c51d740 1 --2- 172.21.15.22:0/964549300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db672d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.627+0000 7fd73c51d740 1 --2- 172.21.15.22:0/964549300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 0x55b0db670400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.627+0000 7fd73c51d740 1 --2- 172.21.15.22:0/964549300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db665b40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.627+0000 7fd73c51d740 1 -- 172.21.15.22:0/964549300 >> 172.21.15.22:0/964549300 conn(0x55b0db661cf0 msgr2=0x55b0db6620d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.627+0000 7fd73c51d740 1 -- 172.21.15.22:0/964549300 shutdown_connections 2024-09-19T08:18:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.627+0000 7fd73c51d740 1 -- 172.21.15.22:0/964549300 wait complete. 2024-09-19T08:18:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.628+0000 7fd73c51d740 1 -- start start 2024-09-19T08:18:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.628+0000 7fd73c51d740 1 Processor -- start 2024-09-19T08:18:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.628+0000 7fd73c51d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db66d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.628+0000 7fd73c51d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 0x55b0db66de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.628+0000 7fd73c51d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db66e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd73c51d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b0db673510 con 0x55b0db670940 2024-09-19T08:18:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd73c51d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b0db5536e0 con 0x55b0db665760 2024-09-19T08:18:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd73c51d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b0db673690 con 0x55b0db666080 2024-09-19T08:18:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db66d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd733fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db66e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db66d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:35556/0 (socket says 172.21.15.22:35556) 2024-09-19T08:18:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd733fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db66e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57104/0 (socket says 172.21.15.22:57104) 2024-09-19T08:18:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd7337fe640 1 -- 172.21.15.22:0/360788173 learned_addr learned my addr 172.21.15.22:0/360788173 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd732ffd640 1 --2- 172.21.15.22:0/360788173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 0x55b0db66de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd73c51d740 10 client.0 _ll_register_callbacks cb 0x55b0db577a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd733fff640 1 -- 172.21.15.22:0/360788173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 msgr2=0x55b0db66de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd733fff640 1 --2- 172.21.15.22:0/360788173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 0x55b0db66de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd733fff640 1 -- 172.21.15.22:0/360788173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 msgr2=0x55b0db66d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd733fff640 1 --2- 172.21.15.22:0/360788173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db66d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd733fff640 1 -- 172.21.15.22:0/360788173 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0db67cae0 con 0x55b0db670940 2024-09-19T08:18:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd732ffd640 1 --2- 172.21.15.22:0/360788173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 0x55b0db66de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:18:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd7337fe640 1 --2- 172.21.15.22:0/360788173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db66d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd733fff640 1 --2- 172.21.15.22:0/360788173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db66e360 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd72c004740 tx=0x7fd72c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767bb8558f18cafe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.629+0000 7fd712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:18:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.630+0000 7fd712ffd640 1 -- 172.21.15.22:0/360788173 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c029020 con 0x55b0db670940 2024-09-19T08:18:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.630+0000 7fd712ffd640 1 -- 172.21.15.22:0/360788173 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd72c00f480 con 0x55b0db670940 2024-09-19T08:18:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.630+0000 7fd73c51d740 1 -- 172.21.15.22:0/360788173 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd72c004050 con 0x55b0db670940 2024-09-19T08:18:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.630+0000 7fd712ffd640 1 -- 172.21.15.22:0/360788173 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd72c034050 con 0x55b0db670940 2024-09-19T08:18:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.630+0000 7fd73c51d740 10 client.16110 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.630+0000 7fd73c51d740 1 -- 172.21.15.22:0/360788173 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b0db67f360 con 0x55b0db670940 2024-09-19T08:18:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.630+0000 7fd712ffd640 1 -- 172.21.15.22:0/360788173 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd72c003e40 con 0x55b0db670940 2024-09-19T08:18:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.630+0000 7fd712ffd640 1 -- 172.21.15.22:0/360788173 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fd72c0197e0 con 0x55b0db670940 2024-09-19T08:18:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.631+0000 7fd73c51d740 10 client.16110 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.631+0000 7fd73c51d740 10 client.16110 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:18:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.631+0000 7fd73c51d740 20 client.16110 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.631+0000 7fd73c51d740 1 --2- 172.21.15.22:0/360788173 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b0db680760 0x55b0db682b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.631+0000 7fd73c51d740 4 client.16110 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:18:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.631+0000 7fd73c51d740 1 -- 172.21.15.22:0/360788173 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd72c004050 con 0x55b0db680760 2024-09-19T08:18:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.631+0000 7fd7337fe640 1 --2- 172.21.15.22:0/360788173 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b0db680760 0x55b0db682b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.631+0000 7fd712ffd640 10 client.16110 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.631+0000 7fd7337fe640 1 --2- 172.21.15.22:0/360788173 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b0db680760 0x55b0db682b50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e2664095c54ed1 server_cookie=7afe8db250191381 in_seq=0 out_seq=0 2024-09-19T08:18:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.632+0000 7fd712ffd640 1 -- 172.21.15.22:0/360788173 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd72c004050 con 0x55b0db680760 2024-09-19T08:18:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.632+0000 7fd712ffd640 10 client.16110 handle_command_reply: tid=0 2024-09-19T08:18:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.632+0000 7fd73c51d740 1 client.16110 shutdown 2024-09-19T08:18:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.633+0000 7fd73c51d740 1 -- 172.21.15.22:0/360788173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 msgr2=0x55b0db66e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.633+0000 7fd73c51d740 1 --2- 172.21.15.22:0/360788173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db66e360 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd72c004740 tx=0x7fd72c004430 comp rx=0 tx=0).stop 2024-09-19T08:18:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.633+0000 7fd73c51d740 1 -- 172.21.15.22:0/360788173 shutdown_connections 2024-09-19T08:18:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.633+0000 7fd73c51d740 1 --2- 172.21.15.22:0/360788173 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b0db680760 0x55b0db682b50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.633+0000 7fd73c51d740 1 --2- 172.21.15.22:0/360788173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db66e360 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.633+0000 7fd73c51d740 1 --2- 172.21.15.22:0/360788173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 0x55b0db66de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.633+0000 7fd73c51d740 1 --2- 172.21.15.22:0/360788173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db66d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.633+0000 7fd73c51d740 1 -- 172.21.15.22:0/360788173 >> 172.21.15.22:0/360788173 conn(0x55b0db661cf0 msgr2=0x55b0db660e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.633+0000 7fd712ffd640 0 client.16110 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.633+0000 7fd73c51d740 1 -- 172.21.15.22:0/360788173 shutdown_connections 2024-09-19T08:18:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.633+0000 7fd73c51d740 1 -- 172.21.15.22:0/360788173 wait complete. 2024-09-19T08:18:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.633+0000 7fd73c51d740 20 client.16110 trim_cache size 0 max 16384 2024-09-19T08:18:31.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.654+0000 7fd73c51d740 1 Processor -- start 2024-09-19T08:18:31.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.654+0000 7fd73c51d740 1 -- start start 2024-09-19T08:18:31.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.655+0000 7fd73c51d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db66c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.655+0000 7fd73c51d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 0x55b0db6681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.655+0000 7fd73c51d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db6882c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.655+0000 7fd73c51d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341adf90 con 0x55b0db670940 2024-09-19T08:18:31.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.655+0000 7fd732ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 0x55b0db6681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.655+0000 7fd732ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 0x55b0db6681c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:50538/0 (socket says 172.21.15.22:50538) 2024-09-19T08:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.655+0000 7fd732ffd640 1 -- 172.21.15.22:0/317331876 learned_addr learned my addr 172.21.15.22:0/317331876 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.655+0000 7fd7337fe640 1 --2- 172.21.15.22:0/317331876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db66c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.655+0000 7fd733fff640 1 --2- 172.21.15.22:0/317331876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db6882c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.655+0000 7fd73c51d740 1 -- 172.21.15.22:0/317331876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341ab800 con 0x55b0db665760 2024-09-19T08:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.655+0000 7fd73c51d740 1 -- 172.21.15.22:0/317331876 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341a9650 con 0x55b0db666080 2024-09-19T08:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.656+0000 7fd733fff640 1 -- 172.21.15.22:0/317331876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 msgr2=0x55b0db6681c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.656+0000 7fd733fff640 1 --2- 172.21.15.22:0/317331876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 0x55b0db6681c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.656+0000 7fd733fff640 1 -- 172.21.15.22:0/317331876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 msgr2=0x55b0db66c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.656+0000 7fd733fff640 1 --2- 172.21.15.22:0/317331876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db66c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.656+0000 7fd733fff640 1 -- 172.21.15.22:0/317331876 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0db65d260 con 0x55b0db670940 2024-09-19T08:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.656+0000 7fd733fff640 1 --2- 172.21.15.22:0/317331876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db6882c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd72c012fd0 tx=0x7fd72c019ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce739ff25848b7db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.656+0000 7fd7197fa640 1 -- 172.21.15.22:0/317331876 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c026040 con 0x55b0db670940 2024-09-19T08:18:31.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.656+0000 7fd7197fa640 1 -- 172.21.15.22:0/317331876 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd72c0300e0 con 0x55b0db670940 2024-09-19T08:18:31.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.656+0000 7fd7197fa640 1 -- 172.21.15.22:0/317331876 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c019890 con 0x55b0db670940 2024-09-19T08:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.657+0000 7fd73c51d740 1 -- 172.21.15.22:0/317331876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 msgr2=0x55b0db6882c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.657+0000 7fd73c51d740 1 --2- 172.21.15.22:0/317331876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db6882c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd72c012fd0 tx=0x7fd72c019ea0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.658+0000 7fd73c51d740 1 -- 172.21.15.22:0/317331876 shutdown_connections 2024-09-19T08:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.658+0000 7fd73c51d740 1 --2- 172.21.15.22:0/317331876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db670940 0x55b0db6882c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.658+0000 7fd73c51d740 1 --2- 172.21.15.22:0/317331876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db666080 0x55b0db6681c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.658+0000 7fd73c51d740 1 --2- 172.21.15.22:0/317331876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db665760 0x55b0db66c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.658+0000 7fd73c51d740 1 -- 172.21.15.22:0/317331876 >> 172.21.15.22:0/317331876 conn(0x55b0db661cf0 msgr2=0x55b0db66f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.658+0000 7fd73c51d740 1 -- 172.21.15.22:0/317331876 shutdown_connections 2024-09-19T08:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.658+0000 7fd73c51d740 1 -- 172.21.15.22:0/317331876 wait complete. 2024-09-19T08:18:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.658+0000 7fd73c51d740 1 -- start start 2024-09-19T08:18:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.659+0000 7fd73c51d740 1 Processor -- start 2024-09-19T08:18:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.659+0000 7fd73c51d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db665760 0x55b0db6788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.659+0000 7fd73c51d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db666080 0x55b0db678e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db665760 0x55b0db6788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db665760 0x55b0db6788e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:50548/0 (socket says 172.21.15.22:50548) 2024-09-19T08:18:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd7337fe640 1 -- 172.21.15.22:0/1304137683 learned_addr learned my addr 172.21.15.22:0/1304137683 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd73c51d740 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db670940 0x55b0db6758b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd73c51d740 1 -- 172.21.15.22:0/1304137683 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341a9cc0 con 0x55b0db666080 2024-09-19T08:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd73c51d740 1 -- 172.21.15.22:0/1304137683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341ab130 con 0x55b0db670940 2024-09-19T08:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd73c51d740 1 -- 172.21.15.22:0/1304137683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341aa390 con 0x55b0db665760 2024-09-19T08:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd732ffd640 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db666080 0x55b0db678e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd73c51d740 10 client.0 _ll_register_callbacks cb 0x7fd72c004dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd733fff640 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db670940 0x55b0db6758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd7337fe640 1 -- 172.21.15.22:0/1304137683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db670940 msgr2=0x55b0db6758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd7337fe640 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db670940 0x55b0db6758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd7337fe640 1 -- 172.21.15.22:0/1304137683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db666080 msgr2=0x55b0db678e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd7337fe640 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db666080 0x55b0db678e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd7337fe640 1 -- 172.21.15.22:0/1304137683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0db662980 con 0x55b0db665760 2024-09-19T08:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.660+0000 7fd732ffd640 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db666080 0x55b0db678e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:18:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.661+0000 7fd7337fe640 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db665760 0x55b0db6788e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55b0db683260 tx=0x7fd7240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f855f84fc6f5f7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.661+0000 7fd712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.661+0000 7fd712ffd640 1 -- 172.21.15.22:0/1304137683 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72401d450 con 0x55b0db665760 2024-09-19T08:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.661+0000 7fd712ffd640 1 -- 172.21.15.22:0/1304137683 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd72401d5f0 con 0x55b0db665760 2024-09-19T08:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.661+0000 7fd73c51d740 1 -- 172.21.15.22:0/1304137683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd72c0300e0 con 0x55b0db665760 2024-09-19T08:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.661+0000 7fd712ffd640 1 -- 172.21.15.22:0/1304137683 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd724042050 con 0x55b0db665760 2024-09-19T08:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.661+0000 7fd73c51d740 10 client.25660 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.661+0000 7fd73c51d740 1 -- 172.21.15.22:0/1304137683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b0db6eb820 con 0x55b0db665760 2024-09-19T08:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.661+0000 7fd712ffd640 1 -- 172.21.15.22:0/1304137683 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd72c0300e0 con 0x55b0db665760 2024-09-19T08:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.662+0000 7fd712ffd640 1 -- 172.21.15.22:0/1304137683 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fd724026040 con 0x55b0db665760 2024-09-19T08:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.662+0000 7fd73c51d740 10 client.25660 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.662+0000 7fd73c51d740 10 client.25660 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.662+0000 7fd73c51d740 20 client.25660 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.662+0000 7fd73c51d740 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b0db6ec060 0x55b0db6ee480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.662+0000 7fd73c51d740 4 client.25660 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-19T08:18:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.662+0000 7fd73c51d740 1 -- 172.21.15.22:0/1304137683 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x55b0db5d89d0 con 0x55b0db6ec060 2024-09-19T08:18:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.662+0000 7fd732ffd640 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b0db6ec060 0x55b0db6ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.662+0000 7fd732ffd640 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b0db6ec060 0x55b0db6ee480 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831c2cc1510e04d1 server_cookie=124e22b9c65b8c32 in_seq=0 out_seq=0 2024-09-19T08:18:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:31.662+0000 7fd712ffd640 10 client.25660 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.280+0000 7fcff419c640 1 -- 172.21.15.22:0/3605729950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfec10e420 msgr2=0x7fcfec110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.280+0000 7fcff419c640 1 --2- 172.21.15.22:0/3605729950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfec10e420 0x7fcfec110810 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009ca0 tx=0x7fcfe402a950 comp rx=0 tx=0).stop 2024-09-19T08:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.280+0000 7fcff419c640 1 -- 172.21.15.22:0/3605729950 shutdown_connections 2024-09-19T08:18:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.280+0000 7fcff419c640 1 --2- 172.21.15.22:0/3605729950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfec10e420 0x7fcfec110810 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.280+0000 7fcff419c640 1 --2- 172.21.15.22:0/3605729950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfec074ee0 0x7fcfec10dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.280+0000 7fcff419c640 1 --2- 172.21.15.22:0/3605729950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec0745c0 0x7fcfec0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.280+0000 7fcff419c640 1 -- 172.21.15.22:0/3605729950 >> 172.21.15.22:0/3605729950 conn(0x7fcfec0fd8c0 msgr2=0x7fcfec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.280+0000 7fcff419c640 1 -- 172.21.15.22:0/3605729950 shutdown_connections 2024-09-19T08:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.280+0000 7fcff419c640 1 -- 172.21.15.22:0/3605729950 wait complete. 2024-09-19T08:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.280+0000 7fcff419c640 1 Processor -- start 2024-09-19T08:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff419c640 1 -- start start 2024-09-19T08:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff419c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfec0745c0 0x7fcfec19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff419c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec074ee0 0x7fcfec19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff419c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfec10e420 0x7fcfec19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff419c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec114170 con 0x7fcfec0745c0 2024-09-19T08:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff419c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec113ff0 con 0x7fcfec074ee0 2024-09-19T08:18:32.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff419c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1142f0 con 0x7fcfec10e420 2024-09-19T08:18:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff1f38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfec0745c0 0x7fcfec19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff1f38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfec0745c0 0x7fcfec19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57148/0 (socket says 172.21.15.22:57148) 2024-09-19T08:18:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff1f38640 1 -- 172.21.15.22:0/2328729370 learned_addr learned my addr 172.21.15.22:0/2328729370 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff2739640 1 --2- 172.21.15.22:0/2328729370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfec10e420 0x7fcfec19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff1737640 1 --2- 172.21.15.22:0/2328729370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec074ee0 0x7fcfec19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff1f38640 1 -- 172.21.15.22:0/2328729370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfec10e420 msgr2=0x7fcfec19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff1f38640 1 --2- 172.21.15.22:0/2328729370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfec10e420 0x7fcfec19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff1f38640 1 -- 172.21.15.22:0/2328729370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec074ee0 msgr2=0x7fcfec19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff1f38640 1 --2- 172.21.15.22:0/2328729370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec074ee0 0x7fcfec19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff1f38640 1 -- 172.21.15.22:0/2328729370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfec19f600 con 0x7fcfec0745c0 2024-09-19T08:18:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.281+0000 7fcff1f38640 1 --2- 172.21.15.22:0/2328729370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfec0745c0 0x7fcfec19a650 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800e9f0 tx=0x7fcfe800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.282+0000 7fcfe2ffd640 1 -- 172.21.15.22:0/2328729370 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800cd80 con 0x7fcfec0745c0 2024-09-19T08:18:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.282+0000 7fcff419c640 1 -- 172.21.15.22:0/2328729370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfec19f8f0 con 0x7fcfec0745c0 2024-09-19T08:18:32.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.282+0000 7fcfe2ffd640 1 -- 172.21.15.22:0/2328729370 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe8010910 con 0x7fcfec0745c0 2024-09-19T08:18:32.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.282+0000 7fcfe2ffd640 1 -- 172.21.15.22:0/2328729370 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe80188f0 con 0x7fcfec0745c0 2024-09-19T08:18:32.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.282+0000 7fcff419c640 1 -- 172.21.15.22:0/2328729370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfec1a71d0 con 0x7fcfec0745c0 2024-09-19T08:18:32.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.284+0000 7fcfe2ffd640 1 -- 172.21.15.22:0/2328729370 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcfe8010ac0 con 0x7fcfec0745c0 2024-09-19T08:18:32.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.285+0000 7fcfe2ffd640 1 --2- 172.21.15.22:0/2328729370 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcfd80a33b0 0x7fcfd80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.285+0000 7fcfe2ffd640 1 -- 172.21.15.22:0/2328729370 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcfe80d0630 con 0x7fcfec0745c0 2024-09-19T08:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.285+0000 7fcff6575740 1 Processor -- start 2024-09-19T08:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.285+0000 7fcff1737640 1 --2- 172.21.15.22:0/2328729370 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcfd80a33b0 0x7fcfd80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.286+0000 7fcff6575740 1 -- start start 2024-09-19T08:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.286+0000 7fcff6575740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9bb60 0x5568b9f9bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.286+0000 7fcff6575740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9c480 0x5568b9fa6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.286+0000 7fcff6575740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9fa6d40 0x5568b9fa9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.286+0000 7fcff6575740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5568b9f01280 con 0x5568b9fa6d40 2024-09-19T08:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.286+0000 7fcff6575740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5568b9ea8b10 con 0x5568b9f9bb60 2024-09-19T08:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.286+0000 7fcff6575740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5568b9efd240 con 0x5568b9f9c480 2024-09-19T08:18:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.286+0000 7fcff2739640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9fa6d40 0x5568b9fa9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.286+0000 7fcff1737640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9c480 0x5568b9fa6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.286+0000 7fcff2739640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9fa6d40 0x5568b9fa9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57164/0 (socket says 172.21.15.22:57164) 2024-09-19T08:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.286+0000 7fcff2739640 1 -- 172.21.15.22:0/2821542461 learned_addr learned my addr 172.21.15.22:0/2821542461 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.286+0000 7fcff1f38640 1 --2- 172.21.15.22:0/2821542461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9bb60 0x5568b9f9bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.286+0000 7fcff1737640 1 --2- 172.21.15.22:0/2328729370 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcfd80a33b0 0x7fcfd80a5870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0ffe50 tx=0x7fcfdc005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.287+0000 7fcff1737640 1 -- 172.21.15.22:0/2821542461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9bb60 msgr2=0x5568b9f9bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.287+0000 7fcff1737640 1 --2- 172.21.15.22:0/2821542461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9bb60 0x5568b9f9bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.287+0000 7fcff1737640 1 -- 172.21.15.22:0/2821542461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9fa6d40 msgr2=0x5568b9fa9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.287+0000 7fcff1737640 1 --2- 172.21.15.22:0/2821542461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9fa6d40 0x5568b9fa9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.287+0000 7fcff1737640 1 -- 172.21.15.22:0/2821542461 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b9eb51f0 con 0x5568b9f9c480 2024-09-19T08:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.287+0000 7fcff2739640 1 --2- 172.21.15.22:0/2821542461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9fa6d40 0x5568b9fa9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.287+0000 7fcff1f38640 1 --2- 172.21.15.22:0/2821542461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9bb60 0x5568b9f9bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:32.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.287+0000 7fcff1737640 1 --2- 172.21.15.22:0/2821542461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9c480 0x5568b9fa6800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc0117b0 tx=0x7fcfdc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6362c63e462c900 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:32.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.287+0000 7fcfd7fff640 1 -- 172.21.15.22:0/2821542461 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc01f420 con 0x5568b9f9c480 2024-09-19T08:18:32.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.287+0000 7fcfd7fff640 1 -- 172.21.15.22:0/2821542461 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfdc01faa0 con 0x5568b9f9c480 2024-09-19T08:18:32.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.287+0000 7fcfd7fff640 1 -- 172.21.15.22:0/2821542461 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc027b40 con 0x5568b9f9c480 2024-09-19T08:18:32.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.288+0000 7fcff6575740 1 -- 172.21.15.22:0/2821542461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9c480 msgr2=0x5568b9fa6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:32.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.288+0000 7fcff6575740 1 --2- 172.21.15.22:0/2821542461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9c480 0x5568b9fa6800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc0117b0 tx=0x7fcfdc014000 comp rx=0 tx=0).stop 2024-09-19T08:18:32.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.288+0000 7fcff6575740 1 -- 172.21.15.22:0/2821542461 shutdown_connections 2024-09-19T08:18:32.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.288+0000 7fcff6575740 1 --2- 172.21.15.22:0/2821542461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9fa6d40 0x5568b9fa9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.288+0000 7fcff6575740 1 --2- 172.21.15.22:0/2821542461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9c480 0x5568b9fa6800 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.288+0000 7fcff6575740 1 --2- 172.21.15.22:0/2821542461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9bb60 0x5568b9f9bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.288+0000 7fcff6575740 1 -- 172.21.15.22:0/2821542461 >> 172.21.15.22:0/2821542461 conn(0x5568b9f980f0 msgr2=0x5568b9f984d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:32.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.288+0000 7fcff6575740 1 -- 172.21.15.22:0/2821542461 shutdown_connections 2024-09-19T08:18:32.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.288+0000 7fcff6575740 1 -- 172.21.15.22:0/2821542461 wait complete. 2024-09-19T08:18:32.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.288+0000 7fcff6575740 1 -- start start 2024-09-19T08:18:32.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.289+0000 7fcff6575740 1 Processor -- start 2024-09-19T08:18:32.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.289+0000 7fcff6575740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9bb60 0x5568b9fa3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.289+0000 7fcff6575740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9f9c480 0x5568b9fa41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.289+0000 7fcff6575740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9fa6d40 0x5568b9fa4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.289+0000 7fcff6575740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5568b9fa9910 con 0x5568b9f9c480 2024-09-19T08:18:32.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.289+0000 7fcff6575740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5568b9ebbf10 con 0x5568b9fa6d40 2024-09-19T08:18:32.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.289+0000 7fcff6575740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5568b9fa9a90 con 0x5568b9f9bb60 2024-09-19T08:18:32.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.289+0000 7fcff6575740 10 client.0 _ll_register_callbacks cb 0x5568b9e84a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:32.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.289+0000 7fcff1737640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9f9c480 0x5568b9fa41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.289+0000 7fcff1f38640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9bb60 0x5568b9fa3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.289+0000 7fcff1737640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9f9c480 0x5568b9fa41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57178/0 (socket says 172.21.15.22:57178) 2024-09-19T08:18:32.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.289+0000 7fcff1737640 1 -- 172.21.15.22:0/2236893530 learned_addr learned my addr 172.21.15.22:0/2236893530 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:32.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.289+0000 7fcff1f38640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9bb60 0x5568b9fa3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:50572/0 (socket says 172.21.15.22:50572) 2024-09-19T08:18:32.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.289+0000 7fcff2739640 1 --2- 172.21.15.22:0/2236893530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9fa6d40 0x5568b9fa4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.290+0000 7fcff1737640 1 -- 172.21.15.22:0/2236893530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9bb60 msgr2=0x5568b9fa3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:32.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.290+0000 7fcff1737640 1 --2- 172.21.15.22:0/2236893530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9bb60 0x5568b9fa3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.290+0000 7fcff1737640 1 -- 172.21.15.22:0/2236893530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9fa6d40 msgr2=0x5568b9fa4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:32.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.290+0000 7fcff1737640 1 --2- 172.21.15.22:0/2236893530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9fa6d40 0x5568b9fa4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.290+0000 7fcff1737640 1 -- 172.21.15.22:0/2236893530 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b9c726a0 con 0x5568b9f9c480 2024-09-19T08:18:32.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.290+0000 7fcff1737640 1 --2- 172.21.15.22:0/2236893530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9f9c480 0x5568b9fa41d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009590 tx=0x7fcfdc011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78dcf187ffc2ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:32.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.290+0000 7fcfd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:18:32.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.290+0000 7fcfd57fa640 1 -- 172.21.15.22:0/2236893530 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc037060 con 0x5568b9f9c480 2024-09-19T08:18:32.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.290+0000 7fcfd57fa640 1 -- 172.21.15.22:0/2236893530 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcfdc01f4e0 con 0x5568b9f9c480 2024-09-19T08:18:32.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.290+0000 7fcff6575740 1 -- 172.21.15.22:0/2236893530 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfdc01faa0 con 0x5568b9f9c480 2024-09-19T08:18:32.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.290+0000 7fcfd57fa640 1 -- 172.21.15.22:0/2236893530 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcfdc01f6e0 con 0x5568b9f9c480 2024-09-19T08:18:32.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.290+0000 7fcff6575740 10 client.16128 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:32.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.290+0000 7fcff6575740 1 -- 172.21.15.22:0/2236893530 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568b9fb5590 con 0x5568b9f9c480 2024-09-19T08:18:32.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.290+0000 7fcfd57fa640 1 -- 172.21.15.22:0/2236893530 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfdc01faa0 con 0x5568b9f9c480 2024-09-19T08:18:32.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.291+0000 7fcfd57fa640 1 -- 172.21.15.22:0/2236893530 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fcfdc005450 con 0x5568b9f9c480 2024-09-19T08:18:32.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.291+0000 7fcff6575740 10 client.16128 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:32.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.291+0000 7fcff6575740 10 client.16128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:18:32.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.291+0000 7fcff6575740 20 client.16128 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:32.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.291+0000 7fcff6575740 1 --2- 172.21.15.22:0/2236893530 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5568b9fb6a50 0x5568b9fb8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.291+0000 7fcff6575740 4 client.16128 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:18:32.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.291+0000 7fcff6575740 1 -- 172.21.15.22:0/2236893530 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568b9f0e7f0 con 0x5568b9fb6a50 2024-09-19T08:18:32.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.291+0000 7fcff1f38640 1 --2- 172.21.15.22:0/2236893530 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5568b9fb6a50 0x5568b9fb8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.291+0000 7fcff1f38640 1 --2- 172.21.15.22:0/2236893530 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5568b9fb6a50 0x5568b9fb8e40 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edaca96e0ea009ba server_cookie=94b71c8b308c06ea in_seq=0 out_seq=0 2024-09-19T08:18:32.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.291+0000 7fcfd57fa640 10 client.16128 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:32.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcfd57fa640 1 -- 172.21.15.22:0/2236893530 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5568b9f0e7f0 con 0x5568b9fb6a50 2024-09-19T08:18:32.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcfd57fa640 10 client.16128 handle_command_reply: tid=0 2024-09-19T08:18:32.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcff6575740 1 client.16128 shutdown 2024-09-19T08:18:32.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcff6575740 1 -- 172.21.15.22:0/2236893530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9f9c480 msgr2=0x5568b9fa41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:32.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcff6575740 1 --2- 172.21.15.22:0/2236893530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9f9c480 0x5568b9fa41d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009590 tx=0x7fcfdc011e40 comp rx=0 tx=0).stop 2024-09-19T08:18:32.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcff6575740 1 -- 172.21.15.22:0/2236893530 shutdown_connections 2024-09-19T08:18:32.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcff6575740 1 --2- 172.21.15.22:0/2236893530 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5568b9fb6a50 0x5568b9fb8e40 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcff6575740 1 --2- 172.21.15.22:0/2236893530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9fa6d40 0x5568b9fa4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcff6575740 1 --2- 172.21.15.22:0/2236893530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9f9c480 0x5568b9fa41d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcff6575740 1 --2- 172.21.15.22:0/2236893530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9bb60 0x5568b9fa3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcfd57fa640 0 client.16128 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:32.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcff6575740 1 -- 172.21.15.22:0/2236893530 >> 172.21.15.22:0/2236893530 conn(0x5568b9f980f0 msgr2=0x5568b9f97270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:32.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcff6575740 1 -- 172.21.15.22:0/2236893530 shutdown_connections 2024-09-19T08:18:32.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcff6575740 1 -- 172.21.15.22:0/2236893530 wait complete. 2024-09-19T08:18:32.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.292+0000 7fcff6575740 20 client.16128 trim_cache size 0 max 16384 2024-09-19T08:18:32.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.317+0000 7fcff6575740 1 Processor -- start 2024-09-19T08:18:32.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.317+0000 7fcff6575740 1 -- start start 2024-09-19T08:18:32.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.317+0000 7fcff6575740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9f9bb60 0x5568b9fa4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.317+0000 7fcff6575740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9c480 0x5568b9fbe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.317+0000 7fcff6575740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9fa6d40 0x5568b9fbe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.317+0000 7fcff6575740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1b2520 con 0x5568b9f9bb60 2024-09-19T08:18:32.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.317+0000 7fcff6575740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1afd90 con 0x5568b9f9c480 2024-09-19T08:18:32.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.317+0000 7fcff6575740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1adbe0 con 0x5568b9fa6d40 2024-09-19T08:18:32.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.317+0000 7fcff1f38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9f9bb60 0x5568b9fa4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.317+0000 7fcff1f38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9f9bb60 0x5568b9fa4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57184/0 (socket says 172.21.15.22:57184) 2024-09-19T08:18:32.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.317+0000 7fcff1f38640 1 -- 172.21.15.22:0/4118880709 learned_addr learned my addr 172.21.15.22:0/4118880709 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:32.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.317+0000 7fcff1737640 1 --2- 172.21.15.22:0/4118880709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9c480 0x5568b9fbe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.318+0000 7fcff2739640 1 --2- 172.21.15.22:0/4118880709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9fa6d40 0x5568b9fbe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.318+0000 7fcff1737640 1 -- 172.21.15.22:0/4118880709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9fa6d40 msgr2=0x5568b9fbe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:32.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.318+0000 7fcff1737640 1 --2- 172.21.15.22:0/4118880709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9fa6d40 0x5568b9fbe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.318+0000 7fcff1737640 1 -- 172.21.15.22:0/4118880709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9f9bb60 msgr2=0x5568b9fa4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:32.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.318+0000 7fcff1737640 1 --2- 172.21.15.22:0/4118880709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9f9bb60 0x5568b9fa4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.318+0000 7fcff1737640 1 -- 172.21.15.22:0/4118880709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b9fb38b0 con 0x5568b9f9c480 2024-09-19T08:18:32.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.318+0000 7fcff1f38640 1 --2- 172.21.15.22:0/4118880709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9f9bb60 0x5568b9fa4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:32.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.318+0000 7fcff1737640 1 --2- 172.21.15.22:0/4118880709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9c480 0x5568b9fbe410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc004850 tx=0x7fcfdc005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6369e7c8ee3a47ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:32.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.318+0000 7fcfd77fe640 1 -- 172.21.15.22:0/4118880709 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc012cb0 con 0x5568b9f9c480 2024-09-19T08:18:32.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.318+0000 7fcfd77fe640 1 -- 172.21.15.22:0/4118880709 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfdc01f4e0 con 0x5568b9f9c480 2024-09-19T08:18:32.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.318+0000 7fcfd77fe640 1 -- 172.21.15.22:0/4118880709 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc027a80 con 0x5568b9f9c480 2024-09-19T08:18:32.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.319+0000 7fcff6575740 1 -- 172.21.15.22:0/4118880709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9c480 msgr2=0x5568b9fbe410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:32.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.319+0000 7fcff6575740 1 --2- 172.21.15.22:0/4118880709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9c480 0x5568b9fbe410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc004850 tx=0x7fcfdc005ab0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.319+0000 7fcff6575740 1 -- 172.21.15.22:0/4118880709 shutdown_connections 2024-09-19T08:18:32.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.319+0000 7fcff6575740 1 --2- 172.21.15.22:0/4118880709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9fa6d40 0x5568b9fbe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.319+0000 7fcff6575740 1 --2- 172.21.15.22:0/4118880709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9c480 0x5568b9fbe410 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.319+0000 7fcff6575740 1 --2- 172.21.15.22:0/4118880709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9f9bb60 0x5568b9fa4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.319+0000 7fcff6575740 1 -- 172.21.15.22:0/4118880709 >> 172.21.15.22:0/4118880709 conn(0x5568b9f980f0 msgr2=0x5568b9fa5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:32.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.319+0000 7fcff6575740 1 -- 172.21.15.22:0/4118880709 shutdown_connections 2024-09-19T08:18:32.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.319+0000 7fcff6575740 1 -- 172.21.15.22:0/4118880709 wait complete. 2024-09-19T08:18:32.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.320+0000 7fcff6575740 1 -- start start 2024-09-19T08:18:32.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.320+0000 7fcff6575740 1 Processor -- start 2024-09-19T08:18:32.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.320+0000 7fcff6575740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9bb60 0x5568b9faece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.320+0000 7fcff6575740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9c480 0x5568b9faf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.320+0000 7fcff6575740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9fa6d40 0x5568b9fabcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.320+0000 7fcff6575740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1ae250 con 0x5568b9fa6d40 2024-09-19T08:18:32.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.320+0000 7fcff6575740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1af6c0 con 0x5568b9f9bb60 2024-09-19T08:18:32.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.320+0000 7fcff6575740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1ae920 con 0x5568b9f9c480 2024-09-19T08:18:32.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcff6575740 10 client.0 _ll_register_callbacks cb 0x7fcfdc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:32.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcff2739640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9fa6d40 0x5568b9fabcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcff1737640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9c480 0x5568b9faf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcff2739640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9fa6d40 0x5568b9fabcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57200/0 (socket says 172.21.15.22:57200) 2024-09-19T08:18:32.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcff1f38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9bb60 0x5568b9faece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcff2739640 1 -- 172.21.15.22:0/1970562443 learned_addr learned my addr 172.21.15.22:0/1970562443 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:32.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcff2739640 1 -- 172.21.15.22:0/1970562443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9c480 msgr2=0x5568b9faf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:32.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcff2739640 1 --2- 172.21.15.22:0/1970562443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9c480 0x5568b9faf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcff2739640 1 -- 172.21.15.22:0/1970562443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9bb60 msgr2=0x5568b9faece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:32.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcff2739640 1 --2- 172.21.15.22:0/1970562443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9bb60 0x5568b9faece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcff2739640 1 -- 172.21.15.22:0/1970562443 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b9fac1f0 con 0x5568b9fa6d40 2024-09-19T08:18:32.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcff1f38640 1 --2- 172.21.15.22:0/1970562443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9bb60 0x5568b9faece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:32.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcff2739640 1 --2- 172.21.15.22:0/1970562443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9fa6d40 0x5568b9fabcb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcfec06b460 tx=0x7fcfe402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9883f95ba18ecd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:32.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcfd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:18:32.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcfd57fa640 1 -- 172.21.15.22:0/1970562443 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4034040 con 0x5568b9fa6d40 2024-09-19T08:18:32.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcff6575740 1 -- 172.21.15.22:0/1970562443 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfdc01f4e0 con 0x5568b9fa6d40 2024-09-19T08:18:32.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.321+0000 7fcfd57fa640 1 -- 172.21.15.22:0/1970562443 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcfe4003970 con 0x5568b9fa6d40 2024-09-19T08:18:32.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.322+0000 7fcfd57fa640 1 -- 172.21.15.22:0/1970562443 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcfe4055050 con 0x5568b9fa6d40 2024-09-19T08:18:32.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.322+0000 7fcff6575740 10 client.16134 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:32.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.322+0000 7fcff6575740 1 -- 172.21.15.22:0/1970562443 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568ba021b40 con 0x5568b9fa6d40 2024-09-19T08:18:32.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.322+0000 7fcfd57fa640 1 -- 172.21.15.22:0/1970562443 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfdc01f4e0 con 0x5568b9fa6d40 2024-09-19T08:18:32.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.322+0000 7fcfd57fa640 1 -- 172.21.15.22:0/1970562443 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fcfe4030460 con 0x5568b9fa6d40 2024-09-19T08:18:32.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.322+0000 7fcff6575740 10 client.16134 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:32.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.322+0000 7fcff6575740 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:18:32.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.322+0000 7fcff6575740 20 client.16134 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:32.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.322+0000 7fcff6575740 1 --2- 172.21.15.22:0/1970562443 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5568ba022340 0x5568ba024760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:32.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.322+0000 7fcff6575740 4 client.16134 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:18:32.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.322+0000 7fcff6575740 1 -- 172.21.15.22:0/1970562443 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5568b9f08d00 con 0x5568ba022340 2024-09-19T08:18:32.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.322+0000 7fcff1f38640 1 --2- 172.21.15.22:0/1970562443 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5568ba022340 0x5568ba024760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:32.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.323+0000 7fcfd57fa640 10 client.16134 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:32.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.323+0000 7fcff1f38640 1 --2- 172.21.15.22:0/1970562443 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5568ba022340 0x5568ba024760 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29b43e764a111e2f server_cookie=90ff053b696755f9 in_seq=0 out_seq=0 2024-09-19T08:18:32.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.324+0000 7fcfd57fa640 1 -- 172.21.15.22:0/1970562443 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5568b9f08d00 con 0x5568ba022340 2024-09-19T08:18:32.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.324+0000 7fcfd57fa640 10 client.16134 handle_command_reply: tid=0 2024-09-19T08:18:32.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.324+0000 7fcff6575740 1 client.16134 shutdown 2024-09-19T08:18:32.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.324+0000 7fcff6575740 1 -- 172.21.15.22:0/1970562443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9fa6d40 msgr2=0x5568b9fabcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:32.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.324+0000 7fcff6575740 1 --2- 172.21.15.22:0/1970562443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9fa6d40 0x5568b9fabcb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcfec06b460 tx=0x7fcfe402f3a0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.324+0000 7fcff6575740 1 -- 172.21.15.22:0/1970562443 shutdown_connections 2024-09-19T08:18:32.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.324+0000 7fcff6575740 1 --2- 172.21.15.22:0/1970562443 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5568ba022340 0x5568ba024760 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.324+0000 7fcff6575740 1 --2- 172.21.15.22:0/1970562443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568b9fa6d40 0x5568b9fabcb0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.324+0000 7fcff6575740 1 --2- 172.21.15.22:0/1970562443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568b9f9c480 0x5568b9faf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.324+0000 7fcfd57fa640 0 client.16134 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:32.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.324+0000 7fcff6575740 1 --2- 172.21.15.22:0/1970562443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568b9f9bb60 0x5568b9faece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.324+0000 7fcff6575740 1 -- 172.21.15.22:0/1970562443 >> 172.21.15.22:0/1970562443 conn(0x5568b9f980f0 msgr2=0x5568b9f96a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:32.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.324+0000 7fcff6575740 1 -- 172.21.15.22:0/1970562443 shutdown_connections 2024-09-19T08:18:32.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.324+0000 7fcff6575740 1 -- 172.21.15.22:0/1970562443 wait complete. 2024-09-19T08:18:32.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.325+0000 7fcff6575740 20 client.16134 trim_cache size 0 max 16384 2024-09-19T08:18:32.330 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:18:32.331 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-19T08:18:32.331 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "a63a5317-709f-48f1-88fb-72323350b6ea", 2024-09-19T08:18:32.331 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-19T08:18:32.331 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:18:32.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.325+0000 7fcff419c640 1 -- 172.21.15.22:0/2328729370 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcfd80a33b0 msgr2=0x7fcfd80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:32.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.325+0000 7fcff419c640 1 --2- 172.21.15.22:0/2328729370 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcfd80a33b0 0x7fcfd80a5870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0ffe50 tx=0x7fcfdc005f90 comp rx=0 tx=0).stop 2024-09-19T08:18:32.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.325+0000 7fcff419c640 1 -- 172.21.15.22:0/2328729370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfec0745c0 msgr2=0x7fcfec19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:32.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.325+0000 7fcff419c640 1 --2- 172.21.15.22:0/2328729370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfec0745c0 0x7fcfec19a650 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800e9f0 tx=0x7fcfe800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.325+0000 7fcff419c640 1 -- 172.21.15.22:0/2328729370 shutdown_connections 2024-09-19T08:18:32.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.325+0000 7fcff419c640 1 --2- 172.21.15.22:0/2328729370 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcfd80a33b0 0x7fcfd80a5870 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.325+0000 7fcff419c640 1 --2- 172.21.15.22:0/2328729370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfec10e420 0x7fcfec19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.325+0000 7fcff419c640 1 --2- 172.21.15.22:0/2328729370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfec074ee0 0x7fcfec19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.325+0000 7fcff419c640 1 --2- 172.21.15.22:0/2328729370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfec0745c0 0x7fcfec19a650 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:32.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.325+0000 7fcff419c640 1 -- 172.21.15.22:0/2328729370 >> 172.21.15.22:0/2328729370 conn(0x7fcfec0fd8c0 msgr2=0x7fcfec0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:32.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.325+0000 7fcff419c640 1 -- 172.21.15.22:0/2328729370 shutdown_connections 2024-09-19T08:18:32.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:32.325+0000 7fcff419c640 1 -- 172.21.15.22:0/2328729370 wait complete. 2024-09-19T08:18:33.122 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:18:33.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:33 smithi064 ceph-mon[34900]: pgmap v108: 97 pgs: 97 active+clean; 378 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.3 MiB/s wr, 409 op/s 2024-09-19T08:18:33.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:33 smithi064 ceph-mon[34900]: scrub queued for path: / 2024-09-19T08:18:33.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:33 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:33.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:33 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:18:33.442 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:18:33.502 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:33 smithi157 ceph-mon[35143]: pgmap v108: 97 pgs: 97 active+clean; 378 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.3 MiB/s wr, 409 op/s 2024-09-19T08:18:33.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:33 smithi157 ceph-mon[35143]: scrub queued for path: / 2024-09-19T08:18:33.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:33 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:33.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:33 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:18:33.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:33 smithi022 ceph-mon[30963]: pgmap v108: 97 pgs: 97 active+clean; 378 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.3 MiB/s wr, 409 op/s 2024-09-19T08:18:33.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:33 smithi022 ceph-mon[30963]: scrub queued for path: / 2024-09-19T08:18:33.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:33 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:33.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:33 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:18:33.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.928+0000 7f9417470640 1 -- 172.21.15.22:0/2023186946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9410100a60 msgr2=0x7f9410100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:33.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.928+0000 7f9417470640 1 --2- 172.21.15.22:0/2023186946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9410100a60 0x7f9410100ec0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9400009930 tx=0x7f940002a420 comp rx=0 tx=0).stop 2024-09-19T08:18:33.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.928+0000 7f9417470640 1 -- 172.21.15.22:0/2023186946 shutdown_connections 2024-09-19T08:18:33.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.928+0000 7f9417470640 1 --2- 172.21.15.22:0/2023186946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9410101400 0x7f941010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.928+0000 7f9417470640 1 --2- 172.21.15.22:0/2023186946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9410100a60 0x7f9410100ec0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.928+0000 7f9417470640 1 --2- 172.21.15.22:0/2023186946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9410106a60 0x7f9410106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.928+0000 7f9417470640 1 -- 172.21.15.22:0/2023186946 >> 172.21.15.22:0/2023186946 conn(0x7f94100fc780 msgr2=0x7f94100feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.928+0000 7f9417470640 1 -- 172.21.15.22:0/2023186946 shutdown_connections 2024-09-19T08:18:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.928+0000 7f9417470640 1 -- 172.21.15.22:0/2023186946 wait complete. 2024-09-19T08:18:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.928+0000 7f9417470640 1 Processor -- start 2024-09-19T08:18:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.928+0000 7f9417470640 1 -- start start 2024-09-19T08:18:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9417470640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9410100a60 0x7f94101973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9417470640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9410101400 0x7f9410197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9417470640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9410106a60 0x7f941019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9417470640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9410110e60 con 0x7f9410106a60 2024-09-19T08:18:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9417470640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9410110ce0 con 0x7f9410100a60 2024-09-19T08:18:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9417470640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9410110fe0 con 0x7f9410101400 2024-09-19T08:18:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9415a0d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9410106a60 0x7f941019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9414a0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9410101400 0x7f9410197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9415a0d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9410106a60 0x7f941019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57220/0 (socket says 172.21.15.22:57220) 2024-09-19T08:18:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f941520c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9410100a60 0x7f94101973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9415a0d640 1 -- 172.21.15.22:0/3087029155 learned_addr learned my addr 172.21.15.22:0/3087029155 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9415a0d640 1 -- 172.21.15.22:0/3087029155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9410101400 msgr2=0x7f9410197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9415a0d640 1 --2- 172.21.15.22:0/3087029155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9410101400 0x7f9410197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9415a0d640 1 -- 172.21.15.22:0/3087029155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9410100a60 msgr2=0x7f94101973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9415a0d640 1 --2- 172.21.15.22:0/3087029155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9410100a60 0x7f94101973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9415a0d640 1 -- 172.21.15.22:0/3087029155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941019c370 con 0x7f9410106a60 2024-09-19T08:18:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f941520c640 1 --2- 172.21.15.22:0/3087029155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9410100a60 0x7f94101973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.929+0000 7f9415a0d640 1 --2- 172.21.15.22:0/3087029155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9410106a60 0x7f941019bc90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f940800eca0 tx=0x7f940800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.930+0000 7f94067fc640 1 -- 172.21.15.22:0/3087029155 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408004b20 con 0x7f9410106a60 2024-09-19T08:18:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.930+0000 7f94067fc640 1 -- 172.21.15.22:0/3087029155 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9408004cc0 con 0x7f9410106a60 2024-09-19T08:18:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.930+0000 7f9417470640 1 -- 172.21.15.22:0/3087029155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941019c660 con 0x7f9410106a60 2024-09-19T08:18:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.930+0000 7f94067fc640 1 -- 172.21.15.22:0/3087029155 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408017650 con 0x7f9410106a60 2024-09-19T08:18:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.930+0000 7f9417470640 1 -- 172.21.15.22:0/3087029155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f941006a790 con 0x7f9410106a60 2024-09-19T08:18:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.932+0000 7f94067fc640 1 -- 172.21.15.22:0/3087029155 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9408005020 con 0x7f9410106a60 2024-09-19T08:18:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.932+0000 7f9419849740 1 Processor -- start 2024-09-19T08:18:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.932+0000 7f9419849740 1 -- start start 2024-09-19T08:18:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.932+0000 7f9419849740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 0x56218ddf8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.932+0000 7f9419849740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218ddf8dc0 0x56218de03140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.932+0000 7f9419849740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218de03680 0x56218de05a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.932+0000 7f9419849740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56218dd5e2e0 con 0x56218ddf8dc0 2024-09-19T08:18:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.932+0000 7f9419849740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56218dd05ce0 con 0x56218de03680 2024-09-19T08:18:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.932+0000 7f9419849740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56218dd5a2a0 con 0x56218ddf84a0 2024-09-19T08:18:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f941520c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218ddf8dc0 0x56218de03140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f9414a0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 0x56218ddf8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f941520c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218ddf8dc0 0x56218de03140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57230/0 (socket says 172.21.15.22:57230) 2024-09-19T08:18:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f941520c640 1 -- 172.21.15.22:0/2490702492 learned_addr learned my addr 172.21.15.22:0/2490702492 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f94067fc640 1 --2- 172.21.15.22:0/3087029155 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f93fc0a3430 0x7f93fc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f9414a0b640 1 --2- 172.21.15.22:0/2490702492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218de03680 0x56218de05a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f9415a0d640 1 --2- 172.21.15.22:0/3087029155 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f93fc0a3430 0x7f93fc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f9414a0b640 1 -- 172.21.15.22:0/2490702492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218de03680 msgr2=0x56218de05a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f9414a0b640 1 --2- 172.21.15.22:0/2490702492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218de03680 0x56218de05a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f9414a0b640 1 -- 172.21.15.22:0/2490702492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218ddf8dc0 msgr2=0x56218de03140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f9414a0b640 1 --2- 172.21.15.22:0/2490702492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218ddf8dc0 0x56218de03140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f9414a0b640 1 -- 172.21.15.22:0/2490702492 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56218dcea440 con 0x56218ddf84a0 2024-09-19T08:18:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f941520c640 1 --2- 172.21.15.22:0/2490702492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218ddf8dc0 0x56218de03140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f94067fc640 1 -- 172.21.15.22:0/3087029155 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f94080d16c0 con 0x7f9410106a60 2024-09-19T08:18:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f9414a0b640 1 --2- 172.21.15.22:0/2490702492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 0x56218ddf8880 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9400005940 tx=0x7f940002a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8245e3f6ebdd59ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f9415a0d640 1 --2- 172.21.15.22:0/3087029155 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f93fc0a3430 0x7f93fc0a58f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f940809d760 tx=0x7f94080099b0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f93fb7fe640 1 -- 172.21.15.22:0/2490702492 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400035070 con 0x56218ddf84a0 2024-09-19T08:18:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f93fb7fe640 1 -- 172.21.15.22:0/2490702492 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9400005500 con 0x56218ddf84a0 2024-09-19T08:18:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.933+0000 7f93fb7fe640 1 -- 172.21.15.22:0/2490702492 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400004cf0 con 0x56218ddf84a0 2024-09-19T08:18:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.934+0000 7f9419849740 1 -- 172.21.15.22:0/2490702492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 msgr2=0x56218ddf8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.934+0000 7f9419849740 1 --2- 172.21.15.22:0/2490702492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 0x56218ddf8880 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9400005940 tx=0x7f940002a420 comp rx=0 tx=0).stop 2024-09-19T08:18:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.934+0000 7f9419849740 1 -- 172.21.15.22:0/2490702492 shutdown_connections 2024-09-19T08:18:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.934+0000 7f9419849740 1 --2- 172.21.15.22:0/2490702492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218de03680 0x56218de05a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.934+0000 7f9419849740 1 --2- 172.21.15.22:0/2490702492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218ddf8dc0 0x56218de03140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.934+0000 7f9419849740 1 --2- 172.21.15.22:0/2490702492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 0x56218ddf8880 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.934+0000 7f9419849740 1 -- 172.21.15.22:0/2490702492 >> 172.21.15.22:0/2490702492 conn(0x56218ddf4a30 msgr2=0x56218ddf4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.934+0000 7f9419849740 1 -- 172.21.15.22:0/2490702492 shutdown_connections 2024-09-19T08:18:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.934+0000 7f9419849740 1 -- 172.21.15.22:0/2490702492 wait complete. 2024-09-19T08:18:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f9419849740 1 -- start start 2024-09-19T08:18:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f9419849740 1 Processor -- start 2024-09-19T08:18:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f9419849740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 0x56218de03050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f9419849740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218ddf8dc0 0x56218de00300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f9419849740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218de03680 0x56218de008f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f9419849740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56218de06250 con 0x56218de03680 2024-09-19T08:18:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f9419849740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56218dd19050 con 0x56218ddf8dc0 2024-09-19T08:18:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f9419849740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56218de063d0 con 0x56218ddf84a0 2024-09-19T08:18:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f9419849740 10 client.0 _ll_register_callbacks cb 0x56218dd59160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:33.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f9414a0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218de03680 0x56218de008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f941520c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218ddf8dc0 0x56218de00300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f941520c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218ddf8dc0 0x56218de00300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:35654/0 (socket says 172.21.15.22:35654) 2024-09-19T08:18:33.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f9414a0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218de03680 0x56218de008f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57246/0 (socket says 172.21.15.22:57246) 2024-09-19T08:18:33.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f941520c640 1 -- 172.21.15.22:0/38667380 learned_addr learned my addr 172.21.15.22:0/38667380 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:33.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.935+0000 7f9414a0b640 1 --2- 172.21.15.22:0/38667380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 0x56218de03050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f941520c640 1 -- 172.21.15.22:0/38667380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 msgr2=0x56218de03050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:33.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f941520c640 1 --2- 172.21.15.22:0/38667380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 0x56218de03050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f941520c640 1 -- 172.21.15.22:0/38667380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218de03680 msgr2=0x56218de008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:33.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f941520c640 1 --2- 172.21.15.22:0/38667380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218de03680 0x56218de008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f941520c640 1 -- 172.21.15.22:0/38667380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56218dacf3c0 con 0x56218ddf8dc0 2024-09-19T08:18:33.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f9414a0b640 1 --2- 172.21.15.22:0/38667380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218de03680 0x56218de008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:33.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f941520c640 1 --2- 172.21.15.22:0/38667380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218ddf8dc0 0x56218de00300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f940c012cf0 tx=0x7f940c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e252613593ab55d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f93f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:18:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f93f8ff9640 1 -- 172.21.15.22:0/38667380 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c0120d0 con 0x56218ddf8dc0 2024-09-19T08:18:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f93f8ff9640 1 -- 172.21.15.22:0/38667380 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f940c004020 con 0x56218ddf8dc0 2024-09-19T08:18:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f9419849740 1 -- 172.21.15.22:0/38667380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9400005500 con 0x56218ddf8dc0 2024-09-19T08:18:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f93f8ff9640 1 -- 172.21.15.22:0/38667380 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f940c012270 con 0x56218ddf8dc0 2024-09-19T08:18:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f9419849740 10 client.25748 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f9419849740 1 -- 172.21.15.22:0/38667380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56218de11d30 con 0x56218ddf8dc0 2024-09-19T08:18:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.936+0000 7f93f8ff9640 1 -- 172.21.15.22:0/38667380 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9400005500 con 0x56218ddf8dc0 2024-09-19T08:18:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.937+0000 7f93f8ff9640 1 -- 172.21.15.22:0/38667380 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f940c01c620 con 0x56218ddf8dc0 2024-09-19T08:18:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.937+0000 7f9419849740 10 client.25748 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.937+0000 7f9419849740 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:18:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.937+0000 7f9419849740 20 client.25748 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.937+0000 7f9419849740 1 --2- 172.21.15.22:0/38667380 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56218de13290 0x56218de15680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.937+0000 7f9419849740 4 client.25748 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:18:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.937+0000 7f9419849740 1 -- 172.21.15.22:0/38667380 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56218dd6b850 con 0x56218de13290 2024-09-19T08:18:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.937+0000 7f9414a0b640 1 --2- 172.21.15.22:0/38667380 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56218de13290 0x56218de15680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.937+0000 7f93f8ff9640 10 client.25748 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.937+0000 7f9414a0b640 1 --2- 172.21.15.22:0/38667380 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56218de13290 0x56218de15680 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a52db2aa21ac87f server_cookie=cd4a1ab6c31a7f51 in_seq=0 out_seq=0 2024-09-19T08:18:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f93f8ff9640 1 -- 172.21.15.22:0/38667380 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56218dd6b850 con 0x56218de13290 2024-09-19T08:18:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f93f8ff9640 10 client.25748 handle_command_reply: tid=0 2024-09-19T08:18:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f9419849740 1 client.25748 shutdown 2024-09-19T08:18:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f9419849740 1 -- 172.21.15.22:0/38667380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218ddf8dc0 msgr2=0x56218de00300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f9419849740 1 --2- 172.21.15.22:0/38667380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218ddf8dc0 0x56218de00300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f940c012cf0 tx=0x7f940c00f790 comp rx=0 tx=0).stop 2024-09-19T08:18:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f9419849740 1 -- 172.21.15.22:0/38667380 shutdown_connections 2024-09-19T08:18:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f9419849740 1 --2- 172.21.15.22:0/38667380 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56218de13290 0x56218de15680 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f9419849740 1 --2- 172.21.15.22:0/38667380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218de03680 0x56218de008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f9419849740 1 --2- 172.21.15.22:0/38667380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218ddf8dc0 0x56218de00300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f9419849740 1 --2- 172.21.15.22:0/38667380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 0x56218de03050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f93f8ff9640 0 client.25748 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:33.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f9419849740 1 -- 172.21.15.22:0/38667380 >> 172.21.15.22:0/38667380 conn(0x56218ddf4a30 msgr2=0x56218ddfa760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:33.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f9419849740 1 -- 172.21.15.22:0/38667380 shutdown_connections 2024-09-19T08:18:33.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f9419849740 1 -- 172.21.15.22:0/38667380 wait complete. 2024-09-19T08:18:33.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.938+0000 7f9419849740 20 client.25748 trim_cache size 0 max 16384 2024-09-19T08:18:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f9419849740 1 Processor -- start 2024-09-19T08:18:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f9419849740 1 -- start start 2024-09-19T08:18:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f9419849740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 0x56218de002d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f9419849740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218ddf8dc0 0x56218de0f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f9419849740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218de03680 0x56218de0ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f9419849740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94101adf30 con 0x56218de03680 2024-09-19T08:18:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f9419849740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f94101ab7a0 con 0x56218ddf8dc0 2024-09-19T08:18:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f9419849740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f94101a95f0 con 0x56218ddf84a0 2024-09-19T08:18:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f9414a0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218de03680 0x56218de0ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f9414a0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218de03680 0x56218de0ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57262/0 (socket says 172.21.15.22:57262) 2024-09-19T08:18:33.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f9414a0b640 1 -- 172.21.15.22:0/3365717982 learned_addr learned my addr 172.21.15.22:0/3365717982 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:33.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f941520c640 1 --2- 172.21.15.22:0/3365717982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218ddf8dc0 0x56218de0f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f9414a0b640 1 --2- 172.21.15.22:0/3365717982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 0x56218de002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f941520c640 1 -- 172.21.15.22:0/3365717982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 msgr2=0x56218de002d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:33.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f941520c640 1 --2- 172.21.15.22:0/3365717982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 0x56218de002d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f941520c640 1 -- 172.21.15.22:0/3365717982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218de03680 msgr2=0x56218de0ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:33.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f941520c640 1 --2- 172.21.15.22:0/3365717982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218de03680 0x56218de0ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.956+0000 7f941520c640 1 -- 172.21.15.22:0/3365717982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56218ddf82d0 con 0x56218ddf8dc0 2024-09-19T08:18:33.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.957+0000 7f941520c640 1 --2- 172.21.15.22:0/3365717982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218ddf8dc0 0x56218de0f4b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f940c001eb0 tx=0x7f940c009d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9416838153cfab61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:33.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.957+0000 7f93faffd640 1 -- 172.21.15.22:0/3365717982 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c029040 con 0x56218ddf8dc0 2024-09-19T08:18:33.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.957+0000 7f93faffd640 1 -- 172.21.15.22:0/3365717982 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f940c01ccb0 con 0x56218ddf8dc0 2024-09-19T08:18:33.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.957+0000 7f93faffd640 1 -- 172.21.15.22:0/3365717982 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c004430 con 0x56218ddf8dc0 2024-09-19T08:18:33.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.957+0000 7f9419849740 1 -- 172.21.15.22:0/3365717982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218ddf8dc0 msgr2=0x56218de0f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:33.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.957+0000 7f9419849740 1 --2- 172.21.15.22:0/3365717982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218ddf8dc0 0x56218de0f4b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f940c001eb0 tx=0x7f940c009d70 comp rx=0 tx=0).stop 2024-09-19T08:18:33.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 -- 172.21.15.22:0/3365717982 shutdown_connections 2024-09-19T08:18:33.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 --2- 172.21.15.22:0/3365717982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218de03680 0x56218de0ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 --2- 172.21.15.22:0/3365717982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218ddf8dc0 0x56218de0f4b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 --2- 172.21.15.22:0/3365717982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf84a0 0x56218de002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 -- 172.21.15.22:0/3365717982 >> 172.21.15.22:0/3365717982 conn(0x56218ddf4a30 msgr2=0x56218ddffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:33.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 -- 172.21.15.22:0/3365717982 shutdown_connections 2024-09-19T08:18:33.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 -- 172.21.15.22:0/3365717982 wait complete. 2024-09-19T08:18:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 -- start start 2024-09-19T08:18:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 Processor -- start 2024-09-19T08:18:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218ddf84a0 0x56218de1abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf8dc0 0x56218de1b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218de03680 0x56218de792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94101a9c60 con 0x56218ddf84a0 2024-09-19T08:18:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f94101ab0d0 con 0x56218de03680 2024-09-19T08:18:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.958+0000 7f9419849740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f94101aa330 con 0x56218ddf8dc0 2024-09-19T08:18:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f9419849740 10 client.0 _ll_register_callbacks cb 0x7f940c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f9414a0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218ddf84a0 0x56218de1abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f941520c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf8dc0 0x56218de1b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f941520c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf8dc0 0x56218de1b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:50646/0 (socket says 172.21.15.22:50646) 2024-09-19T08:18:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f941520c640 1 -- 172.21.15.22:0/1873995618 learned_addr learned my addr 172.21.15.22:0/1873995618 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f9414a0b640 1 --2- 172.21.15.22:0/1873995618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218de03680 0x56218de792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f941520c640 1 -- 172.21.15.22:0/1873995618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218de03680 msgr2=0x56218de792d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f941520c640 1 --2- 172.21.15.22:0/1873995618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218de03680 0x56218de792d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f941520c640 1 -- 172.21.15.22:0/1873995618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218ddf84a0 msgr2=0x56218de1abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f941520c640 1 --2- 172.21.15.22:0/1873995618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218ddf84a0 0x56218de1abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f941520c640 1 -- 172.21.15.22:0/1873995618 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56218de79970 con 0x56218ddf8dc0 2024-09-19T08:18:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f9414a0b640 1 --2- 172.21.15.22:0/1873995618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218ddf84a0 0x56218de1abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f941520c640 1 --2- 172.21.15.22:0/1873995618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf8dc0 0x56218de1b110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f940c01ca40 tx=0x7f940c009280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f20dcd4508301b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f93f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:18:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f93f8ff9640 1 -- 172.21.15.22:0/1873995618 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c029070 con 0x56218ddf8dc0 2024-09-19T08:18:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f9419849740 1 -- 172.21.15.22:0/1873995618 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f940c01ccb0 con 0x56218ddf8dc0 2024-09-19T08:18:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.959+0000 7f93f8ff9640 1 -- 172.21.15.22:0/1873995618 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f940c01c470 con 0x56218ddf8dc0 2024-09-19T08:18:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.960+0000 7f93f8ff9640 1 -- 172.21.15.22:0/1873995618 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f940c03a050 con 0x56218ddf8dc0 2024-09-19T08:18:33.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.960+0000 7f9419849740 10 client.25696 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:33.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.960+0000 7f9419849740 1 -- 172.21.15.22:0/1873995618 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56218de81f10 con 0x56218ddf8dc0 2024-09-19T08:18:33.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.960+0000 7f93f8ff9640 1 -- 172.21.15.22:0/1873995618 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f940c01ccb0 con 0x56218ddf8dc0 2024-09-19T08:18:33.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.960+0000 7f93f8ff9640 1 -- 172.21.15.22:0/1873995618 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f940c004430 con 0x56218ddf8dc0 2024-09-19T08:18:33.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.960+0000 7f9419849740 10 client.25696 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:33.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.960+0000 7f9419849740 10 client.25696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:18:33.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.960+0000 7f9419849740 20 client.25696 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:33.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.960+0000 7f9419849740 1 --2- 172.21.15.22:0/1873995618 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56218de825d0 0x56218de849c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:33.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.960+0000 7f9419849740 4 client.25696 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:18:33.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.960+0000 7f9419849740 1 -- 172.21.15.22:0/1873995618 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56218dd65d60 con 0x56218de825d0 2024-09-19T08:18:33.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.960+0000 7f9414a0b640 1 --2- 172.21.15.22:0/1873995618 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56218de825d0 0x56218de849c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:33.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f93f8ff9640 10 client.25696 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:33.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f9414a0b640 1 --2- 172.21.15.22:0/1873995618 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56218de825d0 0x56218de849c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e3b052c50b7a1cd server_cookie=d20036935000a909 in_seq=0 out_seq=0 2024-09-19T08:18:33.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f93f8ff9640 1 -- 172.21.15.22:0/1873995618 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56218dd65d60 con 0x56218de825d0 2024-09-19T08:18:33.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f93f8ff9640 10 client.25696 handle_command_reply: tid=0 2024-09-19T08:18:33.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f9419849740 1 client.25696 shutdown 2024-09-19T08:18:33.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f9419849740 1 -- 172.21.15.22:0/1873995618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf8dc0 msgr2=0x56218de1b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:33.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f9419849740 1 --2- 172.21.15.22:0/1873995618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf8dc0 0x56218de1b110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f940c01ca40 tx=0x7f940c009280 comp rx=0 tx=0).stop 2024-09-19T08:18:33.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f9419849740 1 -- 172.21.15.22:0/1873995618 shutdown_connections 2024-09-19T08:18:33.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f9419849740 1 --2- 172.21.15.22:0/1873995618 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56218de825d0 0x56218de849c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f9419849740 1 --2- 172.21.15.22:0/1873995618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56218de03680 0x56218de792d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f9419849740 1 --2- 172.21.15.22:0/1873995618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56218ddf8dc0 0x56218de1b110 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f9419849740 1 --2- 172.21.15.22:0/1873995618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56218ddf84a0 0x56218de1abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f93f8ff9640 0 client.25696 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:33.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f9419849740 1 -- 172.21.15.22:0/1873995618 >> 172.21.15.22:0/1873995618 conn(0x56218ddf4a30 msgr2=0x56218de02380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:33.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f9419849740 1 -- 172.21.15.22:0/1873995618 shutdown_connections 2024-09-19T08:18:33.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f9419849740 1 -- 172.21.15.22:0/1873995618 wait complete. 2024-09-19T08:18:33.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.961+0000 7f9419849740 20 client.25696 trim_cache size 0 max 16384 2024-09-19T08:18:33.969 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:18:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:18:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-19T08:18:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: "a63a5317-709f-48f1-88fb-72323350b6ea": { 2024-09-19T08:18:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-19T08:18:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "a63a5317-709f-48f1-88fb-72323350b6ea", 2024-09-19T08:18:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-09-19T08:18:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:18:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:18:33.970 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:18:33.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.962+0000 7f9417470640 1 -- 172.21.15.22:0/3087029155 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f93fc0a3430 msgr2=0x7f93fc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:33.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.962+0000 7f9417470640 1 --2- 172.21.15.22:0/3087029155 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f93fc0a3430 0x7f93fc0a58f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f940809d760 tx=0x7f94080099b0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.962+0000 7f9417470640 1 -- 172.21.15.22:0/3087029155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9410106a60 msgr2=0x7f941019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:33.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.962+0000 7f9417470640 1 --2- 172.21.15.22:0/3087029155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9410106a60 0x7f941019bc90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f940800eca0 tx=0x7f940800b690 comp rx=0 tx=0).stop 2024-09-19T08:18:33.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.962+0000 7f9417470640 1 -- 172.21.15.22:0/3087029155 shutdown_connections 2024-09-19T08:18:33.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.962+0000 7f9417470640 1 --2- 172.21.15.22:0/3087029155 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f93fc0a3430 0x7f93fc0a58f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.962+0000 7f9417470640 1 --2- 172.21.15.22:0/3087029155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9410106a60 0x7f941019bc90 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.962+0000 7f9417470640 1 --2- 172.21.15.22:0/3087029155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9410101400 0x7f9410197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.962+0000 7f9417470640 1 --2- 172.21.15.22:0/3087029155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9410100a60 0x7f94101973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:33.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.962+0000 7f9417470640 1 -- 172.21.15.22:0/3087029155 >> 172.21.15.22:0/3087029155 conn(0x7f94100fc780 msgr2=0x7f94100fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:33.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.962+0000 7f9417470640 1 -- 172.21.15.22:0/3087029155 shutdown_connections 2024-09-19T08:18:33.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:33.962+0000 7f9417470640 1 -- 172.21.15.22:0/3087029155 wait complete. 2024-09-19T08:18:34.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:34 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:18:34.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:34 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:34.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:34 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:18:34.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:34 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:34.502 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:34 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:18:34.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:34 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:34.855 INFO:tasks.cephfs.filesystem:scrub status for tag:a63a5317-709f-48f1-88fb-72323350b6ea - {'path': '/', 'tag': 'a63a5317-709f-48f1-88fb-72323350b6ea', 'options': 'recursive,force'} 2024-09-19T08:18:35.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:35 smithi022 ceph-mon[30963]: pgmap v109: 97 pgs: 97 active+clean; 387 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 483 op/s 2024-09-19T08:18:35.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:35 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:18:35.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:35 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:35.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:35 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:18:35.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:35 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:35.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:35 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:18:35.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:35 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:35.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:35 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:18:35.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:35 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:35.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:35 smithi064 ceph-mon[34900]: pgmap v109: 97 pgs: 97 active+clean; 387 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 483 op/s 2024-09-19T08:18:35.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:35 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:18:35.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:35 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:35.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:35 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:18:35.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:35 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:35.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:35 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:18:35.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:35 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:35.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:35 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:18:35.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:35 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:35.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:35 smithi157 ceph-mon[35143]: pgmap v109: 97 pgs: 97 active+clean; 387 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 483 op/s 2024-09-19T08:18:35.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:35 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:18:35.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:35 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:35.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:35 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:18:35.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:35 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:35.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:35 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:18:35.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:35 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:35.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:35 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:18:35.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:35 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:36.154 INFO:tasks.workunit.client.0.smithi022.stdout: 83 954557 369 555079 235 478069 1054 2024-09-19T08:18:37.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:37 smithi064 ceph-mon[34900]: pgmap v110: 97 pgs: 97 active+clean; 387 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 144 op/s 2024-09-19T08:18:37.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:37 smithi157 ceph-mon[35143]: pgmap v110: 97 pgs: 97 active+clean; 387 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 144 op/s 2024-09-19T08:18:37.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:37 smithi022 ceph-mon[30963]: pgmap v110: 97 pgs: 97 active+clean; 387 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 144 op/s 2024-09-19T08:18:39.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:39 smithi064 ceph-mon[34900]: pgmap v111: 97 pgs: 97 active+clean; 389 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 200 op/s 2024-09-19T08:18:39.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:39 smithi157 ceph-mon[35143]: pgmap v111: 97 pgs: 97 active+clean; 389 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 200 op/s 2024-09-19T08:18:39.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:39 smithi022 ceph-mon[30963]: pgmap v111: 97 pgs: 97 active+clean; 389 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 200 op/s 2024-09-19T08:18:41.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:41 smithi064 ceph-mon[34900]: pgmap v112: 97 pgs: 97 active+clean; 394 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 311 op/s 2024-09-19T08:18:41.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:41 smithi157 ceph-mon[35143]: pgmap v112: 97 pgs: 97 active+clean; 394 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 311 op/s 2024-09-19T08:18:41.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:41 smithi022 ceph-mon[30963]: pgmap v112: 97 pgs: 97 active+clean; 394 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 311 op/s 2024-09-19T08:18:43.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:43 smithi064 ceph-mon[34900]: pgmap v113: 97 pgs: 97 active+clean; 394 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2024-09-19T08:18:43.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:43 smithi157 ceph-mon[35143]: pgmap v113: 97 pgs: 97 active+clean; 394 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2024-09-19T08:18:43.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:43 smithi022 ceph-mon[30963]: pgmap v113: 97 pgs: 97 active+clean; 394 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2024-09-19T08:18:44.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:44 smithi157 ceph-mon[35143]: pgmap v114: 97 pgs: 97 active+clean; 412 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 325 op/s 2024-09-19T08:18:44.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:44 smithi022 ceph-mon[30963]: pgmap v114: 97 pgs: 97 active+clean; 412 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 325 op/s 2024-09-19T08:18:44.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:44 smithi064 ceph-mon[34900]: pgmap v114: 97 pgs: 97 active+clean; 412 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 325 op/s 2024-09-19T08:18:46.154 INFO:tasks.workunit.client.0.smithi022.stdout: 83 0 0 10 0 7 0 2024-09-19T08:18:46.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:46 smithi064 ceph-mon[34900]: pgmap v115: 97 pgs: 97 active+clean; 412 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-09-19T08:18:46.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:46 smithi022 ceph-mon[30963]: pgmap v115: 97 pgs: 97 active+clean; 412 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-09-19T08:18:46.752 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:46 smithi157 ceph-mon[35143]: pgmap v115: 97 pgs: 97 active+clean; 412 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-09-19T08:18:48.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.396+0000 7fd712ffd640 1 -- 172.21.15.22:0/1304137683 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b0db5d89d0 con 0x55b0db6ec060 2024-09-19T08:18:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.396+0000 7fd712ffd640 10 client.25660 handle_command_reply: tid=0 2024-09-19T08:18:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.396+0000 7fd73c51d740 1 client.25660 shutdown 2024-09-19T08:18:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.397+0000 7fd73c51d740 1 -- 172.21.15.22:0/1304137683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db665760 msgr2=0x55b0db6788e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.397+0000 7fd73c51d740 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db665760 0x55b0db6788e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55b0db683260 tx=0x7fd7240046c0 comp rx=0 tx=0).stop 2024-09-19T08:18:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.397+0000 7fd73c51d740 1 -- 172.21.15.22:0/1304137683 shutdown_connections 2024-09-19T08:18:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.397+0000 7fd73c51d740 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b0db6ec060 0x55b0db6ee480 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.397+0000 7fd73c51d740 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b0db670940 0x55b0db6758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.397+0000 7fd73c51d740 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0db666080 0x55b0db678e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.397+0000 7fd73c51d740 1 --2- 172.21.15.22:0/1304137683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b0db665760 0x55b0db6788e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.397+0000 7fd712ffd640 0 client.25660 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.397+0000 7fd73c51d740 1 -- 172.21.15.22:0/1304137683 >> 172.21.15.22:0/1304137683 conn(0x55b0db661cf0 msgr2=0x55b0db66f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.397+0000 7fd73c51d740 1 -- 172.21.15.22:0/1304137683 shutdown_connections 2024-09-19T08:18:48.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.397+0000 7fd73c51d740 1 -- 172.21.15.22:0/1304137683 wait complete. 2024-09-19T08:18:48.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.397+0000 7fd73c51d740 20 client.25660 trim_cache size 0 max 16384 2024-09-19T08:18:48.399 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:18:48.399 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 28, 2024-09-19T08:18:48.399 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:18:48.399 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 14, 2024-09-19T08:18:48.399 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:18:48.399 INFO:teuthology.orchestra.run.smithi022.stdout: "4ab59399": { 2024-09-19T08:18:48.399 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 14, 2024-09-19T08:18:48.400 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:18:48.400 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:18:48.400 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:18:48.400 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:18:48.400 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:18:48.400 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:18:48.400 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 136.6, 2024-09-19T08:18:48.400 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:18:48.400 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:18:48.400 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:18:48.400 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:18:48.400 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:18:48.400 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:18:48.401 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:18:48.401 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:18:48.401 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:18:48.401 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:18:48.401 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:18:48.401 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:18:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.398+0000 7fd73a144640 1 -- 172.21.15.22:0/347769800 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd71c0a33b0 msgr2=0x7fd71c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.398+0000 7fd73a144640 1 --2- 172.21.15.22:0/347769800 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd71c0a33b0 0x7fd71c0a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55b0db662b50 tx=0x7fd724012040 comp rx=0 tx=0).stop 2024-09-19T08:18:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.398+0000 7fd73a144640 1 -- 172.21.15.22:0/347769800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7341027a0 msgr2=0x7fd734198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.398+0000 7fd73a144640 1 --2- 172.21.15.22:0/347769800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7341027a0 0x7fd734198a70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd728004770 tx=0x7fd728003c10 comp rx=0 tx=0).stop 2024-09-19T08:18:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.398+0000 7fd73a144640 1 -- 172.21.15.22:0/347769800 shutdown_connections 2024-09-19T08:18:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.398+0000 7fd73a144640 1 --2- 172.21.15.22:0/347769800 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd71c0a33b0 0x7fd71c0a5870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.398+0000 7fd73a144640 1 --2- 172.21.15.22:0/347769800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd734106f20 0x7fd73419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.398+0000 7fd73a144640 1 --2- 172.21.15.22:0/347769800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7341027a0 0x7fd734198a70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.398+0000 7fd73a144640 1 --2- 172.21.15.22:0/347769800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd734101df0 0x7fd734198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.398+0000 7fd73a144640 1 -- 172.21.15.22:0/347769800 >> 172.21.15.22:0/347769800 conn(0x7fd7340fd8f0 msgr2=0x7fd7340feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.398+0000 7fd73a144640 1 -- 172.21.15.22:0/347769800 shutdown_connections 2024-09-19T08:18:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:48.398+0000 7fd73a144640 1 -- 172.21.15.22:0/347769800 wait complete. 2024-09-19T08:18:48.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:48 smithi064 ceph-mon[34900]: pgmap v116: 97 pgs: 97 active+clean; 413 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 259 op/s 2024-09-19T08:18:48.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:48 smithi157 ceph-mon[35143]: pgmap v116: 97 pgs: 97 active+clean; 413 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 259 op/s 2024-09-19T08:18:48.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:48 smithi022 ceph-mon[30963]: pgmap v116: 97 pgs: 97 active+clean; 413 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 259 op/s 2024-09-19T08:18:49.237 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4ab59399', quiesce took 23.3 seconds. Will release after: -6.699999999999999 2024-09-19T08:18:49.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:18:49.557 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:18:50.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.062+0000 7f3dab60b640 1 -- 172.21.15.22:0/2167612425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3da4102570 msgr2=0x7f3da41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:50.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.062+0000 7f3dab60b640 1 --2- 172.21.15.22:0/2167612425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3da4102570 0x7f3da41029b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009890 tx=0x7f3d9402a410 comp rx=0 tx=0).stop 2024-09-19T08:18:50.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.062+0000 7f3dab60b640 1 -- 172.21.15.22:0/2167612425 shutdown_connections 2024-09-19T08:18:50.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.062+0000 7f3dab60b640 1 --2- 172.21.15.22:0/2167612425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3da4102ef0 0x7f3da410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:50.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.062+0000 7f3dab60b640 1 --2- 172.21.15.22:0/2167612425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3da4102570 0x7f3da41029b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:50.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.062+0000 7f3dab60b640 1 --2- 172.21.15.22:0/2167612425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da4100e20 0x7f3da4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.062+0000 7f3dab60b640 1 -- 172.21.15.22:0/2167612425 >> 172.21.15.22:0/2167612425 conn(0x7f3da40fc7d0 msgr2=0x7f3da40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.062+0000 7f3dab60b640 1 -- 172.21.15.22:0/2167612425 shutdown_connections 2024-09-19T08:18:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.062+0000 7f3dab60b640 1 -- 172.21.15.22:0/2167612425 wait complete. 2024-09-19T08:18:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.063+0000 7f3dab60b640 1 Processor -- start 2024-09-19T08:18:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.063+0000 7f3dab60b640 1 -- start start 2024-09-19T08:18:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.063+0000 7f3dab60b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3da4100e20 0x7f3da4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.063+0000 7f3dab60b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3da4102570 0x7f3da4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.063+0000 7f3dab60b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da4102ef0 0x7f3da419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.063+0000 7f3dab60b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3da4110f80 con 0x7f3da4102ef0 2024-09-19T08:18:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.063+0000 7f3dab60b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3da4110e00 con 0x7f3da4100e20 2024-09-19T08:18:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.063+0000 7f3dab60b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3da4111100 con 0x7f3da4102570 2024-09-19T08:18:50.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.063+0000 7f3da9ba8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da4102ef0 0x7f3da419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:50.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.063+0000 7f3da9ba8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da4102ef0 0x7f3da419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60068/0 (socket says 172.21.15.22:60068) 2024-09-19T08:18:50.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.063+0000 7f3da9ba8640 1 -- 172.21.15.22:0/2641467329 learned_addr learned my addr 172.21.15.22:0/2641467329 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:50.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.063+0000 7f3da93a7640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3da4100e20 0x7f3da4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:50.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.063+0000 7f3da8ba6640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3da4102570 0x7f3da4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:50.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.064+0000 7f3da93a7640 1 -- 172.21.15.22:0/2641467329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3da4102570 msgr2=0x7f3da4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:50.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.064+0000 7f3da93a7640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3da4102570 0x7f3da4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:50.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.064+0000 7f3da93a7640 1 -- 172.21.15.22:0/2641467329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da4102ef0 msgr2=0x7f3da419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:50.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.064+0000 7f3da93a7640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da4102ef0 0x7f3da419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:50.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.064+0000 7f3da93a7640 1 -- 172.21.15.22:0/2641467329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da419d4b0 con 0x7f3da4100e20 2024-09-19T08:18:50.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.064+0000 7f3da9ba8640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da4102ef0 0x7f3da419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:50.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.064+0000 7f3da93a7640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3da4100e20 0x7f3da4198500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009fd0 tx=0x7f3da000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:50.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.064+0000 7f3d9a7fc640 1 -- 172.21.15.22:0/2641467329 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0003970 con 0x7f3da4100e20 2024-09-19T08:18:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.064+0000 7f3dab60b640 1 -- 172.21.15.22:0/2641467329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da419d7a0 con 0x7f3da4100e20 2024-09-19T08:18:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.064+0000 7f3d9a7fc640 1 -- 172.21.15.22:0/2641467329 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3da0004020 con 0x7f3da4100e20 2024-09-19T08:18:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.064+0000 7f3d9a7fc640 1 -- 172.21.15.22:0/2641467329 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da000ba70 con 0x7f3da4100e20 2024-09-19T08:18:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.064+0000 7f3dab60b640 1 -- 172.21.15.22:0/2641467329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da41a5080 con 0x7f3da4100e20 2024-09-19T08:18:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.065+0000 7f3dab60b640 1 -- 172.21.15.22:0/2641467329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da4101200 con 0x7f3da4100e20 2024-09-19T08:18:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.067+0000 7f3d9a7fc640 1 -- 172.21.15.22:0/2641467329 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3da00041d0 con 0x7f3da4100e20 2024-09-19T08:18:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.067+0000 7f3d9a7fc640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3d900a33b0 0x7f3d900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.068+0000 7f3da8ba6640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3d900a33b0 0x7f3d900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.068+0000 7f3d9a7fc640 1 -- 172.21.15.22:0/2641467329 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3da00cfe60 con 0x7f3da4100e20 2024-09-19T08:18:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.068+0000 7f3da8ba6640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3d900a33b0 0x7f3d900a5870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3d940095f0 tx=0x7f3d94005dc0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.072+0000 7f3d9a7fc640 1 -- 172.21.15.22:0/2641467329 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3da009c850 con 0x7f3da4100e20 2024-09-19T08:18:50.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.299+0000 7f3dab60b640 1 -- 172.21.15.22:0/2641467329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3da410a0a0 con 0x7f3da4100e20 2024-09-19T08:18:50.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.300+0000 7f3d9a7fc640 1 -- 172.21.15.22:0/2641467329 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11910 (secure 0 0 0) 0x7f3da009c150 con 0x7f3da4100e20 2024-09-19T08:18:50.301 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:18:50.302 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"btime":"2024-09-19T08:18:06:960387+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:18:06.784015+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:18:50.303 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 28 2024-09-19T08:18:50.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.302+0000 7f3dab60b640 1 -- 172.21.15.22:0/2641467329 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3d900a33b0 msgr2=0x7f3d900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:50.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.302+0000 7f3dab60b640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3d900a33b0 0x7f3d900a5870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3d940095f0 tx=0x7f3d94005dc0 comp rx=0 tx=0).stop 2024-09-19T08:18:50.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.302+0000 7f3dab60b640 1 -- 172.21.15.22:0/2641467329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3da4100e20 msgr2=0x7f3da4198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:50.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.302+0000 7f3dab60b640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3da4100e20 0x7f3da4198500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009fd0 tx=0x7f3da000c880 comp rx=0 tx=0).stop 2024-09-19T08:18:50.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.302+0000 7f3dab60b640 1 -- 172.21.15.22:0/2641467329 shutdown_connections 2024-09-19T08:18:50.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.302+0000 7f3dab60b640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3d900a33b0 0x7f3d900a5870 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.302+0000 7f3dab60b640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da4102ef0 0x7f3da419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.302+0000 7f3dab60b640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3da4102570 0x7f3da4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.302+0000 7f3dab60b640 1 --2- 172.21.15.22:0/2641467329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3da4100e20 0x7f3da4198500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.302+0000 7f3dab60b640 1 -- 172.21.15.22:0/2641467329 >> 172.21.15.22:0/2641467329 conn(0x7f3da40fc7d0 msgr2=0x7f3da406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.302+0000 7f3dab60b640 1 -- 172.21.15.22:0/2641467329 shutdown_connections 2024-09-19T08:18:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:50.302+0000 7f3dab60b640 1 -- 172.21.15.22:0/2641467329 wait complete. 2024-09-19T08:18:50.554 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:50 smithi157 ceph-mon[35143]: pgmap v117: 97 pgs: 97 active+clean; 431 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 418 op/s 2024-09-19T08:18:50.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:50 smithi064 ceph-mon[34900]: pgmap v117: 97 pgs: 97 active+clean; 431 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 418 op/s 2024-09-19T08:18:50.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:50 smithi022 ceph-mon[30963]: pgmap v117: 97 pgs: 97 active+clean; 431 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 418 op/s 2024-09-19T08:18:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:51 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2641467329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:18:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:51 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:51.515 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id 4ab59399 --release --await' 2024-09-19T08:18:51.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id 4ab59399 --release --await 2024-09-19T08:18:51.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:51 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2641467329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:18:51.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:51 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:51.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:51 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2641467329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:18:51.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:51 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:18:51.792 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:18:52.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.294+0000 7f265bca2640 1 -- 172.21.15.22:0/2474462280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2654106f40 msgr2=0x7f265410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:52.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.294+0000 7f265bca2640 1 --2- 172.21.15.22:0/2474462280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2654106f40 0x7f265410eac0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f264c009ca0 tx=0x7f264c02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.294+0000 7f265bca2640 1 -- 172.21.15.22:0/2474462280 shutdown_connections 2024-09-19T08:18:52.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.294+0000 7f265bca2640 1 --2- 172.21.15.22:0/2474462280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2654106f40 0x7f265410eac0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.294+0000 7f265bca2640 1 --2- 172.21.15.22:0/2474462280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26541027c0 0x7f2654106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.294+0000 7f265bca2640 1 --2- 172.21.15.22:0/2474462280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2654101e10 0x7f26541021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.294+0000 7f265bca2640 1 -- 172.21.15.22:0/2474462280 >> 172.21.15.22:0/2474462280 conn(0x7f26540fd8d0 msgr2=0x7f26540ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.294+0000 7f265bca2640 1 -- 172.21.15.22:0/2474462280 shutdown_connections 2024-09-19T08:18:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.294+0000 7f265bca2640 1 -- 172.21.15.22:0/2474462280 wait complete. 2024-09-19T08:18:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.295+0000 7f265bca2640 1 Processor -- start 2024-09-19T08:18:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.295+0000 7f265bca2640 1 -- start start 2024-09-19T08:18:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.295+0000 7f265bca2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2654101e10 0x7f26540744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.295+0000 7f265bca2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26541027c0 0x7f2654074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.295+0000 7f265bca2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2654106f40 0x7f2654074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.295+0000 7f265bca2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2654111ff0 con 0x7f26541027c0 2024-09-19T08:18:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.295+0000 7f265bca2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2654111e70 con 0x7f2654101e10 2024-09-19T08:18:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.295+0000 7f265bca2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2654112170 con 0x7f2654106f40 2024-09-19T08:18:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.295+0000 7f265923d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26541027c0 0x7f2654074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.295+0000 7f265923d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26541027c0 0x7f2654074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60098/0 (socket says 172.21.15.22:60098) 2024-09-19T08:18:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.295+0000 7f265923d640 1 -- 172.21.15.22:0/2589292070 learned_addr learned my addr 172.21.15.22:0/2589292070 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.295+0000 7f2659a3e640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2654101e10 0x7f26540744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.295+0000 7f265a23f640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2654106f40 0x7f2654074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.296+0000 7f265923d640 1 -- 172.21.15.22:0/2589292070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2654106f40 msgr2=0x7f2654074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.296+0000 7f265923d640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2654106f40 0x7f2654074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.296+0000 7f265923d640 1 -- 172.21.15.22:0/2589292070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2654101e10 msgr2=0x7f26540744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.296+0000 7f265923d640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2654101e10 0x7f26540744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.296+0000 7f265923d640 1 -- 172.21.15.22:0/2589292070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26541a68d0 con 0x7f26541027c0 2024-09-19T08:18:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.296+0000 7f2659a3e640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2654101e10 0x7f26540744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.296+0000 7f265923d640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26541027c0 0x7f2654074a30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f264400e9f0 tx=0x7f264400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.296+0000 7f264affd640 1 -- 172.21.15.22:0/2589292070 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264400cd80 con 0x7f26541027c0 2024-09-19T08:18:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.296+0000 7f264affd640 1 -- 172.21.15.22:0/2589292070 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2644010910 con 0x7f26541027c0 2024-09-19T08:18:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.296+0000 7f265bca2640 1 -- 172.21.15.22:0/2589292070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26541a6bc0 con 0x7f26541027c0 2024-09-19T08:18:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.296+0000 7f264affd640 1 -- 172.21.15.22:0/2589292070 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644018990 con 0x7f26541027c0 2024-09-19T08:18:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.296+0000 7f265bca2640 1 -- 172.21.15.22:0/2589292070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26541a7100 con 0x7f26541027c0 2024-09-19T08:18:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.299+0000 7f264affd640 1 -- 172.21.15.22:0/2589292070 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2644010ac0 con 0x7f26541027c0 2024-09-19T08:18:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.299+0000 7f265e07b740 1 Processor -- start 2024-09-19T08:18:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265e07b740 1 -- start start 2024-09-19T08:18:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f264affd640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f26400a33b0 0x7f26400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265e07b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4060 0x55e49d9f4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265e07b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4980 0x55e49d9fed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265e07b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 0x55e49da01630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265e07b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e49d959620 con 0x55e49d9ff240 2024-09-19T08:18:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265e07b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e49d900e50 con 0x55e49d9f4980 2024-09-19T08:18:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265e07b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e49d9555e0 con 0x55e49d9f4060 2024-09-19T08:18:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265a23f640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f26400a33b0 0x7f26400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f264affd640 1 -- 172.21.15.22:0/2589292070 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f26440cffa0 con 0x7f26541027c0 2024-09-19T08:18:52.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f2659a3e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4060 0x55e49d9f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f2659a3e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4060 0x55e49d9f4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:58580/0 (socket says 172.21.15.22:58580) 2024-09-19T08:18:52.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265923d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4980 0x55e49d9fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f2659a3e640 1 -- 172.21.15.22:0/1366964788 learned_addr learned my addr 172.21.15.22:0/1366964788 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:52.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265a23f640 1 --2- 172.21.15.22:0/1366964788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 0x55e49da01630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265923d640 1 -- 172.21.15.22:0/1366964788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4060 msgr2=0x55e49d9f4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:52.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265923d640 1 --2- 172.21.15.22:0/1366964788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4060 0x55e49d9f4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265923d640 1 -- 172.21.15.22:0/1366964788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 msgr2=0x55e49da01630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:52.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265923d640 1 --2- 172.21.15.22:0/1366964788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 0x55e49da01630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.300+0000 7f265923d640 1 -- 172.21.15.22:0/1366964788 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e49d8f1d40 con 0x55e49d9f4980 2024-09-19T08:18:52.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.301+0000 7f265923d640 1 --2- 172.21.15.22:0/1366964788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4980 0x55e49d9fed00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f26440cd700 tx=0x7f2644098710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84d68ac42f08bc75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:52.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.301+0000 7f265a23f640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f26400a33b0 0x7f26400a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f264c006010 tx=0x7f264c009360 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.301+0000 7f263ffff640 1 -- 172.21.15.22:0/1366964788 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264409a640 con 0x55e49d9f4980 2024-09-19T08:18:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.301+0000 7f263ffff640 1 -- 172.21.15.22:0/1366964788 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26440cde50 con 0x55e49d9f4980 2024-09-19T08:18:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.301+0000 7f263ffff640 1 -- 172.21.15.22:0/1366964788 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264404a4a0 con 0x55e49d9f4980 2024-09-19T08:18:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.302+0000 7f265e07b740 1 -- 172.21.15.22:0/1366964788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4980 msgr2=0x55e49d9fed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.302+0000 7f265e07b740 1 --2- 172.21.15.22:0/1366964788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4980 0x55e49d9fed00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f26440cd700 tx=0x7f2644098710 comp rx=0 tx=0).stop 2024-09-19T08:18:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.302+0000 7f265e07b740 1 -- 172.21.15.22:0/1366964788 shutdown_connections 2024-09-19T08:18:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.302+0000 7f265e07b740 1 --2- 172.21.15.22:0/1366964788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 0x55e49da01630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.302+0000 7f265e07b740 1 --2- 172.21.15.22:0/1366964788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4980 0x55e49d9fed00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.302+0000 7f265e07b740 1 --2- 172.21.15.22:0/1366964788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4060 0x55e49d9f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.302+0000 7f265e07b740 1 -- 172.21.15.22:0/1366964788 >> 172.21.15.22:0/1366964788 conn(0x55e49d9f05f0 msgr2=0x55e49d9f09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.302+0000 7f265e07b740 1 -- 172.21.15.22:0/1366964788 shutdown_connections 2024-09-19T08:18:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.302+0000 7f265e07b740 1 -- 172.21.15.22:0/1366964788 wait complete. 2024-09-19T08:18:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.303+0000 7f265e07b740 1 -- start start 2024-09-19T08:18:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.303+0000 7f265e07b740 1 Processor -- start 2024-09-19T08:18:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.303+0000 7f265e07b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4060 0x55e49d9fc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.303+0000 7f265e07b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4980 0x55e49d9fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.303+0000 7f265e07b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 0x55e49d9fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.303+0000 7f265e07b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e49da01e10 con 0x55e49d9ff240 2024-09-19T08:18:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.303+0000 7f265e07b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e49d8e1f10 con 0x55e49d9f4060 2024-09-19T08:18:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.303+0000 7f265e07b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e49da01f90 con 0x55e49d9f4980 2024-09-19T08:18:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.303+0000 7f265e07b740 10 client.0 _ll_register_callbacks cb 0x55e49d905db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.303+0000 7f265923d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4980 0x55e49d9fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.303+0000 7f265a23f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 0x55e49d9fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.303+0000 7f265923d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4980 0x55e49d9fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:58584/0 (socket says 172.21.15.22:58584) 2024-09-19T08:18:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.303+0000 7f2659a3e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4060 0x55e49d9fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.303+0000 7f265923d640 1 -- 172.21.15.22:0/1143263081 learned_addr learned my addr 172.21.15.22:0/1143263081 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f265923d640 1 -- 172.21.15.22:0/1143263081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4060 msgr2=0x55e49d9fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f265923d640 1 --2- 172.21.15.22:0/1143263081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4060 0x55e49d9fc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f265923d640 1 -- 172.21.15.22:0/1143263081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 msgr2=0x55e49d9fcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f265923d640 1 --2- 172.21.15.22:0/1143263081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 0x55e49d9fcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f265923d640 1 -- 172.21.15.22:0/1143263081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e49d668580 con 0x55e49d9f4980 2024-09-19T08:18:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f2659a3e640 1 --2- 172.21.15.22:0/1143263081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4060 0x55e49d9fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f265923d640 1 --2- 172.21.15.22:0/1143263081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4980 0x55e49d9fc680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f26440cd7b0 tx=0x7f264404a820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f2a885ee614bc24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f263d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:18:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f263d7fa640 1 -- 172.21.15.22:0/1143263081 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26440cdac0 con 0x55e49d9f4980 2024-09-19T08:18:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f265e07b740 1 -- 172.21.15.22:0/1143263081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26440cde50 con 0x55e49d9f4980 2024-09-19T08:18:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f263d7fa640 1 -- 172.21.15.22:0/1143263081 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f26440bc0e0 con 0x55e49d9f4980 2024-09-19T08:18:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f263d7fa640 1 -- 172.21.15.22:0/1143263081 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f26440bc300 con 0x55e49d9f4980 2024-09-19T08:18:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f265e07b740 10 client.25714 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f265e07b740 1 -- 172.21.15.22:0/1143263081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e49d732960 con 0x55e49d9f4980 2024-09-19T08:18:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.304+0000 7f263d7fa640 1 -- 172.21.15.22:0/1143263081 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26440cde50 con 0x55e49d9f4980 2024-09-19T08:18:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.305+0000 7f263d7fa640 1 -- 172.21.15.22:0/1143263081 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f2644018b30 con 0x55e49d9f4980 2024-09-19T08:18:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.305+0000 7f265e07b740 10 client.25714 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.305+0000 7f265e07b740 10 client.25714 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:18:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.305+0000 7f265e07b740 20 client.25714 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.305+0000 7f265e07b740 1 --2- 172.21.15.22:0/1143263081 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e49da0ec50 0x55e49da11040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.305+0000 7f265e07b740 4 client.25714 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:18:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.305+0000 7f265e07b740 1 -- 172.21.15.22:0/1143263081 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e49d966b90 con 0x55e49da0ec50 2024-09-19T08:18:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.305+0000 7f2659a3e640 1 --2- 172.21.15.22:0/1143263081 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e49da0ec50 0x55e49da11040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.305+0000 7f2659a3e640 1 --2- 172.21.15.22:0/1143263081 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e49da0ec50 0x55e49da11040 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7706dc776acea361 server_cookie=3c8dbe1909407fa3 in_seq=0 out_seq=0 2024-09-19T08:18:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.305+0000 7f263d7fa640 10 client.25714 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.306+0000 7f263d7fa640 1 -- 172.21.15.22:0/1143263081 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e49d966b90 con 0x55e49da0ec50 2024-09-19T08:18:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.306+0000 7f263d7fa640 10 client.25714 handle_command_reply: tid=0 2024-09-19T08:18:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.306+0000 7f265e07b740 1 client.25714 shutdown 2024-09-19T08:18:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.306+0000 7f265e07b740 1 -- 172.21.15.22:0/1143263081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4980 msgr2=0x55e49d9fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.306+0000 7f265e07b740 1 --2- 172.21.15.22:0/1143263081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4980 0x55e49d9fc680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f26440cd7b0 tx=0x7f264404a820 comp rx=0 tx=0).stop 2024-09-19T08:18:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.306+0000 7f265e07b740 1 -- 172.21.15.22:0/1143263081 shutdown_connections 2024-09-19T08:18:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.306+0000 7f265e07b740 1 --2- 172.21.15.22:0/1143263081 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e49da0ec50 0x55e49da11040 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.306+0000 7f265e07b740 1 --2- 172.21.15.22:0/1143263081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 0x55e49d9fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.306+0000 7f265e07b740 1 --2- 172.21.15.22:0/1143263081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4980 0x55e49d9fc680 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.306+0000 7f263d7fa640 0 client.25714 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:52.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.306+0000 7f265e07b740 1 --2- 172.21.15.22:0/1143263081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4060 0x55e49d9fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.306+0000 7f265e07b740 1 -- 172.21.15.22:0/1143263081 >> 172.21.15.22:0/1143263081 conn(0x55e49d9f05f0 msgr2=0x55e49d9f6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:52.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.306+0000 7f265e07b740 1 -- 172.21.15.22:0/1143263081 shutdown_connections 2024-09-19T08:18:52.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.306+0000 7f265e07b740 1 -- 172.21.15.22:0/1143263081 wait complete. 2024-09-19T08:18:52.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.307+0000 7f265e07b740 20 client.25714 trim_cache size 0 max 16384 2024-09-19T08:18:52.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f265e07b740 1 Processor -- start 2024-09-19T08:18:52.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f265e07b740 1 -- start start 2024-09-19T08:18:52.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f265e07b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4060 0x55e49d9f7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f265e07b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9f4980 0x55e49da0a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f265e07b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9ff240 0x55e49d9f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f265e07b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f26541ae050 con 0x55e49d9f4980 2024-09-19T08:18:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f265e07b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26541ab8c0 con 0x55e49d9f4060 2024-09-19T08:18:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f265e07b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26541a9750 con 0x55e49d9ff240 2024-09-19T08:18:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f2659a3e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4060 0x55e49d9f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f265923d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9f4980 0x55e49da0a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f2659a3e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4060 0x55e49d9f7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:43918/0 (socket says 172.21.15.22:43918) 2024-09-19T08:18:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f2659a3e640 1 -- 172.21.15.22:0/871610797 learned_addr learned my addr 172.21.15.22:0/871610797 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f265a23f640 1 --2- 172.21.15.22:0/871610797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9ff240 0x55e49d9f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f2659a3e640 1 -- 172.21.15.22:0/871610797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9ff240 msgr2=0x55e49d9f93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f2659a3e640 1 --2- 172.21.15.22:0/871610797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9ff240 0x55e49d9f93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f2659a3e640 1 -- 172.21.15.22:0/871610797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9f4980 msgr2=0x55e49da0a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f2659a3e640 1 --2- 172.21.15.22:0/871610797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9f4980 0x55e49da0a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f2659a3e640 1 -- 172.21.15.22:0/871610797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e49d9fe080 con 0x55e49d9f4060 2024-09-19T08:18:52.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.326+0000 7f265923d640 1 --2- 172.21.15.22:0/871610797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9f4980 0x55e49da0a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.327+0000 7f2659a3e640 1 --2- 172.21.15.22:0/871610797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4060 0x55e49d9f7480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f26500129d0 tx=0x7f2650005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc7ea2b909e32723 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.327+0000 7f263f7fe640 1 -- 172.21.15.22:0/871610797 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265000fc80 con 0x55e49d9f4060 2024-09-19T08:18:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.327+0000 7f263f7fe640 1 -- 172.21.15.22:0/871610797 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f265000fe20 con 0x55e49d9f4060 2024-09-19T08:18:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.327+0000 7f263f7fe640 1 -- 172.21.15.22:0/871610797 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650010430 con 0x55e49d9f4060 2024-09-19T08:18:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.328+0000 7f265e07b740 1 -- 172.21.15.22:0/871610797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4060 msgr2=0x55e49d9f7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.328+0000 7f265e07b740 1 --2- 172.21.15.22:0/871610797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4060 0x55e49d9f7480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f26500129d0 tx=0x7f2650005f90 comp rx=0 tx=0).stop 2024-09-19T08:18:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.328+0000 7f265e07b740 1 -- 172.21.15.22:0/871610797 shutdown_connections 2024-09-19T08:18:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.328+0000 7f265e07b740 1 --2- 172.21.15.22:0/871610797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9ff240 0x55e49d9f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.328+0000 7f265e07b740 1 --2- 172.21.15.22:0/871610797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9f4980 0x55e49da0a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.328+0000 7f265e07b740 1 --2- 172.21.15.22:0/871610797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4060 0x55e49d9f7480 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.328+0000 7f265e07b740 1 -- 172.21.15.22:0/871610797 >> 172.21.15.22:0/871610797 conn(0x55e49d9f05f0 msgr2=0x55e49da0a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.328+0000 7f265e07b740 1 -- 172.21.15.22:0/871610797 shutdown_connections 2024-09-19T08:18:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.328+0000 7f265e07b740 1 -- 172.21.15.22:0/871610797 wait complete. 2024-09-19T08:18:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.328+0000 7f265e07b740 1 -- start start 2024-09-19T08:18:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.328+0000 7f265e07b740 1 Processor -- start 2024-09-19T08:18:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.329+0000 7f265e07b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4060 0x55e49da12110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.329+0000 7f265e07b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4980 0x55e49da12650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.329+0000 7f265e07b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 0x55e49da78590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.329+0000 7f265e07b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f26541a9da0 con 0x55e49d9ff240 2024-09-19T08:18:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.329+0000 7f265e07b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26541ab1f0 con 0x55e49d9f4980 2024-09-19T08:18:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.329+0000 7f265e07b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26541aa450 con 0x55e49d9f4060 2024-09-19T08:18:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.329+0000 7f265e07b740 10 client.0 _ll_register_callbacks cb 0x7f264400c7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:18:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.329+0000 7f2659a3e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4060 0x55e49da12110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.329+0000 7f265a23f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 0x55e49da78590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.329+0000 7f265923d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4980 0x55e49da12650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.329+0000 7f2659a3e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4060 0x55e49da12110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:58604/0 (socket says 172.21.15.22:58604) 2024-09-19T08:18:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.329+0000 7f2659a3e640 1 -- 172.21.15.22:0/682530635 learned_addr learned my addr 172.21.15.22:0/682530635 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:18:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.329+0000 7f2659a3e640 1 -- 172.21.15.22:0/682530635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4980 msgr2=0x55e49da12650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.329+0000 7f2659a3e640 1 --2- 172.21.15.22:0/682530635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4980 0x55e49da12650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.330+0000 7f2659a3e640 1 -- 172.21.15.22:0/682530635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 msgr2=0x55e49da78590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:18:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.330+0000 7f2659a3e640 1 --2- 172.21.15.22:0/682530635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 0x55e49da78590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.330+0000 7f2659a3e640 1 -- 172.21.15.22:0/682530635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e49d9f47b0 con 0x55e49d9f4060 2024-09-19T08:18:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.330+0000 7f265923d640 1 --2- 172.21.15.22:0/682530635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4980 0x55e49da12650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:18:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.330+0000 7f2659a3e640 1 --2- 172.21.15.22:0/682530635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4060 0x55e49da12110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2650004650 tx=0x7f26500127f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=624072d6e166a287 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:18:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.330+0000 7f263d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:18:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.330+0000 7f263d7fa640 1 -- 172.21.15.22:0/682530635 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650038060 con 0x55e49d9f4060 2024-09-19T08:18:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.330+0000 7f263d7fa640 1 -- 172.21.15.22:0/682530635 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f26500046c0 con 0x55e49d9f4060 2024-09-19T08:18:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.330+0000 7f265e07b740 1 -- 172.21.15.22:0/682530635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f265000fe20 con 0x55e49d9f4060 2024-09-19T08:18:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.330+0000 7f263d7fa640 1 -- 172.21.15.22:0/682530635 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f26500048c0 con 0x55e49d9f4060 2024-09-19T08:18:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.330+0000 7f265e07b740 10 client.25720 fetch_fsmap learned FSMap version 28 2024-09-19T08:18:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.330+0000 7f265e07b740 1 -- 172.21.15.22:0/682530635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e49da79a00 con 0x55e49d9f4060 2024-09-19T08:18:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.331+0000 7f263d7fa640 1 -- 172.21.15.22:0/682530635 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f265000fe20 con 0x55e49d9f4060 2024-09-19T08:18:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.331+0000 7f263d7fa640 1 -- 172.21.15.22:0/682530635 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f2650029530 con 0x55e49d9f4060 2024-09-19T08:18:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.331+0000 7f265e07b740 10 client.25720 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:18:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.331+0000 7f265e07b740 10 client.25720 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:18:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.331+0000 7f265e07b740 20 client.25720 populate_metadata read hostname 'smithi022' 2024-09-19T08:18:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.331+0000 7f265e07b740 1 --2- 172.21.15.22:0/682530635 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e49da7a090 0x55e49da7c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:18:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.331+0000 7f265e07b740 4 client.25720 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4ab59399", "release": true, "await": true}] 2024-09-19T08:18:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.331+0000 7f265e07b740 1 -- 172.21.15.22:0/682530635 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4ab59399", "release": true, "await": true}) -- 0x55e49d9610a0 con 0x55e49da7a090 2024-09-19T08:18:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.331+0000 7f265923d640 1 --2- 172.21.15.22:0/682530635 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e49da7a090 0x55e49da7c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:18:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.331+0000 7f265923d640 1 --2- 172.21.15.22:0/682530635 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e49da7a090 0x55e49da7c4b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa5580e21daefbe4 server_cookie=e236cf01efc9551b in_seq=0 out_seq=0 2024-09-19T08:18:52.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.331+0000 7f263d7fa640 10 client.25720 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:52.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:52 smithi022 ceph-mon[30963]: pgmap v118: 97 pgs: 97 active+clean; 431 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 298 op/s 2024-09-19T08:18:52.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:52 smithi064 ceph-mon[34900]: pgmap v118: 97 pgs: 97 active+clean; 431 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 298 op/s 2024-09-19T08:18:52.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:52 smithi157 ceph-mon[35143]: pgmap v118: 97 pgs: 97 active+clean; 431 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 298 op/s 2024-09-19T08:18:52.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f263d7fa640 1 -- 172.21.15.22:0/682530635 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e49d9610a0 con 0x55e49da7a090 2024-09-19T08:18:52.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f263d7fa640 10 client.25720 handle_command_reply: tid=0 2024-09-19T08:18:52.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f265e07b740 1 client.25720 shutdown 2024-09-19T08:18:52.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f265e07b740 1 -- 172.21.15.22:0/682530635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4060 msgr2=0x55e49da12110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:52.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f265e07b740 1 --2- 172.21.15.22:0/682530635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4060 0x55e49da12110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2650004650 tx=0x7f26500127f0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f265e07b740 1 -- 172.21.15.22:0/682530635 shutdown_connections 2024-09-19T08:18:52.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f265e07b740 1 --2- 172.21.15.22:0/682530635 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e49da7a090 0x55e49da7c4b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f265e07b740 1 --2- 172.21.15.22:0/682530635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49d9ff240 0x55e49da78590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f265e07b740 1 --2- 172.21.15.22:0/682530635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e49d9f4980 0x55e49da12650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f263d7fa640 0 client.25720 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:18:52.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f265e07b740 1 --2- 172.21.15.22:0/682530635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e49d9f4060 0x55e49da12110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f265e07b740 1 -- 172.21.15.22:0/682530635 >> 172.21.15.22:0/682530635 conn(0x55e49d9f05f0 msgr2=0x55e49da0aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:52.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f265e07b740 1 -- 172.21.15.22:0/682530635 shutdown_connections 2024-09-19T08:18:52.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f265e07b740 1 -- 172.21.15.22:0/682530635 wait complete. 2024-09-19T08:18:52.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.985+0000 7f265e07b740 20 client.25720 trim_cache size 0 max 16384 2024-09-19T08:18:52.987 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:18:52.987 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 28, 2024-09-19T08:18:52.987 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:18:52.987 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 16, 2024-09-19T08:18:52.987 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:18:52.987 INFO:teuthology.orchestra.run.smithi022.stdout: "4ab59399": { 2024-09-19T08:18:52.987 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 16, 2024-09-19T08:18:52.988 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:18:52.988 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:18:52.988 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:18:52.988 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:18:52.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:18:52.988 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:18:52.988 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 136.6, 2024-09-19T08:18:52.988 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:18:52.988 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:18:52.988 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:18:52.988 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:18:52.988 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:18:52.989 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:18:52.989 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:18:52.989 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:18:52.989 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:18:52.989 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:18:52.989 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:18:52.989 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:18:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.986+0000 7f265bca2640 1 -- 172.21.15.22:0/2589292070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f26400a33b0 msgr2=0x7f26400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.986+0000 7f265bca2640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f26400a33b0 0x7f26400a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f264c006010 tx=0x7f264c009360 comp rx=0 tx=0).stop 2024-09-19T08:18:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.986+0000 7f265bca2640 1 -- 172.21.15.22:0/2589292070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26541027c0 msgr2=0x7f2654074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:18:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.986+0000 7f265bca2640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26541027c0 0x7f2654074a30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f264400e9f0 tx=0x7f264400eeb0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.986+0000 7f265bca2640 1 -- 172.21.15.22:0/2589292070 shutdown_connections 2024-09-19T08:18:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.986+0000 7f265bca2640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f26400a33b0 0x7f26400a5870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.986+0000 7f265bca2640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2654106f40 0x7f2654074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.986+0000 7f265bca2640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26541027c0 0x7f2654074a30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.986+0000 7f265bca2640 1 --2- 172.21.15.22:0/2589292070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2654101e10 0x7f26540744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:18:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.986+0000 7f265bca2640 1 -- 172.21.15.22:0/2589292070 >> 172.21.15.22:0/2589292070 conn(0x7f26540fd8d0 msgr2=0x7f26540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:18:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.986+0000 7f265bca2640 1 -- 172.21.15.22:0/2589292070 shutdown_connections 2024-09-19T08:18:52.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:18:52.986+0000 7f265bca2640 1 -- 172.21.15.22:0/2589292070 wait complete. 2024-09-19T08:18:54.028 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4ab59399', total seconds elapsed: 28.1 2024-09-19T08:18:54.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:54 smithi064 ceph-mon[34900]: pgmap v119: 97 pgs: 97 active+clean; 446 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.6 MiB/s wr, 498 op/s 2024-09-19T08:18:54.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:54 smithi157 ceph-mon[35143]: pgmap v119: 97 pgs: 97 active+clean; 446 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.6 MiB/s wr, 498 op/s 2024-09-19T08:18:54.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:54 smithi022 ceph-mon[30963]: pgmap v119: 97 pgs: 97 active+clean; 446 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.6 MiB/s wr, 498 op/s 2024-09-19T08:18:56.154 INFO:tasks.workunit.client.0.smithi022.stdout: 84 11849 25 7414 68 7484 101 2024-09-19T08:18:56.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:56 smithi064 ceph-mon[34900]: pgmap v120: 97 pgs: 97 active+clean; 446 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 422 op/s 2024-09-19T08:18:56.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:56 smithi157 ceph-mon[35143]: pgmap v120: 97 pgs: 97 active+clean; 446 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 422 op/s 2024-09-19T08:18:56.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:56 smithi022 ceph-mon[30963]: pgmap v120: 97 pgs: 97 active+clean; 446 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 422 op/s 2024-09-19T08:18:57.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:57 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:18:57.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:57 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:18:57.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:57 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:18:58.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:58 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:18:58.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:58 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:18:58.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:58 smithi064 ceph-mon[34900]: pgmap v121: 97 pgs: 97 active+clean; 450 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.4 MiB/s wr, 449 op/s 2024-09-19T08:18:58.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:18:58 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:18:58.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:58 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:18:58.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:58 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:18:58.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:58 smithi157 ceph-mon[35143]: pgmap v121: 97 pgs: 97 active+clean; 450 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.4 MiB/s wr, 449 op/s 2024-09-19T08:18:58.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:18:58 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:18:58.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:58 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:18:58.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:58 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:18:58.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:58 smithi022 ceph-mon[30963]: pgmap v121: 97 pgs: 97 active+clean; 450 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.4 MiB/s wr, 449 op/s 2024-09-19T08:18:58.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:18:58 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:19:00.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:00 smithi064 ceph-mon[34900]: pgmap v122: 97 pgs: 97 active+clean; 460 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.4 MiB/s wr, 667 op/s 2024-09-19T08:19:00.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:00 smithi157 ceph-mon[35143]: pgmap v122: 97 pgs: 97 active+clean; 460 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.4 MiB/s wr, 667 op/s 2024-09-19T08:19:00.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:00 smithi022 ceph-mon[30963]: pgmap v122: 97 pgs: 97 active+clean; 460 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.4 MiB/s wr, 667 op/s 2024-09-19T08:19:02.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:02 smithi064 ceph-mon[34900]: pgmap v123: 97 pgs: 97 active+clean; 460 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.7 MiB/s wr, 452 op/s 2024-09-19T08:19:02.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:02 smithi157 ceph-mon[35143]: pgmap v123: 97 pgs: 97 active+clean; 460 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.7 MiB/s wr, 452 op/s 2024-09-19T08:19:02.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:02 smithi022 ceph-mon[30963]: pgmap v123: 97 pgs: 97 active+clean; 460 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.7 MiB/s wr, 452 op/s 2024-09-19T08:19:04.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:04 smithi064 ceph-mon[34900]: pgmap v124: 97 pgs: 97 active+clean; 476 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.3 MiB/s wr, 673 op/s 2024-09-19T08:19:04.752 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:04 smithi157 ceph-mon[35143]: pgmap v124: 97 pgs: 97 active+clean; 476 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.3 MiB/s wr, 673 op/s 2024-09-19T08:19:04.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:04 smithi022 ceph-mon[30963]: pgmap v124: 97 pgs: 97 active+clean; 476 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.3 MiB/s wr, 673 op/s 2024-09-19T08:19:04.856 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:19:05.433 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:19:06.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.148+0000 7f6f93e91640 1 -- 172.21.15.22:0/1807580255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f8c1008f0 msgr2=0x7f6f8c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:06.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.148+0000 7f6f93e91640 1 --2- 172.21.15.22:0/1807580255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f8c1008f0 0x7f6f8c100d50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009930 tx=0x7f6f7c02a420 comp rx=0 tx=0).stop 2024-09-19T08:19:06.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.148+0000 7f6f93e91640 1 -- 172.21.15.22:0/1807580255 shutdown_connections 2024-09-19T08:19:06.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.148+0000 7f6f93e91640 1 --2- 172.21.15.22:0/1807580255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c101290 0x7f6f8c10d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.148+0000 7f6f93e91640 1 --2- 172.21.15.22:0/1807580255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f8c1008f0 0x7f6f8c100d50 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.148+0000 7f6f93e91640 1 --2- 172.21.15.22:0/1807580255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c1068f0 0x7f6f8c106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.148+0000 7f6f93e91640 1 -- 172.21.15.22:0/1807580255 >> 172.21.15.22:0/1807580255 conn(0x7f6f8c0fc630 msgr2=0x7f6f8c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:06.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.149+0000 7f6f93e91640 1 -- 172.21.15.22:0/1807580255 shutdown_connections 2024-09-19T08:19:06.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.149+0000 7f6f93e91640 1 -- 172.21.15.22:0/1807580255 wait complete. 2024-09-19T08:19:06.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.150+0000 7f6f93e91640 1 Processor -- start 2024-09-19T08:19:06.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.150+0000 7f6f93e91640 1 -- start start 2024-09-19T08:19:06.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.150+0000 7f6f93e91640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c1008f0 0x7f6f8c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.150+0000 7f6f93e91640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c101290 0x7f6f8c19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.150+0000 7f6f93e91640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f8c1068f0 0x7f6f8c19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f9142c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c101290 0x7f6f8c19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f9142c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c101290 0x7f6f8c19a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:59108/0 (socket says 172.21.15.22:59108) 2024-09-19T08:19:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f91c2d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c1008f0 0x7f6f8c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f91c2d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c1008f0 0x7f6f8c19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:47972/0 (socket says 172.21.15.22:47972) 2024-09-19T08:19:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f9142c640 1 -- 172.21.15.22:0/1858917300 learned_addr learned my addr 172.21.15.22:0/1858917300 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f93e91640 1 -- 172.21.15.22:0/1858917300 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c10fc40 con 0x7f6f8c1068f0 2024-09-19T08:19:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f93e91640 1 -- 172.21.15.22:0/1858917300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c10fac0 con 0x7f6f8c1008f0 2024-09-19T08:19:06.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f93e91640 1 -- 172.21.15.22:0/1858917300 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c10fdc0 con 0x7f6f8c101290 2024-09-19T08:19:06.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f9242e640 1 --2- 172.21.15.22:0/1858917300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f8c1068f0 0x7f6f8c19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f9142c640 1 -- 172.21.15.22:0/1858917300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c1008f0 msgr2=0x7f6f8c19a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:06.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f9142c640 1 --2- 172.21.15.22:0/1858917300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c1008f0 0x7f6f8c19a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f9142c640 1 -- 172.21.15.22:0/1858917300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f8c1068f0 msgr2=0x7f6f8c19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:06.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f9142c640 1 --2- 172.21.15.22:0/1858917300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f8c1068f0 0x7f6f8c19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f9142c640 1 -- 172.21.15.22:0/1858917300 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c19f300 con 0x7f6f8c101290 2024-09-19T08:19:06.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.151+0000 7f6f9142c640 1 --2- 172.21.15.22:0/1858917300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c101290 0x7f6f8c19a890 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c02a940 tx=0x7f6f7c030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:06.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.152+0000 7f6f82ffd640 1 -- 172.21.15.22:0/1858917300 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c03f070 con 0x7f6f8c101290 2024-09-19T08:19:06.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.152+0000 7f6f82ffd640 1 -- 172.21.15.22:0/1858917300 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f7c004460 con 0x7f6f8c101290 2024-09-19T08:19:06.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.152+0000 7f6f82ffd640 1 -- 172.21.15.22:0/1858917300 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c035030 con 0x7f6f8c101290 2024-09-19T08:19:06.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.152+0000 7f6f93e91640 1 -- 172.21.15.22:0/1858917300 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8c19f590 con 0x7f6f8c101290 2024-09-19T08:19:06.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.152+0000 7f6f93e91640 1 -- 172.21.15.22:0/1858917300 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8c19f930 con 0x7f6f8c101290 2024-09-19T08:19:06.154 INFO:tasks.workunit.client.0.smithi022.stdout: 88 487073 362 287106 275 240222 778 2024-09-19T08:19:06.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.156+0000 7f6f82ffd640 1 -- 172.21.15.22:0/1858917300 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6f7c042070 con 0x7f6f8c101290 2024-09-19T08:19:06.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.156+0000 7f6f9626a740 1 Processor -- start 2024-09-19T08:19:06.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.157+0000 7f6f9626a740 1 -- start start 2024-09-19T08:19:06.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.157+0000 7f6f9626a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d42670 0x5578d6d42a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.157+0000 7f6f9626a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d4d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.157+0000 7f6f9242e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d42670 0x5578d6d42a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.157+0000 7f6f9242e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d42670 0x5578d6d42a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:59118/0 (socket says 172.21.15.22:59118) 2024-09-19T08:19:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.157+0000 7f6f9626a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d4d850 0x5578d6d4fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.158+0000 7f6f9242e640 1 -- 172.21.15.22:0/2657697072 learned_addr learned my addr 172.21.15.22:0/2657697072 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.158+0000 7f6f9626a740 1 -- 172.21.15.22:0/2657697072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5578d6ca82e0 con 0x5578d6d4d850 2024-09-19T08:19:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.158+0000 7f6f9626a740 1 -- 172.21.15.22:0/2657697072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5578d6c4fce0 con 0x5578d6d42f90 2024-09-19T08:19:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.158+0000 7f6f9626a740 1 -- 172.21.15.22:0/2657697072 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5578d6ca42a0 con 0x5578d6d42670 2024-09-19T08:19:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.158+0000 7f6f9142c640 1 --2- 172.21.15.22:0/2657697072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d4d850 0x5578d6d4fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.158+0000 7f6f91c2d640 1 --2- 172.21.15.22:0/2657697072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d4d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.160+0000 7f6f9242e640 1 -- 172.21.15.22:0/2657697072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 msgr2=0x5578d6d4d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:06.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.160+0000 7f6f9242e640 1 --2- 172.21.15.22:0/2657697072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d4d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.160+0000 7f6f9242e640 1 -- 172.21.15.22:0/2657697072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d4d850 msgr2=0x5578d6d4fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:06.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.160+0000 7f6f9242e640 1 --2- 172.21.15.22:0/2657697072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d4d850 0x5578d6d4fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.160+0000 7f6f9242e640 1 -- 172.21.15.22:0/2657697072 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d6c34440 con 0x5578d6d42670 2024-09-19T08:19:06.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.160+0000 7f6f91c2d640 1 --2- 172.21.15.22:0/2657697072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d4d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:19:06.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.161+0000 7f6f9142c640 1 --2- 172.21.15.22:0/2657697072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d4d850 0x5578d6d4fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:06.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.161+0000 7f6f9242e640 1 --2- 172.21.15.22:0/2657697072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d42670 0x5578d6d42a50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6f84003a50 tx=0x7f6f8400b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aa8cfac1713345 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:06.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.163+0000 7f6f82ffd640 1 --2- 172.21.15.22:0/1858917300 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6f780a34b0 0x7f6f780a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.163+0000 7f6f82ffd640 1 -- 172.21.15.22:0/1858917300 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6f7c0f0540 con 0x7f6f8c101290 2024-09-19T08:19:06.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.164+0000 7f6f77fff640 1 -- 172.21.15.22:0/2657697072 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84004690 con 0x5578d6d42670 2024-09-19T08:19:06.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.164+0000 7f6f77fff640 1 -- 172.21.15.22:0/2657697072 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f84004830 con 0x5578d6d42670 2024-09-19T08:19:06.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.164+0000 7f6f77fff640 1 -- 172.21.15.22:0/2657697072 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f840166d0 con 0x5578d6d42670 2024-09-19T08:19:06.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.166+0000 7f6f9626a740 1 -- 172.21.15.22:0/2657697072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d42670 msgr2=0x5578d6d42a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.166+0000 7f6f9626a740 1 --2- 172.21.15.22:0/2657697072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d42670 0x5578d6d42a50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6f84003a50 tx=0x7f6f8400b710 comp rx=0 tx=0).stop 2024-09-19T08:19:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.166+0000 7f6f91c2d640 1 --2- 172.21.15.22:0/1858917300 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6f780a34b0 0x7f6f780a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.167+0000 7f6f91c2d640 1 --2- 172.21.15.22:0/1858917300 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6f780a34b0 0x7f6f780a5970 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x5578d6d3fa60 tx=0x7f6f88005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:06.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.167+0000 7f6f9626a740 1 -- 172.21.15.22:0/2657697072 shutdown_connections 2024-09-19T08:19:06.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.167+0000 7f6f9626a740 1 --2- 172.21.15.22:0/2657697072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d4d850 0x5578d6d4fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.167+0000 7f6f9626a740 1 --2- 172.21.15.22:0/2657697072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d4d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.167+0000 7f6f9626a740 1 --2- 172.21.15.22:0/2657697072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d42670 0x5578d6d42a50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.167+0000 7f6f9626a740 1 -- 172.21.15.22:0/2657697072 >> 172.21.15.22:0/2657697072 conn(0x5578d6d3ec00 msgr2=0x5578d6d3efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:06.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.168+0000 7f6f9626a740 1 -- 172.21.15.22:0/2657697072 shutdown_connections 2024-09-19T08:19:06.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.168+0000 7f6f9626a740 1 -- 172.21.15.22:0/2657697072 wait complete. 2024-09-19T08:19:06.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.169+0000 7f6f9626a740 1 -- start start 2024-09-19T08:19:06.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.169+0000 7f6f9626a740 1 Processor -- start 2024-09-19T08:19:06.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.169+0000 7f6f9626a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d42670 0x5578d6d4d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.169+0000 7f6f9626a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d4a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.169+0000 7f6f9626a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d4d850 0x5578d6d4aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.169+0000 7f6f9626a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5578d6d50420 con 0x5578d6d42670 2024-09-19T08:19:06.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.169+0000 7f6f9626a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5578d6c63050 con 0x5578d6d42f90 2024-09-19T08:19:06.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.169+0000 7f6f9626a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5578d6d505a0 con 0x5578d6d4d850 2024-09-19T08:19:06.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.170+0000 7f6f9626a740 10 client.0 _ll_register_callbacks cb 0x5578d6ca3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:06.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.170+0000 7f6f9242e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d4d850 0x5578d6d4aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.170+0000 7f6f9242e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d4d850 0x5578d6d4aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:59128/0 (socket says 172.21.15.22:59128) 2024-09-19T08:19:06.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.170+0000 7f6f9242e640 1 -- 172.21.15.22:0/2926880910 learned_addr learned my addr 172.21.15.22:0/2926880910 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:06.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.170+0000 7f6f91c2d640 1 --2- 172.21.15.22:0/2926880910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d42670 0x5578d6d4d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.170+0000 7f6f9142c640 1 --2- 172.21.15.22:0/2926880910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d4a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.171+0000 7f6f9142c640 1 -- 172.21.15.22:0/2926880910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d4d850 msgr2=0x5578d6d4aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:06.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.171+0000 7f6f9142c640 1 --2- 172.21.15.22:0/2926880910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d4d850 0x5578d6d4aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.171+0000 7f6f9142c640 1 -- 172.21.15.22:0/2926880910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d42670 msgr2=0x5578d6d4d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:06.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.171+0000 7f6f9142c640 1 --2- 172.21.15.22:0/2926880910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d42670 0x5578d6d4d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.171+0000 7f6f9142c640 1 -- 172.21.15.22:0/2926880910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d6a193c0 con 0x5578d6d42f90 2024-09-19T08:19:06.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.171+0000 7f6f91c2d640 1 --2- 172.21.15.22:0/2926880910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d42670 0x5578d6d4d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:06.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.172+0000 7f6f9142c640 1 --2- 172.21.15.22:0/2926880910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d4a4b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0edc40 tx=0x7f6f7c0037d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13f4488e45be3ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.172+0000 7f6f757fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:19:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.173+0000 7f6f757fa640 1 -- 172.21.15.22:0/2926880910 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c049070 con 0x5578d6d42f90 2024-09-19T08:19:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.173+0000 7f6f757fa640 1 -- 172.21.15.22:0/2926880910 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6f7c0c4fb0 con 0x5578d6d42f90 2024-09-19T08:19:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.173+0000 7f6f9626a740 1 -- 172.21.15.22:0/2926880910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f84004830 con 0x5578d6d42f90 2024-09-19T08:19:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.173+0000 7f6f757fa640 1 -- 172.21.15.22:0/2926880910 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f7c0bc650 con 0x5578d6d42f90 2024-09-19T08:19:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.173+0000 7f6f9626a740 10 client.25790 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.173+0000 7f6f9626a740 1 -- 172.21.15.22:0/2926880910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5578d6d5be30 con 0x5578d6d42f90 2024-09-19T08:19:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.174+0000 7f6f757fa640 1 -- 172.21.15.22:0/2926880910 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f84004830 con 0x5578d6d42f90 2024-09-19T08:19:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.174+0000 7f6f757fa640 1 -- 172.21.15.22:0/2926880910 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6f7c0a5b80 con 0x5578d6d42f90 2024-09-19T08:19:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.174+0000 7f6f9626a740 10 client.25790 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.174+0000 7f6f9626a740 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:19:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.174+0000 7f6f9626a740 20 client.25790 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.174+0000 7f6f9626a740 1 --2- 172.21.15.22:0/2926880910 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5578d6d5d330 0x5578d6d5f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.174+0000 7f6f9626a740 4 client.25790 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:19:06.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.174+0000 7f6f9626a740 1 -- 172.21.15.22:0/2926880910 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578d6cb5850 con 0x5578d6d5d330 2024-09-19T08:19:06.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.175+0000 7f6f91c2d640 1 --2- 172.21.15.22:0/2926880910 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5578d6d5d330 0x5578d6d5f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.175+0000 7f6f91c2d640 1 --2- 172.21.15.22:0/2926880910 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5578d6d5d330 0x5578d6d5f790 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c14fd3b3bd49a server_cookie=2af1dd5e2e3249e in_seq=0 out_seq=0 2024-09-19T08:19:06.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.175+0000 7f6f757fa640 10 client.25790 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:06.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.177+0000 7f6f757fa640 1 -- 172.21.15.22:0/2926880910 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578d6cb5850 con 0x5578d6d5d330 2024-09-19T08:19:06.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.177+0000 7f6f757fa640 10 client.25790 handle_command_reply: tid=0 2024-09-19T08:19:06.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.177+0000 7f6f9626a740 1 client.25790 shutdown 2024-09-19T08:19:06.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.178+0000 7f6f9626a740 1 -- 172.21.15.22:0/2926880910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 msgr2=0x5578d6d4a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:06.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.178+0000 7f6f9626a740 1 --2- 172.21.15.22:0/2926880910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d4a4b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0edc40 tx=0x7f6f7c0037d0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.178+0000 7f6f9626a740 1 -- 172.21.15.22:0/2926880910 shutdown_connections 2024-09-19T08:19:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.178+0000 7f6f9626a740 1 --2- 172.21.15.22:0/2926880910 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5578d6d5d330 0x5578d6d5f790 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.178+0000 7f6f9626a740 1 --2- 172.21.15.22:0/2926880910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d4d850 0x5578d6d4aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.178+0000 7f6f9626a740 1 --2- 172.21.15.22:0/2926880910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d4a4b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.178+0000 7f6f9626a740 1 --2- 172.21.15.22:0/2926880910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d42670 0x5578d6d4d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.178+0000 7f6f9626a740 1 -- 172.21.15.22:0/2926880910 >> 172.21.15.22:0/2926880910 conn(0x5578d6d3ec00 msgr2=0x5578d6d448e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.178+0000 7f6f757fa640 0 client.25790 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:06.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.179+0000 7f6f9626a740 1 -- 172.21.15.22:0/2926880910 shutdown_connections 2024-09-19T08:19:06.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.179+0000 7f6f9626a740 1 -- 172.21.15.22:0/2926880910 wait complete. 2024-09-19T08:19:06.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.179+0000 7f6f9626a740 20 client.25790 trim_cache size 0 max 16384 2024-09-19T08:19:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.217+0000 7f6f9626a740 1 Processor -- start 2024-09-19T08:19:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.217+0000 7f6f9626a740 1 -- start start 2024-09-19T08:19:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.217+0000 7f6f9626a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d42670 0x5578d6d59680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.217+0000 7f6f9626a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d65060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.218+0000 7f6f91c2d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d42670 0x5578d6d59680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.218+0000 7f6f91c2d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d42670 0x5578d6d59680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58756/0 (socket says 172.21.15.22:58756) 2024-09-19T08:19:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.218+0000 7f6f91c2d640 1 -- 172.21.15.22:0/3255223161 learned_addr learned my addr 172.21.15.22:0/3255223161 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.218+0000 7f6f9626a740 1 --2- 172.21.15.22:0/3255223161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d4d850 0x5578d6d655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.218+0000 7f6f9626a740 1 -- 172.21.15.22:0/3255223161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c1b6650 con 0x5578d6d42670 2024-09-19T08:19:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.218+0000 7f6f9626a740 1 -- 172.21.15.22:0/3255223161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c1b3ec0 con 0x5578d6d42f90 2024-09-19T08:19:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.218+0000 7f6f9626a740 1 -- 172.21.15.22:0/3255223161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c1b1d50 con 0x5578d6d4d850 2024-09-19T08:19:06.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.218+0000 7f6f9242e640 1 --2- 172.21.15.22:0/3255223161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d4d850 0x5578d6d655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.218+0000 7f6f91c2d640 1 -- 172.21.15.22:0/3255223161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d4d850 msgr2=0x5578d6d655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:06.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.218+0000 7f6f91c2d640 1 --2- 172.21.15.22:0/3255223161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d4d850 0x5578d6d655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.218+0000 7f6f91c2d640 1 -- 172.21.15.22:0/3255223161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 msgr2=0x5578d6d65060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:19:06.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.218+0000 7f6f91c2d640 1 --2- 172.21.15.22:0/3255223161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d65060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.218+0000 7f6f91c2d640 1 -- 172.21.15.22:0/3255223161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d6d3a200 con 0x5578d6d42670 2024-09-19T08:19:06.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.218+0000 7f6f91c2d640 1 --2- 172.21.15.22:0/3255223161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d42670 0x5578d6d59680 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009b10 tx=0x7f6f8801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81907a37624fc957 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:06.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.219+0000 7f6f777fe640 1 -- 172.21.15.22:0/3255223161 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8802b4e0 con 0x5578d6d42670 2024-09-19T08:19:06.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.219+0000 7f6f777fe640 1 -- 172.21.15.22:0/3255223161 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f8802bb60 con 0x5578d6d42670 2024-09-19T08:19:06.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.219+0000 7f6f777fe640 1 -- 172.21.15.22:0/3255223161 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8801fe50 con 0x5578d6d42670 2024-09-19T08:19:06.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.220+0000 7f6f9626a740 1 -- 172.21.15.22:0/3255223161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d42670 msgr2=0x5578d6d59680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:06.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.220+0000 7f6f9626a740 1 --2- 172.21.15.22:0/3255223161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d42670 0x5578d6d59680 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009b10 tx=0x7f6f8801b000 comp rx=0 tx=0).stop 2024-09-19T08:19:06.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.220+0000 7f6f9626a740 1 -- 172.21.15.22:0/3255223161 shutdown_connections 2024-09-19T08:19:06.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.220+0000 7f6f9626a740 1 --2- 172.21.15.22:0/3255223161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d4d850 0x5578d6d655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.220+0000 7f6f9626a740 1 --2- 172.21.15.22:0/3255223161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d65060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.220+0000 7f6f9626a740 1 --2- 172.21.15.22:0/3255223161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d42670 0x5578d6d59680 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.220+0000 7f6f9626a740 1 -- 172.21.15.22:0/3255223161 >> 172.21.15.22:0/3255223161 conn(0x5578d6d3ec00 msgr2=0x5578d6d448e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:06.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.221+0000 7f6f9626a740 1 -- 172.21.15.22:0/3255223161 shutdown_connections 2024-09-19T08:19:06.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.221+0000 7f6f9626a740 1 -- 172.21.15.22:0/3255223161 wait complete. 2024-09-19T08:19:06.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.221+0000 7f6f9626a740 1 -- start start 2024-09-19T08:19:06.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.222+0000 7f6f9626a740 1 Processor -- start 2024-09-19T08:19:06.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.222+0000 7f6f9626a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d42670 0x5578d6d51480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.223+0000 7f6f9626a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d519c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.223+0000 7f6f91c2d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d42670 0x5578d6d51480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.223+0000 7f6f91c2d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d42670 0x5578d6d51480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:59144/0 (socket says 172.21.15.22:59144) 2024-09-19T08:19:06.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.223+0000 7f6f9142c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.223+0000 7f6f91c2d640 1 -- 172.21.15.22:0/203801072 learned_addr learned my addr 172.21.15.22:0/203801072 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:06.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.223+0000 7f6f9626a740 1 --2- 172.21.15.22:0/203801072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d4d850 0x5578d6d51f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.224+0000 7f6f9242e640 1 --2- 172.21.15.22:0/203801072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d4d850 0x5578d6d51f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.224+0000 7f6f9626a740 1 -- 172.21.15.22:0/203801072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c1b23a0 con 0x5578d6d4d850 2024-09-19T08:19:06.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.224+0000 7f6f9626a740 1 -- 172.21.15.22:0/203801072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c1b37f0 con 0x5578d6d42f90 2024-09-19T08:19:06.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.224+0000 7f6f9626a740 1 -- 172.21.15.22:0/203801072 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c1b2a50 con 0x5578d6d42670 2024-09-19T08:19:06.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.225+0000 7f6f9626a740 10 client.0 _ll_register_callbacks cb 0x7f6f7c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.225+0000 7f6f9242e640 1 -- 172.21.15.22:0/203801072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d42670 msgr2=0x5578d6d51480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.225+0000 7f6f9242e640 1 --2- 172.21.15.22:0/203801072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d42670 0x5578d6d51480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.225+0000 7f6f9242e640 1 -- 172.21.15.22:0/203801072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 msgr2=0x5578d6d519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.225+0000 7f6f9242e640 1 --2- 172.21.15.22:0/203801072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.225+0000 7f6f9242e640 1 -- 172.21.15.22:0/203801072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d6dc7030 con 0x5578d6d4d850 2024-09-19T08:19:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.225+0000 7f6f9142c640 1 --2- 172.21.15.22:0/203801072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.226+0000 7f6f9242e640 1 --2- 172.21.15.22:0/203801072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d4d850 0x5578d6d51f00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6f84000f80 tx=0x7f6f84005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451dedbf4b50c3b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.226+0000 7f6f757fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:19:06.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.226+0000 7f6f757fa640 1 -- 172.21.15.22:0/203801072 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84004690 con 0x5578d6d4d850 2024-09-19T08:19:06.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.226+0000 7f6f757fa640 1 -- 172.21.15.22:0/203801072 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6f84004830 con 0x5578d6d4d850 2024-09-19T08:19:06.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.227+0000 7f6f9626a740 1 -- 172.21.15.22:0/203801072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f8802bb60 con 0x5578d6d4d850 2024-09-19T08:19:06.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.227+0000 7f6f757fa640 1 -- 172.21.15.22:0/203801072 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f84004a50 con 0x5578d6d4d850 2024-09-19T08:19:06.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.227+0000 7f6f9626a740 10 client.16194 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:06.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.227+0000 7f6f9626a740 1 -- 172.21.15.22:0/203801072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5578d6dc8220 con 0x5578d6d4d850 2024-09-19T08:19:06.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.228+0000 7f6f757fa640 1 -- 172.21.15.22:0/203801072 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f8802bb60 con 0x5578d6d4d850 2024-09-19T08:19:06.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.228+0000 7f6f757fa640 1 -- 172.21.15.22:0/203801072 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6f84029070 con 0x5578d6d4d850 2024-09-19T08:19:06.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.228+0000 7f6f9626a740 10 client.16194 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:06.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.228+0000 7f6f9626a740 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:19:06.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.228+0000 7f6f9626a740 20 client.16194 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:06.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.228+0000 7f6f9626a740 1 --2- 172.21.15.22:0/203801072 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5578d6dc8700 0x5578d6dcaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:06.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.228+0000 7f6f9626a740 4 client.16194 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:19:06.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.228+0000 7f6f9626a740 1 -- 172.21.15.22:0/203801072 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578d6cafd60 con 0x5578d6dc8700 2024-09-19T08:19:06.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.228+0000 7f6f91c2d640 1 --2- 172.21.15.22:0/203801072 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5578d6dc8700 0x5578d6dcaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:06.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.229+0000 7f6f91c2d640 1 --2- 172.21.15.22:0/203801072 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5578d6dc8700 0x5578d6dcaaf0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20b2d9aa79158dfa server_cookie=47d42c2e55f19fa in_seq=0 out_seq=0 2024-09-19T08:19:06.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.229+0000 7f6f757fa640 10 client.16194 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:06.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.229+0000 7f6f757fa640 1 -- 172.21.15.22:0/203801072 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5578d6cafd60 con 0x5578d6dc8700 2024-09-19T08:19:06.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.229+0000 7f6f757fa640 10 client.16194 handle_command_reply: tid=0 2024-09-19T08:19:06.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.229+0000 7f6f9626a740 1 client.16194 shutdown 2024-09-19T08:19:06.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.229+0000 7f6f9626a740 1 -- 172.21.15.22:0/203801072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d4d850 msgr2=0x5578d6d51f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:06.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.229+0000 7f6f9626a740 1 --2- 172.21.15.22:0/203801072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d4d850 0x5578d6d51f00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6f84000f80 tx=0x7f6f84005720 comp rx=0 tx=0).stop 2024-09-19T08:19:06.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.229+0000 7f6f9626a740 1 -- 172.21.15.22:0/203801072 shutdown_connections 2024-09-19T08:19:06.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.229+0000 7f6f9626a740 1 --2- 172.21.15.22:0/203801072 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5578d6dc8700 0x5578d6dcaaf0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.230+0000 7f6f9626a740 1 --2- 172.21.15.22:0/203801072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5578d6d4d850 0x5578d6d51f00 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.230+0000 7f6f9626a740 1 --2- 172.21.15.22:0/203801072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578d6d42f90 0x5578d6d519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.230+0000 7f6f757fa640 0 client.16194 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:06.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.230+0000 7f6f9626a740 1 --2- 172.21.15.22:0/203801072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5578d6d42670 0x5578d6d51480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.230+0000 7f6f9626a740 1 -- 172.21.15.22:0/203801072 >> 172.21.15.22:0/203801072 conn(0x5578d6d3ec00 msgr2=0x5578d6d48d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:06.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.230+0000 7f6f9626a740 1 -- 172.21.15.22:0/203801072 shutdown_connections 2024-09-19T08:19:06.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.230+0000 7f6f9626a740 1 -- 172.21.15.22:0/203801072 wait complete. 2024-09-19T08:19:06.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.230+0000 7f6f9626a740 20 client.16194 trim_cache size 0 max 16384 2024-09-19T08:19:06.232 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:19:06.232 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-19T08:19:06.232 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-09-19T08:19:06.232 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:19:06.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.231+0000 7f6f93e91640 1 -- 172.21.15.22:0/1858917300 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6f780a34b0 msgr2=0x7f6f780a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:06.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.231+0000 7f6f93e91640 1 --2- 172.21.15.22:0/1858917300 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6f780a34b0 0x7f6f780a5970 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x5578d6d3fa60 tx=0x7f6f88005f90 comp rx=0 tx=0).stop 2024-09-19T08:19:06.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.231+0000 7f6f93e91640 1 -- 172.21.15.22:0/1858917300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c101290 msgr2=0x7f6f8c19a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:06.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.231+0000 7f6f93e91640 1 --2- 172.21.15.22:0/1858917300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c101290 0x7f6f8c19a890 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c02a940 tx=0x7f6f7c030650 comp rx=0 tx=0).stop 2024-09-19T08:19:06.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.231+0000 7f6f93e91640 1 -- 172.21.15.22:0/1858917300 shutdown_connections 2024-09-19T08:19:06.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.231+0000 7f6f93e91640 1 --2- 172.21.15.22:0/1858917300 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f6f780a34b0 0x7f6f780a5970 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.231+0000 7f6f93e91640 1 --2- 172.21.15.22:0/1858917300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f8c1068f0 0x7f6f8c19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.231+0000 7f6f93e91640 1 --2- 172.21.15.22:0/1858917300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f8c101290 0x7f6f8c19a890 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.231+0000 7f6f93e91640 1 --2- 172.21.15.22:0/1858917300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c1008f0 0x7f6f8c19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:06.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.231+0000 7f6f93e91640 1 -- 172.21.15.22:0/1858917300 >> 172.21.15.22:0/1858917300 conn(0x7f6f8c0fc630 msgr2=0x7f6f8c0fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:06.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.231+0000 7f6f93e91640 1 -- 172.21.15.22:0/1858917300 shutdown_connections 2024-09-19T08:19:06.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:06.231+0000 7f6f93e91640 1 -- 172.21.15.22:0/1858917300 wait complete. 2024-09-19T08:19:06.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:06 smithi022 ceph-mon[30963]: pgmap v125: 97 pgs: 97 active+clean; 476 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 473 op/s 2024-09-19T08:19:06.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:06 smithi064 ceph-mon[34900]: pgmap v125: 97 pgs: 97 active+clean; 476 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 473 op/s 2024-09-19T08:19:06.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:06 smithi157 ceph-mon[35143]: pgmap v125: 97 pgs: 97 active+clean; 476 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 473 op/s 2024-09-19T08:19:07.358 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:19:07.358 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:19:08.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:08 smithi064 ceph-mon[34900]: pgmap v126: 97 pgs: 97 active+clean; 484 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 483 op/s 2024-09-19T08:19:08.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:08 smithi157 ceph-mon[35143]: pgmap v126: 97 pgs: 97 active+clean; 484 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 483 op/s 2024-09-19T08:19:08.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:08 smithi022 ceph-mon[30963]: pgmap v126: 97 pgs: 97 active+clean; 484 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 483 op/s 2024-09-19T08:19:09.526 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:19:10.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.359+0000 7faba3460640 1 -- 172.21.15.22:0/100062739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c100c60 msgr2=0x7fab9c104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:10.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.359+0000 7faba3460640 1 --2- 172.21.15.22:0/100062739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c100c60 0x7fab9c104cb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fab94005fd0 tx=0x7fab9402a6d0 comp rx=0 tx=0).stop 2024-09-19T08:19:10.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.360+0000 7faba3460640 1 -- 172.21.15.22:0/100062739 shutdown_connections 2024-09-19T08:19:10.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.360+0000 7faba3460640 1 --2- 172.21.15.22:0/100062739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab9c1053e0 0x7fab9c10cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:10.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.360+0000 7faba3460640 1 --2- 172.21.15.22:0/100062739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c100c60 0x7fab9c104cb0 secure :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fab94005fd0 tx=0x7fab9402a6d0 comp rx=0 tx=0).stop 2024-09-19T08:19:10.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.360+0000 7faba3460640 1 --2- 172.21.15.22:0/100062739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab9c1002b0 0x7fab9c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:10.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.360+0000 7faba3460640 1 -- 172.21.15.22:0/100062739 >> 172.21.15.22:0/100062739 conn(0x7fab9c077070 msgr2=0x7fab9c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:10.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.370+0000 7faba3460640 1 -- 172.21.15.22:0/100062739 shutdown_connections 2024-09-19T08:19:10.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.374+0000 7faba3460640 1 -- 172.21.15.22:0/100062739 wait complete. 2024-09-19T08:19:10.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.375+0000 7faba3460640 1 Processor -- start 2024-09-19T08:19:10.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.375+0000 7faba3460640 1 -- start start 2024-09-19T08:19:10.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.375+0000 7faba3460640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c1002b0 0x7fab9c10bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:10.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.375+0000 7faba3460640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab9c1053e0 0x7fab9c10c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:10.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.375+0000 7faba3460640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab9c10c890 0x7fab9c114cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:10.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.375+0000 7faba3460640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c10f290 con 0x7fab9c1002b0 2024-09-19T08:19:10.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.375+0000 7faba3460640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c10f110 con 0x7fab9c10c890 2024-09-19T08:19:10.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.375+0000 7faba3460640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c10f410 con 0x7fab9c1053e0 2024-09-19T08:19:10.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.376+0000 7faba2c5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab9c10c890 0x7fab9c114cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:10.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.376+0000 7faba2c5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab9c10c890 0x7fab9c114cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:35408/0 (socket says 172.21.15.22:35408) 2024-09-19T08:19:10.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.376+0000 7faba2c5f640 1 -- 172.21.15.22:0/2922915807 learned_addr learned my addr 172.21.15.22:0/2922915807 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:10.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.376+0000 7faba2c5f640 1 -- 172.21.15.22:0/2922915807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab9c1053e0 msgr2=0x7fab9c10c1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:19:10.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.376+0000 7faba2c5f640 1 --2- 172.21.15.22:0/2922915807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab9c1053e0 0x7fab9c10c1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:10.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.376+0000 7faba2c5f640 1 -- 172.21.15.22:0/2922915807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c1002b0 msgr2=0x7fab9c10bcb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:19:10.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.376+0000 7faba2c5f640 1 --2- 172.21.15.22:0/2922915807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c1002b0 0x7fab9c10bcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:10.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.376+0000 7faba2c5f640 1 -- 172.21.15.22:0/2922915807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9c115570 con 0x7fab9c10c890 2024-09-19T08:19:10.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.381+0000 7faba2c5f640 1 --2- 172.21.15.22:0/2922915807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab9c10c890 0x7fab9c114cb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fab9800a850 tx=0x7fab9800ad10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:10.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.385+0000 7fab8f7fe640 1 -- 172.21.15.22:0/2922915807 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab980079b0 con 0x7fab9c10c890 2024-09-19T08:19:10.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.385+0000 7faba3460640 1 -- 172.21.15.22:0/2922915807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9c115860 con 0x7fab9c10c890 2024-09-19T08:19:10.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.385+0000 7faba3460640 1 -- 172.21.15.22:0/2922915807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9c115da0 con 0x7fab9c10c890 2024-09-19T08:19:10.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.387+0000 7fab8f7fe640 1 -- 172.21.15.22:0/2922915807 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab98005500 con 0x7fab9c10c890 2024-09-19T08:19:10.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.387+0000 7fab8f7fe640 1 -- 172.21.15.22:0/2922915807 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98004de0 con 0x7fab9c10c890 2024-09-19T08:19:10.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.388+0000 7faba3460640 1 -- 172.21.15.22:0/2922915807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab9c104cb0 con 0x7fab9c10c890 2024-09-19T08:19:10.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.397+0000 7fab8f7fe640 1 -- 172.21.15.22:0/2922915807 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fab98005020 con 0x7fab9c10c890 2024-09-19T08:19:10.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.399+0000 7fab8f7fe640 1 --2- 172.21.15.22:0/2922915807 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fab880a35c0 0x7fab880a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:10.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.399+0000 7fab8f7fe640 1 -- 172.21.15.22:0/2922915807 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fab980cfa20 con 0x7fab9c10c890 2024-09-19T08:19:10.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.399+0000 7fab8f7fe640 1 -- 172.21.15.22:0/2922915807 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fab980fe2b0 con 0x7fab9c10c890 2024-09-19T08:19:10.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.403+0000 7faba245e640 1 --2- 172.21.15.22:0/2922915807 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fab880a35c0 0x7fab880a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:10.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.404+0000 7faba245e640 1 --2- 172.21.15.22:0/2922915807 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fab880a35c0 0x7fab880a5a80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fab90009670 tx=0x7fab90006bd0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:10.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:10 smithi064 ceph-mon[34900]: pgmap v127: 97 pgs: 97 active+clean; 491 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 520 op/s 2024-09-19T08:19:10.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:10 smithi157 ceph-mon[35143]: pgmap v127: 97 pgs: 97 active+clean; 491 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 520 op/s 2024-09-19T08:19:10.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:10 smithi022 ceph-mon[30963]: pgmap v127: 97 pgs: 97 active+clean; 491 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 520 op/s 2024-09-19T08:19:10.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.891+0000 7faba3460640 1 -- 172.21.15.22:0/2922915807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab9c10d1e0 con 0x7fab9c10c890 2024-09-19T08:19:10.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.892+0000 7fab8f7fe640 1 -- 172.21.15.22:0/2922915807 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11909 (secure 0 0 0) 0x7fab9809c430 con 0x7fab9c10c890 2024-09-19T08:19:10.892 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:19:10.893 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":29,"btime":"2024-09-19T08:18:57:303741+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:18:56.870055+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:19:10.893 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 29 2024-09-19T08:19:10.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.899+0000 7faba3460640 1 -- 172.21.15.22:0/2922915807 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fab880a35c0 msgr2=0x7fab880a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:10.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.899+0000 7faba3460640 1 --2- 172.21.15.22:0/2922915807 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fab880a35c0 0x7fab880a5a80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fab90009670 tx=0x7fab90006bd0 comp rx=0 tx=0).stop 2024-09-19T08:19:10.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.899+0000 7faba3460640 1 -- 172.21.15.22:0/2922915807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab9c10c890 msgr2=0x7fab9c114cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:10.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.899+0000 7faba3460640 1 --2- 172.21.15.22:0/2922915807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab9c10c890 0x7fab9c114cb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fab9800a850 tx=0x7fab9800ad10 comp rx=0 tx=0).stop 2024-09-19T08:19:10.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.908+0000 7faba3460640 1 -- 172.21.15.22:0/2922915807 shutdown_connections 2024-09-19T08:19:10.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.908+0000 7faba3460640 1 --2- 172.21.15.22:0/2922915807 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fab880a35c0 0x7fab880a5a80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:10.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.908+0000 7faba3460640 1 --2- 172.21.15.22:0/2922915807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab9c10c890 0x7fab9c114cb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:10.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.908+0000 7faba3460640 1 --2- 172.21.15.22:0/2922915807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab9c1053e0 0x7fab9c10c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:10.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.908+0000 7faba3460640 1 --2- 172.21.15.22:0/2922915807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c1002b0 0x7fab9c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:10.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.908+0000 7faba3460640 1 -- 172.21.15.22:0/2922915807 >> 172.21.15.22:0/2922915807 conn(0x7fab9c077070 msgr2=0x7fab9c110d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:10.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.913+0000 7faba3460640 1 -- 172.21.15.22:0/2922915807 shutdown_connections 2024-09-19T08:19:10.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:10.915+0000 7faba3460640 1 -- 172.21.15.22:0/2922915807 wait complete. 2024-09-19T08:19:11.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:11 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2922915807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:19:11.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:11 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2922915807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:19:11.752 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:11 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2922915807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:19:12.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:19:12.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:12 smithi064 ceph-mon[34900]: pgmap v128: 97 pgs: 97 active+clean; 491 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-09-19T08:19:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:12 smithi157 ceph-mon[35143]: pgmap v128: 97 pgs: 97 active+clean; 491 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-09-19T08:19:12.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:12 smithi022 ceph-mon[30963]: pgmap v128: 97 pgs: 97 active+clean; 491 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-09-19T08:19:13.810 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:19:14.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.387+0000 7f22c6eb5640 1 -- 172.21.15.22:0/1769239395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22c010ea80 msgr2=0x7f22c01154f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:14.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.387+0000 7f22c6eb5640 1 --2- 172.21.15.22:0/1769239395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22c010ea80 0x7f22c01154f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f22b8005fd0 tx=0x7f22b802b430 comp rx=0 tx=0).stop 2024-09-19T08:19:14.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.388+0000 7f22c6eb5640 1 -- 172.21.15.22:0/1769239395 shutdown_connections 2024-09-19T08:19:14.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.388+0000 7f22c6eb5640 1 --2- 172.21.15.22:0/1769239395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22c010ea80 0x7f22c01154f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.388+0000 7f22c6eb5640 1 --2- 172.21.15.22:0/1769239395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c0109e00 0x7f22c010e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.388+0000 7f22c6eb5640 1 --2- 172.21.15.22:0/1769239395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c0109450 0x7f22c0109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.388+0000 7f22c6eb5640 1 -- 172.21.15.22:0/1769239395 >> 172.21.15.22:0/1769239395 conn(0x7f22c006e2d0 msgr2=0x7f22c006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:14.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.388+0000 7f22c6eb5640 1 -- 172.21.15.22:0/1769239395 shutdown_connections 2024-09-19T08:19:14.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.388+0000 7f22c6eb5640 1 -- 172.21.15.22:0/1769239395 wait complete. 2024-09-19T08:19:14.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.389+0000 7f22c6eb5640 1 Processor -- start 2024-09-19T08:19:14.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.389+0000 7f22c6eb5640 1 -- start start 2024-09-19T08:19:14.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.389+0000 7f22c6eb5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c0109450 0x7f22c0112740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.389+0000 7f22c6eb5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c0109e00 0x7f22c0112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.389+0000 7f22c6eb5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22c010ea80 0x7f22c01131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.389+0000 7f22c6eb5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01183f0 con 0x7f22c010ea80 2024-09-19T08:19:14.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.389+0000 7f22c6eb5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f22c0118270 con 0x7f22c0109450 2024-09-19T08:19:14.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.389+0000 7f22c6eb5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f22c0118570 con 0x7f22c0109e00 2024-09-19T08:19:14.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.390+0000 7f22c5452640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22c010ea80 0x7f22c01131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:14.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.390+0000 7f22c5452640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22c010ea80 0x7f22c01131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40092/0 (socket says 172.21.15.22:40092) 2024-09-19T08:19:14.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.390+0000 7f22c5452640 1 -- 172.21.15.22:0/3361861445 learned_addr learned my addr 172.21.15.22:0/3361861445 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:14.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.390+0000 7f22c5452640 1 -- 172.21.15.22:0/3361861445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c0109e00 msgr2=0x7f22c0112c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:19:14.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.391+0000 7f22c5452640 1 --2- 172.21.15.22:0/3361861445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c0109e00 0x7f22c0112c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.391+0000 7f22c5452640 1 -- 172.21.15.22:0/3361861445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c0109450 msgr2=0x7f22c0112740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:14.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.391+0000 7f22c4c51640 1 --2- 172.21.15.22:0/3361861445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c0109450 0x7f22c0112740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:14.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.392+0000 7f22c5452640 1 --2- 172.21.15.22:0/3361861445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c0109450 0x7f22c0112740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.393+0000 7f22c5452640 1 -- 172.21.15.22:0/3361861445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c0123100 con 0x7f22c010ea80 2024-09-19T08:19:14.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.393+0000 7f22c4c51640 1 --2- 172.21.15.22:0/3361861445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c0109450 0x7f22c0112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:19:14.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.393+0000 7f22c5452640 1 --2- 172.21.15.22:0/3361861445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22c010ea80 0x7f22c01131c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f22b800a300 tx=0x7f22b800a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:14.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.394+0000 7f22bdffb640 1 -- 172.21.15.22:0/3361861445 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8037040 con 0x7f22c010ea80 2024-09-19T08:19:14.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.394+0000 7f22bdffb640 1 -- 172.21.15.22:0/3361861445 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22b8007c50 con 0x7f22c010ea80 2024-09-19T08:19:14.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.396+0000 7f22c6eb5640 1 -- 172.21.15.22:0/3361861445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c0123390 con 0x7f22c010ea80 2024-09-19T08:19:14.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.396+0000 7f22bdffb640 1 -- 172.21.15.22:0/3361861445 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b803d700 con 0x7f22c010ea80 2024-09-19T08:19:14.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.396+0000 7f22c6eb5640 1 -- 172.21.15.22:0/3361861445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c0123840 con 0x7f22c010ea80 2024-09-19T08:19:14.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.401+0000 7f22c928e740 1 Processor -- start 2024-09-19T08:19:14.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.404+0000 7f22bdffb640 1 -- 172.21.15.22:0/3361861445 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f22b8032950 con 0x7f22c010ea80 2024-09-19T08:19:14.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.404+0000 7f22c928e740 1 -- start start 2024-09-19T08:19:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.404+0000 7f22c928e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c428b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.404+0000 7f22c928e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c428ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.406+0000 7f22bdffb640 1 --2- 172.21.15.22:0/3361861445 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f22a80a3420 0x7f22a80a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.406+0000 7f22bdffb640 1 -- 172.21.15.22:0/3361861445 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f22b80efd30 con 0x7f22c010ea80 2024-09-19T08:19:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.407+0000 7f22c4c51640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c428ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.407+0000 7f22c4c51640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c428ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40098/0 (socket says 172.21.15.22:40098) 2024-09-19T08:19:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.407+0000 7f22bffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c428b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.407+0000 7f22bffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c428b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:60524/0 (socket says 172.21.15.22:60524) 2024-09-19T08:19:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.410+0000 7f22c928e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c4296370 0x5581c4298760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.411+0000 7f22c5452640 1 --2- 172.21.15.22:0/3361861445 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f22a80a3420 0x7f22a80a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.411+0000 7f22c928e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5581c41f12f0 con 0x5581c428b5d0 2024-09-19T08:19:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.411+0000 7f22c928e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5581c4198cf0 con 0x5581c4296370 2024-09-19T08:19:14.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.411+0000 7f22c5452640 1 --2- 172.21.15.22:0/3361861445 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f22a80a3420 0x7f22a80a58e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f22b80ed490 tx=0x7f22b802b430 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:14.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.412+0000 7f22c928e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5581c41ed2b0 con 0x5581c428cb40 2024-09-19T08:19:14.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.412+0000 7f22c4c51640 1 -- 172.21.15.22:0/4232902330 learned_addr learned my addr 172.21.15.22:0/4232902330 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:14.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.413+0000 7f22bffff640 1 --2- 172.21.15.22:0/4232902330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c4296370 0x5581c4298760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:14.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.413+0000 7f22bffff640 1 -- 172.21.15.22:0/4232902330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c4296370 msgr2=0x5581c4298760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:14.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.413+0000 7f22bffff640 1 --2- 172.21.15.22:0/4232902330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c4296370 0x5581c4298760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.413+0000 7f22bffff640 1 -- 172.21.15.22:0/4232902330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 msgr2=0x5581c428ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:14.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.413+0000 7f22bffff640 1 --2- 172.21.15.22:0/4232902330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c428ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.413+0000 7f22bffff640 1 -- 172.21.15.22:0/4232902330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581c417d3a0 con 0x5581c428cb40 2024-09-19T08:19:14.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.413+0000 7f22bffff640 1 --2- 172.21.15.22:0/4232902330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c428b090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f22c0113e60 tx=0x7f22b00093b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d54987b847c0868 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:14.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.414+0000 7f22c4c51640 1 --2- 172.21.15.22:0/4232902330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c428ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:14.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.414+0000 7f22a6ffd640 1 -- 172.21.15.22:0/4232902330 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b000bac0 con 0x5581c428cb40 2024-09-19T08:19:14.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.414+0000 7f22a6ffd640 1 -- 172.21.15.22:0/4232902330 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22b0010040 con 0x5581c428cb40 2024-09-19T08:19:14.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.414+0000 7f22a6ffd640 1 -- 172.21.15.22:0/4232902330 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b0003ca0 con 0x5581c428cb40 2024-09-19T08:19:14.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.415+0000 7f22c928e740 1 -- 172.21.15.22:0/4232902330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 msgr2=0x5581c428b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:14.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.415+0000 7f22c928e740 1 --2- 172.21.15.22:0/4232902330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c428b090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f22c0113e60 tx=0x7f22b00093b0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.415+0000 7f22c928e740 1 -- 172.21.15.22:0/4232902330 shutdown_connections 2024-09-19T08:19:14.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.415+0000 7f22c928e740 1 --2- 172.21.15.22:0/4232902330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c4296370 0x5581c4298760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.415+0000 7f22c928e740 1 --2- 172.21.15.22:0/4232902330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c428ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.415+0000 7f22c928e740 1 --2- 172.21.15.22:0/4232902330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c428b090 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.415+0000 7f22c928e740 1 -- 172.21.15.22:0/4232902330 >> 172.21.15.22:0/4232902330 conn(0x5581c4254310 msgr2=0x5581c4287c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.415+0000 7f22c928e740 1 -- 172.21.15.22:0/4232902330 shutdown_connections 2024-09-19T08:19:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.415+0000 7f22c928e740 1 -- 172.21.15.22:0/4232902330 wait complete. 2024-09-19T08:19:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.416+0000 7f22c928e740 1 -- start start 2024-09-19T08:19:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.416+0000 7f22c928e740 1 Processor -- start 2024-09-19T08:19:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.417+0000 7f22c928e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c4295d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.417+0000 7f22c928e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c428cb40 0x5581c4293060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.417+0000 7f22bffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c4295d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.417+0000 7f22bffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c4295d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40104/0 (socket says 172.21.15.22:40104) 2024-09-19T08:19:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.417+0000 7f22c928e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c4296370 0x5581c42936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.417+0000 7f22c928e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5581c4299070 con 0x5581c428b5d0 2024-09-19T08:19:14.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.417+0000 7f22bffff640 1 -- 172.21.15.22:0/1744815436 learned_addr learned my addr 172.21.15.22:0/1744815436 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:14.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.417+0000 7f22c928e740 1 -- 172.21.15.22:0/1744815436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5581c41ac060 con 0x5581c428cb40 2024-09-19T08:19:14.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.417+0000 7f22c928e740 1 -- 172.21.15.22:0/1744815436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5581c42991f0 con 0x5581c4296370 2024-09-19T08:19:14.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.418+0000 7f22c4c51640 1 --2- 172.21.15.22:0/1744815436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c428cb40 0x5581c4293060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:14.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.418+0000 7f22c4c51640 1 -- 172.21.15.22:0/1744815436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c4296370 msgr2=0x5581c42936c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:19:14.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.418+0000 7f22c4c51640 1 --2- 172.21.15.22:0/1744815436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c4296370 0x5581c42936c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.418+0000 7f22c4c51640 1 -- 172.21.15.22:0/1744815436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 msgr2=0x5581c4295d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:14.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.418+0000 7f22c4c51640 1 --2- 172.21.15.22:0/1744815436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c4295d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.418+0000 7f22c4c51640 1 -- 172.21.15.22:0/1744815436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581c3f623c0 con 0x5581c428cb40 2024-09-19T08:19:14.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.419+0000 7f22c4c51640 1 --2- 172.21.15.22:0/1744815436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c428cb40 0x5581c4293060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22b40314d0 tx=0x7f22b4031990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eff80b134b76fda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:14.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.419+0000 7f22c928e740 10 client.0 _ll_register_callbacks cb 0x5581c3eff8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:14.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.420+0000 7f22c928e740 1 -- 172.21.15.22:0/1744815436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22b0010040 con 0x5581c428cb40 2024-09-19T08:19:14.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.421+0000 7f229ffff640 10 client.25805 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:19:14.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.421+0000 7f229ffff640 1 -- 172.21.15.22:0/1744815436 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b4035660 con 0x5581c428cb40 2024-09-19T08:19:14.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.421+0000 7f229ffff640 1 -- 172.21.15.22:0/1744815436 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f22b4035800 con 0x5581c428cb40 2024-09-19T08:19:14.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.421+0000 7f229ffff640 1 -- 172.21.15.22:0/1744815436 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f22b4035a00 con 0x5581c428cb40 2024-09-19T08:19:14.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.421+0000 7f22c928e740 10 client.25805 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:14.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.421+0000 7f22c928e740 1 -- 172.21.15.22:0/1744815436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5581c42a4a60 con 0x5581c428cb40 2024-09-19T08:19:14.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.423+0000 7f229ffff640 1 -- 172.21.15.22:0/1744815436 <== mon.2 v2:172.21.15.64:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f22b4035ba0 con 0x5581c428cb40 2024-09-19T08:19:14.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.423+0000 7f22c928e740 10 client.25805 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:14.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.423+0000 7f22c928e740 10 client.25805 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:19:14.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.423+0000 7f22c928e740 20 client.25805 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:14.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.424+0000 7f22c928e740 1 --2- 172.21.15.22:0/1744815436 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5581c42a5e10 0x5581c42a8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.424+0000 7f22c928e740 4 client.25805 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:19:14.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.424+0000 7f22c928e740 1 -- 172.21.15.22:0/1744815436 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f22b0010040 con 0x5581c42a5e10 2024-09-19T08:19:14.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.424+0000 7f22bffff640 1 --2- 172.21.15.22:0/1744815436 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5581c42a5e10 0x5581c42a8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:14.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.425+0000 7f22bffff640 1 --2- 172.21.15.22:0/1744815436 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5581c42a5e10 0x5581c42a8200 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3249f068df65f03c server_cookie=4d3203aae856e636 in_seq=0 out_seq=0 2024-09-19T08:19:14.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.425+0000 7f229ffff640 10 client.25805 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:19:14.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.430+0000 7f229ffff640 1 -- 172.21.15.22:0/1744815436 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f22b0010040 con 0x5581c42a5e10 2024-09-19T08:19:14.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.430+0000 7f229ffff640 10 client.25805 handle_command_reply: tid=0 2024-09-19T08:19:14.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.431+0000 7f22c928e740 1 client.25805 shutdown 2024-09-19T08:19:14.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.431+0000 7f22c928e740 1 -- 172.21.15.22:0/1744815436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c428cb40 msgr2=0x5581c4293060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:14.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.431+0000 7f22c928e740 1 --2- 172.21.15.22:0/1744815436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c428cb40 0x5581c4293060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22b40314d0 tx=0x7f22b4031990 comp rx=0 tx=0).stop 2024-09-19T08:19:14.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.431+0000 7f22c928e740 1 -- 172.21.15.22:0/1744815436 shutdown_connections 2024-09-19T08:19:14.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.431+0000 7f22c928e740 1 --2- 172.21.15.22:0/1744815436 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5581c42a5e10 0x5581c42a8200 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.432+0000 7f22c928e740 1 --2- 172.21.15.22:0/1744815436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c4296370 0x5581c42936c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.432+0000 7f22c928e740 1 --2- 172.21.15.22:0/1744815436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c428cb40 0x5581c4293060 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.432+0000 7f22c928e740 1 --2- 172.21.15.22:0/1744815436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c4295d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.432+0000 7f22c928e740 1 -- 172.21.15.22:0/1744815436 >> 172.21.15.22:0/1744815436 conn(0x5581c4254310 msgr2=0x5581c428d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:14.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.432+0000 7f229ffff640 0 client.25805 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:19:14.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.432+0000 7f22c928e740 1 -- 172.21.15.22:0/1744815436 shutdown_connections 2024-09-19T08:19:14.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.432+0000 7f22c928e740 1 -- 172.21.15.22:0/1744815436 wait complete. 2024-09-19T08:19:14.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.433+0000 7f22c928e740 20 client.25805 trim_cache size 0 max 16384 2024-09-19T08:19:14.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.466+0000 7f22c928e740 1 Processor -- start 2024-09-19T08:19:14.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.466+0000 7f22c928e740 1 -- start start 2024-09-19T08:19:14.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.466+0000 7f22c928e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c4292020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.467+0000 7f22c928e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c4294fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.467+0000 7f22c928e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c4296370 0x5581c428e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.467+0000 7f22c928e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f22c007d360 con 0x5581c428b5d0 2024-09-19T08:19:14.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.467+0000 7f22c928e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f22c007abd0 con 0x5581c4296370 2024-09-19T08:19:14.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.467+0000 7f22c928e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f22c0078a20 con 0x5581c428cb40 2024-09-19T08:19:14.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.467+0000 7f22c4c51640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c4294fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:14.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.467+0000 7f22c4c51640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c4294fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:60544/0 (socket says 172.21.15.22:60544) 2024-09-19T08:19:14.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.467+0000 7f22c4c51640 1 -- 172.21.15.22:0/763575489 learned_addr learned my addr 172.21.15.22:0/763575489 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:14.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.468+0000 7f22c4c51640 1 -- 172.21.15.22:0/763575489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c4296370 msgr2=0x5581c428e120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:19:14.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.468+0000 7f22c4c51640 1 --2- 172.21.15.22:0/763575489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c4296370 0x5581c428e120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.468+0000 7f22c4c51640 1 -- 172.21.15.22:0/763575489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 msgr2=0x5581c4292020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:14.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.468+0000 7f22c4c51640 1 --2- 172.21.15.22:0/763575489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c4292020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.468+0000 7f22c4c51640 1 -- 172.21.15.22:0/763575489 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581c42960d0 con 0x5581c428cb40 2024-09-19T08:19:14.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.469+0000 7f22c4c51640 1 --2- 172.21.15.22:0/763575489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c4294fd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f22b400a1a0 tx=0x7f22b4036000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=863761b8a305fe0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:14.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.469+0000 7f22a67fc640 1 -- 172.21.15.22:0/763575489 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b4005990 con 0x5581c428cb40 2024-09-19T08:19:14.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.469+0000 7f22a67fc640 1 -- 172.21.15.22:0/763575489 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22b404c020 con 0x5581c428cb40 2024-09-19T08:19:14.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.469+0000 7f22a67fc640 1 -- 172.21.15.22:0/763575489 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b4005b30 con 0x5581c428cb40 2024-09-19T08:19:14.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.471+0000 7f22c928e740 1 -- 172.21.15.22:0/763575489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 msgr2=0x5581c4294fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:14.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.471+0000 7f22c928e740 1 --2- 172.21.15.22:0/763575489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c4294fd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f22b400a1a0 tx=0x7f22b4036000 comp rx=0 tx=0).stop 2024-09-19T08:19:14.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.474+0000 7f22c928e740 1 -- 172.21.15.22:0/763575489 shutdown_connections 2024-09-19T08:19:14.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.474+0000 7f22c928e740 1 --2- 172.21.15.22:0/763575489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c4296370 0x5581c428e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.474+0000 7f22c928e740 1 --2- 172.21.15.22:0/763575489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c4294fd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.474+0000 7f22c928e740 1 --2- 172.21.15.22:0/763575489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c4292020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.474+0000 7f22c928e740 1 -- 172.21.15.22:0/763575489 >> 172.21.15.22:0/763575489 conn(0x5581c4254310 msgr2=0x5581c42948c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:14.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.474+0000 7f22c928e740 1 -- 172.21.15.22:0/763575489 shutdown_connections 2024-09-19T08:19:14.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.475+0000 7f22c928e740 1 -- 172.21.15.22:0/763575489 wait complete. 2024-09-19T08:19:14.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.475+0000 7f22c928e740 1 -- start start 2024-09-19T08:19:14.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.475+0000 7f22c928e740 1 Processor -- start 2024-09-19T08:19:14.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.476+0000 7f22c928e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c429ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.476+0000 7f22c928e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c428fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.477+0000 7f22c928e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c4296370 0x5581c4290370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.477+0000 7f22c4c51640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c428fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:14.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.477+0000 7f22c4c51640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c428fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:60554/0 (socket says 172.21.15.22:60554) 2024-09-19T08:19:14.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.477+0000 7f22c4c51640 1 -- 172.21.15.22:0/1878757007 learned_addr learned my addr 172.21.15.22:0/1878757007 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:14.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.477+0000 7f22c928e740 1 -- 172.21.15.22:0/1878757007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f22c0079090 con 0x5581c428b5d0 2024-09-19T08:19:14.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.477+0000 7f22bffff640 1 --2- 172.21.15.22:0/1878757007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c4296370 0x5581c4290370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:14.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.477+0000 7f22c928e740 1 -- 172.21.15.22:0/1878757007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f22c007a500 con 0x5581c4296370 2024-09-19T08:19:14.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.477+0000 7f22c928e740 1 -- 172.21.15.22:0/1878757007 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f22c0079760 con 0x5581c428cb40 2024-09-19T08:19:14.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.478+0000 7f22c928e740 10 client.0 _ll_register_callbacks cb 0x7f22b400a4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:14.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.478+0000 7f22c4c51640 1 -- 172.21.15.22:0/1878757007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c4296370 msgr2=0x5581c4290370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:14.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.478+0000 7f22c4c51640 1 --2- 172.21.15.22:0/1878757007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c4296370 0x5581c4290370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.478+0000 7f22c4c51640 1 -- 172.21.15.22:0/1878757007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 msgr2=0x5581c429ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:14.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.478+0000 7f22c4c51640 1 --2- 172.21.15.22:0/1878757007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c429ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.478+0000 7f22c4c51640 1 -- 172.21.15.22:0/1878757007 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581c4290ba0 con 0x5581c428cb40 2024-09-19T08:19:14.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.479+0000 7f22c928e740 1 -- 172.21.15.22:0/1878757007 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22b404c020 con 0x5581c428cb40 2024-09-19T08:19:14.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.479+0000 7f22c4c51640 1 --2- 172.21.15.22:0/1878757007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c428fe10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f22b4046fd0 tx=0x7f22b4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc49c03a2ab35555 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:14.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.479+0000 7f229ffff640 10 client.25747 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:19:14.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.480+0000 7f229ffff640 1 -- 172.21.15.22:0/1878757007 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b4031e90 con 0x5581c428cb40 2024-09-19T08:19:14.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.481+0000 7f229ffff640 1 -- 172.21.15.22:0/1878757007 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f22b4030050 con 0x5581c428cb40 2024-09-19T08:19:14.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.481+0000 7f229ffff640 1 -- 172.21.15.22:0/1878757007 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f22b400ad30 con 0x5581c428cb40 2024-09-19T08:19:14.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.481+0000 7f22c928e740 10 client.25747 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:14.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.481+0000 7f22c928e740 1 -- 172.21.15.22:0/1878757007 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5581c43109d0 con 0x5581c428cb40 2024-09-19T08:19:14.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.482+0000 7f229ffff640 1 -- 172.21.15.22:0/1878757007 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f22b4005990 con 0x5581c428cb40 2024-09-19T08:19:14.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.482+0000 7f22c928e740 10 client.25747 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:14.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.482+0000 7f22c928e740 10 client.25747 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:19:14.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.482+0000 7f22c928e740 20 client.25747 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:14.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.482+0000 7f22c928e740 1 --2- 172.21.15.22:0/1878757007 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5581c4310f10 0x5581c4313330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:14.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.483+0000 7f22c928e740 4 client.25747 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:19:14.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.483+0000 7f22c928e740 1 -- 172.21.15.22:0/1878757007 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581c41fe860 con 0x5581c4310f10 2024-09-19T08:19:14.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.483+0000 7f22bffff640 1 --2- 172.21.15.22:0/1878757007 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5581c4310f10 0x5581c4313330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:14.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.483+0000 7f22bffff640 1 --2- 172.21.15.22:0/1878757007 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5581c4310f10 0x5581c4313330 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bc5ece42f7180d2 server_cookie=c4dca2204ec097ab in_seq=0 out_seq=0 2024-09-19T08:19:14.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.483+0000 7f229ffff640 10 client.25747 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:19:14.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.486+0000 7f229ffff640 1 -- 172.21.15.22:0/1878757007 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581c41fe860 con 0x5581c4310f10 2024-09-19T08:19:14.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.486+0000 7f229ffff640 10 client.25747 handle_command_reply: tid=0 2024-09-19T08:19:14.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.486+0000 7f22c928e740 1 client.25747 shutdown 2024-09-19T08:19:14.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.487+0000 7f22c928e740 1 -- 172.21.15.22:0/1878757007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 msgr2=0x5581c428fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:14.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.487+0000 7f22c928e740 1 --2- 172.21.15.22:0/1878757007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c428fe10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f22b4046fd0 tx=0x7f22b4000bc0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.489+0000 7f22c928e740 1 -- 172.21.15.22:0/1878757007 shutdown_connections 2024-09-19T08:19:14.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.489+0000 7f22c928e740 1 --2- 172.21.15.22:0/1878757007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c4296370 0x5581c4290370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.489+0000 7f22c928e740 1 --2- 172.21.15.22:0/1878757007 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5581c4310f10 0x5581c4313330 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.490+0000 7f22c928e740 1 --2- 172.21.15.22:0/1878757007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581c428cb40 0x5581c428fe10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.490+0000 7f22c928e740 1 --2- 172.21.15.22:0/1878757007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581c428b5d0 0x5581c429ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.490+0000 7f22c928e740 1 -- 172.21.15.22:0/1878757007 >> 172.21.15.22:0/1878757007 conn(0x5581c4254310 msgr2=0x5581c42950e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:14.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.490+0000 7f229ffff640 0 client.25747 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:19:14.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.490+0000 7f22c928e740 1 -- 172.21.15.22:0/1878757007 shutdown_connections 2024-09-19T08:19:14.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.490+0000 7f22c928e740 1 -- 172.21.15.22:0/1878757007 wait complete. 2024-09-19T08:19:14.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.491+0000 7f22c928e740 20 client.25747 trim_cache size 0 max 16384 2024-09-19T08:19:14.491 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:19:14.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.492+0000 7f22c6eb5640 1 -- 172.21.15.22:0/3361861445 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f22a80a3420 msgr2=0x7f22a80a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:14.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.492+0000 7f22c6eb5640 1 --2- 172.21.15.22:0/3361861445 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f22a80a3420 0x7f22a80a58e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f22b80ed490 tx=0x7f22b802b430 comp rx=0 tx=0).stop 2024-09-19T08:19:14.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.492+0000 7f22c6eb5640 1 -- 172.21.15.22:0/3361861445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22c010ea80 msgr2=0x7f22c01131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:14.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.492+0000 7f22c6eb5640 1 --2- 172.21.15.22:0/3361861445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22c010ea80 0x7f22c01131c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f22b800a300 tx=0x7f22b800a330 comp rx=0 tx=0).stop 2024-09-19T08:19:14.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.492+0000 7f22c6eb5640 1 -- 172.21.15.22:0/3361861445 shutdown_connections 2024-09-19T08:19:14.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.492+0000 7f22c6eb5640 1 --2- 172.21.15.22:0/3361861445 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f22a80a3420 0x7f22a80a58e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.492+0000 7f22c6eb5640 1 --2- 172.21.15.22:0/3361861445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22c010ea80 0x7f22c01131c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.492+0000 7f22c6eb5640 1 --2- 172.21.15.22:0/3361861445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c0109e00 0x7f22c0112c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.492+0000 7f22c6eb5640 1 --2- 172.21.15.22:0/3361861445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22c0109450 0x7f22c0112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:14.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.492+0000 7f22c6eb5640 1 -- 172.21.15.22:0/3361861445 >> 172.21.15.22:0/3361861445 conn(0x7f22c006e2d0 msgr2=0x7f22c0072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:14.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.492+0000 7f22c6eb5640 1 -- 172.21.15.22:0/3361861445 shutdown_connections 2024-09-19T08:19:14.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:14.492+0000 7f22c6eb5640 1 -- 172.21.15.22:0/3361861445 wait complete. 2024-09-19T08:19:14.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:14 smithi157 ceph-mon[35143]: pgmap v129: 97 pgs: 97 active+clean; 504 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.0 MiB/s wr, 355 op/s 2024-09-19T08:19:14.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:14 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:19:14.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:14 smithi022 ceph-mon[30963]: pgmap v129: 97 pgs: 97 active+clean; 504 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.0 MiB/s wr, 355 op/s 2024-09-19T08:19:14.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:14 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:19:14.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:14 smithi064 ceph-mon[34900]: pgmap v129: 97 pgs: 97 active+clean; 504 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.0 MiB/s wr, 355 op/s 2024-09-19T08:19:14.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:14 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:19:15.307 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:19:15.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:15 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:19:15.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:15 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:19:15.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:15 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:19:15.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:15 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:19:15.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:15 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:19:15.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:15 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:19:15.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:15 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:19:15.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:15 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:19:15.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:15 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:19:16.155 INFO:tasks.workunit.client.0.smithi022.stdout: 97 1914922 448 1190374 717 991474 1654 2024-09-19T08:19:17.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:16 smithi157 ceph-mon[35143]: pgmap v130: 97 pgs: 97 active+clean; 504 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 134 op/s 2024-09-19T08:19:17.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:16 smithi022 ceph-mon[30963]: pgmap v130: 97 pgs: 97 active+clean; 504 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 134 op/s 2024-09-19T08:19:17.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:16 smithi064 ceph-mon[34900]: pgmap v130: 97 pgs: 97 active+clean; 504 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 134 op/s 2024-09-19T08:19:17.495 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:19:18.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.512+0000 7f5c2ffff640 1 -- 172.21.15.22:0/4245696772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c3006ea30 msgr2=0x7f5c3006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:18.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.512+0000 7f5c2ffff640 1 --2- 172.21.15.22:0/4245696772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c3006ea30 0x7f5c3006ee70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009be0 tx=0x7f5c2802a710 comp rx=0 tx=0).stop 2024-09-19T08:19:18.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.515+0000 7f5c2ffff640 1 -- 172.21.15.22:0/4245696772 shutdown_connections 2024-09-19T08:19:18.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.515+0000 7f5c2ffff640 1 --2- 172.21.15.22:0/4245696772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c3011b140 0x7f5c3011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.515+0000 7f5c2ffff640 1 --2- 172.21.15.22:0/4245696772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c3006ea30 0x7f5c3006ee70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.515+0000 7f5c2ffff640 1 --2- 172.21.15.22:0/4245696772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c30110190 0x7f5c3006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.515+0000 7f5c2ffff640 1 -- 172.21.15.22:0/4245696772 >> 172.21.15.22:0/4245696772 conn(0x7f5c30073180 msgr2=0x7f5c30073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:18.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.523+0000 7f5c2ffff640 1 -- 172.21.15.22:0/4245696772 shutdown_connections 2024-09-19T08:19:18.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.523+0000 7f5c2ffff640 1 -- 172.21.15.22:0/4245696772 wait complete. 2024-09-19T08:19:18.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.524+0000 7f5c2ffff640 1 Processor -- start 2024-09-19T08:19:18.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.524+0000 7f5c2ffff640 1 -- start start 2024-09-19T08:19:18.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.524+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c3006ea30 0x7f5c3011a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.524+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30110190 0x7f5c301169f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.524+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c3011b140 0x7f5c30116f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.525+0000 7f5c2ffff640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5c30113300 con 0x7f5c3011b140 2024-09-19T08:19:18.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.525+0000 7f5c2ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c30113180 con 0x7f5c3006ea30 2024-09-19T08:19:18.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.525+0000 7f5c2ffff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5c30113480 con 0x7f5c30110190 2024-09-19T08:19:18.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.525+0000 7f5c2e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30110190 0x7f5c301169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:18.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.525+0000 7f5c2f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c3011b140 0x7f5c30116f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:18.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.525+0000 7f5c2e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30110190 0x7f5c301169f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:55992/0 (socket says 172.21.15.22:55992) 2024-09-19T08:19:18.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.525+0000 7f5c2e7fc640 1 -- 172.21.15.22:0/2276837919 learned_addr learned my addr 172.21.15.22:0/2276837919 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:18.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.526+0000 7f5c2effd640 1 --2- 172.21.15.22:0/2276837919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c3006ea30 0x7f5c3011a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:18.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.526+0000 7f5c2f7fe640 1 -- 172.21.15.22:0/2276837919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30110190 msgr2=0x7f5c301169f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:18.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.526+0000 7f5c2f7fe640 1 --2- 172.21.15.22:0/2276837919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30110190 0x7f5c301169f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.526+0000 7f5c2f7fe640 1 -- 172.21.15.22:0/2276837919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c3006ea30 msgr2=0x7f5c3011a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:18.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.526+0000 7f5c2f7fe640 1 --2- 172.21.15.22:0/2276837919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c3006ea30 0x7f5c3011a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.526+0000 7f5c2f7fe640 1 -- 172.21.15.22:0/2276837919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c301177f0 con 0x7f5c3011b140 2024-09-19T08:19:18.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.527+0000 7f5c2f7fe640 1 --2- 172.21.15.22:0/2276837919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c3011b140 0x7f5c30116f30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5c3006b4a0 tx=0x7f5c2400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:18.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.528+0000 7f5c17fff640 1 -- 172.21.15.22:0/2276837919 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24017070 con 0x7f5c3011b140 2024-09-19T08:19:18.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.528+0000 7f5c2ffff640 1 -- 172.21.15.22:0/2276837919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c301aaf60 con 0x7f5c3011b140 2024-09-19T08:19:18.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.528+0000 7f5c2ffff640 1 -- 172.21.15.22:0/2276837919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c301ab360 con 0x7f5c3011b140 2024-09-19T08:19:18.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.532+0000 7f5c37afc740 1 Processor -- start 2024-09-19T08:19:18.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.532+0000 7f5c17fff640 1 -- 172.21.15.22:0/2276837919 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c24004220 con 0x7f5c3011b140 2024-09-19T08:19:18.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.533+0000 7f5c17fff640 1 -- 172.21.15.22:0/2276837919 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24003be0 con 0x7f5c3011b140 2024-09-19T08:19:18.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.536+0000 7f5c17fff640 1 -- 172.21.15.22:0/2276837919 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5c24010460 con 0x7f5c3011b140 2024-09-19T08:19:18.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.536+0000 7f5c37afc740 1 -- start start 2024-09-19T08:19:18.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.537+0000 7f5c37afc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824dd60 0x5618d824c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.537+0000 7f5c37afc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 0x5618d824ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.537+0000 7f5c37afc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d8259820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.537+0000 7f5c37afc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5618d81b22f0 con 0x5618d824dd60 2024-09-19T08:19:18.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.537+0000 7f5c37afc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5618d8159cf0 con 0x5618d824d090 2024-09-19T08:19:18.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.537+0000 7f5c37afc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5618d81ae2b0 con 0x5618d824e1d0 2024-09-19T08:19:18.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.537+0000 7f5c2e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d8259820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:18.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.537+0000 7f5c2e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d8259820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:35980/0 (socket says 172.21.15.22:35980) 2024-09-19T08:19:18.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.537+0000 7f5c2e7fc640 1 -- 172.21.15.22:0/2755202183 learned_addr learned my addr 172.21.15.22:0/2755202183 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:18.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.541+0000 7f5c2e7fc640 1 --2- 172.21.15.22:0/2755202183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824dd60 0x5618d824c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.541+0000 7f5c2e7fc640 1 -- 172.21.15.22:0/2755202183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 msgr2=0x5618d824ca70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:19:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.541+0000 7f5c2e7fc640 1 --2- 172.21.15.22:0/2755202183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 0x5618d824ca70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.541+0000 7f5c2e7fc640 1 -- 172.21.15.22:0/2755202183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824dd60 msgr2=0x5618d824c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.541+0000 7f5c2e7fc640 1 --2- 172.21.15.22:0/2755202183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824dd60 0x5618d824c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.541+0000 7f5c2e7fc640 1 -- 172.21.15.22:0/2755202183 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d813e3a0 con 0x5618d824d090 2024-09-19T08:19:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.542+0000 7f5c2e7fc640 1 --2- 172.21.15.22:0/2755202183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d8259820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5c2802ad00 tx=0x7f5c28005e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cbc0c9e88d32da1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.543+0000 7f5c17fff640 1 --2- 172.21.15.22:0/2276837919 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5c180a3540 0x7f5c180a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.544+0000 7f5c17fff640 1 -- 172.21.15.22:0/2276837919 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5c240cf130 con 0x7f5c3011b140 2024-09-19T08:19:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.544+0000 7f5c2ffff640 1 -- 172.21.15.22:0/2755202183 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28044070 con 0x5618d824d090 2024-09-19T08:19:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.544+0000 7f5c2ffff640 1 -- 172.21.15.22:0/2755202183 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c28003e50 con 0x5618d824d090 2024-09-19T08:19:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.545+0000 7f5c37afc740 1 -- 172.21.15.22:0/2755202183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 msgr2=0x5618d8259820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.545+0000 7f5c37afc740 1 --2- 172.21.15.22:0/2755202183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d8259820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5c2802ad00 tx=0x7f5c28005e90 comp rx=0 tx=0).stop 2024-09-19T08:19:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.548+0000 7f5c2effd640 1 --2- 172.21.15.22:0/2276837919 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5c180a3540 0x7f5c180a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.548+0000 7f5c37afc740 1 -- 172.21.15.22:0/2755202183 shutdown_connections 2024-09-19T08:19:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.548+0000 7f5c37afc740 1 --2- 172.21.15.22:0/2755202183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d8259820 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.548+0000 7f5c37afc740 1 --2- 172.21.15.22:0/2755202183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 0x5618d824ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.548+0000 7f5c37afc740 1 --2- 172.21.15.22:0/2755202183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824dd60 0x5618d824c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.548+0000 7f5c37afc740 1 -- 172.21.15.22:0/2755202183 >> 172.21.15.22:0/2755202183 conn(0x5618d8215310 msgr2=0x5618d8248ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:18.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.552+0000 7f5c37afc740 1 -- 172.21.15.22:0/2755202183 shutdown_connections 2024-09-19T08:19:18.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.552+0000 7f5c2effd640 1 --2- 172.21.15.22:0/2276837919 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5c180a3540 0x7f5c180a5a00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5618d8249f60 tx=0x7f5c2001a270 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:18.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.553+0000 7f5c37afc740 1 -- 172.21.15.22:0/2755202183 wait complete. 2024-09-19T08:19:18.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.553+0000 7f5c37afc740 1 -- start start 2024-09-19T08:19:18.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.554+0000 7f5c37afc740 1 Processor -- start 2024-09-19T08:19:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.554+0000 7f5c37afc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d8259710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.554+0000 7f5c37afc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824dd60 0x5618d82569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.554+0000 7f5c37afc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 0x5618d8256f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.554+0000 7f5c37afc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5618d825a240 con 0x5618d824dd60 2024-09-19T08:19:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.554+0000 7f5c37afc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5618d816d060 con 0x5618d824d090 2024-09-19T08:19:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.554+0000 7f5c37afc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5618d825a3c0 con 0x5618d824e1d0 2024-09-19T08:19:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.554+0000 7f5c37afc740 10 client.0 _ll_register_callbacks cb 0x5618d7ec08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.554+0000 7f5c2e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824dd60 0x5618d82569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.554+0000 7f5c2e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824dd60 0x5618d82569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47642/0 (socket says 172.21.15.22:47642) 2024-09-19T08:19:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.555+0000 7f5c2e7fc640 1 -- 172.21.15.22:0/193588320 learned_addr learned my addr 172.21.15.22:0/193588320 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.555+0000 7f5c2e7fc640 1 -- 172.21.15.22:0/193588320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 msgr2=0x5618d8256f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:19:18.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.555+0000 7f5c2e7fc640 1 --2- 172.21.15.22:0/193588320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 0x5618d8256f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.555+0000 7f5c2e7fc640 1 -- 172.21.15.22:0/193588320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 msgr2=0x5618d8259710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:19:18.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.555+0000 7f5c2e7fc640 1 --2- 172.21.15.22:0/193588320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d8259710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.555+0000 7f5c2e7fc640 1 -- 172.21.15.22:0/193588320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d7f233c0 con 0x5618d824dd60 2024-09-19T08:19:18.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.555+0000 7f5c2e7fc640 1 --2- 172.21.15.22:0/193588320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824dd60 0x5618d82569c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5c28004af0 tx=0x7f5c28005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc95a07d146def0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:18.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.556+0000 7f5c12ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:19:18.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.556+0000 7f5c12ff5640 1 -- 172.21.15.22:0/193588320 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28050020 con 0x5618d824dd60 2024-09-19T08:19:18.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.556+0000 7f5c37afc740 1 -- 172.21.15.22:0/193588320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5618d81bf860 con 0x5618d824dd60 2024-09-19T08:19:18.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.556+0000 7f5c12ff5640 1 -- 172.21.15.22:0/193588320 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5c28003970 con 0x5618d824dd60 2024-09-19T08:19:18.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.556+0000 7f5c12ff5640 1 -- 172.21.15.22:0/193588320 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c2805b050 con 0x5618d824dd60 2024-09-19T08:19:18.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.560+0000 7f5c37afc740 10 client.16221 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.560+0000 7f5c37afc740 1 -- 172.21.15.22:0/193588320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5618d8265e00 con 0x5618d824dd60 2024-09-19T08:19:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.560+0000 7f5c12ff5640 1 -- 172.21.15.22:0/193588320 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5618d81bf860 con 0x5618d824dd60 2024-09-19T08:19:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.560+0000 7f5c12ff5640 1 -- 172.21.15.22:0/193588320 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5c2803c040 con 0x5618d824dd60 2024-09-19T08:19:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.562+0000 7f5c37afc740 10 client.16221 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.562+0000 7f5c37afc740 10 client.16221 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:19:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.562+0000 7f5c37afc740 20 client.16221 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.562+0000 7f5c37afc740 1 --2- 172.21.15.22:0/193588320 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x5618d8267350 0x5618d8269810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.562+0000 7f5c37afc740 4 client.16221 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:19:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.562+0000 7f5c37afc740 1 -- 172.21.15.22:0/193588320 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618d81b9d70 con 0x5618d8267350 2024-09-19T08:19:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.568+0000 7f5c2effd640 1 --2- 172.21.15.22:0/193588320 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x5618d8267350 0x5618d8269810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.575+0000 7f5c2effd640 1 --2- 172.21.15.22:0/193588320 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x5618d8267350 0x5618d8269810 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58f691f663f8b69b server_cookie=7a724fc4e3dea3ca in_seq=0 out_seq=0 2024-09-19T08:19:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.575+0000 7f5c12ff5640 10 client.16221 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:19:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.578+0000 7f5c12ff5640 1 -- 172.21.15.22:0/193588320 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618d81b9d70 con 0x5618d8267350 2024-09-19T08:19:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.578+0000 7f5c12ff5640 10 client.16221 handle_command_reply: tid=0 2024-09-19T08:19:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.578+0000 7f5c37afc740 1 client.16221 shutdown 2024-09-19T08:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.578+0000 7f5c37afc740 1 -- 172.21.15.22:0/193588320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824dd60 msgr2=0x5618d82569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.578+0000 7f5c37afc740 1 --2- 172.21.15.22:0/193588320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824dd60 0x5618d82569c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5c28004af0 tx=0x7f5c28005bd0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.578+0000 7f5c37afc740 1 -- 172.21.15.22:0/193588320 shutdown_connections 2024-09-19T08:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.578+0000 7f5c37afc740 1 --2- 172.21.15.22:0/193588320 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x5618d8267350 0x5618d8269810 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.578+0000 7f5c37afc740 1 --2- 172.21.15.22:0/193588320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 0x5618d8256f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.579+0000 7f5c37afc740 1 --2- 172.21.15.22:0/193588320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824dd60 0x5618d82569c0 secure :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5c28004af0 tx=0x7f5c28005bd0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.579+0000 7f5c37afc740 1 --2- 172.21.15.22:0/193588320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d8259710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.579+0000 7f5c37afc740 1 -- 172.21.15.22:0/193588320 >> 172.21.15.22:0/193588320 conn(0x5618d8215310 msgr2=0x5618d824ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.579+0000 7f5c12ff5640 0 client.16221 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:19:18.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.582+0000 7f5c37afc740 1 -- 172.21.15.22:0/193588320 shutdown_connections 2024-09-19T08:19:18.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.586+0000 7f5c37afc740 1 -- 172.21.15.22:0/193588320 wait complete. 2024-09-19T08:19:18.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.587+0000 7f5c37afc740 20 client.16221 trim_cache size 0 max 16384 2024-09-19T08:19:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.628+0000 7f5c37afc740 1 Processor -- start 2024-09-19T08:19:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.629+0000 7f5c37afc740 1 -- start start 2024-09-19T08:19:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.629+0000 7f5c37afc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824d090 0x5618d82635c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.629+0000 7f5c37afc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824e1d0 0x5618d826f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.629+0000 7f5c37afc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d826f660 0x5618d825f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.629+0000 7f5c37afc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c0111e0 con 0x5618d824d090 2024-09-19T08:19:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.629+0000 7f5c37afc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c00ea50 con 0x5618d824e1d0 2024-09-19T08:19:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.629+0000 7f5c37afc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c00c8a0 con 0x5618d826f660 2024-09-19T08:19:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.630+0000 7f5c2f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d826f660 0x5618d825f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.630+0000 7f5c2f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d826f660 0x5618d825f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:56008/0 (socket says 172.21.15.22:56008) 2024-09-19T08:19:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.630+0000 7f5c2f7fe640 1 -- 172.21.15.22:0/801362007 learned_addr learned my addr 172.21.15.22:0/801362007 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:18.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.630+0000 7f5c2effd640 1 --2- 172.21.15.22:0/801362007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824d090 0x5618d82635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:18.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.631+0000 7f5c2effd640 1 -- 172.21.15.22:0/801362007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d826f660 msgr2=0x5618d825f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:18.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.631+0000 7f5c2effd640 1 --2- 172.21.15.22:0/801362007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d826f660 0x5618d825f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.631+0000 7f5c2effd640 1 -- 172.21.15.22:0/801362007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824e1d0 msgr2=0x5618d826f120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:19:18.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.631+0000 7f5c2effd640 1 --2- 172.21.15.22:0/801362007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824e1d0 0x5618d826f120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.631+0000 7f5c2effd640 1 -- 172.21.15.22:0/801362007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d82572d0 con 0x5618d824d090 2024-09-19T08:19:18.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.631+0000 7f5c2effd640 1 --2- 172.21.15.22:0/801362007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824d090 0x5618d82635c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x5618d8269e10 tx=0x7f5c20025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a75eb33bd72584c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.634+0000 7f5c14ff9640 1 -- 172.21.15.22:0/801362007 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2003c070 con 0x5618d824d090 2024-09-19T08:19:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.634+0000 7f5c14ff9640 1 -- 172.21.15.22:0/801362007 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c20003c40 con 0x5618d824d090 2024-09-19T08:19:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.635+0000 7f5c37afc740 1 -- 172.21.15.22:0/801362007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824d090 msgr2=0x5618d82635c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.635+0000 7f5c37afc740 1 --2- 172.21.15.22:0/801362007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824d090 0x5618d82635c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x5618d8269e10 tx=0x7f5c20025000 comp rx=0 tx=0).stop 2024-09-19T08:19:18.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.638+0000 7f5c37afc740 1 -- 172.21.15.22:0/801362007 shutdown_connections 2024-09-19T08:19:18.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.638+0000 7f5c37afc740 1 --2- 172.21.15.22:0/801362007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d826f660 0x5618d825f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.638+0000 7f5c37afc740 1 --2- 172.21.15.22:0/801362007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824e1d0 0x5618d826f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.638+0000 7f5c37afc740 1 --2- 172.21.15.22:0/801362007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d824d090 0x5618d82635c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.638+0000 7f5c37afc740 1 -- 172.21.15.22:0/801362007 >> 172.21.15.22:0/801362007 conn(0x5618d8215310 msgr2=0x5618d8255ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:18.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.644+0000 7f5c37afc740 1 -- 172.21.15.22:0/801362007 shutdown_connections 2024-09-19T08:19:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.646+0000 7f5c37afc740 1 -- 172.21.15.22:0/801362007 wait complete. 2024-09-19T08:19:18.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.646+0000 7f5c37afc740 1 -- start start 2024-09-19T08:19:18.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.646+0000 7f5c37afc740 1 Processor -- start 2024-09-19T08:19:18.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.650+0000 7f5c37afc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d825fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.650+0000 7f5c37afc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 0x5618d825d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.650+0000 7f5c2e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 0x5618d825d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:18.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.650+0000 7f5c2e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 0x5618d825d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:56018/0 (socket says 172.21.15.22:56018) 2024-09-19T08:19:18.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.650+0000 7f5c2effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d825fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:18.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.650+0000 7f5c2effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d825fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:36020/0 (socket says 172.21.15.22:36020) 2024-09-19T08:19:18.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.650+0000 7f5c2e7fc640 1 -- 172.21.15.22:0/3685225525 learned_addr learned my addr 172.21.15.22:0/3685225525 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:18.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.650+0000 7f5c37afc740 1 --2- 172.21.15.22:0/3685225525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d826f660 0x5618d825d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.650+0000 7f5c37afc740 1 -- 172.21.15.22:0/3685225525 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c00cf10 con 0x5618d826f660 2024-09-19T08:19:18.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.650+0000 7f5c37afc740 1 -- 172.21.15.22:0/3685225525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c00e380 con 0x5618d824d090 2024-09-19T08:19:18.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.650+0000 7f5c37afc740 1 -- 172.21.15.22:0/3685225525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c00d5e0 con 0x5618d824e1d0 2024-09-19T08:19:18.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.651+0000 7f5c2e7fc640 1 -- 172.21.15.22:0/3685225525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 msgr2=0x5618d825fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:18.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.651+0000 7f5c2e7fc640 1 --2- 172.21.15.22:0/3685225525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d825fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.651+0000 7f5c2e7fc640 1 -- 172.21.15.22:0/3685225525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d826f660 msgr2=0x5618d825d600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:19:18.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.651+0000 7f5c2e7fc640 1 --2- 172.21.15.22:0/3685225525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d826f660 0x5618d825d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.651+0000 7f5c2e7fc640 1 -- 172.21.15.22:0/3685225525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d825dcf0 con 0x5618d824e1d0 2024-09-19T08:19:18.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.651+0000 7f5c2effd640 1 --2- 172.21.15.22:0/3685225525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d825fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:19:18.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.651+0000 7f5c2e7fc640 1 --2- 172.21.15.22:0/3685225525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 0x5618d825d0c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c28034040 tx=0x7f5c2803e410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11474059f486fb64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:18.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.651+0000 7f5c37afc740 10 client.0 _ll_register_callbacks cb 0x7f5c28009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:18.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.651+0000 7f5c37afc740 1 -- 172.21.15.22:0/3685225525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c20003c40 con 0x5618d824e1d0 2024-09-19T08:19:18.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.653+0000 7f5c12ff5640 10 client.25759 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:19:18.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.653+0000 7f5c12ff5640 1 -- 172.21.15.22:0/3685225525 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28050040 con 0x5618d824e1d0 2024-09-19T08:19:18.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.653+0000 7f5c12ff5640 1 -- 172.21.15.22:0/3685225525 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5c2803e7e0 con 0x5618d824e1d0 2024-09-19T08:19:18.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.653+0000 7f5c12ff5640 1 -- 172.21.15.22:0/3685225525 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c28044040 con 0x5618d824e1d0 2024-09-19T08:19:18.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.653+0000 7f5c37afc740 10 client.25759 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:18.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.653+0000 7f5c37afc740 1 -- 172.21.15.22:0/3685225525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5618d82f3c60 con 0x5618d824e1d0 2024-09-19T08:19:18.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.656+0000 7f5c12ff5640 1 -- 172.21.15.22:0/3685225525 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5c2803e9e0 con 0x5618d824e1d0 2024-09-19T08:19:18.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.656+0000 7f5c37afc740 10 client.25759 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:18.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.656+0000 7f5c37afc740 10 client.25759 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:19:18.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.656+0000 7f5c37afc740 20 client.25759 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:18.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.656+0000 7f5c37afc740 1 --2- 172.21.15.22:0/3685225525 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x5618d82f4300 0x5618d82f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:18.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.656+0000 7f5c37afc740 4 client.25759 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:19:18.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.656+0000 7f5c37afc740 1 -- 172.21.15.22:0/3685225525 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5c20003c40 con 0x5618d82f4300 2024-09-19T08:19:18.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.658+0000 7f5c2effd640 1 --2- 172.21.15.22:0/3685225525 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x5618d82f4300 0x5618d82f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:18.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.661+0000 7f5c2effd640 1 --2- 172.21.15.22:0/3685225525 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x5618d82f4300 0x5618d82f67c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=334eadd117b0e957 server_cookie=8ca19ab458e37179 in_seq=0 out_seq=0 2024-09-19T08:19:18.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.662+0000 7f5c12ff5640 10 client.25759 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:19:18.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.664+0000 7f5c12ff5640 1 -- 172.21.15.22:0/3685225525 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5c20003c40 con 0x5618d82f4300 2024-09-19T08:19:18.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.664+0000 7f5c12ff5640 10 client.25759 handle_command_reply: tid=0 2024-09-19T08:19:18.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.664+0000 7f5c37afc740 1 client.25759 shutdown 2024-09-19T08:19:18.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.666+0000 7f5c37afc740 1 -- 172.21.15.22:0/3685225525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 msgr2=0x5618d825d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:18.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.666+0000 7f5c37afc740 1 --2- 172.21.15.22:0/3685225525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 0x5618d825d0c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c28034040 tx=0x7f5c2803e410 comp rx=0 tx=0).stop 2024-09-19T08:19:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.671+0000 7f5c37afc740 1 -- 172.21.15.22:0/3685225525 shutdown_connections 2024-09-19T08:19:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.671+0000 7f5c37afc740 1 --2- 172.21.15.22:0/3685225525 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x5618d82f4300 0x5618d82f67c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.671+0000 7f5c37afc740 1 --2- 172.21.15.22:0/3685225525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d826f660 0x5618d825d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.671+0000 7f5c37afc740 1 --2- 172.21.15.22:0/3685225525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618d824e1d0 0x5618d825d0c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.671+0000 7f5c37afc740 1 --2- 172.21.15.22:0/3685225525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5618d824d090 0x5618d825fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.671+0000 7f5c12ff5640 0 client.25759 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:19:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.671+0000 7f5c37afc740 1 -- 172.21.15.22:0/3685225525 >> 172.21.15.22:0/3685225525 conn(0x5618d8215310 msgr2=0x5618d8258220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:18.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.676+0000 7f5c37afc740 1 -- 172.21.15.22:0/3685225525 shutdown_connections 2024-09-19T08:19:18.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.677+0000 7f5c37afc740 1 -- 172.21.15.22:0/3685225525 wait complete. 2024-09-19T08:19:18.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.677+0000 7f5c37afc740 20 client.25759 trim_cache size 0 max 16384 2024-09-19T08:19:18.680 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:19:18.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.683+0000 7f5c15ffb640 1 -- 172.21.15.22:0/2276837919 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5c180a3540 msgr2=0x7f5c180a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:18.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.683+0000 7f5c15ffb640 1 --2- 172.21.15.22:0/2276837919 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5c180a3540 0x7f5c180a5a00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5618d8249f60 tx=0x7f5c2001a270 comp rx=0 tx=0).stop 2024-09-19T08:19:18.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.683+0000 7f5c15ffb640 1 -- 172.21.15.22:0/2276837919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c3011b140 msgr2=0x7f5c30116f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:18.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.683+0000 7f5c15ffb640 1 --2- 172.21.15.22:0/2276837919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c3011b140 0x7f5c30116f30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5c3006b4a0 tx=0x7f5c2400d9d0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.685+0000 7f5c15ffb640 1 -- 172.21.15.22:0/2276837919 shutdown_connections 2024-09-19T08:19:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.685+0000 7f5c15ffb640 1 --2- 172.21.15.22:0/2276837919 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5c180a3540 0x7f5c180a5a00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.685+0000 7f5c15ffb640 1 --2- 172.21.15.22:0/2276837919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c3011b140 0x7f5c30116f30 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.685+0000 7f5c15ffb640 1 --2- 172.21.15.22:0/2276837919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30110190 0x7f5c301169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.685+0000 7f5c15ffb640 1 --2- 172.21.15.22:0/2276837919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c3006ea30 0x7f5c3011a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.685+0000 7f5c15ffb640 1 -- 172.21.15.22:0/2276837919 >> 172.21.15.22:0/2276837919 conn(0x7f5c30073180 msgr2=0x7f5c3011b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.686+0000 7f5c15ffb640 1 -- 172.21.15.22:0/2276837919 shutdown_connections 2024-09-19T08:19:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:18.686+0000 7f5c15ffb640 1 -- 172.21.15.22:0/2276837919 wait complete. 2024-09-19T08:19:18.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:18 smithi022 ceph-mon[30963]: pgmap v131: 97 pgs: 97 active+clean; 508 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-09-19T08:19:19.002 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:18 smithi157 ceph-mon[35143]: pgmap v131: 97 pgs: 97 active+clean; 508 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-09-19T08:19:19.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:18 smithi064 ceph-mon[34900]: pgmap v131: 97 pgs: 97 active+clean; 508 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-09-19T08:19:20.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:19:20.690 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:20 smithi022 ceph-mon[30963]: pgmap v132: 97 pgs: 97 active+clean; 515 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 196 op/s 2024-09-19T08:19:21.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:20 smithi157 ceph-mon[35143]: pgmap v132: 97 pgs: 97 active+clean; 515 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 196 op/s 2024-09-19T08:19:21.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:20 smithi064 ceph-mon[34900]: pgmap v132: 97 pgs: 97 active+clean; 515 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 196 op/s 2024-09-19T08:19:21.379 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:19:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.906+0000 7fb7e6ffb640 1 -- 172.21.15.22:0/4190767958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7e0102350 msgr2=0x7fb7e010e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.906+0000 7fb7e6ffb640 1 --2- 172.21.15.22:0/4190767958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7e0102350 0x7fb7e010e880 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009930 tx=0x7fb7d802a420 comp rx=0 tx=0).stop 2024-09-19T08:19:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.906+0000 7fb7e6ffb640 1 -- 172.21.15.22:0/4190767958 shutdown_connections 2024-09-19T08:19:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.906+0000 7fb7e6ffb640 1 --2- 172.21.15.22:0/4190767958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7e0102350 0x7fb7e010e880 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.906+0000 7fb7e6ffb640 1 --2- 172.21.15.22:0/4190767958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7e01019b0 0x7fb7e0101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.906+0000 7fb7e6ffb640 1 --2- 172.21.15.22:0/4190767958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7e01079b0 0x7fb7e0107d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.906+0000 7fb7e6ffb640 1 -- 172.21.15.22:0/4190767958 >> 172.21.15.22:0/4190767958 conn(0x7fb7e00fd6b0 msgr2=0x7fb7e00ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.906+0000 7fb7e6ffb640 1 -- 172.21.15.22:0/4190767958 shutdown_connections 2024-09-19T08:19:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.906+0000 7fb7e6ffb640 1 -- 172.21.15.22:0/4190767958 wait complete. 2024-09-19T08:19:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e6ffb640 1 Processor -- start 2024-09-19T08:19:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e6ffb640 1 -- start start 2024-09-19T08:19:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e6ffb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7e01019b0 0x7fb7e0198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e6ffb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7e0102350 0x7fb7e01987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e6ffb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7e01079b0 0x7fb7e019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e6ffb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e0111db0 con 0x7fb7e01079b0 2024-09-19T08:19:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e6ffb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e0111c30 con 0x7fb7e01019b0 2024-09-19T08:19:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e6ffb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e0111f30 con 0x7fb7e0102350 2024-09-19T08:19:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e67fa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7e01079b0 0x7fb7e019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e57f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7e0102350 0x7fb7e01987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e67fa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7e01079b0 0x7fb7e019cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47656/0 (socket says 172.21.15.22:47656) 2024-09-19T08:19:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e67fa640 1 -- 172.21.15.22:0/1347737223 learned_addr learned my addr 172.21.15.22:0/1347737223 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e5ff9640 1 --2- 172.21.15.22:0/1347737223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7e01019b0 0x7fb7e0198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e67fa640 1 -- 172.21.15.22:0/1347737223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7e0102350 msgr2=0x7fb7e01987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e67fa640 1 --2- 172.21.15.22:0/1347737223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7e0102350 0x7fb7e01987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e67fa640 1 -- 172.21.15.22:0/1347737223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7e01019b0 msgr2=0x7fb7e0198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e67fa640 1 --2- 172.21.15.22:0/1347737223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7e01019b0 0x7fb7e0198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.907+0000 7fb7e67fa640 1 -- 172.21.15.22:0/1347737223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e019d220 con 0x7fb7e01079b0 2024-09-19T08:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.908+0000 7fb7e67fa640 1 --2- 172.21.15.22:0/1347737223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7e01079b0 0x7fb7e019cb40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8005ab0 tx=0x7fb7d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.908+0000 7fb7deffd640 1 -- 172.21.15.22:0/1347737223 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d803f030 con 0x7fb7e01079b0 2024-09-19T08:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.908+0000 7fb7e6ffb640 1 -- 172.21.15.22:0/1347737223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e019d4b0 con 0x7fb7e01079b0 2024-09-19T08:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.908+0000 7fb7deffd640 1 -- 172.21.15.22:0/1347737223 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7d8003ce0 con 0x7fb7e01079b0 2024-09-19T08:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.908+0000 7fb7deffd640 1 -- 172.21.15.22:0/1347737223 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8035070 con 0x7fb7e01079b0 2024-09-19T08:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.908+0000 7fb7e6ffb640 1 -- 172.21.15.22:0/1347737223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e01a4df0 con 0x7fb7e01079b0 2024-09-19T08:19:21.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.910+0000 7fb7deffd640 1 -- 172.21.15.22:0/1347737223 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb7d8042030 con 0x7fb7e01079b0 2024-09-19T08:19:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.910+0000 7fb7ea636740 1 Processor -- start 2024-09-19T08:19:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.910+0000 7fb7ea636740 1 -- start start 2024-09-19T08:19:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7ea636740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c84cf0 0x55a377c83240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7ea636740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 0x55a377c83bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7ea636740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c8e520 0x55a377c90910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7ea636740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a377be92f0 con 0x55a377c84cf0 2024-09-19T08:19:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7ea636740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a377b90cf0 con 0x55a377c8e520 2024-09-19T08:19:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7ea636740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a377be52b0 con 0x55a377c83780 2024-09-19T08:19:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7e57f8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c84cf0 0x55a377c83240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7e57f8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c84cf0 0x55a377c83240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47662/0 (socket says 172.21.15.22:47662) 2024-09-19T08:19:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7e57f8640 1 -- 172.21.15.22:0/2206243165 learned_addr learned my addr 172.21.15.22:0/2206243165 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7e5ff9640 1 --2- 172.21.15.22:0/2206243165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 0x55a377c83bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7e57f8640 1 --2- 172.21.15.22:0/2206243165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c8e520 0x55a377c90910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7e57f8640 1 -- 172.21.15.22:0/2206243165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 msgr2=0x55a377c83bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7e57f8640 1 --2- 172.21.15.22:0/2206243165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 0x55a377c83bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7e57f8640 1 -- 172.21.15.22:0/2206243165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c8e520 msgr2=0x55a377c90910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7deffd640 1 --2- 172.21.15.22:0/1347737223 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb7c80a33b0 0x7fb7c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7e57f8640 1 --2- 172.21.15.22:0/2206243165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c8e520 0x55a377c90910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7e57f8640 1 -- 172.21.15.22:0/2206243165 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a377b753a0 con 0x55a377c84cf0 2024-09-19T08:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7e5ff9640 1 --2- 172.21.15.22:0/2206243165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 0x55a377c83bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7e67fa640 1 --2- 172.21.15.22:0/1347737223 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb7c80a33b0 0x7fb7c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7e57f8640 1 --2- 172.21.15.22:0/2206243165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c84cf0 0x55a377c83240 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4004650 tx=0x7fb7d4004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a6221b117d05fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.911+0000 7fb7deffd640 1 -- 172.21.15.22:0/1347737223 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb7d80f0160 con 0x7fb7e01079b0 2024-09-19T08:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.912+0000 7fb7c7fff640 1 -- 172.21.15.22:0/2206243165 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4005020 con 0x55a377c84cf0 2024-09-19T08:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.912+0000 7fb7c7fff640 1 -- 172.21.15.22:0/2206243165 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7d40056a0 con 0x55a377c84cf0 2024-09-19T08:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.912+0000 7fb7c7fff640 1 -- 172.21.15.22:0/2206243165 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d40106c0 con 0x55a377c84cf0 2024-09-19T08:19:21.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.912+0000 7fb7ea636740 1 -- 172.21.15.22:0/2206243165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c84cf0 msgr2=0x55a377c83240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:21.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.912+0000 7fb7ea636740 1 --2- 172.21.15.22:0/2206243165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c84cf0 0x55a377c83240 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4004650 tx=0x7fb7d4004250 comp rx=0 tx=0).stop 2024-09-19T08:19:21.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.912+0000 7fb7e67fa640 1 --2- 172.21.15.22:0/1347737223 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb7c80a33b0 0x7fb7c80a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80bc1a0 tx=0x7fb7d8009260 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:21.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.912+0000 7fb7ea636740 1 -- 172.21.15.22:0/2206243165 shutdown_connections 2024-09-19T08:19:21.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.912+0000 7fb7ea636740 1 --2- 172.21.15.22:0/2206243165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c8e520 0x55a377c90910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.912+0000 7fb7ea636740 1 --2- 172.21.15.22:0/2206243165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 0x55a377c83bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.912+0000 7fb7ea636740 1 --2- 172.21.15.22:0/2206243165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c84cf0 0x55a377c83240 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.912+0000 7fb7ea636740 1 -- 172.21.15.22:0/2206243165 >> 172.21.15.22:0/2206243165 conn(0x55a377c4c310 msgr2=0x55a377c7fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:21.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.913+0000 7fb7ea636740 1 -- 172.21.15.22:0/2206243165 shutdown_connections 2024-09-19T08:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.913+0000 7fb7ea636740 1 -- 172.21.15.22:0/2206243165 wait complete. 2024-09-19T08:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.913+0000 7fb7ea636740 1 -- start start 2024-09-19T08:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.913+0000 7fb7ea636740 1 Processor -- start 2024-09-19T08:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.913+0000 7fb7ea636740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c83780 0x55a377c8ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.913+0000 7fb7ea636740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c84cf0 0x55a377c8b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.913+0000 7fb7ea636740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c8e520 0x55a377c8b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.913+0000 7fb7ea636740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a377c911c0 con 0x55a377c84cf0 2024-09-19T08:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.913+0000 7fb7ea636740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a377ba4060 con 0x55a377c83780 2024-09-19T08:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.913+0000 7fb7ea636740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a377c91340 con 0x55a377c8e520 2024-09-19T08:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.913+0000 7fb7ea636740 10 client.0 _ll_register_callbacks cb 0x55a3778f78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7e5ff9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c84cf0 0x55a377c8b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7e57f8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c83780 0x55a377c8ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7e5ff9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c84cf0 0x55a377c8b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47678/0 (socket says 172.21.15.22:47678) 2024-09-19T08:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7e5ff9640 1 -- 172.21.15.22:0/2569109897 learned_addr learned my addr 172.21.15.22:0/2569109897 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7e57f8640 1 --2- 172.21.15.22:0/2569109897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c8e520 0x55a377c8b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7e5ff9640 1 -- 172.21.15.22:0/2569109897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c8e520 msgr2=0x55a377c8b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7e5ff9640 1 --2- 172.21.15.22:0/2569109897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c8e520 0x55a377c8b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7e5ff9640 1 -- 172.21.15.22:0/2569109897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c83780 msgr2=0x55a377c8ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7e5ff9640 1 --2- 172.21.15.22:0/2569109897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c83780 0x55a377c8ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7e5ff9640 1 -- 172.21.15.22:0/2569109897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a37795a3c0 con 0x55a377c84cf0 2024-09-19T08:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7e5ff9640 1 --2- 172.21.15.22:0/2569109897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c84cf0 0x55a377c8b180 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0011740 tx=0x7fb7d0011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42d5dd76c257edd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7c57fa640 1 -- 172.21.15.22:0/2569109897 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0010c60 con 0x55a377c84cf0 2024-09-19T08:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7c57fa640 1 -- 172.21.15.22:0/2569109897 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb7d0010e00 con 0x55a377c84cf0 2024-09-19T08:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7ea636740 1 -- 172.21.15.22:0/2569109897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7d40056a0 con 0x55a377c84cf0 2024-09-19T08:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7c57fa640 1 -- 172.21.15.22:0/2569109897 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7d0003970 con 0x55a377c84cf0 2024-09-19T08:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7ea636740 10 client.16236 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.914+0000 7fb7ea636740 1 -- 172.21.15.22:0/2569109897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a377c9cc70 con 0x55a377c84cf0 2024-09-19T08:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.915+0000 7fb7c57fa640 1 -- 172.21.15.22:0/2569109897 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7d40056a0 con 0x55a377c84cf0 2024-09-19T08:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.915+0000 7fb7c57fa640 1 -- 172.21.15.22:0/2569109897 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb7d000f630 con 0x55a377c84cf0 2024-09-19T08:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.915+0000 7fb7ea636740 10 client.16236 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.915+0000 7fb7ea636740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.915+0000 7fb7ea636740 20 client.16236 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.915+0000 7fb7ea636740 1 --2- 172.21.15.22:0/2569109897 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a377c9e180 0x55a377ca05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.915+0000 7fb7ea636740 4 client.16236 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.915+0000 7fb7ea636740 1 -- 172.21.15.22:0/2569109897 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a377bf6860 con 0x55a377c9e180 2024-09-19T08:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.915+0000 7fb7e57f8640 1 --2- 172.21.15.22:0/2569109897 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a377c9e180 0x55a377ca05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.916+0000 7fb7e57f8640 1 --2- 172.21.15.22:0/2569109897 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a377c9e180 0x55a377ca05a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8af65d703d32f5e server_cookie=93d1fe7e2ff06d97 in_seq=0 out_seq=0 2024-09-19T08:19:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.916+0000 7fb7c57fa640 10 client.16236 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.916+0000 7fb7c57fa640 1 -- 172.21.15.22:0/2569109897 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a377bf6860 con 0x55a377c9e180 2024-09-19T08:19:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.916+0000 7fb7c57fa640 10 client.16236 handle_command_reply: tid=0 2024-09-19T08:19:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.916+0000 7fb7ea636740 1 client.16236 shutdown 2024-09-19T08:19:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.917+0000 7fb7ea636740 1 -- 172.21.15.22:0/2569109897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c84cf0 msgr2=0x55a377c8b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.917+0000 7fb7ea636740 1 --2- 172.21.15.22:0/2569109897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c84cf0 0x55a377c8b180 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0011740 tx=0x7fb7d0011c00 comp rx=0 tx=0).stop 2024-09-19T08:19:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.917+0000 7fb7ea636740 1 -- 172.21.15.22:0/2569109897 shutdown_connections 2024-09-19T08:19:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.917+0000 7fb7ea636740 1 --2- 172.21.15.22:0/2569109897 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a377c9e180 0x55a377ca05a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.917+0000 7fb7ea636740 1 --2- 172.21.15.22:0/2569109897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c8e520 0x55a377c8b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.917+0000 7fb7c57fa640 0 client.16236 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.917+0000 7fb7ea636740 1 --2- 172.21.15.22:0/2569109897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c84cf0 0x55a377c8b180 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.917+0000 7fb7ea636740 1 --2- 172.21.15.22:0/2569109897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c83780 0x55a377c8ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.917+0000 7fb7ea636740 1 -- 172.21.15.22:0/2569109897 >> 172.21.15.22:0/2569109897 conn(0x55a377c4c310 msgr2=0x55a377c85b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.917+0000 7fb7ea636740 1 -- 172.21.15.22:0/2569109897 shutdown_connections 2024-09-19T08:19:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.917+0000 7fb7ea636740 1 -- 172.21.15.22:0/2569109897 wait complete. 2024-09-19T08:19:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.917+0000 7fb7ea636740 20 client.16236 trim_cache size 0 max 16384 2024-09-19T08:19:21.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.930+0000 7fb7ea636740 1 Processor -- start 2024-09-19T08:19:21.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.930+0000 7fb7ea636740 1 -- start start 2024-09-19T08:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.930+0000 7fb7ea636740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 0x55a377c8d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.930+0000 7fb7ea636740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c84cf0 0x55a377c862d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.930+0000 7fb7ea636740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c8e520 0x55a377c86810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.930+0000 7fb7ea636740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e01b0070 con 0x55a377c8e520 2024-09-19T08:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.930+0000 7fb7ea636740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e01ad8e0 con 0x55a377c84cf0 2024-09-19T08:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.930+0000 7fb7ea636740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e01ab730 con 0x55a377c83780 2024-09-19T08:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.931+0000 7fb7e57f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 0x55a377c8d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.931+0000 7fb7e5ff9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c84cf0 0x55a377c862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.931+0000 7fb7e57f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 0x55a377c8d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:56068/0 (socket says 172.21.15.22:56068) 2024-09-19T08:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.931+0000 7fb7e57f8640 1 -- 172.21.15.22:0/4221787470 learned_addr learned my addr 172.21.15.22:0/4221787470 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.931+0000 7fb7e57f8640 1 --2- 172.21.15.22:0/4221787470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c8e520 0x55a377c86810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.931+0000 7fb7e5ff9640 1 -- 172.21.15.22:0/4221787470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 msgr2=0x55a377c8d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.931+0000 7fb7e5ff9640 1 --2- 172.21.15.22:0/4221787470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 0x55a377c8d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.931+0000 7fb7e5ff9640 1 -- 172.21.15.22:0/4221787470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c8e520 msgr2=0x55a377c86810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.931+0000 7fb7e5ff9640 1 --2- 172.21.15.22:0/4221787470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c8e520 0x55a377c86810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.931+0000 7fb7e5ff9640 1 -- 172.21.15.22:0/4221787470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a377c8a130 con 0x55a377c84cf0 2024-09-19T08:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.931+0000 7fb7e5ff9640 1 --2- 172.21.15.22:0/4221787470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c84cf0 0x55a377c862d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0011710 tx=0x7fb7d0005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=937b19d41fcf561a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.931+0000 7fb7c77fe640 1 -- 172.21.15.22:0/4221787470 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0028040 con 0x55a377c84cf0 2024-09-19T08:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.931+0000 7fb7c77fe640 1 -- 172.21.15.22:0/4221787470 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7d0011e50 con 0x55a377c84cf0 2024-09-19T08:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.931+0000 7fb7c77fe640 1 -- 172.21.15.22:0/4221787470 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000f6d0 con 0x55a377c84cf0 2024-09-19T08:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.932+0000 7fb7ea636740 1 -- 172.21.15.22:0/4221787470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c84cf0 msgr2=0x55a377c862d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.932+0000 7fb7ea636740 1 --2- 172.21.15.22:0/4221787470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c84cf0 0x55a377c862d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0011710 tx=0x7fb7d0005bd0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.932+0000 7fb7ea636740 1 -- 172.21.15.22:0/4221787470 shutdown_connections 2024-09-19T08:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.932+0000 7fb7ea636740 1 --2- 172.21.15.22:0/4221787470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c8e520 0x55a377c86810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.932+0000 7fb7ea636740 1 --2- 172.21.15.22:0/4221787470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c84cf0 0x55a377c862d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.932+0000 7fb7ea636740 1 --2- 172.21.15.22:0/4221787470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 0x55a377c8d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.932+0000 7fb7ea636740 1 -- 172.21.15.22:0/4221787470 >> 172.21.15.22:0/4221787470 conn(0x55a377c4c310 msgr2=0x55a377c8c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:21.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.932+0000 7fb7ea636740 1 -- 172.21.15.22:0/4221787470 shutdown_connections 2024-09-19T08:19:21.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.932+0000 7fb7ea636740 1 -- 172.21.15.22:0/4221787470 wait complete. 2024-09-19T08:19:21.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.932+0000 7fb7ea636740 1 -- start start 2024-09-19T08:19:21.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.932+0000 7fb7ea636740 1 Processor -- start 2024-09-19T08:19:21.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7ea636740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 0x55a377c96ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7ea636740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c84cf0 0x55a377c92220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7ea636740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c8e520 0x55a377c929b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7ea636740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e01abda0 con 0x55a377c8e520 2024-09-19T08:19:21.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7ea636740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e01ad210 con 0x55a377c84cf0 2024-09-19T08:19:21.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7ea636740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e01ac470 con 0x55a377c83780 2024-09-19T08:19:21.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7ea636740 10 client.0 _ll_register_callbacks cb 0x7fb7d0009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:21.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7e57f8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c8e520 0x55a377c929b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7e57f8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c8e520 0x55a377c929b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47692/0 (socket says 172.21.15.22:47692) 2024-09-19T08:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7e57f8640 1 -- 172.21.15.22:0/422293290 learned_addr learned my addr 172.21.15.22:0/422293290 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7e5ff9640 1 --2- 172.21.15.22:0/422293290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c84cf0 0x55a377c92220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7e57f8640 1 --2- 172.21.15.22:0/422293290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 0x55a377c96ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7e5ff9640 1 -- 172.21.15.22:0/422293290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 msgr2=0x55a377c96ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7e5ff9640 1 --2- 172.21.15.22:0/422293290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 0x55a377c96ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7e5ff9640 1 -- 172.21.15.22:0/422293290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c8e520 msgr2=0x55a377c929b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7e5ff9640 1 --2- 172.21.15.22:0/422293290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c8e520 0x55a377c929b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.933+0000 7fb7e5ff9640 1 -- 172.21.15.22:0/422293290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a377d07e40 con 0x55a377c84cf0 2024-09-19T08:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7e5ff9640 1 --2- 172.21.15.22:0/422293290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c84cf0 0x55a377c92220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55a377c8c2e0 tx=0x7fb7d0005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=998586d7b49beb75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7c57fa640 1 -- 172.21.15.22:0/422293290 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0028070 con 0x55a377c84cf0 2024-09-19T08:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7ea636740 1 -- 172.21.15.22:0/422293290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7d0011e50 con 0x55a377c84cf0 2024-09-19T08:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7c57fa640 1 -- 172.21.15.22:0/422293290 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb7d0009760 con 0x55a377c84cf0 2024-09-19T08:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7c57fa640 1 -- 172.21.15.22:0/422293290 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7d003d050 con 0x55a377c84cf0 2024-09-19T08:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7ea636740 10 client.25838 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7ea636740 1 -- 172.21.15.22:0/422293290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a377d08fd0 con 0x55a377c84cf0 2024-09-19T08:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7c57fa640 1 -- 172.21.15.22:0/422293290 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7d0011e50 con 0x55a377c84cf0 2024-09-19T08:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7c57fa640 1 -- 172.21.15.22:0/422293290 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb7d002b070 con 0x55a377c84cf0 2024-09-19T08:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7ea636740 10 client.25838 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7ea636740 10 client.25838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7ea636740 20 client.25838 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7ea636740 1 --2- 172.21.15.22:0/422293290 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a377d094e0 0x55a377d0b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7ea636740 4 client.25838 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.934+0000 7fb7ea636740 1 -- 172.21.15.22:0/422293290 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a377bf0d70 con 0x55a377d094e0 2024-09-19T08:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.935+0000 7fb7e57f8640 1 --2- 172.21.15.22:0/422293290 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a377d094e0 0x55a377d0b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.935+0000 7fb7c57fa640 10 client.25838 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:21.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.935+0000 7fb7e57f8640 1 --2- 172.21.15.22:0/422293290 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a377d094e0 0x55a377d0b8d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec1e6c6ed24be96c server_cookie=d243da77806bb536 in_seq=0 out_seq=0 2024-09-19T08:19:21.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.935+0000 7fb7c57fa640 1 -- 172.21.15.22:0/422293290 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a377bf0d70 con 0x55a377d094e0 2024-09-19T08:19:21.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.935+0000 7fb7c57fa640 10 client.25838 handle_command_reply: tid=0 2024-09-19T08:19:21.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.935+0000 7fb7ea636740 1 client.25838 shutdown 2024-09-19T08:19:21.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.935+0000 7fb7ea636740 1 -- 172.21.15.22:0/422293290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c84cf0 msgr2=0x55a377c92220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:21.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.935+0000 7fb7ea636740 1 --2- 172.21.15.22:0/422293290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c84cf0 0x55a377c92220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55a377c8c2e0 tx=0x7fb7d0005d20 comp rx=0 tx=0).stop 2024-09-19T08:19:21.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7ea636740 1 -- 172.21.15.22:0/422293290 shutdown_connections 2024-09-19T08:19:21.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7ea636740 1 --2- 172.21.15.22:0/422293290 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a377d094e0 0x55a377d0b8d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7ea636740 1 --2- 172.21.15.22:0/422293290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a377c8e520 0x55a377c929b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7ea636740 1 --2- 172.21.15.22:0/422293290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a377c84cf0 0x55a377c92220 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7c57fa640 0 client.25838 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:21.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7ea636740 1 --2- 172.21.15.22:0/422293290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377c83780 0x55a377c96ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7ea636740 1 -- 172.21.15.22:0/422293290 >> 172.21.15.22:0/422293290 conn(0x55a377c4c310 msgr2=0x55a377c8d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:21.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7ea636740 1 -- 172.21.15.22:0/422293290 shutdown_connections 2024-09-19T08:19:21.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7ea636740 1 -- 172.21.15.22:0/422293290 wait complete. 2024-09-19T08:19:21.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7ea636740 20 client.25838 trim_cache size 0 max 16384 2024-09-19T08:19:21.947 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:19:21.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7e6ffb640 1 -- 172.21.15.22:0/1347737223 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb7c80a33b0 msgr2=0x7fb7c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:21.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7e6ffb640 1 --2- 172.21.15.22:0/1347737223 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb7c80a33b0 0x7fb7c80a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80bc1a0 tx=0x7fb7d8009260 comp rx=0 tx=0).stop 2024-09-19T08:19:21.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7e6ffb640 1 -- 172.21.15.22:0/1347737223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7e01079b0 msgr2=0x7fb7e019cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:21.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7e6ffb640 1 --2- 172.21.15.22:0/1347737223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7e01079b0 0x7fb7e019cb40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8005ab0 tx=0x7fb7d80042f0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7e6ffb640 1 -- 172.21.15.22:0/1347737223 shutdown_connections 2024-09-19T08:19:21.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7e6ffb640 1 --2- 172.21.15.22:0/1347737223 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb7c80a33b0 0x7fb7c80a5870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7e6ffb640 1 --2- 172.21.15.22:0/1347737223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7e01079b0 0x7fb7e019cb40 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7e6ffb640 1 --2- 172.21.15.22:0/1347737223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7e0102350 0x7fb7e01987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7e6ffb640 1 --2- 172.21.15.22:0/1347737223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7e01019b0 0x7fb7e0198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:21.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7e6ffb640 1 -- 172.21.15.22:0/1347737223 >> 172.21.15.22:0/1347737223 conn(0x7fb7e00fd6b0 msgr2=0x7fb7e00fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:21.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7e6ffb640 1 -- 172.21.15.22:0/1347737223 shutdown_connections 2024-09-19T08:19:21.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:21.936+0000 7fb7e6ffb640 1 -- 172.21.15.22:0/1347737223 wait complete. 2024-09-19T08:19:23.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:22 smithi157 ceph-mon[35143]: pgmap v133: 97 pgs: 97 active+clean; 515 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 132 op/s 2024-09-19T08:19:23.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:22 smithi022 ceph-mon[30963]: pgmap v133: 97 pgs: 97 active+clean; 515 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 132 op/s 2024-09-19T08:19:23.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:22 smithi064 ceph-mon[34900]: pgmap v133: 97 pgs: 97 active+clean; 515 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 132 op/s 2024-09-19T08:19:23.721 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:19:23.722 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:19:23.996 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:19:24.858 INFO:tasks.quiescer.fs.[cephfs]:Total duration (59.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T08:19:24.858 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T08:19:24.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:19:24.956 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:24 smithi022 ceph-mon[30963]: pgmap v134: 97 pgs: 97 active+clean; 574 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 623 op/s 2024-09-19T08:19:24.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.990+0000 7f3456b4f640 1 -- 172.21.15.22:0/4010321732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450102570 msgr2=0x7f34501029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:24.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.990+0000 7f3456b4f640 1 --2- 172.21.15.22:0/4010321732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450102570 0x7f34501029b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3444009ca0 tx=0x7f344402a950 comp rx=0 tx=0).stop 2024-09-19T08:19:24.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.991+0000 7f3456b4f640 1 -- 172.21.15.22:0/4010321732 shutdown_connections 2024-09-19T08:19:24.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.991+0000 7f3456b4f640 1 --2- 172.21.15.22:0/4010321732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3450102ef0 0x7f345010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:24.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.991+0000 7f3456b4f640 1 --2- 172.21.15.22:0/4010321732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450102570 0x7f34501029b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:24.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.991+0000 7f3456b4f640 1 --2- 172.21.15.22:0/4010321732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450100e20 0x7f3450101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:24.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.991+0000 7f3456b4f640 1 -- 172.21.15.22:0/4010321732 >> 172.21.15.22:0/4010321732 conn(0x7f34500fc7d0 msgr2=0x7f34500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:24.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.991+0000 7f3456b4f640 1 -- 172.21.15.22:0/4010321732 shutdown_connections 2024-09-19T08:19:24.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.991+0000 7f3456b4f640 1 -- 172.21.15.22:0/4010321732 wait complete. 2024-09-19T08:19:25.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.991+0000 7f3456b4f640 1 Processor -- start 2024-09-19T08:19:25.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f3456b4f640 1 -- start start 2024-09-19T08:19:25.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f3456b4f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3450100e20 0x7f34501984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f3456b4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450102570 0x7f3450198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f344ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450102570 0x7f3450198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f344ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450102570 0x7f3450198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47708/0 (socket says 172.21.15.22:47708) 2024-09-19T08:19:25.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f34548eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3450100e20 0x7f34501984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f3456b4f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450102ef0 0x7f345019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f344ffff640 1 -- 172.21.15.22:0/218925551 learned_addr learned my addr 172.21.15.22:0/218925551 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:25.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f3456b4f640 1 -- 172.21.15.22:0/218925551 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3450110f80 con 0x7f3450102570 2024-09-19T08:19:25.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f3456b4f640 1 -- 172.21.15.22:0/218925551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3450110e00 con 0x7f3450102ef0 2024-09-19T08:19:25.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f3456b4f640 1 -- 172.21.15.22:0/218925551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3450111100 con 0x7f3450100e20 2024-09-19T08:19:25.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f344ffff640 1 -- 172.21.15.22:0/218925551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3450100e20 msgr2=0x7f34501984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f344ffff640 1 --2- 172.21.15.22:0/218925551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3450100e20 0x7f34501984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f34550ec640 1 --2- 172.21.15.22:0/218925551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450102ef0 0x7f345019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f344ffff640 1 -- 172.21.15.22:0/218925551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450102ef0 msgr2=0x7f345019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f344ffff640 1 --2- 172.21.15.22:0/218925551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450102ef0 0x7f345019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f344ffff640 1 -- 172.21.15.22:0/218925551 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345019d490 con 0x7f3450102570 2024-09-19T08:19:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.992+0000 7f34550ec640 1 --2- 172.21.15.22:0/218925551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450102ef0 0x7f345019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:19:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.993+0000 7f344ffff640 1 --2- 172.21.15.22:0/218925551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450102570 0x7f3450198a20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3444005cc0 tx=0x7f344402f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.993+0000 7f344dffb640 1 -- 172.21.15.22:0/218925551 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3444034040 con 0x7f3450102570 2024-09-19T08:19:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.993+0000 7f3456b4f640 1 -- 172.21.15.22:0/218925551 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345019d720 con 0x7f3450102570 2024-09-19T08:19:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.993+0000 7f344dffb640 1 -- 172.21.15.22:0/218925551 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3444030920 con 0x7f3450102570 2024-09-19T08:19:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.993+0000 7f344dffb640 1 -- 172.21.15.22:0/218925551 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34440046e0 con 0x7f3450102570 2024-09-19T08:19:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.993+0000 7f3456b4f640 1 -- 172.21.15.22:0/218925551 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34501a5060 con 0x7f3450102570 2024-09-19T08:19:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.996+0000 7f344dffb640 1 -- 172.21.15.22:0/218925551 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3444037020 con 0x7f3450102570 2024-09-19T08:19:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.996+0000 7f3458f28740 1 Processor -- start 2024-09-19T08:19:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.996+0000 7f3458f28740 1 -- start start 2024-09-19T08:19:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.996+0000 7f3458f28740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b510da0 0x56309b511180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.996+0000 7f3458f28740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b5116c0 0x56309b50b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.996+0000 7f3458f28740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b50bd60 0x56309b50c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.996+0000 7f3458f28740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56309b467280 con 0x56309b5116c0 2024-09-19T08:19:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.996+0000 7f3458f28740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56309b40eb10 con 0x56309b50bd60 2024-09-19T08:19:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.996+0000 7f3458f28740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56309b463240 con 0x56309b510da0 2024-09-19T08:19:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f34548eb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b5116c0 0x56309b50b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f344ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b50bd60 0x56309b50c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f34548eb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b5116c0 0x56309b50b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47714/0 (socket says 172.21.15.22:47714) 2024-09-19T08:19:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f34548eb640 1 -- 172.21.15.22:0/1099645281 learned_addr learned my addr 172.21.15.22:0/1099645281 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f34550ec640 1 --2- 172.21.15.22:0/1099645281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b510da0 0x56309b511180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f344ffff640 1 -- 172.21.15.22:0/1099645281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b510da0 msgr2=0x56309b511180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f344ffff640 1 --2- 172.21.15.22:0/1099645281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b510da0 0x56309b511180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f344dffb640 1 --2- 172.21.15.22:0/218925551 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f34380a33b0 0x7f34380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f344ffff640 1 -- 172.21.15.22:0/1099645281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b5116c0 msgr2=0x56309b50b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f344ffff640 1 --2- 172.21.15.22:0/1099645281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b5116c0 0x56309b50b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f344ffff640 1 -- 172.21.15.22:0/1099645281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56309b41b1f0 con 0x56309b50bd60 2024-09-19T08:19:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f34548eb640 1 --2- 172.21.15.22:0/1099645281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b5116c0 0x56309b50b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f34550ec640 1 --2- 172.21.15.22:0/218925551 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f34380a33b0 0x7f34380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f344dffb640 1 -- 172.21.15.22:0/218925551 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f34440f0b90 con 0x7f3450102570 2024-09-19T08:19:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.997+0000 7f344ffff640 1 --2- 172.21.15.22:0/1099645281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b50bd60 0x56309b50c1e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f34440ee3a0 tx=0x7f34440b9000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94502e3766d62638 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.998+0000 7f3436ffd640 1 -- 172.21.15.22:0/1099645281 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34440f5040 con 0x56309b50bd60 2024-09-19T08:19:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.998+0000 7f3436ffd640 1 -- 172.21.15.22:0/1099645281 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344402c030 con 0x56309b50bd60 2024-09-19T08:19:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.998+0000 7f3436ffd640 1 -- 172.21.15.22:0/1099645281 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34440bcd00 con 0x56309b50bd60 2024-09-19T08:19:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.998+0000 7f3458f28740 1 -- 172.21.15.22:0/1099645281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b50bd60 msgr2=0x56309b50c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.998+0000 7f3458f28740 1 --2- 172.21.15.22:0/1099645281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b50bd60 0x56309b50c1e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f34440ee3a0 tx=0x7f34440b9000 comp rx=0 tx=0).stop 2024-09-19T08:19:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.998+0000 7f34550ec640 1 --2- 172.21.15.22:0/218925551 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f34380a33b0 0x7f34380a5870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3448009ce0 tx=0x7f3448005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.998+0000 7f3458f28740 1 -- 172.21.15.22:0/1099645281 shutdown_connections 2024-09-19T08:19:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.998+0000 7f3458f28740 1 --2- 172.21.15.22:0/1099645281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b50bd60 0x56309b50c1e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.998+0000 7f3458f28740 1 --2- 172.21.15.22:0/1099645281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b5116c0 0x56309b50b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.998+0000 7f3458f28740 1 --2- 172.21.15.22:0/1099645281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b510da0 0x56309b511180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.999+0000 7f3458f28740 1 -- 172.21.15.22:0/1099645281 >> 172.21.15.22:0/1099645281 conn(0x56309b4fdba0 msgr2=0x56309b4fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.999+0000 7f3458f28740 1 -- 172.21.15.22:0/1099645281 shutdown_connections 2024-09-19T08:19:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.999+0000 7f3458f28740 1 -- 172.21.15.22:0/1099645281 wait complete. 2024-09-19T08:19:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.999+0000 7f3458f28740 1 -- start start 2024-09-19T08:19:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:24.999+0000 7f3458f28740 1 Processor -- start 2024-09-19T08:19:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f3458f28740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b50bd60 0x56309b514cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f3458f28740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b510da0 0x56309b5151f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f3458f28740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b5116c0 0x56309b515730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f3458f28740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56309b517ea0 con 0x56309b50bd60 2024-09-19T08:19:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f3458f28740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56309b421f10 con 0x56309b5116c0 2024-09-19T08:19:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f3458f28740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56309b518020 con 0x56309b510da0 2024-09-19T08:19:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f3458f28740 10 client.0 _ll_register_callbacks cb 0x56309b3eaa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f34550ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b5116c0 0x56309b515730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f344ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b510da0 0x56309b5151f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f344ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b510da0 0x56309b5151f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:56122/0 (socket says 172.21.15.22:56122) 2024-09-19T08:19:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f344ffff640 1 -- 172.21.15.22:0/2779272543 learned_addr learned my addr 172.21.15.22:0/2779272543 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f34550ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b5116c0 0x56309b515730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:36104/0 (socket says 172.21.15.22:36104) 2024-09-19T08:19:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f34548eb640 1 --2- 172.21.15.22:0/2779272543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b50bd60 0x56309b514cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f34550ec640 1 -- 172.21.15.22:0/2779272543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b510da0 msgr2=0x56309b5151f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f34550ec640 1 --2- 172.21.15.22:0/2779272543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b510da0 0x56309b5151f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f34550ec640 1 -- 172.21.15.22:0/2779272543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b50bd60 msgr2=0x56309b514cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f34550ec640 1 --2- 172.21.15.22:0/2779272543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b50bd60 0x56309b514cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f34550ec640 1 -- 172.21.15.22:0/2779272543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56309b1d86a0 con 0x56309b5116c0 2024-09-19T08:19:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f344ffff640 1 --2- 172.21.15.22:0/2779272543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b510da0 0x56309b5151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f34550ec640 1 --2- 172.21.15.22:0/2779272543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b5116c0 0x56309b515730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f344801e630 tx=0x7f34480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=994b1ddbc0009e61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.000+0000 7f342ffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:19:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.001+0000 7f342ffff640 1 -- 172.21.15.22:0/2779272543 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448004020 con 0x56309b5116c0 2024-09-19T08:19:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.001+0000 7f342ffff640 1 -- 172.21.15.22:0/2779272543 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f34480041c0 con 0x56309b5116c0 2024-09-19T08:19:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.001+0000 7f3458f28740 1 -- 172.21.15.22:0/2779272543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f344402c030 con 0x56309b5116c0 2024-09-19T08:19:25.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:24 smithi157 ceph-mon[35143]: pgmap v134: 97 pgs: 97 active+clean; 574 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 623 op/s 2024-09-19T08:19:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.001+0000 7f342ffff640 1 -- 172.21.15.22:0/2779272543 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f34480043c0 con 0x56309b5116c0 2024-09-19T08:19:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.001+0000 7f3458f28740 10 client.25847 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.001+0000 7f3458f28740 1 -- 172.21.15.22:0/2779272543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56309b523bf0 con 0x56309b5116c0 2024-09-19T08:19:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.001+0000 7f342ffff640 1 -- 172.21.15.22:0/2779272543 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344402c030 con 0x56309b5116c0 2024-09-19T08:19:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.001+0000 7f342ffff640 1 -- 172.21.15.22:0/2779272543 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f344801a040 con 0x56309b5116c0 2024-09-19T08:19:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.001+0000 7f3458f28740 10 client.25847 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.001+0000 7f3458f28740 10 client.25847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:19:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.001+0000 7f3458f28740 20 client.25847 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.001+0000 7f3458f28740 1 --2- 172.21.15.22:0/2779272543 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56309b5250b0 0x56309b5274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.002+0000 7f3458f28740 4 client.25847 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:19:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.002+0000 7f3458f28740 1 -- 172.21.15.22:0/2779272543 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56309b4747f0 con 0x56309b5250b0 2024-09-19T08:19:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.002+0000 7f34548eb640 1 --2- 172.21.15.22:0/2779272543 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56309b5250b0 0x56309b5274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.002+0000 7f34548eb640 1 --2- 172.21.15.22:0/2779272543 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56309b5250b0 0x56309b5274d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac64e9397887dd7 server_cookie=502ff656b8ecd15a in_seq=0 out_seq=0 2024-09-19T08:19:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.002+0000 7f342ffff640 10 client.25847 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.003+0000 7f342ffff640 1 -- 172.21.15.22:0/2779272543 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56309b4747f0 con 0x56309b5250b0 2024-09-19T08:19:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.003+0000 7f342ffff640 10 client.25847 handle_command_reply: tid=0 2024-09-19T08:19:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.003+0000 7f3458f28740 1 client.25847 shutdown 2024-09-19T08:19:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.003+0000 7f3458f28740 1 -- 172.21.15.22:0/2779272543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b5116c0 msgr2=0x56309b515730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.003+0000 7f3458f28740 1 --2- 172.21.15.22:0/2779272543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b5116c0 0x56309b515730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f344801e630 tx=0x7f34480046c0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.003+0000 7f3458f28740 1 -- 172.21.15.22:0/2779272543 shutdown_connections 2024-09-19T08:19:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.003+0000 7f3458f28740 1 --2- 172.21.15.22:0/2779272543 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56309b5250b0 0x56309b5274d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.003+0000 7f3458f28740 1 --2- 172.21.15.22:0/2779272543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b5116c0 0x56309b515730 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.003+0000 7f3458f28740 1 --2- 172.21.15.22:0/2779272543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b510da0 0x56309b5151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.003+0000 7f3458f28740 1 --2- 172.21.15.22:0/2779272543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b50bd60 0x56309b514cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.003+0000 7f3458f28740 1 -- 172.21.15.22:0/2779272543 >> 172.21.15.22:0/2779272543 conn(0x56309b4fdba0 msgr2=0x56309b4fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.003+0000 7f342ffff640 0 client.25847 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.003+0000 7f3458f28740 1 -- 172.21.15.22:0/2779272543 shutdown_connections 2024-09-19T08:19:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.004+0000 7f3458f28740 1 -- 172.21.15.22:0/2779272543 wait complete. 2024-09-19T08:19:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.004+0000 7f3458f28740 20 client.25847 trim_cache size 0 max 16384 2024-09-19T08:19:25.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f3458f28740 1 Processor -- start 2024-09-19T08:19:25.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f3458f28740 1 -- start start 2024-09-19T08:19:25.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f3458f28740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b50bd60 0x56309b50b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f3458f28740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b510da0 0x56309b515580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f3458f28740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b5116c0 0x56309b516b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f3458f28740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f34501adf40 con 0x56309b5116c0 2024-09-19T08:19:25.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f3458f28740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f34501ab7b0 con 0x56309b510da0 2024-09-19T08:19:25.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f3458f28740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34501a9600 con 0x56309b50bd60 2024-09-19T08:19:25.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f344ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b510da0 0x56309b515580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f34548eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b50bd60 0x56309b50b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f34550ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b5116c0 0x56309b516b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f344ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b510da0 0x56309b515580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:36114/0 (socket says 172.21.15.22:36114) 2024-09-19T08:19:25.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f34548eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b50bd60 0x56309b50b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:56130/0 (socket says 172.21.15.22:56130) 2024-09-19T08:19:25.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f344ffff640 1 -- 172.21.15.22:0/3839040224 learned_addr learned my addr 172.21.15.22:0/3839040224 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:25.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f34550ec640 1 -- 172.21.15.22:0/3839040224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b50bd60 msgr2=0x56309b50b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:25.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f34550ec640 1 --2- 172.21.15.22:0/3839040224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b50bd60 0x56309b50b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f34550ec640 1 -- 172.21.15.22:0/3839040224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b510da0 msgr2=0x56309b515580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:25.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f34550ec640 1 --2- 172.21.15.22:0/3839040224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b510da0 0x56309b515580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f34550ec640 1 -- 172.21.15.22:0/3839040224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56309b513200 con 0x56309b5116c0 2024-09-19T08:19:25.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.022+0000 7f344ffff640 1 --2- 172.21.15.22:0/3839040224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b510da0 0x56309b515580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:25.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.023+0000 7f34550ec640 1 --2- 172.21.15.22:0/3839040224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b5116c0 0x56309b516b10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3448003cb0 tx=0x7f34480045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ed4da7b3587f4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.023+0000 7f34367fc640 1 -- 172.21.15.22:0/3839040224 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448004020 con 0x56309b5116c0 2024-09-19T08:19:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.023+0000 7f34367fc640 1 -- 172.21.15.22:0/3839040224 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34480041c0 con 0x56309b5116c0 2024-09-19T08:19:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.023+0000 7f34367fc640 1 -- 172.21.15.22:0/3839040224 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448004990 con 0x56309b5116c0 2024-09-19T08:19:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.024+0000 7f3458f28740 1 -- 172.21.15.22:0/3839040224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b5116c0 msgr2=0x56309b516b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.024+0000 7f3458f28740 1 --2- 172.21.15.22:0/3839040224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b5116c0 0x56309b516b10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3448003cb0 tx=0x7f34480045e0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.024+0000 7f3458f28740 1 -- 172.21.15.22:0/3839040224 shutdown_connections 2024-09-19T08:19:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.024+0000 7f3458f28740 1 --2- 172.21.15.22:0/3839040224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b5116c0 0x56309b516b10 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.024+0000 7f3458f28740 1 --2- 172.21.15.22:0/3839040224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b510da0 0x56309b515580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.024+0000 7f3458f28740 1 --2- 172.21.15.22:0/3839040224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b50bd60 0x56309b50b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.024+0000 7f3458f28740 1 -- 172.21.15.22:0/3839040224 >> 172.21.15.22:0/3839040224 conn(0x56309b4fdba0 msgr2=0x56309b516400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.024+0000 7f3458f28740 1 -- 172.21.15.22:0/3839040224 shutdown_connections 2024-09-19T08:19:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.024+0000 7f3458f28740 1 -- 172.21.15.22:0/3839040224 wait complete. 2024-09-19T08:19:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.024+0000 7f3458f28740 1 -- start start 2024-09-19T08:19:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.025+0000 7f3458f28740 1 Processor -- start 2024-09-19T08:19:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.025+0000 7f3458f28740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b50bd60 0x56309b52ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.025+0000 7f3458f28740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b510da0 0x56309b52cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.025+0000 7f3458f28740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b5116c0 0x56309b52d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.025+0000 7f3458f28740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f34501a9c70 con 0x56309b510da0 2024-09-19T08:19:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.025+0000 7f3458f28740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f34501ab0e0 con 0x56309b5116c0 2024-09-19T08:19:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.025+0000 7f3458f28740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34501aa340 con 0x56309b50bd60 2024-09-19T08:19:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.025+0000 7f3458f28740 10 client.0 _ll_register_callbacks cb 0x7f3448013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.025+0000 7f34548eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b50bd60 0x56309b52ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.025+0000 7f34548eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b50bd60 0x56309b52ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:56146/0 (socket says 172.21.15.22:56146) 2024-09-19T08:19:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.025+0000 7f344ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b510da0 0x56309b52cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.025+0000 7f34548eb640 1 -- 172.21.15.22:0/2760649023 learned_addr learned my addr 172.21.15.22:0/2760649023 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.026+0000 7f34550ec640 1 --2- 172.21.15.22:0/2760649023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b5116c0 0x56309b52d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.026+0000 7f34548eb640 1 -- 172.21.15.22:0/2760649023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b5116c0 msgr2=0x56309b52d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:25.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.026+0000 7f34548eb640 1 --2- 172.21.15.22:0/2760649023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b5116c0 0x56309b52d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.026+0000 7f34548eb640 1 -- 172.21.15.22:0/2760649023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b510da0 msgr2=0x56309b52cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:25.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.026+0000 7f34548eb640 1 --2- 172.21.15.22:0/2760649023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b510da0 0x56309b52cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.026+0000 7f34548eb640 1 -- 172.21.15.22:0/2760649023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56309b58ee10 con 0x56309b50bd60 2024-09-19T08:19:25.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.026+0000 7f34548eb640 1 --2- 172.21.15.22:0/2760649023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b50bd60 0x56309b52ca20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3440009a90 tx=0x7f3440004350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d4058e0a215962b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:25.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.026+0000 7f342ffff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:19:25.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.026+0000 7f342ffff640 1 -- 172.21.15.22:0/2760649023 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440005020 con 0x56309b50bd60 2024-09-19T08:19:25.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.026+0000 7f342ffff640 1 -- 172.21.15.22:0/2760649023 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f34400051c0 con 0x56309b50bd60 2024-09-19T08:19:25.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.026+0000 7f3458f28740 1 -- 172.21.15.22:0/2760649023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34480041c0 con 0x56309b50bd60 2024-09-19T08:19:25.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.026+0000 7f342ffff640 1 -- 172.21.15.22:0/2760649023 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f34400053c0 con 0x56309b50bd60 2024-09-19T08:19:25.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.026+0000 7f3458f28740 10 client.25795 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:25.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.026+0000 7f3458f28740 1 -- 172.21.15.22:0/2760649023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56309b590190 con 0x56309b50bd60 2024-09-19T08:19:25.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.027+0000 7f342ffff640 1 -- 172.21.15.22:0/2760649023 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34480041c0 con 0x56309b50bd60 2024-09-19T08:19:25.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.027+0000 7f342ffff640 1 -- 172.21.15.22:0/2760649023 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f34400205a0 con 0x56309b50bd60 2024-09-19T08:19:25.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.027+0000 7f3458f28740 10 client.25795 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:25.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.027+0000 7f3458f28740 10 client.25795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:19:25.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.027+0000 7f3458f28740 20 client.25795 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:25.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.027+0000 7f3458f28740 1 --2- 172.21.15.22:0/2760649023 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56309b5908e0 0x56309b592d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:25.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.027+0000 7f3458f28740 4 client.25795 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:19:25.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.027+0000 7f3458f28740 1 -- 172.21.15.22:0/2760649023 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56309b46ed00 con 0x56309b5908e0 2024-09-19T08:19:25.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.027+0000 7f344ffff640 1 --2- 172.21.15.22:0/2760649023 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56309b5908e0 0x56309b592d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:25.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.028+0000 7f342ffff640 10 client.25795 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:25.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.028+0000 7f344ffff640 1 --2- 172.21.15.22:0/2760649023 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56309b5908e0 0x56309b592d00 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60fa4004171a38e0 server_cookie=b805d09c2b712cb1 in_seq=0 out_seq=0 2024-09-19T08:19:25.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.029+0000 7f342ffff640 1 -- 172.21.15.22:0/2760649023 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56309b46ed00 con 0x56309b5908e0 2024-09-19T08:19:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.029+0000 7f342ffff640 10 client.25795 handle_command_reply: tid=0 2024-09-19T08:19:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.029+0000 7f3458f28740 1 client.25795 shutdown 2024-09-19T08:19:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.029+0000 7f3458f28740 1 -- 172.21.15.22:0/2760649023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b50bd60 msgr2=0x56309b52ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.029+0000 7f3458f28740 1 --2- 172.21.15.22:0/2760649023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b50bd60 0x56309b52ca20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3440009a90 tx=0x7f3440004350 comp rx=0 tx=0).stop 2024-09-19T08:19:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.029+0000 7f3458f28740 1 -- 172.21.15.22:0/2760649023 shutdown_connections 2024-09-19T08:19:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.029+0000 7f3458f28740 1 --2- 172.21.15.22:0/2760649023 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56309b5908e0 0x56309b592d00 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.029+0000 7f3458f28740 1 --2- 172.21.15.22:0/2760649023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56309b5116c0 0x56309b52d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.029+0000 7f3458f28740 1 --2- 172.21.15.22:0/2760649023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56309b510da0 0x56309b52cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.029+0000 7f342ffff640 0 client.25795 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.029+0000 7f3458f28740 1 --2- 172.21.15.22:0/2760649023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56309b50bd60 0x56309b52ca20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.029+0000 7f3458f28740 1 -- 172.21.15.22:0/2760649023 >> 172.21.15.22:0/2760649023 conn(0x56309b4fdba0 msgr2=0x56309b4fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.029+0000 7f3458f28740 1 -- 172.21.15.22:0/2760649023 shutdown_connections 2024-09-19T08:19:25.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.029+0000 7f3458f28740 1 -- 172.21.15.22:0/2760649023 wait complete. 2024-09-19T08:19:25.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.030+0000 7f3458f28740 20 client.25795 trim_cache size 0 max 16384 2024-09-19T08:19:25.034 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:19:25.034 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-19T08:19:25.034 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "3891546b-3bbd-41df-a8a9-195fb42e7eb6", 2024-09-19T08:19:25.034 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-19T08:19:25.034 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:19:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.030+0000 7f3456b4f640 1 -- 172.21.15.22:0/218925551 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f34380a33b0 msgr2=0x7f34380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.030+0000 7f3456b4f640 1 --2- 172.21.15.22:0/218925551 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f34380a33b0 0x7f34380a5870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3448009ce0 tx=0x7f3448005f90 comp rx=0 tx=0).stop 2024-09-19T08:19:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.030+0000 7f3456b4f640 1 -- 172.21.15.22:0/218925551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450102570 msgr2=0x7f3450198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.030+0000 7f3456b4f640 1 --2- 172.21.15.22:0/218925551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450102570 0x7f3450198a20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3444005cc0 tx=0x7f344402f420 comp rx=0 tx=0).stop 2024-09-19T08:19:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.030+0000 7f3456b4f640 1 -- 172.21.15.22:0/218925551 shutdown_connections 2024-09-19T08:19:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.030+0000 7f3456b4f640 1 --2- 172.21.15.22:0/218925551 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f34380a33b0 0x7f34380a5870 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.030+0000 7f3456b4f640 1 --2- 172.21.15.22:0/218925551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450102ef0 0x7f345019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.030+0000 7f3456b4f640 1 --2- 172.21.15.22:0/218925551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3450102570 0x7f3450198a20 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.030+0000 7f3456b4f640 1 --2- 172.21.15.22:0/218925551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3450100e20 0x7f34501984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.030+0000 7f3456b4f640 1 -- 172.21.15.22:0/218925551 >> 172.21.15.22:0/218925551 conn(0x7f34500fc7d0 msgr2=0x7f345006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.031+0000 7f3456b4f640 1 -- 172.21.15.22:0/218925551 shutdown_connections 2024-09-19T08:19:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:25.031+0000 7f3456b4f640 1 -- 172.21.15.22:0/218925551 wait complete. 2024-09-19T08:19:25.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:24 smithi064 ceph-mon[34900]: pgmap v134: 97 pgs: 97 active+clean; 574 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 623 op/s 2024-09-19T08:19:25.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:19:26.155 INFO:tasks.workunit.client.0.smithi022.stdout: 106 1348871 545 837329 328 712795 1091 2024-09-19T08:19:26.212 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:19:27.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:26 smithi157 ceph-mon[35143]: scrub queued for path: / 2024-09-19T08:19:27.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:26 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:27.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:26 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:19:27.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:26 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:19:27.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:26 smithi157 ceph-mon[35143]: pgmap v135: 97 pgs: 97 active+clean; 574 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.2 MiB/s wr, 562 op/s 2024-09-19T08:19:27.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:26 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:27.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:26 smithi022 ceph-mon[30963]: scrub queued for path: / 2024-09-19T08:19:27.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:26 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:27.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:26 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:19:27.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:26 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:19:27.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:26 smithi022 ceph-mon[30963]: pgmap v135: 97 pgs: 97 active+clean; 574 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.2 MiB/s wr, 562 op/s 2024-09-19T08:19:27.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:26 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:27.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:26 smithi064 ceph-mon[34900]: scrub queued for path: / 2024-09-19T08:19:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:26 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:26 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:19:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:26 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:19:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:26 smithi064 ceph-mon[34900]: pgmap v135: 97 pgs: 97 active+clean; 574 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.2 MiB/s wr, 562 op/s 2024-09-19T08:19:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:26 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:27.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.129+0000 7fa261075640 1 -- 172.21.15.22:0/3490821161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa25c1082e0 msgr2=0x7fa25c10ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:27.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.129+0000 7fa261075640 1 --2- 172.21.15.22:0/3490821161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa25c1082e0 0x7fa25c10ec50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa254009cd0 tx=0x7fa25402a8e0 comp rx=0 tx=0).stop 2024-09-19T08:19:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.129+0000 7fa261075640 1 -- 172.21.15.22:0/3490821161 shutdown_connections 2024-09-19T08:19:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.129+0000 7fa261075640 1 --2- 172.21.15.22:0/3490821161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa25c1082e0 0x7fa25c10ec50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.129+0000 7fa261075640 1 --2- 172.21.15.22:0/3490821161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25c103760 0x7fa25c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.129+0000 7fa261075640 1 --2- 172.21.15.22:0/3490821161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa25c102db0 0x7fa25c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.129+0000 7fa261075640 1 -- 172.21.15.22:0/3490821161 >> 172.21.15.22:0/3490821161 conn(0x7fa25c0fc7d0 msgr2=0x7fa25c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.129+0000 7fa261075640 1 -- 172.21.15.22:0/3490821161 shutdown_connections 2024-09-19T08:19:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.129+0000 7fa261075640 1 -- 172.21.15.22:0/3490821161 wait complete. 2024-09-19T08:19:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.130+0000 7fa261075640 1 Processor -- start 2024-09-19T08:19:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.130+0000 7fa261075640 1 -- start start 2024-09-19T08:19:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.130+0000 7fa261075640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25c102db0 0x7fa25c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.130+0000 7fa261075640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa25c103760 0x7fa25c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.130+0000 7fa261075640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa25c1082e0 0x7fa25c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.130+0000 7fa261075640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c1120b0 con 0x7fa25c1082e0 2024-09-19T08:19:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.130+0000 7fa261075640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c111f30 con 0x7fa25c102db0 2024-09-19T08:19:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.130+0000 7fa261075640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c112230 con 0x7fa25c103760 2024-09-19T08:19:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.130+0000 7fa25ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25c102db0 0x7fa25c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.130+0000 7fa25a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa25c103760 0x7fa25c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.130+0000 7fa25ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25c102db0 0x7fa25c198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:36136/0 (socket says 172.21.15.22:36136) 2024-09-19T08:19:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.130+0000 7fa25ad9d640 1 -- 172.21.15.22:0/1655680728 learned_addr learned my addr 172.21.15.22:0/1655680728 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.130+0000 7fa25b59e640 1 --2- 172.21.15.22:0/1655680728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa25c1082e0 0x7fa25c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.131+0000 7fa25ad9d640 1 -- 172.21.15.22:0/1655680728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa25c103760 msgr2=0x7fa25c198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.131+0000 7fa25ad9d640 1 --2- 172.21.15.22:0/1655680728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa25c103760 0x7fa25c198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.131+0000 7fa25ad9d640 1 -- 172.21.15.22:0/1655680728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa25c1082e0 msgr2=0x7fa25c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.131+0000 7fa25ad9d640 1 --2- 172.21.15.22:0/1655680728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa25c1082e0 0x7fa25c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.131+0000 7fa25ad9d640 1 -- 172.21.15.22:0/1655680728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25c19d510 con 0x7fa25c102db0 2024-09-19T08:19:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.131+0000 7fa25a59c640 1 --2- 172.21.15.22:0/1655680728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa25c103760 0x7fa25c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.131+0000 7fa25ad9d640 1 --2- 172.21.15.22:0/1655680728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25c102db0 0x7fa25c198560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00e9f0 tx=0x7fa24c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.131+0000 7fa253fff640 1 -- 172.21.15.22:0/1655680728 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c00cd80 con 0x7fa25c102db0 2024-09-19T08:19:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.131+0000 7fa253fff640 1 -- 172.21.15.22:0/1655680728 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24c010910 con 0x7fa25c102db0 2024-09-19T08:19:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.131+0000 7fa261075640 1 -- 172.21.15.22:0/1655680728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25c19d800 con 0x7fa25c102db0 2024-09-19T08:19:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.131+0000 7fa253fff640 1 -- 172.21.15.22:0/1655680728 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c018990 con 0x7fa25c102db0 2024-09-19T08:19:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.131+0000 7fa261075640 1 -- 172.21.15.22:0/1655680728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa25c1a50e0 con 0x7fa25c102db0 2024-09-19T08:19:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.132+0000 7fa261075640 1 -- 172.21.15.22:0/1655680728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa25c103190 con 0x7fa25c102db0 2024-09-19T08:19:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.134+0000 7fa253fff640 1 -- 172.21.15.22:0/1655680728 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa24c010ac0 con 0x7fa25c102db0 2024-09-19T08:19:27.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.135+0000 7fa253fff640 1 --2- 172.21.15.22:0/1655680728 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa2480a33b0 0x7fa2480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:27.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.135+0000 7fa25a59c640 1 --2- 172.21.15.22:0/1655680728 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa2480a33b0 0x7fa2480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:27.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.135+0000 7fa253fff640 1 -- 172.21.15.22:0/1655680728 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa24c0d0460 con 0x7fa25c102db0 2024-09-19T08:19:27.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.135+0000 7fa25a59c640 1 --2- 172.21.15.22:0/1655680728 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa2480a33b0 0x7fa2480a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0fee30 tx=0x7fa244005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:27.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.138+0000 7fa253fff640 1 -- 172.21.15.22:0/1655680728 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa24c09c800 con 0x7fa25c102db0 2024-09-19T08:19:27.248 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:19:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.408+0000 7fa261075640 1 -- 172.21.15.22:0/1655680728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa25c10a2c0 con 0x7fa25c102db0 2024-09-19T08:19:27.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.409+0000 7fa253fff640 1 -- 172.21.15.22:0/1655680728 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11909 (secure 0 0 0) 0x7fa24c0a0b00 con 0x7fa25c102db0 2024-09-19T08:19:27.409 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:19:27.410 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":29,"btime":"2024-09-19T08:18:57:303741+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:18:56.870055+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:19:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 29 2024-09-19T08:19:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.412+0000 7fa261075640 1 -- 172.21.15.22:0/1655680728 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa2480a33b0 msgr2=0x7fa2480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.412+0000 7fa261075640 1 --2- 172.21.15.22:0/1655680728 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa2480a33b0 0x7fa2480a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0fee30 tx=0x7fa244005f90 comp rx=0 tx=0).stop 2024-09-19T08:19:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.412+0000 7fa261075640 1 -- 172.21.15.22:0/1655680728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25c102db0 msgr2=0x7fa25c198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.412+0000 7fa261075640 1 --2- 172.21.15.22:0/1655680728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25c102db0 0x7fa25c198560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00e9f0 tx=0x7fa24c00eeb0 comp rx=0 tx=0).stop 2024-09-19T08:19:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.412+0000 7fa261075640 1 -- 172.21.15.22:0/1655680728 shutdown_connections 2024-09-19T08:19:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.412+0000 7fa261075640 1 --2- 172.21.15.22:0/1655680728 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa2480a33b0 0x7fa2480a5870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.412+0000 7fa261075640 1 --2- 172.21.15.22:0/1655680728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa25c1082e0 0x7fa25c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.412+0000 7fa261075640 1 --2- 172.21.15.22:0/1655680728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa25c103760 0x7fa25c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.412+0000 7fa261075640 1 --2- 172.21.15.22:0/1655680728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa25c102db0 0x7fa25c198560 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.412+0000 7fa261075640 1 -- 172.21.15.22:0/1655680728 >> 172.21.15.22:0/1655680728 conn(0x7fa25c0fc7d0 msgr2=0x7fa25c105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.412+0000 7fa261075640 1 -- 172.21.15.22:0/1655680728 shutdown_connections 2024-09-19T08:19:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:27.412+0000 7fa261075640 1 -- 172.21.15.22:0/1655680728 wait complete. 2024-09-19T08:19:27.929 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:27 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1655680728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:19:28.002 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:27 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1655680728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:19:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.013+0000 7f52b70d6640 1 -- 172.21.15.22:0/2439679168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52b010c180 msgr2=0x7f52b010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.013+0000 7f52b70d6640 1 --2- 172.21.15.22:0/2439679168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52b010c180 0x7f52b010e600 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009ca0 tx=0x7f52a802a950 comp rx=0 tx=0).stop 2024-09-19T08:19:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.013+0000 7f52b70d6640 1 -- 172.21.15.22:0/2439679168 shutdown_connections 2024-09-19T08:19:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.013+0000 7f52b70d6640 1 --2- 172.21.15.22:0/2439679168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52b010c180 0x7f52b010e600 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.013+0000 7f52b70d6640 1 --2- 172.21.15.22:0/2439679168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b01036c0 0x7f52b010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.013+0000 7f52b70d6640 1 --2- 172.21.15.22:0/2439679168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b0102da0 0x7f52b0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.013+0000 7f52b70d6640 1 -- 172.21.15.22:0/2439679168 >> 172.21.15.22:0/2439679168 conn(0x7f52b00fc7a0 msgr2=0x7f52b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.013+0000 7f52b70d6640 1 -- 172.21.15.22:0/2439679168 shutdown_connections 2024-09-19T08:19:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.013+0000 7f52b70d6640 1 -- 172.21.15.22:0/2439679168 wait complete. 2024-09-19T08:19:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.014+0000 7f52b70d6640 1 Processor -- start 2024-09-19T08:19:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.014+0000 7f52b70d6640 1 -- start start 2024-09-19T08:19:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.014+0000 7f52b70d6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b0102da0 0x7f52b0100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.014+0000 7f52b70d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b01036c0 0x7f52b0100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.014+0000 7f52b70d6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52b010c180 0x7f52b0101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.014+0000 7f52b70d6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0111f60 con 0x7f52b010c180 2024-09-19T08:19:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.014+0000 7f52b70d6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0111de0 con 0x7f52b01036c0 2024-09-19T08:19:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.014+0000 7f52b70d6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01120e0 con 0x7f52b0102da0 2024-09-19T08:19:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.014+0000 7f52b5673640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52b010c180 0x7f52b0101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.014+0000 7f52b5673640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52b010c180 0x7f52b0101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43328/0 (socket says 172.21.15.22:43328) 2024-09-19T08:19:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.014+0000 7f52b5673640 1 -- 172.21.15.22:0/977257027 learned_addr learned my addr 172.21.15.22:0/977257027 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.015+0000 7f52affff640 1 --2- 172.21.15.22:0/977257027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b01036c0 0x7f52b0100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.015+0000 7f52b4e72640 1 --2- 172.21.15.22:0/977257027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b0102da0 0x7f52b0100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.015+0000 7f52b5673640 1 -- 172.21.15.22:0/977257027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b0102da0 msgr2=0x7f52b0100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.015+0000 7f52b5673640 1 --2- 172.21.15.22:0/977257027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b0102da0 0x7f52b0100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.015+0000 7f52b5673640 1 -- 172.21.15.22:0/977257027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b01036c0 msgr2=0x7f52b0100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.015+0000 7f52b5673640 1 --2- 172.21.15.22:0/977257027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b01036c0 0x7f52b0100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.015+0000 7f52b5673640 1 -- 172.21.15.22:0/977257027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b01a6880 con 0x7f52b010c180 2024-09-19T08:19:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.015+0000 7f52b5673640 1 --2- 172.21.15.22:0/977257027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52b010c180 0x7f52b0101480 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f52a802ae70 tx=0x7f52a802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.015+0000 7f52adffb640 1 -- 172.21.15.22:0/977257027 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8034040 con 0x7f52b010c180 2024-09-19T08:19:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.015+0000 7f52adffb640 1 -- 172.21.15.22:0/977257027 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a8030920 con 0x7f52b010c180 2024-09-19T08:19:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.015+0000 7f52b70d6640 1 -- 172.21.15.22:0/977257027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b01a6b10 con 0x7f52b010c180 2024-09-19T08:19:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.015+0000 7f52adffb640 1 -- 172.21.15.22:0/977257027 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8045480 con 0x7f52b010c180 2024-09-19T08:19:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.015+0000 7f52b70d6640 1 -- 172.21.15.22:0/977257027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b01a6ff0 con 0x7f52b010c180 2024-09-19T08:19:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.017+0000 7f52adffb640 1 -- 172.21.15.22:0/977257027 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f52a8037020 con 0x7f52b010c180 2024-09-19T08:19:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.018+0000 7f52b94af740 1 Processor -- start 2024-09-19T08:19:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.018+0000 7f52b94af740 1 -- start start 2024-09-19T08:19:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.018+0000 7f52adffb640 1 --2- 172.21.15.22:0/977257027 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f52a00a33b0 0x7f52a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.018+0000 7f52b94af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b0024a0 0x55836b002880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.018+0000 7f52b94af740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b002dc0 0x55836b00d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.018+0000 7f52b94af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b00fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.018+0000 7f52b94af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55836af682e0 con 0x55836b00d680 2024-09-19T08:19:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.018+0000 7f52b94af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55836af0fce0 con 0x55836b0024a0 2024-09-19T08:19:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.018+0000 7f52b94af740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55836af642a0 con 0x55836b002dc0 2024-09-19T08:19:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.018+0000 7f52affff640 1 --2- 172.21.15.22:0/977257027 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f52a00a33b0 0x7f52a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.018+0000 7f52adffb640 1 -- 172.21.15.22:0/977257027 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f52a80f0bf0 con 0x7f52b010c180 2024-09-19T08:19:28.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.018+0000 7f52b5673640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b00fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.018+0000 7f52affff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b002dc0 0x55836b00d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f52b5673640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b00fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43338/0 (socket says 172.21.15.22:43338) 2024-09-19T08:19:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f52b5673640 1 -- 172.21.15.22:0/3940897256 learned_addr learned my addr 172.21.15.22:0/3940897256 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f52b4e72640 1 --2- 172.21.15.22:0/3940897256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b0024a0 0x55836b002880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f52b5673640 1 -- 172.21.15.22:0/3940897256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b002dc0 msgr2=0x55836b00d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f52b5673640 1 --2- 172.21.15.22:0/3940897256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b002dc0 0x55836b00d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f52b5673640 1 -- 172.21.15.22:0/3940897256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b0024a0 msgr2=0x55836b002880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f52b5673640 1 --2- 172.21.15.22:0/3940897256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b0024a0 0x55836b002880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f52b5673640 1 -- 172.21.15.22:0/3940897256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55836aef4440 con 0x55836b00d680 2024-09-19T08:19:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f52affff640 1 --2- 172.21.15.22:0/3940897256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b002dc0 0x55836b00d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f52b4e72640 1 --2- 172.21.15.22:0/3940897256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b0024a0 0x55836b002880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f52affff640 1 --2- 172.21.15.22:0/977257027 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f52a00a33b0 0x7f52a00a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f529c0058c0 tx=0x7f529c00caf0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:28.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f52b5673640 1 --2- 172.21.15.22:0/3940897256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b00fa70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f52a80b99c0 tx=0x7f52a802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823b12c5dfb0ceb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f529affd640 1 -- 172.21.15.22:0/3940897256 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a80098d0 con 0x55836b00d680 2024-09-19T08:19:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f529affd640 1 -- 172.21.15.22:0/3940897256 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a80dd0d0 con 0x55836b00d680 2024-09-19T08:19:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.019+0000 7f529affd640 1 -- 172.21.15.22:0/3940897256 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a80c5c60 con 0x55836b00d680 2024-09-19T08:19:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.020+0000 7f52b94af740 1 -- 172.21.15.22:0/3940897256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 msgr2=0x55836b00fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.020+0000 7f52b94af740 1 --2- 172.21.15.22:0/3940897256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b00fa70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f52a80b99c0 tx=0x7f52a802a950 comp rx=0 tx=0).stop 2024-09-19T08:19:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.020+0000 7f52b94af740 1 -- 172.21.15.22:0/3940897256 shutdown_connections 2024-09-19T08:19:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.020+0000 7f52b94af740 1 --2- 172.21.15.22:0/3940897256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b00fa70 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.020+0000 7f52b94af740 1 --2- 172.21.15.22:0/3940897256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b002dc0 0x55836b00d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.020+0000 7f52b94af740 1 --2- 172.21.15.22:0/3940897256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b0024a0 0x55836b002880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.020+0000 7f52b94af740 1 -- 172.21.15.22:0/3940897256 >> 172.21.15.22:0/3940897256 conn(0x55836affea30 msgr2=0x55836affee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.020+0000 7f52b94af740 1 -- 172.21.15.22:0/3940897256 shutdown_connections 2024-09-19T08:19:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.020+0000 7f52b94af740 1 -- 172.21.15.22:0/3940897256 wait complete. 2024-09-19T08:19:28.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.020+0000 7f52b94af740 1 -- start start 2024-09-19T08:19:28.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.020+0000 7f52b94af740 1 Processor -- start 2024-09-19T08:19:28.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b94af740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b0024a0 0x55836b00cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b94af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b002dc0 0x55836b00a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b94af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b00a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b94af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55836b010010 con 0x55836b00d680 2024-09-19T08:19:28.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b94af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55836af23050 con 0x55836b002dc0 2024-09-19T08:19:28.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b94af740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55836b010190 con 0x55836b0024a0 2024-09-19T08:19:28.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b94af740 10 client.0 _ll_register_callbacks cb 0x55836af63160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:28.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b5673640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b00a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b5673640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b00a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43344/0 (socket says 172.21.15.22:43344) 2024-09-19T08:19:28.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b5673640 1 -- 172.21.15.22:0/3796143755 learned_addr learned my addr 172.21.15.22:0/3796143755 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:28.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52affff640 1 --2- 172.21.15.22:0/3796143755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b002dc0 0x55836b00a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b4e72640 1 --2- 172.21.15.22:0/3796143755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b0024a0 0x55836b00cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b5673640 1 -- 172.21.15.22:0/3796143755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b0024a0 msgr2=0x55836b00cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:28.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b5673640 1 --2- 172.21.15.22:0/3796143755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b0024a0 0x55836b00cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b5673640 1 -- 172.21.15.22:0/3796143755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b002dc0 msgr2=0x55836b00a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:28.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b5673640 1 --2- 172.21.15.22:0/3796143755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b002dc0 0x55836b00a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52b5673640 1 -- 172.21.15.22:0/3796143755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55836acd93c0 con 0x55836b00d680 2024-09-19T08:19:28.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.021+0000 7f52affff640 1 --2- 172.21.15.22:0/3796143755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b002dc0 0x55836b00a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:19:28.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f52b5673640 1 --2- 172.21.15.22:0/3796143755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b00a800 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f52a80b9450 tx=0x7f52a8004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955fbc7c2b229641 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:28.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f528ffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:19:28.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f528ffff640 1 -- 172.21.15.22:0/3796143755 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a80bd450 con 0x55836b00d680 2024-09-19T08:19:28.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f528ffff640 1 -- 172.21.15.22:0/3796143755 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f52a80c57d0 con 0x55836b00d680 2024-09-19T08:19:28.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f52b94af740 1 -- 172.21.15.22:0/3796143755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55836af75850 con 0x55836b00d680 2024-09-19T08:19:28.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f528ffff640 1 -- 172.21.15.22:0/3796143755 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52a80098d0 con 0x55836b00d680 2024-09-19T08:19:28.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f52b94af740 10 client.16278 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:28.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f52b94af740 1 -- 172.21.15.22:0/3796143755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55836b010010 con 0x55836b00d680 2024-09-19T08:19:28.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f528ffff640 1 -- 172.21.15.22:0/3796143755 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55836af75850 con 0x55836b00d680 2024-09-19T08:19:28.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f528ffff640 1 -- 172.21.15.22:0/3796143755 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f52a80d9030 con 0x55836b00d680 2024-09-19T08:19:28.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f52b94af740 10 client.16278 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:28.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f52b94af740 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:19:28.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f52b94af740 20 client.16278 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:28.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f52b94af740 1 --2- 172.21.15.22:0/3796143755 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55836b01cb90 0x55836b01efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f52b94af740 4 client.16278 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:19:28.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.022+0000 7f52b94af740 1 -- 172.21.15.22:0/3796143755 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55836af6fd60 con 0x55836b01cb90 2024-09-19T08:19:28.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.023+0000 7f52b4e72640 1 --2- 172.21.15.22:0/3796143755 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55836b01cb90 0x55836b01efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.023+0000 7f528ffff640 10 client.16278 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:28.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.023+0000 7f52b4e72640 1 --2- 172.21.15.22:0/3796143755 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55836b01cb90 0x55836b01efb0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58045c03ac5b0cb server_cookie=42e19b53c7b3ebd6 in_seq=0 out_seq=0 2024-09-19T08:19:28.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f528ffff640 1 -- 172.21.15.22:0/3796143755 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55836af6fd60 con 0x55836b01cb90 2024-09-19T08:19:28.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f528ffff640 10 client.16278 handle_command_reply: tid=0 2024-09-19T08:19:28.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f52b94af740 1 client.16278 shutdown 2024-09-19T08:19:28.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f52b94af740 1 -- 172.21.15.22:0/3796143755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 msgr2=0x55836b00a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:28.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f52b94af740 1 --2- 172.21.15.22:0/3796143755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b00a800 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f52a80b9450 tx=0x7f52a8004950 comp rx=0 tx=0).stop 2024-09-19T08:19:28.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f52b94af740 1 -- 172.21.15.22:0/3796143755 shutdown_connections 2024-09-19T08:19:28.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f52b94af740 1 --2- 172.21.15.22:0/3796143755 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55836b01cb90 0x55836b01efb0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f52b94af740 1 --2- 172.21.15.22:0/3796143755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b00a800 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f52b94af740 1 --2- 172.21.15.22:0/3796143755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b002dc0 0x55836b00a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f52b94af740 1 --2- 172.21.15.22:0/3796143755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b0024a0 0x55836b00cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f528ffff640 0 client.16278 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:28.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f52b94af740 1 -- 172.21.15.22:0/3796143755 >> 172.21.15.22:0/3796143755 conn(0x55836affea30 msgr2=0x55836b004710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:28.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f52b94af740 1 -- 172.21.15.22:0/3796143755 shutdown_connections 2024-09-19T08:19:28.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f52b94af740 1 -- 172.21.15.22:0/3796143755 wait complete. 2024-09-19T08:19:28.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.024+0000 7f52b94af740 20 client.16278 trim_cache size 0 max 16384 2024-09-19T08:19:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.041+0000 7f52b94af740 1 Processor -- start 2024-09-19T08:19:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52b94af740 1 -- start start 2024-09-19T08:19:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52b94af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b0024a0 0x55836b0191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52b94af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b002dc0 0x55836b024860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52b94af740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b00d680 0x55836b024da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52b94af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01b2270 con 0x55836b002dc0 2024-09-19T08:19:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52b94af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01afae0 con 0x55836b0024a0 2024-09-19T08:19:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52b94af740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01ad930 con 0x55836b00d680 2024-09-19T08:19:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52affff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b002dc0 0x55836b024860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52affff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b002dc0 0x55836b024860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43350/0 (socket says 172.21.15.22:43350) 2024-09-19T08:19:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52affff640 1 -- 172.21.15.22:0/2304286439 learned_addr learned my addr 172.21.15.22:0/2304286439 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52b5673640 1 --2- 172.21.15.22:0/2304286439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b00d680 0x55836b024da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52b4e72640 1 --2- 172.21.15.22:0/2304286439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b0024a0 0x55836b0191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52affff640 1 -- 172.21.15.22:0/2304286439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b00d680 msgr2=0x55836b024da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52affff640 1 --2- 172.21.15.22:0/2304286439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b00d680 0x55836b024da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52affff640 1 -- 172.21.15.22:0/2304286439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b0024a0 msgr2=0x55836b0191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52affff640 1 --2- 172.21.15.22:0/2304286439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b0024a0 0x55836b0191e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52affff640 1 -- 172.21.15.22:0/2304286439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55836b00ab70 con 0x55836b002dc0 2024-09-19T08:19:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.042+0000 7f52affff640 1 --2- 172.21.15.22:0/2304286439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b002dc0 0x55836b024860 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f529c00cfd0 tx=0x7f529c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96dee833ad442155 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.043+0000 7f529a7fc640 1 -- 172.21.15.22:0/2304286439 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c011bb0 con 0x55836b002dc0 2024-09-19T08:19:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.043+0000 7f529a7fc640 1 -- 172.21.15.22:0/2304286439 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f529c011d50 con 0x55836b002dc0 2024-09-19T08:19:28.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.043+0000 7f529a7fc640 1 -- 172.21.15.22:0/2304286439 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c012740 con 0x55836b002dc0 2024-09-19T08:19:28.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.043+0000 7f52b94af740 1 -- 172.21.15.22:0/2304286439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b002dc0 msgr2=0x55836b024860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:28.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.043+0000 7f52b94af740 1 --2- 172.21.15.22:0/2304286439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b002dc0 0x55836b024860 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f529c00cfd0 tx=0x7f529c004730 comp rx=0 tx=0).stop 2024-09-19T08:19:28.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.043+0000 7f52b94af740 1 -- 172.21.15.22:0/2304286439 shutdown_connections 2024-09-19T08:19:28.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.043+0000 7f52b94af740 1 --2- 172.21.15.22:0/2304286439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b00d680 0x55836b024da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.043+0000 7f52b94af740 1 --2- 172.21.15.22:0/2304286439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b002dc0 0x55836b024860 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.043+0000 7f52b94af740 1 --2- 172.21.15.22:0/2304286439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b0024a0 0x55836b0191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.043+0000 7f52b94af740 1 -- 172.21.15.22:0/2304286439 >> 172.21.15.22:0/2304286439 conn(0x55836affea30 msgr2=0x55836b00c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:28.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.043+0000 7f52b94af740 1 -- 172.21.15.22:0/2304286439 shutdown_connections 2024-09-19T08:19:28.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.044+0000 7f52b94af740 1 -- 172.21.15.22:0/2304286439 wait complete. 2024-09-19T08:19:28.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.044+0000 7f52b94af740 1 -- start start 2024-09-19T08:19:28.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.044+0000 7f52b94af740 1 Processor -- start 2024-09-19T08:19:28.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.044+0000 7f52b94af740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b0024a0 0x55836b015c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.044+0000 7f52b94af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b002dc0 0x55836b010f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.044+0000 7f52b94af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b011720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.044+0000 7f52b94af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01adfa0 con 0x55836b00d680 2024-09-19T08:19:28.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.044+0000 7f52b94af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01af410 con 0x55836b002dc0 2024-09-19T08:19:28.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.044+0000 7f52b94af740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01ae670 con 0x55836b0024a0 2024-09-19T08:19:28.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.044+0000 7f52b94af740 10 client.0 _ll_register_callbacks cb 0x7f52a400d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:28.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f52b4e72640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b0024a0 0x55836b015c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f52b5673640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b011720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f52b4e72640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b0024a0 0x55836b015c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:47080/0 (socket says 172.21.15.22:47080) 2024-09-19T08:19:28.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f52b4e72640 1 -- 172.21.15.22:0/1940090020 learned_addr learned my addr 172.21.15.22:0/1940090020 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:28.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f52affff640 1 --2- 172.21.15.22:0/1940090020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b002dc0 0x55836b010f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f52b5673640 1 -- 172.21.15.22:0/1940090020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b0024a0 msgr2=0x55836b015c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:28.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f52b5673640 1 --2- 172.21.15.22:0/1940090020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b0024a0 0x55836b015c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f52b5673640 1 -- 172.21.15.22:0/1940090020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b002dc0 msgr2=0x55836b010f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:28.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f52b5673640 1 --2- 172.21.15.22:0/1940090020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b002dc0 0x55836b010f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f52b5673640 1 -- 172.21.15.22:0/1940090020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55836b086830 con 0x55836b00d680 2024-09-19T08:19:28.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f52b5673640 1 --2- 172.21.15.22:0/1940090020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b011720 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f52a80bcf80 tx=0x7f52a8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23bfb4996eff359 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:28.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f528ffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:19:28.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f528ffff640 1 -- 172.21.15.22:0/1940090020 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a80bcb30 con 0x55836b00d680 2024-09-19T08:19:28.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f528ffff640 1 -- 172.21.15.22:0/1940090020 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f52a80bc490 con 0x55836b00d680 2024-09-19T08:19:28.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f52b94af740 1 -- 172.21.15.22:0/1940090020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f529c011d50 con 0x55836b00d680 2024-09-19T08:19:28.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f528ffff640 1 -- 172.21.15.22:0/1940090020 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52a80bd6f0 con 0x55836b00d680 2024-09-19T08:19:28.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f52b94af740 10 client.16284 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:28.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.045+0000 7f52b94af740 1 -- 172.21.15.22:0/1940090020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55836b086830 con 0x55836b00d680 2024-09-19T08:19:28.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.046+0000 7f528ffff640 1 -- 172.21.15.22:0/1940090020 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f529c011d50 con 0x55836b00d680 2024-09-19T08:19:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.046+0000 7f528ffff640 1 -- 172.21.15.22:0/1940090020 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f52a80a2050 con 0x55836b00d680 2024-09-19T08:19:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.046+0000 7f52b94af740 10 client.16284 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.046+0000 7f52b94af740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:19:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.046+0000 7f52b94af740 20 client.16284 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.046+0000 7f52b94af740 1 --2- 172.21.15.22:0/1940090020 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55836b087d00 0x55836b08a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.046+0000 7f52b94af740 4 client.16284 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:19:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.046+0000 7f52b94af740 1 -- 172.21.15.22:0/1940090020 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55836aeec210 con 0x55836b087d00 2024-09-19T08:19:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.046+0000 7f52b4e72640 1 --2- 172.21.15.22:0/1940090020 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55836b087d00 0x55836b08a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f528ffff640 10 client.16284 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.046+0000 7f52b4e72640 1 --2- 172.21.15.22:0/1940090020 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55836b087d00 0x55836b08a0f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46bac99807285d16 server_cookie=5241cad3f1d07841 in_seq=0 out_seq=0 2024-09-19T08:19:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f528ffff640 1 -- 172.21.15.22:0/1940090020 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55836aeec210 con 0x55836b087d00 2024-09-19T08:19:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f528ffff640 10 client.16284 handle_command_reply: tid=0 2024-09-19T08:19:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f52b94af740 1 client.16284 shutdown 2024-09-19T08:19:28.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f52b94af740 1 -- 172.21.15.22:0/1940090020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 msgr2=0x55836b011720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:28.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f52b94af740 1 --2- 172.21.15.22:0/1940090020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b011720 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f52a80bcf80 tx=0x7f52a8004360 comp rx=0 tx=0).stop 2024-09-19T08:19:28.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f52b94af740 1 -- 172.21.15.22:0/1940090020 shutdown_connections 2024-09-19T08:19:28.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f52b94af740 1 --2- 172.21.15.22:0/1940090020 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55836b087d00 0x55836b08a0f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f52b94af740 1 --2- 172.21.15.22:0/1940090020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55836b00d680 0x55836b011720 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f52b94af740 1 --2- 172.21.15.22:0/1940090020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55836b002dc0 0x55836b010f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f52b94af740 1 --2- 172.21.15.22:0/1940090020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55836b0024a0 0x55836b015c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f528ffff640 0 client.16284 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:28.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f52b94af740 1 -- 172.21.15.22:0/1940090020 >> 172.21.15.22:0/1940090020 conn(0x55836affea30 msgr2=0x55836b009b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:28.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f52b94af740 1 -- 172.21.15.22:0/1940090020 shutdown_connections 2024-09-19T08:19:28.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f52b94af740 1 -- 172.21.15.22:0/1940090020 wait complete. 2024-09-19T08:19:28.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.047+0000 7f52b94af740 20 client.16284 trim_cache size 0 max 16384 2024-09-19T08:19:28.059 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:19:28.059 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:19:28.059 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-19T08:19:28.059 INFO:teuthology.orchestra.run.smithi022.stdout: "3891546b-3bbd-41df-a8a9-195fb42e7eb6": { 2024-09-19T08:19:28.059 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-19T08:19:28.059 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "3891546b-3bbd-41df-a8a9-195fb42e7eb6", 2024-09-19T08:19:28.059 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-09-19T08:19:28.059 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:19:28.059 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:19:28.059 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:19:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.048+0000 7f52b70d6640 1 -- 172.21.15.22:0/977257027 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f52a00a33b0 msgr2=0x7f52a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.048+0000 7f52b70d6640 1 --2- 172.21.15.22:0/977257027 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f52a00a33b0 0x7f52a00a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f529c0058c0 tx=0x7f529c00caf0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.048+0000 7f52b70d6640 1 -- 172.21.15.22:0/977257027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52b010c180 msgr2=0x7f52b0101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.048+0000 7f52b70d6640 1 --2- 172.21.15.22:0/977257027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52b010c180 0x7f52b0101480 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f52a802ae70 tx=0x7f52a802f5b0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.048+0000 7f52b70d6640 1 -- 172.21.15.22:0/977257027 shutdown_connections 2024-09-19T08:19:28.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.048+0000 7f52b70d6640 1 --2- 172.21.15.22:0/977257027 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f52a00a33b0 0x7f52a00a5870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.048+0000 7f52b70d6640 1 --2- 172.21.15.22:0/977257027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52b010c180 0x7f52b0101480 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.048+0000 7f52b70d6640 1 --2- 172.21.15.22:0/977257027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52b01036c0 0x7f52b0100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.048+0000 7f52b70d6640 1 --2- 172.21.15.22:0/977257027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b0102da0 0x7f52b0100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:28.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.048+0000 7f52b70d6640 1 -- 172.21.15.22:0/977257027 >> 172.21.15.22:0/977257027 conn(0x7f52b00fc7a0 msgr2=0x7f52b00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:28.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.048+0000 7f52b70d6640 1 -- 172.21.15.22:0/977257027 shutdown_connections 2024-09-19T08:19:28.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:28.048+0000 7f52b70d6640 1 -- 172.21.15.22:0/977257027 wait complete. 2024-09-19T08:19:28.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:27 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1655680728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:19:28.883 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T08:19:28.883 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T08:19:29.002 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:28 smithi157 ceph-mon[35143]: pgmap v136: 97 pgs: 97 active+clean; 582 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.9 MiB/s wr, 618 op/s 2024-09-19T08:19:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:28 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:19:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:28 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:28 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:19:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:28 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:29.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:28 smithi022 ceph-mon[30963]: pgmap v136: 97 pgs: 97 active+clean; 582 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.9 MiB/s wr, 618 op/s 2024-09-19T08:19:29.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:28 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:19:29.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:28 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:29.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:28 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:19:29.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:28 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:29.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:28 smithi064 ceph-mon[34900]: pgmap v136: 97 pgs: 97 active+clean; 582 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.9 MiB/s wr, 618 op/s 2024-09-19T08:19:29.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:28 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:19:29.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:28 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:29.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:28 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:19:29.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:28 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:29.661 INFO:tasks.cephfs.filesystem:scrub status for tag:3891546b-3bbd-41df-a8a9-195fb42e7eb6 - {'path': '/', 'tag': '3891546b-3bbd-41df-a8a9-195fb42e7eb6', 'options': 'recursive,force'} 2024-09-19T08:19:29.704 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:19:29.826 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:29 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:19:29.827 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:29 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:29.827 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:29 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:19:29.827 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:29 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:30.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:29 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:19:30.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:29 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:30.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:29 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:19:30.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:29 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:30.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:29 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:19:30.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:29 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:30.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:29 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:19:30.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:29 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.239+0000 7f5084790640 1 -- 172.21.15.22:0/2109892546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101420 msgr2=0x7f507c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.239+0000 7f5084790640 1 --2- 172.21.15.22:0/2109892546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101420 0x7f507c10d950 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5074009cd0 tx=0x7f507402a8e0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.239+0000 7f5084790640 1 -- 172.21.15.22:0/2109892546 shutdown_connections 2024-09-19T08:19:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.239+0000 7f5084790640 1 --2- 172.21.15.22:0/2109892546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101420 0x7f507c10d950 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.239+0000 7f5084790640 1 --2- 172.21.15.22:0/2109892546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f507c100a80 0x7f507c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.239+0000 7f5084790640 1 --2- 172.21.15.22:0/2109892546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f507c106a80 0x7f507c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.239+0000 7f5084790640 1 -- 172.21.15.22:0/2109892546 >> 172.21.15.22:0/2109892546 conn(0x7f507c0fc7a0 msgr2=0x7f507c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.239+0000 7f5084790640 1 -- 172.21.15.22:0/2109892546 shutdown_connections 2024-09-19T08:19:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.239+0000 7f5084790640 1 -- 172.21.15.22:0/2109892546 wait complete. 2024-09-19T08:19:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.240+0000 7f5084790640 1 Processor -- start 2024-09-19T08:19:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.240+0000 7f5084790640 1 -- start start 2024-09-19T08:19:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.240+0000 7f5084790640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f507c100a80 0x7f507c197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.240+0000 7f5084790640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101420 0x7f507c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.240+0000 7f5084790640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f507c106a80 0x7f507c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.240+0000 7f5084790640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f507c110e80 con 0x7f507c101420 2024-09-19T08:19:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.240+0000 7f5084790640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f507c110d00 con 0x7f507c100a80 2024-09-19T08:19:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.240+0000 7f5084790640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f507c111000 con 0x7f507c106a80 2024-09-19T08:19:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.240+0000 7f5081d2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101420 0x7f507c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.240+0000 7f5081d2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101420 0x7f507c197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43384/0 (socket says 172.21.15.22:43384) 2024-09-19T08:19:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.240+0000 7f5082d2d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f507c106a80 0x7f507c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.240+0000 7f5081d2b640 1 -- 172.21.15.22:0/1563350304 learned_addr learned my addr 172.21.15.22:0/1563350304 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.241+0000 7f508252c640 1 --2- 172.21.15.22:0/1563350304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f507c100a80 0x7f507c197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.241+0000 7f5081d2b640 1 -- 172.21.15.22:0/1563350304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f507c106a80 msgr2=0x7f507c19bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.241+0000 7f5081d2b640 1 --2- 172.21.15.22:0/1563350304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f507c106a80 0x7f507c19bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.241+0000 7f5081d2b640 1 -- 172.21.15.22:0/1563350304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f507c100a80 msgr2=0x7f507c197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.241+0000 7f5081d2b640 1 --2- 172.21.15.22:0/1563350304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f507c100a80 0x7f507c197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.241+0000 7f5081d2b640 1 -- 172.21.15.22:0/1563350304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507c19c2d0 con 0x7f507c101420 2024-09-19T08:19:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.241+0000 7f5082d2d640 1 --2- 172.21.15.22:0/1563350304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f507c106a80 0x7f507c19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.241+0000 7f5081d2b640 1 --2- 172.21.15.22:0/1563350304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101420 0x7f507c197860 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f506c00d950 tx=0x7f506c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.241+0000 7f50737fe640 1 -- 172.21.15.22:0/1563350304 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c017030 con 0x7f507c101420 2024-09-19T08:19:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.241+0000 7f5084790640 1 -- 172.21.15.22:0/1563350304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507c19c5c0 con 0x7f507c101420 2024-09-19T08:19:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.241+0000 7f50737fe640 1 -- 172.21.15.22:0/1563350304 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f506c004020 con 0x7f507c101420 2024-09-19T08:19:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.241+0000 7f50737fe640 1 -- 172.21.15.22:0/1563350304 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c0047a0 con 0x7f507c101420 2024-09-19T08:19:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.241+0000 7f5084790640 1 -- 172.21.15.22:0/1563350304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f507c06a790 con 0x7f507c101420 2024-09-19T08:19:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.243+0000 7f50737fe640 1 -- 172.21.15.22:0/1563350304 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f506c0041d0 con 0x7f507c101420 2024-09-19T08:19:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.244+0000 7f5086b69740 1 Processor -- start 2024-09-19T08:19:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.244+0000 7f5086b69740 1 -- start start 2024-09-19T08:19:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.244+0000 7f5086b69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2640 0x55898f2d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.244+0000 7f5086b69740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2f60 0x55898f2e1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.244+0000 7f5086b69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2e1d50 0x55898f2e4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.244+0000 7f5086b69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55898f2382a0 con 0x55898f2d2640 2024-09-19T08:19:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.244+0000 7f5086b69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55898f1dfb10 con 0x55898f2e1d50 2024-09-19T08:19:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.244+0000 7f5086b69740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55898f234260 con 0x55898f2d2f60 2024-09-19T08:19:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.244+0000 7f50737fe640 1 --2- 172.21.15.22:0/1563350304 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f50680a33b0 0x7f50680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f508252c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2f60 0x55898f2e1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f5082d2d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2640 0x55898f2d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f508252c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2f60 0x55898f2e1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:47096/0 (socket says 172.21.15.22:47096) 2024-09-19T08:19:30.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f508252c640 1 -- 172.21.15.22:0/44190127 learned_addr learned my addr 172.21.15.22:0/44190127 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:30.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f5081d2b640 1 --2- 172.21.15.22:0/44190127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2e1d50 0x55898f2e4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f5082d2d640 1 --2- 172.21.15.22:0/1563350304 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f50680a33b0 0x7f50680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f50737fe640 1 -- 172.21.15.22:0/1563350304 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f506c0cf700 con 0x7f507c101420 2024-09-19T08:19:30.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f508252c640 1 -- 172.21.15.22:0/44190127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2e1d50 msgr2=0x55898f2e4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:30.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f508252c640 1 --2- 172.21.15.22:0/44190127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2e1d50 0x55898f2e4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f508252c640 1 -- 172.21.15.22:0/44190127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2640 msgr2=0x55898f2d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:30.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f508252c640 1 --2- 172.21.15.22:0/44190127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2640 0x55898f2d2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f508252c640 1 -- 172.21.15.22:0/44190127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55898efa9b90 con 0x55898f2d2f60 2024-09-19T08:19:30.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f508252c640 1 --2- 172.21.15.22:0/44190127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2f60 0x55898f2e1810 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5078009c00 tx=0x7f50780117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f68b7b40a46877d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f5082d2d640 1 --2- 172.21.15.22:0/1563350304 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f50680a33b0 0x7f50680a5870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5074009430 tx=0x7f50740317e0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f5070ff9640 1 -- 172.21.15.22:0/44190127 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078012450 con 0x55898f2d2f60 2024-09-19T08:19:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f5070ff9640 1 -- 172.21.15.22:0/44190127 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5078012ad0 con 0x55898f2d2f60 2024-09-19T08:19:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.245+0000 7f5070ff9640 1 -- 172.21.15.22:0/44190127 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507800fb80 con 0x55898f2d2f60 2024-09-19T08:19:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.246+0000 7f5086b69740 1 -- 172.21.15.22:0/44190127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2f60 msgr2=0x55898f2e1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.246+0000 7f5086b69740 1 --2- 172.21.15.22:0/44190127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2f60 0x55898f2e1810 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5078009c00 tx=0x7f50780117b0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.246+0000 7f5086b69740 1 -- 172.21.15.22:0/44190127 shutdown_connections 2024-09-19T08:19:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.246+0000 7f5086b69740 1 --2- 172.21.15.22:0/44190127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2e1d50 0x55898f2e4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.246+0000 7f5086b69740 1 --2- 172.21.15.22:0/44190127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2f60 0x55898f2e1810 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.246+0000 7f5086b69740 1 --2- 172.21.15.22:0/44190127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2640 0x55898f2d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.246+0000 7f5086b69740 1 -- 172.21.15.22:0/44190127 >> 172.21.15.22:0/44190127 conn(0x55898f2cecf0 msgr2=0x55898f2cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.247+0000 7f5086b69740 1 -- 172.21.15.22:0/44190127 shutdown_connections 2024-09-19T08:19:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.247+0000 7f5086b69740 1 -- 172.21.15.22:0/44190127 wait complete. 2024-09-19T08:19:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.247+0000 7f5086b69740 1 -- start start 2024-09-19T08:19:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.247+0000 7f5086b69740 1 Processor -- start 2024-09-19T08:19:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.247+0000 7f5086b69740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2640 0x55898f2dec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.247+0000 7f5086b69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2f60 0x55898f2df150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.247+0000 7f5086b69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2e1d50 0x55898f2df690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.247+0000 7f5086b69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55898f2e4920 con 0x55898f2d2f60 2024-09-19T08:19:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.247+0000 7f5086b69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55898f1c06e0 con 0x55898f2e1d50 2024-09-19T08:19:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.247+0000 7f5086b69740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55898f2e4aa0 con 0x55898f2d2640 2024-09-19T08:19:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.247+0000 7f5086b69740 10 client.0 _ll_register_callbacks cb 0x55898f1e4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f5082d2d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2e1d50 0x55898f2df690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f508252c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2640 0x55898f2dec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f5082d2d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2e1d50 0x55898f2df690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:35706/0 (socket says 172.21.15.22:35706) 2024-09-19T08:19:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f508252c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2640 0x55898f2dec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:47110/0 (socket says 172.21.15.22:47110) 2024-09-19T08:19:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f5082d2d640 1 -- 172.21.15.22:0/1191876059 learned_addr learned my addr 172.21.15.22:0/1191876059 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f5081d2b640 1 --2- 172.21.15.22:0/1191876059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2f60 0x55898f2df150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f5082d2d640 1 -- 172.21.15.22:0/1191876059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2640 msgr2=0x55898f2dec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f5082d2d640 1 --2- 172.21.15.22:0/1191876059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2640 0x55898f2dec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f5082d2d640 1 -- 172.21.15.22:0/1191876059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2f60 msgr2=0x55898f2df150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f5082d2d640 1 --2- 172.21.15.22:0/1191876059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2f60 0x55898f2df150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f5082d2d640 1 -- 172.21.15.22:0/1191876059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55898f2ede00 con 0x55898f2e1d50 2024-09-19T08:19:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f5081d2b640 1 --2- 172.21.15.22:0/1191876059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2f60 0x55898f2df150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f508252c640 1 --2- 172.21.15.22:0/1191876059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2640 0x55898f2dec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f5082d2d640 1 --2- 172.21.15.22:0/1191876059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2e1d50 0x55898f2df690 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5074009cd0 tx=0x7f50740095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f28fd624b82ca0a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f5065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:19:30.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.248+0000 7f5065ffb640 1 -- 172.21.15.22:0/1191876059 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074051070 con 0x55898f2e1d50 2024-09-19T08:19:30.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.249+0000 7f5086b69740 1 -- 172.21.15.22:0/1191876059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5078012ad0 con 0x55898f2e1d50 2024-09-19T08:19:30.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.249+0000 7f5065ffb640 1 -- 172.21.15.22:0/1191876059 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f50740309b0 con 0x55898f2e1d50 2024-09-19T08:19:30.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.249+0000 7f5065ffb640 1 -- 172.21.15.22:0/1191876059 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5074030bb0 con 0x55898f2e1d50 2024-09-19T08:19:30.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.249+0000 7f5086b69740 10 client.25889 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:30.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.249+0000 7f5086b69740 1 -- 172.21.15.22:0/1191876059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55898f2f07e0 con 0x55898f2e1d50 2024-09-19T08:19:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.249+0000 7f5065ffb640 1 -- 172.21.15.22:0/1191876059 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5078012ad0 con 0x55898f2e1d50 2024-09-19T08:19:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.249+0000 7f5065ffb640 1 -- 172.21.15.22:0/1191876059 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5074030070 con 0x55898f2e1d50 2024-09-19T08:19:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.249+0000 7f5086b69740 10 client.25889 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.249+0000 7f5086b69740 10 client.25889 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:19:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.249+0000 7f5086b69740 20 client.25889 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.249+0000 7f5086b69740 1 --2- 172.21.15.22:0/1191876059 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55898f2f1bf0 0x55898f2f4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.249+0000 7f5086b69740 4 client.25889 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:19:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.249+0000 7f5086b69740 1 -- 172.21.15.22:0/1191876059 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55898f2459d0 con 0x55898f2f1bf0 2024-09-19T08:19:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.250+0000 7f508252c640 1 --2- 172.21.15.22:0/1191876059 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55898f2f1bf0 0x55898f2f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.251+0000 7f5065ffb640 10 client.25889 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.251+0000 7f508252c640 1 --2- 172.21.15.22:0/1191876059 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55898f2f1bf0 0x55898f2f4040 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45d9f570399a909f server_cookie=6b2e18a791f28d1a in_seq=0 out_seq=0 2024-09-19T08:19:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5065ffb640 1 -- 172.21.15.22:0/1191876059 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55898f2459d0 con 0x55898f2f1bf0 2024-09-19T08:19:30.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5065ffb640 10 client.25889 handle_command_reply: tid=0 2024-09-19T08:19:30.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5086b69740 1 client.25889 shutdown 2024-09-19T08:19:30.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5086b69740 1 -- 172.21.15.22:0/1191876059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2e1d50 msgr2=0x55898f2df690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:30.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5086b69740 1 --2- 172.21.15.22:0/1191876059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2e1d50 0x55898f2df690 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5074009cd0 tx=0x7f50740095c0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5086b69740 1 -- 172.21.15.22:0/1191876059 shutdown_connections 2024-09-19T08:19:30.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5086b69740 1 --2- 172.21.15.22:0/1191876059 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55898f2f1bf0 0x55898f2f4040 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5086b69740 1 --2- 172.21.15.22:0/1191876059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2e1d50 0x55898f2df690 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5086b69740 1 --2- 172.21.15.22:0/1191876059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2f60 0x55898f2df150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5086b69740 1 --2- 172.21.15.22:0/1191876059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2640 0x55898f2dec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5065ffb640 0 client.25889 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:30.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5086b69740 1 -- 172.21.15.22:0/1191876059 >> 172.21.15.22:0/1191876059 conn(0x55898f2cecf0 msgr2=0x55898f2d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:30.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5086b69740 1 -- 172.21.15.22:0/1191876059 shutdown_connections 2024-09-19T08:19:30.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5086b69740 1 -- 172.21.15.22:0/1191876059 wait complete. 2024-09-19T08:19:30.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.252+0000 7f5086b69740 20 client.25889 trim_cache size 0 max 16384 2024-09-19T08:19:30.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.269+0000 7f5086b69740 1 Processor -- start 2024-09-19T08:19:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.269+0000 7f5086b69740 1 -- start start 2024-09-19T08:19:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f5086b69740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2640 0x55898f2cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f5086b69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2d2f60 0x55898f2f9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f5086b69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2e1d50 0x55898f2f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f5086b69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f507c1ae000 con 0x55898f2e1d50 2024-09-19T08:19:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f5086b69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f507c1ab870 con 0x55898f2d2f60 2024-09-19T08:19:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f5086b69740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f507c1a96c0 con 0x55898f2d2640 2024-09-19T08:19:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f5082d2d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2e1d50 0x55898f2f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f508252c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2640 0x55898f2cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f5082d2d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2e1d50 0x55898f2f9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43410/0 (socket says 172.21.15.22:43410) 2024-09-19T08:19:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f508252c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2640 0x55898f2cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:47122/0 (socket says 172.21.15.22:47122) 2024-09-19T08:19:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f5082d2d640 1 -- 172.21.15.22:0/570334910 learned_addr learned my addr 172.21.15.22:0/570334910 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f5081d2b640 1 --2- 172.21.15.22:0/570334910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2d2f60 0x55898f2f9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f508252c640 1 -- 172.21.15.22:0/570334910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2d2f60 msgr2=0x55898f2f9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f508252c640 1 --2- 172.21.15.22:0/570334910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2d2f60 0x55898f2f9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f508252c640 1 -- 172.21.15.22:0/570334910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2e1d50 msgr2=0x55898f2f9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f508252c640 1 --2- 172.21.15.22:0/570334910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2e1d50 0x55898f2f9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f508252c640 1 -- 172.21.15.22:0/570334910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55898f2ca260 con 0x55898f2d2640 2024-09-19T08:19:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.270+0000 7f5082d2d640 1 --2- 172.21.15.22:0/570334910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2e1d50 0x55898f2f9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.271+0000 7f508252c640 1 --2- 172.21.15.22:0/570334910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2640 0x55898f2cdd60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f507800f760 tx=0x7f507800f680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4247f7677c68c501 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.271+0000 7f5067fff640 1 -- 172.21.15.22:0/570334910 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50780052e0 con 0x55898f2d2640 2024-09-19T08:19:30.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.271+0000 7f5067fff640 1 -- 172.21.15.22:0/570334910 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f507800fd30 con 0x55898f2d2640 2024-09-19T08:19:30.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.271+0000 7f5067fff640 1 -- 172.21.15.22:0/570334910 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078003830 con 0x55898f2d2640 2024-09-19T08:19:30.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.271+0000 7f5086b69740 1 -- 172.21.15.22:0/570334910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2640 msgr2=0x55898f2cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:30.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.271+0000 7f5086b69740 1 --2- 172.21.15.22:0/570334910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2640 0x55898f2cdd60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f507800f760 tx=0x7f507800f680 comp rx=0 tx=0).stop 2024-09-19T08:19:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.271+0000 7f5086b69740 1 -- 172.21.15.22:0/570334910 shutdown_connections 2024-09-19T08:19:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.271+0000 7f5086b69740 1 --2- 172.21.15.22:0/570334910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2e1d50 0x55898f2f9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.271+0000 7f5086b69740 1 --2- 172.21.15.22:0/570334910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2d2f60 0x55898f2f9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.271+0000 7f5086b69740 1 --2- 172.21.15.22:0/570334910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2d2640 0x55898f2cdd60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.271+0000 7f5086b69740 1 -- 172.21.15.22:0/570334910 >> 172.21.15.22:0/570334910 conn(0x55898f2cecf0 msgr2=0x55898f2cd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5086b69740 1 -- 172.21.15.22:0/570334910 shutdown_connections 2024-09-19T08:19:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5086b69740 1 -- 172.21.15.22:0/570334910 wait complete. 2024-09-19T08:19:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5086b69740 1 -- start start 2024-09-19T08:19:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5086b69740 1 Processor -- start 2024-09-19T08:19:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5086b69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2d2640 0x55898f2e59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5086b69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2f60 0x55898f2e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5086b69740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2e1d50 0x55898f2e6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5086b69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f507c1a9d30 con 0x55898f2d2f60 2024-09-19T08:19:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5086b69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f507c1ab1a0 con 0x55898f2d2640 2024-09-19T08:19:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5086b69740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f507c1aa400 con 0x55898f2e1d50 2024-09-19T08:19:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5081d2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2f60 0x55898f2e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5086b69740 10 client.0 _ll_register_callbacks cb 0x7f5074003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5081d2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2f60 0x55898f2e5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43418/0 (socket says 172.21.15.22:43418) 2024-09-19T08:19:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5081d2b640 1 -- 172.21.15.22:0/3594930483 learned_addr learned my addr 172.21.15.22:0/3594930483 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f5082d2d640 1 --2- 172.21.15.22:0/3594930483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2e1d50 0x55898f2e6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.272+0000 7f508252c640 1 --2- 172.21.15.22:0/3594930483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2d2640 0x55898f2e59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.273+0000 7f5081d2b640 1 -- 172.21.15.22:0/3594930483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2e1d50 msgr2=0x55898f2e6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.273+0000 7f5081d2b640 1 --2- 172.21.15.22:0/3594930483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2e1d50 0x55898f2e6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.273+0000 7f5081d2b640 1 -- 172.21.15.22:0/3594930483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2d2640 msgr2=0x55898f2e59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.273+0000 7f5081d2b640 1 --2- 172.21.15.22:0/3594930483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2d2640 0x55898f2e59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.273+0000 7f5081d2b640 1 -- 172.21.15.22:0/3594930483 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55898f2d99e0 con 0x55898f2d2f60 2024-09-19T08:19:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.273+0000 7f5081d2b640 1 --2- 172.21.15.22:0/3594930483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2f60 0x55898f2e5f30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f506c0ccf10 tx=0x7f506c06a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed0cd4fe16e09a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.273+0000 7f5065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:19:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.273+0000 7f5065ffb640 1 -- 172.21.15.22:0/3594930483 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c09b9f0 con 0x55898f2d2f60 2024-09-19T08:19:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.273+0000 7f5086b69740 1 -- 172.21.15.22:0/3594930483 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f507800fd30 con 0x55898f2d2f60 2024-09-19T08:19:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.273+0000 7f5065ffb640 1 -- 172.21.15.22:0/3594930483 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f506c098060 con 0x55898f2d2f60 2024-09-19T08:19:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.273+0000 7f5065ffb640 1 -- 172.21.15.22:0/3594930483 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f506c09fda0 con 0x55898f2d2f60 2024-09-19T08:19:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.273+0000 7f5086b69740 10 client.16302 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.273+0000 7f5086b69740 1 -- 172.21.15.22:0/3594930483 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55898f35c980 con 0x55898f2d2f60 2024-09-19T08:19:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.274+0000 7f5065ffb640 1 -- 172.21.15.22:0/3594930483 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f507800fd30 con 0x55898f2d2f60 2024-09-19T08:19:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.274+0000 7f5065ffb640 1 -- 172.21.15.22:0/3594930483 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f506c09ae40 con 0x55898f2d2f60 2024-09-19T08:19:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.274+0000 7f5086b69740 10 client.16302 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.274+0000 7f5086b69740 10 client.16302 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:19:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.274+0000 7f5086b69740 20 client.16302 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.274+0000 7f5086b69740 1 --2- 172.21.15.22:0/3594930483 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55898f35d070 0x55898f35f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.274+0000 7f5086b69740 4 client.16302 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T08:19:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.274+0000 7f5086b69740 1 -- 172.21.15.22:0/3594930483 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x55898f23fd20 con 0x55898f35d070 2024-09-19T08:19:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.274+0000 7f508252c640 1 --2- 172.21.15.22:0/3594930483 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55898f35d070 0x55898f35f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.274+0000 7f5065ffb640 10 client.16302 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:30.274+0000 7f508252c640 1 --2- 172.21.15.22:0/3594930483 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55898f35d070 0x55898f35f460 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5e56c2f7ab05c00 server_cookie=85277f38b38703f5 in_seq=0 out_seq=0 2024-09-19T08:19:31.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:30 smithi157 ceph-mon[35143]: pgmap v137: 97 pgs: 97 active+clean; 618 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.14k op/s 2024-09-19T08:19:31.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:30 smithi022 ceph-mon[30963]: pgmap v137: 97 pgs: 97 active+clean; 618 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.14k op/s 2024-09-19T08:19:31.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:30 smithi064 ceph-mon[34900]: pgmap v137: 97 pgs: 97 active+clean; 618 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.14k op/s 2024-09-19T08:19:33.002 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:32 smithi157 ceph-mon[35143]: pgmap v138: 97 pgs: 97 active+clean; 618 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.6 MiB/s wr, 1.07k op/s 2024-09-19T08:19:33.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:32 smithi022 ceph-mon[30963]: pgmap v138: 97 pgs: 97 active+clean; 618 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.6 MiB/s wr, 1.07k op/s 2024-09-19T08:19:33.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:32 smithi064 ceph-mon[34900]: pgmap v138: 97 pgs: 97 active+clean; 618 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.6 MiB/s wr, 1.07k op/s 2024-09-19T08:19:35.002 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:34 smithi157 ceph-mon[35143]: pgmap v139: 97 pgs: 97 active+clean; 600 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-09-19T08:19:35.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:34 smithi022 ceph-mon[30963]: pgmap v139: 97 pgs: 97 active+clean; 600 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-09-19T08:19:35.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:34 smithi064 ceph-mon[34900]: pgmap v139: 97 pgs: 97 active+clean; 600 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-09-19T08:19:36.155 INFO:tasks.workunit.client.0.smithi022.stdout: 108 206650 49 127344 160 109559 305 2024-09-19T08:19:37.002 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:36 smithi157 ceph-mon[35143]: pgmap v140: 97 pgs: 97 active+clean; 600 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 744 op/s 2024-09-19T08:19:37.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:36 smithi022 ceph-mon[30963]: pgmap v140: 97 pgs: 97 active+clean; 600 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 744 op/s 2024-09-19T08:19:37.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:36 smithi064 ceph-mon[34900]: pgmap v140: 97 pgs: 97 active+clean; 600 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 744 op/s 2024-09-19T08:19:39.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:38 smithi157 ceph-mon[35143]: pgmap v141: 97 pgs: 97 active+clean; 601 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 813 op/s 2024-09-19T08:19:39.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:38 smithi022 ceph-mon[30963]: pgmap v141: 97 pgs: 97 active+clean; 601 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 813 op/s 2024-09-19T08:19:39.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:38 smithi064 ceph-mon[34900]: pgmap v141: 97 pgs: 97 active+clean; 601 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 813 op/s 2024-09-19T08:19:41.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:40 smithi157 ceph-mon[35143]: pgmap v142: 97 pgs: 97 active+clean; 605 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 970 op/s 2024-09-19T08:19:41.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:40 smithi022 ceph-mon[30963]: pgmap v142: 97 pgs: 97 active+clean; 605 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 970 op/s 2024-09-19T08:19:41.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:40 smithi064 ceph-mon[34900]: pgmap v142: 97 pgs: 97 active+clean; 605 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 970 op/s 2024-09-19T08:19:43.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:42 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:43.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:42 smithi157 ceph-mon[35143]: pgmap v143: 97 pgs: 97 active+clean; 605 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 441 op/s 2024-09-19T08:19:43.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:42 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:43.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:42 smithi022 ceph-mon[30963]: pgmap v143: 97 pgs: 97 active+clean; 605 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 441 op/s 2024-09-19T08:19:43.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:42 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:19:43.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:42 smithi064 ceph-mon[34900]: pgmap v143: 97 pgs: 97 active+clean; 605 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 441 op/s 2024-09-19T08:19:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.603+0000 7f5065ffb640 1 -- 172.21.15.22:0/3594930483 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55898f23fd20 con 0x55898f35d070 2024-09-19T08:19:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.603+0000 7f5065ffb640 10 client.16302 handle_command_reply: tid=0 2024-09-19T08:19:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.603+0000 7f5086b69740 1 client.16302 shutdown 2024-09-19T08:19:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.604+0000 7f5086b69740 1 -- 172.21.15.22:0/3594930483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2f60 msgr2=0x55898f2e5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.604+0000 7f5086b69740 1 --2- 172.21.15.22:0/3594930483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2f60 0x55898f2e5f30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f506c0ccf10 tx=0x7f506c06a6e0 comp rx=0 tx=0).stop 2024-09-19T08:19:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.604+0000 7f5086b69740 1 -- 172.21.15.22:0/3594930483 shutdown_connections 2024-09-19T08:19:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.604+0000 7f5086b69740 1 --2- 172.21.15.22:0/3594930483 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55898f35d070 0x55898f35f460 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.604+0000 7f5086b69740 1 --2- 172.21.15.22:0/3594930483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898f2e1d50 0x55898f2e6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.604+0000 7f5086b69740 1 --2- 172.21.15.22:0/3594930483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55898f2d2f60 0x55898f2e5f30 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.604+0000 7f5086b69740 1 --2- 172.21.15.22:0/3594930483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55898f2d2640 0x55898f2e59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.604+0000 7f5086b69740 1 -- 172.21.15.22:0/3594930483 >> 172.21.15.22:0/3594930483 conn(0x55898f2cecf0 msgr2=0x55898f2df3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.604+0000 7f5065ffb640 0 client.16302 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.604+0000 7f5086b69740 1 -- 172.21.15.22:0/3594930483 shutdown_connections 2024-09-19T08:19:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.604+0000 7f5086b69740 1 -- 172.21.15.22:0/3594930483 wait complete. 2024-09-19T08:19:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.604+0000 7f5086b69740 20 client.16302 trim_cache size 0 max 16384 2024-09-19T08:19:44.607 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:19:44.607 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 29, 2024-09-19T08:19:44.607 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:19:44.607 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 18, 2024-09-19T08:19:44.607 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:19:44.607 INFO:teuthology.orchestra.run.smithi022.stdout: "69e267f5": { 2024-09-19T08:19:44.607 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 18, 2024-09-19T08:19:44.607 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:19:44.607 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:19:44.607 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:19:44.607 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:19:44.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:19:44.608 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:19:44.608 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 149.8, 2024-09-19T08:19:44.608 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:19:44.608 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:19:44.608 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:19:44.608 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:19:44.608 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:19:44.608 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:19:44.608 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:19:44.608 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:19:44.608 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:19:44.608 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:19:44.608 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:19:44.609 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:19:44.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.605+0000 7f5084790640 1 -- 172.21.15.22:0/1563350304 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f50680a33b0 msgr2=0x7f50680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:44.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.605+0000 7f5084790640 1 --2- 172.21.15.22:0/1563350304 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f50680a33b0 0x7f50680a5870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5074009430 tx=0x7f50740317e0 comp rx=0 tx=0).stop 2024-09-19T08:19:44.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.605+0000 7f5084790640 1 -- 172.21.15.22:0/1563350304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101420 msgr2=0x7f507c197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:44.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.605+0000 7f5084790640 1 --2- 172.21.15.22:0/1563350304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101420 0x7f507c197860 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f506c00d950 tx=0x7f506c00de10 comp rx=0 tx=0).stop 2024-09-19T08:19:44.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.605+0000 7f5084790640 1 -- 172.21.15.22:0/1563350304 shutdown_connections 2024-09-19T08:19:44.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.605+0000 7f5084790640 1 --2- 172.21.15.22:0/1563350304 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f50680a33b0 0x7f50680a5870 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:44.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.605+0000 7f5084790640 1 --2- 172.21.15.22:0/1563350304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f507c106a80 0x7f507c19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:44.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.605+0000 7f5084790640 1 --2- 172.21.15.22:0/1563350304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101420 0x7f507c197860 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:44.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.605+0000 7f5084790640 1 --2- 172.21.15.22:0/1563350304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f507c100a80 0x7f507c197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:44.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.605+0000 7f5084790640 1 -- 172.21.15.22:0/1563350304 >> 172.21.15.22:0/1563350304 conn(0x7f507c0fc7a0 msgr2=0x7f507c0fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:44.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.605+0000 7f5084790640 1 -- 172.21.15.22:0/1563350304 shutdown_connections 2024-09-19T08:19:44.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:44.606+0000 7f5084790640 1 -- 172.21.15.22:0/1563350304 wait complete. 2024-09-19T08:19:45.002 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:44 smithi157 ceph-mon[35143]: pgmap v144: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 763 op/s 2024-09-19T08:19:45.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:44 smithi022 ceph-mon[30963]: pgmap v144: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 763 op/s 2024-09-19T08:19:45.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:44 smithi064 ceph-mon[34900]: pgmap v144: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 763 op/s 2024-09-19T08:19:45.709 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '69e267f5', quiesce took 20.9 seconds. Will release after: 8.900000000000002 2024-09-19T08:19:46.155 INFO:tasks.workunit.client.0.smithi022.stdout: 108 409 0 316 0 498 0 2024-09-19T08:19:47.002 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:46 smithi157 ceph-mon[35143]: pgmap v145: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.3 MiB/s wr, 602 op/s 2024-09-19T08:19:47.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:46 smithi022 ceph-mon[30963]: pgmap v145: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.3 MiB/s wr, 602 op/s 2024-09-19T08:19:47.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:46 smithi064 ceph-mon[34900]: pgmap v145: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.3 MiB/s wr, 602 op/s 2024-09-19T08:19:48.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:47 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:19:48.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:47 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:19:48.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:47 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:19:48.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:47 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:19:48.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:47 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:19:48.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:47 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:19:49.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:48 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:19:49.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:48 smithi157 ceph-mon[35143]: pgmap v146: 97 pgs: 97 active+clean; 616 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 645 op/s 2024-09-19T08:19:49.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:48 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:19:49.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:48 smithi022 ceph-mon[30963]: pgmap v146: 97 pgs: 97 active+clean; 616 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 645 op/s 2024-09-19T08:19:49.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:48 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:19:49.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:48 smithi064 ceph-mon[34900]: pgmap v146: 97 pgs: 97 active+clean; 616 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 645 op/s 2024-09-19T08:19:51.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:50 smithi157 ceph-mon[35143]: pgmap v147: 97 pgs: 97 active+clean; 595 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 722 op/s 2024-09-19T08:19:51.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:50 smithi022 ceph-mon[30963]: pgmap v147: 97 pgs: 97 active+clean; 595 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 722 op/s 2024-09-19T08:19:51.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:50 smithi064 ceph-mon[34900]: pgmap v147: 97 pgs: 97 active+clean; 595 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 722 op/s 2024-09-19T08:19:53.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:52 smithi157 ceph-mon[35143]: pgmap v148: 97 pgs: 97 active+clean; 595 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 510 op/s 2024-09-19T08:19:53.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:52 smithi022 ceph-mon[30963]: pgmap v148: 97 pgs: 97 active+clean; 595 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 510 op/s 2024-09-19T08:19:53.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:52 smithi064 ceph-mon[34900]: pgmap v148: 97 pgs: 97 active+clean; 595 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 510 op/s 2024-09-19T08:19:54.611 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:19:54.878 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:19:55.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:54 smithi022 ceph-mon[30963]: pgmap v149: 97 pgs: 97 active+clean; 611 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 631 op/s 2024-09-19T08:19:55.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:54 smithi157 ceph-mon[35143]: pgmap v149: 97 pgs: 97 active+clean; 611 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 631 op/s 2024-09-19T08:19:55.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:54 smithi064 ceph-mon[34900]: pgmap v149: 97 pgs: 97 active+clean; 611 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 631 op/s 2024-09-19T08:19:55.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.387+0000 7fb6408b1640 1 -- 172.21.15.22:0/2603928506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c102790 msgr2=0x7fb63c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:55.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.387+0000 7fb6408b1640 1 --2- 172.21.15.22:0/2603928506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c102790 0x7fb63c1067e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb630009900 tx=0x7fb63002a410 comp rx=0 tx=0).stop 2024-09-19T08:19:55.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.387+0000 7fb6408b1640 1 -- 172.21.15.22:0/2603928506 shutdown_connections 2024-09-19T08:19:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.387+0000 7fb6408b1640 1 --2- 172.21.15.22:0/2603928506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c106f10 0x7fb63c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.387+0000 7fb6408b1640 1 --2- 172.21.15.22:0/2603928506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c102790 0x7fb63c1067e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.387+0000 7fb6408b1640 1 --2- 172.21.15.22:0/2603928506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c101de0 0x7fb63c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.387+0000 7fb6408b1640 1 -- 172.21.15.22:0/2603928506 >> 172.21.15.22:0/2603928506 conn(0x7fb63c0fd8c0 msgr2=0x7fb63c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.387+0000 7fb6408b1640 1 -- 172.21.15.22:0/2603928506 shutdown_connections 2024-09-19T08:19:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.387+0000 7fb6408b1640 1 -- 172.21.15.22:0/2603928506 wait complete. 2024-09-19T08:19:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.388+0000 7fb6408b1640 1 Processor -- start 2024-09-19T08:19:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.388+0000 7fb6408b1640 1 -- start start 2024-09-19T08:19:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.388+0000 7fb6408b1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c101de0 0x7fb63c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb6408b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c102790 0x7fb63c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb6408b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c106f10 0x7fb63c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb6408b1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c111f70 con 0x7fb63c106f10 2024-09-19T08:19:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb6408b1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c111df0 con 0x7fb63c102790 2024-09-19T08:19:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb6408b1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1120f0 con 0x7fb63c101de0 2024-09-19T08:19:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb63ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c106f10 0x7fb63c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb63a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c101de0 0x7fb63c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb63ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c106f10 0x7fb63c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42386/0 (socket says 172.21.15.22:42386) 2024-09-19T08:19:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb639d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c102790 0x7fb63c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb63ad9d640 1 -- 172.21.15.22:0/3182883328 learned_addr learned my addr 172.21.15.22:0/3182883328 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb63a59c640 1 -- 172.21.15.22:0/3182883328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c102790 msgr2=0x7fb63c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb63a59c640 1 --2- 172.21.15.22:0/3182883328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c102790 0x7fb63c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb63a59c640 1 -- 172.21.15.22:0/3182883328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c106f10 msgr2=0x7fb63c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb63a59c640 1 --2- 172.21.15.22:0/3182883328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c106f10 0x7fb63c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb63a59c640 1 -- 172.21.15.22:0/3182883328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63c1a6900 con 0x7fb63c101de0 2024-09-19T08:19:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb63ad9d640 1 --2- 172.21.15.22:0/3182883328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c106f10 0x7fb63c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:19:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb63a59c640 1 --2- 172.21.15.22:0/3182883328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c101de0 0x7fb63c0744f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00d6d0 tx=0x7fb62c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.389+0000 7fb62b7fe640 1 -- 172.21.15.22:0/3182883328 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c017070 con 0x7fb63c101de0 2024-09-19T08:19:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.390+0000 7fb6408b1640 1 -- 172.21.15.22:0/3182883328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63c1a6bf0 con 0x7fb63c101de0 2024-09-19T08:19:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.390+0000 7fb62b7fe640 1 -- 172.21.15.22:0/3182883328 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb62c009820 con 0x7fb63c101de0 2024-09-19T08:19:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.390+0000 7fb62b7fe640 1 -- 172.21.15.22:0/3182883328 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c004cd0 con 0x7fb63c101de0 2024-09-19T08:19:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.390+0000 7fb6408b1640 1 -- 172.21.15.22:0/3182883328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb63c1a7150 con 0x7fb63c101de0 2024-09-19T08:19:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.391+0000 7fb6408b1640 1 -- 172.21.15.22:0/3182883328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb63c1021c0 con 0x7fb63c101de0 2024-09-19T08:19:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.392+0000 7fb62b7fe640 1 -- 172.21.15.22:0/3182883328 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb62c0099d0 con 0x7fb63c101de0 2024-09-19T08:19:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.392+0000 7fb62b7fe640 1 --2- 172.21.15.22:0/3182883328 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb6240a33b0 0x7fb6240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:55.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.393+0000 7fb639d9b640 1 --2- 172.21.15.22:0/3182883328 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb6240a33b0 0x7fb6240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:55.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.393+0000 7fb62b7fe640 1 -- 172.21.15.22:0/3182883328 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb62c0cf790 con 0x7fb63c101de0 2024-09-19T08:19:55.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.393+0000 7fb639d9b640 1 --2- 172.21.15.22:0/3182883328 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb6240a33b0 0x7fb6240a5870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb630009230 tx=0x7fb6300058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:55.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.396+0000 7fb62b7fe640 1 -- 172.21.15.22:0/3182883328 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb62c09bb30 con 0x7fb63c101de0 2024-09-19T08:19:55.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.610+0000 7fb6408b1640 1 -- 172.21.15.22:0/3182883328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb63c10a150 con 0x7fb63c101de0 2024-09-19T08:19:55.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.611+0000 7fb62b7fe640 1 -- 172.21.15.22:0/3182883328 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11909 (secure 0 0 0) 0x7fb62c09fe30 con 0x7fb63c101de0 2024-09-19T08:19:55.612 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:19:55.612 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":29,"btime":"2024-09-19T08:18:57:303741+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:18:56.870055+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:19:55.614 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 29 2024-09-19T08:19:55.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.613+0000 7fb6408b1640 1 -- 172.21.15.22:0/3182883328 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb6240a33b0 msgr2=0x7fb6240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:55.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.613+0000 7fb6408b1640 1 --2- 172.21.15.22:0/3182883328 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb6240a33b0 0x7fb6240a5870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb630009230 tx=0x7fb6300058d0 comp rx=0 tx=0).stop 2024-09-19T08:19:55.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.613+0000 7fb6408b1640 1 -- 172.21.15.22:0/3182883328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c101de0 msgr2=0x7fb63c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:55.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.613+0000 7fb6408b1640 1 --2- 172.21.15.22:0/3182883328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c101de0 0x7fb63c0744f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00d6d0 tx=0x7fb62c00db90 comp rx=0 tx=0).stop 2024-09-19T08:19:55.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.613+0000 7fb6408b1640 1 -- 172.21.15.22:0/3182883328 shutdown_connections 2024-09-19T08:19:55.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.613+0000 7fb6408b1640 1 --2- 172.21.15.22:0/3182883328 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb6240a33b0 0x7fb6240a5870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:55.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.613+0000 7fb6408b1640 1 --2- 172.21.15.22:0/3182883328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c106f10 0x7fb63c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:55.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.613+0000 7fb6408b1640 1 --2- 172.21.15.22:0/3182883328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb63c102790 0x7fb63c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:55.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.613+0000 7fb6408b1640 1 --2- 172.21.15.22:0/3182883328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c101de0 0x7fb63c0744f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:55.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.613+0000 7fb6408b1640 1 -- 172.21.15.22:0/3182883328 >> 172.21.15.22:0/3182883328 conn(0x7fb63c0fd8c0 msgr2=0x7fb63c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:55.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.613+0000 7fb6408b1640 1 -- 172.21.15.22:0/3182883328 shutdown_connections 2024-09-19T08:19:55.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:55.613+0000 7fb6408b1640 1 -- 172.21.15.22:0/3182883328 wait complete. 2024-09-19T08:19:56.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:55 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3182883328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:19:56.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:55 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3182883328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:19:56.155 INFO:tasks.workunit.client.0.smithi022.stdout: 108 50544 0 32382 0 28155 0 2024-09-19T08:19:56.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:55 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3182883328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:19:56.588 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id 69e267f5 --release --await' 2024-09-19T08:19:56.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id 69e267f5 --release --await 2024-09-19T08:19:56.858 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:19:56.980 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:56 smithi022 ceph-mon[30963]: pgmap v150: 97 pgs: 97 active+clean; 611 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 309 op/s 2024-09-19T08:19:57.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:56 smithi064 ceph-mon[34900]: pgmap v150: 97 pgs: 97 active+clean; 611 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 309 op/s 2024-09-19T08:19:57.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:56 smithi157 ceph-mon[35143]: pgmap v150: 97 pgs: 97 active+clean; 611 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 309 op/s 2024-09-19T08:19:57.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.347+0000 7f3e4cadd640 1 -- 172.21.15.22:0/3992337516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e48102ef0 msgr2=0x7f3e4810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:57.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.347+0000 7f3e4cadd640 1 --2- 172.21.15.22:0/3992337516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e48102ef0 0x7f3e4810da40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009ca0 tx=0x7f3e4002a950 comp rx=0 tx=0).stop 2024-09-19T08:19:57.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.347+0000 7f3e4cadd640 1 -- 172.21.15.22:0/3992337516 shutdown_connections 2024-09-19T08:19:57.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.347+0000 7f3e4cadd640 1 --2- 172.21.15.22:0/3992337516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e48102ef0 0x7f3e4810da40 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.347+0000 7f3e4cadd640 1 --2- 172.21.15.22:0/3992337516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e48102570 0x7f3e481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.347+0000 7f3e4cadd640 1 --2- 172.21.15.22:0/3992337516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e48100e20 0x7f3e48101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.347+0000 7f3e4cadd640 1 -- 172.21.15.22:0/3992337516 >> 172.21.15.22:0/3992337516 conn(0x7f3e480fc7d0 msgr2=0x7f3e480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:57.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.347+0000 7f3e4cadd640 1 -- 172.21.15.22:0/3992337516 shutdown_connections 2024-09-19T08:19:57.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.347+0000 7f3e4cadd640 1 -- 172.21.15.22:0/3992337516 wait complete. 2024-09-19T08:19:57.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.348+0000 7f3e4cadd640 1 Processor -- start 2024-09-19T08:19:57.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.348+0000 7f3e4cadd640 1 -- start start 2024-09-19T08:19:57.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.348+0000 7f3e4cadd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e48100e20 0x7f3e480fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.348+0000 7f3e4cadd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e48102570 0x7f3e480fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.348+0000 7f3e4cadd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e48102ef0 0x7f3e480fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.348+0000 7f3e4cadd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e48110fa0 con 0x7f3e48102570 2024-09-19T08:19:57.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.348+0000 7f3e4cadd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3e48110e20 con 0x7f3e48102ef0 2024-09-19T08:19:57.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.348+0000 7f3e4cadd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3e48111120 con 0x7f3e48100e20 2024-09-19T08:19:57.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.348+0000 7f3e45d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e48102570 0x7f3e480fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.348+0000 7f3e45d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e48102570 0x7f3e480fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34488/0 (socket says 172.21.15.22:34488) 2024-09-19T08:19:57.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.348+0000 7f3e46d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e48102ef0 0x7f3e480fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.348+0000 7f3e45d9b640 1 -- 172.21.15.22:0/2630180793 learned_addr learned my addr 172.21.15.22:0/2630180793 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:57.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.349+0000 7f3e4659c640 1 --2- 172.21.15.22:0/2630180793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e48100e20 0x7f3e480fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.349+0000 7f3e45d9b640 1 -- 172.21.15.22:0/2630180793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e48100e20 msgr2=0x7f3e480fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:57.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.349+0000 7f3e45d9b640 1 --2- 172.21.15.22:0/2630180793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e48100e20 0x7f3e480fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.349+0000 7f3e45d9b640 1 -- 172.21.15.22:0/2630180793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e48102ef0 msgr2=0x7f3e480fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:57.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.349+0000 7f3e45d9b640 1 --2- 172.21.15.22:0/2630180793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e48102ef0 0x7f3e480fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.349+0000 7f3e45d9b640 1 -- 172.21.15.22:0/2630180793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e481a6a40 con 0x7f3e48102570 2024-09-19T08:19:57.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.349+0000 7f3e4659c640 1 --2- 172.21.15.22:0/2630180793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e48100e20 0x7f3e480fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:19:57.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.349+0000 7f3e45d9b640 1 --2- 172.21.15.22:0/2630180793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e48102570 0x7f3e480fd1c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000b4a0 tx=0x7f3e3000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:57.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.349+0000 7f3e3f7fe640 1 -- 172.21.15.22:0/2630180793 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30014070 con 0x7f3e48102570 2024-09-19T08:19:57.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.349+0000 7f3e3f7fe640 1 -- 172.21.15.22:0/2630180793 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e30010af0 con 0x7f3e48102570 2024-09-19T08:19:57.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.349+0000 7f3e4cadd640 1 -- 172.21.15.22:0/2630180793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e481a6d30 con 0x7f3e48102570 2024-09-19T08:19:57.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.349+0000 7f3e3f7fe640 1 -- 172.21.15.22:0/2630180793 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30017d10 con 0x7f3e48102570 2024-09-19T08:19:57.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.349+0000 7f3e4cadd640 1 -- 172.21.15.22:0/2630180793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e481a7260 con 0x7f3e48102570 2024-09-19T08:19:57.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.351+0000 7f3e3f7fe640 1 -- 172.21.15.22:0/2630180793 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3e30017070 con 0x7f3e48102570 2024-09-19T08:19:57.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.352+0000 7f3e4eeb6740 1 Processor -- start 2024-09-19T08:19:57.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.352+0000 7f3e4eeb6740 1 -- start start 2024-09-19T08:19:57.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.352+0000 7f3e4eeb6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac47060 0x56284ac47440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.352+0000 7f3e4eeb6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47980 0x56284ac51d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.352+0000 7f3e4eeb6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac52240 0x56284ac54630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.352+0000 7f3e4eeb6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56284abac620 con 0x56284ac52240 2024-09-19T08:19:57.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.352+0000 7f3e4eeb6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56284ab53e50 con 0x56284ac47060 2024-09-19T08:19:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.352+0000 7f3e4eeb6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56284aba85e0 con 0x56284ac47980 2024-09-19T08:19:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.352+0000 7f3e3f7fe640 1 --2- 172.21.15.22:0/2630180793 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3e340a3430 0x7f3e340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.352+0000 7f3e45d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac52240 0x56284ac54630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e45d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac52240 0x56284ac54630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34492/0 (socket says 172.21.15.22:34492) 2024-09-19T08:19:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e45d9b640 1 -- 172.21.15.22:0/876339165 learned_addr learned my addr 172.21.15.22:0/876339165 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e4659c640 1 --2- 172.21.15.22:0/876339165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47980 0x56284ac51d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e46d9d640 1 --2- 172.21.15.22:0/2630180793 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3e340a3430 0x7f3e340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e46d9d640 1 --2- 172.21.15.22:0/876339165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac47060 0x56284ac47440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e3f7fe640 1 -- 172.21.15.22:0/2630180793 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3e300cef50 con 0x7f3e48102570 2024-09-19T08:19:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e45d9b640 1 -- 172.21.15.22:0/876339165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47980 msgr2=0x56284ac51d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e45d9b640 1 --2- 172.21.15.22:0/876339165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47980 0x56284ac51d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e45d9b640 1 -- 172.21.15.22:0/876339165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac47060 msgr2=0x56284ac47440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:57.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e45d9b640 1 --2- 172.21.15.22:0/876339165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac47060 0x56284ac47440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e45d9b640 1 -- 172.21.15.22:0/876339165 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56284ab44d40 con 0x56284ac52240 2024-09-19T08:19:57.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e46d9d640 1 --2- 172.21.15.22:0/2630180793 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3e340a3430 0x7f3e340a58f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009a60 tx=0x7f3e40009890 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:57.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e45d9b640 1 --2- 172.21.15.22:0/876339165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac52240 0x56284ac54630 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3e300cc6d0 tx=0x7f3e30017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec0ffba66a92c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:57.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e3cff9640 1 -- 172.21.15.22:0/876339165 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3001e030 con 0x56284ac52240 2024-09-19T08:19:57.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e3cff9640 1 -- 172.21.15.22:0/876339165 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3009b9f0 con 0x56284ac52240 2024-09-19T08:19:57.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.353+0000 7f3e3cff9640 1 -- 172.21.15.22:0/876339165 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30097960 con 0x56284ac52240 2024-09-19T08:19:57.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.354+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/876339165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac52240 msgr2=0x56284ac54630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:57.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.354+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/876339165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac52240 0x56284ac54630 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3e300cc6d0 tx=0x7f3e30017730 comp rx=0 tx=0).stop 2024-09-19T08:19:57.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.354+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/876339165 shutdown_connections 2024-09-19T08:19:57.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.354+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/876339165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac52240 0x56284ac54630 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.354+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/876339165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47980 0x56284ac51d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.354+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/876339165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac47060 0x56284ac47440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.354+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/876339165 >> 172.21.15.22:0/876339165 conn(0x56284ac435f0 msgr2=0x56284ac439d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:57.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.354+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/876339165 shutdown_connections 2024-09-19T08:19:57.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.354+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/876339165 wait complete. 2024-09-19T08:19:57.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e4eeb6740 1 -- start start 2024-09-19T08:19:57.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e4eeb6740 1 Processor -- start 2024-09-19T08:19:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e4eeb6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47060 0x56284ac4f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e4eeb6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47980 0x56284ac4f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e4eeb6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac52240 0x56284ac4fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e4eeb6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56284ac54e10 con 0x56284ac47980 2024-09-19T08:19:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e4eeb6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56284ab34f10 con 0x56284ac52240 2024-09-19T08:19:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e4eeb6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56284ac54f90 con 0x56284ac47060 2024-09-19T08:19:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e45d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47980 0x56284ac4f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e4eeb6740 10 client.0 _ll_register_callbacks cb 0x56284ab58db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e45d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47980 0x56284ac4f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34498/0 (socket says 172.21.15.22:34498) 2024-09-19T08:19:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e45d9b640 1 -- 172.21.15.22:0/690423487 learned_addr learned my addr 172.21.15.22:0/690423487 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e4659c640 1 --2- 172.21.15.22:0/690423487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47060 0x56284ac4f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e46d9d640 1 --2- 172.21.15.22:0/690423487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac52240 0x56284ac4fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e45d9b640 1 -- 172.21.15.22:0/690423487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47060 msgr2=0x56284ac4f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:57.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e45d9b640 1 --2- 172.21.15.22:0/690423487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47060 0x56284ac4f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e45d9b640 1 -- 172.21.15.22:0/690423487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac52240 msgr2=0x56284ac4fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:57.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.355+0000 7f3e45d9b640 1 --2- 172.21.15.22:0/690423487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac52240 0x56284ac4fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e45d9b640 1 -- 172.21.15.22:0/690423487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56284a8bb580 con 0x56284ac47980 2024-09-19T08:19:57.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e46d9d640 1 --2- 172.21.15.22:0/690423487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac52240 0x56284ac4fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:19:57.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e45d9b640 1 --2- 172.21.15.22:0/690423487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47980 0x56284ac4f6b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3e30097750 tx=0x7f3e300a3f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ecb74dc039cbcd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:57.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:19:57.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e2dffb640 1 -- 172.21.15.22:0/690423487 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3001e070 con 0x56284ac47980 2024-09-19T08:19:57.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e2dffb640 1 -- 172.21.15.22:0/690423487 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3e300cc8d0 con 0x56284ac47980 2024-09-19T08:19:57.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/690423487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3009b9f0 con 0x56284ac47980 2024-09-19T08:19:57.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e2dffb640 1 -- 172.21.15.22:0/690423487 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e30017460 con 0x56284ac47980 2024-09-19T08:19:57.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e4eeb6740 10 client.16320 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:57.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/690423487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56284a985960 con 0x56284ac47980 2024-09-19T08:19:57.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e2dffb640 1 -- 172.21.15.22:0/690423487 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3009b9f0 con 0x56284ac47980 2024-09-19T08:19:57.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e2dffb640 1 -- 172.21.15.22:0/690423487 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3e300a3ad0 con 0x56284ac47980 2024-09-19T08:19:57.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e4eeb6740 10 client.16320 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:57.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e4eeb6740 10 client.16320 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:19:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e4eeb6740 20 client.16320 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/690423487 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56284ac61cc0 0x56284ac640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e4eeb6740 4 client.16320 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:19:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.356+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/690423487 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56284abb9b90 con 0x56284ac61cc0 2024-09-19T08:19:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.357+0000 7f3e4659c640 1 --2- 172.21.15.22:0/690423487 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56284ac61cc0 0x56284ac640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.357+0000 7f3e4659c640 1 --2- 172.21.15.22:0/690423487 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56284ac61cc0 0x56284ac640b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c00404c38fba3f server_cookie=6c5644b6f61b3945 in_seq=0 out_seq=0 2024-09-19T08:19:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.357+0000 7f3e2dffb640 10 client.16320 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e2dffb640 1 -- 172.21.15.22:0/690423487 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56284abb9b90 con 0x56284ac61cc0 2024-09-19T08:19:57.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e2dffb640 10 client.16320 handle_command_reply: tid=0 2024-09-19T08:19:57.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e4eeb6740 1 client.16320 shutdown 2024-09-19T08:19:57.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/690423487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47980 msgr2=0x56284ac4f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:57.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/690423487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47980 0x56284ac4f6b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3e30097750 tx=0x7f3e300a3f60 comp rx=0 tx=0).stop 2024-09-19T08:19:57.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/690423487 shutdown_connections 2024-09-19T08:19:57.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/690423487 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56284ac61cc0 0x56284ac640b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/690423487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac52240 0x56284ac4fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/690423487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47980 0x56284ac4f6b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e2dffb640 0 client.16320 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:57.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/690423487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47060 0x56284ac4f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/690423487 >> 172.21.15.22:0/690423487 conn(0x56284ac435f0 msgr2=0x56284ac492d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:57.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/690423487 shutdown_connections 2024-09-19T08:19:57.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/690423487 wait complete. 2024-09-19T08:19:57.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.358+0000 7f3e4eeb6740 20 client.16320 trim_cache size 0 max 16384 2024-09-19T08:19:57.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.375+0000 7f3e4eeb6740 1 Processor -- start 2024-09-19T08:19:57.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.375+0000 7f3e4eeb6740 1 -- start start 2024-09-19T08:19:57.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4eeb6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47060 0x56284ac670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4eeb6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47980 0x56284ac67610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4eeb6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac52240 0x56284ac4c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4eeb6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481b23f0 con 0x56284ac47060 2024-09-19T08:19:57.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4eeb6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481afc60 con 0x56284ac52240 2024-09-19T08:19:57.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4eeb6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481adab0 con 0x56284ac47980 2024-09-19T08:19:57.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47060 0x56284ac670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47060 0x56284ac670d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34500/0 (socket says 172.21.15.22:34500) 2024-09-19T08:19:57.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4659c640 1 -- 172.21.15.22:0/3318303413 learned_addr learned my addr 172.21.15.22:0/3318303413 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:57.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e45d9b640 1 --2- 172.21.15.22:0/3318303413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47980 0x56284ac67610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e46d9d640 1 --2- 172.21.15.22:0/3318303413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac52240 0x56284ac4c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4659c640 1 -- 172.21.15.22:0/3318303413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47980 msgr2=0x56284ac67610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:57.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4659c640 1 --2- 172.21.15.22:0/3318303413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47980 0x56284ac67610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4659c640 1 -- 172.21.15.22:0/3318303413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac52240 msgr2=0x56284ac4c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4659c640 1 --2- 172.21.15.22:0/3318303413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac52240 0x56284ac4c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4659c640 1 -- 172.21.15.22:0/3318303413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56284ac51ae0 con 0x56284ac47060 2024-09-19T08:19:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e46d9d640 1 --2- 172.21.15.22:0/3318303413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac52240 0x56284ac4c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:19:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e4659c640 1 --2- 172.21.15.22:0/3318303413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47060 0x56284ac670d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800be40 tx=0x7f3e38013b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef7ec6c51a809e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e2ffff640 1 -- 172.21.15.22:0/3318303413 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38004030 con 0x56284ac47060 2024-09-19T08:19:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e2ffff640 1 -- 172.21.15.22:0/3318303413 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e380041d0 con 0x56284ac47060 2024-09-19T08:19:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.376+0000 7f3e2ffff640 1 -- 172.21.15.22:0/3318303413 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380117c0 con 0x56284ac47060 2024-09-19T08:19:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.377+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/3318303413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47060 msgr2=0x56284ac670d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.377+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/3318303413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47060 0x56284ac670d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800be40 tx=0x7f3e38013b00 comp rx=0 tx=0).stop 2024-09-19T08:19:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.377+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/3318303413 shutdown_connections 2024-09-19T08:19:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.377+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/3318303413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac52240 0x56284ac4c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.377+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/3318303413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac47980 0x56284ac67610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.377+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/3318303413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47060 0x56284ac670d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.377+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/3318303413 >> 172.21.15.22:0/3318303413 conn(0x56284ac435f0 msgr2=0x56284ac492d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.377+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/3318303413 shutdown_connections 2024-09-19T08:19:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.377+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/3318303413 wait complete. 2024-09-19T08:19:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.377+0000 7f3e4eeb6740 1 -- start start 2024-09-19T08:19:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e4eeb6740 1 Processor -- start 2024-09-19T08:19:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e4eeb6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac47060 0x56284ac67870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e4eeb6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47980 0x56284ac65c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e4eeb6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac52240 0x56284ac66180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e4eeb6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481ae120 con 0x56284ac47980 2024-09-19T08:19:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e4eeb6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481af590 con 0x56284ac47060 2024-09-19T08:19:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e4eeb6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481ae7f0 con 0x56284ac52240 2024-09-19T08:19:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e45d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47980 0x56284ac65c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e45d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47980 0x56284ac65c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34506/0 (socket says 172.21.15.22:34506) 2024-09-19T08:19:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e45d9b640 1 -- 172.21.15.22:0/4219140056 learned_addr learned my addr 172.21.15.22:0/4219140056 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:19:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e4eeb6740 10 client.0 _ll_register_callbacks cb 0x7f3e3009b2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:19:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e4659c640 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac47060 0x56284ac67870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e46d9d640 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac52240 0x56284ac66180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e45d9b640 1 -- 172.21.15.22:0/4219140056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac52240 msgr2=0x56284ac66180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e45d9b640 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac52240 0x56284ac66180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e45d9b640 1 -- 172.21.15.22:0/4219140056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac47060 msgr2=0x56284ac67870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:19:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e45d9b640 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac47060 0x56284ac67870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e45d9b640 1 -- 172.21.15.22:0/4219140056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56284ac669b0 con 0x56284ac47980 2024-09-19T08:19:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e46d9d640 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac52240 0x56284ac66180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:19:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.378+0000 7f3e4659c640 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac47060 0x56284ac67870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:19:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e45d9b640 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47980 0x56284ac65c20 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3e300995a0 tx=0x7f3e30099800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beecc552d96d5406 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:19:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:19:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e2dffb640 1 -- 172.21.15.22:0/4219140056 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3001e070 con 0x56284ac47980 2024-09-19T08:19:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e2dffb640 1 -- 172.21.15.22:0/4219140056 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3e300700f0 con 0x56284ac47980 2024-09-19T08:19:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/4219140056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e380041d0 con 0x56284ac47980 2024-09-19T08:19:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e2dffb640 1 -- 172.21.15.22:0/4219140056 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e30095e60 con 0x56284ac47980 2024-09-19T08:19:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e4eeb6740 10 client.16326 fetch_fsmap learned FSMap version 29 2024-09-19T08:19:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/4219140056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56284ac669b0 con 0x56284ac47980 2024-09-19T08:19:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e2dffb640 1 -- 172.21.15.22:0/4219140056 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e380041d0 con 0x56284ac47980 2024-09-19T08:19:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e2dffb640 1 -- 172.21.15.22:0/4219140056 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3e30077020 con 0x56284ac47980 2024-09-19T08:19:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e4eeb6740 10 client.16326 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:19:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e4eeb6740 10 client.16326 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:19:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e4eeb6740 20 client.16326 populate_metadata read hostname 'smithi022' 2024-09-19T08:19:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56284accd980 0x56284accfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:19:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e4eeb6740 4 client.16326 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "69e267f5", "release": true, "await": true}] 2024-09-19T08:19:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.379+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/4219140056 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "set_id": "69e267f5", "release": true, "await": true}) -- 0x56284abb40a0 con 0x56284accd980 2024-09-19T08:19:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.380+0000 7f3e4659c640 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56284accd980 0x56284accfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:19:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.380+0000 7f3e2dffb640 10 client.16326 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:57.380+0000 7f3e4659c640 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56284accd980 0x56284accfd90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35ed4e2e059adde9 server_cookie=8dc38c668b616616 in_seq=0 out_seq=0 2024-09-19T08:19:58.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.266+0000 7f3e2dffb640 1 -- 172.21.15.22:0/4219140056 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56284abb40a0 con 0x56284accd980 2024-09-19T08:19:58.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.266+0000 7f3e2dffb640 10 client.16326 handle_command_reply: tid=0 2024-09-19T08:19:58.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.266+0000 7f3e4eeb6740 1 client.16326 shutdown 2024-09-19T08:19:58.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.266+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/4219140056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47980 msgr2=0x56284ac65c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:58.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.266+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47980 0x56284ac65c20 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3e300995a0 tx=0x7f3e30099800 comp rx=0 tx=0).stop 2024-09-19T08:19:58.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.266+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/4219140056 shutdown_connections 2024-09-19T08:19:58.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.266+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56284accd980 0x56284accfd90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:58.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.266+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56284ac52240 0x56284ac66180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:58.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.266+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56284ac47980 0x56284ac65c20 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:58.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.266+0000 7f3e4eeb6740 1 --2- 172.21.15.22:0/4219140056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56284ac47060 0x56284ac67870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:58.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.266+0000 7f3e2dffb640 0 client.16326 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:19:58.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.266+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/4219140056 >> 172.21.15.22:0/4219140056 conn(0x56284ac435f0 msgr2=0x56284ac5d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:58.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.267+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/4219140056 shutdown_connections 2024-09-19T08:19:58.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.267+0000 7f3e4eeb6740 1 -- 172.21.15.22:0/4219140056 wait complete. 2024-09-19T08:19:58.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.267+0000 7f3e4eeb6740 20 client.16326 trim_cache size 0 max 16384 2024-09-19T08:19:58.268 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:19:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 29, 2024-09-19T08:19:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:19:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 20, 2024-09-19T08:19:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:19:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "69e267f5": { 2024-09-19T08:19:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 20, 2024-09-19T08:19:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:19:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:19:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:19:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:19:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:19:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:19:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 149.8, 2024-09-19T08:19:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:19:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:19:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:19:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:19:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:19:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:19:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:19:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:19:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:19:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:19:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:19:58.270 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:19:58.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.268+0000 7f3e4cadd640 1 -- 172.21.15.22:0/2630180793 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3e340a3430 msgr2=0x7f3e340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:58.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.268+0000 7f3e4cadd640 1 --2- 172.21.15.22:0/2630180793 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3e340a3430 0x7f3e340a58f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009a60 tx=0x7f3e40009890 comp rx=0 tx=0).stop 2024-09-19T08:19:58.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.268+0000 7f3e4cadd640 1 -- 172.21.15.22:0/2630180793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e48102570 msgr2=0x7f3e480fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:19:58.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.268+0000 7f3e4cadd640 1 --2- 172.21.15.22:0/2630180793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e48102570 0x7f3e480fd1c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000b4a0 tx=0x7f3e3000b960 comp rx=0 tx=0).stop 2024-09-19T08:19:58.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.268+0000 7f3e4cadd640 1 -- 172.21.15.22:0/2630180793 shutdown_connections 2024-09-19T08:19:58.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.268+0000 7f3e4cadd640 1 --2- 172.21.15.22:0/2630180793 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3e340a3430 0x7f3e340a58f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:58.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.268+0000 7f3e4cadd640 1 --2- 172.21.15.22:0/2630180793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e48102ef0 0x7f3e480fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:58.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.268+0000 7f3e4cadd640 1 --2- 172.21.15.22:0/2630180793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e48102570 0x7f3e480fd1c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:58.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.268+0000 7f3e4cadd640 1 --2- 172.21.15.22:0/2630180793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e48100e20 0x7f3e480fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:19:58.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.268+0000 7f3e4cadd640 1 -- 172.21.15.22:0/2630180793 >> 172.21.15.22:0/2630180793 conn(0x7f3e480fc7d0 msgr2=0x7f3e4806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:19:58.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.268+0000 7f3e4cadd640 1 -- 172.21.15.22:0/2630180793 shutdown_connections 2024-09-19T08:19:58.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:19:58.268+0000 7f3e4cadd640 1 -- 172.21.15.22:0/2630180793 wait complete. 2024-09-19T08:19:58.983 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:19:58 smithi022 ceph-mon[30963]: pgmap v151: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-09-19T08:19:59.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:19:58 smithi064 ceph-mon[34900]: pgmap v151: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-09-19T08:19:59.140 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '69e267f5', total seconds elapsed: 34.3 2024-09-19T08:19:59.140 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T08:19:59.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:19:59.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:19:58 smithi157 ceph-mon[35143]: pgmap v151: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-09-19T08:19:59.506 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:19:59.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:20:00.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.025+0000 7f2dd73bc640 1 -- 172.21.15.22:0/2785491587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dd0100aa0 msgr2=0x7f2dd010ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:00.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.025+0000 7f2dd73bc640 1 --2- 172.21.15.22:0/2785491587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dd0100aa0 0x7f2dd010ea20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009ca0 tx=0x7f2dc802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.025+0000 7f2dd73bc640 1 -- 172.21.15.22:0/2785491587 shutdown_connections 2024-09-19T08:20:00.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.025+0000 7f2dd73bc640 1 --2- 172.21.15.22:0/2785491587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dd0100aa0 0x7f2dd010ea20 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.025+0000 7f2dd73bc640 1 --2- 172.21.15.22:0/2785491587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd0100100 0x7f2dd0100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.025+0000 7f2dd73bc640 1 --2- 172.21.15.22:0/2785491587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd0106b80 0x7f2dd0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.025+0000 7f2dd73bc640 1 -- 172.21.15.22:0/2785491587 >> 172.21.15.22:0/2785491587 conn(0x7f2dd0077230 msgr2=0x7f2dd00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:00.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.025+0000 7f2dd73bc640 1 -- 172.21.15.22:0/2785491587 shutdown_connections 2024-09-19T08:20:00.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.025+0000 7f2dd73bc640 1 -- 172.21.15.22:0/2785491587 wait complete. 2024-09-19T08:20:00.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.026+0000 7f2dd73bc640 1 Processor -- start 2024-09-19T08:20:00.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.026+0000 7f2dd73bc640 1 -- start start 2024-09-19T08:20:00.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.026+0000 7f2dd73bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd0100100 0x7f2dd0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.026+0000 7f2dd73bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dd0100aa0 0x7f2dd01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.026+0000 7f2dd73bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd0106b80 0x7f2dd019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.026+0000 7f2dd73bc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01120a0 con 0x7f2dd0100aa0 2024-09-19T08:20:00.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.026+0000 7f2dd73bc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0111f20 con 0x7f2dd0106b80 2024-09-19T08:20:00.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.026+0000 7f2dd73bc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0112220 con 0x7f2dd0100100 2024-09-19T08:20:00.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.027+0000 7f2dd5158640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd0100100 0x7f2dd0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.027+0000 7f2dd5158640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd0100100 0x7f2dd0198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:54538/0 (socket says 172.21.15.22:54538) 2024-09-19T08:20:00.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.027+0000 7f2dd5158640 1 -- 172.21.15.22:0/1717715183 learned_addr learned my addr 172.21.15.22:0/1717715183 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:00.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.027+0000 7f2dd4957640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dd0100aa0 0x7f2dd01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.027+0000 7f2dd5959640 1 --2- 172.21.15.22:0/1717715183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd0106b80 0x7f2dd019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.027+0000 7f2dd5158640 1 -- 172.21.15.22:0/1717715183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd0106b80 msgr2=0x7f2dd019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:00.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.027+0000 7f2dd5158640 1 --2- 172.21.15.22:0/1717715183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd0106b80 0x7f2dd019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.027+0000 7f2dd5158640 1 -- 172.21.15.22:0/1717715183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dd0100aa0 msgr2=0x7f2dd01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:00.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.027+0000 7f2dd5158640 1 --2- 172.21.15.22:0/1717715183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dd0100aa0 0x7f2dd01989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.027+0000 7f2dd5158640 1 -- 172.21.15.22:0/1717715183 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd019d420 con 0x7f2dd0100100 2024-09-19T08:20:00.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.027+0000 7f2dd5158640 1 --2- 172.21.15.22:0/1717715183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd0100100 0x7f2dd0198470 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00d950 tx=0x7f2dcc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:00.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.027+0000 7f2dc67fc640 1 -- 172.21.15.22:0/1717715183 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc017030 con 0x7f2dd0100100 2024-09-19T08:20:00.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.027+0000 7f2dd73bc640 1 -- 172.21.15.22:0/1717715183 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd019d710 con 0x7f2dd0100100 2024-09-19T08:20:00.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.027+0000 7f2dc67fc640 1 -- 172.21.15.22:0/1717715183 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dcc004020 con 0x7f2dd0100100 2024-09-19T08:20:00.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.028+0000 7f2dc67fc640 1 -- 172.21.15.22:0/1717715183 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc0047a0 con 0x7f2dd0100100 2024-09-19T08:20:00.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.028+0000 7f2dd73bc640 1 -- 172.21.15.22:0/1717715183 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd01a4ff0 con 0x7f2dd0100100 2024-09-19T08:20:00.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.029+0000 7f2dd73bc640 1 -- 172.21.15.22:0/1717715183 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd0100560 con 0x7f2dd0100100 2024-09-19T08:20:00.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.030+0000 7f2dc67fc640 1 -- 172.21.15.22:0/1717715183 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2dcc0041d0 con 0x7f2dd0100100 2024-09-19T08:20:00.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.031+0000 7f2dc67fc640 1 --2- 172.21.15.22:0/1717715183 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2dbc0a33b0 0x7f2dbc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.031+0000 7f2dd4957640 1 --2- 172.21.15.22:0/1717715183 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2dbc0a33b0 0x7f2dbc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.031+0000 7f2dc67fc640 1 -- 172.21.15.22:0/1717715183 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2dcc0cfce0 con 0x7f2dd0100100 2024-09-19T08:20:00.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.032+0000 7f2dd4957640 1 --2- 172.21.15.22:0/1717715183 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2dbc0a33b0 0x7f2dbc0a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2dd00ff780 tx=0x7f2dc0005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:00.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.038+0000 7f2dc67fc640 1 -- 172.21.15.22:0/1717715183 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2dcc09c6d0 con 0x7f2dd0100100 2024-09-19T08:20:00.157 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:00.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.285+0000 7f2dd73bc640 1 -- 172.21.15.22:0/1717715183 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dd01a5220 con 0x7f2dd0100100 2024-09-19T08:20:00.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.285+0000 7f2dc67fc640 1 -- 172.21.15.22:0/1717715183 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11909 (secure 0 0 0) 0x7f2dcc09bfd0 con 0x7f2dd0100100 2024-09-19T08:20:00.286 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:20:00.286 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":29,"btime":"2024-09-19T08:18:57:303741+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:18:56.870055+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:20:00.286 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 29 2024-09-19T08:20:00.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.288+0000 7f2dd73bc640 1 -- 172.21.15.22:0/1717715183 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2dbc0a33b0 msgr2=0x7f2dbc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:00.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.288+0000 7f2dd73bc640 1 --2- 172.21.15.22:0/1717715183 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2dbc0a33b0 0x7f2dbc0a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2dd00ff780 tx=0x7f2dc0005f90 comp rx=0 tx=0).stop 2024-09-19T08:20:00.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.288+0000 7f2dd73bc640 1 -- 172.21.15.22:0/1717715183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd0100100 msgr2=0x7f2dd0198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:00.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.288+0000 7f2dd73bc640 1 --2- 172.21.15.22:0/1717715183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd0100100 0x7f2dd0198470 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00d950 tx=0x7f2dcc00de10 comp rx=0 tx=0).stop 2024-09-19T08:20:00.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.288+0000 7f2dd73bc640 1 -- 172.21.15.22:0/1717715183 shutdown_connections 2024-09-19T08:20:00.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.288+0000 7f2dd73bc640 1 --2- 172.21.15.22:0/1717715183 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2dbc0a33b0 0x7f2dbc0a5870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.288+0000 7f2dd73bc640 1 --2- 172.21.15.22:0/1717715183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd0106b80 0x7f2dd019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.288+0000 7f2dd73bc640 1 --2- 172.21.15.22:0/1717715183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dd0100aa0 0x7f2dd01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.288+0000 7f2dd73bc640 1 --2- 172.21.15.22:0/1717715183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2dd0100100 0x7f2dd0198470 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.288+0000 7f2dd73bc640 1 -- 172.21.15.22:0/1717715183 >> 172.21.15.22:0/1717715183 conn(0x7f2dd0077230 msgr2=0x7f2dd0104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:00.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.288+0000 7f2dd73bc640 1 -- 172.21.15.22:0/1717715183 shutdown_connections 2024-09-19T08:20:00.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.288+0000 7f2dd73bc640 1 -- 172.21.15.22:0/1717715183 wait complete. 2024-09-19T08:20:00.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.912+0000 7f8cf2c5e640 1 -- 172.21.15.22:0/4041009545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cec102790 msgr2=0x7f8cec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:00.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.912+0000 7f8cf2c5e640 1 --2- 172.21.15.22:0/4041009545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cec102790 0x7f8cec1067e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009900 tx=0x7f8cd402a410 comp rx=0 tx=0).stop 2024-09-19T08:20:00.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.912+0000 7f8cf2c5e640 1 -- 172.21.15.22:0/4041009545 shutdown_connections 2024-09-19T08:20:00.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.912+0000 7f8cf2c5e640 1 --2- 172.21.15.22:0/4041009545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cec106f10 0x7f8cec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.912+0000 7f8cf2c5e640 1 --2- 172.21.15.22:0/4041009545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cec102790 0x7f8cec1067e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.912+0000 7f8cf2c5e640 1 --2- 172.21.15.22:0/4041009545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cec101de0 0x7f8cec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.912+0000 7f8cf2c5e640 1 -- 172.21.15.22:0/4041009545 >> 172.21.15.22:0/4041009545 conn(0x7f8cec0fd880 msgr2=0x7f8cec0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:00.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.912+0000 7f8cf2c5e640 1 -- 172.21.15.22:0/4041009545 shutdown_connections 2024-09-19T08:20:00.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.912+0000 7f8cf2c5e640 1 -- 172.21.15.22:0/4041009545 wait complete. 2024-09-19T08:20:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cf2c5e640 1 Processor -- start 2024-09-19T08:20:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cf2c5e640 1 -- start start 2024-09-19T08:20:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cf2c5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cec101de0 0x7f8cec0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cf2c5e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cec102790 0x7f8cec074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cf2c5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cec106f10 0x7f8cec074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cf2c5e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec111f70 con 0x7f8cec102790 2024-09-19T08:20:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cf2c5e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec111df0 con 0x7f8cec101de0 2024-09-19T08:20:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cf2c5e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec1120f0 con 0x7f8cec106f10 2024-09-19T08:20:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cec102790 0x7f8cec074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cec102790 0x7f8cec074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34536/0 (socket says 172.21.15.22:34536) 2024-09-19T08:20:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cebfff640 1 -- 172.21.15.22:0/2524723000 learned_addr learned my addr 172.21.15.22:0/2524723000 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cf09fa640 1 --2- 172.21.15.22:0/2524723000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cec101de0 0x7f8cec0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cf11fb640 1 --2- 172.21.15.22:0/2524723000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cec106f10 0x7f8cec074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cebfff640 1 -- 172.21.15.22:0/2524723000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cec106f10 msgr2=0x7f8cec074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cebfff640 1 --2- 172.21.15.22:0/2524723000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cec106f10 0x7f8cec074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cebfff640 1 -- 172.21.15.22:0/2524723000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cec101de0 msgr2=0x7f8cec0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cebfff640 1 --2- 172.21.15.22:0/2524723000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cec101de0 0x7f8cec0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.913+0000 7f8cebfff640 1 -- 172.21.15.22:0/2524723000 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cec1a68f0 con 0x7f8cec102790 2024-09-19T08:20:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.914+0000 7f8cebfff640 1 --2- 172.21.15.22:0/2524723000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cec102790 0x7f8cec074a30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4006010 tx=0x7f8cd4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.914+0000 7f8ce9ffb640 1 -- 172.21.15.22:0/2524723000 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd403f030 con 0x7f8cec102790 2024-09-19T08:20:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.914+0000 7f8cf2c5e640 1 -- 172.21.15.22:0/2524723000 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cec1a6b80 con 0x7f8cec102790 2024-09-19T08:20:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.914+0000 7f8ce9ffb640 1 -- 172.21.15.22:0/2524723000 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cd4004440 con 0x7f8cec102790 2024-09-19T08:20:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.914+0000 7f8ce9ffb640 1 -- 172.21.15.22:0/2524723000 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4035070 con 0x7f8cec102790 2024-09-19T08:20:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.914+0000 7f8cf2c5e640 1 -- 172.21.15.22:0/2524723000 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cec1a70b0 con 0x7f8cec102790 2024-09-19T08:20:00.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.916+0000 7f8ce9ffb640 1 -- 172.21.15.22:0/2524723000 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8cd4042030 con 0x7f8cec102790 2024-09-19T08:20:00.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.916+0000 7f8cf5037740 1 Processor -- start 2024-09-19T08:20:00.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf5037740 1 -- start start 2024-09-19T08:20:00.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf5037740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f14a0 0x5640082f1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf5037740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082f1dc0 0x5640082fc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf5037740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082fc680 0x5640082fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf5037740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5640082572e0 con 0x5640082f1dc0 2024-09-19T08:20:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf5037740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5640081fece0 con 0x5640082fc680 2024-09-19T08:20:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf5037740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5640082532a0 con 0x5640082f14a0 2024-09-19T08:20:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf09fa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082f1dc0 0x5640082fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8ce9ffb640 1 --2- 172.21.15.22:0/2524723000 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f8cd80a33b0 0x7f8cd80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf09fa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082f1dc0 0x5640082fc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34546/0 (socket says 172.21.15.22:34546) 2024-09-19T08:20:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf09fa640 1 -- 172.21.15.22:0/3613276334 learned_addr learned my addr 172.21.15.22:0/3613276334 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf11fb640 1 --2- 172.21.15.22:0/3613276334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f14a0 0x5640082f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf11fb640 1 --2- 172.21.15.22:0/2524723000 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f8cd80a33b0 0x7f8cd80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8ce9ffb640 1 -- 172.21.15.22:0/2524723000 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8cd40f0160 con 0x7f8cec102790 2024-09-19T08:20:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cebfff640 1 --2- 172.21.15.22:0/3613276334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082fc680 0x5640082fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf11fb640 1 -- 172.21.15.22:0/3613276334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082fc680 msgr2=0x5640082fea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf11fb640 1 --2- 172.21.15.22:0/3613276334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082fc680 0x5640082fea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf11fb640 1 -- 172.21.15.22:0/3613276334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082f1dc0 msgr2=0x5640082fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf11fb640 1 --2- 172.21.15.22:0/3613276334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082f1dc0 0x5640082fc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.917+0000 7f8cf11fb640 1 -- 172.21.15.22:0/3613276334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640081e3440 con 0x5640082f14a0 2024-09-19T08:20:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.918+0000 7f8cf11fb640 1 --2- 172.21.15.22:0/3613276334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f14a0 0x5640082f1880 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4005fd0 tx=0x7f8ce4013e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6707a49931719127 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.918+0000 7f8cf11fb640 1 --2- 172.21.15.22:0/2524723000 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f8cd80a33b0 0x7f8cd80a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8ce401e000 tx=0x7f8ce4020000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.918+0000 7f8ce2ffd640 1 -- 172.21.15.22:0/3613276334 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400fc80 con 0x5640082f14a0 2024-09-19T08:20:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.918+0000 7f8ce2ffd640 1 -- 172.21.15.22:0/3613276334 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ce4023030 con 0x5640082f14a0 2024-09-19T08:20:00.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.918+0000 7f8ce2ffd640 1 -- 172.21.15.22:0/3613276334 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4010ad0 con 0x5640082f14a0 2024-09-19T08:20:00.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.919+0000 7f8cf5037740 1 -- 172.21.15.22:0/3613276334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f14a0 msgr2=0x5640082f1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:00.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.919+0000 7f8cf5037740 1 --2- 172.21.15.22:0/3613276334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f14a0 0x5640082f1880 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4005fd0 tx=0x7f8ce4013e50 comp rx=0 tx=0).stop 2024-09-19T08:20:00.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.919+0000 7f8cf5037740 1 -- 172.21.15.22:0/3613276334 shutdown_connections 2024-09-19T08:20:00.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.919+0000 7f8cf5037740 1 --2- 172.21.15.22:0/3613276334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082fc680 0x5640082fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.919+0000 7f8cf5037740 1 --2- 172.21.15.22:0/3613276334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082f1dc0 0x5640082fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.919+0000 7f8cf5037740 1 --2- 172.21.15.22:0/3613276334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f14a0 0x5640082f1880 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.919+0000 7f8cf5037740 1 -- 172.21.15.22:0/3613276334 >> 172.21.15.22:0/3613276334 conn(0x5640082eda30 msgr2=0x5640082ede10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.919+0000 7f8cf5037740 1 -- 172.21.15.22:0/3613276334 shutdown_connections 2024-09-19T08:20:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.919+0000 7f8cf5037740 1 -- 172.21.15.22:0/3613276334 wait complete. 2024-09-19T08:20:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.919+0000 7f8cf5037740 1 -- start start 2024-09-19T08:20:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.919+0000 7f8cf5037740 1 Processor -- start 2024-09-19T08:20:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf5037740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f14a0 0x5640082fc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf5037740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082f1dc0 0x5640082f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf5037740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082fc680 0x5640082f98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf5037740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5640082ff250 con 0x5640082fc680 2024-09-19T08:20:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf5037740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564008212050 con 0x5640082f1dc0 2024-09-19T08:20:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf5037740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5640082ff3d0 con 0x5640082f14a0 2024-09-19T08:20:00.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf5037740 10 client.0 _ll_register_callbacks cb 0x564008252160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf11fb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082fc680 0x5640082f98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf11fb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082fc680 0x5640082f98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34550/0 (socket says 172.21.15.22:34550) 2024-09-19T08:20:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf11fb640 1 -- 172.21.15.22:0/44126172 learned_addr learned my addr 172.21.15.22:0/44126172 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf09fa640 1 --2- 172.21.15.22:0/44126172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f14a0 0x5640082fc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cebfff640 1 --2- 172.21.15.22:0/44126172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082f1dc0 0x5640082f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf11fb640 1 -- 172.21.15.22:0/44126172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f14a0 msgr2=0x5640082fc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf11fb640 1 --2- 172.21.15.22:0/44126172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f14a0 0x5640082fc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf11fb640 1 -- 172.21.15.22:0/44126172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082f1dc0 msgr2=0x5640082f92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf11fb640 1 --2- 172.21.15.22:0/44126172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082f1dc0 0x5640082f92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf11fb640 1 -- 172.21.15.22:0/44126172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564007fc83c0 con 0x5640082fc680 2024-09-19T08:20:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8cf11fb640 1 --2- 172.21.15.22:0/44126172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082fc680 0x5640082f98e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4005a90 tx=0x7f8ce4005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745934f047e21e9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.920+0000 7f8ccbfff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:00.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8ccbfff640 1 -- 172.21.15.22:0/44126172 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce401fe60 con 0x5640082fc680 2024-09-19T08:20:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8ccbfff640 1 -- 172.21.15.22:0/44126172 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8ce4013320 con 0x5640082fc680 2024-09-19T08:20:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8cf5037740 1 -- 172.21.15.22:0/44126172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ce4023030 con 0x5640082fc680 2024-09-19T08:20:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8ccbfff640 1 -- 172.21.15.22:0/44126172 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ce4013520 con 0x5640082fc680 2024-09-19T08:20:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8cf5037740 10 client.16344 fetch_fsmap learned FSMap version 29 2024-09-19T08:20:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8cf5037740 1 -- 172.21.15.22:0/44126172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56400830ac70 con 0x5640082fc680 2024-09-19T08:20:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8ccbfff640 1 -- 172.21.15.22:0/44126172 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ce4023030 con 0x5640082fc680 2024-09-19T08:20:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8ccbfff640 1 -- 172.21.15.22:0/44126172 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8ce40288e0 con 0x5640082fc680 2024-09-19T08:20:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8cf5037740 10 client.16344 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:20:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8cf5037740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:20:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8cf5037740 20 client.16344 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8cf5037740 1 --2- 172.21.15.22:0/44126172 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56400830c020 0x56400830e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8cf5037740 4 client.16344 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:20:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8cf5037740 1 -- 172.21.15.22:0/44126172 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564008264850 con 0x56400830c020 2024-09-19T08:20:00.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.921+0000 7f8cf09fa640 1 --2- 172.21.15.22:0/44126172 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56400830c020 0x56400830e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.922+0000 7f8cf09fa640 1 --2- 172.21.15.22:0/44126172 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56400830c020 0x56400830e410 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce4741f6175d5730 server_cookie=57fd5429914cf159 in_seq=0 out_seq=0 2024-09-19T08:20:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.922+0000 7f8ccbfff640 10 client.16344 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8ccbfff640 1 -- 172.21.15.22:0/44126172 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564008264850 con 0x56400830c020 2024-09-19T08:20:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8ccbfff640 10 client.16344 handle_command_reply: tid=0 2024-09-19T08:20:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8cf5037740 1 client.16344 shutdown 2024-09-19T08:20:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8cf5037740 1 -- 172.21.15.22:0/44126172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082fc680 msgr2=0x5640082f98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8cf5037740 1 --2- 172.21.15.22:0/44126172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082fc680 0x5640082f98e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4005a90 tx=0x7f8ce4005ac0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8cf5037740 1 -- 172.21.15.22:0/44126172 shutdown_connections 2024-09-19T08:20:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8cf5037740 1 --2- 172.21.15.22:0/44126172 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56400830c020 0x56400830e410 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8cf5037740 1 --2- 172.21.15.22:0/44126172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082fc680 0x5640082f98e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8cf5037740 1 --2- 172.21.15.22:0/44126172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082f1dc0 0x5640082f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8cf5037740 1 --2- 172.21.15.22:0/44126172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f14a0 0x5640082fc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8cf5037740 1 -- 172.21.15.22:0/44126172 >> 172.21.15.22:0/44126172 conn(0x5640082eda30 msgr2=0x5640082f3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:00.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8ccbfff640 0 client.16344 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:00.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8cf5037740 1 -- 172.21.15.22:0/44126172 shutdown_connections 2024-09-19T08:20:00.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8cf5037740 1 -- 172.21.15.22:0/44126172 wait complete. 2024-09-19T08:20:00.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.923+0000 7f8cf5037740 20 client.16344 trim_cache size 0 max 16384 2024-09-19T08:20:00.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.940+0000 7f8cf5037740 1 Processor -- start 2024-09-19T08:20:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.940+0000 7f8cf5037740 1 -- start start 2024-09-19T08:20:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf5037740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082f14a0 0x5640082f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf5037740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f1dc0 0x564008313bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf5037740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082fc680 0x564008314110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf5037740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec1b2360 con 0x5640082f14a0 2024-09-19T08:20:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf5037740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec1afbd0 con 0x5640082fc680 2024-09-19T08:20:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf5037740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec1ada20 con 0x5640082f1dc0 2024-09-19T08:20:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf09fa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082f14a0 0x5640082f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf09fa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082f14a0 0x5640082f7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34556/0 (socket says 172.21.15.22:34556) 2024-09-19T08:20:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf09fa640 1 -- 172.21.15.22:0/3507413852 learned_addr learned my addr 172.21.15.22:0/3507413852 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf11fb640 1 --2- 172.21.15.22:0/3507413852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082fc680 0x564008314110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cebfff640 1 --2- 172.21.15.22:0/3507413852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f1dc0 0x564008313bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf09fa640 1 -- 172.21.15.22:0/3507413852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f1dc0 msgr2=0x564008313bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf09fa640 1 --2- 172.21.15.22:0/3507413852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f1dc0 0x564008313bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf09fa640 1 -- 172.21.15.22:0/3507413852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082fc680 msgr2=0x564008314110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf09fa640 1 --2- 172.21.15.22:0/3507413852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082fc680 0x564008314110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf09fa640 1 -- 172.21.15.22:0/3507413852 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56400830e780 con 0x5640082f14a0 2024-09-19T08:20:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.941+0000 7f8cf09fa640 1 --2- 172.21.15.22:0/3507413852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082f14a0 0x5640082f7bc0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x56400830eb40 tx=0x7f8cdc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e7ec6b8508b1fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.942+0000 7f8ce27fc640 1 -- 172.21.15.22:0/3507413852 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc003d20 con 0x5640082f14a0 2024-09-19T08:20:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.942+0000 7f8ce27fc640 1 -- 172.21.15.22:0/3507413852 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cdc012930 con 0x5640082f14a0 2024-09-19T08:20:00.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.942+0000 7f8ce27fc640 1 -- 172.21.15.22:0/3507413852 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc0279a0 con 0x5640082f14a0 2024-09-19T08:20:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.942+0000 7f8cf5037740 1 -- 172.21.15.22:0/3507413852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082f14a0 msgr2=0x5640082f7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.942+0000 7f8cf5037740 1 --2- 172.21.15.22:0/3507413852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082f14a0 0x5640082f7bc0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x56400830eb40 tx=0x7f8cdc005f90 comp rx=0 tx=0).stop 2024-09-19T08:20:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.942+0000 7f8cf5037740 1 -- 172.21.15.22:0/3507413852 shutdown_connections 2024-09-19T08:20:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.942+0000 7f8cf5037740 1 --2- 172.21.15.22:0/3507413852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082fc680 0x564008314110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.942+0000 7f8cf5037740 1 --2- 172.21.15.22:0/3507413852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f1dc0 0x564008313bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.942+0000 7f8cf5037740 1 --2- 172.21.15.22:0/3507413852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082f14a0 0x5640082f7bc0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.942+0000 7f8cf5037740 1 -- 172.21.15.22:0/3507413852 >> 172.21.15.22:0/3507413852 conn(0x5640082eda30 msgr2=0x5640082fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:00.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.942+0000 7f8cf5037740 1 -- 172.21.15.22:0/3507413852 shutdown_connections 2024-09-19T08:20:00.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.942+0000 7f8cf5037740 1 -- 172.21.15.22:0/3507413852 wait complete. 2024-09-19T08:20:00.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cf5037740 1 -- start start 2024-09-19T08:20:00.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cf5037740 1 Processor -- start 2024-09-19T08:20:00.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cf5037740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082f14a0 0x564008302370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cf5037740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f1dc0 0x5640083028b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cf5037740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082fc680 0x564008302df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cf5037740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec1ae090 con 0x5640082fc680 2024-09-19T08:20:00.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cf5037740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec1af500 con 0x5640082f14a0 2024-09-19T08:20:00.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cf5037740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec1ae760 con 0x5640082f1dc0 2024-09-19T08:20:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cf5037740 10 client.0 _ll_register_callbacks cb 0x7f8cdc0096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cf11fb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082fc680 0x564008302df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f1dc0 0x5640083028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cf09fa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082f14a0 0x564008302370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f1dc0 0x5640083028b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:54578/0 (socket says 172.21.15.22:54578) 2024-09-19T08:20:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cebfff640 1 -- 172.21.15.22:0/1335944067 learned_addr learned my addr 172.21.15.22:0/1335944067 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.943+0000 7f8cf11fb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082fc680 0x564008302df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34564/0 (socket says 172.21.15.22:34564) 2024-09-19T08:20:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8cebfff640 1 -- 172.21.15.22:0/1335944067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082f14a0 msgr2=0x564008302370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8cebfff640 1 --2- 172.21.15.22:0/1335944067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082f14a0 0x564008302370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8cebfff640 1 -- 172.21.15.22:0/1335944067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082fc680 msgr2=0x564008302df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8cebfff640 1 --2- 172.21.15.22:0/1335944067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082fc680 0x564008302df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8cebfff640 1 -- 172.21.15.22:0/1335944067 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564008375bb0 con 0x5640082f1dc0 2024-09-19T08:20:00.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8cf09fa640 1 --2- 172.21.15.22:0/1335944067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082f14a0 0x564008302370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8cf11fb640 1 --2- 172.21.15.22:0/1335944067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082fc680 0x564008302df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8cebfff640 1 --2- 172.21.15.22:0/1335944067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f1dc0 0x5640083028b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4000c00 tx=0x7f8cd40bbe10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44cfedea4adad0d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8ccbfff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:20:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8ccbfff640 1 -- 172.21.15.22:0/1335944067 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd40d4070 con 0x5640082f1dc0 2024-09-19T08:20:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8ccbfff640 1 -- 172.21.15.22:0/1335944067 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8cd40b8a60 con 0x5640082f1dc0 2024-09-19T08:20:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8cf5037740 1 -- 172.21.15.22:0/1335944067 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cdc012930 con 0x5640082f1dc0 2024-09-19T08:20:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8ccbfff640 1 -- 172.21.15.22:0/1335944067 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8cd4049040 con 0x5640082f1dc0 2024-09-19T08:20:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8cf5037740 10 client.25879 fetch_fsmap learned FSMap version 29 2024-09-19T08:20:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.944+0000 7f8cf5037740 1 -- 172.21.15.22:0/1335944067 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564008376ed0 con 0x5640082f1dc0 2024-09-19T08:20:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.945+0000 7f8ccbfff640 1 -- 172.21.15.22:0/1335944067 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cdc012930 con 0x5640082f1dc0 2024-09-19T08:20:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.945+0000 7f8ccbfff640 1 -- 172.21.15.22:0/1335944067 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8cd40bc650 con 0x5640082f1dc0 2024-09-19T08:20:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.945+0000 7f8cf5037740 10 client.25879 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:20:00.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.945+0000 7f8cf5037740 10 client.25879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:20:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.945+0000 7f8cf5037740 20 client.25879 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.945+0000 7f8cf5037740 1 --2- 172.21.15.22:0/1335944067 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564008377650 0x564008379a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.945+0000 7f8cf5037740 4 client.25879 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:20:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.945+0000 7f8cf5037740 1 -- 172.21.15.22:0/1335944067 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56400825ed60 con 0x564008377650 2024-09-19T08:20:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.945+0000 7f8cf09fa640 1 --2- 172.21.15.22:0/1335944067 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564008377650 0x564008379a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.945+0000 7f8cf09fa640 1 --2- 172.21.15.22:0/1335944067 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564008377650 0x564008379a70 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d8b09d2704b4ad server_cookie=87de54fc0bc3dbc1 in_seq=0 out_seq=0 2024-09-19T08:20:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.945+0000 7f8ccbfff640 10 client.25879 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.945+0000 7f8ccbfff640 1 -- 172.21.15.22:0/1335944067 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56400825ed60 con 0x564008377650 2024-09-19T08:20:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.945+0000 7f8ccbfff640 10 client.25879 handle_command_reply: tid=0 2024-09-19T08:20:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.945+0000 7f8cf5037740 1 client.25879 shutdown 2024-09-19T08:20:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf5037740 1 -- 172.21.15.22:0/1335944067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f1dc0 msgr2=0x5640083028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf5037740 1 --2- 172.21.15.22:0/1335944067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f1dc0 0x5640083028b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4000c00 tx=0x7f8cd40bbe10 comp rx=0 tx=0).stop 2024-09-19T08:20:00.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf5037740 1 -- 172.21.15.22:0/1335944067 shutdown_connections 2024-09-19T08:20:00.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf5037740 1 --2- 172.21.15.22:0/1335944067 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564008377650 0x564008379a70 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf5037740 1 --2- 172.21.15.22:0/1335944067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640082fc680 0x564008302df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf5037740 1 --2- 172.21.15.22:0/1335944067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640082f1dc0 0x5640083028b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8ccbfff640 0 client.25879 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:00.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf5037740 1 --2- 172.21.15.22:0/1335944067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5640082f14a0 0x564008302370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf5037740 1 -- 172.21.15.22:0/1335944067 >> 172.21.15.22:0/1335944067 conn(0x5640082eda30 msgr2=0x5640082ec350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:00.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf5037740 1 -- 172.21.15.22:0/1335944067 shutdown_connections 2024-09-19T08:20:00.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf5037740 1 -- 172.21.15.22:0/1335944067 wait complete. 2024-09-19T08:20:00.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf5037740 20 client.25879 trim_cache size 0 max 16384 2024-09-19T08:20:00.951 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:20:00.951 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-19T08:20:00.952 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-09-19T08:20:00.952 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:20:00.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf2c5e640 1 -- 172.21.15.22:0/2524723000 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f8cd80a33b0 msgr2=0x7f8cd80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:00.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf2c5e640 1 --2- 172.21.15.22:0/2524723000 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f8cd80a33b0 0x7f8cd80a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8ce401e000 tx=0x7f8ce4020000 comp rx=0 tx=0).stop 2024-09-19T08:20:00.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf2c5e640 1 -- 172.21.15.22:0/2524723000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cec102790 msgr2=0x7f8cec074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:00.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.946+0000 7f8cf2c5e640 1 --2- 172.21.15.22:0/2524723000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cec102790 0x7f8cec074a30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4006010 tx=0x7f8cd4004040 comp rx=0 tx=0).stop 2024-09-19T08:20:00.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.947+0000 7f8cf2c5e640 1 -- 172.21.15.22:0/2524723000 shutdown_connections 2024-09-19T08:20:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.947+0000 7f8cf2c5e640 1 --2- 172.21.15.22:0/2524723000 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f8cd80a33b0 0x7f8cd80a5870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.947+0000 7f8cf2c5e640 1 --2- 172.21.15.22:0/2524723000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cec106f10 0x7f8cec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.947+0000 7f8cf2c5e640 1 --2- 172.21.15.22:0/2524723000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cec102790 0x7f8cec074a30 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.947+0000 7f8cf2c5e640 1 --2- 172.21.15.22:0/2524723000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cec101de0 0x7f8cec0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.947+0000 7f8cf2c5e640 1 -- 172.21.15.22:0/2524723000 >> 172.21.15.22:0/2524723000 conn(0x7f8cec0fd880 msgr2=0x7f8cec0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.947+0000 7f8cf2c5e640 1 -- 172.21.15.22:0/2524723000 shutdown_connections 2024-09-19T08:20:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:00.947+0000 7f8cf2c5e640 1 -- 172.21.15.22:0/2524723000 wait complete. 2024-09-19T08:20:01.045 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:00 smithi022 ceph-mon[30963]: pgmap v152: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-09-19T08:20:01.045 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:00 smithi022 ceph-mon[30963]: overall HEALTH_OK 2024-09-19T08:20:01.045 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:00 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1717715183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:20:01.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:00 smithi064 ceph-mon[34900]: pgmap v152: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-09-19T08:20:01.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:00 smithi064 ceph-mon[34900]: overall HEALTH_OK 2024-09-19T08:20:01.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:00 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1717715183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:20:01.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:00 smithi157 ceph-mon[35143]: pgmap v152: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-09-19T08:20:01.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:00 smithi157 ceph-mon[35143]: overall HEALTH_OK 2024-09-19T08:20:01.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:00 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1717715183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:20:01.466 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T08:20:01.466 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T08:20:02.157 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:20:02.157 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:20:02.193 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.711+0000 7ffa7bd29640 1 -- 172.21.15.22:0/991174691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa741036b0 msgr2=0x7ffa7410bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.711+0000 7ffa7bd29640 1 --2- 172.21.15.22:0/991174691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa741036b0 0x7ffa7410bba0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009900 tx=0x7ffa6802a480 comp rx=0 tx=0).stop 2024-09-19T08:20:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.711+0000 7ffa7bd29640 1 -- 172.21.15.22:0/991174691 shutdown_connections 2024-09-19T08:20:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.711+0000 7ffa7bd29640 1 --2- 172.21.15.22:0/991174691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7410c170 0x7ffa7410e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.711+0000 7ffa7bd29640 1 --2- 172.21.15.22:0/991174691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa741036b0 0x7ffa7410bba0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.711+0000 7ffa7bd29640 1 --2- 172.21.15.22:0/991174691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa74102d90 0x7ffa74103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.711+0000 7ffa7bd29640 1 -- 172.21.15.22:0/991174691 >> 172.21.15.22:0/991174691 conn(0x7ffa740fc7d0 msgr2=0x7ffa740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.711+0000 7ffa7bd29640 1 -- 172.21.15.22:0/991174691 shutdown_connections 2024-09-19T08:20:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.711+0000 7ffa7bd29640 1 -- 172.21.15.22:0/991174691 wait complete. 2024-09-19T08:20:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.712+0000 7ffa7bd29640 1 Processor -- start 2024-09-19T08:20:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.712+0000 7ffa7bd29640 1 -- start start 2024-09-19T08:20:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.712+0000 7ffa7bd29640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa74102d90 0x7ffa741009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.712+0000 7ffa7bd29640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa741036b0 0x7ffa74100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.712+0000 7ffa7bd29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7410c170 0x7ffa74101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.712+0000 7ffa7bd29640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffa74111f50 con 0x7ffa7410c170 2024-09-19T08:20:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.712+0000 7ffa7bd29640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffa74111dd0 con 0x7ffa74102d90 2024-09-19T08:20:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.712+0000 7ffa7bd29640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa741120d0 con 0x7ffa741036b0 2024-09-19T08:20:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa7a2c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7410c170 0x7ffa74101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa7a2c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7410c170 0x7ffa74101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34586/0 (socket says 172.21.15.22:34586) 2024-09-19T08:20:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa792c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa741036b0 0x7ffa74100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa7a2c6640 1 -- 172.21.15.22:0/404938714 learned_addr learned my addr 172.21.15.22:0/404938714 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa79ac5640 1 --2- 172.21.15.22:0/404938714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa74102d90 0x7ffa741009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa7a2c6640 1 -- 172.21.15.22:0/404938714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa741036b0 msgr2=0x7ffa74100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa7a2c6640 1 --2- 172.21.15.22:0/404938714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa741036b0 0x7ffa74100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa7a2c6640 1 -- 172.21.15.22:0/404938714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa74102d90 msgr2=0x7ffa741009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa7a2c6640 1 --2- 172.21.15.22:0/404938714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa74102d90 0x7ffa741009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa7a2c6640 1 -- 172.21.15.22:0/404938714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa741a68e0 con 0x7ffa7410c170 2024-09-19T08:20:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa79ac5640 1 --2- 172.21.15.22:0/404938714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa74102d90 0x7ffa741009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:20:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa7a2c6640 1 --2- 172.21.15.22:0/404938714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7410c170 0x7ffa74101470 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00eb10 tx=0x7ffa6c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa66ffd640 1 -- 172.21.15.22:0/404938714 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c004b20 con 0x7ffa7410c170 2024-09-19T08:20:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa66ffd640 1 -- 172.21.15.22:0/404938714 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa6c004cc0 con 0x7ffa7410c170 2024-09-19T08:20:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa7bd29640 1 -- 172.21.15.22:0/404938714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa741a6bd0 con 0x7ffa7410c170 2024-09-19T08:20:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa66ffd640 1 -- 172.21.15.22:0/404938714 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c010760 con 0x7ffa7410c170 2024-09-19T08:20:02.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.713+0000 7ffa7bd29640 1 -- 172.21.15.22:0/404938714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa741a7130 con 0x7ffa7410c170 2024-09-19T08:20:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.717+0000 7ffa66ffd640 1 -- 172.21.15.22:0/404938714 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffa6c005020 con 0x7ffa7410c170 2024-09-19T08:20:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.717+0000 7ffa7e102740 1 Processor -- start 2024-09-19T08:20:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa7e102740 1 -- start start 2024-09-19T08:20:02.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa7e102740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2a760 0x558d4bd2ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa7e102740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2b080 0x558d4bd35400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa7e102740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd35940 0x558d4bd37d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa66ffd640 1 --2- 172.21.15.22:0/404938714 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ffa600a3430 0x7ffa600a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa7e102740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558d4bc902a0 con 0x558d4bd2a760 2024-09-19T08:20:02.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa7e102740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558d4bc37b10 con 0x558d4bd35940 2024-09-19T08:20:02.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa7e102740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558d4bc8c260 con 0x558d4bd2b080 2024-09-19T08:20:02.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa79ac5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2b080 0x558d4bd35400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa79ac5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2b080 0x558d4bd35400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:54598/0 (socket says 172.21.15.22:54598) 2024-09-19T08:20:02.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa79ac5640 1 -- 172.21.15.22:0/1832380003 learned_addr learned my addr 172.21.15.22:0/1832380003 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa792c4640 1 --2- 172.21.15.22:0/1832380003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2a760 0x558d4bd2ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa66ffd640 1 -- 172.21.15.22:0/404938714 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ffa6c0d0390 con 0x7ffa7410c170 2024-09-19T08:20:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa7a2c6640 1 --2- 172.21.15.22:0/404938714 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ffa600a3430 0x7ffa600a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa79ac5640 1 -- 172.21.15.22:0/1832380003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd35940 msgr2=0x558d4bd37d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa79ac5640 1 --2- 172.21.15.22:0/1832380003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd35940 0x558d4bd37d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa79ac5640 1 -- 172.21.15.22:0/1832380003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2a760 msgr2=0x558d4bd2ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa79ac5640 1 --2- 172.21.15.22:0/1832380003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2a760 0x558d4bd2ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.718+0000 7ffa79ac5640 1 -- 172.21.15.22:0/1832380003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d4ba01b90 con 0x558d4bd2b080 2024-09-19T08:20:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.719+0000 7ffa79ac5640 1 --2- 172.21.15.22:0/1832380003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2b080 0x558d4bd35400 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000bcc0 tx=0x7ffa7000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4e5aa19b070ee34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.719+0000 7ffa7a2c6640 1 --2- 172.21.15.22:0/404938714 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ffa600a3430 0x7ffa600a58f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0cdba0 tx=0x7ffa6c00efd0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.719+0000 7ffa5ffff640 1 -- 172.21.15.22:0/1832380003 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70005020 con 0x558d4bd2b080 2024-09-19T08:20:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.719+0000 7ffa5ffff640 1 -- 172.21.15.22:0/1832380003 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa700047a0 con 0x558d4bd2b080 2024-09-19T08:20:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.719+0000 7ffa5ffff640 1 -- 172.21.15.22:0/1832380003 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70011710 con 0x558d4bd2b080 2024-09-19T08:20:02.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.720+0000 7ffa7e102740 1 -- 172.21.15.22:0/1832380003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2b080 msgr2=0x558d4bd35400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:02.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.720+0000 7ffa7e102740 1 --2- 172.21.15.22:0/1832380003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2b080 0x558d4bd35400 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000bcc0 tx=0x7ffa7000f730 comp rx=0 tx=0).stop 2024-09-19T08:20:02.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.720+0000 7ffa7e102740 1 -- 172.21.15.22:0/1832380003 shutdown_connections 2024-09-19T08:20:02.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.720+0000 7ffa7e102740 1 --2- 172.21.15.22:0/1832380003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd35940 0x558d4bd37d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.720+0000 7ffa7e102740 1 --2- 172.21.15.22:0/1832380003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2b080 0x558d4bd35400 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.720+0000 7ffa7e102740 1 --2- 172.21.15.22:0/1832380003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2a760 0x558d4bd2ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.720+0000 7ffa7e102740 1 -- 172.21.15.22:0/1832380003 >> 172.21.15.22:0/1832380003 conn(0x558d4bd26cf0 msgr2=0x558d4bd270d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:02.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.720+0000 7ffa7e102740 1 -- 172.21.15.22:0/1832380003 shutdown_connections 2024-09-19T08:20:02.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.720+0000 7ffa7e102740 1 -- 172.21.15.22:0/1832380003 wait complete. 2024-09-19T08:20:02.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.720+0000 7ffa7e102740 1 -- start start 2024-09-19T08:20:02.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.721+0000 7ffa7e102740 1 Processor -- start 2024-09-19T08:20:02.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.721+0000 7ffa7e102740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2a760 0x558d4bd32770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.721+0000 7ffa7e102740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd2b080 0x558d4bd32cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.721+0000 7ffa7e102740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd35940 0x558d4bd331f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.721+0000 7ffa7e102740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558d4bd38510 con 0x558d4bd35940 2024-09-19T08:20:02.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.721+0000 7ffa7e102740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558d4bc186e0 con 0x558d4bd2b080 2024-09-19T08:20:02.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.721+0000 7ffa7e102740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558d4bd38690 con 0x558d4bd2a760 2024-09-19T08:20:02.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.721+0000 7ffa7e102740 10 client.0 _ll_register_callbacks cb 0x558d4bc3ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:02.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.721+0000 7ffa792c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2a760 0x558d4bd32770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.721+0000 7ffa792c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2a760 0x558d4bd32770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:54614/0 (socket says 172.21.15.22:54614) 2024-09-19T08:20:02.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.721+0000 7ffa792c4640 1 -- 172.21.15.22:0/2183816973 learned_addr learned my addr 172.21.15.22:0/2183816973 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:02.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.721+0000 7ffa792c4640 1 --2- 172.21.15.22:0/2183816973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd35940 0x558d4bd331f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.721+0000 7ffa79ac5640 1 --2- 172.21.15.22:0/2183816973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd2b080 0x558d4bd32cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.722+0000 7ffa792c4640 1 -- 172.21.15.22:0/2183816973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd2b080 msgr2=0x558d4bd32cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:02.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.722+0000 7ffa792c4640 1 --2- 172.21.15.22:0/2183816973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd2b080 0x558d4bd32cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.722+0000 7ffa792c4640 1 -- 172.21.15.22:0/2183816973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd35940 msgr2=0x558d4bd331f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:02.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.722+0000 7ffa792c4640 1 --2- 172.21.15.22:0/2183816973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd35940 0x558d4bd331f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.722+0000 7ffa792c4640 1 -- 172.21.15.22:0/2183816973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d4bd41980 con 0x558d4bd2a760 2024-09-19T08:20:02.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.722+0000 7ffa792c4640 1 --2- 172.21.15.22:0/2183816973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2a760 0x558d4bd32770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffa68033000 tx=0x7ffa6802a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f201f5df75663b7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:02.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.722+0000 7ffa5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:20:02.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.722+0000 7ffa5d7fa640 1 -- 172.21.15.22:0/2183816973 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68035040 con 0x558d4bd2a760 2024-09-19T08:20:02.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.722+0000 7ffa7e102740 1 -- 172.21.15.22:0/2183816973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa700047a0 con 0x558d4bd2a760 2024-09-19T08:20:02.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.722+0000 7ffa5d7fa640 1 -- 172.21.15.22:0/2183816973 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ffa68004260 con 0x558d4bd2a760 2024-09-19T08:20:02.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.722+0000 7ffa5d7fa640 1 -- 172.21.15.22:0/2183816973 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa68052050 con 0x558d4bd2a760 2024-09-19T08:20:02.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.722+0000 7ffa7e102740 10 client.25891 fetch_fsmap learned FSMap version 29 2024-09-19T08:20:02.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.722+0000 7ffa7e102740 1 -- 172.21.15.22:0/2183816973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558d4bd44330 con 0x558d4bd2a760 2024-09-19T08:20:02.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.722+0000 7ffa5d7fa640 1 -- 172.21.15.22:0/2183816973 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa700047a0 con 0x558d4bd2a760 2024-09-19T08:20:02.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.723+0000 7ffa5d7fa640 1 -- 172.21.15.22:0/2183816973 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ffa68003710 con 0x558d4bd2a760 2024-09-19T08:20:02.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.723+0000 7ffa7e102740 10 client.25891 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:20:02.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.723+0000 7ffa7e102740 10 client.25891 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:20:02.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.723+0000 7ffa7e102740 20 client.25891 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:02.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.723+0000 7ffa7e102740 1 --2- 172.21.15.22:0/2183816973 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558d4bd45740 0x558d4bd47b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.723+0000 7ffa7e102740 4 client.25891 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:20:02.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.723+0000 7ffa7e102740 1 -- 172.21.15.22:0/2183816973 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d4bc9d9d0 con 0x558d4bd45740 2024-09-19T08:20:02.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.723+0000 7ffa79ac5640 1 --2- 172.21.15.22:0/2183816973 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558d4bd45740 0x558d4bd47b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.723+0000 7ffa79ac5640 1 --2- 172.21.15.22:0/2183816973 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558d4bd45740 0x558d4bd47b30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c11617d91e9a4646 server_cookie=85dfeda64958f8d9 in_seq=0 out_seq=0 2024-09-19T08:20:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.723+0000 7ffa5d7fa640 10 client.25891 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.724+0000 7ffa5d7fa640 1 -- 172.21.15.22:0/2183816973 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d4bc9d9d0 con 0x558d4bd45740 2024-09-19T08:20:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.724+0000 7ffa5d7fa640 10 client.25891 handle_command_reply: tid=0 2024-09-19T08:20:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.724+0000 7ffa7e102740 1 client.25891 shutdown 2024-09-19T08:20:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.724+0000 7ffa7e102740 1 -- 172.21.15.22:0/2183816973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2a760 msgr2=0x558d4bd32770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.724+0000 7ffa7e102740 1 --2- 172.21.15.22:0/2183816973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2a760 0x558d4bd32770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffa68033000 tx=0x7ffa6802a480 comp rx=0 tx=0).stop 2024-09-19T08:20:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.725+0000 7ffa7e102740 1 -- 172.21.15.22:0/2183816973 shutdown_connections 2024-09-19T08:20:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.725+0000 7ffa7e102740 1 --2- 172.21.15.22:0/2183816973 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558d4bd45740 0x558d4bd47b30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.725+0000 7ffa7e102740 1 --2- 172.21.15.22:0/2183816973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd35940 0x558d4bd331f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.725+0000 7ffa7e102740 1 --2- 172.21.15.22:0/2183816973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd2b080 0x558d4bd32cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.725+0000 7ffa5d7fa640 0 client.25891 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:02.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.725+0000 7ffa7e102740 1 --2- 172.21.15.22:0/2183816973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2a760 0x558d4bd32770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.725+0000 7ffa7e102740 1 -- 172.21.15.22:0/2183816973 >> 172.21.15.22:0/2183816973 conn(0x558d4bd26cf0 msgr2=0x558d4bd25e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:02.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.725+0000 7ffa7e102740 1 -- 172.21.15.22:0/2183816973 shutdown_connections 2024-09-19T08:20:02.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.725+0000 7ffa7e102740 1 -- 172.21.15.22:0/2183816973 wait complete. 2024-09-19T08:20:02.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.725+0000 7ffa7e102740 20 client.25891 trim_cache size 0 max 16384 2024-09-19T08:20:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa7e102740 1 Processor -- start 2024-09-19T08:20:02.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa7e102740 1 -- start start 2024-09-19T08:20:02.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa7e102740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2a760 0x558d4bd32050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa7e102740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd2b080 0x558d4bd32e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa7e102740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd35940 0x558d4bd33380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa7e102740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffa741b23f0 con 0x558d4bd2a760 2024-09-19T08:20:02.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa7e102740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffa741afc60 con 0x558d4bd2b080 2024-09-19T08:20:02.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa7e102740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa741adab0 con 0x558d4bd35940 2024-09-19T08:20:02.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa792c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2a760 0x558d4bd32050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa792c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2a760 0x558d4bd32050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34626/0 (socket says 172.21.15.22:34626) 2024-09-19T08:20:02.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa792c4640 1 -- 172.21.15.22:0/401555034 learned_addr learned my addr 172.21.15.22:0/401555034 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:02.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa79ac5640 1 --2- 172.21.15.22:0/401555034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd2b080 0x558d4bd32e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa792c4640 1 --2- 172.21.15.22:0/401555034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd35940 0x558d4bd33380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa792c4640 1 -- 172.21.15.22:0/401555034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd35940 msgr2=0x558d4bd33380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa792c4640 1 --2- 172.21.15.22:0/401555034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd35940 0x558d4bd33380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa792c4640 1 -- 172.21.15.22:0/401555034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd2b080 msgr2=0x558d4bd32e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa792c4640 1 --2- 172.21.15.22:0/401555034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd2b080 0x558d4bd32e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.745+0000 7ffa792c4640 1 -- 172.21.15.22:0/401555034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d4bd48190 con 0x558d4bd2a760 2024-09-19T08:20:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.746+0000 7ffa792c4640 1 --2- 172.21.15.22:0/401555034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2a760 0x558d4bd32050 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ffa6803a3f0 tx=0x7ffa68043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c148806978ae3019 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.746+0000 7ffa5f7fe640 1 -- 172.21.15.22:0/401555034 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68035070 con 0x558d4bd2a760 2024-09-19T08:20:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.746+0000 7ffa5f7fe640 1 -- 172.21.15.22:0/401555034 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa68003cb0 con 0x558d4bd2a760 2024-09-19T08:20:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.746+0000 7ffa5f7fe640 1 -- 172.21.15.22:0/401555034 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6803e970 con 0x558d4bd2a760 2024-09-19T08:20:02.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.747+0000 7ffa7e102740 1 -- 172.21.15.22:0/401555034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2a760 msgr2=0x558d4bd32050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:02.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.747+0000 7ffa7e102740 1 --2- 172.21.15.22:0/401555034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2a760 0x558d4bd32050 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ffa6803a3f0 tx=0x7ffa68043040 comp rx=0 tx=0).stop 2024-09-19T08:20:02.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.747+0000 7ffa7e102740 1 -- 172.21.15.22:0/401555034 shutdown_connections 2024-09-19T08:20:02.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.747+0000 7ffa7e102740 1 --2- 172.21.15.22:0/401555034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd35940 0x558d4bd33380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.747+0000 7ffa7e102740 1 --2- 172.21.15.22:0/401555034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd2b080 0x558d4bd32e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.747+0000 7ffa7e102740 1 --2- 172.21.15.22:0/401555034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2a760 0x558d4bd32050 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.747+0000 7ffa7e102740 1 -- 172.21.15.22:0/401555034 >> 172.21.15.22:0/401555034 conn(0x558d4bd26cf0 msgr2=0x558d4bd32730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.747+0000 7ffa7e102740 1 -- 172.21.15.22:0/401555034 shutdown_connections 2024-09-19T08:20:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.747+0000 7ffa7e102740 1 -- 172.21.15.22:0/401555034 wait complete. 2024-09-19T08:20:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.747+0000 7ffa7e102740 1 -- start start 2024-09-19T08:20:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.747+0000 7ffa7e102740 1 Processor -- start 2024-09-19T08:20:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa7e102740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2a760 0x558d4bd39d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa7e102740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2b080 0x558d4bd3a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa7e102740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd35940 0x558d4bdaf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa7e102740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffa741ae120 con 0x558d4bd2b080 2024-09-19T08:20:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa7e102740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffa741af590 con 0x558d4bd35940 2024-09-19T08:20:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa7e102740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa741ae7f0 con 0x558d4bd2a760 2024-09-19T08:20:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa7e102740 10 client.0 _ll_register_callbacks cb 0x7ffa68031ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa79ac5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2b080 0x558d4bd3a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa792c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2a760 0x558d4bd39d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa79ac5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2b080 0x558d4bd3a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34642/0 (socket says 172.21.15.22:34642) 2024-09-19T08:20:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa79ac5640 1 -- 172.21.15.22:0/191130775 learned_addr learned my addr 172.21.15.22:0/191130775 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa792c4640 1 --2- 172.21.15.22:0/191130775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd35940 0x558d4bdaf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa79ac5640 1 -- 172.21.15.22:0/191130775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2a760 msgr2=0x558d4bd39d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa79ac5640 1 --2- 172.21.15.22:0/191130775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2a760 0x558d4bd39d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa79ac5640 1 -- 172.21.15.22:0/191130775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd35940 msgr2=0x558d4bdaf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa79ac5640 1 --2- 172.21.15.22:0/191130775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd35940 0x558d4bdaf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa79ac5640 1 -- 172.21.15.22:0/191130775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d4bd22260 con 0x558d4bd2b080 2024-09-19T08:20:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.748+0000 7ffa792c4640 1 --2- 172.21.15.22:0/191130775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd35940 0x558d4bdaf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:20:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.749+0000 7ffa79ac5640 1 --2- 172.21.15.22:0/191130775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2b080 0x558d4bd3a2c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x558d4bd27b50 tx=0x7ffa7000b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fb254872b04e7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.749+0000 7ffa5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.749+0000 7ffa5d7fa640 1 -- 172.21.15.22:0/191130775 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7002c020 con 0x558d4bd2b080 2024-09-19T08:20:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.749+0000 7ffa7e102740 1 -- 172.21.15.22:0/191130775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa68003cb0 con 0x558d4bd2b080 2024-09-19T08:20:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.749+0000 7ffa5d7fa640 1 -- 172.21.15.22:0/191130775 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ffa70005510 con 0x558d4bd2b080 2024-09-19T08:20:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.749+0000 7ffa5d7fa640 1 -- 172.21.15.22:0/191130775 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa70037050 con 0x558d4bd2b080 2024-09-19T08:20:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.749+0000 7ffa7e102740 10 client.16368 fetch_fsmap learned FSMap version 29 2024-09-19T08:20:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.749+0000 7ffa7e102740 1 -- 172.21.15.22:0/191130775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558d4bd26b20 con 0x558d4bd2b080 2024-09-19T08:20:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.749+0000 7ffa5d7fa640 1 -- 172.21.15.22:0/191130775 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa68003cb0 con 0x558d4bd2b080 2024-09-19T08:20:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.749+0000 7ffa5d7fa640 1 -- 172.21.15.22:0/191130775 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ffa70011710 con 0x558d4bd2b080 2024-09-19T08:20:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.750+0000 7ffa7e102740 10 client.16368 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:20:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.750+0000 7ffa7e102740 10 client.16368 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:20:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.750+0000 7ffa7e102740 20 client.16368 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.750+0000 7ffa7e102740 1 --2- 172.21.15.22:0/191130775 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558d4bdb0e60 0x558d4bdb3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.750+0000 7ffa7e102740 4 client.16368 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T08:20:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.750+0000 7ffa7e102740 1 -- 172.21.15.22:0/191130775 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x558d4bc97d20 con 0x558d4bdb0e60 2024-09-19T08:20:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.750+0000 7ffa792c4640 1 --2- 172.21.15.22:0/191130775 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558d4bdb0e60 0x558d4bdb3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.750+0000 7ffa5d7fa640 10 client.16368 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:02.750+0000 7ffa792c4640 1 --2- 172.21.15.22:0/191130775 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558d4bdb0e60 0x558d4bdb3280 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec0fdc33da48074f server_cookie=47d0e3d8a6faf658 in_seq=0 out_seq=0 2024-09-19T08:20:02.770 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:03.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:02 smithi022 ceph-mon[30963]: pgmap v153: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-09-19T08:20:03.089 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:02 smithi064 ceph-mon[34900]: pgmap v153: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-09-19T08:20:03.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.239+0000 7fa386420640 1 -- 172.21.15.22:0/1105180442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa380106a60 msgr2=0x7fa380106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:03.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.239+0000 7fa386420640 1 --2- 172.21.15.22:0/1105180442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa380106a60 0x7fa380106e40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa370009930 tx=0x7fa37002a420 comp rx=0 tx=0).stop 2024-09-19T08:20:03.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.239+0000 7fa386420640 1 -- 172.21.15.22:0/1105180442 shutdown_connections 2024-09-19T08:20:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.239+0000 7fa386420640 1 --2- 172.21.15.22:0/1105180442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380101400 0x7fa38010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.239+0000 7fa386420640 1 --2- 172.21.15.22:0/1105180442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa380100a60 0x7fa380100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.239+0000 7fa386420640 1 --2- 172.21.15.22:0/1105180442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa380106a60 0x7fa380106e40 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.239+0000 7fa386420640 1 -- 172.21.15.22:0/1105180442 >> 172.21.15.22:0/1105180442 conn(0x7fa3800fc780 msgr2=0x7fa3800feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.239+0000 7fa386420640 1 -- 172.21.15.22:0/1105180442 shutdown_connections 2024-09-19T08:20:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.239+0000 7fa386420640 1 -- 172.21.15.22:0/1105180442 wait complete. 2024-09-19T08:20:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.239+0000 7fa386420640 1 Processor -- start 2024-09-19T08:20:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.239+0000 7fa386420640 1 -- start start 2024-09-19T08:20:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa386420640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa380100a60 0x7fa380198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa386420640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa380101400 0x7fa380198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa386420640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380106a60 0x7fa38019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa386420640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa380110e60 con 0x7fa380101400 2024-09-19T08:20:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa386420640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa380110ce0 con 0x7fa380100a60 2024-09-19T08:20:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa386420640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa380110fe0 con 0x7fa380106a60 2024-09-19T08:20:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa37f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa380101400 0x7fa380198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa37ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa380100a60 0x7fa380198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:03.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa37ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa380100a60 0x7fa380198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:32792/0 (socket says 172.21.15.22:32792) 2024-09-19T08:20:03.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa37ffff640 1 -- 172.21.15.22:0/2429652225 learned_addr learned my addr 172.21.15.22:0/2429652225 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:03.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa37f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa380101400 0x7fa380198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34658/0 (socket says 172.21.15.22:34658) 2024-09-19T08:20:03.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa3849bd640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380106a60 0x7fa38019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:03.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa37ffff640 1 -- 172.21.15.22:0/2429652225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380106a60 msgr2=0x7fa38019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:03.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa37ffff640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380106a60 0x7fa38019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:03.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa37ffff640 1 -- 172.21.15.22:0/2429652225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa380101400 msgr2=0x7fa380198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:03.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa37ffff640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa380101400 0x7fa380198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:03.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa37ffff640 1 -- 172.21.15.22:0/2429652225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38019d400 con 0x7fa380100a60 2024-09-19T08:20:03.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa3849bd640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380106a60 0x7fa38019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:03.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.240+0000 7fa37f7fe640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa380101400 0x7fa380198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:03.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.241+0000 7fa37ffff640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa380100a60 0x7fa380198450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa370009230 tx=0x7fa370004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:03.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.241+0000 7fa37d7fa640 1 -- 172.21.15.22:0/2429652225 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37003f030 con 0x7fa380100a60 2024-09-19T08:20:03.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.241+0000 7fa37d7fa640 1 -- 172.21.15.22:0/2429652225 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa370004440 con 0x7fa380100a60 2024-09-19T08:20:03.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.241+0000 7fa386420640 1 -- 172.21.15.22:0/2429652225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38019d690 con 0x7fa380100a60 2024-09-19T08:20:03.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.241+0000 7fa37d7fa640 1 -- 172.21.15.22:0/2429652225 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370035070 con 0x7fa380100a60 2024-09-19T08:20:03.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.241+0000 7fa386420640 1 -- 172.21.15.22:0/2429652225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3800fcda0 con 0x7fa380100a60 2024-09-19T08:20:03.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.242+0000 7fa386420640 1 -- 172.21.15.22:0/2429652225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3801021a0 con 0x7fa380100a60 2024-09-19T08:20:03.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.244+0000 7fa37d7fa640 1 -- 172.21.15.22:0/2429652225 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa370042030 con 0x7fa380100a60 2024-09-19T08:20:03.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.244+0000 7fa37d7fa640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3680a33b0 0x7fa3680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:03.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.245+0000 7fa37f7fe640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3680a33b0 0x7fa3680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:03.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.245+0000 7fa37d7fa640 1 -- 172.21.15.22:0/2429652225 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa3700f0420 con 0x7fa380100a60 2024-09-19T08:20:03.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.245+0000 7fa37f7fe640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3680a33b0 0x7fa3680a5870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa380104a60 tx=0x7fa374005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:03.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.248+0000 7fa37d7fa640 1 -- 172.21.15.22:0/2429652225 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa3700bc7c0 con 0x7fa380100a60 2024-09-19T08:20:03.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:02 smithi157 ceph-mon[35143]: pgmap v153: 97 pgs: 97 active+clean; 612 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-09-19T08:20:03.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.462+0000 7fa386420640 1 -- 172.21.15.22:0/2429652225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa380109fe0 con 0x7fa380100a60 2024-09-19T08:20:03.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.463+0000 7fa37d7fa640 1 -- 172.21.15.22:0/2429652225 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11909 (secure 0 0 0) 0x7fa3700c0ac0 con 0x7fa380100a60 2024-09-19T08:20:03.463 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:20:03.463 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":29,"btime":"2024-09-19T08:18:57:303741+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:18:56.870055+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:20:03.463 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 29 2024-09-19T08:20:03.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.465+0000 7fa386420640 1 -- 172.21.15.22:0/2429652225 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3680a33b0 msgr2=0x7fa3680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:03.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.465+0000 7fa386420640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3680a33b0 0x7fa3680a5870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa380104a60 tx=0x7fa374005f90 comp rx=0 tx=0).stop 2024-09-19T08:20:03.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.465+0000 7fa386420640 1 -- 172.21.15.22:0/2429652225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa380100a60 msgr2=0x7fa380198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:03.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.465+0000 7fa386420640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa380100a60 0x7fa380198450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa370009230 tx=0x7fa370004040 comp rx=0 tx=0).stop 2024-09-19T08:20:03.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.465+0000 7fa386420640 1 -- 172.21.15.22:0/2429652225 shutdown_connections 2024-09-19T08:20:03.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.465+0000 7fa386420640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa3680a33b0 0x7fa3680a5870 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:03.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.465+0000 7fa386420640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa380106a60 0x7fa38019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:03.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.465+0000 7fa386420640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa380101400 0x7fa380198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:03.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.465+0000 7fa386420640 1 --2- 172.21.15.22:0/2429652225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa380100a60 0x7fa380198450 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:03.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.465+0000 7fa386420640 1 -- 172.21.15.22:0/2429652225 >> 172.21.15.22:0/2429652225 conn(0x7fa3800fc780 msgr2=0x7fa38006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:03.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.465+0000 7fa386420640 1 -- 172.21.15.22:0/2429652225 shutdown_connections 2024-09-19T08:20:03.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:03.465+0000 7fa386420640 1 -- 172.21.15.22:0/2429652225 wait complete. 2024-09-19T08:20:04.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:03 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2429652225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:20:04.148 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:03 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2429652225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:20:04.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:20:04.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:03 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2429652225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:20:04.490 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:04.860 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:04 smithi022 ceph-mon[30963]: pgmap v154: 97 pgs: 97 active+clean; 613 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-09-19T08:20:04.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.949+0000 7f3ae6703640 1 -- 172.21.15.22:0/1538331849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ae0102d20 msgr2=0x7f3ae0103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:04.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.949+0000 7f3ae6703640 1 --2- 172.21.15.22:0/1538331849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ae0102d20 0x7f3ae0103100 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009900 tx=0x7f3ad002a480 comp rx=0 tx=0).stop 2024-09-19T08:20:04.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.949+0000 7f3ae6703640 1 -- 172.21.15.22:0/1538331849 shutdown_connections 2024-09-19T08:20:04.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.949+0000 7f3ae6703640 1 --2- 172.21.15.22:0/1538331849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ae010c150 0x7f3ae010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:04.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.949+0000 7f3ae6703640 1 --2- 172.21.15.22:0/1538331849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae0103640 0x7f3ae010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:04.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.949+0000 7f3ae6703640 1 --2- 172.21.15.22:0/1538331849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ae0102d20 0x7f3ae0103100 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:04.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.949+0000 7f3ae6703640 1 -- 172.21.15.22:0/1538331849 >> 172.21.15.22:0/1538331849 conn(0x7f3ae00fc780 msgr2=0x7f3ae00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:04.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.949+0000 7f3ae6703640 1 -- 172.21.15.22:0/1538331849 shutdown_connections 2024-09-19T08:20:04.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.949+0000 7f3ae6703640 1 -- 172.21.15.22:0/1538331849 wait complete. 2024-09-19T08:20:04.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.950+0000 7f3ae6703640 1 Processor -- start 2024-09-19T08:20:04.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.950+0000 7f3ae6703640 1 -- start start 2024-09-19T08:20:04.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.950+0000 7f3ae6703640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ae0102d20 0x7f3ae01984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:04.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.950+0000 7f3ae6703640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae0103640 0x7f3ae01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:04.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.950+0000 7f3ae6703640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ae010c150 0x7f3ae019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:04.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.950+0000 7f3ae6703640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae0111f50 con 0x7f3ae0102d20 2024-09-19T08:20:04.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.950+0000 7f3ae6703640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae0111dd0 con 0x7f3ae010c150 2024-09-19T08:20:04.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.950+0000 7f3ae6703640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae01120d0 con 0x7f3ae0103640 2024-09-19T08:20:04.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.950+0000 7f3adffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ae0102d20 0x7f3ae01984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:04.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.950+0000 7f3adffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ae0102d20 0x7f3ae01984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34670/0 (socket says 172.21.15.22:34670) 2024-09-19T08:20:04.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.950+0000 7f3adffff640 1 -- 172.21.15.22:0/4070159180 learned_addr learned my addr 172.21.15.22:0/4070159180 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:04.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.950+0000 7f3ad7fff640 1 --2- 172.21.15.22:0/4070159180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae0103640 0x7f3ae01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:04.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.950+0000 7f3ae4ca0640 1 --2- 172.21.15.22:0/4070159180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ae010c150 0x7f3ae019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:04.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.951+0000 7f3adffff640 1 -- 172.21.15.22:0/4070159180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae0103640 msgr2=0x7f3ae01989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:04.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.951+0000 7f3adffff640 1 --2- 172.21.15.22:0/4070159180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae0103640 0x7f3ae01989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:04.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.951+0000 7f3adffff640 1 -- 172.21.15.22:0/4070159180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ae010c150 msgr2=0x7f3ae019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:04.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.951+0000 7f3adffff640 1 --2- 172.21.15.22:0/4070159180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ae010c150 0x7f3ae019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:04.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.951+0000 7f3adffff640 1 -- 172.21.15.22:0/4070159180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae019d450 con 0x7f3ae0102d20 2024-09-19T08:20:04.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.951+0000 7f3adffff640 1 --2- 172.21.15.22:0/4070159180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ae0102d20 0x7f3ae01984a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0006010 tx=0x7f3ad00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:04.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.951+0000 7f3addffb640 1 -- 172.21.15.22:0/4070159180 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad003f030 con 0x7f3ae0102d20 2024-09-19T08:20:04.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.951+0000 7f3addffb640 1 -- 172.21.15.22:0/4070159180 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ad0003ce0 con 0x7f3ae0102d20 2024-09-19T08:20:04.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.951+0000 7f3ae6703640 1 -- 172.21.15.22:0/4070159180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae019d6e0 con 0x7f3ae0102d20 2024-09-19T08:20:04.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.951+0000 7f3addffb640 1 -- 172.21.15.22:0/4070159180 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0035070 con 0x7f3ae0102d20 2024-09-19T08:20:04.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.951+0000 7f3ae6703640 1 -- 172.21.15.22:0/4070159180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae01a5020 con 0x7f3ae0102d20 2024-09-19T08:20:04.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3addffb640 1 -- 172.21.15.22:0/4070159180 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3ad0042030 con 0x7f3ae0102d20 2024-09-19T08:20:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3ae8adc740 1 Processor -- start 2024-09-19T08:20:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3ae8adc740 1 -- start start 2024-09-19T08:20:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3ae8adc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a9b40 0x5576f59a8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3ae8adc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59a85d0 0x5576f59a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3ae8adc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59b7780 0x5576f59b9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3ae8adc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5576f590e2f0 con 0x5576f59a85d0 2024-09-19T08:20:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3ae8adc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5576f58b5cf0 con 0x5576f59b7780 2024-09-19T08:20:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3ae8adc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5576f590a2b0 con 0x5576f59a9b40 2024-09-19T08:20:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3adffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59a85d0 0x5576f59a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3ae4ca0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a9b40 0x5576f59a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3ae4ca0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a9b40 0x5576f59a8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:54694/0 (socket says 172.21.15.22:54694) 2024-09-19T08:20:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3ae4ca0640 1 -- 172.21.15.22:0/3882749734 learned_addr learned my addr 172.21.15.22:0/3882749734 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:04.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3ad7fff640 1 --2- 172.21.15.22:0/3882749734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59b7780 0x5576f59b9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.954+0000 7f3addffb640 1 --2- 172.21.15.22:0/4070159180 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3acc0a33b0 0x7f3acc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.955+0000 7f3ae4ca0640 1 -- 172.21.15.22:0/3882749734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59b7780 msgr2=0x5576f59b9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.955+0000 7f3ae4ca0640 1 --2- 172.21.15.22:0/3882749734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59b7780 0x5576f59b9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.955+0000 7f3ae4ca0640 1 -- 172.21.15.22:0/3882749734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59a85d0 msgr2=0x5576f59a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.955+0000 7f3ae4ca0640 1 --2- 172.21.15.22:0/3882749734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59a85d0 0x5576f59a8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.955+0000 7f3ae4ca0640 1 -- 172.21.15.22:0/3882749734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576f589a3a0 con 0x5576f59a9b40 2024-09-19T08:20:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.955+0000 7f3adffff640 1 --2- 172.21.15.22:0/3882749734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59a85d0 0x5576f59a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.955+0000 7f3ad7fff640 1 --2- 172.21.15.22:0/3882749734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59b7780 0x5576f59b9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:20:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.955+0000 7f3addffb640 1 -- 172.21.15.22:0/4070159180 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3ad00f00b0 con 0x7f3ae0102d20 2024-09-19T08:20:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.955+0000 7f3ae4ca0640 1 --2- 172.21.15.22:0/3882749734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a9b40 0x5576f59a8090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009f70 tx=0x7f3ad8010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe43add6a1891c19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.955+0000 7f3ae4ca0640 1 --2- 172.21.15.22:0/4070159180 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3acc0a33b0 0x7f3acc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:04.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.955+0000 7f3ad67fc640 1 -- 172.21.15.22:0/3882749734 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8016070 con 0x5576f59a9b40 2024-09-19T08:20:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.955+0000 7f3ad67fc640 1 -- 172.21.15.22:0/3882749734 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ad8012450 con 0x5576f59a9b40 2024-09-19T08:20:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.955+0000 7f3ad67fc640 1 -- 172.21.15.22:0/3882749734 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8011700 con 0x5576f59a9b40 2024-09-19T08:20:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.956+0000 7f3ae8adc740 1 -- 172.21.15.22:0/3882749734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a9b40 msgr2=0x5576f59a8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.956+0000 7f3ae4ca0640 1 --2- 172.21.15.22:0/4070159180 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3acc0a33b0 0x7f3acc0a5870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8011010 tx=0x7f3ad8011e20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.956+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/3882749734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a9b40 0x5576f59a8090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009f70 tx=0x7f3ad8010800 comp rx=0 tx=0).stop 2024-09-19T08:20:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.956+0000 7f3ae8adc740 1 -- 172.21.15.22:0/3882749734 shutdown_connections 2024-09-19T08:20:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.956+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/3882749734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59b7780 0x5576f59b9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.956+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/3882749734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59a85d0 0x5576f59a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.956+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/3882749734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a9b40 0x5576f59a8090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.956+0000 7f3ae8adc740 1 -- 172.21.15.22:0/3882749734 >> 172.21.15.22:0/3882749734 conn(0x5576f5971310 msgr2=0x5576f59a4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.956+0000 7f3ae8adc740 1 -- 172.21.15.22:0/3882749734 shutdown_connections 2024-09-19T08:20:04.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.956+0000 7f3ae8adc740 1 -- 172.21.15.22:0/3882749734 wait complete. 2024-09-19T08:20:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.956+0000 7f3ae8adc740 1 -- start start 2024-09-19T08:20:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.956+0000 7f3ae8adc740 1 Processor -- start 2024-09-19T08:20:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae8adc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a85d0 0x5576f59b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae8adc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a9b40 0x5576f59b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae8adc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59b7780 0x5576f59b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae8adc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5576f59ba480 con 0x5576f59b7780 2024-09-19T08:20:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae8adc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5576f58c9060 con 0x5576f59a85d0 2024-09-19T08:20:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae8adc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5576f59ba600 con 0x5576f59a9b40 2024-09-19T08:20:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae8adc740 10 client.0 _ll_register_callbacks cb 0x5576f561c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae4ca0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59b7780 0x5576f59b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ad7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a9b40 0x5576f59b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:04.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae4ca0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59b7780 0x5576f59b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34694/0 (socket says 172.21.15.22:34694) 2024-09-19T08:20:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae4ca0640 1 -- 172.21.15.22:0/3162196100 learned_addr learned my addr 172.21.15.22:0/3162196100 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3adffff640 1 --2- 172.21.15.22:0/3162196100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a85d0 0x5576f59b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae4ca0640 1 -- 172.21.15.22:0/3162196100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a9b40 msgr2=0x5576f59b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae4ca0640 1 --2- 172.21.15.22:0/3162196100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a9b40 0x5576f59b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae4ca0640 1 -- 172.21.15.22:0/3162196100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a85d0 msgr2=0x5576f59b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae4ca0640 1 --2- 172.21.15.22:0/3162196100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a85d0 0x5576f59b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae4ca0640 1 -- 172.21.15.22:0/3162196100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576f567f3c0 con 0x5576f59b7780 2024-09-19T08:20:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3ae4ca0640 1 --2- 172.21.15.22:0/3162196100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59b7780 0x5576f59b49e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8012eb0 tx=0x7f3ad801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc5050c208b5af4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:04.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.957+0000 7f3abf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3abf7fe640 1 -- 172.21.15.22:0/3162196100 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8035060 con 0x5576f59b7780 2024-09-19T08:20:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3ae8adc740 1 -- 172.21.15.22:0/3162196100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ad8012450 con 0x5576f59b7780 2024-09-19T08:20:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3abf7fe640 1 -- 172.21.15.22:0/3162196100 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3ad801bc30 con 0x5576f59b7780 2024-09-19T08:20:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3abf7fe640 1 -- 172.21.15.22:0/3162196100 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3ad8016040 con 0x5576f59b7780 2024-09-19T08:20:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3ae8adc740 10 client.16386 fetch_fsmap learned FSMap version 29 2024-09-19T08:20:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3ae8adc740 1 -- 172.21.15.22:0/3162196100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5576f59c5e50 con 0x5576f59b7780 2024-09-19T08:20:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3abf7fe640 1 -- 172.21.15.22:0/3162196100 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ad8012450 con 0x5576f59b7780 2024-09-19T08:20:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3abf7fe640 1 -- 172.21.15.22:0/3162196100 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3ad800f040 con 0x5576f59b7780 2024-09-19T08:20:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3ae8adc740 10 client.16386 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:20:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3ae8adc740 10 client.16386 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:20:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3ae8adc740 20 client.16386 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:04.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/3162196100 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5576f59c72b0 0x5576f59c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:04.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3ae8adc740 4 client.16386 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:20:04.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3ae8adc740 1 -- 172.21.15.22:0/3162196100 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576f591b860 con 0x5576f59c72b0 2024-09-19T08:20:04.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.958+0000 7f3adffff640 1 --2- 172.21.15.22:0/3162196100 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5576f59c72b0 0x5576f59c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:04.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.959+0000 7f3adffff640 1 --2- 172.21.15.22:0/3162196100 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5576f59c72b0 0x5576f59c96a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37c678d52f34eaba server_cookie=3929443e23da7380 in_seq=0 out_seq=0 2024-09-19T08:20:04.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:04.959+0000 7f3abf7fe640 10 client.16386 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:20:05.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.020+0000 7f3abf7fe640 1 -- 172.21.15.22:0/3162196100 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576f591b860 con 0x5576f59c72b0 2024-09-19T08:20:05.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.020+0000 7f3abf7fe640 10 client.16386 handle_command_reply: tid=0 2024-09-19T08:20:05.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.020+0000 7f3ae8adc740 1 client.16386 shutdown 2024-09-19T08:20:05.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.021+0000 7f3ae8adc740 1 -- 172.21.15.22:0/3162196100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59b7780 msgr2=0x5576f59b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:05.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.021+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/3162196100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59b7780 0x5576f59b49e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8012eb0 tx=0x7f3ad801b000 comp rx=0 tx=0).stop 2024-09-19T08:20:05.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.021+0000 7f3ae8adc740 1 -- 172.21.15.22:0/3162196100 shutdown_connections 2024-09-19T08:20:05.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.021+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/3162196100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59b7780 0x5576f59b49e0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.021+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/3162196100 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5576f59c72b0 0x5576f59c96a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.021+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/3162196100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a9b40 0x5576f59b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.021+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/3162196100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a85d0 0x5576f59b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.021+0000 7f3abf7fe640 0 client.16386 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:20:05.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.021+0000 7f3ae8adc740 1 -- 172.21.15.22:0/3162196100 >> 172.21.15.22:0/3162196100 conn(0x5576f5971310 msgr2=0x5576f59aa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:05.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.021+0000 7f3ae8adc740 1 -- 172.21.15.22:0/3162196100 shutdown_connections 2024-09-19T08:20:05.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.021+0000 7f3ae8adc740 1 -- 172.21.15.22:0/3162196100 wait complete. 2024-09-19T08:20:05.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.021+0000 7f3ae8adc740 20 client.16386 trim_cache size 0 max 16384 2024-09-19T08:20:05.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.038+0000 7f3ae8adc740 1 Processor -- start 2024-09-19T08:20:05.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.038+0000 7f3ae8adc740 1 -- start start 2024-09-19T08:20:05.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ae8adc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a85d0 0x5576f59c3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:05.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ae8adc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59a9b40 0x5576f59a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:05.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ae8adc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59b7780 0x5576f59cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:05.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ae8adc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae01ae050 con 0x5576f59a9b40 2024-09-19T08:20:05.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ae8adc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae01ab8c0 con 0x5576f59a85d0 2024-09-19T08:20:05.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ae8adc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae01a9710 con 0x5576f59b7780 2024-09-19T08:20:05.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ad7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59a9b40 0x5576f59a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:05.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ad7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59a9b40 0x5576f59a7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34710/0 (socket says 172.21.15.22:34710) 2024-09-19T08:20:05.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ad7fff640 1 -- 172.21.15.22:0/4165119567 learned_addr learned my addr 172.21.15.22:0/4165119567 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:05.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3adffff640 1 --2- 172.21.15.22:0/4165119567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a85d0 0x5576f59c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:05.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ae4ca0640 1 --2- 172.21.15.22:0/4165119567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59b7780 0x5576f59cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:05.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ad7fff640 1 -- 172.21.15.22:0/4165119567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59b7780 msgr2=0x5576f59cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:05.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ad7fff640 1 --2- 172.21.15.22:0/4165119567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59b7780 0x5576f59cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ad7fff640 1 -- 172.21.15.22:0/4165119567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a85d0 msgr2=0x5576f59c3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:05.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ad7fff640 1 --2- 172.21.15.22:0/4165119567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a85d0 0x5576f59c3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ad7fff640 1 -- 172.21.15.22:0/4165119567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576f59a5960 con 0x5576f59a9b40 2024-09-19T08:20:05.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.039+0000 7f3ad7fff640 1 --2- 172.21.15.22:0/4165119567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59a9b40 0x5576f59a7de0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009e60 tx=0x7f3ac8013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5352880d51b237a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:05.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.040+0000 7f3ad5ffb640 1 -- 172.21.15.22:0/4165119567 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8021070 con 0x5576f59a9b40 2024-09-19T08:20:05.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.040+0000 7f3ad5ffb640 1 -- 172.21.15.22:0/4165119567 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac8010910 con 0x5576f59a9b40 2024-09-19T08:20:05.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.040+0000 7f3ad5ffb640 1 -- 172.21.15.22:0/4165119567 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac801cad0 con 0x5576f59a9b40 2024-09-19T08:20:05.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.041+0000 7f3ae8adc740 1 -- 172.21.15.22:0/4165119567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59a9b40 msgr2=0x5576f59a7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:05.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.041+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/4165119567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59a9b40 0x5576f59a7de0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009e60 tx=0x7f3ac8013830 comp rx=0 tx=0).stop 2024-09-19T08:20:05.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.041+0000 7f3ae8adc740 1 -- 172.21.15.22:0/4165119567 shutdown_connections 2024-09-19T08:20:05.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.041+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/4165119567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59b7780 0x5576f59cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.041+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/4165119567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59a9b40 0x5576f59a7de0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.041+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/4165119567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a85d0 0x5576f59c3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.041+0000 7f3ae8adc740 1 -- 172.21.15.22:0/4165119567 >> 172.21.15.22:0/4165119567 conn(0x5576f5971310 msgr2=0x5576f59b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:05.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.041+0000 7f3ae8adc740 1 -- 172.21.15.22:0/4165119567 shutdown_connections 2024-09-19T08:20:05.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.041+0000 7f3ae8adc740 1 -- 172.21.15.22:0/4165119567 wait complete. 2024-09-19T08:20:05.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.041+0000 7f3ae8adc740 1 -- start start 2024-09-19T08:20:05.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.041+0000 7f3ae8adc740 1 Processor -- start 2024-09-19T08:20:05.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae8adc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a85d0 0x5576f59c0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:05.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae8adc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a9b40 0x5576f59bb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:05.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae8adc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59b7780 0x5576f59bb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:05.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae8adc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae01a9d80 con 0x5576f59b7780 2024-09-19T08:20:05.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae8adc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae01ab1f0 con 0x5576f59a9b40 2024-09-19T08:20:05.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae8adc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae01aa450 con 0x5576f59a85d0 2024-09-19T08:20:05.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae8adc740 10 client.0 _ll_register_callbacks cb 0x7f3ad00f3b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:05.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae4ca0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59b7780 0x5576f59bb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:05.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae4ca0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59b7780 0x5576f59bb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34714/0 (socket says 172.21.15.22:34714) 2024-09-19T08:20:05.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae4ca0640 1 -- 172.21.15.22:0/2822195372 learned_addr learned my addr 172.21.15.22:0/2822195372 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:05.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3adffff640 1 --2- 172.21.15.22:0/2822195372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a85d0 0x5576f59c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:05.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ad7fff640 1 --2- 172.21.15.22:0/2822195372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a9b40 0x5576f59bb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:05.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae4ca0640 1 -- 172.21.15.22:0/2822195372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a85d0 msgr2=0x5576f59c0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:05.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae4ca0640 1 --2- 172.21.15.22:0/2822195372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a85d0 0x5576f59c0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae4ca0640 1 -- 172.21.15.22:0/2822195372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a9b40 msgr2=0x5576f59bb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:05.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae4ca0640 1 --2- 172.21.15.22:0/2822195372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a9b40 0x5576f59bb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae4ca0640 1 -- 172.21.15.22:0/2822195372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576f59bc0e0 con 0x5576f59b7780 2024-09-19T08:20:05.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3ae4ca0640 1 --2- 172.21.15.22:0/2822195372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59b7780 0x5576f59bb8b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8011fd0 tx=0x7f3ad8009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d214937ec17007b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:05.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3abf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:05.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3abf7fe640 1 -- 172.21.15.22:0/2822195372 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8046060 con 0x5576f59b7780 2024-09-19T08:20:05.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.042+0000 7f3abf7fe640 1 -- 172.21.15.22:0/2822195372 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3ad801ac30 con 0x5576f59b7780 2024-09-19T08:20:05.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.043+0000 7f3ae8adc740 1 -- 172.21.15.22:0/2822195372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ac8010910 con 0x5576f59b7780 2024-09-19T08:20:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.043+0000 7f3abf7fe640 1 -- 172.21.15.22:0/2822195372 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3ad8035040 con 0x5576f59b7780 2024-09-19T08:20:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.043+0000 7f3ae8adc740 10 client.16392 fetch_fsmap learned FSMap version 29 2024-09-19T08:20:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.043+0000 7f3ae8adc740 1 -- 172.21.15.22:0/2822195372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5576f5a31e10 con 0x5576f59b7780 2024-09-19T08:20:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.043+0000 7f3abf7fe640 1 -- 172.21.15.22:0/2822195372 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac8010910 con 0x5576f59b7780 2024-09-19T08:20:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.043+0000 7f3abf7fe640 1 -- 172.21.15.22:0/2822195372 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3ad8011040 con 0x5576f59b7780 2024-09-19T08:20:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.043+0000 7f3ae8adc740 10 client.16392 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:20:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.043+0000 7f3ae8adc740 10 client.16392 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:20:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.043+0000 7f3ae8adc740 20 client.16392 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.043+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/2822195372 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5576f5a32560 0x5576f5a34980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.043+0000 7f3ae8adc740 4 client.16392 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:20:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.043+0000 7f3ae8adc740 1 -- 172.21.15.22:0/2822195372 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576f5915d70 con 0x5576f5a32560 2024-09-19T08:20:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.043+0000 7f3adffff640 1 --2- 172.21.15.22:0/2822195372 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5576f5a32560 0x5576f5a34980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.044+0000 7f3abf7fe640 10 client.16392 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:20:05.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.044+0000 7f3adffff640 1 --2- 172.21.15.22:0/2822195372 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5576f5a32560 0x5576f5a34980 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b19fd23020c41543 server_cookie=5359acbd7b52435f in_seq=0 out_seq=0 2024-09-19T08:20:05.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:04 smithi064 ceph-mon[34900]: pgmap v154: 97 pgs: 97 active+clean; 613 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-09-19T08:20:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.127+0000 7f3abf7fe640 1 -- 172.21.15.22:0/2822195372 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576f5915d70 con 0x5576f5a32560 2024-09-19T08:20:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.127+0000 7f3abf7fe640 10 client.16392 handle_command_reply: tid=0 2024-09-19T08:20:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.127+0000 7f3ae8adc740 1 client.16392 shutdown 2024-09-19T08:20:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.127+0000 7f3ae8adc740 1 -- 172.21.15.22:0/2822195372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59b7780 msgr2=0x5576f59bb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.127+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/2822195372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59b7780 0x5576f59bb8b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8011fd0 tx=0x7f3ad8009430 comp rx=0 tx=0).stop 2024-09-19T08:20:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.127+0000 7f3ae8adc740 1 -- 172.21.15.22:0/2822195372 shutdown_connections 2024-09-19T08:20:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.127+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/2822195372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576f59b7780 0x5576f59bb8b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.127+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/2822195372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576f59a9b40 0x5576f59bb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.127+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/2822195372 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5576f5a32560 0x5576f5a34980 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.127+0000 7f3ae8adc740 1 --2- 172.21.15.22:0/2822195372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576f59a85d0 0x5576f59c0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.127+0000 7f3ae8adc740 1 -- 172.21.15.22:0/2822195372 >> 172.21.15.22:0/2822195372 conn(0x5576f5971310 msgr2=0x5576f59b5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:05.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.127+0000 7f3abf7fe640 0 client.16392 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:20:05.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae8adc740 1 -- 172.21.15.22:0/2822195372 shutdown_connections 2024-09-19T08:20:05.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae8adc740 1 -- 172.21.15.22:0/2822195372 wait complete. 2024-09-19T08:20:05.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae8adc740 20 client.16392 trim_cache size 0 max 16384 2024-09-19T08:20:05.133 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:20:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae6703640 1 -- 172.21.15.22:0/4070159180 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3acc0a33b0 msgr2=0x7f3acc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae6703640 1 --2- 172.21.15.22:0/4070159180 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3acc0a33b0 0x7f3acc0a5870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8011010 tx=0x7f3ad8011e20 comp rx=0 tx=0).stop 2024-09-19T08:20:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae6703640 1 -- 172.21.15.22:0/4070159180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ae0102d20 msgr2=0x7f3ae01984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae6703640 1 --2- 172.21.15.22:0/4070159180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ae0102d20 0x7f3ae01984a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0006010 tx=0x7f3ad00042f0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae6703640 1 -- 172.21.15.22:0/4070159180 shutdown_connections 2024-09-19T08:20:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae6703640 1 --2- 172.21.15.22:0/4070159180 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3acc0a33b0 0x7f3acc0a5870 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae6703640 1 --2- 172.21.15.22:0/4070159180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ae010c150 0x7f3ae019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae6703640 1 --2- 172.21.15.22:0/4070159180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae0103640 0x7f3ae01989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae6703640 1 --2- 172.21.15.22:0/4070159180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ae0102d20 0x7f3ae01984a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae6703640 1 -- 172.21.15.22:0/4070159180 >> 172.21.15.22:0/4070159180 conn(0x7f3ae00fc780 msgr2=0x7f3ae00fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae6703640 1 -- 172.21.15.22:0/4070159180 shutdown_connections 2024-09-19T08:20:05.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:05.128+0000 7f3ae6703640 1 -- 172.21.15.22:0/4070159180 wait complete. 2024-09-19T08:20:05.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:04 smithi157 ceph-mon[35143]: pgmap v154: 97 pgs: 97 active+clean; 613 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-09-19T08:20:05.952 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:20:06.155 INFO:tasks.workunit.client.0.smithi022.stdout: 109 169620 63 104945 80 91423 181 2024-09-19T08:20:06.208 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:06.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.702+0000 7ff848486640 1 -- 172.21.15.22:0/3024218723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff840106a80 msgr2=0x7ff840106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:06.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.702+0000 7ff848486640 1 --2- 172.21.15.22:0/3024218723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff840106a80 0x7ff840106e60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009930 tx=0x7ff83c02a420 comp rx=0 tx=0).stop 2024-09-19T08:20:06.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.702+0000 7ff848486640 1 -- 172.21.15.22:0/3024218723 shutdown_connections 2024-09-19T08:20:06.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.702+0000 7ff848486640 1 --2- 172.21.15.22:0/3024218723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff840101420 0x7ff84010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.702+0000 7ff848486640 1 --2- 172.21.15.22:0/3024218723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840100a80 0x7ff840100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.702+0000 7ff848486640 1 --2- 172.21.15.22:0/3024218723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff840106a80 0x7ff840106e60 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.702+0000 7ff848486640 1 -- 172.21.15.22:0/3024218723 >> 172.21.15.22:0/3024218723 conn(0x7ff8400fc7a0 msgr2=0x7ff8400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:06.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.702+0000 7ff848486640 1 -- 172.21.15.22:0/3024218723 shutdown_connections 2024-09-19T08:20:06.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.702+0000 7ff848486640 1 -- 172.21.15.22:0/3024218723 wait complete. 2024-09-19T08:20:06.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.703+0000 7ff848486640 1 Processor -- start 2024-09-19T08:20:06.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.703+0000 7ff848486640 1 -- start start 2024-09-19T08:20:06.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.703+0000 7ff848486640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff840100a80 0x7ff840198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.703+0000 7ff848486640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff840101420 0x7ff840198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.703+0000 7ff848486640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840106a80 0x7ff84019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.703+0000 7ff848486640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff840110e80 con 0x7ff840100a80 2024-09-19T08:20:06.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.703+0000 7ff848486640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff840110d00 con 0x7ff840106a80 2024-09-19T08:20:06.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.703+0000 7ff848486640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff840111000 con 0x7ff840101420 2024-09-19T08:20:06.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.703+0000 7ff846222640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff840100a80 0x7ff840198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.703+0000 7ff846222640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff840100a80 0x7ff840198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34726/0 (socket says 172.21.15.22:34726) 2024-09-19T08:20:06.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.703+0000 7ff846222640 1 -- 172.21.15.22:0/2152191871 learned_addr learned my addr 172.21.15.22:0/2152191871 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:06.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.703+0000 7ff845a21640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff840101420 0x7ff840198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.704+0000 7ff846a23640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840106a80 0x7ff84019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.704+0000 7ff846222640 1 -- 172.21.15.22:0/2152191871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff840101420 msgr2=0x7ff840198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:06.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.704+0000 7ff846222640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff840101420 0x7ff840198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.704+0000 7ff846222640 1 -- 172.21.15.22:0/2152191871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840106a80 msgr2=0x7ff84019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:06.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.704+0000 7ff846222640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840106a80 0x7ff84019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.704+0000 7ff846222640 1 -- 172.21.15.22:0/2152191871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84019d3e0 con 0x7ff840100a80 2024-09-19T08:20:06.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.704+0000 7ff845a21640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff840101420 0x7ff840198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:06.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.704+0000 7ff846222640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff840100a80 0x7ff840198430 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009230 tx=0x7ff83c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:06.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.704+0000 7ff8377fe640 1 -- 172.21.15.22:0/2152191871 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c03f030 con 0x7ff840100a80 2024-09-19T08:20:06.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.704+0000 7ff8377fe640 1 -- 172.21.15.22:0/2152191871 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83c003ad0 con 0x7ff840100a80 2024-09-19T08:20:06.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.704+0000 7ff848486640 1 -- 172.21.15.22:0/2152191871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84019d670 con 0x7ff840100a80 2024-09-19T08:20:06.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.704+0000 7ff8377fe640 1 -- 172.21.15.22:0/2152191871 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c035070 con 0x7ff840100a80 2024-09-19T08:20:06.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.704+0000 7ff848486640 1 -- 172.21.15.22:0/2152191871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8401a4fb0 con 0x7ff840100a80 2024-09-19T08:20:06.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.707+0000 7ff8377fe640 1 -- 172.21.15.22:0/2152191871 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff83c042030 con 0x7ff840100a80 2024-09-19T08:20:06.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.707+0000 7ff84a85f740 1 Processor -- start 2024-09-19T08:20:06.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.707+0000 7ff84a85f740 1 -- start start 2024-09-19T08:20:06.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.707+0000 7ff84a85f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 0x555734906090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.707+0000 7ff84a85f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5557349065d0 0x555734906a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.707+0000 7ff84a85f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555734915780 0x555734917b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.707+0000 7ff84a85f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55573486c2f0 con 0x555734915780 2024-09-19T08:20:06.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.707+0000 7ff84a85f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555734813cf0 con 0x555734907b40 2024-09-19T08:20:06.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.707+0000 7ff84a85f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5557348682b0 con 0x5557349065d0 2024-09-19T08:20:06.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.707+0000 7ff846222640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5557349065d0 0x555734906a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.707+0000 7ff845a21640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555734915780 0x555734917b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.707+0000 7ff846222640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5557349065d0 0x555734906a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:54756/0 (socket says 172.21.15.22:54756) 2024-09-19T08:20:06.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.707+0000 7ff846222640 1 -- 172.21.15.22:0/2447054314 learned_addr learned my addr 172.21.15.22:0/2447054314 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:06.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.707+0000 7ff846a23640 1 --2- 172.21.15.22:0/2447054314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 0x555734906090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.708+0000 7ff8377fe640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff82c0a33b0 0x7ff82c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.708+0000 7ff845a21640 1 -- 172.21.15.22:0/2447054314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5557349065d0 msgr2=0x555734906a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:06.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.708+0000 7ff846a23640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff82c0a33b0 0x7ff82c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.708+0000 7ff845a21640 1 --2- 172.21.15.22:0/2447054314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5557349065d0 0x555734906a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.708+0000 7ff845a21640 1 -- 172.21.15.22:0/2447054314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 msgr2=0x555734906090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:06.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.708+0000 7ff845a21640 1 --2- 172.21.15.22:0/2447054314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 0x555734906090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.708+0000 7ff845a21640 1 -- 172.21.15.22:0/2447054314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557347f83a0 con 0x555734915780 2024-09-19T08:20:06.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.708+0000 7ff846222640 1 --2- 172.21.15.22:0/2447054314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5557349065d0 0x555734906a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:06.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.708+0000 7ff8377fe640 1 -- 172.21.15.22:0/2152191871 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff83c0f02f0 con 0x7ff840100a80 2024-09-19T08:20:06.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.708+0000 7ff845a21640 1 --2- 172.21.15.22:0/2447054314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555734915780 0x555734917b70 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff83000af90 tx=0x7ff83000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2ee4e0a4adaa9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:06.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.708+0000 7ff846a23640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff82c0a33b0 0x7ff82c0a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff838009fd0 tx=0x7ff838004ab0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:06.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.708+0000 7ff834ff9640 1 -- 172.21.15.22:0/2447054314 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830005220 con 0x555734915780 2024-09-19T08:20:06.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.708+0000 7ff834ff9640 1 -- 172.21.15.22:0/2447054314 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8300047a0 con 0x555734915780 2024-09-19T08:20:06.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.708+0000 7ff834ff9640 1 -- 172.21.15.22:0/2447054314 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830010b30 con 0x555734915780 2024-09-19T08:20:06.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.709+0000 7ff84a85f740 1 -- 172.21.15.22:0/2447054314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555734915780 msgr2=0x555734917b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:06.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.709+0000 7ff84a85f740 1 --2- 172.21.15.22:0/2447054314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555734915780 0x555734917b70 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff83000af90 tx=0x7ff83000f850 comp rx=0 tx=0).stop 2024-09-19T08:20:06.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.709+0000 7ff84a85f740 1 -- 172.21.15.22:0/2447054314 shutdown_connections 2024-09-19T08:20:06.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.709+0000 7ff84a85f740 1 --2- 172.21.15.22:0/2447054314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555734915780 0x555734917b70 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.709+0000 7ff84a85f740 1 --2- 172.21.15.22:0/2447054314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5557349065d0 0x555734906a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.709+0000 7ff84a85f740 1 --2- 172.21.15.22:0/2447054314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 0x555734906090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.709+0000 7ff84a85f740 1 -- 172.21.15.22:0/2447054314 >> 172.21.15.22:0/2447054314 conn(0x5557348cf310 msgr2=0x555734902c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:06.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.709+0000 7ff84a85f740 1 -- 172.21.15.22:0/2447054314 shutdown_connections 2024-09-19T08:20:06.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.709+0000 7ff84a85f740 1 -- 172.21.15.22:0/2447054314 wait complete. 2024-09-19T08:20:06.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff84a85f740 1 -- start start 2024-09-19T08:20:06.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff84a85f740 1 Processor -- start 2024-09-19T08:20:06.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff84a85f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557349065d0 0x5557349150e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff84a85f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 0x555734912390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff84a85f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734915780 0x5557349128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff84a85f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555734918480 con 0x5557349065d0 2024-09-19T08:20:06.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff84a85f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555734827060 con 0x555734907b40 2024-09-19T08:20:06.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff84a85f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555734918600 con 0x555734915780 2024-09-19T08:20:06.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff84a85f740 10 client.0 _ll_register_callbacks cb 0x55573457a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:06.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff846222640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557349065d0 0x5557349150e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff846a23640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734915780 0x5557349128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff846222640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557349065d0 0x5557349150e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34750/0 (socket says 172.21.15.22:34750) 2024-09-19T08:20:06.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff846222640 1 -- 172.21.15.22:0/197353841 learned_addr learned my addr 172.21.15.22:0/197353841 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:06.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff845a21640 1 --2- 172.21.15.22:0/197353841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 0x555734912390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff846222640 1 -- 172.21.15.22:0/197353841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734915780 msgr2=0x5557349128f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:06.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff846222640 1 --2- 172.21.15.22:0/197353841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734915780 0x5557349128f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff846222640 1 -- 172.21.15.22:0/197353841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 msgr2=0x555734912390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:06.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff846222640 1 --2- 172.21.15.22:0/197353841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 0x555734912390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff846222640 1 -- 172.21.15.22:0/197353841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557345dd3c0 con 0x5557349065d0 2024-09-19T08:20:06.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.710+0000 7ff846a23640 1 --2- 172.21.15.22:0/197353841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734915780 0x5557349128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:06.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff846222640 1 --2- 172.21.15.22:0/197353841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557349065d0 0x5557349150e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0edb00 tx=0x7ff83c0f3950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2250c058bdc58994 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:06.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:06.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff829ffb640 1 -- 172.21.15.22:0/197353841 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c049070 con 0x5557349065d0 2024-09-19T08:20:06.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff829ffb640 1 -- 172.21.15.22:0/197353841 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff83c0b8c50 con 0x5557349065d0 2024-09-19T08:20:06.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff84a85f740 1 -- 172.21.15.22:0/197353841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8300047a0 con 0x5557349065d0 2024-09-19T08:20:06.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff829ffb640 1 -- 172.21.15.22:0/197353841 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff83c0bc120 con 0x5557349065d0 2024-09-19T08:20:06.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff84a85f740 10 client.16404 fetch_fsmap learned FSMap version 29 2024-09-19T08:20:06.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff84a85f740 1 -- 172.21.15.22:0/197353841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555734918480 con 0x5557349065d0 2024-09-19T08:20:06.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff829ffb640 1 -- 172.21.15.22:0/197353841 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8300047a0 con 0x5557349065d0 2024-09-19T08:20:06.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff829ffb640 1 -- 172.21.15.22:0/197353841 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff83c03bc80 con 0x5557349065d0 2024-09-19T08:20:06.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff84a85f740 10 client.16404 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:20:06.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff84a85f740 10 client.16404 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:20:06.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff84a85f740 20 client.16404 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:06.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff84a85f740 1 --2- 172.21.15.22:0/197353841 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x555734925260 0x5557349276f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff84a85f740 4 client.16404 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:20:06.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.711+0000 7ff84a85f740 1 -- 172.21.15.22:0/197353841 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555734879860 con 0x555734925260 2024-09-19T08:20:06.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.712+0000 7ff845a21640 1 --2- 172.21.15.22:0/197353841 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x555734925260 0x5557349276f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.713+0000 7ff829ffb640 10 client.16404 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:20:06.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.713+0000 7ff845a21640 1 --2- 172.21.15.22:0/197353841 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x555734925260 0x5557349276f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6805a1e5737aefc9 server_cookie=ccbac6ef41cd3093 in_seq=0 out_seq=0 2024-09-19T08:20:06.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff829ffb640 1 -- 172.21.15.22:0/197353841 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555734879860 con 0x555734925260 2024-09-19T08:20:06.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff829ffb640 10 client.16404 handle_command_reply: tid=0 2024-09-19T08:20:06.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff84a85f740 1 client.16404 shutdown 2024-09-19T08:20:06.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff84a85f740 1 -- 172.21.15.22:0/197353841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557349065d0 msgr2=0x5557349150e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:06.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff84a85f740 1 --2- 172.21.15.22:0/197353841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557349065d0 0x5557349150e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0edb00 tx=0x7ff83c0f3950 comp rx=0 tx=0).stop 2024-09-19T08:20:06.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff84a85f740 1 -- 172.21.15.22:0/197353841 shutdown_connections 2024-09-19T08:20:06.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff84a85f740 1 --2- 172.21.15.22:0/197353841 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x555734925260 0x5557349276f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff84a85f740 1 --2- 172.21.15.22:0/197353841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734915780 0x5557349128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff84a85f740 1 --2- 172.21.15.22:0/197353841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 0x555734912390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff84a85f740 1 --2- 172.21.15.22:0/197353841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557349065d0 0x5557349150e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff84a85f740 1 -- 172.21.15.22:0/197353841 >> 172.21.15.22:0/197353841 conn(0x5557348cf310 msgr2=0x555734908960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:06.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff829ffb640 0 client.16404 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:20:06.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff84a85f740 1 -- 172.21.15.22:0/197353841 shutdown_connections 2024-09-19T08:20:06.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff84a85f740 1 -- 172.21.15.22:0/197353841 wait complete. 2024-09-19T08:20:06.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.714+0000 7ff84a85f740 20 client.16404 trim_cache size 0 max 16384 2024-09-19T08:20:06.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.727+0000 7ff84a85f740 1 Processor -- start 2024-09-19T08:20:06.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.727+0000 7ff84a85f740 1 -- start start 2024-09-19T08:20:06.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.727+0000 7ff84a85f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557349065d0 0x555734902e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.727+0000 7ff84a85f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 0x555734921720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.727+0000 7ff84a85f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734915780 0x55573492cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.727+0000 7ff84a85f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401adf00 con 0x5557349065d0 2024-09-19T08:20:06.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.727+0000 7ff84a85f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401ab770 con 0x555734907b40 2024-09-19T08:20:06.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.727+0000 7ff84a85f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401a95c0 con 0x555734915780 2024-09-19T08:20:06.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.727+0000 7ff846222640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557349065d0 0x555734902e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.727+0000 7ff846a23640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734915780 0x55573492cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.727+0000 7ff846a23640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734915780 0x55573492cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:54774/0 (socket says 172.21.15.22:54774) 2024-09-19T08:20:06.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.727+0000 7ff846a23640 1 -- 172.21.15.22:0/2144078408 learned_addr learned my addr 172.21.15.22:0/2144078408 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:06.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.727+0000 7ff846222640 1 --2- 172.21.15.22:0/2144078408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557349065d0 0x555734902e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34766/0 (socket says 172.21.15.22:34766) 2024-09-19T08:20:06.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.728+0000 7ff845a21640 1 --2- 172.21.15.22:0/2144078408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 0x555734921720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.728+0000 7ff846222640 1 -- 172.21.15.22:0/2144078408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734915780 msgr2=0x55573492cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:06.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.728+0000 7ff846222640 1 --2- 172.21.15.22:0/2144078408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734915780 0x55573492cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.728+0000 7ff846222640 1 -- 172.21.15.22:0/2144078408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 msgr2=0x555734921720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:06.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.728+0000 7ff846222640 1 --2- 172.21.15.22:0/2144078408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 0x555734921720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.728+0000 7ff846222640 1 -- 172.21.15.22:0/2144078408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557349024f0 con 0x5557349065d0 2024-09-19T08:20:06.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.728+0000 7ff846a23640 1 --2- 172.21.15.22:0/2144078408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734915780 0x55573492cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:06.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.728+0000 7ff846222640 1 --2- 172.21.15.22:0/2144078408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557349065d0 0x555734902e10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0b8fb0 tx=0x7ff83c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667efd6c9c65783e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:06.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.728+0000 7ff82bfff640 1 -- 172.21.15.22:0/2144078408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c049040 con 0x5557349065d0 2024-09-19T08:20:06.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.728+0000 7ff82bfff640 1 -- 172.21.15.22:0/2144078408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83c0bcce0 con 0x5557349065d0 2024-09-19T08:20:06.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.728+0000 7ff82bfff640 1 -- 172.21.15.22:0/2144078408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c0b8880 con 0x5557349065d0 2024-09-19T08:20:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.729+0000 7ff84a85f740 1 -- 172.21.15.22:0/2144078408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557349065d0 msgr2=0x555734902e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.729+0000 7ff84a85f740 1 --2- 172.21.15.22:0/2144078408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557349065d0 0x555734902e10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0b8fb0 tx=0x7ff83c02a420 comp rx=0 tx=0).stop 2024-09-19T08:20:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.729+0000 7ff84a85f740 1 -- 172.21.15.22:0/2144078408 shutdown_connections 2024-09-19T08:20:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.729+0000 7ff84a85f740 1 --2- 172.21.15.22:0/2144078408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734915780 0x55573492cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.729+0000 7ff84a85f740 1 --2- 172.21.15.22:0/2144078408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555734907b40 0x555734921720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.729+0000 7ff84a85f740 1 --2- 172.21.15.22:0/2144078408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557349065d0 0x555734902e10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.729+0000 7ff84a85f740 1 -- 172.21.15.22:0/2144078408 >> 172.21.15.22:0/2144078408 conn(0x5557348cf310 msgr2=0x555734902700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.729+0000 7ff84a85f740 1 -- 172.21.15.22:0/2144078408 shutdown_connections 2024-09-19T08:20:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.729+0000 7ff84a85f740 1 -- 172.21.15.22:0/2144078408 wait complete. 2024-09-19T08:20:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.729+0000 7ff84a85f740 1 -- start start 2024-09-19T08:20:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff84a85f740 1 Processor -- start 2024-09-19T08:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff84a85f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557349065d0 0x555734918850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff84a85f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734907b40 0x555734918d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff84a85f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555734915780 0x5557349192f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff84a85f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401a9c30 con 0x555734915780 2024-09-19T08:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff84a85f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401ab0a0 con 0x5557349065d0 2024-09-19T08:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff84a85f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401aa300 con 0x555734907b40 2024-09-19T08:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff84a85f740 10 client.0 _ll_register_callbacks cb 0x7ff83c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff846a23640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555734915780 0x5557349192f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff845a21640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734907b40 0x555734918d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff846222640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557349065d0 0x555734918850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff845a21640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734907b40 0x555734918d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:54780/0 (socket says 172.21.15.22:54780) 2024-09-19T08:20:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff846a23640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555734915780 0x5557349192f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34768/0 (socket says 172.21.15.22:34768) 2024-09-19T08:20:06.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff845a21640 1 -- 172.21.15.22:0/4292855586 learned_addr learned my addr 172.21.15.22:0/4292855586 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:06.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff845a21640 1 -- 172.21.15.22:0/4292855586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557349065d0 msgr2=0x555734918850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:06.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff845a21640 1 --2- 172.21.15.22:0/4292855586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557349065d0 0x555734918850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff845a21640 1 -- 172.21.15.22:0/4292855586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555734915780 msgr2=0x5557349192f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:06.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff845a21640 1 --2- 172.21.15.22:0/4292855586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555734915780 0x5557349192f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff845a21640 1 -- 172.21.15.22:0/4292855586 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55573498e220 con 0x555734907b40 2024-09-19T08:20:06.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.730+0000 7ff846a23640 1 --2- 172.21.15.22:0/4292855586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555734915780 0x5557349192f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:06.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff845a21640 1 --2- 172.21.15.22:0/4292855586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734907b40 0x555734918d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff83000af90 tx=0x7ff830005b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=554e66b247386533 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:06.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:20:06.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff829ffb640 1 -- 172.21.15.22:0/4292855586 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830004720 con 0x555734907b40 2024-09-19T08:20:06.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff829ffb640 1 -- 172.21.15.22:0/4292855586 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff830010700 con 0x555734907b40 2024-09-19T08:20:06.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff84a85f740 1 -- 172.21.15.22:0/4292855586 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff83c0bcce0 con 0x555734907b40 2024-09-19T08:20:06.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff829ffb640 1 -- 172.21.15.22:0/4292855586 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff830010900 con 0x555734907b40 2024-09-19T08:20:06.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff84a85f740 10 client.25939 fetch_fsmap learned FSMap version 29 2024-09-19T08:20:06.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff84a85f740 1 -- 172.21.15.22:0/4292855586 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55573498f520 con 0x555734907b40 2024-09-19T08:20:06.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff829ffb640 1 -- 172.21.15.22:0/4292855586 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83c0bcce0 con 0x555734907b40 2024-09-19T08:20:06.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff829ffb640 1 -- 172.21.15.22:0/4292855586 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff830034b60 con 0x555734907b40 2024-09-19T08:20:06.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff84a85f740 10 client.25939 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:20:06.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff84a85f740 10 client.25939 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:20:06.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff84a85f740 20 client.25939 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:06.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff84a85f740 1 --2- 172.21.15.22:0/4292855586 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55573498f960 0x555734991d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:06.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff84a85f740 4 client.25939 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:20:06.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.731+0000 7ff84a85f740 1 -- 172.21.15.22:0/4292855586 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555734873d70 con 0x55573498f960 2024-09-19T08:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.732+0000 7ff846222640 1 --2- 172.21.15.22:0/4292855586 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55573498f960 0x555734991d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.732+0000 7ff846222640 1 --2- 172.21.15.22:0/4292855586 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55573498f960 0x555734991d80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2211c94bf3267891 server_cookie=ccd83296d8654742 in_seq=0 out_seq=0 2024-09-19T08:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.732+0000 7ff829ffb640 10 client.25939 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.732+0000 7ff829ffb640 1 -- 172.21.15.22:0/4292855586 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555734873d70 con 0x55573498f960 2024-09-19T08:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.732+0000 7ff829ffb640 10 client.25939 handle_command_reply: tid=0 2024-09-19T08:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.732+0000 7ff84a85f740 1 client.25939 shutdown 2024-09-19T08:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.732+0000 7ff84a85f740 1 -- 172.21.15.22:0/4292855586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734907b40 msgr2=0x555734918d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.732+0000 7ff84a85f740 1 --2- 172.21.15.22:0/4292855586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734907b40 0x555734918d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff83000af90 tx=0x7ff830005b60 comp rx=0 tx=0).stop 2024-09-19T08:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff84a85f740 1 -- 172.21.15.22:0/4292855586 shutdown_connections 2024-09-19T08:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff84a85f740 1 --2- 172.21.15.22:0/4292855586 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55573498f960 0x555734991d80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff84a85f740 1 --2- 172.21.15.22:0/4292855586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555734915780 0x5557349192f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff84a85f740 1 --2- 172.21.15.22:0/4292855586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555734907b40 0x555734918d90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff829ffb640 0 client.25939 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:20:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff84a85f740 1 --2- 172.21.15.22:0/4292855586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557349065d0 0x555734918850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff84a85f740 1 -- 172.21.15.22:0/4292855586 >> 172.21.15.22:0/4292855586 conn(0x5557348cf310 msgr2=0x555734908110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff84a85f740 1 -- 172.21.15.22:0/4292855586 shutdown_connections 2024-09-19T08:20:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff84a85f740 1 -- 172.21.15.22:0/4292855586 wait complete. 2024-09-19T08:20:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff84a85f740 20 client.25939 trim_cache size 0 max 16384 2024-09-19T08:20:06.739 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:20:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff848486640 1 -- 172.21.15.22:0/2152191871 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff82c0a33b0 msgr2=0x7ff82c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff848486640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff82c0a33b0 0x7ff82c0a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff838009fd0 tx=0x7ff838004ab0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff848486640 1 -- 172.21.15.22:0/2152191871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff840100a80 msgr2=0x7ff840198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff848486640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff840100a80 0x7ff840198430 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009230 tx=0x7ff83c004270 comp rx=0 tx=0).stop 2024-09-19T08:20:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff848486640 1 -- 172.21.15.22:0/2152191871 shutdown_connections 2024-09-19T08:20:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff848486640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff82c0a33b0 0x7ff82c0a5870 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff848486640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840106a80 0x7ff84019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff848486640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff840101420 0x7ff840198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff848486640 1 --2- 172.21.15.22:0/2152191871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff840100a80 0x7ff840198430 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff848486640 1 -- 172.21.15.22:0/2152191871 >> 172.21.15.22:0/2152191871 conn(0x7ff8400fc7a0 msgr2=0x7ff84006aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff848486640 1 -- 172.21.15.22:0/2152191871 shutdown_connections 2024-09-19T08:20:06.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:06.733+0000 7ff848486640 1 -- 172.21.15.22:0/2152191871 wait complete. 2024-09-19T08:20:07.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:06 smithi022 ceph-mon[30963]: pgmap v155: 97 pgs: 97 active+clean; 613 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 241 KiB/s wr, 92 op/s 2024-09-19T08:20:07.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:06 smithi064 ceph-mon[34900]: pgmap v155: 97 pgs: 97 active+clean; 613 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 241 KiB/s wr, 92 op/s 2024-09-19T08:20:07.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:06 smithi157 ceph-mon[35143]: pgmap v155: 97 pgs: 97 active+clean; 613 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 241 KiB/s wr, 92 op/s 2024-09-19T08:20:07.621 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:20:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:08.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.366+0000 7f234d13d640 1 -- 172.21.15.22:0/2181594954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2348102570 msgr2=0x7f23481029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:08.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.366+0000 7f234d13d640 1 --2- 172.21.15.22:0/2181594954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2348102570 0x7f23481029b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2330009900 tx=0x7f233002a480 comp rx=0 tx=0).stop 2024-09-19T08:20:08.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.366+0000 7f234d13d640 1 -- 172.21.15.22:0/2181594954 shutdown_connections 2024-09-19T08:20:08.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.366+0000 7f234d13d640 1 --2- 172.21.15.22:0/2181594954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2348102ef0 0x7f234810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.366+0000 7f234d13d640 1 --2- 172.21.15.22:0/2181594954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2348102570 0x7f23481029b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.366+0000 7f234d13d640 1 --2- 172.21.15.22:0/2181594954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2348100e20 0x7f2348101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.366+0000 7f234d13d640 1 -- 172.21.15.22:0/2181594954 >> 172.21.15.22:0/2181594954 conn(0x7f23480fc7d0 msgr2=0x7f23480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:08.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.366+0000 7f234d13d640 1 -- 172.21.15.22:0/2181594954 shutdown_connections 2024-09-19T08:20:08.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.366+0000 7f234d13d640 1 -- 172.21.15.22:0/2181594954 wait complete. 2024-09-19T08:20:08.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.367+0000 7f234d13d640 1 Processor -- start 2024-09-19T08:20:08.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.367+0000 7f234d13d640 1 -- start start 2024-09-19T08:20:08.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.367+0000 7f234d13d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2348100e20 0x7f23480fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.367+0000 7f234d13d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2348102570 0x7f23480fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f234d13d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2348102ef0 0x7f23480fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f234d13d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2348110f80 con 0x7f2348102ef0 2024-09-19T08:20:08.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f234d13d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2348110e00 con 0x7f2348100e20 2024-09-19T08:20:08.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f234d13d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2348111100 con 0x7f2348102570 2024-09-19T08:20:08.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f234759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2348102ef0 0x7f23480fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f233ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2348102570 0x7f23480fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f234759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2348102ef0 0x7f23480fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56958/0 (socket says 172.21.15.22:56958) 2024-09-19T08:20:08.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f2346d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2348100e20 0x7f23480fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f234759e640 1 -- 172.21.15.22:0/1299266369 learned_addr learned my addr 172.21.15.22:0/1299266369 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:08.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f234759e640 1 -- 172.21.15.22:0/1299266369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2348102570 msgr2=0x7f23480fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:08.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f234759e640 1 --2- 172.21.15.22:0/1299266369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2348102570 0x7f23480fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f234759e640 1 -- 172.21.15.22:0/1299266369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2348100e20 msgr2=0x7f23480fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:08.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f234759e640 1 --2- 172.21.15.22:0/1299266369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2348100e20 0x7f23480fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f234759e640 1 -- 172.21.15.22:0/1299266369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23481a6990 con 0x7f2348102ef0 2024-09-19T08:20:08.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f234759e640 1 --2- 172.21.15.22:0/1299266369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2348102ef0 0x7f23480fd700 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f234000eb10 tx=0x7f234000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:08.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f2344d99640 1 -- 172.21.15.22:0/1299266369 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340004b20 con 0x7f2348102ef0 2024-09-19T08:20:08.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f234d13d640 1 -- 172.21.15.22:0/1299266369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23481a6c80 con 0x7f2348102ef0 2024-09-19T08:20:08.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f2344d99640 1 -- 172.21.15.22:0/1299266369 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2340004cc0 con 0x7f2348102ef0 2024-09-19T08:20:08.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.368+0000 7f2344d99640 1 -- 172.21.15.22:0/1299266369 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340010760 con 0x7f2348102ef0 2024-09-19T08:20:08.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.369+0000 7f234d13d640 1 -- 172.21.15.22:0/1299266369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23481a71e0 con 0x7f2348102ef0 2024-09-19T08:20:08.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.371+0000 7f234f516740 1 Processor -- start 2024-09-19T08:20:08.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.371+0000 7f2344d99640 1 -- 172.21.15.22:0/1299266369 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2340005020 con 0x7f2348102ef0 2024-09-19T08:20:08.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.371+0000 7f234f516740 1 -- start start 2024-09-19T08:20:08.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f234f516740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x55807903a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f234f516740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903a5d0 0x55807903aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f234f516740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 0x558079047760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f234f516740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558078fa02f0 con 0x55807903bb40 2024-09-19T08:20:08.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f234f516740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558078f47cf0 con 0x558079045370 2024-09-19T08:20:08.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f234f516740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558078f9c2b0 con 0x55807903a5d0 2024-09-19T08:20:08.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f233ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x55807903a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f233ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x55807903a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56968/0 (socket says 172.21.15.22:56968) 2024-09-19T08:20:08.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f233ffff640 1 -- 172.21.15.22:0/589036581 learned_addr learned my addr 172.21.15.22:0/589036581 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:08.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f2346d9d640 1 --2- 172.21.15.22:0/589036581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903a5d0 0x55807903aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f233ffff640 1 --2- 172.21.15.22:0/589036581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 0x558079047760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f2344d99640 1 --2- 172.21.15.22:0/1299266369 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f23340a33b0 0x7f23340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f2346d9d640 1 -- 172.21.15.22:0/589036581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 msgr2=0x558079047760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:08.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f2346d9d640 1 --2- 172.21.15.22:0/589036581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 0x558079047760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f2346d9d640 1 -- 172.21.15.22:0/589036581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 msgr2=0x55807903a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:08.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f2346d9d640 1 --2- 172.21.15.22:0/589036581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x55807903a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f2346d9d640 1 -- 172.21.15.22:0/589036581 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558078f2c3a0 con 0x55807903a5d0 2024-09-19T08:20:08.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f233ffff640 1 --2- 172.21.15.22:0/589036581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 0x558079047760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:20:08.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f234759e640 1 --2- 172.21.15.22:0/1299266369 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f23340a33b0 0x7f23340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.372+0000 7f2344d99640 1 -- 172.21.15.22:0/1299266369 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f23400d0340 con 0x7f2348102ef0 2024-09-19T08:20:08.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.373+0000 7f2346d9d640 1 --2- 172.21.15.22:0/589036581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903a5d0 0x55807903aa10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f233800bbf0 tx=0x7f23380107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a187ba1da57b97e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:08.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.373+0000 7f233d7fa640 1 -- 172.21.15.22:0/589036581 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338004030 con 0x55807903a5d0 2024-09-19T08:20:08.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.373+0000 7f233d7fa640 1 -- 172.21.15.22:0/589036581 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23380041d0 con 0x55807903a5d0 2024-09-19T08:20:08.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.373+0000 7f234759e640 1 --2- 172.21.15.22:0/1299266369 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f23340a33b0 0x7f23340a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f234009c5c0 tx=0x7f2340091ac0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:08.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.373+0000 7f233d7fa640 1 -- 172.21.15.22:0/589036581 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23380116c0 con 0x55807903a5d0 2024-09-19T08:20:08.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.373+0000 7f234f516740 1 -- 172.21.15.22:0/589036581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903a5d0 msgr2=0x55807903aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:08.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.373+0000 7f234f516740 1 --2- 172.21.15.22:0/589036581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903a5d0 0x55807903aa10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f233800bbf0 tx=0x7f23380107a0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 -- 172.21.15.22:0/589036581 shutdown_connections 2024-09-19T08:20:08.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 --2- 172.21.15.22:0/589036581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 0x558079047760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 --2- 172.21.15.22:0/589036581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903a5d0 0x55807903aa10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 --2- 172.21.15.22:0/589036581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x55807903a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 -- 172.21.15.22:0/589036581 >> 172.21.15.22:0/589036581 conn(0x558079003310 msgr2=0x558079036c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:08.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 -- 172.21.15.22:0/589036581 shutdown_connections 2024-09-19T08:20:08.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 -- 172.21.15.22:0/589036581 wait complete. 2024-09-19T08:20:08.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 -- start start 2024-09-19T08:20:08.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 Processor -- start 2024-09-19T08:20:08.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903a5d0 0x558079044d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x558079042010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 0x558079042600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558079048070 con 0x55807903bb40 2024-09-19T08:20:08.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558078f5b060 con 0x558079045370 2024-09-19T08:20:08.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5580790481f0 con 0x55807903a5d0 2024-09-19T08:20:08.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f234f516740 10 client.0 _ll_register_callbacks cb 0x558078cae8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:08.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.374+0000 7f2346d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x558079042010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f233ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903a5d0 0x558079044d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f2346d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x558079042010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56970/0 (socket says 172.21.15.22:56970) 2024-09-19T08:20:08.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f2346d9d640 1 -- 172.21.15.22:0/3898886413 learned_addr learned my addr 172.21.15.22:0/3898886413 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:08.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f233ffff640 1 --2- 172.21.15.22:0/3898886413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 0x558079042600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f2346d9d640 1 -- 172.21.15.22:0/3898886413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903a5d0 msgr2=0x558079044d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:08.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f2346d9d640 1 --2- 172.21.15.22:0/3898886413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903a5d0 0x558079044d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f2346d9d640 1 -- 172.21.15.22:0/3898886413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 msgr2=0x558079042600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:08.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f2346d9d640 1 --2- 172.21.15.22:0/3898886413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 0x558079042600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f2346d9d640 1 -- 172.21.15.22:0/3898886413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558078d113c0 con 0x55807903bb40 2024-09-19T08:20:08.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f233ffff640 1 --2- 172.21.15.22:0/3898886413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903a5d0 0x558079044d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:08.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f2346d9d640 1 --2- 172.21.15.22:0/3898886413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x558079042010 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2338011010 tx=0x7f233800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b148504b43e1cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:08.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f232e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:08.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f232e7fc640 1 -- 172.21.15.22:0/3898886413 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23380035f0 con 0x55807903bb40 2024-09-19T08:20:08.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f232e7fc640 1 -- 172.21.15.22:0/3898886413 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2338003790 con 0x55807903bb40 2024-09-19T08:20:08.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f234f516740 1 -- 172.21.15.22:0/3898886413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23380041d0 con 0x55807903bb40 2024-09-19T08:20:08.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f232e7fc640 1 -- 172.21.15.22:0/3898886413 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2338003990 con 0x55807903bb40 2024-09-19T08:20:08.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f234f516740 10 client.16422 fetch_fsmap learned FSMap version 29 2024-09-19T08:20:08.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.375+0000 7f234f516740 1 -- 172.21.15.22:0/3898886413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558079053ab0 con 0x55807903bb40 2024-09-19T08:20:08.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.376+0000 7f232e7fc640 1 -- 172.21.15.22:0/3898886413 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23380041d0 con 0x55807903bb40 2024-09-19T08:20:08.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.376+0000 7f232e7fc640 1 -- 172.21.15.22:0/3898886413 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f23380049f0 con 0x55807903bb40 2024-09-19T08:20:08.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.376+0000 7f234f516740 10 client.16422 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:20:08.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.376+0000 7f234f516740 10 client.16422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:20:08.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.376+0000 7f234f516740 20 client.16422 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:08.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.376+0000 7f234f516740 1 --2- 172.21.15.22:0/3898886413 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558079054eb0 0x5580790572a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.376+0000 7f234f516740 4 client.16422 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:20:08.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.376+0000 7f234f516740 1 -- 172.21.15.22:0/3898886413 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558078fad860 con 0x558079054eb0 2024-09-19T08:20:08.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.376+0000 7f233ffff640 1 --2- 172.21.15.22:0/3898886413 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558079054eb0 0x5580790572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.376+0000 7f232e7fc640 10 client.16422 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:08.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.376+0000 7f233ffff640 1 --2- 172.21.15.22:0/3898886413 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558079054eb0 0x5580790572a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdadf0c5567f7958 server_cookie=409d2ab89a4c0bb5 in_seq=0 out_seq=0 2024-09-19T08:20:08.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.377+0000 7f232e7fc640 1 -- 172.21.15.22:0/3898886413 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558078fad860 con 0x558079054eb0 2024-09-19T08:20:08.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.377+0000 7f232e7fc640 10 client.16422 handle_command_reply: tid=0 2024-09-19T08:20:08.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.377+0000 7f234f516740 1 client.16422 shutdown 2024-09-19T08:20:08.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.378+0000 7f234f516740 1 -- 172.21.15.22:0/3898886413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 msgr2=0x558079042010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:08.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.378+0000 7f234f516740 1 --2- 172.21.15.22:0/3898886413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x558079042010 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2338011010 tx=0x7f233800bfd0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.378+0000 7f234f516740 1 -- 172.21.15.22:0/3898886413 shutdown_connections 2024-09-19T08:20:08.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.378+0000 7f234f516740 1 --2- 172.21.15.22:0/3898886413 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558079054eb0 0x5580790572a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.378+0000 7f234f516740 1 --2- 172.21.15.22:0/3898886413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 0x558079042600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.378+0000 7f234f516740 1 --2- 172.21.15.22:0/3898886413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x558079042010 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.378+0000 7f232e7fc640 0 client.16422 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:08.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.378+0000 7f234f516740 1 --2- 172.21.15.22:0/3898886413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903a5d0 0x558079044d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.378+0000 7f234f516740 1 -- 172.21.15.22:0/3898886413 >> 172.21.15.22:0/3898886413 conn(0x558079003310 msgr2=0x55807903c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:08.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.378+0000 7f234f516740 1 -- 172.21.15.22:0/3898886413 shutdown_connections 2024-09-19T08:20:08.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.378+0000 7f234f516740 1 -- 172.21.15.22:0/3898886413 wait complete. 2024-09-19T08:20:08.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.378+0000 7f234f516740 20 client.16422 trim_cache size 0 max 16384 2024-09-19T08:20:08.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.391+0000 7f234f516740 1 Processor -- start 2024-09-19T08:20:08.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.391+0000 7f234f516740 1 -- start start 2024-09-19T08:20:08.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.391+0000 7f234f516740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55807903a5d0 0x558079051340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.391+0000 7f234f516740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x558079040ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.391+0000 7f234f516740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558079045370 0x558079046760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.391+0000 7f234f516740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f23481b2480 con 0x55807903bb40 2024-09-19T08:20:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.391+0000 7f234f516740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f23481afcf0 con 0x55807903a5d0 2024-09-19T08:20:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.391+0000 7f234f516740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f23481adb40 con 0x558079045370 2024-09-19T08:20:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.391+0000 7f2346d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x558079040ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.391+0000 7f2346d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x558079040ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56982/0 (socket says 172.21.15.22:56982) 2024-09-19T08:20:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.391+0000 7f2346d9d640 1 -- 172.21.15.22:0/2736497823 learned_addr learned my addr 172.21.15.22:0/2736497823 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.391+0000 7f233ffff640 1 --2- 172.21.15.22:0/2736497823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558079045370 0x558079046760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.391+0000 7f233ffff640 1 --2- 172.21.15.22:0/2736497823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55807903a5d0 0x558079051340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.392+0000 7f2346d9d640 1 -- 172.21.15.22:0/2736497823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558079045370 msgr2=0x558079046760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.392+0000 7f2346d9d640 1 --2- 172.21.15.22:0/2736497823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558079045370 0x558079046760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.392+0000 7f2346d9d640 1 -- 172.21.15.22:0/2736497823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55807903a5d0 msgr2=0x558079051340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.392+0000 7f2346d9d640 1 --2- 172.21.15.22:0/2736497823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55807903a5d0 0x558079051340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.392+0000 7f2346d9d640 1 -- 172.21.15.22:0/2736497823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558079057830 con 0x55807903bb40 2024-09-19T08:20:08.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.392+0000 7f2346d9d640 1 --2- 172.21.15.22:0/2736497823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x558079040ec0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2338000c00 tx=0x7f233800b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c497e0588ddd8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:08.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.392+0000 7f233cff9640 1 -- 172.21.15.22:0/2736497823 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338011440 con 0x55807903bb40 2024-09-19T08:20:08.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.392+0000 7f233cff9640 1 -- 172.21.15.22:0/2736497823 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2338004a70 con 0x55807903bb40 2024-09-19T08:20:08.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.392+0000 7f233cff9640 1 -- 172.21.15.22:0/2736497823 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23380041a0 con 0x55807903bb40 2024-09-19T08:20:08.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.393+0000 7f234f516740 1 -- 172.21.15.22:0/2736497823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 msgr2=0x558079040ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:08.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.393+0000 7f234f516740 1 --2- 172.21.15.22:0/2736497823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x558079040ec0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2338000c00 tx=0x7f233800b440 comp rx=0 tx=0).stop 2024-09-19T08:20:08.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.393+0000 7f234f516740 1 -- 172.21.15.22:0/2736497823 shutdown_connections 2024-09-19T08:20:08.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.393+0000 7f234f516740 1 --2- 172.21.15.22:0/2736497823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558079045370 0x558079046760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.393+0000 7f234f516740 1 --2- 172.21.15.22:0/2736497823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903bb40 0x558079040ec0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.393+0000 7f234f516740 1 --2- 172.21.15.22:0/2736497823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55807903a5d0 0x558079051340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.393+0000 7f234f516740 1 -- 172.21.15.22:0/2736497823 >> 172.21.15.22:0/2736497823 conn(0x558079003310 msgr2=0x5580790407b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.393+0000 7f234f516740 1 -- 172.21.15.22:0/2736497823 shutdown_connections 2024-09-19T08:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.393+0000 7f234f516740 1 -- 172.21.15.22:0/2736497823 wait complete. 2024-09-19T08:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.393+0000 7f234f516740 1 -- start start 2024-09-19T08:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f234f516740 1 Processor -- start 2024-09-19T08:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f234f516740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903a5d0 0x55807905be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f234f516740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903bb40 0x55807905c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f234f516740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 0x55807904c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f234f516740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f23481ae1b0 con 0x55807903a5d0 2024-09-19T08:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f234f516740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f23481af620 con 0x558079045370 2024-09-19T08:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f234f516740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f23481ae880 con 0x55807903bb40 2024-09-19T08:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f234f516740 10 client.0 _ll_register_callbacks cb 0x7f2330009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f233ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903a5d0 0x55807905be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f233ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903a5d0 0x55807905be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56994/0 (socket says 172.21.15.22:56994) 2024-09-19T08:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f233ffff640 1 -- 172.21.15.22:0/3651345133 learned_addr learned my addr 172.21.15.22:0/3651345133 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f2346d9d640 1 --2- 172.21.15.22:0/3651345133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903bb40 0x55807905c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f233ffff640 1 --2- 172.21.15.22:0/3651345133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 0x55807904c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f233ffff640 1 -- 172.21.15.22:0/3651345133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903bb40 msgr2=0x55807905c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f233ffff640 1 --2- 172.21.15.22:0/3651345133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903bb40 0x55807905c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f233ffff640 1 -- 172.21.15.22:0/3651345133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 msgr2=0x55807904c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f233ffff640 1 --2- 172.21.15.22:0/3651345133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 0x55807904c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f233ffff640 1 -- 172.21.15.22:0/3651345133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558079037960 con 0x55807903a5d0 2024-09-19T08:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f233ffff640 1 --2- 172.21.15.22:0/3651345133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903a5d0 0x55807905be30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2330009230 tx=0x7f2330009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d8bf407087ae5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.394+0000 7f232e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.395+0000 7f232e7fc640 1 -- 172.21.15.22:0/3651345133 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330050070 con 0x55807903a5d0 2024-09-19T08:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.395+0000 7f232e7fc640 1 -- 172.21.15.22:0/3651345133 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2330005350 con 0x55807903a5d0 2024-09-19T08:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.395+0000 7f234f516740 1 -- 172.21.15.22:0/3651345133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2338004a70 con 0x55807903a5d0 2024-09-19T08:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.395+0000 7f232e7fc640 1 -- 172.21.15.22:0/3651345133 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2330035070 con 0x55807903a5d0 2024-09-19T08:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.395+0000 7f234f516740 10 client.16428 fetch_fsmap learned FSMap version 29 2024-09-19T08:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.395+0000 7f234f516740 1 -- 172.21.15.22:0/3651345133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55807904da20 con 0x55807903a5d0 2024-09-19T08:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.395+0000 7f232e7fc640 1 -- 172.21.15.22:0/3651345133 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2338004a70 con 0x55807903a5d0 2024-09-19T08:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.395+0000 7f232e7fc640 1 -- 172.21.15.22:0/3651345133 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2330055030 con 0x55807903a5d0 2024-09-19T08:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.395+0000 7f234f516740 10 client.16428 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.395+0000 7f234f516740 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.395+0000 7f234f516740 20 client.16428 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.395+0000 7f234f516740 1 --2- 172.21.15.22:0/3651345133 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5580790bfa30 0x5580790c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.395+0000 7f234f516740 4 client.16428 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.395+0000 7f234f516740 1 -- 172.21.15.22:0/3651345133 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558078fa7d70 con 0x5580790bfa30 2024-09-19T08:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f2346d9d640 1 --2- 172.21.15.22:0/3651345133 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5580790bfa30 0x5580790c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f2346d9d640 1 --2- 172.21.15.22:0/3651345133 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5580790bfa30 0x5580790c1e20 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fdab3bb1f33d99 server_cookie=4a129e13f047405c in_seq=0 out_seq=0 2024-09-19T08:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f232e7fc640 10 client.16428 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f232e7fc640 1 -- 172.21.15.22:0/3651345133 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558078fa7d70 con 0x5580790bfa30 2024-09-19T08:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f232e7fc640 10 client.16428 handle_command_reply: tid=0 2024-09-19T08:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f234f516740 1 client.16428 shutdown 2024-09-19T08:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f234f516740 1 -- 172.21.15.22:0/3651345133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903a5d0 msgr2=0x55807905be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f234f516740 1 --2- 172.21.15.22:0/3651345133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903a5d0 0x55807905be30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2330009230 tx=0x7f2330009260 comp rx=0 tx=0).stop 2024-09-19T08:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f234f516740 1 -- 172.21.15.22:0/3651345133 shutdown_connections 2024-09-19T08:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f234f516740 1 --2- 172.21.15.22:0/3651345133 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5580790bfa30 0x5580790c1e20 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f234f516740 1 --2- 172.21.15.22:0/3651345133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558079045370 0x55807904c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f234f516740 1 --2- 172.21.15.22:0/3651345133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55807903bb40 0x55807905c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f234f516740 1 --2- 172.21.15.22:0/3651345133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55807903a5d0 0x55807905be30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f234f516740 1 -- 172.21.15.22:0/3651345133 >> 172.21.15.22:0/3651345133 conn(0x558079003310 msgr2=0x55807903c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f232e7fc640 0 client.16428 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:08.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f234f516740 1 -- 172.21.15.22:0/3651345133 shutdown_connections 2024-09-19T08:20:08.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f234f516740 1 -- 172.21.15.22:0/3651345133 wait complete. 2024-09-19T08:20:08.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.396+0000 7f234f516740 20 client.16428 trim_cache size 0 max 16384 2024-09-19T08:20:08.404 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.397+0000 7f234d13d640 1 -- 172.21.15.22:0/1299266369 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f23340a33b0 msgr2=0x7f23340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.397+0000 7f234d13d640 1 --2- 172.21.15.22:0/1299266369 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f23340a33b0 0x7f23340a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f234009c5c0 tx=0x7f2340091ac0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.397+0000 7f234d13d640 1 -- 172.21.15.22:0/1299266369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2348102ef0 msgr2=0x7f23480fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.397+0000 7f234d13d640 1 --2- 172.21.15.22:0/1299266369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2348102ef0 0x7f23480fd700 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f234000eb10 tx=0x7f234000c2d0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.397+0000 7f234d13d640 1 -- 172.21.15.22:0/1299266369 shutdown_connections 2024-09-19T08:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.397+0000 7f234d13d640 1 --2- 172.21.15.22:0/1299266369 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f23340a33b0 0x7f23340a5870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.397+0000 7f234d13d640 1 --2- 172.21.15.22:0/1299266369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2348102ef0 0x7f23480fd700 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.397+0000 7f234d13d640 1 --2- 172.21.15.22:0/1299266369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2348102570 0x7f23480fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.397+0000 7f234d13d640 1 --2- 172.21.15.22:0/1299266369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2348100e20 0x7f23480fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.397+0000 7f234d13d640 1 -- 172.21.15.22:0/1299266369 >> 172.21.15.22:0/1299266369 conn(0x7f23480fc7d0 msgr2=0x7f234806a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.397+0000 7f234d13d640 1 -- 172.21.15.22:0/1299266369 shutdown_connections 2024-09-19T08:20:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:08.397+0000 7f234d13d640 1 -- 172.21.15.22:0/1299266369 wait complete. 2024-09-19T08:20:09.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:08 smithi064 ceph-mon[34900]: pgmap v156: 97 pgs: 97 active+clean; 615 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 372 KiB/s wr, 106 op/s 2024-09-19T08:20:09.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:08 smithi022 ceph-mon[30963]: pgmap v156: 97 pgs: 97 active+clean; 615 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 372 KiB/s wr, 106 op/s 2024-09-19T08:20:09.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:08 smithi157 ceph-mon[35143]: pgmap v156: 97 pgs: 97 active+clean; 615 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 372 KiB/s wr, 106 op/s 2024-09-19T08:20:10.191 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:20:10.192 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:20:10.460 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:10.842 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:10 smithi022 ceph-mon[30963]: pgmap v157: 97 pgs: 97 active+clean; 616 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 517 KiB/s wr, 141 op/s 2024-09-19T08:20:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.927+0000 7f9b9a01e640 1 -- 172.21.15.22:0/288549337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b94101420 msgr2=0x7f9b9410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.927+0000 7f9b9a01e640 1 --2- 172.21.15.22:0/288549337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b94101420 0x7f9b9410d950 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009cd0 tx=0x7f9b8c02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.927+0000 7f9b9a01e640 1 -- 172.21.15.22:0/288549337 shutdown_connections 2024-09-19T08:20:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.927+0000 7f9b9a01e640 1 --2- 172.21.15.22:0/288549337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b94101420 0x7f9b9410d950 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.927+0000 7f9b9a01e640 1 --2- 172.21.15.22:0/288549337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b94100a80 0x7f9b94100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.927+0000 7f9b9a01e640 1 --2- 172.21.15.22:0/288549337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b94106a80 0x7f9b94106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.927+0000 7f9b9a01e640 1 -- 172.21.15.22:0/288549337 >> 172.21.15.22:0/288549337 conn(0x7f9b940fc7a0 msgr2=0x7f9b940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.927+0000 7f9b9a01e640 1 -- 172.21.15.22:0/288549337 shutdown_connections 2024-09-19T08:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.927+0000 7f9b9a01e640 1 -- 172.21.15.22:0/288549337 wait complete. 2024-09-19T08:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.928+0000 7f9b9a01e640 1 Processor -- start 2024-09-19T08:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.928+0000 7f9b9a01e640 1 -- start start 2024-09-19T08:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.928+0000 7f9b9a01e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b94100a80 0x7f9b94198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.928+0000 7f9b9a01e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b94101420 0x7f9b94198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.928+0000 7f9b9a01e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b94106a80 0x7f9b9419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.928+0000 7f9b9a01e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94110e80 con 0x7f9b94100a80 2024-09-19T08:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.928+0000 7f9b9a01e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94110d00 con 0x7f9b94101420 2024-09-19T08:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.928+0000 7f9b9a01e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94111000 con 0x7f9b94106a80 2024-09-19T08:20:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.928+0000 7f9b937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b94100a80 0x7f9b94198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.928+0000 7f9b937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b94100a80 0x7f9b94198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57000/0 (socket says 172.21.15.22:57000) 2024-09-19T08:20:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.928+0000 7f9b937fe640 1 -- 172.21.15.22:0/139733648 learned_addr learned my addr 172.21.15.22:0/139733648 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.928+0000 7f9b93fff640 1 --2- 172.21.15.22:0/139733648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b94106a80 0x7f9b9419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.929+0000 7f9b92ffd640 1 --2- 172.21.15.22:0/139733648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b94101420 0x7f9b94198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.929+0000 7f9b937fe640 1 -- 172.21.15.22:0/139733648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b94106a80 msgr2=0x7f9b9419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.929+0000 7f9b937fe640 1 --2- 172.21.15.22:0/139733648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b94106a80 0x7f9b9419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.929+0000 7f9b937fe640 1 -- 172.21.15.22:0/139733648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b94101420 msgr2=0x7f9b94198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.929+0000 7f9b937fe640 1 --2- 172.21.15.22:0/139733648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b94101420 0x7f9b94198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.929+0000 7f9b937fe640 1 -- 172.21.15.22:0/139733648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9419d3e0 con 0x7f9b94100a80 2024-09-19T08:20:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.929+0000 7f9b937fe640 1 --2- 172.21.15.22:0/139733648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b94100a80 0x7f9b94198430 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400e9f0 tx=0x7f9b8400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.929+0000 7f9b90ff9640 1 -- 172.21.15.22:0/139733648 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8400cd80 con 0x7f9b94100a80 2024-09-19T08:20:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.929+0000 7f9b9a01e640 1 -- 172.21.15.22:0/139733648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b9419d6d0 con 0x7f9b94100a80 2024-09-19T08:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.929+0000 7f9b90ff9640 1 -- 172.21.15.22:0/139733648 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b84005500 con 0x7f9b94100a80 2024-09-19T08:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.929+0000 7f9b90ff9640 1 -- 172.21.15.22:0/139733648 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84010690 con 0x7f9b94100a80 2024-09-19T08:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.929+0000 7f9b9a01e640 1 -- 172.21.15.22:0/139733648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b941a4fb0 con 0x7f9b94100a80 2024-09-19T08:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.932+0000 7f9b9c3f7740 1 Processor -- start 2024-09-19T08:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.932+0000 7f9b9c3f7740 1 -- start start 2024-09-19T08:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.932+0000 7f9b90ff9640 1 -- 172.21.15.22:0/139733648 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9b840046c0 con 0x7f9b94100a80 2024-09-19T08:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.932+0000 7f9b9c3f7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d1fb60 0x55ed62d1ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.932+0000 7f9b9c3f7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d20480 0x55ed62d2a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b9c3f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d2ad40 0x55ed62d2d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b9c3f7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ed62c85280 con 0x55ed62d1fb60 2024-09-19T08:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b9c3f7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ed62c2cb10 con 0x55ed62d20480 2024-09-19T08:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b9c3f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ed62c81240 con 0x55ed62d2ad40 2024-09-19T08:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b93fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d1fb60 0x55ed62d1ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b93fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d1fb60 0x55ed62d1ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57014/0 (socket says 172.21.15.22:57014) 2024-09-19T08:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b93fff640 1 -- 172.21.15.22:0/732153402 learned_addr learned my addr 172.21.15.22:0/732153402 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b92ffd640 1 --2- 172.21.15.22:0/732153402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d2ad40 0x55ed62d2d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b937fe640 1 --2- 172.21.15.22:0/732153402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d20480 0x55ed62d2a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b93fff640 1 -- 172.21.15.22:0/732153402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d2ad40 msgr2=0x55ed62d2d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b93fff640 1 --2- 172.21.15.22:0/732153402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d2ad40 0x55ed62d2d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b93fff640 1 -- 172.21.15.22:0/732153402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d20480 msgr2=0x55ed62d2a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b93fff640 1 --2- 172.21.15.22:0/732153402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d20480 0x55ed62d2a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b93fff640 1 -- 172.21.15.22:0/732153402 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed62c391f0 con 0x55ed62d1fb60 2024-09-19T08:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b93fff640 1 --2- 172.21.15.22:0/732153402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d1fb60 0x55ed62d1ff40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009cd0 tx=0x7f9b8c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83d4dd5a090d2a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b90ff9640 1 --2- 172.21.15.22:0/139733648 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9b7c0a33b0 0x7f9b7c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.933+0000 7f9b79ffb640 1 -- 172.21.15.22:0/732153402 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c048070 con 0x55ed62d1fb60 2024-09-19T08:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.934+0000 7f9b79ffb640 1 -- 172.21.15.22:0/732153402 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b8c036070 con 0x55ed62d1fb60 2024-09-19T08:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.934+0000 7f9b79ffb640 1 -- 172.21.15.22:0/732153402 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c004420 con 0x55ed62d1fb60 2024-09-19T08:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.934+0000 7f9b937fe640 1 --2- 172.21.15.22:0/139733648 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9b7c0a33b0 0x7f9b7c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.934+0000 7f9b90ff9640 1 -- 172.21.15.22:0/139733648 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9b840d00c0 con 0x7f9b94100a80 2024-09-19T08:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.934+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/732153402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d1fb60 msgr2=0x55ed62d1ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.934+0000 7f9b937fe640 1 --2- 172.21.15.22:0/139733648 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9b7c0a33b0 0x7f9b7c0a5870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9b840cd8d0 tx=0x7f9b84004a40 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.934+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/732153402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d1fb60 0x55ed62d1ff40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009cd0 tx=0x7f9b8c009a20 comp rx=0 tx=0).stop 2024-09-19T08:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.934+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/732153402 shutdown_connections 2024-09-19T08:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.934+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/732153402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d2ad40 0x55ed62d2d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.934+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/732153402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d20480 0x55ed62d2a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.934+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/732153402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d1fb60 0x55ed62d1ff40 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.934+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/732153402 >> 172.21.15.22:0/732153402 conn(0x55ed62d1c0f0 msgr2=0x55ed62d1c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.934+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/732153402 shutdown_connections 2024-09-19T08:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.934+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/732153402 wait complete. 2024-09-19T08:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.935+0000 7f9b9c3f7740 1 -- start start 2024-09-19T08:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.935+0000 7f9b9c3f7740 1 Processor -- start 2024-09-19T08:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.935+0000 7f9b9c3f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d1fb60 0x55ed62d27bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.935+0000 7f9b9c3f7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d20480 0x55ed62d28110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.935+0000 7f9b9c3f7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d2ad40 0x55ed62d28650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.935+0000 7f9b9c3f7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ed62d31d20 con 0x55ed62d20480 2024-09-19T08:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.935+0000 7f9b9c3f7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ed62c3ff10 con 0x55ed62d2ad40 2024-09-19T08:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.935+0000 7f9b9c3f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ed62d31ea0 con 0x55ed62d1fb60 2024-09-19T08:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.935+0000 7f9b9c3f7740 10 client.0 _ll_register_callbacks cb 0x55ed62c08a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.935+0000 7f9b92ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d20480 0x55ed62d28110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.935+0000 7f9b92ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d20480 0x55ed62d28110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57022/0 (socket says 172.21.15.22:57022) 2024-09-19T08:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.935+0000 7f9b92ffd640 1 -- 172.21.15.22:0/3084587690 learned_addr learned my addr 172.21.15.22:0/3084587690 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.935+0000 7f9b93fff640 1 --2- 172.21.15.22:0/3084587690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d1fb60 0x55ed62d27bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b93fff640 1 --2- 172.21.15.22:0/3084587690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d2ad40 0x55ed62d28650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b92ffd640 1 -- 172.21.15.22:0/3084587690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d1fb60 msgr2=0x55ed62d27bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b92ffd640 1 --2- 172.21.15.22:0/3084587690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d1fb60 0x55ed62d27bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b92ffd640 1 -- 172.21.15.22:0/3084587690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d2ad40 msgr2=0x55ed62d28650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b92ffd640 1 --2- 172.21.15.22:0/3084587690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d2ad40 0x55ed62d28650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b92ffd640 1 -- 172.21.15.22:0/3084587690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed629f66a0 con 0x55ed62d20480 2024-09-19T08:20:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b93fff640 1 --2- 172.21.15.22:0/3084587690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d2ad40 0x55ed62d28650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:20:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b92ffd640 1 --2- 172.21.15.22:0/3084587690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d20480 0x55ed62d28110 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800bad0 tx=0x7f9b8800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378854a2d0b89c61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b72ffd640 1 -- 172.21.15.22:0/3084587690 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88003e20 con 0x55ed62d20480 2024-09-19T08:20:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b72ffd640 1 -- 172.21.15.22:0/3084587690 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9b88011440 con 0x55ed62d20480 2024-09-19T08:20:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/3084587690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b8c036070 con 0x55ed62d20480 2024-09-19T08:20:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b72ffd640 1 -- 172.21.15.22:0/3084587690 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9b88005020 con 0x55ed62d20480 2024-09-19T08:20:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b9c3f7740 10 client.16440 fetch_fsmap learned FSMap version 29 2024-09-19T08:20:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/3084587690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed62d3d930 con 0x55ed62d20480 2024-09-19T08:20:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.936+0000 7f9b72ffd640 1 -- 172.21.15.22:0/3084587690 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b8c036070 con 0x55ed62d20480 2024-09-19T08:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.937+0000 7f9b72ffd640 1 -- 172.21.15.22:0/3084587690 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9b880116d0 con 0x55ed62d20480 2024-09-19T08:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.937+0000 7f9b9c3f7740 10 client.16440 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.937+0000 7f9b9c3f7740 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.937+0000 7f9b9c3f7740 20 client.16440 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.937+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/3084587690 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ed62d3ee90 0x55ed62d412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.937+0000 7f9b9c3f7740 4 client.16440 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.937+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/3084587690 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed62c927f0 con 0x55ed62d3ee90 2024-09-19T08:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.937+0000 7f9b93fff640 1 --2- 172.21.15.22:0/3084587690 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ed62d3ee90 0x55ed62d412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.937+0000 7f9b72ffd640 10 client.16440 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.937+0000 7f9b93fff640 1 --2- 172.21.15.22:0/3084587690 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ed62d3ee90 0x55ed62d412b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afb172ab16fb67d6 server_cookie=5b109db3d3831cb in_seq=0 out_seq=0 2024-09-19T08:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.938+0000 7f9b72ffd640 1 -- 172.21.15.22:0/3084587690 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed62c927f0 con 0x55ed62d3ee90 2024-09-19T08:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.938+0000 7f9b72ffd640 10 client.16440 handle_command_reply: tid=0 2024-09-19T08:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.938+0000 7f9b9c3f7740 1 client.16440 shutdown 2024-09-19T08:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.939+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/3084587690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d20480 msgr2=0x55ed62d28110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.939+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/3084587690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d20480 0x55ed62d28110 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800bad0 tx=0x7f9b8800bf90 comp rx=0 tx=0).stop 2024-09-19T08:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.939+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/3084587690 shutdown_connections 2024-09-19T08:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.939+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/3084587690 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ed62d3ee90 0x55ed62d412b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.939+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/3084587690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d2ad40 0x55ed62d28650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.939+0000 7f9b72ffd640 0 client.16440 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.939+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/3084587690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d20480 0x55ed62d28110 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.939+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/3084587690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d1fb60 0x55ed62d27bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.939+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/3084587690 >> 172.21.15.22:0/3084587690 conn(0x55ed62d1c0f0 msgr2=0x55ed62d21e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.939+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/3084587690 shutdown_connections 2024-09-19T08:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.939+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/3084587690 wait complete. 2024-09-19T08:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.939+0000 7f9b9c3f7740 20 client.16440 trim_cache size 0 max 16384 2024-09-19T08:20:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.956+0000 7f9b9c3f7740 1 Processor -- start 2024-09-19T08:20:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.956+0000 7f9b9c3f7740 1 -- start start 2024-09-19T08:20:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.956+0000 7f9b9c3f7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d1fb60 0x55ed62d26a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b9c3f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d20480 0x55ed62d284a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b9c3f7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d2ad40 0x55ed62d46b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b9c3f7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941adf40 con 0x55ed62d1fb60 2024-09-19T08:20:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b9c3f7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941ab7b0 con 0x55ed62d2ad40 2024-09-19T08:20:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b9c3f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941a9600 con 0x55ed62d20480 2024-09-19T08:20:10.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b93fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d1fb60 0x55ed62d26a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b93fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d1fb60 0x55ed62d26a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57030/0 (socket says 172.21.15.22:57030) 2024-09-19T08:20:10.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b93fff640 1 -- 172.21.15.22:0/159917325 learned_addr learned my addr 172.21.15.22:0/159917325 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:10.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b92ffd640 1 --2- 172.21.15.22:0/159917325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d20480 0x55ed62d284a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b93fff640 1 -- 172.21.15.22:0/159917325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d20480 msgr2=0x55ed62d284a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:10.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b93fff640 1 --2- 172.21.15.22:0/159917325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d20480 0x55ed62d284a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b93fff640 1 -- 172.21.15.22:0/159917325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d2ad40 msgr2=0x55ed62d46b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:10.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b93fff640 1 --2- 172.21.15.22:0/159917325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d2ad40 0x55ed62d46b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b93fff640 1 -- 172.21.15.22:0/159917325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed62d2d470 con 0x55ed62d1fb60 2024-09-19T08:20:10.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b93fff640 1 --2- 172.21.15.22:0/159917325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d1fb60 0x55ed62d26a80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c005e30 tx=0x7f9b8c003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e88addbe4827bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:10.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b797fa640 1 -- 172.21.15.22:0/159917325 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c048040 con 0x55ed62d1fb60 2024-09-19T08:20:10.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b797fa640 1 -- 172.21.15.22:0/159917325 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b8c036040 con 0x55ed62d1fb60 2024-09-19T08:20:10.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.957+0000 7f9b797fa640 1 -- 172.21.15.22:0/159917325 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c030d70 con 0x55ed62d1fb60 2024-09-19T08:20:10.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.958+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/159917325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d1fb60 msgr2=0x55ed62d26a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:10.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.958+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/159917325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d1fb60 0x55ed62d26a80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c005e30 tx=0x7f9b8c003ea0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.958+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/159917325 shutdown_connections 2024-09-19T08:20:10.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.958+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/159917325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d2ad40 0x55ed62d46b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.958+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/159917325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d20480 0x55ed62d284a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.958+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/159917325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d1fb60 0x55ed62d26a80 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.958+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/159917325 >> 172.21.15.22:0/159917325 conn(0x55ed62d1c0f0 msgr2=0x55ed62d26370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:10.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.958+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/159917325 shutdown_connections 2024-09-19T08:20:10.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.958+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/159917325 wait complete. 2024-09-19T08:20:10.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.958+0000 7f9b9c3f7740 1 -- start start 2024-09-19T08:20:10.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b9c3f7740 1 Processor -- start 2024-09-19T08:20:10.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b9c3f7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d1fb60 0x55ed62d24be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b9c3f7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d20480 0x55ed62d25120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b9c3f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d2ad40 0x55ed62d25660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b9c3f7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941a9c70 con 0x55ed62d20480 2024-09-19T08:20:10.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b9c3f7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941ab0e0 con 0x55ed62d1fb60 2024-09-19T08:20:10.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b9c3f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941aa340 con 0x55ed62d2ad40 2024-09-19T08:20:10.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b9c3f7740 10 client.0 _ll_register_callbacks cb 0x7f9b88004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:10.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b92ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d20480 0x55ed62d25120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b92ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d20480 0x55ed62d25120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57036/0 (socket says 172.21.15.22:57036) 2024-09-19T08:20:10.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b92ffd640 1 -- 172.21.15.22:0/1698139527 learned_addr learned my addr 172.21.15.22:0/1698139527 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:10.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b93fff640 1 --2- 172.21.15.22:0/1698139527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d2ad40 0x55ed62d25660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b93fff640 1 --2- 172.21.15.22:0/1698139527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d1fb60 0x55ed62d24be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b92ffd640 1 -- 172.21.15.22:0/1698139527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d2ad40 msgr2=0x55ed62d25660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:10.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b92ffd640 1 --2- 172.21.15.22:0/1698139527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d2ad40 0x55ed62d25660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b92ffd640 1 -- 172.21.15.22:0/1698139527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d1fb60 msgr2=0x55ed62d24be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:10.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b92ffd640 1 --2- 172.21.15.22:0/1698139527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d1fb60 0x55ed62d24be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b92ffd640 1 -- 172.21.15.22:0/1698139527 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed62da8b30 con 0x55ed62d20480 2024-09-19T08:20:10.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.959+0000 7f9b93fff640 1 --2- 172.21.15.22:0/1698139527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d1fb60 0x55ed62d24be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:20:10.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.960+0000 7f9b92ffd640 1 --2- 172.21.15.22:0/1698139527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d20480 0x55ed62d25120 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800baa0 tx=0x7f9b88005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659d5d19662264b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:10.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.960+0000 7f9b72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:10.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.960+0000 7f9b72ffd640 1 -- 172.21.15.22:0/1698139527 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88031060 con 0x55ed62d20480 2024-09-19T08:20:10.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.960+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/1698139527 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b8c036040 con 0x55ed62d20480 2024-09-19T08:20:10.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.960+0000 7f9b72ffd640 1 -- 172.21.15.22:0/1698139527 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9b8800b400 con 0x55ed62d20480 2024-09-19T08:20:10.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.960+0000 7f9b72ffd640 1 -- 172.21.15.22:0/1698139527 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9b88027040 con 0x55ed62d20480 2024-09-19T08:20:10.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.960+0000 7f9b9c3f7740 10 client.16446 fetch_fsmap learned FSMap version 29 2024-09-19T08:20:10.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.960+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/1698139527 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed62da9e60 con 0x55ed62d20480 2024-09-19T08:20:10.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.960+0000 7f9b72ffd640 1 -- 172.21.15.22:0/1698139527 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b8c036040 con 0x55ed62d20480 2024-09-19T08:20:10.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.960+0000 7f9b72ffd640 1 -- 172.21.15.22:0/1698139527 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9b88004cf0 con 0x55ed62d20480 2024-09-19T08:20:10.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.961+0000 7f9b9c3f7740 10 client.16446 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:20:10.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.961+0000 7f9b9c3f7740 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:20:10.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.961+0000 7f9b9c3f7740 20 client.16446 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:10.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.961+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/1698139527 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ed62daa610 0x55ed62daca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:10.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.961+0000 7f9b9c3f7740 4 client.16446 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:20:10.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.961+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/1698139527 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ed62c8cd00 con 0x55ed62daa610 2024-09-19T08:20:10.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.961+0000 7f9b93fff640 1 --2- 172.21.15.22:0/1698139527 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ed62daa610 0x55ed62daca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:10.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.961+0000 7f9b72ffd640 10 client.16446 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:10.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.961+0000 7f9b93fff640 1 --2- 172.21.15.22:0/1698139527 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ed62daa610 0x55ed62daca30 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba9ea9b522bb111 server_cookie=e5e64738cb063dc9 in_seq=0 out_seq=0 2024-09-19T08:20:10.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.962+0000 7f9b72ffd640 1 -- 172.21.15.22:0/1698139527 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ed62c8cd00 con 0x55ed62daa610 2024-09-19T08:20:10.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.962+0000 7f9b72ffd640 10 client.16446 handle_command_reply: tid=0 2024-09-19T08:20:10.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.962+0000 7f9b9c3f7740 1 client.16446 shutdown 2024-09-19T08:20:10.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.963+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/1698139527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d20480 msgr2=0x55ed62d25120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:10.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.963+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/1698139527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d20480 0x55ed62d25120 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800baa0 tx=0x7f9b88005770 comp rx=0 tx=0).stop 2024-09-19T08:20:10.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.963+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/1698139527 shutdown_connections 2024-09-19T08:20:10.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.963+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/1698139527 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55ed62daa610 0x55ed62daca30 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.963+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/1698139527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ed62d2ad40 0x55ed62d25660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.963+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/1698139527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed62d20480 0x55ed62d25120 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.963+0000 7f9b72ffd640 0 client.16446 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:10.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.963+0000 7f9b9c3f7740 1 --2- 172.21.15.22:0/1698139527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed62d1fb60 0x55ed62d24be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.963+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/1698139527 >> 172.21.15.22:0/1698139527 conn(0x55ed62d1c0f0 msgr2=0x55ed62d1b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:10.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.963+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/1698139527 shutdown_connections 2024-09-19T08:20:10.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.963+0000 7f9b9c3f7740 1 -- 172.21.15.22:0/1698139527 wait complete. 2024-09-19T08:20:10.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.963+0000 7f9b9c3f7740 20 client.16446 trim_cache size 0 max 16384 2024-09-19T08:20:10.968 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:20:10.968 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-19T08:20:10.968 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "fb8b065c-674f-4534-96ac-744d3fb32f90", 2024-09-19T08:20:10.968 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-19T08:20:10.968 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:20:10.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.964+0000 7f9b9a01e640 1 -- 172.21.15.22:0/139733648 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9b7c0a33b0 msgr2=0x7f9b7c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:10.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.964+0000 7f9b9a01e640 1 --2- 172.21.15.22:0/139733648 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9b7c0a33b0 0x7f9b7c0a5870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9b840cd8d0 tx=0x7f9b84004a40 comp rx=0 tx=0).stop 2024-09-19T08:20:10.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.964+0000 7f9b9a01e640 1 -- 172.21.15.22:0/139733648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b94100a80 msgr2=0x7f9b94198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:10.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.964+0000 7f9b9a01e640 1 --2- 172.21.15.22:0/139733648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b94100a80 0x7f9b94198430 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400e9f0 tx=0x7f9b8400edf0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.964+0000 7f9b9a01e640 1 -- 172.21.15.22:0/139733648 shutdown_connections 2024-09-19T08:20:10.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.964+0000 7f9b9a01e640 1 --2- 172.21.15.22:0/139733648 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9b7c0a33b0 0x7f9b7c0a5870 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.964+0000 7f9b9a01e640 1 --2- 172.21.15.22:0/139733648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b94106a80 0x7f9b9419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.964+0000 7f9b9a01e640 1 --2- 172.21.15.22:0/139733648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b94101420 0x7f9b94198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.964+0000 7f9b9a01e640 1 --2- 172.21.15.22:0/139733648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b94100a80 0x7f9b94198430 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:10.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.964+0000 7f9b9a01e640 1 -- 172.21.15.22:0/139733648 >> 172.21.15.22:0/139733648 conn(0x7f9b940fc7a0 msgr2=0x7f9b9406a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:10.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.964+0000 7f9b9a01e640 1 -- 172.21.15.22:0/139733648 shutdown_connections 2024-09-19T08:20:10.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:10.964+0000 7f9b9a01e640 1 -- 172.21.15.22:0/139733648 wait complete. 2024-09-19T08:20:11.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:10 smithi064 ceph-mon[34900]: pgmap v157: 97 pgs: 97 active+clean; 616 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 517 KiB/s wr, 141 op/s 2024-09-19T08:20:11.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:10 smithi157 ceph-mon[35143]: pgmap v157: 97 pgs: 97 active+clean; 616 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 517 KiB/s wr, 141 op/s 2024-09-19T08:20:11.791 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:20:12.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:11 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:20:12.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:11 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:12.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:11 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:20:12.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:11 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:12.061 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:12.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:11 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:20:12.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:11 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:12.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:11 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:20:12.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:11 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:12.107 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:20:12.107 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 29, 2024-09-19T08:20:12.107 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:20:12.107 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 22, 2024-09-19T08:20:12.107 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:20:12.108 INFO:teuthology.orchestra.run.smithi022.stdout: "2af777fa": { 2024-09-19T08:20:12.108 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 22, 2024-09-19T08:20:12.108 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:20:12.108 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:20:12.108 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:20:12.108 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:20:12.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:20:12.108 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:20:12.108 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 149.8, 2024-09-19T08:20:12.108 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:20:12.108 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:20:12.108 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:20:12.109 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:20:12.109 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:20:12.109 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:20:12.109 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:20:12.109 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:20:12.109 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:20:12.109 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:20:12.109 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:20:12.109 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:20:12.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.106+0000 7ffa5d7fa640 1 -- 172.21.15.22:0/191130775 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558d4bc97d20 con 0x558d4bdb0e60 2024-09-19T08:20:12.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.106+0000 7ffa5d7fa640 10 client.16368 handle_command_reply: tid=0 2024-09-19T08:20:12.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.106+0000 7ffa7e102740 1 client.16368 shutdown 2024-09-19T08:20:12.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.106+0000 7ffa7e102740 1 -- 172.21.15.22:0/191130775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2b080 msgr2=0x558d4bd3a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:12.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.106+0000 7ffa7e102740 1 --2- 172.21.15.22:0/191130775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2b080 0x558d4bd3a2c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x558d4bd27b50 tx=0x7ffa7000b4b0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.106+0000 7ffa7e102740 1 -- 172.21.15.22:0/191130775 shutdown_connections 2024-09-19T08:20:12.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.106+0000 7ffa7e102740 1 --2- 172.21.15.22:0/191130775 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558d4bdb0e60 0x558d4bdb3280 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.106+0000 7ffa7e102740 1 --2- 172.21.15.22:0/191130775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d4bd35940 0x558d4bdaf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.106+0000 7ffa7e102740 1 --2- 172.21.15.22:0/191130775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d4bd2b080 0x558d4bd3a2c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.106+0000 7ffa7e102740 1 --2- 172.21.15.22:0/191130775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558d4bd2a760 0x558d4bd39d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.106+0000 7ffa7e102740 1 -- 172.21.15.22:0/191130775 >> 172.21.15.22:0/191130775 conn(0x558d4bd26cf0 msgr2=0x558d4bd32f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.106+0000 7ffa5d7fa640 0 client.16368 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.106+0000 7ffa7e102740 1 -- 172.21.15.22:0/191130775 shutdown_connections 2024-09-19T08:20:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.106+0000 7ffa7e102740 1 -- 172.21.15.22:0/191130775 wait complete. 2024-09-19T08:20:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.107+0000 7ffa7e102740 20 client.16368 trim_cache size 0 max 16384 2024-09-19T08:20:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.107+0000 7ffa7bd29640 1 -- 172.21.15.22:0/404938714 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ffa600a3430 msgr2=0x7ffa600a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.107+0000 7ffa7bd29640 1 --2- 172.21.15.22:0/404938714 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ffa600a3430 0x7ffa600a58f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0cdba0 tx=0x7ffa6c00efd0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.107+0000 7ffa7bd29640 1 -- 172.21.15.22:0/404938714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7410c170 msgr2=0x7ffa74101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.107+0000 7ffa7bd29640 1 --2- 172.21.15.22:0/404938714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7410c170 0x7ffa74101470 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00eb10 tx=0x7ffa6c00c2d0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.107+0000 7ffa7bd29640 1 -- 172.21.15.22:0/404938714 shutdown_connections 2024-09-19T08:20:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.107+0000 7ffa7bd29640 1 --2- 172.21.15.22:0/404938714 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ffa600a3430 0x7ffa600a58f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.107+0000 7ffa7bd29640 1 --2- 172.21.15.22:0/404938714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7410c170 0x7ffa74101470 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.107+0000 7ffa7bd29640 1 --2- 172.21.15.22:0/404938714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa741036b0 0x7ffa74100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.107+0000 7ffa7bd29640 1 --2- 172.21.15.22:0/404938714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa74102d90 0x7ffa741009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.107+0000 7ffa7bd29640 1 -- 172.21.15.22:0/404938714 >> 172.21.15.22:0/404938714 conn(0x7ffa740fc7d0 msgr2=0x7ffa740fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:12.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.107+0000 7ffa7bd29640 1 -- 172.21.15.22:0/404938714 shutdown_connections 2024-09-19T08:20:12.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.108+0000 7ffa7bd29640 1 -- 172.21.15.22:0/404938714 wait complete. 2024-09-19T08:20:12.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:11 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:20:12.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:11 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:12.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:11 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:20:12.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:11 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:12.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.796+0000 7f17b3130640 1 -- 172.21.15.22:0/1799168748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17ac10c180 msgr2=0x7f17ac10e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:12.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.796+0000 7f17b3130640 1 --2- 172.21.15.22:0/1799168748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17ac10c180 0x7f17ac10e620 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009ca0 tx=0x7f17a402a950 comp rx=0 tx=0).stop 2024-09-19T08:20:12.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.796+0000 7f17b3130640 1 -- 172.21.15.22:0/1799168748 shutdown_connections 2024-09-19T08:20:12.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.796+0000 7f17b3130640 1 --2- 172.21.15.22:0/1799168748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17ac10c180 0x7f17ac10e620 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.796+0000 7f17b3130640 1 --2- 172.21.15.22:0/1799168748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17ac103670 0x7f17ac10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.796+0000 7f17b3130640 1 --2- 172.21.15.22:0/1799168748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17ac102d50 0x7f17ac103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.796+0000 7f17b3130640 1 -- 172.21.15.22:0/1799168748 >> 172.21.15.22:0/1799168748 conn(0x7f17ac0fc7d0 msgr2=0x7f17ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:12.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.796+0000 7f17b3130640 1 -- 172.21.15.22:0/1799168748 shutdown_connections 2024-09-19T08:20:12.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.796+0000 7f17b3130640 1 -- 172.21.15.22:0/1799168748 wait complete. 2024-09-19T08:20:12.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.797+0000 7f17b3130640 1 Processor -- start 2024-09-19T08:20:12.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.797+0000 7f17b3130640 1 -- start start 2024-09-19T08:20:12.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.797+0000 7f17b3130640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17ac102d50 0x7f17ac1009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.797+0000 7f17b3130640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17ac103670 0x7f17ac100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.797+0000 7f17b3130640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17ac10c180 0x7f17ac101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.797+0000 7f17b3130640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac111f60 con 0x7f17ac10c180 2024-09-19T08:20:12.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.797+0000 7f17b3130640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac111de0 con 0x7f17ac103670 2024-09-19T08:20:12.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.797+0000 7f17b3130640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1120e0 con 0x7f17ac102d50 2024-09-19T08:20:12.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.797+0000 7f17b16cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17ac10c180 0x7f17ac101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.797+0000 7f17b0ecc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17ac102d50 0x7f17ac1009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.797+0000 7f17b16cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17ac10c180 0x7f17ac101430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57048/0 (socket says 172.21.15.22:57048) 2024-09-19T08:20:12.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.797+0000 7f17b16cd640 1 -- 172.21.15.22:0/534634481 learned_addr learned my addr 172.21.15.22:0/534634481 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:12.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.797+0000 7f17abfff640 1 --2- 172.21.15.22:0/534634481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17ac103670 0x7f17ac100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.798+0000 7f17b0ecc640 1 -- 172.21.15.22:0/534634481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17ac103670 msgr2=0x7f17ac100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:12.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.798+0000 7f17b0ecc640 1 --2- 172.21.15.22:0/534634481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17ac103670 0x7f17ac100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.798+0000 7f17b0ecc640 1 -- 172.21.15.22:0/534634481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17ac10c180 msgr2=0x7f17ac101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:12.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.798+0000 7f17b0ecc640 1 --2- 172.21.15.22:0/534634481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17ac10c180 0x7f17ac101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.798+0000 7f17b0ecc640 1 -- 172.21.15.22:0/534634481 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ac1a6900 con 0x7f17ac102d50 2024-09-19T08:20:12.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.798+0000 7f17b16cd640 1 --2- 172.21.15.22:0/534634481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17ac10c180 0x7f17ac101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:12.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.798+0000 7f17b0ecc640 1 --2- 172.21.15.22:0/534634481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17ac102d50 0x7f17ac1009b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f17a000e9f0 tx=0x7f17a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:12.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.798+0000 7f17a9ffb640 1 -- 172.21.15.22:0/534634481 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a000ccc0 con 0x7f17ac102d50 2024-09-19T08:20:12.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.798+0000 7f17a9ffb640 1 -- 172.21.15.22:0/534634481 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a0010910 con 0x7f17ac102d50 2024-09-19T08:20:12.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.798+0000 7f17b3130640 1 -- 172.21.15.22:0/534634481 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ac1a6bf0 con 0x7f17ac102d50 2024-09-19T08:20:12.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.798+0000 7f17a9ffb640 1 -- 172.21.15.22:0/534634481 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0018960 con 0x7f17ac102d50 2024-09-19T08:20:12.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.798+0000 7f17b3130640 1 -- 172.21.15.22:0/534634481 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ac1a7130 con 0x7f17ac102d50 2024-09-19T08:20:12.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.800+0000 7f17a9ffb640 1 -- 172.21.15.22:0/534634481 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f17a0018b00 con 0x7f17ac102d50 2024-09-19T08:20:12.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.801+0000 7f17b5509740 1 Processor -- start 2024-09-19T08:20:12.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.801+0000 7f17b5509740 1 -- start start 2024-09-19T08:20:12.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.801+0000 7f17b5509740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.801+0000 7f17b5509740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 0x56417fb2a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.801+0000 7f17b5509740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb2d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.801+0000 7f17b5509740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56417fa7d2e0 con 0x56417fb173c0 2024-09-19T08:20:12.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.801+0000 7f17b5509740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56417fa24ce0 con 0x56417fb2ae90 2024-09-19T08:20:12.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.801+0000 7f17b5509740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56417fa792a0 con 0x56417fb17ce0 2024-09-19T08:20:12.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.801+0000 7f17b16cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.801+0000 7f17b16cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb177a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57054/0 (socket says 172.21.15.22:57054) 2024-09-19T08:20:12.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.801+0000 7f17b16cd640 1 -- 172.21.15.22:0/1095025911 learned_addr learned my addr 172.21.15.22:0/1095025911 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:12.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.801+0000 7f17a9ffb640 1 --2- 172.21.15.22:0/534634481 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f179c0a33b0 0x7f179c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.801+0000 7f17b0ecc640 1 --2- 172.21.15.22:0/1095025911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 0x56417fb2a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.801+0000 7f17abfff640 1 --2- 172.21.15.22:0/1095025911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb2d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.802+0000 7f17b16cd640 1 --2- 172.21.15.22:0/534634481 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f179c0a33b0 0x7f179c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.802+0000 7f17b0ecc640 1 -- 172.21.15.22:0/1095025911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 msgr2=0x56417fb2d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:12.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.802+0000 7f17b0ecc640 1 --2- 172.21.15.22:0/1095025911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb2d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.802+0000 7f17b0ecc640 1 -- 172.21.15.22:0/1095025911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 msgr2=0x56417fb177a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:12.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.802+0000 7f17b0ecc640 1 --2- 172.21.15.22:0/1095025911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb177a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.802+0000 7f17b0ecc640 1 -- 172.21.15.22:0/1095025911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56417fa09440 con 0x56417fb17ce0 2024-09-19T08:20:12.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.802+0000 7f17abfff640 1 --2- 172.21.15.22:0/1095025911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb2d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:12.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.802+0000 7f17b16cd640 1 --2- 172.21.15.22:0/1095025911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb177a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:12.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.802+0000 7f17a9ffb640 1 -- 172.21.15.22:0/534634481 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f17a00d00b0 con 0x7f17ac102d50 2024-09-19T08:20:12.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.802+0000 7f17b0ecc640 1 --2- 172.21.15.22:0/1095025911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 0x56417fb2a950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f17a00cd810 tx=0x7f17a0024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cb582554e2b24f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:12.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.802+0000 7f17b16cd640 1 --2- 172.21.15.22:0/534634481 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f179c0a33b0 0x7f179c0a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f17a402f610 tx=0x7f17a402a950 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:12.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.802+0000 7f1796ffd640 1 -- 172.21.15.22:0/1095025911 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0016740 con 0x56417fb17ce0 2024-09-19T08:20:12.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.802+0000 7f1796ffd640 1 -- 172.21.15.22:0/1095025911 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a00a0750 con 0x56417fb17ce0 2024-09-19T08:20:12.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.802+0000 7f1796ffd640 1 -- 172.21.15.22:0/1095025911 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a00986f0 con 0x56417fb17ce0 2024-09-19T08:20:12.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.803+0000 7f17b5509740 1 -- 172.21.15.22:0/1095025911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 msgr2=0x56417fb2a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.803+0000 7f17b5509740 1 --2- 172.21.15.22:0/1095025911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 0x56417fb2a950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f17a00cd810 tx=0x7f17a0024000 comp rx=0 tx=0).stop 2024-09-19T08:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.803+0000 7f17b5509740 1 -- 172.21.15.22:0/1095025911 shutdown_connections 2024-09-19T08:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.803+0000 7f17b5509740 1 --2- 172.21.15.22:0/1095025911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb2d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.803+0000 7f17b5509740 1 --2- 172.21.15.22:0/1095025911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 0x56417fb2a950 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.803+0000 7f17b5509740 1 --2- 172.21.15.22:0/1095025911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb177a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.803+0000 7f17b5509740 1 -- 172.21.15.22:0/1095025911 >> 172.21.15.22:0/1095025911 conn(0x56417fb13a30 msgr2=0x56417fb13e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.803+0000 7f17b5509740 1 -- 172.21.15.22:0/1095025911 shutdown_connections 2024-09-19T08:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.803+0000 7f17b5509740 1 -- 172.21.15.22:0/1095025911 wait complete. 2024-09-19T08:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.803+0000 7f17b5509740 1 -- start start 2024-09-19T08:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b5509740 1 Processor -- start 2024-09-19T08:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b5509740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb2a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b5509740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 0x56417fb27b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b5509740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb28130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b5509740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56417fb2d9a0 con 0x56417fb173c0 2024-09-19T08:20:12.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b5509740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56417fa38050 con 0x56417fb2ae90 2024-09-19T08:20:12.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b5509740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56417fb2db20 con 0x56417fb17ce0 2024-09-19T08:20:12.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b0ecc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb2a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b0ecc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb2a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57068/0 (socket says 172.21.15.22:57068) 2024-09-19T08:20:12.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b0ecc640 1 -- 172.21.15.22:0/647703541 learned_addr learned my addr 172.21.15.22:0/647703541 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:12.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b5509740 10 client.0 _ll_register_callbacks cb 0x56417fa78160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:12.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17abfff640 1 --2- 172.21.15.22:0/647703541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 0x56417fb27b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b16cd640 1 --2- 172.21.15.22:0/647703541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb28130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b0ecc640 1 -- 172.21.15.22:0/647703541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 msgr2=0x56417fb27b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:12.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b0ecc640 1 --2- 172.21.15.22:0/647703541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 0x56417fb27b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b0ecc640 1 -- 172.21.15.22:0/647703541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 msgr2=0x56417fb28130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:12.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b0ecc640 1 --2- 172.21.15.22:0/647703541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb28130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.804+0000 7f17b0ecc640 1 -- 172.21.15.22:0/647703541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56417f7ee3c0 con 0x56417fb173c0 2024-09-19T08:20:12.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.805+0000 7f17b0ecc640 1 --2- 172.21.15.22:0/647703541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb2a890 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f17a009c150 tx=0x7f17a0016980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3030e419f2dbf23f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:12.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.805+0000 7f178bfff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:12.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.805+0000 7f178bfff640 1 -- 172.21.15.22:0/647703541 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a00259a0 con 0x56417fb173c0 2024-09-19T08:20:12.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.805+0000 7f178bfff640 1 -- 172.21.15.22:0/647703541 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f17a0010ac0 con 0x56417fb173c0 2024-09-19T08:20:12.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.805+0000 7f17b5509740 1 -- 172.21.15.22:0/647703541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a00a0750 con 0x56417fb173c0 2024-09-19T08:20:12.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.805+0000 7f178bfff640 1 -- 172.21.15.22:0/647703541 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17a00986f0 con 0x56417fb173c0 2024-09-19T08:20:12.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.805+0000 7f17b5509740 10 client.16458 fetch_fsmap learned FSMap version 30 2024-09-19T08:20:12.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.805+0000 7f17b5509740 1 -- 172.21.15.22:0/647703541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56417fb39440 con 0x56417fb173c0 2024-09-19T08:20:12.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.805+0000 7f178bfff640 1 -- 172.21.15.22:0/647703541 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a00a0750 con 0x56417fb173c0 2024-09-19T08:20:12.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.805+0000 7f178bfff640 1 -- 172.21.15.22:0/647703541 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f17a00cda90 con 0x56417fb173c0 2024-09-19T08:20:12.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.806+0000 7f17b5509740 10 client.16458 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:20:12.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.806+0000 7f17b5509740 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:20:12.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.806+0000 7f17b5509740 20 client.16458 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:12.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.806+0000 7f17b5509740 1 --2- 172.21.15.22:0/647703541 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56417fb3a790 0x56417fb3cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.806+0000 7f17b5509740 4 client.16458 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:20:12.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.806+0000 7f17b5509740 1 -- 172.21.15.22:0/647703541 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56417fa8a850 con 0x56417fb3a790 2024-09-19T08:20:12.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.806+0000 7f17abfff640 1 --2- 172.21.15.22:0/647703541 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56417fb3a790 0x56417fb3cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.806+0000 7f178bfff640 10 client.16458 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:12.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.806+0000 7f17abfff640 1 --2- 172.21.15.22:0/647703541 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56417fb3a790 0x56417fb3cb80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce5c75f6f73ec96 server_cookie=b318a6a56269a13d in_seq=0 out_seq=0 2024-09-19T08:20:12.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.807+0000 7f178bfff640 1 -- 172.21.15.22:0/647703541 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56417fa8a850 con 0x56417fb3a790 2024-09-19T08:20:12.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.807+0000 7f178bfff640 10 client.16458 handle_command_reply: tid=0 2024-09-19T08:20:12.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.807+0000 7f17b5509740 1 client.16458 shutdown 2024-09-19T08:20:12.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.807+0000 7f17b5509740 1 -- 172.21.15.22:0/647703541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 msgr2=0x56417fb2a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:12.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.807+0000 7f17b5509740 1 --2- 172.21.15.22:0/647703541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb2a890 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f17a009c150 tx=0x7f17a0016980 comp rx=0 tx=0).stop 2024-09-19T08:20:12.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.807+0000 7f17b5509740 1 -- 172.21.15.22:0/647703541 shutdown_connections 2024-09-19T08:20:12.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.807+0000 7f17b5509740 1 --2- 172.21.15.22:0/647703541 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56417fb3a790 0x56417fb3cb80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.807+0000 7f17b5509740 1 --2- 172.21.15.22:0/647703541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb28130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.807+0000 7f17b5509740 1 --2- 172.21.15.22:0/647703541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 0x56417fb27b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.807+0000 7f178bfff640 0 client.16458 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:12.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.807+0000 7f17b5509740 1 --2- 172.21.15.22:0/647703541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb2a890 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.807+0000 7f17b5509740 1 -- 172.21.15.22:0/647703541 >> 172.21.15.22:0/647703541 conn(0x56417fb13a30 msgr2=0x56417fb196c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:12.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.808+0000 7f17b5509740 1 -- 172.21.15.22:0/647703541 shutdown_connections 2024-09-19T08:20:12.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.808+0000 7f17b5509740 1 -- 172.21.15.22:0/647703541 wait complete. 2024-09-19T08:20:12.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.808+0000 7f17b5509740 20 client.16458 trim_cache size 0 max 16384 2024-09-19T08:20:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b5509740 1 Processor -- start 2024-09-19T08:20:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b5509740 1 -- start start 2024-09-19T08:20:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b5509740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb228f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b5509740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 0x56417fb3fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b5509740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb400e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b5509740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1adf80 con 0x56417fb173c0 2024-09-19T08:20:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b5509740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1ab820 con 0x56417fb2ae90 2024-09-19T08:20:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b5509740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1a9690 con 0x56417fb17ce0 2024-09-19T08:20:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b0ecc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb228f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b0ecc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb228f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57080/0 (socket says 172.21.15.22:57080) 2024-09-19T08:20:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b0ecc640 1 -- 172.21.15.22:0/2064375447 learned_addr learned my addr 172.21.15.22:0/2064375447 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b16cd640 1 --2- 172.21.15.22:0/2064375447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17abfff640 1 --2- 172.21.15.22:0/2064375447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 0x56417fb3fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b16cd640 1 -- 172.21.15.22:0/2064375447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 msgr2=0x56417fb3fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b16cd640 1 --2- 172.21.15.22:0/2064375447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 0x56417fb3fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b16cd640 1 -- 172.21.15.22:0/2064375447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 msgr2=0x56417fb228f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b16cd640 1 --2- 172.21.15.22:0/2064375447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb228f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b16cd640 1 -- 172.21.15.22:0/2064375447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56417fb2abc0 con 0x56417fb2ae90 2024-09-19T08:20:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b0ecc640 1 --2- 172.21.15.22:0/2064375447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb228f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:12.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.826+0000 7f17b16cd640 1 --2- 172.21.15.22:0/2064375447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb400e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f17a4004af0 tx=0x7f17a4045000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eb18b9679c22734 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:12.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.827+0000 7f17967fc640 1 -- 172.21.15.22:0/2064375447 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4043050 con 0x56417fb2ae90 2024-09-19T08:20:12.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.827+0000 7f17967fc640 1 -- 172.21.15.22:0/2064375447 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a4044040 con 0x56417fb2ae90 2024-09-19T08:20:12.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.827+0000 7f17967fc640 1 -- 172.21.15.22:0/2064375447 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4034040 con 0x56417fb2ae90 2024-09-19T08:20:12.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.827+0000 7f17b5509740 1 -- 172.21.15.22:0/2064375447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 msgr2=0x56417fb400e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.827+0000 7f17b5509740 1 --2- 172.21.15.22:0/2064375447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb400e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f17a4004af0 tx=0x7f17a4045000 comp rx=0 tx=0).stop 2024-09-19T08:20:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.827+0000 7f17b5509740 1 -- 172.21.15.22:0/2064375447 shutdown_connections 2024-09-19T08:20:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.827+0000 7f17b5509740 1 --2- 172.21.15.22:0/2064375447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb400e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.827+0000 7f17b5509740 1 --2- 172.21.15.22:0/2064375447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb17ce0 0x56417fb3fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.827+0000 7f17b5509740 1 --2- 172.21.15.22:0/2064375447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb173c0 0x56417fb228f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.827+0000 7f17b5509740 1 -- 172.21.15.22:0/2064375447 >> 172.21.15.22:0/2064375447 conn(0x56417fb13a30 msgr2=0x56417fb26b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.828+0000 7f17b5509740 1 -- 172.21.15.22:0/2064375447 shutdown_connections 2024-09-19T08:20:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.828+0000 7f17b5509740 1 -- 172.21.15.22:0/2064375447 wait complete. 2024-09-19T08:20:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.828+0000 7f17b5509740 1 -- start start 2024-09-19T08:20:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.828+0000 7f17b5509740 1 Processor -- start 2024-09-19T08:20:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.828+0000 7f17b5509740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb173c0 0x56417fb3db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.828+0000 7f17b5509740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb17ce0 0x56417fb3e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.828+0000 7f17b5509740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb3e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b5509740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1a9ce0 con 0x56417fb17ce0 2024-09-19T08:20:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b5509740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1ab150 con 0x56417fb2ae90 2024-09-19T08:20:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b5509740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1aa3b0 con 0x56417fb173c0 2024-09-19T08:20:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b5509740 10 client.0 _ll_register_callbacks cb 0x7f1798005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b0ecc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb173c0 0x56417fb3db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b16cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb3e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17abfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb17ce0 0x56417fb3e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b0ecc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb173c0 0x56417fb3db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:48480/0 (socket says 172.21.15.22:48480) 2024-09-19T08:20:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b0ecc640 1 -- 172.21.15.22:0/638685115 learned_addr learned my addr 172.21.15.22:0/638685115 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b16cd640 1 -- 172.21.15.22:0/638685115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb173c0 msgr2=0x56417fb3db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b16cd640 1 --2- 172.21.15.22:0/638685115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb173c0 0x56417fb3db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b16cd640 1 -- 172.21.15.22:0/638685115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb17ce0 msgr2=0x56417fb3e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b16cd640 1 --2- 172.21.15.22:0/638685115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb17ce0 0x56417fb3e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b16cd640 1 -- 172.21.15.22:0/638685115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56417fba42d0 con 0x56417fb2ae90 2024-09-19T08:20:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b16cd640 1 --2- 172.21.15.22:0/638685115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb3e580 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f17a402ae70 tx=0x7f17a404c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88e8f8ec4abb1032 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:12.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f178bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:20:12.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f178bfff640 1 -- 172.21.15.22:0/638685115 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4043040 con 0x56417fb2ae90 2024-09-19T08:20:12.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f178bfff640 1 -- 172.21.15.22:0/638685115 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f17a4003f80 con 0x56417fb2ae90 2024-09-19T08:20:12.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.829+0000 7f17b5509740 1 -- 172.21.15.22:0/638685115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a4044040 con 0x56417fb2ae90 2024-09-19T08:20:12.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.830+0000 7f178bfff640 1 -- 172.21.15.22:0/638685115 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17a4004180 con 0x56417fb2ae90 2024-09-19T08:20:12.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.830+0000 7f17b5509740 10 client.26033 fetch_fsmap learned FSMap version 30 2024-09-19T08:20:12.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.830+0000 7f17b5509740 1 -- 172.21.15.22:0/638685115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56417fba5580 con 0x56417fb2ae90 2024-09-19T08:20:12.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.830+0000 7f178bfff640 1 -- 172.21.15.22:0/638685115 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a4044040 con 0x56417fb2ae90 2024-09-19T08:20:12.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.830+0000 7f178bfff640 1 -- 172.21.15.22:0/638685115 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f17a4066800 con 0x56417fb2ae90 2024-09-19T08:20:12.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.830+0000 7f17b5509740 10 client.26033 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:20:12.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.830+0000 7f17b5509740 10 client.26033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:20:12.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.830+0000 7f17b5509740 20 client.26033 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:12.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.830+0000 7f17b5509740 1 --2- 172.21.15.22:0/638685115 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56417fba5c40 0x56417fba8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:12.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.830+0000 7f17b5509740 4 client.26033 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:20:12.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.830+0000 7f17b5509740 1 -- 172.21.15.22:0/638685115 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56417fa84d60 con 0x56417fba5c40 2024-09-19T08:20:12.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.830+0000 7f17b0ecc640 1 --2- 172.21.15.22:0/638685115 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56417fba5c40 0x56417fba8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:12.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.831+0000 7f17b0ecc640 1 --2- 172.21.15.22:0/638685115 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56417fba5c40 0x56417fba8030 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0a0feeff10392a server_cookie=da5d23bdc839ff66 in_seq=0 out_seq=0 2024-09-19T08:20:12.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.831+0000 7f178bfff640 10 client.26033 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:12.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.831+0000 7f178bfff640 1 -- 172.21.15.22:0/638685115 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56417fa84d60 con 0x56417fba5c40 2024-09-19T08:20:12.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.831+0000 7f178bfff640 10 client.26033 handle_command_reply: tid=0 2024-09-19T08:20:12.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.831+0000 7f17b5509740 1 client.26033 shutdown 2024-09-19T08:20:12.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.831+0000 7f17b5509740 1 -- 172.21.15.22:0/638685115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 msgr2=0x56417fb3e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:12.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.831+0000 7f17b5509740 1 --2- 172.21.15.22:0/638685115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb3e580 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f17a402ae70 tx=0x7f17a404c960 comp rx=0 tx=0).stop 2024-09-19T08:20:12.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.831+0000 7f17b5509740 1 -- 172.21.15.22:0/638685115 shutdown_connections 2024-09-19T08:20:12.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.831+0000 7f17b5509740 1 --2- 172.21.15.22:0/638685115 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56417fba5c40 0x56417fba8030 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.831+0000 7f17b5509740 1 --2- 172.21.15.22:0/638685115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417fb2ae90 0x56417fb3e580 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.831+0000 7f17b5509740 1 --2- 172.21.15.22:0/638685115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56417fb17ce0 0x56417fb3e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.831+0000 7f178bfff640 0 client.26033 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:12.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.831+0000 7f17b5509740 1 --2- 172.21.15.22:0/638685115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56417fb173c0 0x56417fb3db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.831+0000 7f17b5509740 1 -- 172.21.15.22:0/638685115 >> 172.21.15.22:0/638685115 conn(0x56417fb13a30 msgr2=0x56417fb262e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:12.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.832+0000 7f17b5509740 1 -- 172.21.15.22:0/638685115 shutdown_connections 2024-09-19T08:20:12.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.832+0000 7f17b5509740 1 -- 172.21.15.22:0/638685115 wait complete. 2024-09-19T08:20:12.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.832+0000 7f17b5509740 20 client.26033 trim_cache size 0 max 16384 2024-09-19T08:20:12.836 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:20:12.836 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:20:12.836 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-19T08:20:12.836 INFO:teuthology.orchestra.run.smithi022.stdout: "fb8b065c-674f-4534-96ac-744d3fb32f90": { 2024-09-19T08:20:12.836 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-19T08:20:12.836 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "fb8b065c-674f-4534-96ac-744d3fb32f90", 2024-09-19T08:20:12.836 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-09-19T08:20:12.836 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:20:12.837 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:20:12.837 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:20:12.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.832+0000 7f17b3130640 1 -- 172.21.15.22:0/534634481 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f179c0a33b0 msgr2=0x7f179c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:12.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.832+0000 7f17b3130640 1 --2- 172.21.15.22:0/534634481 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f179c0a33b0 0x7f179c0a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f17a402f610 tx=0x7f17a402a950 comp rx=0 tx=0).stop 2024-09-19T08:20:12.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.832+0000 7f17b3130640 1 -- 172.21.15.22:0/534634481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17ac102d50 msgr2=0x7f17ac1009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:12.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.832+0000 7f17b3130640 1 --2- 172.21.15.22:0/534634481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17ac102d50 0x7f17ac1009b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f17a000e9f0 tx=0x7f17a000eeb0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.832+0000 7f17b3130640 1 -- 172.21.15.22:0/534634481 shutdown_connections 2024-09-19T08:20:12.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.832+0000 7f17b3130640 1 --2- 172.21.15.22:0/534634481 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f179c0a33b0 0x7f179c0a5870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.832+0000 7f17b3130640 1 --2- 172.21.15.22:0/534634481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17ac10c180 0x7f17ac101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.832+0000 7f17b3130640 1 --2- 172.21.15.22:0/534634481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17ac103670 0x7f17ac100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.832+0000 7f17b3130640 1 --2- 172.21.15.22:0/534634481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17ac102d50 0x7f17ac1009b0 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f17a000e9f0 tx=0x7f17a000eeb0 comp rx=0 tx=0).stop 2024-09-19T08:20:12.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.832+0000 7f17b3130640 1 -- 172.21.15.22:0/534634481 >> 172.21.15.22:0/534634481 conn(0x7f17ac0fc7d0 msgr2=0x7f17ac0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:12.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.833+0000 7f17b3130640 1 -- 172.21.15.22:0/534634481 shutdown_connections 2024-09-19T08:20:12.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:12.833+0000 7f17b3130640 1 -- 172.21.15.22:0/534634481 wait complete. 2024-09-19T08:20:13.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:12 smithi022 ceph-mon[30963]: scrub queued for path: / 2024-09-19T08:20:13.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:12 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:13.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:12 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:20:13.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:12 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:20:13.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:12 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:13.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:12 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:20:13.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:12 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:13.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:12 smithi022 ceph-mon[30963]: pgmap v158: 97 pgs: 97 active+clean; 616 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 513 KiB/s wr, 103 op/s 2024-09-19T08:20:13.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:12 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:20:13.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:12 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:13.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:12 smithi064 ceph-mon[34900]: scrub queued for path: / 2024-09-19T08:20:13.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:12 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:13.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:12 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:20:13.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:12 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:20:13.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:12 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:13.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:12 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:20:13.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:12 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:13.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:12 smithi064 ceph-mon[34900]: pgmap v158: 97 pgs: 97 active+clean; 616 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 513 KiB/s wr, 103 op/s 2024-09-19T08:20:13.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:12 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:20:13.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:12 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:12 smithi157 ceph-mon[35143]: scrub queued for path: / 2024-09-19T08:20:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:12 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:12 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:20:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:12 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:20:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:12 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:12 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:20:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:12 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:12 smithi157 ceph-mon[35143]: pgmap v158: 97 pgs: 97 active+clean; 616 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 513 KiB/s wr, 103 op/s 2024-09-19T08:20:13.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:12 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:20:13.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:12 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:13.619 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2af777fa', quiesce took 14.5 seconds. Will release after: 15.3 2024-09-19T08:20:14.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:13 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:20:14.062 INFO:tasks.cephfs.filesystem:scrub status for tag:fb8b065c-674f-4534-96ac-744d3fb32f90 - {'path': '/', 'tag': 'fb8b065c-674f-4534-96ac-744d3fb32f90', 'options': 'recursive,force'} 2024-09-19T08:20:14.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:13 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:20:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:13 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:20:15.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:14 smithi064 ceph-mon[34900]: pgmap v159: 97 pgs: 97 active+clean; 626 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-09-19T08:20:15.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:14 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:20:15.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:14 smithi157 ceph-mon[35143]: pgmap v159: 97 pgs: 97 active+clean; 626 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-09-19T08:20:15.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:14 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:20:15.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:14 smithi022 ceph-mon[30963]: pgmap v159: 97 pgs: 97 active+clean; 626 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-09-19T08:20:15.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:14 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:20:16.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:15 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:20:16.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:15 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:20:16.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:15 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:20:16.155 INFO:tasks.workunit.client.0.smithi022.stdout: 109 3763 0 2938 0 1676 0 2024-09-19T08:20:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:15 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:20:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:15 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:20:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:15 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:20:16.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:15 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:20:16.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:15 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:20:16.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:15 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:20:17.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:16 smithi064 ceph-mon[34900]: pgmap v160: 97 pgs: 97 active+clean; 626 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-19T08:20:17.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:16 smithi022 ceph-mon[30963]: pgmap v160: 97 pgs: 97 active+clean; 626 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-19T08:20:17.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:16 smithi157 ceph-mon[35143]: pgmap v160: 97 pgs: 97 active+clean; 626 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-19T08:20:19.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:18 smithi064 ceph-mon[34900]: pgmap v161: 97 pgs: 97 active+clean; 626 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 284 op/s 2024-09-19T08:20:19.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:18 smithi157 ceph-mon[35143]: pgmap v161: 97 pgs: 97 active+clean; 626 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 284 op/s 2024-09-19T08:20:19.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:18 smithi022 ceph-mon[30963]: pgmap v161: 97 pgs: 97 active+clean; 626 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 284 op/s 2024-09-19T08:20:21.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:20 smithi157 ceph-mon[35143]: pgmap v162: 97 pgs: 97 active+clean; 628 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 629 op/s 2024-09-19T08:20:21.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:20 smithi064 ceph-mon[34900]: pgmap v162: 97 pgs: 97 active+clean; 628 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 629 op/s 2024-09-19T08:20:21.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:20 smithi022 ceph-mon[30963]: pgmap v162: 97 pgs: 97 active+clean; 628 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 629 op/s 2024-09-19T08:20:23.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:22 smithi064 ceph-mon[34900]: pgmap v163: 97 pgs: 97 active+clean; 628 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.1 MiB/s wr, 582 op/s 2024-09-19T08:20:23.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:22 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:23.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:22 smithi157 ceph-mon[35143]: pgmap v163: 97 pgs: 97 active+clean; 628 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.1 MiB/s wr, 582 op/s 2024-09-19T08:20:23.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:22 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:23.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:22 smithi022 ceph-mon[30963]: pgmap v163: 97 pgs: 97 active+clean; 628 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.1 MiB/s wr, 582 op/s 2024-09-19T08:20:23.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:22 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:20:25.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:24 smithi064 ceph-mon[34900]: pgmap v164: 97 pgs: 97 active+clean; 607 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.3 MiB/s wr, 966 op/s 2024-09-19T08:20:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:24 smithi157 ceph-mon[35143]: pgmap v164: 97 pgs: 97 active+clean; 607 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.3 MiB/s wr, 966 op/s 2024-09-19T08:20:25.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:24 smithi022 ceph-mon[30963]: pgmap v164: 97 pgs: 97 active+clean; 607 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.3 MiB/s wr, 966 op/s 2024-09-19T08:20:26.155 INFO:tasks.workunit.client.0.smithi022.stdout: 109 69930 0 44858 0 36947 0 2024-09-19T08:20:27.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:26 smithi064 ceph-mon[34900]: pgmap v165: 97 pgs: 97 active+clean; 607 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 431 KiB/s wr, 845 op/s 2024-09-19T08:20:27.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:26 smithi157 ceph-mon[35143]: pgmap v165: 97 pgs: 97 active+clean; 607 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 431 KiB/s wr, 845 op/s 2024-09-19T08:20:27.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:26 smithi022 ceph-mon[30963]: pgmap v165: 97 pgs: 97 active+clean; 607 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 431 KiB/s wr, 845 op/s 2024-09-19T08:20:28.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:27 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:20:28.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:27 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:20:28.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:27 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:20:28.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:27 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:20:28.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:27 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:20:28.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:27 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:20:28.920 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:20:29.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:28 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:20:29.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:28 smithi064 ceph-mon[34900]: pgmap v166: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 598 KiB/s wr, 887 op/s 2024-09-19T08:20:29.198 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:29.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:28 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:20:29.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:28 smithi157 ceph-mon[35143]: pgmap v166: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 598 KiB/s wr, 887 op/s 2024-09-19T08:20:29.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:28 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:20:29.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:28 smithi022 ceph-mon[30963]: pgmap v166: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 598 KiB/s wr, 887 op/s 2024-09-19T08:20:29.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.754+0000 7f161bc47640 1 -- 172.21.15.22:0/866655053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161410e490 msgr2=0x7f1614110880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:29.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.754+0000 7f161bc47640 1 --2- 172.21.15.22:0/866655053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161410e490 0x7f1614110880 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f160c009ca0 tx=0x7f160c02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:20:29.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.754+0000 7f161bc47640 1 -- 172.21.15.22:0/866655053 shutdown_connections 2024-09-19T08:20:29.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.754+0000 7f161bc47640 1 --2- 172.21.15.22:0/866655053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161410e490 0x7f1614110880 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:29.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.754+0000 7f161bc47640 1 --2- 172.21.15.22:0/866655053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1614101290 0x7f161410df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:29.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.754+0000 7f161bc47640 1 --2- 172.21.15.22:0/866655053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16141008e0 0x7f1614100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:29.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.754+0000 7f161bc47640 1 -- 172.21.15.22:0/866655053 >> 172.21.15.22:0/866655053 conn(0x7f16140fc7d0 msgr2=0x7f16140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:29.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.754+0000 7f161bc47640 1 -- 172.21.15.22:0/866655053 shutdown_connections 2024-09-19T08:20:29.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.754+0000 7f161bc47640 1 -- 172.21.15.22:0/866655053 wait complete. 2024-09-19T08:20:29.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.755+0000 7f161bc47640 1 Processor -- start 2024-09-19T08:20:29.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.755+0000 7f161bc47640 1 -- start start 2024-09-19T08:20:29.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.755+0000 7f161bc47640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16141008e0 0x7f161419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:29.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.755+0000 7f161bc47640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1614101290 0x7f161419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:29.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.755+0000 7f161bc47640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161410e490 0x7f161419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:29.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.755+0000 7f161bc47640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1614114190 con 0x7f161410e490 2024-09-19T08:20:29.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.755+0000 7f161bc47640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1614114010 con 0x7f16141008e0 2024-09-19T08:20:29.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.755+0000 7f161bc47640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1614114310 con 0x7f1614101290 2024-09-19T08:20:29.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.755+0000 7f161a1e4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161410e490 0x7f161419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:29.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.755+0000 7f161a1e4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161410e490 0x7f161419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45974/0 (socket says 172.21.15.22:45974) 2024-09-19T08:20:29.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.755+0000 7f161a1e4640 1 -- 172.21.15.22:0/2191263686 learned_addr learned my addr 172.21.15.22:0/2191263686 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:29.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.755+0000 7f16191e2640 1 --2- 172.21.15.22:0/2191263686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1614101290 0x7f161419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:29.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.756+0000 7f16199e3640 1 --2- 172.21.15.22:0/2191263686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16141008e0 0x7f161419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:29.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.756+0000 7f161a1e4640 1 -- 172.21.15.22:0/2191263686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1614101290 msgr2=0x7f161419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.756+0000 7f161a1e4640 1 --2- 172.21.15.22:0/2191263686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1614101290 0x7f161419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.756+0000 7f161a1e4640 1 -- 172.21.15.22:0/2191263686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16141008e0 msgr2=0x7f161419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.756+0000 7f161a1e4640 1 --2- 172.21.15.22:0/2191263686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16141008e0 0x7f161419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.756+0000 7f161a1e4640 1 -- 172.21.15.22:0/2191263686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161419f600 con 0x7f161410e490 2024-09-19T08:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.756+0000 7f161a1e4640 1 --2- 172.21.15.22:0/2191263686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161410e490 0x7f161419ef20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f160c02ade0 tx=0x7f160c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.756+0000 7f160affd640 1 -- 172.21.15.22:0/2191263686 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c003c30 con 0x7f161410e490 2024-09-19T08:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.756+0000 7f161bc47640 1 -- 172.21.15.22:0/2191263686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161419f890 con 0x7f161410e490 2024-09-19T08:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.756+0000 7f160affd640 1 -- 172.21.15.22:0/2191263686 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f160c003dd0 con 0x7f161410e490 2024-09-19T08:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.756+0000 7f160affd640 1 -- 172.21.15.22:0/2191263686 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c02fd60 con 0x7f161410e490 2024-09-19T08:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.756+0000 7f161bc47640 1 -- 172.21.15.22:0/2191263686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16141a71d0 con 0x7f161410e490 2024-09-19T08:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.757+0000 7f161bc47640 1 -- 172.21.15.22:0/2191263686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1614100cc0 con 0x7f161410e490 2024-09-19T08:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.758+0000 7f160affd640 1 -- 172.21.15.22:0/2191263686 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f160c004050 con 0x7f161410e490 2024-09-19T08:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.759+0000 7f160affd640 1 --2- 172.21.15.22:0/2191263686 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16000a33b0 0x7f16000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:29.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.759+0000 7f16199e3640 1 --2- 172.21.15.22:0/2191263686 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16000a33b0 0x7f16000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:29.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.759+0000 7f160affd640 1 -- 172.21.15.22:0/2191263686 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f160c0f10c0 con 0x7f161410e490 2024-09-19T08:20:29.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.759+0000 7f16199e3640 1 --2- 172.21.15.22:0/2191263686 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16000a33b0 0x7f16000a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1610005e10 tx=0x7f1610005da0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:29.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.763+0000 7f160affd640 1 -- 172.21.15.22:0/2191263686 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f160c0bdc50 con 0x7f161410e490 2024-09-19T08:20:29.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.976+0000 7f161bc47640 1 -- 172.21.15.22:0/2191263686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16141a7400 con 0x7f161410e490 2024-09-19T08:20:29.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.977+0000 7f160affd640 1 -- 172.21.15.22:0/2191263686 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11907 (secure 0 0 0) 0x7f160c0bd550 con 0x7f161410e490 2024-09-19T08:20:29.978 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:20:29.978 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":30,"btime":"2024-09-19T08:20:12:788245+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:20:11.821718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:20:29.978 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 30 2024-09-19T08:20:29.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.980+0000 7f161bc47640 1 -- 172.21.15.22:0/2191263686 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16000a33b0 msgr2=0x7f16000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:29.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.980+0000 7f161bc47640 1 --2- 172.21.15.22:0/2191263686 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16000a33b0 0x7f16000a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1610005e10 tx=0x7f1610005da0 comp rx=0 tx=0).stop 2024-09-19T08:20:29.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.980+0000 7f161bc47640 1 -- 172.21.15.22:0/2191263686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161410e490 msgr2=0x7f161419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:29.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.980+0000 7f161bc47640 1 --2- 172.21.15.22:0/2191263686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161410e490 0x7f161419ef20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f160c02ade0 tx=0x7f160c0319f0 comp rx=0 tx=0).stop 2024-09-19T08:20:29.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.980+0000 7f161bc47640 1 -- 172.21.15.22:0/2191263686 shutdown_connections 2024-09-19T08:20:29.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.980+0000 7f161bc47640 1 --2- 172.21.15.22:0/2191263686 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16000a33b0 0x7f16000a5870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:29.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.980+0000 7f161bc47640 1 --2- 172.21.15.22:0/2191263686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f161410e490 0x7f161419ef20 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:29.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.980+0000 7f161bc47640 1 --2- 172.21.15.22:0/2191263686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1614101290 0x7f161419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:29.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.980+0000 7f161bc47640 1 --2- 172.21.15.22:0/2191263686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16141008e0 0x7f161419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:29.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.980+0000 7f161bc47640 1 -- 172.21.15.22:0/2191263686 >> 172.21.15.22:0/2191263686 conn(0x7f16140fc7d0 msgr2=0x7f16140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:29.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.980+0000 7f161bc47640 1 -- 172.21.15.22:0/2191263686 shutdown_connections 2024-09-19T08:20:29.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:29.980+0000 7f161bc47640 1 -- 172.21.15.22:0/2191263686 wait complete. 2024-09-19T08:20:30.945 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id 2af777fa --release --await' 2024-09-19T08:20:30.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id 2af777fa --release --await 2024-09-19T08:20:31.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:30 smithi064 ceph-mon[34900]: pgmap v167: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 647 KiB/s wr, 918 op/s 2024-09-19T08:20:31.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:30 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2191263686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:20:31.209 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:31.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:30 smithi157 ceph-mon[35143]: pgmap v167: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 647 KiB/s wr, 918 op/s 2024-09-19T08:20:31.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:30 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2191263686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:20:31.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:30 smithi022 ceph-mon[30963]: pgmap v167: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 647 KiB/s wr, 918 op/s 2024-09-19T08:20:31.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:30 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2191263686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:20:31.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.712+0000 7fdc91699640 1 -- 172.21.15.22:0/282799653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc8c105ad0 msgr2=0x7fdc8c105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:31.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.712+0000 7fdc91699640 1 --2- 172.21.15.22:0/282799653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc8c105ad0 0x7fdc8c105eb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009900 tx=0x7fdc7c02a410 comp rx=0 tx=0).stop 2024-09-19T08:20:31.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.712+0000 7fdc91699640 1 -- 172.21.15.22:0/282799653 shutdown_connections 2024-09-19T08:20:31.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.712+0000 7fdc91699640 1 --2- 172.21.15.22:0/282799653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc8c101b60 0x7fdc8c10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.712+0000 7fdc91699640 1 --2- 172.21.15.22:0/282799653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc8c1011c0 0x7fdc8c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.712+0000 7fdc91699640 1 --2- 172.21.15.22:0/282799653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc8c105ad0 0x7fdc8c105eb0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.712+0000 7fdc91699640 1 -- 172.21.15.22:0/282799653 >> 172.21.15.22:0/282799653 conn(0x7fdc8c077210 msgr2=0x7fdc8c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:31.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.712+0000 7fdc91699640 1 -- 172.21.15.22:0/282799653 shutdown_connections 2024-09-19T08:20:31.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.712+0000 7fdc91699640 1 -- 172.21.15.22:0/282799653 wait complete. 2024-09-19T08:20:31.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.713+0000 7fdc91699640 1 Processor -- start 2024-09-19T08:20:31.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.713+0000 7fdc91699640 1 -- start start 2024-09-19T08:20:31.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.713+0000 7fdc91699640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc8c1011c0 0x7fdc8c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.713+0000 7fdc91699640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc8c101b60 0x7fdc8c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.713+0000 7fdc91699640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc8c105ad0 0x7fdc8c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.713+0000 7fdc91699640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c110ef0 con 0x7fdc8c1011c0 2024-09-19T08:20:31.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.713+0000 7fdc91699640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c110d70 con 0x7fdc8c105ad0 2024-09-19T08:20:31.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.713+0000 7fdc91699640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c111070 con 0x7fdc8c101b60 2024-09-19T08:20:31.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.713+0000 7fdc8affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc8c1011c0 0x7fdc8c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.713+0000 7fdc8affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc8c1011c0 0x7fdc8c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46004/0 (socket says 172.21.15.22:46004) 2024-09-19T08:20:31.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.713+0000 7fdc8affd640 1 -- 172.21.15.22:0/1445957975 learned_addr learned my addr 172.21.15.22:0/1445957975 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:31.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.713+0000 7fdc8a7fc640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc8c101b60 0x7fdc8c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.714+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc8c105ad0 0x7fdc8c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.714+0000 7fdc8affd640 1 -- 172.21.15.22:0/1445957975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc8c101b60 msgr2=0x7fdc8c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:31.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.714+0000 7fdc8affd640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc8c101b60 0x7fdc8c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.714+0000 7fdc8affd640 1 -- 172.21.15.22:0/1445957975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc8c105ad0 msgr2=0x7fdc8c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:31.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.714+0000 7fdc8affd640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc8c105ad0 0x7fdc8c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.714+0000 7fdc8affd640 1 -- 172.21.15.22:0/1445957975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c19d330 con 0x7fdc8c1011c0 2024-09-19T08:20:31.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.714+0000 7fdc8a7fc640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc8c101b60 0x7fdc8c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:20:31.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.714+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc8c105ad0 0x7fdc8c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:20:31.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.714+0000 7fdc8affd640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc8c1011c0 0x7fdc8c198380 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c006010 tx=0x7fdc7c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:31.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.714+0000 7fdc73fff640 1 -- 172.21.15.22:0/1445957975 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c03f070 con 0x7fdc8c1011c0 2024-09-19T08:20:31.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.714+0000 7fdc73fff640 1 -- 172.21.15.22:0/1445957975 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc7c031c30 con 0x7fdc8c1011c0 2024-09-19T08:20:31.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.714+0000 7fdc91699640 1 -- 172.21.15.22:0/1445957975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8c19d5c0 con 0x7fdc8c1011c0 2024-09-19T08:20:31.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.714+0000 7fdc73fff640 1 -- 172.21.15.22:0/1445957975 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c035030 con 0x7fdc8c1011c0 2024-09-19T08:20:31.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.714+0000 7fdc91699640 1 -- 172.21.15.22:0/1445957975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8c1a4f00 con 0x7fdc8c1011c0 2024-09-19T08:20:31.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.716+0000 7fdc73fff640 1 -- 172.21.15.22:0/1445957975 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdc7c042070 con 0x7fdc8c1011c0 2024-09-19T08:20:31.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc93a72740 1 Processor -- start 2024-09-19T08:20:31.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc93a72740 1 -- start start 2024-09-19T08:20:31.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc93a72740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41470 0x557350a41850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc73fff640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdc740a33b0 0x7fdc740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc93a72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41d90 0x557350a4c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc93a72740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a4c650 0x557350a4ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc93a72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5573509a2620 con 0x557350a41d90 2024-09-19T08:20:31.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc93a72740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557350949e50 con 0x557350a4c650 2024-09-19T08:20:31.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc93a72740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55735099e5e0 con 0x557350a41470 2024-09-19T08:20:31.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc8a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41d90 0x557350a4c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc8a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41d90 0x557350a4c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46016/0 (socket says 172.21.15.22:46016) 2024-09-19T08:20:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc8a7fc640 1 -- 172.21.15.22:0/1567709397 learned_addr learned my addr 172.21.15.22:0/1567709397 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc73fff640 1 -- 172.21.15.22:0/1445957975 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fdc7c0f0680 con 0x7fdc8c1011c0 2024-09-19T08:20:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc8affd640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdc740a33b0 0x7fdc740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/1567709397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a4c650 0x557350a4ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/1567709397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41470 0x557350a41850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc8a7fc640 1 -- 172.21.15.22:0/1567709397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41470 msgr2=0x557350a41850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.717+0000 7fdc8a7fc640 1 --2- 172.21.15.22:0/1567709397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41470 0x557350a41850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.718+0000 7fdc8a7fc640 1 -- 172.21.15.22:0/1567709397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a4c650 msgr2=0x557350a4ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.718+0000 7fdc8a7fc640 1 --2- 172.21.15.22:0/1567709397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a4c650 0x557350a4ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.718+0000 7fdc8a7fc640 1 -- 172.21.15.22:0/1567709397 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735093ad40 con 0x557350a41d90 2024-09-19T08:20:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.718+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/1567709397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a4c650 0x557350a4ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:20:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.718+0000 7fdc8a7fc640 1 --2- 172.21.15.22:0/1567709397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41d90 0x557350a4c110 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000bd30 tx=0x7fdc8000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e3fd74669a9fdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.718+0000 7fdc8affd640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdc740a33b0 0x7fdc740a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c005ab0 tx=0x7fdc7c02a410 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.718+0000 7fdc717fa640 1 -- 172.21.15.22:0/1567709397 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80005220 con 0x557350a41d90 2024-09-19T08:20:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.718+0000 7fdc717fa640 1 -- 172.21.15.22:0/1567709397 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc8000be20 con 0x557350a41d90 2024-09-19T08:20:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.718+0000 7fdc717fa640 1 -- 172.21.15.22:0/1567709397 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc800117c0 con 0x557350a41d90 2024-09-19T08:20:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 -- 172.21.15.22:0/1567709397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41d90 msgr2=0x557350a4c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 --2- 172.21.15.22:0/1567709397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41d90 0x557350a4c110 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000bd30 tx=0x7fdc8000f730 comp rx=0 tx=0).stop 2024-09-19T08:20:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 -- 172.21.15.22:0/1567709397 shutdown_connections 2024-09-19T08:20:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 --2- 172.21.15.22:0/1567709397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a4c650 0x557350a4ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 --2- 172.21.15.22:0/1567709397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41d90 0x557350a4c110 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 --2- 172.21.15.22:0/1567709397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41470 0x557350a41850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 -- 172.21.15.22:0/1567709397 >> 172.21.15.22:0/1567709397 conn(0x557350a3da00 msgr2=0x557350a3dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:31.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 -- 172.21.15.22:0/1567709397 shutdown_connections 2024-09-19T08:20:31.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 -- 172.21.15.22:0/1567709397 wait complete. 2024-09-19T08:20:31.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 -- start start 2024-09-19T08:20:31.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 Processor -- start 2024-09-19T08:20:31.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a41470 0x557350a49530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41d90 0x557350a49a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a4c650 0x557350a49fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557350a4f220 con 0x557350a4c650 2024-09-19T08:20:31.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55735092af10 con 0x557350a41470 2024-09-19T08:20:31.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.719+0000 7fdc93a72740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557350a4f3a0 con 0x557350a41d90 2024-09-19T08:20:31.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc93a72740 10 client.0 _ll_register_callbacks cb 0x55735094edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:31.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a4c650 0x557350a49fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a4c650 0x557350a49fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46024/0 (socket says 172.21.15.22:46024) 2024-09-19T08:20:31.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc8b7fe640 1 -- 172.21.15.22:0/966135181 learned_addr learned my addr 172.21.15.22:0/966135181 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:31.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/966135181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a41470 0x557350a49530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc8a7fc640 1 --2- 172.21.15.22:0/966135181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41d90 0x557350a49a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc8b7fe640 1 -- 172.21.15.22:0/966135181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41d90 msgr2=0x557350a49a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:31.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/966135181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41d90 0x557350a49a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc8b7fe640 1 -- 172.21.15.22:0/966135181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a41470 msgr2=0x557350a49530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:31.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/966135181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a41470 0x557350a49530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc8b7fe640 1 -- 172.21.15.22:0/966135181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573506b1580 con 0x557350a4c650 2024-09-19T08:20:31.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/966135181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a4c650 0x557350a49fb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fdc84013ec0 tx=0x7fdc84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e988f48496bd515 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:31.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc6eff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:31.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc6eff5640 1 -- 172.21.15.22:0/966135181 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc84013260 con 0x557350a4c650 2024-09-19T08:20:31.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc6eff5640 1 -- 172.21.15.22:0/966135181 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fdc84004b20 con 0x557350a4c650 2024-09-19T08:20:31.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc93a72740 1 -- 172.21.15.22:0/966135181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc8000be20 con 0x557350a4c650 2024-09-19T08:20:31.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.720+0000 7fdc6eff5640 1 -- 172.21.15.22:0/966135181 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdc84026050 con 0x557350a4c650 2024-09-19T08:20:31.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.721+0000 7fdc93a72740 10 client.16482 fetch_fsmap learned FSMap version 30 2024-09-19T08:20:31.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.721+0000 7fdc93a72740 1 -- 172.21.15.22:0/966135181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55735077b960 con 0x557350a4c650 2024-09-19T08:20:31.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.721+0000 7fdc6eff5640 1 -- 172.21.15.22:0/966135181 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc8000be20 con 0x557350a4c650 2024-09-19T08:20:31.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.721+0000 7fdc6eff5640 1 -- 172.21.15.22:0/966135181 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fdc84021460 con 0x557350a4c650 2024-09-19T08:20:31.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.721+0000 7fdc93a72740 10 client.16482 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:20:31.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.721+0000 7fdc93a72740 10 client.16482 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:20:31.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.721+0000 7fdc93a72740 20 client.16482 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:31.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.721+0000 7fdc93a72740 1 --2- 172.21.15.22:0/966135181 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557350a5c210 0x557350a5e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.721+0000 7fdc93a72740 4 client.16482 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:20:31.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.721+0000 7fdc93a72740 1 -- 172.21.15.22:0/966135181 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573509afb90 con 0x557350a5c210 2024-09-19T08:20:31.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.721+0000 7fdc8a7fc640 1 --2- 172.21.15.22:0/966135181 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557350a5c210 0x557350a5e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.722+0000 7fdc6eff5640 10 client.16482 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:31.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.722+0000 7fdc8a7fc640 1 --2- 172.21.15.22:0/966135181 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557350a5c210 0x557350a5e600 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e86e0c83803a25d server_cookie=825e87228e7c08be in_seq=0 out_seq=0 2024-09-19T08:20:31.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc6eff5640 1 -- 172.21.15.22:0/966135181 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573509afb90 con 0x557350a5c210 2024-09-19T08:20:31.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc6eff5640 10 client.16482 handle_command_reply: tid=0 2024-09-19T08:20:31.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc93a72740 1 client.16482 shutdown 2024-09-19T08:20:31.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc93a72740 1 -- 172.21.15.22:0/966135181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a4c650 msgr2=0x557350a49fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:31.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc93a72740 1 --2- 172.21.15.22:0/966135181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a4c650 0x557350a49fb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fdc84013ec0 tx=0x7fdc84004a40 comp rx=0 tx=0).stop 2024-09-19T08:20:31.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc93a72740 1 -- 172.21.15.22:0/966135181 shutdown_connections 2024-09-19T08:20:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc93a72740 1 --2- 172.21.15.22:0/966135181 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557350a5c210 0x557350a5e600 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc93a72740 1 --2- 172.21.15.22:0/966135181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a4c650 0x557350a49fb0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc93a72740 1 --2- 172.21.15.22:0/966135181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41d90 0x557350a49a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc93a72740 1 --2- 172.21.15.22:0/966135181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a41470 0x557350a49530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc93a72740 1 -- 172.21.15.22:0/966135181 >> 172.21.15.22:0/966135181 conn(0x557350a3da00 msgr2=0x557350a38750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc6eff5640 0 client.16482 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc93a72740 1 -- 172.21.15.22:0/966135181 shutdown_connections 2024-09-19T08:20:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc93a72740 1 -- 172.21.15.22:0/966135181 wait complete. 2024-09-19T08:20:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.723+0000 7fdc93a72740 20 client.16482 trim_cache size 0 max 16384 2024-09-19T08:20:31.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc93a72740 1 Processor -- start 2024-09-19T08:20:31.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc93a72740 1 -- start start 2024-09-19T08:20:31.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc93a72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41470 0x557350a43ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc93a72740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a41d90 0x557350a38640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc93a72740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a4c650 0x557350a63e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc93a72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c1adea0 con 0x557350a41470 2024-09-19T08:20:31.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc93a72740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c1ab740 con 0x557350a41d90 2024-09-19T08:20:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc93a72740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c1a9590 con 0x557350a4c650 2024-09-19T08:20:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41470 0x557350a43ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41470 0x557350a43ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46028/0 (socket says 172.21.15.22:46028) 2024-09-19T08:20:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc8b7fe640 1 -- 172.21.15.22:0/4081491436 learned_addr learned my addr 172.21.15.22:0/4081491436 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc8a7fc640 1 --2- 172.21.15.22:0/4081491436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a41d90 0x557350a38640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/4081491436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a4c650 0x557350a63e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc8b7fe640 1 -- 172.21.15.22:0/4081491436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a4c650 msgr2=0x557350a63e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/4081491436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a4c650 0x557350a63e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc8b7fe640 1 -- 172.21.15.22:0/4081491436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a41d90 msgr2=0x557350a38640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/4081491436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a41d90 0x557350a38640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.741+0000 7fdc8b7fe640 1 -- 172.21.15.22:0/4081491436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557350a5ec60 con 0x557350a41470 2024-09-19T08:20:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.742+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/4081491436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41470 0x557350a43ed0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fdc84005b40 tx=0x7fdc840021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1c4276362f4f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.742+0000 7fdc70ff9640 1 -- 172.21.15.22:0/4081491436 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc84013260 con 0x557350a41470 2024-09-19T08:20:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.742+0000 7fdc70ff9640 1 -- 172.21.15.22:0/4081491436 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc84021a50 con 0x557350a41470 2024-09-19T08:20:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.742+0000 7fdc70ff9640 1 -- 172.21.15.22:0/4081491436 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8402ac40 con 0x557350a41470 2024-09-19T08:20:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.743+0000 7fdc93a72740 1 -- 172.21.15.22:0/4081491436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41470 msgr2=0x557350a43ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.743+0000 7fdc93a72740 1 --2- 172.21.15.22:0/4081491436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41470 0x557350a43ed0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fdc84005b40 tx=0x7fdc840021f0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.743+0000 7fdc93a72740 1 -- 172.21.15.22:0/4081491436 shutdown_connections 2024-09-19T08:20:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.743+0000 7fdc93a72740 1 --2- 172.21.15.22:0/4081491436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a4c650 0x557350a63e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.743+0000 7fdc93a72740 1 --2- 172.21.15.22:0/4081491436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a41d90 0x557350a38640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.743+0000 7fdc93a72740 1 --2- 172.21.15.22:0/4081491436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41470 0x557350a43ed0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.743+0000 7fdc93a72740 1 -- 172.21.15.22:0/4081491436 >> 172.21.15.22:0/4081491436 conn(0x557350a3da00 msgr2=0x557350a37f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.743+0000 7fdc93a72740 1 -- 172.21.15.22:0/4081491436 shutdown_connections 2024-09-19T08:20:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.743+0000 7fdc93a72740 1 -- 172.21.15.22:0/4081491436 wait complete. 2024-09-19T08:20:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.743+0000 7fdc93a72740 1 -- start start 2024-09-19T08:20:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.743+0000 7fdc93a72740 1 Processor -- start 2024-09-19T08:20:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc93a72740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41470 0x557350a505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc93a72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41d90 0x557350a50b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc93a72740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a4c650 0x557350a51070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc93a72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c1a9c00 con 0x557350a41d90 2024-09-19T08:20:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc93a72740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c1ab070 con 0x557350a4c650 2024-09-19T08:20:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc93a72740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c1aa2d0 con 0x557350a41470 2024-09-19T08:20:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc93a72740 10 client.0 _ll_register_callbacks cb 0x7fdc84013d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc8a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41d90 0x557350a50b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc8a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41d90 0x557350a50b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46040/0 (socket says 172.21.15.22:46040) 2024-09-19T08:20:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc8a7fc640 1 -- 172.21.15.22:0/2723509364 learned_addr learned my addr 172.21.15.22:0/2723509364 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/2723509364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a4c650 0x557350a51070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/2723509364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41470 0x557350a505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc8a7fc640 1 -- 172.21.15.22:0/2723509364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41470 msgr2=0x557350a505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc8a7fc640 1 --2- 172.21.15.22:0/2723509364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41470 0x557350a505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc8a7fc640 1 -- 172.21.15.22:0/2723509364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a4c650 msgr2=0x557350a51070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc8a7fc640 1 --2- 172.21.15.22:0/2723509364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a4c650 0x557350a51070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc8a7fc640 1 -- 172.21.15.22:0/2723509364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557350ac6100 con 0x557350a41d90 2024-09-19T08:20:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc8a7fc640 1 --2- 172.21.15.22:0/2723509364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41d90 0x557350a50b30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fdc80005780 tx=0x7fdc80005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaaba1f1a97e2090 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.744+0000 7fdc6eff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.745+0000 7fdc6eff5640 1 -- 172.21.15.22:0/2723509364 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8002c030 con 0x557350a41d90 2024-09-19T08:20:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.745+0000 7fdc6eff5640 1 -- 172.21.15.22:0/2723509364 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fdc80005080 con 0x557350a41d90 2024-09-19T08:20:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.745+0000 7fdc93a72740 1 -- 172.21.15.22:0/2723509364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc84021a50 con 0x557350a41d90 2024-09-19T08:20:31.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.745+0000 7fdc6eff5640 1 -- 172.21.15.22:0/2723509364 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdc80005280 con 0x557350a41d90 2024-09-19T08:20:31.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.745+0000 7fdc93a72740 10 client.16488 fetch_fsmap learned FSMap version 30 2024-09-19T08:20:31.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.745+0000 7fdc93a72740 1 -- 172.21.15.22:0/2723509364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557350ac7420 con 0x557350a41d90 2024-09-19T08:20:31.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.745+0000 7fdc6eff5640 1 -- 172.21.15.22:0/2723509364 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc84021a50 con 0x557350a41d90 2024-09-19T08:20:31.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.745+0000 7fdc6eff5640 1 -- 172.21.15.22:0/2723509364 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fdc8003a870 con 0x557350a41d90 2024-09-19T08:20:31.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.745+0000 7fdc93a72740 10 client.16488 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:20:31.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.745+0000 7fdc93a72740 10 client.16488 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:20:31.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.745+0000 7fdc93a72740 20 client.16488 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:31.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.745+0000 7fdc93a72740 1 --2- 172.21.15.22:0/2723509364 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557350ac7870 0x557350ac9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:31.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.745+0000 7fdc93a72740 4 client.16488 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2af777fa", "release": true, "await": true}] 2024-09-19T08:20:31.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.745+0000 7fdc93a72740 1 -- 172.21.15.22:0/2723509364 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2af777fa", "release": true, "await": true}) -- 0x5573509aa0a0 con 0x557350ac7870 2024-09-19T08:20:31.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.746+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/2723509364 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557350ac7870 0x557350ac9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:31.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.746+0000 7fdc8b7fe640 1 --2- 172.21.15.22:0/2723509364 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557350ac7870 0x557350ac9c90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b2d83c7a598d3f0 server_cookie=439a0fc2609a2004 in_seq=0 out_seq=0 2024-09-19T08:20:31.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:31.746+0000 7fdc6eff5640 10 client.16488 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:32.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.616+0000 7fdc6eff5640 1 -- 172.21.15.22:0/2723509364 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5573509aa0a0 con 0x557350ac7870 2024-09-19T08:20:32.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.616+0000 7fdc6eff5640 10 client.16488 handle_command_reply: tid=0 2024-09-19T08:20:32.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.616+0000 7fdc93a72740 1 client.16488 shutdown 2024-09-19T08:20:32.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.616+0000 7fdc93a72740 1 -- 172.21.15.22:0/2723509364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41d90 msgr2=0x557350a50b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:32.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.616+0000 7fdc93a72740 1 --2- 172.21.15.22:0/2723509364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41d90 0x557350a50b30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fdc80005780 tx=0x7fdc80005710 comp rx=0 tx=0).stop 2024-09-19T08:20:32.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.616+0000 7fdc93a72740 1 -- 172.21.15.22:0/2723509364 shutdown_connections 2024-09-19T08:20:32.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.616+0000 7fdc93a72740 1 --2- 172.21.15.22:0/2723509364 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x557350ac7870 0x557350ac9c90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:32.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.616+0000 7fdc93a72740 1 --2- 172.21.15.22:0/2723509364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557350a4c650 0x557350a51070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:32.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.616+0000 7fdc93a72740 1 --2- 172.21.15.22:0/2723509364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557350a41d90 0x557350a50b30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:32.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.616+0000 7fdc93a72740 1 --2- 172.21.15.22:0/2723509364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557350a41470 0x557350a505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:32.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.616+0000 7fdc93a72740 1 -- 172.21.15.22:0/2723509364 >> 172.21.15.22:0/2723509364 conn(0x557350a3da00 msgr2=0x557350a494f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:32.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.616+0000 7fdc6eff5640 0 client.16488 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:32.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.616+0000 7fdc93a72740 1 -- 172.21.15.22:0/2723509364 shutdown_connections 2024-09-19T08:20:32.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.616+0000 7fdc93a72740 1 -- 172.21.15.22:0/2723509364 wait complete. 2024-09-19T08:20:32.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.617+0000 7fdc93a72740 20 client.16488 trim_cache size 0 max 16384 2024-09-19T08:20:32.618 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:20:32.618 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 30, 2024-09-19T08:20:32.619 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:20:32.619 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 24, 2024-09-19T08:20:32.619 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:20:32.619 INFO:teuthology.orchestra.run.smithi022.stdout: "2af777fa": { 2024-09-19T08:20:32.619 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 24, 2024-09-19T08:20:32.619 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:20:32.619 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:20:32.619 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:20:32.619 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:20:32.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:20:32.619 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:20:32.619 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 149.8, 2024-09-19T08:20:32.619 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:20:32.620 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:20:32.620 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:20:32.620 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:20:32.620 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:20:32.620 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:20:32.620 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:20:32.620 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:20:32.620 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:20:32.620 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:20:32.620 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:20:32.620 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:20:32.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.617+0000 7fdc91699640 1 -- 172.21.15.22:0/1445957975 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdc740a33b0 msgr2=0x7fdc740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:32.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.618+0000 7fdc91699640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdc740a33b0 0x7fdc740a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c005ab0 tx=0x7fdc7c02a410 comp rx=0 tx=0).stop 2024-09-19T08:20:32.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.618+0000 7fdc91699640 1 -- 172.21.15.22:0/1445957975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc8c1011c0 msgr2=0x7fdc8c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:32.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.618+0000 7fdc91699640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc8c1011c0 0x7fdc8c198380 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c006010 tx=0x7fdc7c003c10 comp rx=0 tx=0).stop 2024-09-19T08:20:32.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.618+0000 7fdc91699640 1 -- 172.21.15.22:0/1445957975 shutdown_connections 2024-09-19T08:20:32.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.618+0000 7fdc91699640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdc740a33b0 0x7fdc740a5870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:32.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.618+0000 7fdc91699640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc8c105ad0 0x7fdc8c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:32.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.618+0000 7fdc91699640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc8c101b60 0x7fdc8c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:32.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.618+0000 7fdc91699640 1 --2- 172.21.15.22:0/1445957975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc8c1011c0 0x7fdc8c198380 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:32.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.618+0000 7fdc91699640 1 -- 172.21.15.22:0/1445957975 >> 172.21.15.22:0/1445957975 conn(0x7fdc8c077210 msgr2=0x7fdc8c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:32.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.618+0000 7fdc91699640 1 -- 172.21.15.22:0/1445957975 shutdown_connections 2024-09-19T08:20:32.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:32.618+0000 7fdc91699640 1 -- 172.21.15.22:0/1445957975 wait complete. 2024-09-19T08:20:32.982 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:32 smithi022 ceph-mon[30963]: pgmap v168: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 494 KiB/s wr, 559 op/s 2024-09-19T08:20:33.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:32 smithi064 ceph-mon[34900]: pgmap v168: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 494 KiB/s wr, 559 op/s 2024-09-19T08:20:33.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:32 smithi157 ceph-mon[35143]: pgmap v168: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 494 KiB/s wr, 559 op/s 2024-09-19T08:20:33.338 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2af777fa', total seconds elapsed: 34.2 2024-09-19T08:20:35.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:34 smithi064 ceph-mon[34900]: pgmap v169: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 495 KiB/s wr, 563 op/s 2024-09-19T08:20:35.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:34 smithi157 ceph-mon[35143]: pgmap v169: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 495 KiB/s wr, 563 op/s 2024-09-19T08:20:35.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:34 smithi022 ceph-mon[30963]: pgmap v169: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 495 KiB/s wr, 563 op/s 2024-09-19T08:20:36.155 INFO:tasks.workunit.client.0.smithi022.stdout: 111 350714 124 219631 101 191988 191 2024-09-19T08:20:37.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:36 smithi064 ceph-mon[34900]: pgmap v170: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 231 KiB/s wr, 179 op/s 2024-09-19T08:20:37.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:36 smithi157 ceph-mon[35143]: pgmap v170: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 231 KiB/s wr, 179 op/s 2024-09-19T08:20:37.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:36 smithi022 ceph-mon[30963]: pgmap v170: 97 pgs: 97 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 231 KiB/s wr, 179 op/s 2024-09-19T08:20:39.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:38 smithi157 ceph-mon[35143]: pgmap v171: 97 pgs: 97 active+clean; 614 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 703 KiB/s wr, 207 op/s 2024-09-19T08:20:39.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:38 smithi022 ceph-mon[30963]: pgmap v171: 97 pgs: 97 active+clean; 614 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 703 KiB/s wr, 207 op/s 2024-09-19T08:20:39.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:38 smithi064 ceph-mon[34900]: pgmap v171: 97 pgs: 97 active+clean; 614 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 703 KiB/s wr, 207 op/s 2024-09-19T08:20:41.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:40 smithi157 ceph-mon[35143]: pgmap v172: 97 pgs: 97 active+clean; 617 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 887 KiB/s wr, 252 op/s 2024-09-19T08:20:41.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:40 smithi022 ceph-mon[30963]: pgmap v172: 97 pgs: 97 active+clean; 617 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 887 KiB/s wr, 252 op/s 2024-09-19T08:20:41.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:40 smithi064 ceph-mon[34900]: pgmap v172: 97 pgs: 97 active+clean; 617 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 887 KiB/s wr, 252 op/s 2024-09-19T08:20:43.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:42 smithi157 ceph-mon[35143]: pgmap v173: 97 pgs: 97 active+clean; 617 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 823 KiB/s wr, 119 op/s 2024-09-19T08:20:43.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:42 smithi022 ceph-mon[30963]: pgmap v173: 97 pgs: 97 active+clean; 617 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 823 KiB/s wr, 119 op/s 2024-09-19T08:20:43.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:42 smithi064 ceph-mon[34900]: pgmap v173: 97 pgs: 97 active+clean; 617 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 823 KiB/s wr, 119 op/s 2024-09-19T08:20:44.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:20:45.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:44 smithi022 ceph-mon[30963]: pgmap v174: 97 pgs: 97 active+clean; 629 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-09-19T08:20:45.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:44 smithi157 ceph-mon[35143]: pgmap v174: 97 pgs: 97 active+clean; 629 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-09-19T08:20:45.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:44 smithi064 ceph-mon[34900]: pgmap v174: 97 pgs: 97 active+clean; 629 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-09-19T08:20:46.155 INFO:tasks.workunit.client.0.smithi022.stdout: 118 1846357 449 1179456 480 1052468 1205 2024-09-19T08:20:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:46.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.818+0000 7f0eaf25c640 1 -- 172.21.15.22:0/3703792264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ea8106f10 msgr2=0x7f0ea810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:46.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.818+0000 7f0eaf25c640 1 --2- 172.21.15.22:0/3703792264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ea8106f10 0x7f0ea810ea90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009be0 tx=0x7f0ea002a6c0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.819+0000 7f0eaf25c640 1 -- 172.21.15.22:0/3703792264 shutdown_connections 2024-09-19T08:20:46.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.819+0000 7f0eaf25c640 1 --2- 172.21.15.22:0/3703792264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ea8106f10 0x7f0ea810ea90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.819+0000 7f0eaf25c640 1 --2- 172.21.15.22:0/3703792264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ea8102790 0x7f0ea81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.819+0000 7f0eaf25c640 1 --2- 172.21.15.22:0/3703792264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ea8101de0 0x7f0ea81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.819+0000 7f0eaf25c640 1 -- 172.21.15.22:0/3703792264 >> 172.21.15.22:0/3703792264 conn(0x7f0ea80fd880 msgr2=0x7f0ea80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:46.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.819+0000 7f0eaf25c640 1 -- 172.21.15.22:0/3703792264 shutdown_connections 2024-09-19T08:20:46.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.819+0000 7f0eaf25c640 1 -- 172.21.15.22:0/3703792264 wait complete. 2024-09-19T08:20:46.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.820+0000 7f0eaf25c640 1 Processor -- start 2024-09-19T08:20:46.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.820+0000 7f0eaf25c640 1 -- start start 2024-09-19T08:20:46.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.820+0000 7f0eaf25c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ea8101de0 0x7f0ea81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.820+0000 7f0eaf25c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ea8102790 0x7f0ea8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.820+0000 7f0eaf25c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ea8106f10 0x7f0ea819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.820+0000 7f0eaf25c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8111f70 con 0x7f0ea8101de0 2024-09-19T08:20:46.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.820+0000 7f0eaf25c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8111df0 con 0x7f0ea8106f10 2024-09-19T08:20:46.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.820+0000 7f0eaf25c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81120f0 con 0x7f0ea8102790 2024-09-19T08:20:46.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.820+0000 7f0ea7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ea8102790 0x7f0ea8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.820+0000 7f0ea7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ea8102790 0x7f0ea8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:47658/0 (socket says 172.21.15.22:47658) 2024-09-19T08:20:46.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.820+0000 7f0ea7fff640 1 -- 172.21.15.22:0/3229758382 learned_addr learned my addr 172.21.15.22:0/3229758382 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:46.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.821+0000 7f0ead7f9640 1 --2- 172.21.15.22:0/3229758382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ea8106f10 0x7f0ea819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.821+0000 7f0eacff8640 1 --2- 172.21.15.22:0/3229758382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ea8101de0 0x7f0ea81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.821+0000 7f0ea7fff640 1 -- 172.21.15.22:0/3229758382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ea8106f10 msgr2=0x7f0ea819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:46.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.821+0000 7f0ea7fff640 1 --2- 172.21.15.22:0/3229758382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ea8106f10 0x7f0ea819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.821+0000 7f0ea7fff640 1 -- 172.21.15.22:0/3229758382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ea8101de0 msgr2=0x7f0ea81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:46.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.821+0000 7f0ea7fff640 1 --2- 172.21.15.22:0/3229758382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ea8101de0 0x7f0ea81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.821+0000 7f0ea7fff640 1 -- 172.21.15.22:0/3229758382 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea819d3a0 con 0x7f0ea8102790 2024-09-19T08:20:46.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.821+0000 7f0ea7fff640 1 --2- 172.21.15.22:0/3229758382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ea8102790 0x7f0ea8198930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400e9f0 tx=0x7f0e9400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:46.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.821+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/3229758382 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9400ccc0 con 0x7f0ea8102790 2024-09-19T08:20:46.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.821+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/3229758382 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e94010910 con 0x7f0ea8102790 2024-09-19T08:20:46.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.821+0000 7f0eaf25c640 1 -- 172.21.15.22:0/3229758382 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea819d690 con 0x7f0ea8102790 2024-09-19T08:20:46.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.822+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/3229758382 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e940188a0 con 0x7f0ea8102790 2024-09-19T08:20:46.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.822+0000 7f0eaf25c640 1 -- 172.21.15.22:0/3229758382 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea8074420 con 0x7f0ea8102790 2024-09-19T08:20:46.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.824+0000 7f0eb1635740 1 Processor -- start 2024-09-19T08:20:46.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.825+0000 7f0eb1635740 1 -- start start 2024-09-19T08:20:46.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.825+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/3229758382 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0e94010ac0 con 0x7f0ea8102790 2024-09-19T08:20:46.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.825+0000 7f0eb1635740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa175123c0 0x55aa175127a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.825+0000 7f0eb1635740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa17525950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.825+0000 7f0eb1635740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa17525e90 0x55aa17528280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.825+0000 7f0eb1635740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55aa174782e0 con 0x55aa17512ce0 2024-09-19T08:20:46.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.825+0000 7f0eb1635740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55aa1741fce0 con 0x55aa175123c0 2024-09-19T08:20:46.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.825+0000 7f0eb1635740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55aa174742a0 con 0x55aa17525e90 2024-09-19T08:20:46.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.826+0000 7f0eacff8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa17525950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.826+0000 7f0eacff8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa17525950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47028/0 (socket says 172.21.15.22:47028) 2024-09-19T08:20:46.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.826+0000 7f0eacff8640 1 -- 172.21.15.22:0/1107078478 learned_addr learned my addr 172.21.15.22:0/1107078478 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:46.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.826+0000 7f0ea7fff640 1 --2- 172.21.15.22:0/1107078478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa17525e90 0x55aa17528280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.826+0000 7f0eacff8640 1 -- 172.21.15.22:0/1107078478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa17525e90 msgr2=0x55aa17528280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:46.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.827+0000 7f0eacff8640 1 --2- 172.21.15.22:0/1107078478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa17525e90 0x55aa17528280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.827+0000 7f0eacff8640 1 -- 172.21.15.22:0/1107078478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa175123c0 msgr2=0x55aa175127a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:46.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.827+0000 7f0eacff8640 1 --2- 172.21.15.22:0/1107078478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa175123c0 0x55aa175127a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.827+0000 7f0eacff8640 1 -- 172.21.15.22:0/1107078478 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa17404440 con 0x55aa17512ce0 2024-09-19T08:20:46.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.827+0000 7f0ea7fff640 1 --2- 172.21.15.22:0/1107078478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa17525e90 0x55aa17528280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:20:46.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.827+0000 7f0eacff8640 1 --2- 172.21.15.22:0/1107078478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa17525950 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009cd0 tx=0x7f0e9c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9db5f735b4573e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:46.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.828+0000 7f0e92ffd640 1 -- 172.21.15.22:0/1107078478 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c01e070 con 0x55aa17512ce0 2024-09-19T08:20:46.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.828+0000 7f0e92ffd640 1 -- 172.21.15.22:0/1107078478 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c0198f0 con 0x55aa17512ce0 2024-09-19T08:20:46.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.828+0000 7f0e92ffd640 1 -- 172.21.15.22:0/1107078478 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c018750 con 0x55aa17512ce0 2024-09-19T08:20:46.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.828+0000 7f0ea5ffb640 1 --2- 172.21.15.22:0/3229758382 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0e980a3430 0x7f0e980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.828+0000 7f0ead7f9640 1 --2- 172.21.15.22:0/3229758382 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0e980a3430 0x7f0e980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.828+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/3229758382 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0e940d00a0 con 0x7f0ea8102790 2024-09-19T08:20:46.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.829+0000 7f0eb1635740 1 -- 172.21.15.22:0/1107078478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 msgr2=0x55aa17525950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:46.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.829+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1107078478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa17525950 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009cd0 tx=0x7f0e9c011810 comp rx=0 tx=0).stop 2024-09-19T08:20:46.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.829+0000 7f0ead7f9640 1 --2- 172.21.15.22:0/3229758382 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0e980a3430 0x7f0e980a58f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0004af0 tx=0x7f0ea0009860 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:46.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.829+0000 7f0eb1635740 1 -- 172.21.15.22:0/1107078478 shutdown_connections 2024-09-19T08:20:46.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.829+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1107078478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa17525e90 0x55aa17528280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.829+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1107078478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa17525950 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.829+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1107078478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa175123c0 0x55aa175127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.829+0000 7f0eb1635740 1 -- 172.21.15.22:0/1107078478 >> 172.21.15.22:0/1107078478 conn(0x55aa1750ea30 msgr2=0x55aa1750ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:46.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.829+0000 7f0eb1635740 1 -- 172.21.15.22:0/1107078478 shutdown_connections 2024-09-19T08:20:46.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.829+0000 7f0eb1635740 1 -- 172.21.15.22:0/1107078478 wait complete. 2024-09-19T08:20:46.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.829+0000 7f0eb1635740 1 -- start start 2024-09-19T08:20:46.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.829+0000 7f0eb1635740 1 Processor -- start 2024-09-19T08:20:46.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.830+0000 7f0eb1635740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa175123c0 0x55aa17525860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.830+0000 7f0eb1635740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa17512ce0 0x55aa17522b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.830+0000 7f0eb1635740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17525e90 0x55aa17523100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.830+0000 7f0eb1635740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55aa17528a00 con 0x55aa17525e90 2024-09-19T08:20:46.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.830+0000 7f0eb1635740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55aa17433050 con 0x55aa175123c0 2024-09-19T08:20:46.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.830+0000 7f0eb1635740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55aa17528b80 con 0x55aa17512ce0 2024-09-19T08:20:46.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.830+0000 7f0eb1635740 10 client.0 _ll_register_callbacks cb 0x55aa17473160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:46.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.830+0000 7f0eacff8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa175123c0 0x55aa17525860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.830+0000 7f0ead7f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17525e90 0x55aa17523100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.830+0000 7f0eacff8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa175123c0 0x55aa17525860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:54800/0 (socket says 172.21.15.22:54800) 2024-09-19T08:20:46.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.830+0000 7f0eacff8640 1 -- 172.21.15.22:0/614932792 learned_addr learned my addr 172.21.15.22:0/614932792 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:46.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.830+0000 7f0eacff8640 1 -- 172.21.15.22:0/614932792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa17512ce0 msgr2=0x55aa17522b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:46.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.830+0000 7f0ea7fff640 1 --2- 172.21.15.22:0/614932792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa17512ce0 0x55aa17522b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.831+0000 7f0eacff8640 1 --2- 172.21.15.22:0/614932792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa17512ce0 0x55aa17522b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.831+0000 7f0eacff8640 1 -- 172.21.15.22:0/614932792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17525e90 msgr2=0x55aa17523100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:46.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.831+0000 7f0eacff8640 1 --2- 172.21.15.22:0/614932792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17525e90 0x55aa17523100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.831+0000 7f0eacff8640 1 -- 172.21.15.22:0/614932792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa171e93c0 con 0x55aa175123c0 2024-09-19T08:20:46.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.831+0000 7f0ead7f9640 1 --2- 172.21.15.22:0/614932792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17525e90 0x55aa17523100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:46.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.831+0000 7f0ea7fff640 1 --2- 172.21.15.22:0/614932792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa17512ce0 0x55aa17522b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:20:46.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.831+0000 7f0eacff8640 1 --2- 172.21.15.22:0/614932792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa175123c0 0x55aa17525860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c01c040 tx=0x7f0e9c019cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8793371285beb975 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:46.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.831+0000 7f0e87fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:20:46.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.831+0000 7f0e87fff640 1 -- 172.21.15.22:0/614932792 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c01e040 con 0x55aa175123c0 2024-09-19T08:20:46.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.831+0000 7f0e87fff640 1 -- 172.21.15.22:0/614932792 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0e9c005040 con 0x55aa175123c0 2024-09-19T08:20:46.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.831+0000 7f0eb1635740 1 -- 172.21.15.22:0/614932792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9c0198f0 con 0x55aa175123c0 2024-09-19T08:20:46.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.831+0000 7f0e87fff640 1 -- 172.21.15.22:0/614932792 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e9c032050 con 0x55aa175123c0 2024-09-19T08:20:46.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.831+0000 7f0eb1635740 10 client.26060 fetch_fsmap learned FSMap version 30 2024-09-19T08:20:46.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.831+0000 7f0eb1635740 1 -- 172.21.15.22:0/614932792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aa17534410 con 0x55aa175123c0 2024-09-19T08:20:46.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.832+0000 7f0e87fff640 1 -- 172.21.15.22:0/614932792 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c0198f0 con 0x55aa175123c0 2024-09-19T08:20:46.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.832+0000 7f0e87fff640 1 -- 172.21.15.22:0/614932792 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f0e9c004cc0 con 0x55aa175123c0 2024-09-19T08:20:46.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.832+0000 7f0eb1635740 10 client.26060 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:20:46.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.832+0000 7f0eb1635740 10 client.26060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:20:46.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.832+0000 7f0eb1635740 20 client.26060 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:46.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.832+0000 7f0eb1635740 1 --2- 172.21.15.22:0/614932792 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55aa17535810 0x55aa17537c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.832+0000 7f0eb1635740 4 client.26060 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:20:46.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.832+0000 7f0eb1635740 1 -- 172.21.15.22:0/614932792 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa17485850 con 0x55aa17535810 2024-09-19T08:20:46.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.832+0000 7f0ea7fff640 1 --2- 172.21.15.22:0/614932792 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55aa17535810 0x55aa17537c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.832+0000 7f0e87fff640 10 client.26060 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:46.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.833+0000 7f0ea7fff640 1 --2- 172.21.15.22:0/614932792 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55aa17535810 0x55aa17537c00 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e2d2702d4043079 server_cookie=85f9a8357b3fd1b in_seq=0 out_seq=0 2024-09-19T08:20:46.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.833+0000 7f0e87fff640 1 -- 172.21.15.22:0/614932792 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa17485850 con 0x55aa17535810 2024-09-19T08:20:46.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.833+0000 7f0e87fff640 10 client.26060 handle_command_reply: tid=0 2024-09-19T08:20:46.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.833+0000 7f0eb1635740 1 client.26060 shutdown 2024-09-19T08:20:46.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.834+0000 7f0eb1635740 1 -- 172.21.15.22:0/614932792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa175123c0 msgr2=0x55aa17525860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:46.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.834+0000 7f0eb1635740 1 --2- 172.21.15.22:0/614932792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa175123c0 0x55aa17525860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c01c040 tx=0x7f0e9c019cd0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.834+0000 7f0eb1635740 1 -- 172.21.15.22:0/614932792 shutdown_connections 2024-09-19T08:20:46.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.834+0000 7f0eb1635740 1 --2- 172.21.15.22:0/614932792 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55aa17535810 0x55aa17537c00 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.834+0000 7f0eb1635740 1 --2- 172.21.15.22:0/614932792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17525e90 0x55aa17523100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.834+0000 7f0eb1635740 1 --2- 172.21.15.22:0/614932792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa17512ce0 0x55aa17522b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.834+0000 7f0eb1635740 1 --2- 172.21.15.22:0/614932792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa175123c0 0x55aa17525860 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.834+0000 7f0eb1635740 1 -- 172.21.15.22:0/614932792 >> 172.21.15.22:0/614932792 conn(0x55aa1750ea30 msgr2=0x55aa175146c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:46.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.834+0000 7f0e87fff640 0 client.26060 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:46.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.834+0000 7f0eb1635740 1 -- 172.21.15.22:0/614932792 shutdown_connections 2024-09-19T08:20:46.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.834+0000 7f0eb1635740 1 -- 172.21.15.22:0/614932792 wait complete. 2024-09-19T08:20:46.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.834+0000 7f0eb1635740 20 client.26060 trim_cache size 0 max 16384 2024-09-19T08:20:46.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.869+0000 7f0eb1635740 1 Processor -- start 2024-09-19T08:20:46.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.869+0000 7f0eb1635740 1 -- start start 2024-09-19T08:20:46.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0eb1635740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa175123c0 0x55aa17531cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0eb1635740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa175223f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0eb1635740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa17525e90 0x55aa1753d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0eb1635740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81ae010 con 0x55aa17512ce0 2024-09-19T08:20:46.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0eb1635740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81ab8b0 con 0x55aa17525e90 2024-09-19T08:20:46.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0eb1635740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81a9700 con 0x55aa175123c0 2024-09-19T08:20:46.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0ea7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa175223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0ea7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa175223f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47056/0 (socket says 172.21.15.22:47056) 2024-09-19T08:20:46.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0eacff8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa175123c0 0x55aa17531cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0ea7fff640 1 -- 172.21.15.22:0/1394629012 learned_addr learned my addr 172.21.15.22:0/1394629012 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:46.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0ead7f9640 1 --2- 172.21.15.22:0/1394629012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa17525e90 0x55aa1753d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0eacff8640 1 -- 172.21.15.22:0/1394629012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa17525e90 msgr2=0x55aa1753d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:46.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0eacff8640 1 --2- 172.21.15.22:0/1394629012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa17525e90 0x55aa1753d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0eacff8640 1 -- 172.21.15.22:0/1394629012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 msgr2=0x55aa175223f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:46.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0eacff8640 1 --2- 172.21.15.22:0/1394629012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa175223f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0eacff8640 1 -- 172.21.15.22:0/1394629012 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa17523700 con 0x55aa175123c0 2024-09-19T08:20:46.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.870+0000 7f0ea7fff640 1 --2- 172.21.15.22:0/1394629012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa175223f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:46.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.871+0000 7f0eacff8640 1 --2- 172.21.15.22:0/1394629012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa175123c0 0x55aa17531cd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c018600 tx=0x7f0e9c018940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7b7eec5ba5e2778 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:46.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.871+0000 7f0e927fc640 1 -- 172.21.15.22:0/1394629012 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c01e070 con 0x55aa175123c0 2024-09-19T08:20:46.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.871+0000 7f0e927fc640 1 -- 172.21.15.22:0/1394629012 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c029070 con 0x55aa175123c0 2024-09-19T08:20:46.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.871+0000 7f0e927fc640 1 -- 172.21.15.22:0/1394629012 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0052a0 con 0x55aa175123c0 2024-09-19T08:20:46.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.872+0000 7f0eb1635740 1 -- 172.21.15.22:0/1394629012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa175123c0 msgr2=0x55aa17531cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:46.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.872+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1394629012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa175123c0 0x55aa17531cd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c018600 tx=0x7f0e9c018940 comp rx=0 tx=0).stop 2024-09-19T08:20:46.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.872+0000 7f0eb1635740 1 -- 172.21.15.22:0/1394629012 shutdown_connections 2024-09-19T08:20:46.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.872+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1394629012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa17525e90 0x55aa1753d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.872+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1394629012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa175223f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.872+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1394629012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa175123c0 0x55aa17531cd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.872+0000 7f0eb1635740 1 -- 172.21.15.22:0/1394629012 >> 172.21.15.22:0/1394629012 conn(0x55aa1750ea30 msgr2=0x55aa1751e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:46.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.872+0000 7f0eb1635740 1 -- 172.21.15.22:0/1394629012 shutdown_connections 2024-09-19T08:20:46.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.872+0000 7f0eb1635740 1 -- 172.21.15.22:0/1394629012 wait complete. 2024-09-19T08:20:46.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.873+0000 7f0eb1635740 1 -- start start 2024-09-19T08:20:46.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.873+0000 7f0eb1635740 1 Processor -- start 2024-09-19T08:20:46.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.874+0000 7f0eb1635740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa175123c0 0x55aa17528e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.874+0000 7f0eb1635740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa17529340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.874+0000 7f0eacff8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa175123c0 0x55aa17528e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.874+0000 7f0eacff8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa175123c0 0x55aa17528e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:47708/0 (socket says 172.21.15.22:47708) 2024-09-19T08:20:46.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.874+0000 7f0ea7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa17529340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.874+0000 7f0eacff8640 1 -- 172.21.15.22:0/1477436312 learned_addr learned my addr 172.21.15.22:0/1477436312 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:46.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.874+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1477436312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa17525e90 0x55aa17529880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.874+0000 7f0eb1635740 1 -- 172.21.15.22:0/1477436312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81a9d70 con 0x55aa17512ce0 2024-09-19T08:20:46.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.875+0000 7f0eb1635740 1 -- 172.21.15.22:0/1477436312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81ab1e0 con 0x55aa17525e90 2024-09-19T08:20:46.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.875+0000 7f0ead7f9640 1 --2- 172.21.15.22:0/1477436312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa17525e90 0x55aa17529880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.875+0000 7f0eb1635740 1 -- 172.21.15.22:0/1477436312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81aa440 con 0x55aa175123c0 2024-09-19T08:20:46.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.875+0000 7f0eb1635740 10 client.0 _ll_register_callbacks cb 0x7f0e9c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:46.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.875+0000 7f0ead7f9640 1 -- 172.21.15.22:0/1477436312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa175123c0 msgr2=0x55aa17528e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:46.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.875+0000 7f0ead7f9640 1 --2- 172.21.15.22:0/1477436312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa175123c0 0x55aa17528e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.875+0000 7f0ead7f9640 1 -- 172.21.15.22:0/1477436312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 msgr2=0x55aa17529340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:46.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.875+0000 7f0ead7f9640 1 --2- 172.21.15.22:0/1477436312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa17529340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.875+0000 7f0ead7f9640 1 -- 172.21.15.22:0/1477436312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa175af470 con 0x55aa17525e90 2024-09-19T08:20:46.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.876+0000 7f0ead7f9640 1 --2- 172.21.15.22:0/1477436312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa17525e90 0x55aa17529880 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009b30 tx=0x7f0ea002a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ba1f04c3afa32f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:46.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.876+0000 7f0e87fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:20:46.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.876+0000 7f0eb1635740 1 -- 172.21.15.22:0/1477436312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9c029070 con 0x55aa17525e90 2024-09-19T08:20:46.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.878+0000 7f0e87fff640 1 -- 172.21.15.22:0/1477436312 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0036040 con 0x55aa17525e90 2024-09-19T08:20:46.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.878+0000 7f0e87fff640 1 -- 172.21.15.22:0/1477436312 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0ea0004040 con 0x55aa17525e90 2024-09-19T08:20:46.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.879+0000 7f0e87fff640 1 -- 172.21.15.22:0/1477436312 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ea0004240 con 0x55aa17525e90 2024-09-19T08:20:46.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.883+0000 7f0eb1635740 10 client.26066 fetch_fsmap learned FSMap version 30 2024-09-19T08:20:46.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.883+0000 7f0eb1635740 1 -- 172.21.15.22:0/1477436312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55aa17515130 con 0x55aa17525e90 2024-09-19T08:20:46.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.886+0000 7f0e87fff640 1 -- 172.21.15.22:0/1477436312 <== mon.2 v2:172.21.15.64:3300/0 4 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f0ea00043e0 con 0x55aa17525e90 2024-09-19T08:20:46.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.887+0000 7f0eb1635740 10 client.26066 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:20:46.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.887+0000 7f0eb1635740 10 client.26066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:20:46.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.887+0000 7f0eb1635740 20 client.26066 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:46.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.887+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1477436312 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55aa17515650 0x55aa17517a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:46.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.887+0000 7f0eb1635740 4 client.26066 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:20:46.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.887+0000 7f0eb1635740 1 -- 172.21.15.22:0/1477436312 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aa1747fd60 con 0x55aa17515650 2024-09-19T08:20:46.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.887+0000 7f0eacff8640 1 --2- 172.21.15.22:0/1477436312 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55aa17515650 0x55aa17517a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.889+0000 7f0eacff8640 1 --2- 172.21.15.22:0/1477436312 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55aa17515650 0x55aa17517a70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a0e5a3ca0551214 server_cookie=df2e8b50d4c22dd5 in_seq=0 out_seq=0 2024-09-19T08:20:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.889+0000 7f0e87fff640 10 client.26066 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.889+0000 7f0e87fff640 1 -- 172.21.15.22:0/1477436312 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aa1747fd60 con 0x55aa17515650 2024-09-19T08:20:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.889+0000 7f0e87fff640 10 client.26066 handle_command_reply: tid=0 2024-09-19T08:20:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.890+0000 7f0eb1635740 1 client.26066 shutdown 2024-09-19T08:20:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.890+0000 7f0eb1635740 1 -- 172.21.15.22:0/1477436312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa17525e90 msgr2=0x55aa17529880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.890+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1477436312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa17525e90 0x55aa17529880 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009b30 tx=0x7f0ea002a6c0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.890+0000 7f0eb1635740 1 -- 172.21.15.22:0/1477436312 shutdown_connections 2024-09-19T08:20:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.890+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1477436312 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55aa17515650 0x55aa17517a70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.891+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1477436312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa17525e90 0x55aa17529880 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.891+0000 7f0e87fff640 0 client.26066 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.891+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1477436312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa17512ce0 0x55aa17529340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.891+0000 7f0eb1635740 1 --2- 172.21.15.22:0/1477436312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa175123c0 0x55aa17528e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.891+0000 7f0eb1635740 1 -- 172.21.15.22:0/1477436312 >> 172.21.15.22:0/1477436312 conn(0x55aa1750ea30 msgr2=0x55aa1750d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.892+0000 7f0eb1635740 1 -- 172.21.15.22:0/1477436312 shutdown_connections 2024-09-19T08:20:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.892+0000 7f0eb1635740 1 -- 172.21.15.22:0/1477436312 wait complete. 2024-09-19T08:20:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.892+0000 7f0eb1635740 20 client.26066 trim_cache size 0 max 16384 2024-09-19T08:20:46.892 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:20:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-19T08:20:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-09-19T08:20:46.893 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:20:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.894+0000 7f0eaf25c640 1 -- 172.21.15.22:0/3229758382 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0e980a3430 msgr2=0x7f0e980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.894+0000 7f0eaf25c640 1 --2- 172.21.15.22:0/3229758382 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0e980a3430 0x7f0e980a58f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0004af0 tx=0x7f0ea0009860 comp rx=0 tx=0).stop 2024-09-19T08:20:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.894+0000 7f0eaf25c640 1 -- 172.21.15.22:0/3229758382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ea8102790 msgr2=0x7f0ea8198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.894+0000 7f0eaf25c640 1 --2- 172.21.15.22:0/3229758382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ea8102790 0x7f0ea8198930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400e9f0 tx=0x7f0e9400edf0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.894+0000 7f0eaf25c640 1 -- 172.21.15.22:0/3229758382 shutdown_connections 2024-09-19T08:20:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.894+0000 7f0eaf25c640 1 --2- 172.21.15.22:0/3229758382 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0e980a3430 0x7f0e980a58f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.894+0000 7f0eaf25c640 1 --2- 172.21.15.22:0/3229758382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ea8106f10 0x7f0ea819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.894+0000 7f0eaf25c640 1 --2- 172.21.15.22:0/3229758382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ea8102790 0x7f0ea8198930 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.895+0000 7f0eaf25c640 1 --2- 172.21.15.22:0/3229758382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ea8101de0 0x7f0ea81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.895+0000 7f0eaf25c640 1 -- 172.21.15.22:0/3229758382 >> 172.21.15.22:0/3229758382 conn(0x7f0ea80fd880 msgr2=0x7f0ea80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.895+0000 7f0eaf25c640 1 -- 172.21.15.22:0/3229758382 shutdown_connections 2024-09-19T08:20:46.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:46.895+0000 7f0eaf25c640 1 -- 172.21.15.22:0/3229758382 wait complete. 2024-09-19T08:20:47.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:46 smithi022 ceph-mon[30963]: pgmap v175: 97 pgs: 97 active+clean; 629 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 260 op/s 2024-09-19T08:20:47.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:46 smithi157 ceph-mon[35143]: pgmap v175: 97 pgs: 97 active+clean; 629 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 260 op/s 2024-09-19T08:20:47.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:46 smithi064 ceph-mon[34900]: pgmap v175: 97 pgs: 97 active+clean; 629 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 260 op/s 2024-09-19T08:20:47.760 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:20:47.761 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:20:48.046 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.697+0000 7fcef0df9640 1 -- 172.21.15.22:0/2445733072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fceec101290 msgr2=0x7fceec10d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.698+0000 7fcef0df9640 1 --2- 172.21.15.22:0/2445733072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fceec101290 0x7fceec10d7c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009cd0 tx=0x7fcee402a910 comp rx=0 tx=0).stop 2024-09-19T08:20:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.699+0000 7fcef0df9640 1 -- 172.21.15.22:0/2445733072 shutdown_connections 2024-09-19T08:20:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.700+0000 7fcef0df9640 1 --2- 172.21.15.22:0/2445733072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fceec101290 0x7fceec10d7c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.700+0000 7fcef0df9640 1 --2- 172.21.15.22:0/2445733072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fceec1008f0 0x7fceec100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.700+0000 7fcef0df9640 1 --2- 172.21.15.22:0/2445733072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceec1068f0 0x7fceec106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.700+0000 7fcef0df9640 1 -- 172.21.15.22:0/2445733072 >> 172.21.15.22:0/2445733072 conn(0x7fceec0fc630 msgr2=0x7fceec0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.701+0000 7fcef0df9640 1 -- 172.21.15.22:0/2445733072 shutdown_connections 2024-09-19T08:20:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.704+0000 7fcef0df9640 1 -- 172.21.15.22:0/2445733072 wait complete. 2024-09-19T08:20:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.706+0000 7fcef0df9640 1 Processor -- start 2024-09-19T08:20:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.706+0000 7fcef0df9640 1 -- start start 2024-09-19T08:20:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.707+0000 7fcef0df9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceec1008f0 0x7fceec196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.710+0000 7fcef0df9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fceec101290 0x7fceec197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.710+0000 7fcef0df9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fceec1068f0 0x7fceec19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.711+0000 7fceead9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fceec1068f0 0x7fceec19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.711+0000 7fceead9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fceec1068f0 0x7fceec19b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47126/0 (socket says 172.21.15.22:47126) 2024-09-19T08:20:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.711+0000 7fceead9d640 1 -- 172.21.15.22:0/2776983887 learned_addr learned my addr 172.21.15.22:0/2776983887 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.711+0000 7fcee9d9b640 1 --2- 172.21.15.22:0/2776983887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fceec101290 0x7fceec197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.711+0000 7fceea59c640 1 --2- 172.21.15.22:0/2776983887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceec1008f0 0x7fceec196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.712+0000 7fcef0df9640 1 -- 172.21.15.22:0/2776983887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fceec110d40 con 0x7fceec1068f0 2024-09-19T08:20:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.712+0000 7fcef0df9640 1 -- 172.21.15.22:0/2776983887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fceec110bc0 con 0x7fceec1008f0 2024-09-19T08:20:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.712+0000 7fcef0df9640 1 -- 172.21.15.22:0/2776983887 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fceec110ec0 con 0x7fceec101290 2024-09-19T08:20:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.713+0000 7fceead9d640 1 -- 172.21.15.22:0/2776983887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fceec101290 msgr2=0x7fceec197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.713+0000 7fceead9d640 1 --2- 172.21.15.22:0/2776983887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fceec101290 0x7fceec197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.713+0000 7fceead9d640 1 -- 172.21.15.22:0/2776983887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceec1008f0 msgr2=0x7fceec196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:48.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.713+0000 7fceead9d640 1 --2- 172.21.15.22:0/2776983887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceec1008f0 0x7fceec196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:48.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.713+0000 7fceead9d640 1 -- 172.21.15.22:0/2776983887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec19bf70 con 0x7fceec1068f0 2024-09-19T08:20:48.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.714+0000 7fceead9d640 1 --2- 172.21.15.22:0/2776983887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fceec1068f0 0x7fceec19b890 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcee402ae30 tx=0x7fcee4031690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:48.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.715+0000 7fcedb7fe640 1 -- 172.21.15.22:0/2776983887 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4004040 con 0x7fceec1068f0 2024-09-19T08:20:48.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.715+0000 7fcedb7fe640 1 -- 172.21.15.22:0/2776983887 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcee40046c0 con 0x7fceec1068f0 2024-09-19T08:20:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.716+0000 7fcedb7fe640 1 -- 172.21.15.22:0/2776983887 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee40305d0 con 0x7fceec1068f0 2024-09-19T08:20:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.718+0000 7fcef0df9640 1 -- 172.21.15.22:0/2776983887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceec19c200 con 0x7fceec1068f0 2024-09-19T08:20:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.718+0000 7fcef0df9640 1 -- 172.21.15.22:0/2776983887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceec19c600 con 0x7fceec1068f0 2024-09-19T08:20:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.724+0000 7fcedb7fe640 1 -- 172.21.15.22:0/2776983887 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcee402f620 con 0x7fceec1068f0 2024-09-19T08:20:48.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.725+0000 7fcedb7fe640 1 --2- 172.21.15.22:0/2776983887 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fced40a3460 0x7fced40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:48.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.726+0000 7fcedb7fe640 1 -- 172.21.15.22:0/2776983887 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcee40f1160 con 0x7fceec1068f0 2024-09-19T08:20:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.726+0000 7fced97fa640 1 -- 172.21.15.22:0/2776983887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceec102030 con 0x7fceec1068f0 2024-09-19T08:20:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.727+0000 7fceea59c640 1 --2- 172.21.15.22:0/2776983887 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fced40a3460 0x7fced40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.730+0000 7fceea59c640 1 --2- 172.21.15.22:0/2776983887 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fced40a3460 0x7fced40a5920 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0058c0 tx=0x7fcedc005be0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:48.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:48.737+0000 7fcedb7fe640 1 -- 172.21.15.22:0/2776983887 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcee40bdc00 con 0x7fceec1068f0 2024-09-19T08:20:49.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:49.142+0000 7fced97fa640 1 -- 172.21.15.22:0/2776983887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fceec198320 con 0x7fceec1068f0 2024-09-19T08:20:49.142 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:48 smithi022 ceph-mon[30963]: pgmap v176: 97 pgs: 97 active+clean; 633 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 312 op/s 2024-09-19T08:20:49.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:49.143+0000 7fcedb7fe640 1 -- 172.21.15.22:0/2776983887 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11907 (secure 0 0 0) 0x7fcee40bd500 con 0x7fceec1068f0 2024-09-19T08:20:49.143 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:20:49.143 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":30,"btime":"2024-09-19T08:20:12:788245+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:20:11.821718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:20:49.144 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 30 2024-09-19T08:20:49.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:49.145+0000 7fced97fa640 1 -- 172.21.15.22:0/2776983887 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fced40a3460 msgr2=0x7fced40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:49.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:49.145+0000 7fced97fa640 1 --2- 172.21.15.22:0/2776983887 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fced40a3460 0x7fced40a5920 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0058c0 tx=0x7fcedc005be0 comp rx=0 tx=0).stop 2024-09-19T08:20:49.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:49.145+0000 7fced97fa640 1 -- 172.21.15.22:0/2776983887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fceec1068f0 msgr2=0x7fceec19b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:49.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:49.145+0000 7fced97fa640 1 --2- 172.21.15.22:0/2776983887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fceec1068f0 0x7fceec19b890 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcee402ae30 tx=0x7fcee4031690 comp rx=0 tx=0).stop 2024-09-19T08:20:49.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:49.145+0000 7fced97fa640 1 -- 172.21.15.22:0/2776983887 shutdown_connections 2024-09-19T08:20:49.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:49.145+0000 7fced97fa640 1 --2- 172.21.15.22:0/2776983887 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fced40a3460 0x7fced40a5920 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:49.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:49.145+0000 7fced97fa640 1 --2- 172.21.15.22:0/2776983887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fceec1068f0 0x7fceec19b890 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:49.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:49.145+0000 7fced97fa640 1 --2- 172.21.15.22:0/2776983887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fceec101290 0x7fceec197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:49.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:49.145+0000 7fced97fa640 1 --2- 172.21.15.22:0/2776983887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceec1008f0 0x7fceec196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:49.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:49.145+0000 7fced97fa640 1 -- 172.21.15.22:0/2776983887 >> 172.21.15.22:0/2776983887 conn(0x7fceec0fc630 msgr2=0x7fceec0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:49.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:49.145+0000 7fced97fa640 1 -- 172.21.15.22:0/2776983887 shutdown_connections 2024-09-19T08:20:49.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:49.145+0000 7fced97fa640 1 -- 172.21.15.22:0/2776983887 wait complete. 2024-09-19T08:20:49.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:48 smithi157 ceph-mon[35143]: pgmap v176: 97 pgs: 97 active+clean; 633 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 312 op/s 2024-09-19T08:20:49.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:48 smithi064 ceph-mon[34900]: pgmap v176: 97 pgs: 97 active+clean; 633 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 312 op/s 2024-09-19T08:20:50.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:49 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2776983887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:20:50.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:49 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2776983887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:20:50.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:49 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2776983887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:20:50.355 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:20:51.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:50 smithi022 ceph-mon[30963]: pgmap v177: 97 pgs: 97 active+clean; 639 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 426 op/s 2024-09-19T08:20:51.153 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:50 smithi157 ceph-mon[35143]: pgmap v177: 97 pgs: 97 active+clean; 639 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 426 op/s 2024-09-19T08:20:51.326 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:51.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:50 smithi064 ceph-mon[34900]: pgmap v177: 97 pgs: 97 active+clean; 639 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 426 op/s 2024-09-19T08:20:51.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.934+0000 7f99bd38a640 1 -- 172.21.15.22:0/4114312156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b81027a0 msgr2=0x7f99b81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:51.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.934+0000 7f99bd38a640 1 --2- 172.21.15.22:0/4114312156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b81027a0 0x7f99b81067f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009900 tx=0x7f99a002a410 comp rx=0 tx=0).stop 2024-09-19T08:20:51.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.934+0000 7f99bd38a640 1 -- 172.21.15.22:0/4114312156 shutdown_connections 2024-09-19T08:20:51.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.934+0000 7f99bd38a640 1 --2- 172.21.15.22:0/4114312156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b8106f20 0x7f99b810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.934+0000 7f99bd38a640 1 --2- 172.21.15.22:0/4114312156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b81027a0 0x7f99b81067f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.934+0000 7f99bd38a640 1 --2- 172.21.15.22:0/4114312156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b8101df0 0x7f99b81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.934+0000 7f99bd38a640 1 -- 172.21.15.22:0/4114312156 >> 172.21.15.22:0/4114312156 conn(0x7f99b80fd8f0 msgr2=0x7f99b80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:51.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.934+0000 7f99bd38a640 1 -- 172.21.15.22:0/4114312156 shutdown_connections 2024-09-19T08:20:51.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.934+0000 7f99bd38a640 1 -- 172.21.15.22:0/4114312156 wait complete. 2024-09-19T08:20:51.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.935+0000 7f99bd38a640 1 Processor -- start 2024-09-19T08:20:51.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.935+0000 7f99bd38a640 1 -- start start 2024-09-19T08:20:51.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.935+0000 7f99bd38a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8101df0 0x7f99b80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.935+0000 7f99bd38a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b81027a0 0x7f99b8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.935+0000 7f99bd38a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b8106f20 0x7f99b8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.935+0000 7f99bd38a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99b8111f80 con 0x7f99b8101df0 2024-09-19T08:20:51.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.935+0000 7f99bd38a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99b8111e00 con 0x7f99b8106f20 2024-09-19T08:20:51.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.935+0000 7f99b6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8101df0 0x7f99b80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.935+0000 7f99bd38a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99b8112100 con 0x7f99b81027a0 2024-09-19T08:20:51.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.935+0000 7f99b6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8101df0 0x7f99b80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47144/0 (socket says 172.21.15.22:47144) 2024-09-19T08:20:51.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.935+0000 7f99b6ffd640 1 -- 172.21.15.22:0/2053175959 learned_addr learned my addr 172.21.15.22:0/2053175959 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:51.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.935+0000 7f99b77fe640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b8106f20 0x7f99b8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.935+0000 7f99b67fc640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b81027a0 0x7f99b8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.936+0000 7f99b77fe640 1 -- 172.21.15.22:0/2053175959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b81027a0 msgr2=0x7f99b8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:51.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.936+0000 7f99b77fe640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b81027a0 0x7f99b8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.936+0000 7f99b77fe640 1 -- 172.21.15.22:0/2053175959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8101df0 msgr2=0x7f99b80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:51.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.936+0000 7f99b77fe640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8101df0 0x7f99b80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.936+0000 7f99b77fe640 1 -- 172.21.15.22:0/2053175959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b81a6920 con 0x7f99b8106f20 2024-09-19T08:20:51.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.936+0000 7f99b6ffd640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8101df0 0x7f99b80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:51.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.936+0000 7f99b77fe640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b8106f20 0x7f99b8074f70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f99b000e970 tx=0x7f99b000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:51.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.936+0000 7f99affff640 1 -- 172.21.15.22:0/2053175959 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b000ccc0 con 0x7f99b8106f20 2024-09-19T08:20:51.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.936+0000 7f99affff640 1 -- 172.21.15.22:0/2053175959 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99b0005020 con 0x7f99b8106f20 2024-09-19T08:20:51.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.936+0000 7f99affff640 1 -- 172.21.15.22:0/2053175959 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0004a00 con 0x7f99b8106f20 2024-09-19T08:20:51.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.936+0000 7f99bd38a640 1 -- 172.21.15.22:0/2053175959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b81a6c10 con 0x7f99b8106f20 2024-09-19T08:20:51.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.936+0000 7f99bd38a640 1 -- 172.21.15.22:0/2053175959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b81a70d0 con 0x7f99b8106f20 2024-09-19T08:20:51.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.939+0000 7f99bf763740 1 Processor -- start 2024-09-19T08:20:51.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.940+0000 7f99affff640 1 -- 172.21.15.22:0/2053175959 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f99b00051d0 con 0x7f99b8106f20 2024-09-19T08:20:51.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.940+0000 7f99bf763740 1 -- start start 2024-09-19T08:20:51.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.940+0000 7f99bf763740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.940+0000 7f99bf763740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1ec5d0 0x556d6b1eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.940+0000 7f99bf763740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1f7370 0x556d6b1f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.940+0000 7f99bf763740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556d6b1522f0 con 0x556d6b1edb40 2024-09-19T08:20:51.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.940+0000 7f99bf763740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556d6b0f9cf0 con 0x556d6b1f7370 2024-09-19T08:20:51.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.940+0000 7f99bf763740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556d6b14e2b0 con 0x556d6b1ec5d0 2024-09-19T08:20:51.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99b6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1ec5d0 0x556d6b1eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99b6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1ec5d0 0x556d6b1eca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:47644/0 (socket says 172.21.15.22:47644) 2024-09-19T08:20:51.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99b67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99b6ffd640 1 -- 172.21.15.22:0/2589320400 learned_addr learned my addr 172.21.15.22:0/2589320400 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:51.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99b67fc640 1 --2- 172.21.15.22:0/2589320400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1f7370 0x556d6b1f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99b67fc640 1 -- 172.21.15.22:0/2589320400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1ec5d0 msgr2=0x556d6b1eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:51.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99b67fc640 1 --2- 172.21.15.22:0/2589320400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1ec5d0 0x556d6b1eca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99b67fc640 1 -- 172.21.15.22:0/2589320400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1f7370 msgr2=0x556d6b1f9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:51.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99b67fc640 1 --2- 172.21.15.22:0/2589320400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1f7370 0x556d6b1f9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99b67fc640 1 -- 172.21.15.22:0/2589320400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d6b0de3a0 con 0x556d6b1edb40 2024-09-19T08:20:51.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99b6ffd640 1 --2- 172.21.15.22:0/2589320400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1ec5d0 0x556d6b1eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:51.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99affff640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f99a40a3430 0x7f99a40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99b67fc640 1 --2- 172.21.15.22:0/2589320400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1ec090 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f99a0005940 tx=0x7f99a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d7f65ba97fb8a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:51.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99affff640 1 -- 172.21.15.22:0/2053175959 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f99b00cfbd0 con 0x7f99b8106f20 2024-09-19T08:20:51.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.941+0000 7f99b6ffd640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f99a40a3430 0x7f99a40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.942+0000 7f99ad7fa640 1 -- 172.21.15.22:0/2589320400 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0044030 con 0x556d6b1edb40 2024-09-19T08:20:51.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.942+0000 7f99ad7fa640 1 -- 172.21.15.22:0/2589320400 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a0031910 con 0x556d6b1edb40 2024-09-19T08:20:51.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.942+0000 7f99ad7fa640 1 -- 172.21.15.22:0/2589320400 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0035040 con 0x556d6b1edb40 2024-09-19T08:20:51.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.942+0000 7f99bf763740 1 -- 172.21.15.22:0/2589320400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 msgr2=0x556d6b1ec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:51.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.942+0000 7f99b6ffd640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f99a40a3430 0x7f99a40a58f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x556d6b1e9ac0 tx=0x7f99a8009500 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:51.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.942+0000 7f99bf763740 1 --2- 172.21.15.22:0/2589320400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1ec090 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f99a0005940 tx=0x7f99a0000bc0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.943+0000 7f99bf763740 1 -- 172.21.15.22:0/2589320400 shutdown_connections 2024-09-19T08:20:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.943+0000 7f99bf763740 1 --2- 172.21.15.22:0/2589320400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1f7370 0x556d6b1f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.943+0000 7f99bf763740 1 --2- 172.21.15.22:0/2589320400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1ec5d0 0x556d6b1eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.943+0000 7f99bf763740 1 --2- 172.21.15.22:0/2589320400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1ec090 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.943+0000 7f99bf763740 1 -- 172.21.15.22:0/2589320400 >> 172.21.15.22:0/2589320400 conn(0x556d6b1b5310 msgr2=0x556d6b1e8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:51.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.943+0000 7f99bf763740 1 -- 172.21.15.22:0/2589320400 shutdown_connections 2024-09-19T08:20:51.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.943+0000 7f99bf763740 1 -- 172.21.15.22:0/2589320400 wait complete. 2024-09-19T08:20:51.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.943+0000 7f99bf763740 1 -- start start 2024-09-19T08:20:51.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.943+0000 7f99bf763740 1 Processor -- start 2024-09-19T08:20:51.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99bf763740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1ec5d0 0x556d6b1f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99bf763740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1edb40 0x556d6b1f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99bf763740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1f7370 0x556d6b1f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99bf763740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556d6b1fa010 con 0x556d6b1ec5d0 2024-09-19T08:20:51.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99bf763740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556d6b10d060 con 0x556d6b1edb40 2024-09-19T08:20:51.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99bf763740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556d6b1fa190 con 0x556d6b1f7370 2024-09-19T08:20:51.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99bf763740 10 client.0 _ll_register_callbacks cb 0x556d6ae608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:51.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99b6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1ec5d0 0x556d6b1f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99b6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1ec5d0 0x556d6b1f6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47168/0 (socket says 172.21.15.22:47168) 2024-09-19T08:20:51.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99b6ffd640 1 -- 172.21.15.22:0/3054867142 learned_addr learned my addr 172.21.15.22:0/3054867142 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:51.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99b77fe640 1 --2- 172.21.15.22:0/3054867142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1f7370 0x556d6b1f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99b67fc640 1 --2- 172.21.15.22:0/3054867142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1edb40 0x556d6b1f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99b77fe640 1 -- 172.21.15.22:0/3054867142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1edb40 msgr2=0x556d6b1f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:51.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99b77fe640 1 --2- 172.21.15.22:0/3054867142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1edb40 0x556d6b1f4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99b77fe640 1 -- 172.21.15.22:0/3054867142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1ec5d0 msgr2=0x556d6b1f6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:51.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99b77fe640 1 --2- 172.21.15.22:0/3054867142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1ec5d0 0x556d6b1f6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99b77fe640 1 -- 172.21.15.22:0/3054867142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d6aec33c0 con 0x556d6b1f7370 2024-09-19T08:20:51.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.944+0000 7f99b6ffd640 1 --2- 172.21.15.22:0/3054867142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1ec5d0 0x556d6b1f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:51.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f99b77fe640 1 --2- 172.21.15.22:0/3054867142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1f7370 0x556d6b1f47d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f99b00cd330 tx=0x7f99b009c5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd0e9f60e1cc2847 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:51.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f999e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:20:51.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f999e7fc640 1 -- 172.21.15.22:0/3054867142 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b001c040 con 0x556d6b1f7370 2024-09-19T08:20:51.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f999e7fc640 1 -- 172.21.15.22:0/3054867142 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f99b00a48a0 con 0x556d6b1f7370 2024-09-19T08:20:51.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f99bf763740 1 -- 172.21.15.22:0/3054867142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99a0031910 con 0x556d6b1f7370 2024-09-19T08:20:51.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f999e7fc640 1 -- 172.21.15.22:0/3054867142 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f99b00a4b30 con 0x556d6b1f7370 2024-09-19T08:20:51.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f99bf763740 10 client.26047 fetch_fsmap learned FSMap version 30 2024-09-19T08:20:51.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f99bf763740 1 -- 172.21.15.22:0/3054867142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d6b205ba0 con 0x556d6b1f7370 2024-09-19T08:20:51.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f999e7fc640 1 -- 172.21.15.22:0/3054867142 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a0031910 con 0x556d6b1f7370 2024-09-19T08:20:51.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f999e7fc640 1 -- 172.21.15.22:0/3054867142 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f99b00ba020 con 0x556d6b1f7370 2024-09-19T08:20:51.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f99bf763740 10 client.26047 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:20:51.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f99bf763740 10 client.26047 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:20:51.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f99bf763740 20 client.26047 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:51.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f99bf763740 1 --2- 172.21.15.22:0/3054867142 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x556d6b206fb0 0x556d6b209410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f99bf763740 4 client.26047 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:20:51.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.945+0000 7f99bf763740 1 -- 172.21.15.22:0/3054867142 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d6b15f860 con 0x556d6b206fb0 2024-09-19T08:20:51.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.946+0000 7f99b6ffd640 1 --2- 172.21.15.22:0/3054867142 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x556d6b206fb0 0x556d6b209410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.946+0000 7f99b6ffd640 1 --2- 172.21.15.22:0/3054867142 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x556d6b206fb0 0x556d6b209410 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e7cb436a0449dd3 server_cookie=491621557375b4c8 in_seq=0 out_seq=0 2024-09-19T08:20:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.946+0000 7f999e7fc640 10 client.26047 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:20:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.947+0000 7f999e7fc640 1 -- 172.21.15.22:0/3054867142 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d6b15f860 con 0x556d6b206fb0 2024-09-19T08:20:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.947+0000 7f999e7fc640 10 client.26047 handle_command_reply: tid=0 2024-09-19T08:20:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.948+0000 7f99bf763740 1 client.26047 shutdown 2024-09-19T08:20:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.948+0000 7f99bf763740 1 -- 172.21.15.22:0/3054867142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1f7370 msgr2=0x556d6b1f47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.948+0000 7f99bf763740 1 --2- 172.21.15.22:0/3054867142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1f7370 0x556d6b1f47d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f99b00cd330 tx=0x7f99b009c5c0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.948+0000 7f99bf763740 1 -- 172.21.15.22:0/3054867142 shutdown_connections 2024-09-19T08:20:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.948+0000 7f99bf763740 1 --2- 172.21.15.22:0/3054867142 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x556d6b206fb0 0x556d6b209410 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.948+0000 7f99bf763740 1 --2- 172.21.15.22:0/3054867142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1f7370 0x556d6b1f47d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.948+0000 7f99bf763740 1 --2- 172.21.15.22:0/3054867142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1edb40 0x556d6b1f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.948+0000 7f999e7fc640 0 client.26047 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:20:51.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.948+0000 7f99bf763740 1 --2- 172.21.15.22:0/3054867142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1ec5d0 0x556d6b1f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.948+0000 7f99bf763740 1 -- 172.21.15.22:0/3054867142 >> 172.21.15.22:0/3054867142 conn(0x556d6b1b5310 msgr2=0x556d6b1ee9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:51.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.948+0000 7f99bf763740 1 -- 172.21.15.22:0/3054867142 shutdown_connections 2024-09-19T08:20:51.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.948+0000 7f99bf763740 1 -- 172.21.15.22:0/3054867142 wait complete. 2024-09-19T08:20:51.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.949+0000 7f99bf763740 20 client.26047 trim_cache size 0 max 16384 2024-09-19T08:20:51.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.962+0000 7f99bf763740 1 Processor -- start 2024-09-19T08:20:51.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.962+0000 7f99bf763740 1 -- start start 2024-09-19T08:20:51.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.962+0000 7f99bf763740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1ec5d0 0x556d6b2032b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.962+0000 7f99bf763740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1f3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.962+0000 7f99bf763740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1f7370 0x556d6b1ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.962+0000 7f99bf763740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99b81b22d0 con 0x556d6b1edb40 2024-09-19T08:20:51.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.962+0000 7f99bf763740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99b81afb70 con 0x556d6b1f7370 2024-09-19T08:20:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.962+0000 7f99bf763740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99b81ad9c0 con 0x556d6b1ec5d0 2024-09-19T08:20:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.962+0000 7f99b6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1ec5d0 0x556d6b2032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99b77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1f7370 0x556d6b1ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99b67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99b6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1ec5d0 0x556d6b2032b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:47672/0 (socket says 172.21.15.22:47672) 2024-09-19T08:20:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99b6ffd640 1 -- 172.21.15.22:0/1960739128 learned_addr learned my addr 172.21.15.22:0/1960739128 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99b6ffd640 1 -- 172.21.15.22:0/1960739128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1f7370 msgr2=0x556d6b1ef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99b6ffd640 1 --2- 172.21.15.22:0/1960739128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1f7370 0x556d6b1ef120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99b6ffd640 1 -- 172.21.15.22:0/1960739128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 msgr2=0x556d6b1f3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99b6ffd640 1 --2- 172.21.15.22:0/1960739128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1f3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99b6ffd640 1 -- 172.21.15.22:0/1960739128 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d6b2037f0 con 0x556d6b1ec5d0 2024-09-19T08:20:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99b77fe640 1 --2- 172.21.15.22:0/1960739128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1f7370 0x556d6b1ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99b67fc640 1 --2- 172.21.15.22:0/1960739128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1f3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99b6ffd640 1 --2- 172.21.15.22:0/1960739128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1ec5d0 0x556d6b2032b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f99a800fc40 tx=0x7f99a801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ddf7b61def2772 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99acff9640 1 -- 172.21.15.22:0/1960739128 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8020980 con 0x556d6b1ec5d0 2024-09-19T08:20:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99acff9640 1 -- 172.21.15.22:0/1960739128 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a802b470 con 0x556d6b1ec5d0 2024-09-19T08:20:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.963+0000 7f99acff9640 1 -- 172.21.15.22:0/1960739128 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8010720 con 0x556d6b1ec5d0 2024-09-19T08:20:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.964+0000 7f99bf763740 1 -- 172.21.15.22:0/1960739128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1ec5d0 msgr2=0x556d6b2032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.964+0000 7f99bf763740 1 --2- 172.21.15.22:0/1960739128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1ec5d0 0x556d6b2032b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f99a800fc40 tx=0x7f99a801a000 comp rx=0 tx=0).stop 2024-09-19T08:20:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.964+0000 7f99bf763740 1 -- 172.21.15.22:0/1960739128 shutdown_connections 2024-09-19T08:20:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.964+0000 7f99bf763740 1 --2- 172.21.15.22:0/1960739128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1f7370 0x556d6b1ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.964+0000 7f99bf763740 1 --2- 172.21.15.22:0/1960739128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1f3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.964+0000 7f99bf763740 1 --2- 172.21.15.22:0/1960739128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1ec5d0 0x556d6b2032b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.964+0000 7f99bf763740 1 -- 172.21.15.22:0/1960739128 >> 172.21.15.22:0/1960739128 conn(0x556d6b1b5310 msgr2=0x556d6b1e3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.964+0000 7f99bf763740 1 -- 172.21.15.22:0/1960739128 shutdown_connections 2024-09-19T08:20:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.964+0000 7f99bf763740 1 -- 172.21.15.22:0/1960739128 wait complete. 2024-09-19T08:20:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.965+0000 7f99bf763740 1 -- start start 2024-09-19T08:20:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.965+0000 7f99bf763740 1 Processor -- start 2024-09-19T08:20:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.965+0000 7f99bf763740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1ec5d0 0x556d6b1fa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.965+0000 7f99bf763740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1faec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.965+0000 7f99bf763740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1f7370 0x556d6b26fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.965+0000 7f99bf763740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99b81ae030 con 0x556d6b1edb40 2024-09-19T08:20:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.965+0000 7f99bf763740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99b81af4a0 con 0x556d6b1ec5d0 2024-09-19T08:20:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.965+0000 7f99bf763740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99b81ae700 con 0x556d6b1f7370 2024-09-19T08:20:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.965+0000 7f99bf763740 10 client.0 _ll_register_callbacks cb 0x7f99b009b890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.965+0000 7f99b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1ec5d0 0x556d6b1fa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.965+0000 7f99b77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1f7370 0x556d6b26fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.965+0000 7f99b67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.965+0000 7f99b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1ec5d0 0x556d6b1fa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:49930/0 (socket says 172.21.15.22:49930) 2024-09-19T08:20:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.965+0000 7f99b6ffd640 1 -- 172.21.15.22:0/2665435127 learned_addr learned my addr 172.21.15.22:0/2665435127 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.966+0000 7f99b6ffd640 1 -- 172.21.15.22:0/2665435127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1f7370 msgr2=0x556d6b26fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.966+0000 7f99b6ffd640 1 --2- 172.21.15.22:0/2665435127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1f7370 0x556d6b26fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.966+0000 7f99b6ffd640 1 -- 172.21.15.22:0/2665435127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 msgr2=0x556d6b1faec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.966+0000 7f99b6ffd640 1 --2- 172.21.15.22:0/2665435127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1faec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.966+0000 7f99b6ffd640 1 -- 172.21.15.22:0/2665435127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d6b2701e0 con 0x556d6b1ec5d0 2024-09-19T08:20:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.966+0000 7f99b67fc640 1 --2- 172.21.15.22:0/2665435127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:20:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.966+0000 7f99b6ffd640 1 --2- 172.21.15.22:0/2665435127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1ec5d0 0x556d6b1fa980 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009600 tx=0x7f99a802bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=435cbc856d47d923 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.966+0000 7f999e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:20:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.966+0000 7f999e7fc640 1 -- 172.21.15.22:0/2665435127 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a802b820 con 0x556d6b1ec5d0 2024-09-19T08:20:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.966+0000 7f99bf763740 1 -- 172.21.15.22:0/2665435127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99a802b470 con 0x556d6b1ec5d0 2024-09-19T08:20:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.966+0000 7f999e7fc640 1 -- 172.21.15.22:0/2665435127 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f99a802b9c0 con 0x556d6b1ec5d0 2024-09-19T08:20:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.966+0000 7f999e7fc640 1 -- 172.21.15.22:0/2665435127 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f99a8020900 con 0x556d6b1ec5d0 2024-09-19T08:20:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.966+0000 7f99bf763740 10 client.26084 fetch_fsmap learned FSMap version 30 2024-09-19T08:20:51.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.966+0000 7f99bf763740 1 -- 172.21.15.22:0/2665435127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d6b2714e0 con 0x556d6b1ec5d0 2024-09-19T08:20:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.967+0000 7f999e7fc640 1 -- 172.21.15.22:0/2665435127 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a802b470 con 0x556d6b1ec5d0 2024-09-19T08:20:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.967+0000 7f999e7fc640 1 -- 172.21.15.22:0/2665435127 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f99a8046050 con 0x556d6b1ec5d0 2024-09-19T08:20:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.967+0000 7f99bf763740 10 client.26084 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:20:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.967+0000 7f99bf763740 10 client.26084 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:20:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.967+0000 7f99bf763740 20 client.26084 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.967+0000 7f99bf763740 1 --2- 172.21.15.22:0/2665435127 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x556d6b206fb0 0x556d6b273c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.967+0000 7f99bf763740 4 client.26084 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:20:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.967+0000 7f99bf763740 1 -- 172.21.15.22:0/2665435127 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d6b159d70 con 0x556d6b206fb0 2024-09-19T08:20:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.967+0000 7f99b67fc640 1 --2- 172.21.15.22:0/2665435127 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x556d6b206fb0 0x556d6b273c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.968+0000 7f99b67fc640 1 --2- 172.21.15.22:0/2665435127 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x556d6b206fb0 0x556d6b273c80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=718e6c1f3480daaf server_cookie=eeef067b616cd96c in_seq=0 out_seq=0 2024-09-19T08:20:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.968+0000 7f999e7fc640 10 client.26084 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:20:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f999e7fc640 1 -- 172.21.15.22:0/2665435127 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d6b159d70 con 0x556d6b206fb0 2024-09-19T08:20:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f999e7fc640 10 client.26084 handle_command_reply: tid=0 2024-09-19T08:20:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f99bf763740 1 client.26084 shutdown 2024-09-19T08:20:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f99bf763740 1 -- 172.21.15.22:0/2665435127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1ec5d0 msgr2=0x556d6b1fa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f99bf763740 1 --2- 172.21.15.22:0/2665435127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1ec5d0 0x556d6b1fa980 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009600 tx=0x7f99a802bcb0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f99bf763740 1 -- 172.21.15.22:0/2665435127 shutdown_connections 2024-09-19T08:20:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f99bf763740 1 --2- 172.21.15.22:0/2665435127 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x556d6b206fb0 0x556d6b273c80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f99bf763740 1 --2- 172.21.15.22:0/2665435127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556d6b1f7370 0x556d6b26fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f999e7fc640 0 client.26084 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:20:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f99bf763740 1 --2- 172.21.15.22:0/2665435127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d6b1edb40 0x556d6b1faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f99bf763740 1 --2- 172.21.15.22:0/2665435127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556d6b1ec5d0 0x556d6b1fa980 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f99bf763740 1 -- 172.21.15.22:0/2665435127 >> 172.21.15.22:0/2665435127 conn(0x556d6b1b5310 msgr2=0x556d6b1f3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f99bf763740 1 -- 172.21.15.22:0/2665435127 shutdown_connections 2024-09-19T08:20:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f99bf763740 1 -- 172.21.15.22:0/2665435127 wait complete. 2024-09-19T08:20:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.969+0000 7f99bf763740 20 client.26084 trim_cache size 0 max 16384 2024-09-19T08:20:51.994 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:20:51.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.970+0000 7f99bd38a640 1 -- 172.21.15.22:0/2053175959 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f99a40a3430 msgr2=0x7f99a40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:51.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.970+0000 7f99bd38a640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f99a40a3430 0x7f99a40a58f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x556d6b1e9ac0 tx=0x7f99a8009500 comp rx=0 tx=0).stop 2024-09-19T08:20:51.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.970+0000 7f99bd38a640 1 -- 172.21.15.22:0/2053175959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b8106f20 msgr2=0x7f99b8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.970+0000 7f99bd38a640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b8106f20 0x7f99b8074f70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f99b000e970 tx=0x7f99b000ee30 comp rx=0 tx=0).stop 2024-09-19T08:20:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.970+0000 7f99bd38a640 1 -- 172.21.15.22:0/2053175959 shutdown_connections 2024-09-19T08:20:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.970+0000 7f99bd38a640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f99a40a3430 0x7f99a40a58f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.970+0000 7f99bd38a640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b8106f20 0x7f99b8074f70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.970+0000 7f99bd38a640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b81027a0 0x7f99b8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.970+0000 7f99bd38a640 1 --2- 172.21.15.22:0/2053175959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99b8101df0 0x7f99b80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.970+0000 7f99bd38a640 1 -- 172.21.15.22:0/2053175959 >> 172.21.15.22:0/2053175959 conn(0x7f99b80fd8f0 msgr2=0x7f99b80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.970+0000 7f99bd38a640 1 -- 172.21.15.22:0/2053175959 shutdown_connections 2024-09-19T08:20:51.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:51.970+0000 7f99bd38a640 1 -- 172.21.15.22:0/2053175959 wait complete. 2024-09-19T08:20:53.035 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:20:53.071 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:52 smithi022 ceph-mon[30963]: pgmap v178: 97 pgs: 97 active+clean; 639 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 339 op/s 2024-09-19T08:20:53.139 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:52 smithi064 ceph-mon[34900]: pgmap v178: 97 pgs: 97 active+clean; 639 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 339 op/s 2024-09-19T08:20:53.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:52 smithi157 ceph-mon[35143]: pgmap v178: 97 pgs: 97 active+clean; 639 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 339 op/s 2024-09-19T08:20:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.261+0000 7fecc7d38640 1 -- 172.21.15.22:0/2865629685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc01012c0 msgr2=0x7fecc0105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.261+0000 7fecc7d38640 1 --2- 172.21.15.22:0/2865629685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc01012c0 0x7fecc0105790 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fecb0009860 tx=0x7fecb002a400 comp rx=0 tx=0).stop 2024-09-19T08:20:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.262+0000 7fecc7d38640 1 -- 172.21.15.22:0/2865629685 shutdown_connections 2024-09-19T08:20:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.262+0000 7fecc7d38640 1 --2- 172.21.15.22:0/2865629685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecc0105db0 0x7fecc0106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.262+0000 7fecc7d38640 1 --2- 172.21.15.22:0/2865629685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc01012c0 0x7fecc0105790 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.262+0000 7fecc7d38640 1 --2- 172.21.15.22:0/2865629685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc01009a0 0x7fecc0100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.262+0000 7fecc7d38640 1 -- 172.21.15.22:0/2865629685 >> 172.21.15.22:0/2865629685 conn(0x7fecc00fc630 msgr2=0x7fecc00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:54.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.265+0000 7fecc7d38640 1 -- 172.21.15.22:0/2865629685 shutdown_connections 2024-09-19T08:20:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.269+0000 7fecc7d38640 1 -- 172.21.15.22:0/2865629685 wait complete. 2024-09-19T08:20:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.270+0000 7fecc7d38640 1 Processor -- start 2024-09-19T08:20:54.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.270+0000 7fecc7d38640 1 -- start start 2024-09-19T08:20:54.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.270+0000 7fecc7d38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecc01009a0 0x7fecc0073a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.270+0000 7fecc7d38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc01012c0 0x7fecc006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.270+0000 7fecc7d38640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc0105db0 0x7fecc006d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.270+0000 7fecc7d38640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0118450 con 0x7fecc01012c0 2024-09-19T08:20:54.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.270+0000 7fecc7d38640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01182d0 con 0x7fecc01009a0 2024-09-19T08:20:54.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.270+0000 7fecc7d38640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01185d0 con 0x7fecc0105db0 2024-09-19T08:20:54.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.271+0000 7fecc5ad4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecc01009a0 0x7fecc0073a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:54.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.271+0000 7fecc5ad4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecc01009a0 0x7fecc0073a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:49956/0 (socket says 172.21.15.22:49956) 2024-09-19T08:20:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.271+0000 7fecc5ad4640 1 -- 172.21.15.22:0/342009559 learned_addr learned my addr 172.21.15.22:0/342009559 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.274+0000 7fecc5ad4640 1 -- 172.21.15.22:0/342009559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc0105db0 msgr2=0x7fecc006d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.274+0000 7fecc62d5640 1 --2- 172.21.15.22:0/342009559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc0105db0 0x7fecc006d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.274+0000 7fecc5ad4640 1 --2- 172.21.15.22:0/342009559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc0105db0 0x7fecc006d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.274+0000 7fecc5ad4640 1 -- 172.21.15.22:0/342009559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc01012c0 msgr2=0x7fecc006caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.274+0000 7fecc5ad4640 1 --2- 172.21.15.22:0/342009559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc01012c0 0x7fecc006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.274+0000 7fecc5ad4640 1 -- 172.21.15.22:0/342009559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc006d670 con 0x7fecc01009a0 2024-09-19T08:20:54.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.278+0000 7fecc5ad4640 1 --2- 172.21.15.22:0/342009559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecc01009a0 0x7fecc0073a60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fecb800efb0 tx=0x7fecb800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:54.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.283+0000 7fecb6ffd640 1 -- 172.21.15.22:0/342009559 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb801b070 con 0x7fecc01009a0 2024-09-19T08:20:54.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.283+0000 7fecc7d38640 1 -- 172.21.15.22:0/342009559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc0071590 con 0x7fecc01009a0 2024-09-19T08:20:54.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.283+0000 7fecc7d38640 1 -- 172.21.15.22:0/342009559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc0071ad0 con 0x7fecc01009a0 2024-09-19T08:20:54.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.283+0000 7fecb6ffd640 1 -- 172.21.15.22:0/342009559 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb800ba50 con 0x7fecc01009a0 2024-09-19T08:20:54.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.283+0000 7fecb6ffd640 1 -- 172.21.15.22:0/342009559 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8016c00 con 0x7fecc01009a0 2024-09-19T08:20:54.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.295+0000 7fecca111740 1 Processor -- start 2024-09-19T08:20:54.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.299+0000 7fecca111740 1 -- start start 2024-09-19T08:20:54.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.299+0000 7fecb6ffd640 1 -- 172.21.15.22:0/342009559 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fecb801e070 con 0x7fecc01009a0 2024-09-19T08:20:54.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.302+0000 7fecca111740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ebccf0 0x556f33ebb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.302+0000 7fecca111740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebb780 0x556f33ebbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.302+0000 7fecca111740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ec6520 0x556f33ec8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.302+0000 7fecca111740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556f33e212f0 con 0x556f33ec6520 2024-09-19T08:20:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.302+0000 7fecca111740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556f33dc8cf0 con 0x556f33ebccf0 2024-09-19T08:20:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.302+0000 7fecca111740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556f33e1d2b0 con 0x556f33ebb780 2024-09-19T08:20:54.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.304+0000 7fecc62d5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ebccf0 0x556f33ebb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:54.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.305+0000 7fecc52d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ec6520 0x556f33ec8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:54.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.305+0000 7fecc52d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ec6520 0x556f33ec8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47226/0 (socket says 172.21.15.22:47226) 2024-09-19T08:20:54.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.305+0000 7fecc52d3640 1 -- 172.21.15.22:0/2154908002 learned_addr learned my addr 172.21.15.22:0/2154908002 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:54.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.306+0000 7fecc52d3640 1 -- 172.21.15.22:0/2154908002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebb780 msgr2=0x556f33ebbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:54.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.307+0000 7fecb6ffd640 1 --2- 172.21.15.22:0/342009559 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fecac0a3580 0x7fecac0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.307+0000 7fecc5ad4640 1 --2- 172.21.15.22:0/2154908002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebb780 0x556f33ebbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:54.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.308+0000 7fecb6ffd640 1 -- 172.21.15.22:0/342009559 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fecb80cfbd0 con 0x7fecc01009a0 2024-09-19T08:20:54.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.310+0000 7fecc52d3640 1 --2- 172.21.15.22:0/2154908002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebb780 0x556f33ebbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.310+0000 7fecc52d3640 1 -- 172.21.15.22:0/2154908002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ebccf0 msgr2=0x556f33ebb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:54.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.310+0000 7fecc52d3640 1 --2- 172.21.15.22:0/2154908002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ebccf0 0x556f33ebb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.310+0000 7fecc52d3640 1 -- 172.21.15.22:0/2154908002 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f33dad3a0 con 0x556f33ec6520 2024-09-19T08:20:54.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.310+0000 7fecc5ad4640 1 --2- 172.21.15.22:0/2154908002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebb780 0x556f33ebbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:20:54.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.310+0000 7fecc62d5640 1 --2- 172.21.15.22:0/2154908002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ebccf0 0x556f33ebb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:20:54.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.310+0000 7fecc52d3640 1 --2- 172.21.15.22:0/2154908002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ec6520 0x556f33ec8910 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fecb002a920 tx=0x7fecb0030ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efafbd5136b0f468 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.310+0000 7fecc62d5640 1 --2- 172.21.15.22:0/342009559 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fecac0a3580 0x7fecac0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.311+0000 7fecabfff640 1 -- 172.21.15.22:0/2154908002 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0040070 con 0x556f33ec6520 2024-09-19T08:20:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.311+0000 7fecabfff640 1 -- 172.21.15.22:0/2154908002 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb0030e40 con 0x556f33ec6520 2024-09-19T08:20:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.311+0000 7fecabfff640 1 -- 172.21.15.22:0/2154908002 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb00054e0 con 0x556f33ec6520 2024-09-19T08:20:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.313+0000 7fecca111740 1 -- 172.21.15.22:0/2154908002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ec6520 msgr2=0x556f33ec8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.313+0000 7fecca111740 1 --2- 172.21.15.22:0/2154908002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ec6520 0x556f33ec8910 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fecb002a920 tx=0x7fecb0030ab0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.313+0000 7fecc62d5640 1 --2- 172.21.15.22:0/342009559 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fecac0a3580 0x7fecac0a5a40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fecbc006fd0 tx=0x7fecbc004750 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:54.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.315+0000 7fecca111740 1 -- 172.21.15.22:0/2154908002 shutdown_connections 2024-09-19T08:20:54.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.315+0000 7fecca111740 1 --2- 172.21.15.22:0/2154908002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ec6520 0x556f33ec8910 secure :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fecb002a920 tx=0x7fecb0030ab0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.315+0000 7fecca111740 1 --2- 172.21.15.22:0/2154908002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebb780 0x556f33ebbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.315+0000 7fecca111740 1 --2- 172.21.15.22:0/2154908002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ebccf0 0x556f33ebb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.315+0000 7fecca111740 1 -- 172.21.15.22:0/2154908002 >> 172.21.15.22:0/2154908002 conn(0x556f33e84310 msgr2=0x556f33eb7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:54.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.319+0000 7fecca111740 1 -- 172.21.15.22:0/2154908002 shutdown_connections 2024-09-19T08:20:54.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.320+0000 7fecca111740 1 -- 172.21.15.22:0/2154908002 wait complete. 2024-09-19T08:20:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.320+0000 7fecca111740 1 -- start start 2024-09-19T08:20:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.320+0000 7fecca111740 1 Processor -- start 2024-09-19T08:20:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.321+0000 7fecca111740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebb780 0x556f33ec32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.321+0000 7fecca111740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ebccf0 0x556f33ec37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.321+0000 7fecca111740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ec3d30 0x556f33ed26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.321+0000 7fecca111740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556f33ec91c0 con 0x556f33ebccf0 2024-09-19T08:20:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.321+0000 7fecca111740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556f33ddc060 con 0x556f33ec3d30 2024-09-19T08:20:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.321+0000 7fecca111740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556f33ec9340 con 0x556f33ebb780 2024-09-19T08:20:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.321+0000 7fecca111740 10 client.0 _ll_register_callbacks cb 0x556f33b2f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.323+0000 7fecc62d5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ec3d30 0x556f33ed26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.324+0000 7fecc62d5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ec3d30 0x556f33ed26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:49960/0 (socket says 172.21.15.22:49960) 2024-09-19T08:20:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.324+0000 7fecc62d5640 1 -- 172.21.15.22:0/2949668786 learned_addr learned my addr 172.21.15.22:0/2949668786 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.324+0000 7fecc52d3640 1 --2- 172.21.15.22:0/2949668786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ebccf0 0x556f33ec37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.324+0000 7fecc52d3640 1 -- 172.21.15.22:0/2949668786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebb780 msgr2=0x556f33ec32b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:20:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.324+0000 7fecc52d3640 1 --2- 172.21.15.22:0/2949668786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebb780 0x556f33ec32b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.325+0000 7fecc52d3640 1 -- 172.21.15.22:0/2949668786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ec3d30 msgr2=0x556f33ed26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.325+0000 7fecc52d3640 1 --2- 172.21.15.22:0/2949668786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ec3d30 0x556f33ed26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.325+0000 7fecc52d3640 1 -- 172.21.15.22:0/2949668786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f33b923c0 con 0x556f33ebccf0 2024-09-19T08:20:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.325+0000 7fecc52d3640 1 --2- 172.21.15.22:0/2949668786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ebccf0 0x556f33ec37f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x556f33ebd160 tx=0x7fecb0004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4309dbe7ec6252c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:54.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.328+0000 7feca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:54.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.328+0000 7feca97fa640 1 -- 172.21.15.22:0/2949668786 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0040070 con 0x556f33ebccf0 2024-09-19T08:20:54.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.328+0000 7fecca111740 1 -- 172.21.15.22:0/2949668786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb0030e40 con 0x556f33ebccf0 2024-09-19T08:20:54.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.328+0000 7feca97fa640 1 -- 172.21.15.22:0/2949668786 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fecb00303a0 con 0x556f33ebccf0 2024-09-19T08:20:54.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.328+0000 7feca97fa640 1 -- 172.21.15.22:0/2949668786 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fecb0055050 con 0x556f33ebccf0 2024-09-19T08:20:54.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.329+0000 7fecca111740 10 client.16539 fetch_fsmap learned FSMap version 30 2024-09-19T08:20:54.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.329+0000 7fecca111740 1 -- 172.21.15.22:0/2949668786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556f33ee6100 con 0x556f33ebccf0 2024-09-19T08:20:54.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.330+0000 7feca97fa640 1 -- 172.21.15.22:0/2949668786 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb0030e40 con 0x556f33ebccf0 2024-09-19T08:20:54.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.330+0000 7feca97fa640 1 -- 172.21.15.22:0/2949668786 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fecb00035f0 con 0x556f33ebccf0 2024-09-19T08:20:54.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.330+0000 7fecca111740 10 client.16539 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:20:54.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.330+0000 7fecca111740 10 client.16539 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:20:54.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.331+0000 7fecca111740 20 client.16539 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:54.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.331+0000 7fecca111740 1 --2- 172.21.15.22:0/2949668786 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x556f33ee75b0 0x556f33ee9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.331+0000 7fecca111740 4 client.16539 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:20:54.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.331+0000 7fecca111740 1 -- 172.21.15.22:0/2949668786 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f33e2e860 con 0x556f33ee75b0 2024-09-19T08:20:54.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.332+0000 7fecc5ad4640 1 --2- 172.21.15.22:0/2949668786 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x556f33ee75b0 0x556f33ee9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:54.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.336+0000 7feca97fa640 10 client.16539 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:20:54.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.336+0000 7fecc5ad4640 1 --2- 172.21.15.22:0/2949668786 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x556f33ee75b0 0x556f33ee9a10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1261c1275619606d server_cookie=712c43075d19f300 in_seq=0 out_seq=0 2024-09-19T08:20:54.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.339+0000 7feca97fa640 1 -- 172.21.15.22:0/2949668786 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f33e2e860 con 0x556f33ee75b0 2024-09-19T08:20:54.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.339+0000 7feca97fa640 10 client.16539 handle_command_reply: tid=0 2024-09-19T08:20:54.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.339+0000 7fecca111740 1 client.16539 shutdown 2024-09-19T08:20:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.345+0000 7fecca111740 1 -- 172.21.15.22:0/2949668786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ebccf0 msgr2=0x556f33ec37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.345+0000 7fecca111740 1 --2- 172.21.15.22:0/2949668786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ebccf0 0x556f33ec37f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x556f33ebd160 tx=0x7fecb0004280 comp rx=0 tx=0).stop 2024-09-19T08:20:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.346+0000 7fecca111740 1 -- 172.21.15.22:0/2949668786 shutdown_connections 2024-09-19T08:20:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.346+0000 7fecca111740 1 --2- 172.21.15.22:0/2949668786 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x556f33ee75b0 0x556f33ee9a10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.346+0000 7fecca111740 1 --2- 172.21.15.22:0/2949668786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ec3d30 0x556f33ed26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.346+0000 7fecca111740 1 --2- 172.21.15.22:0/2949668786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ebccf0 0x556f33ec37f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.346+0000 7fecca111740 1 --2- 172.21.15.22:0/2949668786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebb780 0x556f33ec32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.346+0000 7fecca111740 1 -- 172.21.15.22:0/2949668786 >> 172.21.15.22:0/2949668786 conn(0x556f33e84310 msgr2=0x556f33ebdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.346+0000 7feca97fa640 0 client.16539 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:20:54.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.352+0000 7fecca111740 1 -- 172.21.15.22:0/2949668786 shutdown_connections 2024-09-19T08:20:54.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.352+0000 7fecca111740 1 -- 172.21.15.22:0/2949668786 wait complete. 2024-09-19T08:20:54.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.353+0000 7fecca111740 20 client.16539 trim_cache size 0 max 16384 2024-09-19T08:20:54.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.387+0000 7fecca111740 1 Processor -- start 2024-09-19T08:20:54.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.387+0000 7fecca111740 1 -- start start 2024-09-19T08:20:54.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.387+0000 7fecca111740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ebb780 0x556f33ed24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.387+0000 7fecca111740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebccf0 0x556f33ec2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.387+0000 7fecca111740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ec3d30 0x556f33eef320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.388+0000 7fecc52d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebccf0 0x556f33ec2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:54.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.388+0000 7fecc52d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebccf0 0x556f33ec2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:47716/0 (socket says 172.21.15.22:47716) 2024-09-19T08:20:54.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.388+0000 7fecc62d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ec3d30 0x556f33eef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:54.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.388+0000 7fecc62d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ec3d30 0x556f33eef320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47256/0 (socket says 172.21.15.22:47256) 2024-09-19T08:20:54.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.388+0000 7fecc52d3640 1 -- 172.21.15.22:0/2861720598 learned_addr learned my addr 172.21.15.22:0/2861720598 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:54.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.388+0000 7fecca111740 1 -- 172.21.15.22:0/2861720598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feca4011140 con 0x556f33ec3d30 2024-09-19T08:20:54.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.388+0000 7fecca111740 1 -- 172.21.15.22:0/2861720598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feca400e9e0 con 0x556f33ebb780 2024-09-19T08:20:54.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.388+0000 7fecca111740 1 -- 172.21.15.22:0/2861720598 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feca400c830 con 0x556f33ebccf0 2024-09-19T08:20:54.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.389+0000 7fecc52d3640 1 -- 172.21.15.22:0/2861720598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ebb780 msgr2=0x556f33ed24c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:20:54.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.389+0000 7fecc52d3640 1 --2- 172.21.15.22:0/2861720598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ebb780 0x556f33ed24c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.389+0000 7fecc52d3640 1 -- 172.21.15.22:0/2861720598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ec3d30 msgr2=0x556f33eef320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:54.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.389+0000 7fecc52d3640 1 --2- 172.21.15.22:0/2861720598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ec3d30 0x556f33eef320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.389+0000 7fecc52d3640 1 -- 172.21.15.22:0/2861720598 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f33eb8b10 con 0x556f33ebccf0 2024-09-19T08:20:54.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.390+0000 7fecc52d3640 1 --2- 172.21.15.22:0/2861720598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebccf0 0x556f33ec2290 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fecb002a920 tx=0x7fecb0001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=451f64908b8a50c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:54.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.390+0000 7fecab7fe640 1 -- 172.21.15.22:0/2861720598 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb005f070 con 0x556f33ebccf0 2024-09-19T08:20:54.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.390+0000 7fecab7fe640 1 -- 172.21.15.22:0/2861720598 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb0005850 con 0x556f33ebccf0 2024-09-19T08:20:54.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.390+0000 7fecab7fe640 1 -- 172.21.15.22:0/2861720598 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0040040 con 0x556f33ebccf0 2024-09-19T08:20:54.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.395+0000 7fecca111740 1 -- 172.21.15.22:0/2861720598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebccf0 msgr2=0x556f33ec2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:54.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.395+0000 7fecca111740 1 --2- 172.21.15.22:0/2861720598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebccf0 0x556f33ec2290 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fecb002a920 tx=0x7fecb0001e50 comp rx=0 tx=0).stop 2024-09-19T08:20:54.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.397+0000 7fecca111740 1 -- 172.21.15.22:0/2861720598 shutdown_connections 2024-09-19T08:20:54.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.397+0000 7fecca111740 1 --2- 172.21.15.22:0/2861720598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ec3d30 0x556f33eef320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.397+0000 7fecca111740 1 --2- 172.21.15.22:0/2861720598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebccf0 0x556f33ec2290 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.397+0000 7fecca111740 1 --2- 172.21.15.22:0/2861720598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ebb780 0x556f33ed24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.397+0000 7fecca111740 1 -- 172.21.15.22:0/2861720598 >> 172.21.15.22:0/2861720598 conn(0x556f33e84310 msgr2=0x556f33ec1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:54.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.398+0000 7fecca111740 1 -- 172.21.15.22:0/2861720598 shutdown_connections 2024-09-19T08:20:54.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.400+0000 7fecca111740 1 -- 172.21.15.22:0/2861720598 wait complete. 2024-09-19T08:20:54.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.401+0000 7fecca111740 1 -- start start 2024-09-19T08:20:54.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.401+0000 7fecca111740 1 Processor -- start 2024-09-19T08:20:54.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.405+0000 7fecca111740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ebb780 0x556f33eceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.405+0000 7fecca111740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebccf0 0x556f33eca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.405+0000 7fecca111740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ec3d30 0x556f33eca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.405+0000 7fecca111740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feca400cea0 con 0x556f33ebb780 2024-09-19T08:20:54.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.405+0000 7fecca111740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feca400e310 con 0x556f33ec3d30 2024-09-19T08:20:54.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.405+0000 7fecca111740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feca400d570 con 0x556f33ebccf0 2024-09-19T08:20:54.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.405+0000 7fecca111740 10 client.0 _ll_register_callbacks cb 0x7fecb8009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:54.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.405+0000 7fecc52d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebccf0 0x556f33eca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:54.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.405+0000 7fecc52d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebccf0 0x556f33eca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:47730/0 (socket says 172.21.15.22:47730) 2024-09-19T08:20:54.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.405+0000 7fecc52d3640 1 -- 172.21.15.22:0/4091984253 learned_addr learned my addr 172.21.15.22:0/4091984253 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:54.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.405+0000 7fecc5ad4640 1 --2- 172.21.15.22:0/4091984253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ebb780 0x556f33eceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:54.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.406+0000 7fecc5ad4640 1 -- 172.21.15.22:0/4091984253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebccf0 msgr2=0x556f33eca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:54.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.406+0000 7fecc5ad4640 1 --2- 172.21.15.22:0/4091984253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebccf0 0x556f33eca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.406+0000 7fecc5ad4640 1 -- 172.21.15.22:0/4091984253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ec3d30 msgr2=0x556f33eca7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:20:54.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.406+0000 7fecc5ad4640 1 --2- 172.21.15.22:0/4091984253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ec3d30 0x556f33eca7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.406+0000 7fecc5ad4640 1 -- 172.21.15.22:0/4091984253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f33ebe2d0 con 0x556f33ebb780 2024-09-19T08:20:54.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.406+0000 7fecc5ad4640 1 --2- 172.21.15.22:0/4091984253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ebb780 0x556f33eceeb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fecb80cd2d0 tx=0x7fecb8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d1557ecd43c047 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:54.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.406+0000 7feca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:54.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.407+0000 7feca97fa640 1 -- 172.21.15.22:0/4091984253 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80a4ad0 con 0x556f33ebb780 2024-09-19T08:20:54.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.407+0000 7feca97fa640 1 -- 172.21.15.22:0/4091984253 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fecb80cd550 con 0x556f33ebb780 2024-09-19T08:20:54.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.407+0000 7fecca111740 1 -- 172.21.15.22:0/4091984253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb0005850 con 0x556f33ebb780 2024-09-19T08:20:54.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.409+0000 7feca97fa640 1 -- 172.21.15.22:0/4091984253 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fecb80984d0 con 0x556f33ebb780 2024-09-19T08:20:54.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.410+0000 7fecca111740 10 client.16545 fetch_fsmap learned FSMap version 30 2024-09-19T08:20:54.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.410+0000 7fecca111740 1 -- 172.21.15.22:0/4091984253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556f33f52260 con 0x556f33ebb780 2024-09-19T08:20:54.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.411+0000 7feca97fa640 1 -- 172.21.15.22:0/4091984253 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb0005850 con 0x556f33ebb780 2024-09-19T08:20:54.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.411+0000 7feca97fa640 1 -- 172.21.15.22:0/4091984253 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fecb800b040 con 0x556f33ebb780 2024-09-19T08:20:54.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.412+0000 7fecca111740 10 client.16545 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:20:54.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.412+0000 7fecca111740 10 client.16545 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:20:54.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.412+0000 7fecca111740 20 client.16545 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:54.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.412+0000 7fecca111740 1 --2- 172.21.15.22:0/4091984253 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x556f33f52960 0x556f33f54d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:54.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.412+0000 7fecca111740 4 client.16545 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:20:54.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.412+0000 7fecca111740 1 -- 172.21.15.22:0/4091984253 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f33e28d70 con 0x556f33f52960 2024-09-19T08:20:54.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.413+0000 7fecc52d3640 1 --2- 172.21.15.22:0/4091984253 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x556f33f52960 0x556f33f54d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:54.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.415+0000 7fecc52d3640 1 --2- 172.21.15.22:0/4091984253 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x556f33f52960 0x556f33f54d80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=acbce50d17d29b22 server_cookie=2d3923dbcfdea348 in_seq=0 out_seq=0 2024-09-19T08:20:54.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.416+0000 7feca97fa640 10 client.16545 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:20:54.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.416+0000 7feca97fa640 1 -- 172.21.15.22:0/4091984253 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f33e28d70 con 0x556f33f52960 2024-09-19T08:20:54.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.416+0000 7feca97fa640 10 client.16545 handle_command_reply: tid=0 2024-09-19T08:20:54.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.417+0000 7fecca111740 1 client.16545 shutdown 2024-09-19T08:20:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.418+0000 7fecca111740 1 -- 172.21.15.22:0/4091984253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ebb780 msgr2=0x556f33eceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.418+0000 7fecca111740 1 --2- 172.21.15.22:0/4091984253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ebb780 0x556f33eceeb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fecb80cd2d0 tx=0x7fecb8009650 comp rx=0 tx=0).stop 2024-09-19T08:20:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.420+0000 7fecca111740 1 -- 172.21.15.22:0/4091984253 shutdown_connections 2024-09-19T08:20:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.420+0000 7fecca111740 1 --2- 172.21.15.22:0/4091984253 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x556f33f52960 0x556f33f54d80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.420+0000 7fecca111740 1 --2- 172.21.15.22:0/4091984253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f33ec3d30 0x556f33eca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.420+0000 7fecca111740 1 --2- 172.21.15.22:0/4091984253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f33ebccf0 0x556f33eca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.420+0000 7fecca111740 1 --2- 172.21.15.22:0/4091984253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f33ebb780 0x556f33eceeb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.420+0000 7fecca111740 1 -- 172.21.15.22:0/4091984253 >> 172.21.15.22:0/4091984253 conn(0x556f33e84310 msgr2=0x556f33ec23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.421+0000 7feca97fa640 0 client.16545 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:20:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.421+0000 7fecca111740 1 -- 172.21.15.22:0/4091984253 shutdown_connections 2024-09-19T08:20:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.421+0000 7fecca111740 1 -- 172.21.15.22:0/4091984253 wait complete. 2024-09-19T08:20:54.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.421+0000 7fecca111740 20 client.16545 trim_cache size 0 max 16384 2024-09-19T08:20:54.425 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:20:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.426+0000 7fecc7d38640 1 -- 172.21.15.22:0/342009559 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fecac0a3580 msgr2=0x7fecac0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.426+0000 7fecc7d38640 1 --2- 172.21.15.22:0/342009559 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fecac0a3580 0x7fecac0a5a40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fecbc006fd0 tx=0x7fecbc004750 comp rx=0 tx=0).stop 2024-09-19T08:20:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.426+0000 7fecc7d38640 1 -- 172.21.15.22:0/342009559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecc01009a0 msgr2=0x7fecc0073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.426+0000 7fecc7d38640 1 --2- 172.21.15.22:0/342009559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecc01009a0 0x7fecc0073a60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fecb800efb0 tx=0x7fecb800c800 comp rx=0 tx=0).stop 2024-09-19T08:20:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.428+0000 7fecc7d38640 1 -- 172.21.15.22:0/342009559 shutdown_connections 2024-09-19T08:20:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.428+0000 7fecc7d38640 1 --2- 172.21.15.22:0/342009559 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fecac0a3580 0x7fecac0a5a40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.428+0000 7fecc7d38640 1 --2- 172.21.15.22:0/342009559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc0105db0 0x7fecc006d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.428+0000 7fecc7d38640 1 --2- 172.21.15.22:0/342009559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecc01012c0 0x7fecc006caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.428+0000 7fecc7d38640 1 --2- 172.21.15.22:0/342009559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecc01009a0 0x7fecc0073a60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.428+0000 7fecc7d38640 1 -- 172.21.15.22:0/342009559 >> 172.21.15.22:0/342009559 conn(0x7fecc00fc630 msgr2=0x7fecc00fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.428+0000 7fecc7d38640 1 -- 172.21.15.22:0/342009559 shutdown_connections 2024-09-19T08:20:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:54.428+0000 7fecc7d38640 1 -- 172.21.15.22:0/342009559 wait complete. 2024-09-19T08:20:55.152 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:54 smithi022 ceph-mon[30963]: pgmap v179: 97 pgs: 97 active+clean; 695 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.9 MiB/s wr, 681 op/s 2024-09-19T08:20:55.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:54 smithi157 ceph-mon[35143]: pgmap v179: 97 pgs: 97 active+clean; 695 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.9 MiB/s wr, 681 op/s 2024-09-19T08:20:55.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:54 smithi064 ceph-mon[34900]: pgmap v179: 97 pgs: 97 active+clean; 695 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.9 MiB/s wr, 681 op/s 2024-09-19T08:20:55.685 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:20:56.156 INFO:tasks.workunit.client.0.smithi022.stdout: 124 1175455 356 747072 309 644638 1035 2024-09-19T08:20:56.281 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:20:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.005+0000 7f2be93b6640 1 -- 172.21.15.22:0/3423186953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be4110220 msgr2=0x7f2be406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.005+0000 7f2be93b6640 1 --2- 172.21.15.22:0/3423186953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be4110220 0x7f2be406e4f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009ca0 tx=0x7f2bdc02a8f0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.009+0000 7f2be93b6640 1 -- 172.21.15.22:0/3423186953 shutdown_connections 2024-09-19T08:20:57.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.009+0000 7f2be93b6640 1 --2- 172.21.15.22:0/3423186953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be411b140 0x7f2be411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.009+0000 7f2be93b6640 1 --2- 172.21.15.22:0/3423186953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2be406ea30 0x7f2be406ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.009+0000 7f2be93b6640 1 --2- 172.21.15.22:0/3423186953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be4110220 0x7f2be406e4f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.009+0000 7f2be93b6640 1 -- 172.21.15.22:0/3423186953 >> 172.21.15.22:0/3423186953 conn(0x7f2be4073210 msgr2=0x7f2be4073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:57.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.013+0000 7f2be93b6640 1 -- 172.21.15.22:0/3423186953 shutdown_connections 2024-09-19T08:20:57.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.017+0000 7f2be93b6640 1 -- 172.21.15.22:0/3423186953 wait complete. 2024-09-19T08:20:57.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.018+0000 7f2be93b6640 1 Processor -- start 2024-09-19T08:20:57.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.018+0000 7f2be93b6640 1 -- start start 2024-09-19T08:20:57.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.018+0000 7f2be93b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be406ea30 0x7f2be4119940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.018+0000 7f2be93b6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be411b140 0x7f2be4112a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.019+0000 7f2be93b6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2be4112fb0 0x7f2be4113460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.019+0000 7f2be93b6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4120e00 con 0x7f2be406ea30 2024-09-19T08:20:57.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.019+0000 7f2be93b6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4120c80 con 0x7f2be4112fb0 2024-09-19T08:20:57.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.019+0000 7f2be93b6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4120f80 con 0x7f2be411b140 2024-09-19T08:20:57.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.021+0000 7f2be2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be406ea30 0x7f2be4119940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.021+0000 7f2be2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be406ea30 0x7f2be4119940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47282/0 (socket says 172.21.15.22:47282) 2024-09-19T08:20:57.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.021+0000 7f2be2ffd640 1 -- 172.21.15.22:0/1000113019 learned_addr learned my addr 172.21.15.22:0/1000113019 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:57.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.022+0000 7f2be2ffd640 1 -- 172.21.15.22:0/1000113019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be411b140 msgr2=0x7f2be4112a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:20:57.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.022+0000 7f2be2ffd640 1 --2- 172.21.15.22:0/1000113019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be411b140 0x7f2be4112a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.022+0000 7f2be2ffd640 1 -- 172.21.15.22:0/1000113019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2be4112fb0 msgr2=0x7f2be4113460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:20:57.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.022+0000 7f2be2ffd640 1 --2- 172.21.15.22:0/1000113019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2be4112fb0 0x7f2be4113460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.022+0000 7f2be2ffd640 1 -- 172.21.15.22:0/1000113019 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be4117520 con 0x7f2be406ea30 2024-09-19T08:20:57.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.028+0000 7f2be2ffd640 1 --2- 172.21.15.22:0/1000113019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be406ea30 0x7f2be4119940 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc02aee0 tx=0x7f2bdc030fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:57.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.029+0000 7f2bcbfff640 1 -- 172.21.15.22:0/1000113019 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc003c70 con 0x7f2be406ea30 2024-09-19T08:20:57.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.029+0000 7f2bcbfff640 1 -- 172.21.15.22:0/1000113019 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bdc02d070 con 0x7f2be406ea30 2024-09-19T08:20:57.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.029+0000 7f2bcbfff640 1 -- 172.21.15.22:0/1000113019 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0047f0 con 0x7f2be406ea30 2024-09-19T08:20:57.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.029+0000 7f2be93b6640 1 -- 172.21.15.22:0/1000113019 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be41177b0 con 0x7f2be406ea30 2024-09-19T08:20:57.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.030+0000 7f2be93b6640 1 -- 172.21.15.22:0/1000113019 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be4117c40 con 0x7f2be406ea30 2024-09-19T08:20:57.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.035+0000 7f2bcbfff640 1 -- 172.21.15.22:0/1000113019 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2bdc004040 con 0x7f2be406ea30 2024-09-19T08:20:57.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.036+0000 7f2bcbfff640 1 --2- 172.21.15.22:0/1000113019 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2bcc0a35c0 0x7f2bcc0a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.037+0000 7f2bcbfff640 1 -- 172.21.15.22:0/1000113019 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2bdc0f11b0 con 0x7f2be406ea30 2024-09-19T08:20:57.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.039+0000 7f2beb78f740 1 Processor -- start 2024-09-19T08:20:57.041 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:56 smithi022 ceph-mon[30963]: pgmap v180: 97 pgs: 97 active+clean; 695 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 536 op/s 2024-09-19T08:20:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.043+0000 7f2be27fc640 1 --2- 172.21.15.22:0/1000113019 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2bcc0a35c0 0x7f2bcc0a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.048+0000 7f2be27fc640 1 --2- 172.21.15.22:0/1000113019 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2bcc0a35c0 0x7f2bcc0a5a80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2be4114190 tx=0x7f2bd8006bd0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:57.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.052+0000 7f2beb78f740 1 -- start start 2024-09-19T08:20:57.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.052+0000 7f2beb78f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0da240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.052+0000 7f2beb78f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0da780 0x5645ee0dabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.052+0000 7f2beb78f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0e5520 0x5645ee0e7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.052+0000 7f2beb78f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5645ee0402f0 con 0x5645ee0e5520 2024-09-19T08:20:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.052+0000 7f2beb78f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5645edfe7cf0 con 0x5645ee0da780 2024-09-19T08:20:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.052+0000 7f2beb78f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5645ee03c2b0 con 0x5645ee0dbcf0 2024-09-19T08:20:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.053+0000 7f2be37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0e5520 0x5645ee0e7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.053+0000 7f2be37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0e5520 0x5645ee0e7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47290/0 (socket says 172.21.15.22:47290) 2024-09-19T08:20:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.053+0000 7f2be37fe640 1 -- 172.21.15.22:0/3768937832 learned_addr learned my addr 172.21.15.22:0/3768937832 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.053+0000 7f2be27fc640 1 --2- 172.21.15.22:0/3768937832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0da780 0x5645ee0dabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.053+0000 7f2be2ffd640 1 --2- 172.21.15.22:0/3768937832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0da240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.054+0000 7f2be37fe640 1 -- 172.21.15.22:0/3768937832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 msgr2=0x5645ee0da240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.054+0000 7f2be37fe640 1 --2- 172.21.15.22:0/3768937832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0da240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.054+0000 7f2be37fe640 1 -- 172.21.15.22:0/3768937832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0da780 msgr2=0x5645ee0dabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:57.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.054+0000 7f2be37fe640 1 --2- 172.21.15.22:0/3768937832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0da780 0x5645ee0dabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.054+0000 7f2be37fe640 1 -- 172.21.15.22:0/3768937832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645edfcc3a0 con 0x5645ee0e5520 2024-09-19T08:20:57.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.054+0000 7f2be37fe640 1 --2- 172.21.15.22:0/3768937832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0e5520 0x5645ee0e7910 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40096e0 tx=0x7f2bd4009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3623f9038ed38058 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:57.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.056+0000 7f2bc97fa640 1 -- 172.21.15.22:0/3768937832 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4015020 con 0x5645ee0e5520 2024-09-19T08:20:57.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.056+0000 7f2bc97fa640 1 -- 172.21.15.22:0/3768937832 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd400bda0 con 0x5645ee0e5520 2024-09-19T08:20:57.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.057+0000 7f2beb78f740 1 -- 172.21.15.22:0/3768937832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0e5520 msgr2=0x5645ee0e7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:57.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.057+0000 7f2beb78f740 1 --2- 172.21.15.22:0/3768937832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0e5520 0x5645ee0e7910 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40096e0 tx=0x7f2bd4009ba0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.063+0000 7f2beb78f740 1 -- 172.21.15.22:0/3768937832 shutdown_connections 2024-09-19T08:20:57.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.063+0000 7f2beb78f740 1 --2- 172.21.15.22:0/3768937832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0e5520 0x5645ee0e7910 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.063+0000 7f2beb78f740 1 --2- 172.21.15.22:0/3768937832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0da780 0x5645ee0dabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.063+0000 7f2beb78f740 1 --2- 172.21.15.22:0/3768937832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0da240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.063+0000 7f2beb78f740 1 -- 172.21.15.22:0/3768937832 >> 172.21.15.22:0/3768937832 conn(0x5645ee0a3310 msgr2=0x5645ee0d6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:57.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.064+0000 7f2beb78f740 1 -- 172.21.15.22:0/3768937832 shutdown_connections 2024-09-19T08:20:57.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.067+0000 7f2beb78f740 1 -- 172.21.15.22:0/3768937832 wait complete. 2024-09-19T08:20:57.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.067+0000 7f2beb78f740 1 -- start start 2024-09-19T08:20:57.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.067+0000 7f2beb78f740 1 Processor -- start 2024-09-19T08:20:57.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.068+0000 7f2beb78f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.068+0000 7f2beb78f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0e2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.068+0000 7f2beb78f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 0x5645ee0e2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.068+0000 7f2beb78f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5645ee0e8270 con 0x5645ee0da780 2024-09-19T08:20:57.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.068+0000 7f2beb78f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5645edffb060 con 0x5645ee0e5520 2024-09-19T08:20:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.068+0000 7f2beb78f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5645ee0e83f0 con 0x5645ee0dbcf0 2024-09-19T08:20:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.068+0000 7f2beb78f740 10 client.0 _ll_register_callbacks cb 0x5645edd4e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.069+0000 7f2be27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0e2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.069+0000 7f2be2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.069+0000 7f2be2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0e4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47292/0 (socket says 172.21.15.22:47292) 2024-09-19T08:20:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.069+0000 7f2be2ffd640 1 -- 172.21.15.22:0/1748365550 learned_addr learned my addr 172.21.15.22:0/1748365550 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.069+0000 7f2be2ffd640 1 -- 172.21.15.22:0/1748365550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 msgr2=0x5645ee0e2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.069+0000 7f2be2ffd640 1 --2- 172.21.15.22:0/1748365550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0e2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.069+0000 7f2be2ffd640 1 -- 172.21.15.22:0/1748365550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 msgr2=0x5645ee0e2570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:20:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.069+0000 7f2be2ffd640 1 --2- 172.21.15.22:0/1748365550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 0x5645ee0e2570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.069+0000 7f2be2ffd640 1 -- 172.21.15.22:0/1748365550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645eddb13c0 con 0x5645ee0da780 2024-09-19T08:20:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.070+0000 7f2be2ffd640 1 --2- 172.21.15.22:0/1748365550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0e4d80 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc031b60 tx=0x7f2bdc044e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91423516de53506 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:57.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.070+0000 7f2bc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:57.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.071+0000 7f2bc6ff5640 1 -- 172.21.15.22:0/1748365550 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0c1900 con 0x5645ee0da780 2024-09-19T08:20:57.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.071+0000 7f2bc6ff5640 1 -- 172.21.15.22:0/1748365550 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2bdc0eeb30 con 0x5645ee0da780 2024-09-19T08:20:57.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.072+0000 7f2beb78f740 1 -- 172.21.15.22:0/1748365550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd400bda0 con 0x5645ee0da780 2024-09-19T08:20:57.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.072+0000 7f2bc6ff5640 1 -- 172.21.15.22:0/1748365550 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2bdc0eed30 con 0x5645ee0da780 2024-09-19T08:20:57.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.072+0000 7f2beb78f740 10 client.16557 fetch_fsmap learned FSMap version 30 2024-09-19T08:20:57.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.072+0000 7f2beb78f740 1 -- 172.21.15.22:0/1748365550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645ee0f3c80 con 0x5645ee0da780 2024-09-19T08:20:57.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.073+0000 7f2bc6ff5640 1 -- 172.21.15.22:0/1748365550 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd400bda0 con 0x5645ee0da780 2024-09-19T08:20:57.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.073+0000 7f2bc6ff5640 1 -- 172.21.15.22:0/1748365550 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f2bdc0d6260 con 0x5645ee0da780 2024-09-19T08:20:57.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.073+0000 7f2beb78f740 10 client.16557 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:20:57.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.073+0000 7f2beb78f740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:20:57.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.073+0000 7f2beb78f740 20 client.16557 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:57.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.073+0000 7f2beb78f740 1 --2- 172.21.15.22:0/1748365550 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5645ee0f5350 0x5645ee0f7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.073+0000 7f2beb78f740 4 client.16557 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:20:57.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.073+0000 7f2beb78f740 1 -- 172.21.15.22:0/1748365550 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645ee04d860 con 0x5645ee0f5350 2024-09-19T08:20:57.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.074+0000 7f2be27fc640 1 --2- 172.21.15.22:0/1748365550 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5645ee0f5350 0x5645ee0f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.075+0000 7f2bc6ff5640 10 client.16557 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:57.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.076+0000 7f2be27fc640 1 --2- 172.21.15.22:0/1748365550 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5645ee0f5350 0x5645ee0f7810 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6533bce93aa18667 server_cookie=b72f0e4e87ef1748 in_seq=0 out_seq=0 2024-09-19T08:20:57.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.077+0000 7f2bc6ff5640 1 -- 172.21.15.22:0/1748365550 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645ee04d860 con 0x5645ee0f5350 2024-09-19T08:20:57.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.077+0000 7f2bc6ff5640 10 client.16557 handle_command_reply: tid=0 2024-09-19T08:20:57.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.077+0000 7f2beb78f740 1 client.16557 shutdown 2024-09-19T08:20:57.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.078+0000 7f2beb78f740 1 -- 172.21.15.22:0/1748365550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 msgr2=0x5645ee0e4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:57.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.078+0000 7f2beb78f740 1 --2- 172.21.15.22:0/1748365550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0e4d80 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc031b60 tx=0x7f2bdc044e50 comp rx=0 tx=0).stop 2024-09-19T08:20:57.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.079+0000 7f2beb78f740 1 -- 172.21.15.22:0/1748365550 shutdown_connections 2024-09-19T08:20:57.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.079+0000 7f2beb78f740 1 --2- 172.21.15.22:0/1748365550 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5645ee0f5350 0x5645ee0f7810 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.079+0000 7f2beb78f740 1 --2- 172.21.15.22:0/1748365550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 0x5645ee0e2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.079+0000 7f2beb78f740 1 --2- 172.21.15.22:0/1748365550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0e2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.079+0000 7f2beb78f740 1 --2- 172.21.15.22:0/1748365550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0e4d80 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.079+0000 7f2beb78f740 1 -- 172.21.15.22:0/1748365550 >> 172.21.15.22:0/1748365550 conn(0x5645ee0a3310 msgr2=0x5645ee0dcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:57.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.081+0000 7f2bc6ff5640 0 client.16557 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:57.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.084+0000 7f2beb78f740 1 -- 172.21.15.22:0/1748365550 shutdown_connections 2024-09-19T08:20:57.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.084+0000 7f2beb78f740 1 -- 172.21.15.22:0/1748365550 wait complete. 2024-09-19T08:20:57.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.084+0000 7f2beb78f740 20 client.16557 trim_cache size 0 max 16384 2024-09-19T08:20:57.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.124+0000 7f2beb78f740 1 Processor -- start 2024-09-19T08:20:57.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.128+0000 7f2beb78f740 1 -- start start 2024-09-19T08:20:57.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.128+0000 7f2beb78f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.129+0000 7f2beb78f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0dd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.129+0000 7f2beb78f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 0x5645ee0dd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.129+0000 7f2beb78f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0011110 con 0x5645ee0da780 2024-09-19T08:20:57.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.129+0000 7f2beb78f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd000e9b0 con 0x5645ee0e5520 2024-09-19T08:20:57.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.129+0000 7f2beb78f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd000c800 con 0x5645ee0dbcf0 2024-09-19T08:20:57.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.129+0000 7f2be37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 0x5645ee0dd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.129+0000 7f2be37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 0x5645ee0dd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:50012/0 (socket says 172.21.15.22:50012) 2024-09-19T08:20:57.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.129+0000 7f2be37fe640 1 -- 172.21.15.22:0/2981494672 learned_addr learned my addr 172.21.15.22:0/2981494672 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:57.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.130+0000 7f2be2ffd640 1 --2- 172.21.15.22:0/2981494672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.132+0000 7f2be27fc640 1 --2- 172.21.15.22:0/2981494672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0dd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.132+0000 7f2be2ffd640 1 -- 172.21.15.22:0/2981494672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 msgr2=0x5645ee0dd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:57.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.132+0000 7f2be2ffd640 1 --2- 172.21.15.22:0/2981494672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0dd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.132+0000 7f2be2ffd640 1 -- 172.21.15.22:0/2981494672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 msgr2=0x5645ee0dd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:57.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.132+0000 7f2be2ffd640 1 --2- 172.21.15.22:0/2981494672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 0x5645ee0dd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.132+0000 7f2be2ffd640 1 -- 172.21.15.22:0/2981494672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ee0e50f0 con 0x5645ee0da780 2024-09-19T08:20:57.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.133+0000 7f2be2ffd640 1 --2- 172.21.15.22:0/2981494672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0d5dc0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0bb800 tx=0x7f2bdc009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb8503db457fe8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:57.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.133+0000 7f2bc8ff9640 1 -- 172.21.15.22:0/2981494672 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0bd090 con 0x5645ee0da780 2024-09-19T08:20:57.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.134+0000 7f2bc8ff9640 1 -- 172.21.15.22:0/2981494672 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bdc036030 con 0x5645ee0da780 2024-09-19T08:20:57.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.134+0000 7f2bc8ff9640 1 -- 172.21.15.22:0/2981494672 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0c5d30 con 0x5645ee0da780 2024-09-19T08:20:57.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.135+0000 7f2beb78f740 1 -- 172.21.15.22:0/2981494672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 msgr2=0x5645ee0d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:57.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.135+0000 7f2beb78f740 1 --2- 172.21.15.22:0/2981494672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0d5dc0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0bb800 tx=0x7f2bdc009a20 comp rx=0 tx=0).stop 2024-09-19T08:20:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.139+0000 7f2beb78f740 1 -- 172.21.15.22:0/2981494672 shutdown_connections 2024-09-19T08:20:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.139+0000 7f2beb78f740 1 --2- 172.21.15.22:0/2981494672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 0x5645ee0dd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.139+0000 7f2beb78f740 1 --2- 172.21.15.22:0/2981494672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0dd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.139+0000 7f2beb78f740 1 --2- 172.21.15.22:0/2981494672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0d5dc0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.139+0000 7f2beb78f740 1 -- 172.21.15.22:0/2981494672 >> 172.21.15.22:0/2981494672 conn(0x5645ee0a3310 msgr2=0x5645ee0d56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:57.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.148+0000 7f2beb78f740 1 -- 172.21.15.22:0/2981494672 shutdown_connections 2024-09-19T08:20:57.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.153+0000 7f2beb78f740 1 -- 172.21.15.22:0/2981494672 wait complete. 2024-09-19T08:20:57.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.153+0000 7f2beb78f740 1 -- start start 2024-09-19T08:20:57.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.154+0000 7f2beb78f740 1 Processor -- start 2024-09-19T08:20:57.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.154+0000 7f2beb78f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0eda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.155+0000 7f2beb78f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0edfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.155+0000 7f2beb78f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 0x5645ee0e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.155+0000 7f2be2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.155+0000 7f2be2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0eda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47310/0 (socket says 172.21.15.22:47310) 2024-09-19T08:20:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.155+0000 7f2be2ffd640 1 -- 172.21.15.22:0/4080437961 learned_addr learned my addr 172.21.15.22:0/4080437961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:20:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.155+0000 7f2beb78f740 1 -- 172.21.15.22:0/4080437961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd000ce70 con 0x5645ee0da780 2024-09-19T08:20:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.155+0000 7f2beb78f740 1 -- 172.21.15.22:0/4080437961 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd000e2e0 con 0x5645ee0e5520 2024-09-19T08:20:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.155+0000 7f2be37fe640 1 --2- 172.21.15.22:0/4080437961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 0x5645ee0e8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.155+0000 7f2beb78f740 1 -- 172.21.15.22:0/4080437961 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd000d540 con 0x5645ee0dbcf0 2024-09-19T08:20:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.155+0000 7f2be27fc640 1 --2- 172.21.15.22:0/4080437961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0edfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.156+0000 7f2be2ffd640 1 -- 172.21.15.22:0/4080437961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 msgr2=0x5645ee0edfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.156+0000 7f2be2ffd640 1 --2- 172.21.15.22:0/4080437961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0edfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.156+0000 7f2be2ffd640 1 -- 172.21.15.22:0/4080437961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 msgr2=0x5645ee0e8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:20:57.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.156+0000 7f2be2ffd640 1 --2- 172.21.15.22:0/4080437961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 0x5645ee0e8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.156+0000 7f2be2ffd640 1 -- 172.21.15.22:0/4080437961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ee0e96b0 con 0x5645ee0da780 2024-09-19T08:20:57.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.156+0000 7f2be2ffd640 1 --2- 172.21.15.22:0/4080437961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0eda70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc045000 tx=0x7f2bdc004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40662437e2102732 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:20:57.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.157+0000 7f2beb78f740 10 client.0 _ll_register_callbacks cb 0x7f2bdc0b9130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:20:57.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.158+0000 7f2beb78f740 1 -- 172.21.15.22:0/4080437961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bdc036030 con 0x5645ee0da780 2024-09-19T08:20:57.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.160+0000 7f2bc6ff5640 10 client.16563 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:20:57.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.160+0000 7f2bc6ff5640 1 -- 172.21.15.22:0/4080437961 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0eed40 con 0x5645ee0da780 2024-09-19T08:20:57.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.160+0000 7f2bc6ff5640 1 -- 172.21.15.22:0/4080437961 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2bdc0c5f30 con 0x5645ee0da780 2024-09-19T08:20:57.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.160+0000 7f2bc6ff5640 1 -- 172.21.15.22:0/4080437961 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2bdc0c61c0 con 0x5645ee0da780 2024-09-19T08:20:57.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.161+0000 7f2beb78f740 10 client.16563 fetch_fsmap learned FSMap version 30 2024-09-19T08:20:57.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.161+0000 7f2beb78f740 1 -- 172.21.15.22:0/4080437961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5645ee15f790 con 0x5645ee0da780 2024-09-19T08:20:57.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.162+0000 7f2bc6ff5640 1 -- 172.21.15.22:0/4080437961 <== mon.0 v2:172.21.15.22:3300/0 4 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f2bdc0bc9a0 con 0x5645ee0da780 2024-09-19T08:20:57.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.162+0000 7f2beb78f740 10 client.16563 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:20:57.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.162+0000 7f2beb78f740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:20:57.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.162+0000 7f2beb78f740 20 client.16563 populate_metadata read hostname 'smithi022' 2024-09-19T08:20:57.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.162+0000 7f2beb78f740 1 --2- 172.21.15.22:0/4080437961 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5645ee15fbd0 0x5645ee162070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:20:57.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.162+0000 7f2beb78f740 4 client.16563 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:20:57.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.162+0000 7f2beb78f740 1 -- 172.21.15.22:0/4080437961 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2bdc036030 con 0x5645ee15fbd0 2024-09-19T08:20:57.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.166+0000 7f2be27fc640 1 --2- 172.21.15.22:0/4080437961 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5645ee15fbd0 0x5645ee162070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:20:57.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.168+0000 7f2be27fc640 1 --2- 172.21.15.22:0/4080437961 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5645ee15fbd0 0x5645ee162070 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f499e3ea7420ae52 server_cookie=5be6394edfe162b3 in_seq=0 out_seq=0 2024-09-19T08:20:57.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.168+0000 7f2bc6ff5640 10 client.16563 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:57.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.170+0000 7f2bc6ff5640 1 -- 172.21.15.22:0/4080437961 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2bdc036030 con 0x5645ee15fbd0 2024-09-19T08:20:57.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.170+0000 7f2bc6ff5640 10 client.16563 handle_command_reply: tid=0 2024-09-19T08:20:57.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.170+0000 7f2beb78f740 1 client.16563 shutdown 2024-09-19T08:20:57.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.182+0000 7f2beb78f740 1 -- 172.21.15.22:0/4080437961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 msgr2=0x5645ee0eda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:57.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.182+0000 7f2beb78f740 1 --2- 172.21.15.22:0/4080437961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0eda70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc045000 tx=0x7f2bdc004990 comp rx=0 tx=0).stop 2024-09-19T08:20:57.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.184+0000 7f2beb78f740 1 -- 172.21.15.22:0/4080437961 shutdown_connections 2024-09-19T08:20:57.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.184+0000 7f2beb78f740 1 --2- 172.21.15.22:0/4080437961 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5645ee15fbd0 0x5645ee162070 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.184+0000 7f2beb78f740 1 --2- 172.21.15.22:0/4080437961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645ee0e5520 0x5645ee0e8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.184+0000 7f2beb78f740 1 --2- 172.21.15.22:0/4080437961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5645ee0dbcf0 0x5645ee0edfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.184+0000 7f2beb78f740 1 --2- 172.21.15.22:0/4080437961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5645ee0da780 0x5645ee0eda70 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.184+0000 7f2beb78f740 1 -- 172.21.15.22:0/4080437961 >> 172.21.15.22:0/4080437961 conn(0x5645ee0a3310 msgr2=0x5645ee0d5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:57.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.184+0000 7f2bc6ff5640 0 client.16563 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:20:57.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.185+0000 7f2beb78f740 1 -- 172.21.15.22:0/4080437961 shutdown_connections 2024-09-19T08:20:57.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.185+0000 7f2beb78f740 1 -- 172.21.15.22:0/4080437961 wait complete. 2024-09-19T08:20:57.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.186+0000 7f2beb78f740 20 client.16563 trim_cache size 0 max 16384 2024-09-19T08:20:57.188 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:20:57.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.188+0000 7f2be93b6640 1 -- 172.21.15.22:0/1000113019 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2bcc0a35c0 msgr2=0x7f2bcc0a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:57.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.188+0000 7f2be93b6640 1 --2- 172.21.15.22:0/1000113019 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2bcc0a35c0 0x7f2bcc0a5a80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2be4114190 tx=0x7f2bd8006bd0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.188+0000 7f2be93b6640 1 -- 172.21.15.22:0/1000113019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be406ea30 msgr2=0x7f2be4119940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:20:57.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.188+0000 7f2be93b6640 1 --2- 172.21.15.22:0/1000113019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be406ea30 0x7f2be4119940 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc02aee0 tx=0x7f2bdc030fd0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.188+0000 7f2be93b6640 1 -- 172.21.15.22:0/1000113019 shutdown_connections 2024-09-19T08:20:57.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.188+0000 7f2be93b6640 1 --2- 172.21.15.22:0/1000113019 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2bcc0a35c0 0x7f2bcc0a5a80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.188+0000 7f2be93b6640 1 --2- 172.21.15.22:0/1000113019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2be4112fb0 0x7f2be4113460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.188+0000 7f2be93b6640 1 --2- 172.21.15.22:0/1000113019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be411b140 0x7f2be4112a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.188+0000 7f2be93b6640 1 --2- 172.21.15.22:0/1000113019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be406ea30 0x7f2be4119940 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:20:57.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.188+0000 7f2be93b6640 1 -- 172.21.15.22:0/1000113019 >> 172.21.15.22:0/1000113019 conn(0x7f2be4073210 msgr2=0x7f2be406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:20:57.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.188+0000 7f2be93b6640 1 -- 172.21.15.22:0/1000113019 shutdown_connections 2024-09-19T08:20:57.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:20:57.188+0000 7f2be93b6640 1 -- 172.21.15.22:0/1000113019 wait complete. 2024-09-19T08:20:57.252 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:56 smithi157 ceph-mon[35143]: pgmap v180: 97 pgs: 97 active+clean; 695 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 536 op/s 2024-09-19T08:20:57.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:56 smithi064 ceph-mon[34900]: pgmap v180: 97 pgs: 97 active+clean; 695 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 536 op/s 2024-09-19T08:20:58.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:20:58 smithi022 ceph-mon[30963]: pgmap v181: 97 pgs: 97 active+clean; 689 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.4 MiB/s wr, 556 op/s 2024-09-19T08:20:59.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:20:58 smithi157 ceph-mon[35143]: pgmap v181: 97 pgs: 97 active+clean; 689 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.4 MiB/s wr, 556 op/s 2024-09-19T08:20:59.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:20:58 smithi064 ceph-mon[34900]: pgmap v181: 97 pgs: 97 active+clean; 689 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.4 MiB/s wr, 556 op/s 2024-09-19T08:20:59.521 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:20:59.522 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:21:01.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:00 smithi157 ceph-mon[35143]: pgmap v182: 97 pgs: 97 active+clean; 666 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 553 op/s 2024-09-19T08:21:01.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:00 smithi022 ceph-mon[30963]: pgmap v182: 97 pgs: 97 active+clean; 666 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 553 op/s 2024-09-19T08:21:01.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:00 smithi064 ceph-mon[34900]: pgmap v182: 97 pgs: 97 active+clean; 666 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 553 op/s 2024-09-19T08:21:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:21:02.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.675+0000 7fced1ce4640 1 -- 172.21.15.22:0/261562439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcecc108da0 msgr2=0x7fcecc10cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:02.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.675+0000 7fced1ce4640 1 --2- 172.21.15.22:0/261562439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcecc108da0 0x7fcecc10cdf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009860 tx=0x7fcebc02a360 comp rx=0 tx=0).stop 2024-09-19T08:21:02.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.676+0000 7fced1ce4640 1 -- 172.21.15.22:0/261562439 shutdown_connections 2024-09-19T08:21:02.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.676+0000 7fced1ce4640 1 --2- 172.21.15.22:0/261562439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcecc10d520 0x7fcecc1150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.676+0000 7fced1ce4640 1 --2- 172.21.15.22:0/261562439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcecc108da0 0x7fcecc10cdf0 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009860 tx=0x7fcebc02a360 comp rx=0 tx=0).stop 2024-09-19T08:21:02.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.676+0000 7fced1ce4640 1 --2- 172.21.15.22:0/261562439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcecc1083f0 0x7fcecc1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.676+0000 7fced1ce4640 1 -- 172.21.15.22:0/261562439 >> 172.21.15.22:0/261562439 conn(0x7fcecc077af0 msgr2=0x7fcecc079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:02.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.683+0000 7fced1ce4640 1 -- 172.21.15.22:0/261562439 shutdown_connections 2024-09-19T08:21:02.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.684+0000 7fced1ce4640 1 -- 172.21.15.22:0/261562439 wait complete. 2024-09-19T08:21:02.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.685+0000 7fced1ce4640 1 Processor -- start 2024-09-19T08:21:02.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.689+0000 7fced1ce4640 1 -- start start 2024-09-19T08:21:02.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.690+0000 7fced1ce4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcecc1083f0 0x7fcecc073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.690+0000 7fced1ce4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcecc10d520 0x7fcecc06cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.690+0000 7fced1ce4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcecc06d1e0 0x7fcecc06d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.690+0000 7fced1ce4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc118610 con 0x7fcecc1083f0 2024-09-19T08:21:02.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.690+0000 7fcecbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcecc06d1e0 0x7fcecc06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:02.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.690+0000 7fcecbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcecc06d1e0 0x7fcecc06d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:43996/0 (socket says 172.21.15.22:43996) 2024-09-19T08:21:02.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.690+0000 7fcecbfff640 1 -- 172.21.15.22:0/1082442986 learned_addr learned my addr 172.21.15.22:0/1082442986 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:02.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.691+0000 7fced1ce4640 1 -- 172.21.15.22:0/1082442986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc118490 con 0x7fcecc10d520 2024-09-19T08:21:02.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.691+0000 7fced1ce4640 1 -- 172.21.15.22:0/1082442986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc118790 con 0x7fcecc06d1e0 2024-09-19T08:21:02.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.691+0000 7fcecaffd640 1 --2- 172.21.15.22:0/1082442986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcecc10d520 0x7fcecc06cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:02.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.693+0000 7fcecaffd640 1 -- 172.21.15.22:0/1082442986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcecc06d1e0 msgr2=0x7fcecc06d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:02.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.693+0000 7fcecaffd640 1 --2- 172.21.15.22:0/1082442986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcecc06d1e0 0x7fcecc06d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.693+0000 7fcecaffd640 1 -- 172.21.15.22:0/1082442986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcecc1083f0 msgr2=0x7fcecc073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:02.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.693+0000 7fcecaffd640 1 --2- 172.21.15.22:0/1082442986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcecc1083f0 0x7fcecc073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.693+0000 7fcecaffd640 1 -- 172.21.15.22:0/1082442986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcecc071840 con 0x7fcecc10d520 2024-09-19T08:21:02.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.694+0000 7fcecaffd640 1 --2- 172.21.15.22:0/1082442986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcecc10d520 0x7fcecc06cca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcecc109300 tx=0x7fcebc033d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:02.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.694+0000 7fcec8ff9640 1 -- 172.21.15.22:0/1082442986 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc037040 con 0x7fcecc10d520 2024-09-19T08:21:02.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.694+0000 7fced1ce4640 1 -- 172.21.15.22:0/1082442986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcecc071ad0 con 0x7fcecc10d520 2024-09-19T08:21:02.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.695+0000 7fced1ce4640 1 -- 172.21.15.22:0/1082442986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcecc071ed0 con 0x7fcecc10d520 2024-09-19T08:21:02.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.700+0000 7fced40bd740 1 Processor -- start 2024-09-19T08:21:02.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.700+0000 7fcec8ff9640 1 -- 172.21.15.22:0/1082442986 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcebc004220 con 0x7fcecc10d520 2024-09-19T08:21:02.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.700+0000 7fcec8ff9640 1 -- 172.21.15.22:0/1082442986 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc003970 con 0x7fcecc10d520 2024-09-19T08:21:02.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.701+0000 7fced40bd740 1 -- start start 2024-09-19T08:21:02.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.703+0000 7fced40bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 0x55a32b626f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.703+0000 7fced40bd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 0x55a32b631800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.703+0000 7fcecbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 0x55a32b626f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:02.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.703+0000 7fcecbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 0x55a32b626f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52882/0 (socket says 172.21.15.22:52882) 2024-09-19T08:21:02.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.705+0000 7fced40bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b631d40 0x55a32b634130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.705+0000 7fced40bd740 1 -- 172.21.15.22:0/1150518464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a32b58c280 con 0x55a32b626b60 2024-09-19T08:21:02.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.705+0000 7fcecbfff640 1 -- 172.21.15.22:0/1150518464 learned_addr learned my addr 172.21.15.22:0/1150518464 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:02.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.705+0000 7fced40bd740 1 -- 172.21.15.22:0/1150518464 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a32b533b10 con 0x55a32b631d40 2024-09-19T08:21:02.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.705+0000 7fced40bd740 1 -- 172.21.15.22:0/1150518464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a32b588240 con 0x55a32b627480 2024-09-19T08:21:02.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.705+0000 7fcecb7fe640 1 --2- 172.21.15.22:0/1150518464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 0x55a32b631800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.712+0000 7fcecb7fe640 1 -- 172.21.15.22:0/1150518464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b631d40 msgr2=0x55a32b634130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:21:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.712+0000 7fcecb7fe640 1 --2- 172.21.15.22:0/1150518464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b631d40 0x55a32b634130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.712+0000 7fcecb7fe640 1 -- 172.21.15.22:0/1150518464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 msgr2=0x55a32b626f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.712+0000 7fcecb7fe640 1 --2- 172.21.15.22:0/1150518464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 0x55a32b626f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.712+0000 7fcecb7fe640 1 -- 172.21.15.22:0/1150518464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a32b5401f0 con 0x55a32b627480 2024-09-19T08:21:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.720+0000 7fcec8ff9640 1 -- 172.21.15.22:0/1082442986 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcebc003b10 con 0x7fcecc10d520 2024-09-19T08:21:02.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.722+0000 7fcec8ff9640 1 --2- 172.21.15.22:0/1082442986 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fceb40a35c0 0x7fceb40a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.722+0000 7fcecb7fe640 1 --2- 172.21.15.22:0/1150518464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 0x55a32b631800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009d60 tx=0x7fcec400e760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e85b2f94d3b87ee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:02.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.722+0000 7fcecbfff640 1 --2- 172.21.15.22:0/1082442986 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fceb40a35c0 0x7fceb40a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:02.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.722+0000 7fcec8ff9640 1 -- 172.21.15.22:0/1082442986 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcebc0eed20 con 0x7fcecc10d520 2024-09-19T08:21:02.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.729+0000 7fceb1ffb640 1 -- 172.21.15.22:0/1150518464 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4003b70 con 0x55a32b627480 2024-09-19T08:21:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.729+0000 7fceb1ffb640 1 -- 172.21.15.22:0/1150518464 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec4009e50 con 0x55a32b627480 2024-09-19T08:21:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.729+0000 7fceb1ffb640 1 -- 172.21.15.22:0/1150518464 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec400fad0 con 0x55a32b627480 2024-09-19T08:21:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.730+0000 7fced40bd740 1 -- 172.21.15.22:0/1150518464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 msgr2=0x55a32b631800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.730+0000 7fced40bd740 1 --2- 172.21.15.22:0/1150518464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 0x55a32b631800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009d60 tx=0x7fcec400e760 comp rx=0 tx=0).stop 2024-09-19T08:21:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.732+0000 7fcecbfff640 1 --2- 172.21.15.22:0/1082442986 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fceb40a35c0 0x7fceb40a5a80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcec0006fd0 tx=0x7fcec0008040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.741+0000 7fced40bd740 1 -- 172.21.15.22:0/1150518464 shutdown_connections 2024-09-19T08:21:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.741+0000 7fced40bd740 1 --2- 172.21.15.22:0/1150518464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b631d40 0x55a32b634130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.741+0000 7fced40bd740 1 --2- 172.21.15.22:0/1150518464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 0x55a32b631800 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.741+0000 7fced40bd740 1 --2- 172.21.15.22:0/1150518464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 0x55a32b626f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.741+0000 7fced40bd740 1 -- 172.21.15.22:0/1150518464 >> 172.21.15.22:0/1150518464 conn(0x55a32b6230f0 msgr2=0x55a32b6234d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:02.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.743+0000 7fced40bd740 1 -- 172.21.15.22:0/1150518464 shutdown_connections 2024-09-19T08:21:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.745+0000 7fced40bd740 1 -- 172.21.15.22:0/1150518464 wait complete. 2024-09-19T08:21:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.746+0000 7fced40bd740 1 -- start start 2024-09-19T08:21:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.746+0000 7fced40bd740 1 Processor -- start 2024-09-19T08:21:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.750+0000 7fced40bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 0x55a32b62ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.750+0000 7fced40bd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 0x55a32b62f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.750+0000 7fced40bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b631d40 0x55a32b62f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.750+0000 7fced40bd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a32b634910 con 0x55a32b626b60 2024-09-19T08:21:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.750+0000 7fced40bd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a32b546f10 con 0x55a32b631d40 2024-09-19T08:21:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.750+0000 7fced40bd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a32b634a90 con 0x55a32b627480 2024-09-19T08:21:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.750+0000 7fced40bd740 10 client.0 _ll_register_callbacks cb 0x55a32b50fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:21:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.751+0000 7fcecbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b631d40 0x55a32b62f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.751+0000 7fcecb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 0x55a32b62ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.751+0000 7fcecb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 0x55a32b62ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52894/0 (socket says 172.21.15.22:52894) 2024-09-19T08:21:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.751+0000 7fcecb7fe640 1 -- 172.21.15.22:0/270586195 learned_addr learned my addr 172.21.15.22:0/270586195 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.751+0000 7fcecb7fe640 1 -- 172.21.15.22:0/270586195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 msgr2=0x55a32b62f160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:21:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.751+0000 7fcecb7fe640 1 --2- 172.21.15.22:0/270586195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 0x55a32b62f160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.751+0000 7fcecb7fe640 1 -- 172.21.15.22:0/270586195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b631d40 msgr2=0x55a32b62f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.751+0000 7fcecb7fe640 1 --2- 172.21.15.22:0/270586195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b631d40 0x55a32b62f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.751+0000 7fcecb7fe640 1 -- 172.21.15.22:0/270586195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32b2fd6a0 con 0x55a32b626b60 2024-09-19T08:21:02.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.752+0000 7fcecb7fe640 1 --2- 172.21.15.22:0/270586195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 0x55a32b62ec20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fcec400f650 tx=0x7fcec4003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d642b82013850584 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:02.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.752+0000 7fceaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:21:02.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.753+0000 7fceaaffd640 1 -- 172.21.15.22:0/270586195 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4024030 con 0x55a32b626b60 2024-09-19T08:21:02.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.753+0000 7fceaaffd640 1 -- 172.21.15.22:0/270586195 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcec401c050 con 0x55a32b626b60 2024-09-19T08:21:02.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.754+0000 7fced40bd740 1 -- 172.21.15.22:0/270586195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcec4009e50 con 0x55a32b626b60 2024-09-19T08:21:02.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.757+0000 7fceaaffd640 1 -- 172.21.15.22:0/270586195 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcec4009980 con 0x55a32b626b60 2024-09-19T08:21:02.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.757+0000 7fced40bd740 10 client.16572 fetch_fsmap learned FSMap version 30 2024-09-19T08:21:02.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.757+0000 7fced40bd740 1 -- 172.21.15.22:0/270586195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a32b6405a0 con 0x55a32b626b60 2024-09-19T08:21:02.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.758+0000 7fceaaffd640 1 -- 172.21.15.22:0/270586195 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec4009e50 con 0x55a32b626b60 2024-09-19T08:21:02.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.758+0000 7fceaaffd640 1 -- 172.21.15.22:0/270586195 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fcec400b030 con 0x55a32b626b60 2024-09-19T08:21:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.760+0000 7fced40bd740 10 client.16572 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:21:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.760+0000 7fced40bd740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:21:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.760+0000 7fced40bd740 20 client.16572 populate_metadata read hostname 'smithi022' 2024-09-19T08:21:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.760+0000 7fced40bd740 1 --2- 172.21.15.22:0/270586195 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a32b641a00 0x55a32b643df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.760+0000 7fced40bd740 4 client.16572 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:21:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.760+0000 7fced40bd740 1 -- 172.21.15.22:0/270586195 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a32b5997f0 con 0x55a32b641a00 2024-09-19T08:21:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.762+0000 7fcecaffd640 1 --2- 172.21.15.22:0/270586195 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a32b641a00 0x55a32b643df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:02.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.765+0000 7fcecaffd640 1 --2- 172.21.15.22:0/270586195 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a32b641a00 0x55a32b643df0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c296c9a56c74a9 server_cookie=f3008bde05bed391 in_seq=0 out_seq=0 2024-09-19T08:21:02.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.766+0000 7fceaaffd640 10 client.16572 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:21:02.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.769+0000 7fceaaffd640 1 -- 172.21.15.22:0/270586195 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a32b5997f0 con 0x55a32b641a00 2024-09-19T08:21:02.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.769+0000 7fceaaffd640 10 client.16572 handle_command_reply: tid=0 2024-09-19T08:21:02.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.770+0000 7fced40bd740 1 client.16572 shutdown 2024-09-19T08:21:02.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.778+0000 7fced40bd740 1 -- 172.21.15.22:0/270586195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 msgr2=0x55a32b62ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:02.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.778+0000 7fced40bd740 1 --2- 172.21.15.22:0/270586195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 0x55a32b62ec20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fcec400f650 tx=0x7fcec4003e90 comp rx=0 tx=0).stop 2024-09-19T08:21:02.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.780+0000 7fced40bd740 1 -- 172.21.15.22:0/270586195 shutdown_connections 2024-09-19T08:21:02.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.780+0000 7fced40bd740 1 --2- 172.21.15.22:0/270586195 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a32b641a00 0x55a32b643df0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.780+0000 7fced40bd740 1 --2- 172.21.15.22:0/270586195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b631d40 0x55a32b62f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.780+0000 7fced40bd740 1 --2- 172.21.15.22:0/270586195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 0x55a32b62f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.780+0000 7fced40bd740 1 --2- 172.21.15.22:0/270586195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 0x55a32b62ec20 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.780+0000 7fceaaffd640 0 client.16572 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:21:02.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.780+0000 7fced40bd740 1 -- 172.21.15.22:0/270586195 >> 172.21.15.22:0/270586195 conn(0x55a32b6230f0 msgr2=0x55a32b628e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:02.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.784+0000 7fced40bd740 1 -- 172.21.15.22:0/270586195 shutdown_connections 2024-09-19T08:21:02.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.785+0000 7fced40bd740 1 -- 172.21.15.22:0/270586195 wait complete. 2024-09-19T08:21:02.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.787+0000 7fced40bd740 20 client.16572 trim_cache size 0 max 16384 2024-09-19T08:21:02.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.837+0000 7fced40bd740 1 Processor -- start 2024-09-19T08:21:02.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.837+0000 7fced40bd740 1 -- start start 2024-09-19T08:21:02.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.837+0000 7fced40bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 0x55a32b62e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.837+0000 7fced40bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b627480 0x55a32b649420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.838+0000 7fced40bd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b631d40 0x55a32b649960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.838+0000 7fced40bd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fceb8011150 con 0x55a32b626b60 2024-09-19T08:21:02.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.838+0000 7fced40bd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fceb800e9f0 con 0x55a32b627480 2024-09-19T08:21:02.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.838+0000 7fced40bd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fceb800c840 con 0x55a32b631d40 2024-09-19T08:21:02.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.838+0000 7fcecaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b627480 0x55a32b649420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:02.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.838+0000 7fcecaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b627480 0x55a32b649420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:45894/0 (socket says 172.21.15.22:45894) 2024-09-19T08:21:02.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.838+0000 7fcecaffd640 1 -- 172.21.15.22:0/1166711645 learned_addr learned my addr 172.21.15.22:0/1166711645 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:02.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.838+0000 7fcecbfff640 1 --2- 172.21.15.22:0/1166711645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b631d40 0x55a32b649960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:02.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.838+0000 7fcecaffd640 1 -- 172.21.15.22:0/1166711645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b631d40 msgr2=0x55a32b649960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:02.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.838+0000 7fcecaffd640 1 --2- 172.21.15.22:0/1166711645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b631d40 0x55a32b649960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.838+0000 7fcecaffd640 1 -- 172.21.15.22:0/1166711645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 msgr2=0x55a32b62e3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:21:02.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.838+0000 7fcecaffd640 1 --2- 172.21.15.22:0/1166711645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 0x55a32b62e3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.838+0000 7fcecaffd640 1 -- 172.21.15.22:0/1166711645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a32b624260 con 0x55a32b627480 2024-09-19T08:21:02.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.839+0000 7fcecaffd640 1 --2- 172.21.15.22:0/1166711645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b627480 0x55a32b649420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcebc001eb0 tx=0x7fcebc0db040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b47b2d5bb3d3f6ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:02.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.841+0000 7fceb17fa640 1 -- 172.21.15.22:0/1166711645 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc0bb7c0 con 0x55a32b627480 2024-09-19T08:21:02.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.841+0000 7fceb17fa640 1 -- 172.21.15.22:0/1166711645 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcebc0bb0c0 con 0x55a32b627480 2024-09-19T08:21:02.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.841+0000 7fceb17fa640 1 -- 172.21.15.22:0/1166711645 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc0db0b0 con 0x55a32b627480 2024-09-19T08:21:02.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.844+0000 7fced40bd740 1 -- 172.21.15.22:0/1166711645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b627480 msgr2=0x55a32b649420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:02.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.844+0000 7fced40bd740 1 --2- 172.21.15.22:0/1166711645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b627480 0x55a32b649420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcebc001eb0 tx=0x7fcebc0db040 comp rx=0 tx=0).stop 2024-09-19T08:21:02.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.847+0000 7fced40bd740 1 -- 172.21.15.22:0/1166711645 shutdown_connections 2024-09-19T08:21:02.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.847+0000 7fced40bd740 1 --2- 172.21.15.22:0/1166711645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b631d40 0x55a32b649960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.847+0000 7fced40bd740 1 --2- 172.21.15.22:0/1166711645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b627480 0x55a32b649420 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.847+0000 7fced40bd740 1 --2- 172.21.15.22:0/1166711645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b626b60 0x55a32b62e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.847+0000 7fced40bd740 1 -- 172.21.15.22:0/1166711645 >> 172.21.15.22:0/1166711645 conn(0x55a32b6230f0 msgr2=0x55a32b630b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:02.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.851+0000 7fced40bd740 1 -- 172.21.15.22:0/1166711645 shutdown_connections 2024-09-19T08:21:02.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.851+0000 7fced40bd740 1 -- 172.21.15.22:0/1166711645 wait complete. 2024-09-19T08:21:02.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.851+0000 7fced40bd740 1 -- start start 2024-09-19T08:21:02.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.852+0000 7fced40bd740 1 Processor -- start 2024-09-19T08:21:02.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.853+0000 7fced40bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b626b60 0x55a32b635e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.853+0000 7fced40bd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 0x55a32b6363c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.853+0000 7fcecb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b626b60 0x55a32b635e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:02.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.853+0000 7fcecb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b626b60 0x55a32b635e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:45902/0 (socket says 172.21.15.22:45902) 2024-09-19T08:21:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.853+0000 7fcecaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 0x55a32b6363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.853+0000 7fcecaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 0x55a32b6363c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:44024/0 (socket says 172.21.15.22:44024) 2024-09-19T08:21:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.853+0000 7fcecb7fe640 1 -- 172.21.15.22:0/3545271346 learned_addr learned my addr 172.21.15.22:0/3545271346 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.853+0000 7fced40bd740 1 --2- 172.21.15.22:0/3545271346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b631d40 0x55a32b6ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.854+0000 7fcecbfff640 1 --2- 172.21.15.22:0/3545271346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b631d40 0x55a32b6ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.854+0000 7fced40bd740 1 -- 172.21.15.22:0/3545271346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fceb800ceb0 con 0x55a32b631d40 2024-09-19T08:21:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.854+0000 7fced40bd740 1 -- 172.21.15.22:0/3545271346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fceb800e320 con 0x55a32b626b60 2024-09-19T08:21:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.854+0000 7fced40bd740 1 -- 172.21.15.22:0/3545271346 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fceb800d580 con 0x55a32b627480 2024-09-19T08:21:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.855+0000 7fcecaffd640 1 -- 172.21.15.22:0/3545271346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b626b60 msgr2=0x55a32b635e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.855+0000 7fcecaffd640 1 --2- 172.21.15.22:0/3545271346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b626b60 0x55a32b635e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.855+0000 7fcecaffd640 1 -- 172.21.15.22:0/3545271346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b631d40 msgr2=0x55a32b6ab3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.855+0000 7fcecaffd640 1 --2- 172.21.15.22:0/3545271346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b631d40 0x55a32b6ab3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.855+0000 7fcecaffd640 1 -- 172.21.15.22:0/3545271346 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32b636c80 con 0x55a32b627480 2024-09-19T08:21:02.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.855+0000 7fcecbfff640 1 --2- 172.21.15.22:0/3545271346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b631d40 0x55a32b6ab3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:21:02.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.856+0000 7fced40bd740 10 client.0 _ll_register_callbacks cb 0x7fcec0009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:21:02.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.856+0000 7fced40bd740 1 -- 172.21.15.22:0/3545271346 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcebc0bb0c0 con 0x55a32b627480 2024-09-19T08:21:02.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.868+0000 7fcecaffd640 1 --2- 172.21.15.22:0/3545271346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 0x55a32b6363c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcebc001eb0 tx=0x7fcebc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ca3cf2e1a5f0df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:02.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.869+0000 7fceaaffd640 10 client.26077 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:21:02.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.869+0000 7fceaaffd640 1 -- 172.21.15.22:0/3545271346 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc0bb7c0 con 0x55a32b627480 2024-09-19T08:21:02.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.869+0000 7fceaaffd640 1 -- 172.21.15.22:0/3545271346 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcebc0b93c0 con 0x55a32b627480 2024-09-19T08:21:02.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.870+0000 7fceaaffd640 1 -- 172.21.15.22:0/3545271346 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcebc0bf160 con 0x55a32b627480 2024-09-19T08:21:02.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.871+0000 7fced40bd740 10 client.26077 fetch_fsmap learned FSMap version 30 2024-09-19T08:21:02.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.871+0000 7fced40bd740 1 -- 172.21.15.22:0/3545271346 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55a32b6ac740 con 0x55a32b627480 2024-09-19T08:21:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.875+0000 7fceaaffd640 1 -- 172.21.15.22:0/3545271346 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fcebc03a070 con 0x55a32b627480 2024-09-19T08:21:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.875+0000 7fced40bd740 10 client.26077 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:21:02.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.875+0000 7fced40bd740 10 client.26077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:21:02.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.875+0000 7fced40bd740 20 client.26077 populate_metadata read hostname 'smithi022' 2024-09-19T08:21:02.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.875+0000 7fced40bd740 1 --2- 172.21.15.22:0/3545271346 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a32b6acd80 0x55a32b6af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:02.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.875+0000 7fced40bd740 4 client.26077 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:21:02.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.875+0000 7fced40bd740 1 -- 172.21.15.22:0/3545271346 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a32b593d00 con 0x55a32b6acd80 2024-09-19T08:21:02.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.876+0000 7fcecb7fe640 1 --2- 172.21.15.22:0/3545271346 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a32b6acd80 0x55a32b6af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:02.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.879+0000 7fcecb7fe640 1 --2- 172.21.15.22:0/3545271346 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a32b6acd80 0x55a32b6af240 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b77089139ce4fd server_cookie=e0b109a90b21fe77 in_seq=0 out_seq=0 2024-09-19T08:21:02.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.879+0000 7fceaaffd640 10 client.26077 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:21:02.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.884+0000 7fceaaffd640 1 -- 172.21.15.22:0/3545271346 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a32b593d00 con 0x55a32b6acd80 2024-09-19T08:21:02.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.884+0000 7fceaaffd640 10 client.26077 handle_command_reply: tid=0 2024-09-19T08:21:02.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.884+0000 7fced40bd740 1 client.26077 shutdown 2024-09-19T08:21:02.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.886+0000 7fced40bd740 1 -- 172.21.15.22:0/3545271346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 msgr2=0x55a32b6363c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:02.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.886+0000 7fced40bd740 1 --2- 172.21.15.22:0/3545271346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 0x55a32b6363c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcebc001eb0 tx=0x7fcebc009310 comp rx=0 tx=0).stop 2024-09-19T08:21:02.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.886+0000 7fced40bd740 1 -- 172.21.15.22:0/3545271346 shutdown_connections 2024-09-19T08:21:02.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.886+0000 7fced40bd740 1 --2- 172.21.15.22:0/3545271346 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a32b6acd80 0x55a32b6af240 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.886+0000 7fced40bd740 1 --2- 172.21.15.22:0/3545271346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a32b631d40 0x55a32b6ab3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.886+0000 7fceaaffd640 0 client.26077 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:21:02.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.886+0000 7fced40bd740 1 --2- 172.21.15.22:0/3545271346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a32b627480 0x55a32b6363c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.886+0000 7fced40bd740 1 --2- 172.21.15.22:0/3545271346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a32b626b60 0x55a32b635e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.886+0000 7fced40bd740 1 -- 172.21.15.22:0/3545271346 >> 172.21.15.22:0/3545271346 conn(0x55a32b6230f0 msgr2=0x55a32b621a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:02.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.887+0000 7fced40bd740 1 -- 172.21.15.22:0/3545271346 shutdown_connections 2024-09-19T08:21:02.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.887+0000 7fced40bd740 1 -- 172.21.15.22:0/3545271346 wait complete. 2024-09-19T08:21:02.894 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:21:02.894 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-19T08:21:02.894 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "a4ff4855-83f3-407a-9efb-8f696af28963", 2024-09-19T08:21:02.894 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-19T08:21:02.895 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:21:02.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.891+0000 7fced40bd740 20 client.26077 trim_cache size 0 max 16384 2024-09-19T08:21:02.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.894+0000 7fced1ce4640 1 -- 172.21.15.22:0/1082442986 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fceb40a35c0 msgr2=0x7fceb40a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:02.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.894+0000 7fced1ce4640 1 --2- 172.21.15.22:0/1082442986 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fceb40a35c0 0x7fceb40a5a80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcec0006fd0 tx=0x7fcec0008040 comp rx=0 tx=0).stop 2024-09-19T08:21:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.894+0000 7fced1ce4640 1 -- 172.21.15.22:0/1082442986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcecc10d520 msgr2=0x7fcecc06cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.894+0000 7fced1ce4640 1 --2- 172.21.15.22:0/1082442986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcecc10d520 0x7fcecc06cca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcecc109300 tx=0x7fcebc033d10 comp rx=0 tx=0).stop 2024-09-19T08:21:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.894+0000 7fced1ce4640 1 -- 172.21.15.22:0/1082442986 shutdown_connections 2024-09-19T08:21:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.894+0000 7fced1ce4640 1 --2- 172.21.15.22:0/1082442986 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fceb40a35c0 0x7fceb40a5a80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.894+0000 7fced1ce4640 1 --2- 172.21.15.22:0/1082442986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcecc06d1e0 0x7fcecc06d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.894+0000 7fced1ce4640 1 --2- 172.21.15.22:0/1082442986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcecc10d520 0x7fcecc06cca0 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcecc109300 tx=0x7fcebc033d10 comp rx=0 tx=0).stop 2024-09-19T08:21:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.894+0000 7fced1ce4640 1 --2- 172.21.15.22:0/1082442986 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcecc1083f0 0x7fcecc073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.894+0000 7fced1ce4640 1 -- 172.21.15.22:0/1082442986 >> 172.21.15.22:0/1082442986 conn(0x7fcecc077af0 msgr2=0x7fcecc078560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.895+0000 7fced1ce4640 1 -- 172.21.15.22:0/1082442986 shutdown_connections 2024-09-19T08:21:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:02.895+0000 7fced1ce4640 1 -- 172.21.15.22:0/1082442986 wait complete. 2024-09-19T08:21:03.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:02 smithi157 ceph-mon[35143]: pgmap v183: 97 pgs: 97 active+clean; 666 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 412 op/s 2024-09-19T08:21:03.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:02 smithi022 ceph-mon[30963]: pgmap v183: 97 pgs: 97 active+clean; 666 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 412 op/s 2024-09-19T08:21:03.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:02 smithi064 ceph-mon[34900]: pgmap v183: 97 pgs: 97 active+clean; 666 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 412 op/s 2024-09-19T08:21:04.077 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:21:04.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:03 smithi022 ceph-mon[30963]: scrub queued for path: / 2024-09-19T08:21:04.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:03 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:04.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:03 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:21:04.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:03 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:21:04.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:03 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:21:04.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:03 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:21:04.196 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:03 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:04.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:03 smithi157 ceph-mon[35143]: scrub queued for path: / 2024-09-19T08:21:04.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:03 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:04.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:03 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:21:04.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:03 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:21:04.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:03 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:21:04.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:03 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:21:04.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:03 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:04.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:03 smithi064 ceph-mon[34900]: scrub queued for path: / 2024-09-19T08:21:04.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:03 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:04.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:03 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:21:04.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:03 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:21:04.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:03 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:21:04.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:03 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:21:04.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:03 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:04.611 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:21:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:04 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:04 smithi022 ceph-mon[30963]: pgmap v184: 97 pgs: 97 active+clean; 679 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 534 op/s 2024-09-19T08:21:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:04 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:21:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:04 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:04 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:21:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:04 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:04 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:21:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:04 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:04 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:21:05.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:04 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:04 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:21:05.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:04 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:04 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:04 smithi157 ceph-mon[35143]: pgmap v184: 97 pgs: 97 active+clean; 679 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 534 op/s 2024-09-19T08:21:05.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:04 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:21:05.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:04 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:04 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:21:05.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:04 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:04 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:21:05.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:04 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:04 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:21:05.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:04 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:04 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:21:05.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:04 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:04 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:04 smithi064 ceph-mon[34900]: pgmap v184: 97 pgs: 97 active+clean; 679 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 534 op/s 2024-09-19T08:21:05.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:04 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:21:05.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:04 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:04 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:21:05.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:04 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:04 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:21:05.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:04 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:04 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:21:05.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:04 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:04 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:21:05.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:04 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:21:05.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.429+0000 7f120189a640 1 -- 172.21.15.22:0/2992470032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11fc106a00 msgr2=0x7f11fc106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:05.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.429+0000 7f120189a640 1 --2- 172.21.15.22:0/2992470032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11fc106a00 0x7f11fc106de0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f11ec01a910 tx=0x7f11ec03b470 comp rx=0 tx=0).stop 2024-09-19T08:21:05.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.430+0000 7f120189a640 1 -- 172.21.15.22:0/2992470032 shutdown_connections 2024-09-19T08:21:05.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.430+0000 7f120189a640 1 --2- 172.21.15.22:0/2992470032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11fc1008c0 0x7f11fc104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.430+0000 7f120189a640 1 --2- 172.21.15.22:0/2992470032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11fc0fff40 0x7f11fc100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.430+0000 7f120189a640 1 --2- 172.21.15.22:0/2992470032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11fc106a00 0x7f11fc106de0 secure :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f11ec01a910 tx=0x7f11ec03b470 comp rx=0 tx=0).stop 2024-09-19T08:21:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.430+0000 7f120189a640 1 -- 172.21.15.22:0/2992470032 >> 172.21.15.22:0/2992470032 conn(0x7f11fc077070 msgr2=0x7f11fc0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.435+0000 7f120189a640 1 -- 172.21.15.22:0/2992470032 shutdown_connections 2024-09-19T08:21:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.436+0000 7f120189a640 1 -- 172.21.15.22:0/2992470032 wait complete. 2024-09-19T08:21:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.437+0000 7f120189a640 1 Processor -- start 2024-09-19T08:21:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.438+0000 7f120189a640 1 -- start start 2024-09-19T08:21:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.438+0000 7f120189a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11fc0fff40 0x7f11fc195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.438+0000 7f120189a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11fc1008c0 0x7f11fc196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.438+0000 7f120189a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11fc19a6c0 0x7f11fc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.438+0000 7f120189a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc109ca0 con 0x7f11fc19a6c0 2024-09-19T08:21:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.438+0000 7f120189a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc109b20 con 0x7f11fc1008c0 2024-09-19T08:21:05.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.438+0000 7f120189a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc109e20 con 0x7f11fc0fff40 2024-09-19T08:21:05.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.438+0000 7f1200898640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11fc0fff40 0x7f11fc195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:05.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.438+0000 7f1200898640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11fc0fff40 0x7f11fc195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:44042/0 (socket says 172.21.15.22:44042) 2024-09-19T08:21:05.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.438+0000 7f1200898640 1 -- 172.21.15.22:0/905218960 learned_addr learned my addr 172.21.15.22:0/905218960 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:05.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.439+0000 7f1200898640 1 -- 172.21.15.22:0/905218960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11fc1008c0 msgr2=0x7f11fc196330 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:21:05.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.439+0000 7f1200898640 1 --2- 172.21.15.22:0/905218960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11fc1008c0 0x7f11fc196330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.439+0000 7f1200898640 1 -- 172.21.15.22:0/905218960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11fc19a6c0 msgr2=0x7f11fc19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:05.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.439+0000 7f1200898640 1 --2- 172.21.15.22:0/905218960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11fc19a6c0 0x7f11fc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.439+0000 7f1200898640 1 -- 172.21.15.22:0/905218960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc19b210 con 0x7f11fc0fff40 2024-09-19T08:21:05.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.439+0000 7f1200898640 1 --2- 172.21.15.22:0/905218960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11fc0fff40 0x7f11fc195df0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f11ec040830 tx=0x7f11ec0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:05.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.439+0000 7f11f9ffb640 1 -- 172.21.15.22:0/905218960 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec011020 con 0x7f11fc0fff40 2024-09-19T08:21:05.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.439+0000 7f11f9ffb640 1 -- 172.21.15.22:0/905218960 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11ec003ce0 con 0x7f11fc0fff40 2024-09-19T08:21:05.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.439+0000 7f11f9ffb640 1 -- 172.21.15.22:0/905218960 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec046040 con 0x7f11fc0fff40 2024-09-19T08:21:05.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.439+0000 7f120189a640 1 -- 172.21.15.22:0/905218960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11fc19b4a0 con 0x7f11fc0fff40 2024-09-19T08:21:05.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.440+0000 7f120189a640 1 -- 172.21.15.22:0/905218960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11fc1c4c10 con 0x7f11fc0fff40 2024-09-19T08:21:05.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.449+0000 7f1204ed5740 1 Processor -- start 2024-09-19T08:21:05.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.451+0000 7f1204ed5740 1 -- start start 2024-09-19T08:21:05.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.451+0000 7f11f9ffb640 1 -- 172.21.15.22:0/905218960 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f11ec049070 con 0x7f11fc0fff40 2024-09-19T08:21:05.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.451+0000 7f1204ed5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302eb670 0x5637302eba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.452+0000 7f1201099640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302eb670 0x5637302eba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:05.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.452+0000 7f1201099640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302eb670 0x5637302eba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52934/0 (socket says 172.21.15.22:52934) 2024-09-19T08:21:05.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.452+0000 7f1201099640 1 -- 172.21.15.22:0/826091305 learned_addr learned my addr 172.21.15.22:0/826091305 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:05.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.452+0000 7f1204ed5740 1 --2- 172.21.15.22:0/826091305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302ebf90 0x5637302f6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.452+0000 7f1204ed5740 1 --2- 172.21.15.22:0/826091305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302f6850 0x5637302f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.452+0000 7f1204ed5740 1 -- 172.21.15.22:0/826091305 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5637302512e0 con 0x5637302eb670 2024-09-19T08:21:05.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.452+0000 7f1204ed5740 1 -- 172.21.15.22:0/826091305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5637301f8ce0 con 0x5637302ebf90 2024-09-19T08:21:05.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.452+0000 7f1204ed5740 1 -- 172.21.15.22:0/826091305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56373024d2a0 con 0x5637302f6850 2024-09-19T08:21:05.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.452+0000 7f1201099640 1 -- 172.21.15.22:0/826091305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302f6850 msgr2=0x5637302f8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:05.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.452+0000 7f1201099640 1 --2- 172.21.15.22:0/826091305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302f6850 0x5637302f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.452+0000 7f1201099640 1 -- 172.21.15.22:0/826091305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302ebf90 msgr2=0x5637302f6310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:21:05.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.452+0000 7f1201099640 1 --2- 172.21.15.22:0/826091305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302ebf90 0x5637302f6310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.452+0000 7f1201099640 1 -- 172.21.15.22:0/826091305 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637301dd440 con 0x5637302eb670 2024-09-19T08:21:05.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.453+0000 7f11f9ffb640 1 --2- 172.21.15.22:0/905218960 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f11e40a34f0 0x7f11e40a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.453+0000 7f1201099640 1 --2- 172.21.15.22:0/826091305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302eb670 0x5637302eba50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f11f000bab0 tx=0x7f11f000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92157e4d3a10668 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:05.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.453+0000 7f11f9ffb640 1 -- 172.21.15.22:0/905218960 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f11ec0f07d0 con 0x7f11fc0fff40 2024-09-19T08:21:05.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.453+0000 7f1200898640 1 --2- 172.21.15.22:0/905218960 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f11e40a34f0 0x7f11e40a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:05.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.453+0000 7f11e2ffd640 1 -- 172.21.15.22:0/826091305 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0016070 con 0x5637302eb670 2024-09-19T08:21:05.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.453+0000 7f11e2ffd640 1 -- 172.21.15.22:0/826091305 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f0005500 con 0x5637302eb670 2024-09-19T08:21:05.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.453+0000 7f11e2ffd640 1 -- 172.21.15.22:0/826091305 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0010460 con 0x5637302eb670 2024-09-19T08:21:05.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.454+0000 7f1200898640 1 --2- 172.21.15.22:0/905218960 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f11e40a34f0 0x7f11e40a59b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x5637302e8a60 tx=0x7f11ec03b470 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:05.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.455+0000 7f1204ed5740 1 -- 172.21.15.22:0/826091305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302eb670 msgr2=0x5637302eba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:05.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.455+0000 7f1204ed5740 1 --2- 172.21.15.22:0/826091305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302eb670 0x5637302eba50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f11f000bab0 tx=0x7f11f000bf70 comp rx=0 tx=0).stop 2024-09-19T08:21:05.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.458+0000 7f1204ed5740 1 -- 172.21.15.22:0/826091305 shutdown_connections 2024-09-19T08:21:05.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.458+0000 7f1204ed5740 1 --2- 172.21.15.22:0/826091305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302f6850 0x5637302f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.458+0000 7f1204ed5740 1 --2- 172.21.15.22:0/826091305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302ebf90 0x5637302f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.458+0000 7f1204ed5740 1 --2- 172.21.15.22:0/826091305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302eb670 0x5637302eba50 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.458+0000 7f1204ed5740 1 -- 172.21.15.22:0/826091305 >> 172.21.15.22:0/826091305 conn(0x5637302e7c00 msgr2=0x5637302e7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:05.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.467+0000 7f1204ed5740 1 -- 172.21.15.22:0/826091305 shutdown_connections 2024-09-19T08:21:05.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.469+0000 7f1204ed5740 1 -- 172.21.15.22:0/826091305 wait complete. 2024-09-19T08:21:05.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.470+0000 7f1204ed5740 1 -- start start 2024-09-19T08:21:05.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.470+0000 7f1204ed5740 1 Processor -- start 2024-09-19T08:21:05.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.471+0000 7f1204ed5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302eb670 0x5637302f6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.471+0000 7f1201099640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302eb670 0x5637302f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:05.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.471+0000 7f1201099640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302eb670 0x5637302f6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52936/0 (socket says 172.21.15.22:52936) 2024-09-19T08:21:05.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.472+0000 7f1201099640 1 -- 172.21.15.22:0/750608294 learned_addr learned my addr 172.21.15.22:0/750608294 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:05.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.472+0000 7f1204ed5740 1 --2- 172.21.15.22:0/750608294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302ebf90 0x5637302f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.472+0000 7f1204ed5740 1 --2- 172.21.15.22:0/750608294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302f6850 0x5637302f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.472+0000 7f11fbfff640 1 --2- 172.21.15.22:0/750608294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302ebf90 0x5637302f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:05.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.472+0000 7f1204ed5740 1 -- 172.21.15.22:0/750608294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5637302f9420 con 0x5637302eb670 2024-09-19T08:21:05.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.472+0000 7f1204ed5740 1 -- 172.21.15.22:0/750608294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56373020c050 con 0x5637302ebf90 2024-09-19T08:21:05.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.472+0000 7f1204ed5740 1 -- 172.21.15.22:0/750608294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5637302f95a0 con 0x5637302f6850 2024-09-19T08:21:05.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.473+0000 7f1201099640 1 -- 172.21.15.22:0/750608294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302f6850 msgr2=0x5637302f3ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:21:05.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.473+0000 7f1201099640 1 --2- 172.21.15.22:0/750608294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302f6850 0x5637302f3ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.473+0000 7f1201099640 1 -- 172.21.15.22:0/750608294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302ebf90 msgr2=0x5637302f34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:05.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.473+0000 7f1201099640 1 --2- 172.21.15.22:0/750608294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302ebf90 0x5637302f34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.473+0000 7f1201099640 1 -- 172.21.15.22:0/750608294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56372ffc23c0 con 0x5637302eb670 2024-09-19T08:21:05.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.473+0000 7f1201099640 1 --2- 172.21.15.22:0/750608294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302eb670 0x5637302f6220 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f11f0004700 tx=0x7f11f000dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df0debae99a736b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:05.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.474+0000 7f1204ed5740 10 client.0 _ll_register_callbacks cb 0x56373024c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:21:05.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.475+0000 7f1204ed5740 1 -- 172.21.15.22:0/750608294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f0005500 con 0x5637302eb670 2024-09-19T08:21:05.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.477+0000 7f11dbfff640 10 client.16584 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:21:05.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.477+0000 7f11dbfff640 1 -- 172.21.15.22:0/750608294 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0016070 con 0x5637302eb670 2024-09-19T08:21:05.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.477+0000 7f11dbfff640 1 -- 172.21.15.22:0/750608294 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f11f0004040 con 0x5637302eb670 2024-09-19T08:21:05.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.477+0000 7f11dbfff640 1 -- 172.21.15.22:0/750608294 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f11f0004240 con 0x5637302eb670 2024-09-19T08:21:05.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.478+0000 7f1204ed5740 10 client.16584 fetch_fsmap learned FSMap version 31 2024-09-19T08:21:05.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.478+0000 7f1204ed5740 1 -- 172.21.15.22:0/750608294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=31}) -- 0x563730304cb0 con 0x5637302eb670 2024-09-19T08:21:05.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.482+0000 7f11dbfff640 1 -- 172.21.15.22:0/750608294 <== mon.0 v2:172.21.15.22:3300/0 4 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f11f0022020 con 0x5637302eb670 2024-09-19T08:21:05.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.482+0000 7f1204ed5740 10 client.16584 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:21:05.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.482+0000 7f1204ed5740 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:21:05.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.482+0000 7f1204ed5740 20 client.16584 populate_metadata read hostname 'smithi022' 2024-09-19T08:21:05.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.482+0000 7f1204ed5740 1 --2- 172.21.15.22:0/750608294 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x563730306060 0x563730308450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.482+0000 7f1204ed5740 4 client.16584 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:21:05.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.482+0000 7f1204ed5740 1 -- 172.21.15.22:0/750608294 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f11f0005500 con 0x563730306060 2024-09-19T08:21:05.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.483+0000 7f11fbfff640 1 --2- 172.21.15.22:0/750608294 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x563730306060 0x563730308450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:05.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.485+0000 7f11fbfff640 1 --2- 172.21.15.22:0/750608294 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x563730306060 0x563730308450 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6fe5999ed637398 server_cookie=2e1726b42b7ffe9b in_seq=0 out_seq=0 2024-09-19T08:21:05.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.486+0000 7f11dbfff640 10 client.16584 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:21:05.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.487+0000 7f11dbfff640 1 -- 172.21.15.22:0/750608294 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f11f0005500 con 0x563730306060 2024-09-19T08:21:05.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.487+0000 7f11dbfff640 10 client.16584 handle_command_reply: tid=0 2024-09-19T08:21:05.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.487+0000 7f1204ed5740 1 client.16584 shutdown 2024-09-19T08:21:05.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.497+0000 7f1204ed5740 1 -- 172.21.15.22:0/750608294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302eb670 msgr2=0x5637302f6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:05.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.497+0000 7f1204ed5740 1 --2- 172.21.15.22:0/750608294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302eb670 0x5637302f6220 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f11f0004700 tx=0x7f11f000dcd0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.497+0000 7f1204ed5740 1 -- 172.21.15.22:0/750608294 shutdown_connections 2024-09-19T08:21:05.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.497+0000 7f1204ed5740 1 --2- 172.21.15.22:0/750608294 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x563730306060 0x563730308450 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.497+0000 7f1204ed5740 1 --2- 172.21.15.22:0/750608294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302f6850 0x5637302f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.497+0000 7f1204ed5740 1 --2- 172.21.15.22:0/750608294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302ebf90 0x5637302f34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.497+0000 7f1204ed5740 1 --2- 172.21.15.22:0/750608294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302eb670 0x5637302f6220 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.497+0000 7f1204ed5740 1 -- 172.21.15.22:0/750608294 >> 172.21.15.22:0/750608294 conn(0x5637302e7c00 msgr2=0x5637302ed930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:05.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.498+0000 7f11dbfff640 0 client.16584 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:21:05.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.498+0000 7f1204ed5740 1 -- 172.21.15.22:0/750608294 shutdown_connections 2024-09-19T08:21:05.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.499+0000 7f1204ed5740 1 -- 172.21.15.22:0/750608294 wait complete. 2024-09-19T08:21:05.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.503+0000 7f1204ed5740 20 client.16584 trim_cache size 0 max 16384 2024-09-19T08:21:05.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.570+0000 7f1204ed5740 1 Processor -- start 2024-09-19T08:21:05.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.570+0000 7f1204ed5740 1 -- start start 2024-09-19T08:21:05.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.570+0000 7f1204ed5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302eb670 0x5637302e7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.570+0000 7f1204ed5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302ebf90 0x56373030db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.570+0000 7f1204ed5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302f6850 0x56373030e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.570+0000 7f1201099640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302eb670 0x5637302e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:05.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.570+0000 7f1201099640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302eb670 0x5637302e7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:45938/0 (socket says 172.21.15.22:45938) 2024-09-19T08:21:05.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.570+0000 7f1201099640 1 -- 172.21.15.22:0/1113418050 learned_addr learned my addr 172.21.15.22:0/1113418050 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:05.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.570+0000 7f1204ed5740 1 -- 172.21.15.22:0/1113418050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc1d0060 con 0x5637302ebf90 2024-09-19T08:21:05.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.570+0000 7f1204ed5740 1 -- 172.21.15.22:0/1113418050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc1cd8d0 con 0x5637302eb670 2024-09-19T08:21:05.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.571+0000 7f1204ed5740 1 -- 172.21.15.22:0/1113418050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc1cb720 con 0x5637302f6850 2024-09-19T08:21:05.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.572+0000 7f1201099640 1 --2- 172.21.15.22:0/1113418050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302f6850 0x56373030e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:05.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.573+0000 7f11fbfff640 1 --2- 172.21.15.22:0/1113418050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302ebf90 0x56373030db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:05.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.573+0000 7f11fbfff640 1 -- 172.21.15.22:0/1113418050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302f6850 msgr2=0x56373030e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:05.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.573+0000 7f11fbfff640 1 --2- 172.21.15.22:0/1113418050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302f6850 0x56373030e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.573+0000 7f11fbfff640 1 -- 172.21.15.22:0/1113418050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302eb670 msgr2=0x5637302e7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:05.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.573+0000 7f11fbfff640 1 --2- 172.21.15.22:0/1113418050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302eb670 0x5637302e7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.574+0000 7f11fbfff640 1 -- 172.21.15.22:0/1113418050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563730302dc0 con 0x5637302ebf90 2024-09-19T08:21:05.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.574+0000 7f11fbfff640 1 --2- 172.21.15.22:0/1113418050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302ebf90 0x56373030db70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x563730308b10 tx=0x7f11f4009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d58d9f0d337af32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:05.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.575+0000 7f11e27fc640 1 -- 172.21.15.22:0/1113418050 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4018650 con 0x5637302ebf90 2024-09-19T08:21:05.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.575+0000 7f11e27fc640 1 -- 172.21.15.22:0/1113418050 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f4018cd0 con 0x5637302ebf90 2024-09-19T08:21:05.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.576+0000 7f1204ed5740 1 -- 172.21.15.22:0/1113418050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302ebf90 msgr2=0x56373030db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.576+0000 7f1204ed5740 1 --2- 172.21.15.22:0/1113418050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302ebf90 0x56373030db70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x563730308b10 tx=0x7f11f4009550 comp rx=0 tx=0).stop 2024-09-19T08:21:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.577+0000 7f1204ed5740 1 -- 172.21.15.22:0/1113418050 shutdown_connections 2024-09-19T08:21:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.577+0000 7f1204ed5740 1 --2- 172.21.15.22:0/1113418050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302f6850 0x56373030e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.577+0000 7f1204ed5740 1 --2- 172.21.15.22:0/1113418050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302ebf90 0x56373030db70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.577+0000 7f1204ed5740 1 --2- 172.21.15.22:0/1113418050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302eb670 0x5637302e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.577+0000 7f1204ed5740 1 -- 172.21.15.22:0/1113418050 >> 172.21.15.22:0/1113418050 conn(0x5637302e7c00 msgr2=0x5637302eead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.579+0000 7f1204ed5740 1 -- 172.21.15.22:0/1113418050 shutdown_connections 2024-09-19T08:21:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.580+0000 7f1204ed5740 1 -- 172.21.15.22:0/1113418050 wait complete. 2024-09-19T08:21:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.580+0000 7f1204ed5740 1 -- start start 2024-09-19T08:21:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.580+0000 7f1204ed5740 1 Processor -- start 2024-09-19T08:21:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.581+0000 7f1204ed5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302eb670 0x5637302fee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.581+0000 7f1204ed5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302ebf90 0x5637302f0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.581+0000 7f1204ed5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302f6850 0x5637302f0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.581+0000 7f1204ed5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc1cbd90 con 0x5637302f6850 2024-09-19T08:21:05.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.581+0000 7f1204ed5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc1cd200 con 0x5637302eb670 2024-09-19T08:21:05.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.581+0000 7f1204ed5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc1cc460 con 0x5637302ebf90 2024-09-19T08:21:05.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.581+0000 7f1204ed5740 10 client.0 _ll_register_callbacks cb 0x7f11f000b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:21:05.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.582+0000 7f1201099640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302eb670 0x5637302fee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:05.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.582+0000 7f1201099640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302eb670 0x5637302fee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:45948/0 (socket says 172.21.15.22:45948) 2024-09-19T08:21:05.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.582+0000 7f1201099640 1 -- 172.21.15.22:0/1925874017 learned_addr learned my addr 172.21.15.22:0/1925874017 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:05.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.582+0000 7f1201099640 1 --2- 172.21.15.22:0/1925874017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302f6850 0x5637302f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:05.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.583+0000 7f1201099640 1 -- 172.21.15.22:0/1925874017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302ebf90 msgr2=0x5637302f0220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:21:05.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.583+0000 7f1201099640 1 --2- 172.21.15.22:0/1925874017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302ebf90 0x5637302f0220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.583+0000 7f1201099640 1 -- 172.21.15.22:0/1925874017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302eb670 msgr2=0x5637302fee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:05.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.583+0000 7f1201099640 1 --2- 172.21.15.22:0/1925874017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302eb670 0x5637302fee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.583+0000 7f1201099640 1 -- 172.21.15.22:0/1925874017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637302f0dd0 con 0x5637302f6850 2024-09-19T08:21:05.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.584+0000 7f1201099640 1 --2- 172.21.15.22:0/1925874017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302f6850 0x5637302f0760 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f11f000d770 tx=0x7f11f001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c29a3c4d2d98143 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:05.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.586+0000 7f11dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:21:05.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.586+0000 7f11dbfff640 1 -- 172.21.15.22:0/1925874017 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0035070 con 0x5637302f6850 2024-09-19T08:21:05.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.586+0000 7f11dbfff640 1 -- 172.21.15.22:0/1925874017 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f11f002d470 con 0x5637302f6850 2024-09-19T08:21:05.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.587+0000 7f1204ed5740 1 -- 172.21.15.22:0/1925874017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f4018cd0 con 0x5637302f6850 2024-09-19T08:21:05.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.587+0000 7f11dbfff640 1 -- 172.21.15.22:0/1925874017 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f11f002d670 con 0x5637302f6850 2024-09-19T08:21:05.726 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:21:05.726 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:21:05.727 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-19T08:21:05.727 INFO:teuthology.orchestra.run.smithi022.stdout: "a4ff4855-83f3-407a-9efb-8f696af28963": { 2024-09-19T08:21:05.727 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-19T08:21:05.727 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "a4ff4855-83f3-407a-9efb-8f696af28963", 2024-09-19T08:21:05.727 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-09-19T08:21:05.727 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:21:05.727 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:21:05.727 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:21:05.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.589+0000 7f1204ed5740 10 client.16590 fetch_fsmap learned FSMap version 31 2024-09-19T08:21:05.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.589+0000 7f1204ed5740 1 -- 172.21.15.22:0/1925874017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5637303819d0 con 0x5637302f6850 2024-09-19T08:21:05.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.589+0000 7f11dbfff640 1 -- 172.21.15.22:0/1925874017 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f4018cd0 con 0x5637302f6850 2024-09-19T08:21:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.590+0000 7f11dbfff640 1 -- 172.21.15.22:0/1925874017 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f11f00107b0 con 0x5637302f6850 2024-09-19T08:21:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.590+0000 7f1204ed5740 10 client.16590 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:21:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.590+0000 7f1204ed5740 10 client.16590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:21:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.590+0000 7f1204ed5740 20 client.16590 populate_metadata read hostname 'smithi022' 2024-09-19T08:21:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.590+0000 7f1204ed5740 1 --2- 172.21.15.22:0/1925874017 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x563730382020 0x563730384410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.590+0000 7f1204ed5740 4 client.16590 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:21:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.590+0000 7f1204ed5740 1 -- 172.21.15.22:0/1925874017 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56373025e850 con 0x563730382020 2024-09-19T08:21:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.591+0000 7f11fbfff640 1 --2- 172.21.15.22:0/1925874017 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x563730382020 0x563730384410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.595+0000 7f11fbfff640 1 --2- 172.21.15.22:0/1925874017 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x563730382020 0x563730384410 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=800710171f48ba84 server_cookie=4dd7ebc8fe13598c in_seq=0 out_seq=0 2024-09-19T08:21:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.595+0000 7f11dbfff640 10 client.16590 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:21:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.596+0000 7f11dbfff640 1 -- 172.21.15.22:0/1925874017 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56373025e850 con 0x563730382020 2024-09-19T08:21:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.596+0000 7f11dbfff640 10 client.16590 handle_command_reply: tid=0 2024-09-19T08:21:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.596+0000 7f1204ed5740 1 client.16590 shutdown 2024-09-19T08:21:05.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.598+0000 7f1204ed5740 1 -- 172.21.15.22:0/1925874017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302f6850 msgr2=0x5637302f0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:05.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.598+0000 7f1204ed5740 1 --2- 172.21.15.22:0/1925874017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302f6850 0x5637302f0760 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f11f000d770 tx=0x7f11f001e000 comp rx=0 tx=0).stop 2024-09-19T08:21:05.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.599+0000 7f1204ed5740 1 -- 172.21.15.22:0/1925874017 shutdown_connections 2024-09-19T08:21:05.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.599+0000 7f1204ed5740 1 --2- 172.21.15.22:0/1925874017 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x563730382020 0x563730384410 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.599+0000 7f1204ed5740 1 --2- 172.21.15.22:0/1925874017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5637302f6850 0x5637302f0760 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.599+0000 7f1204ed5740 1 --2- 172.21.15.22:0/1925874017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637302ebf90 0x5637302f0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.599+0000 7f1204ed5740 1 --2- 172.21.15.22:0/1925874017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637302eb670 0x5637302fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.599+0000 7f1204ed5740 1 -- 172.21.15.22:0/1925874017 >> 172.21.15.22:0/1925874017 conn(0x5637302e7c00 msgr2=0x5637302ed0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:05.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.600+0000 7f11dbfff640 0 client.16590 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:21:05.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.602+0000 7f1204ed5740 1 -- 172.21.15.22:0/1925874017 shutdown_connections 2024-09-19T08:21:05.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.606+0000 7f1204ed5740 1 -- 172.21.15.22:0/1925874017 wait complete. 2024-09-19T08:21:05.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.606+0000 7f1204ed5740 20 client.16590 trim_cache size 0 max 16384 2024-09-19T08:21:05.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.610+0000 7f120189a640 1 -- 172.21.15.22:0/905218960 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f11e40a34f0 msgr2=0x7f11e40a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:05.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.610+0000 7f120189a640 1 --2- 172.21.15.22:0/905218960 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f11e40a34f0 0x7f11e40a59b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x5637302e8a60 tx=0x7f11ec03b470 comp rx=0 tx=0).stop 2024-09-19T08:21:05.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.610+0000 7f120189a640 1 -- 172.21.15.22:0/905218960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11fc0fff40 msgr2=0x7f11fc195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:05.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.610+0000 7f120189a640 1 --2- 172.21.15.22:0/905218960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11fc0fff40 0x7f11fc195df0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f11ec040830 tx=0x7f11ec0045e0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.611+0000 7f120189a640 1 -- 172.21.15.22:0/905218960 shutdown_connections 2024-09-19T08:21:05.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.611+0000 7f120189a640 1 --2- 172.21.15.22:0/905218960 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f11e40a34f0 0x7f11e40a59b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.611+0000 7f120189a640 1 --2- 172.21.15.22:0/905218960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11fc19a6c0 0x7f11fc19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.611+0000 7f120189a640 1 --2- 172.21.15.22:0/905218960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11fc1008c0 0x7f11fc196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.611+0000 7f120189a640 1 --2- 172.21.15.22:0/905218960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11fc0fff40 0x7f11fc195df0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:05.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.611+0000 7f120189a640 1 -- 172.21.15.22:0/905218960 >> 172.21.15.22:0/905218960 conn(0x7f11fc077070 msgr2=0x7f11fc1040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:05.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.612+0000 7f120189a640 1 -- 172.21.15.22:0/905218960 shutdown_connections 2024-09-19T08:21:05.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:05.612+0000 7f120189a640 1 -- 172.21.15.22:0/905218960 wait complete. 2024-09-19T08:21:06.155 INFO:tasks.workunit.client.0.smithi022.stdout: 131 1907609 480 1231411 483 1092193 1324 2024-09-19T08:21:06.664 INFO:tasks.cephfs.filesystem:scrub status for tag:a4ff4855-83f3-407a-9efb-8f696af28963 - {'path': '/', 'tag': 'a4ff4855-83f3-407a-9efb-8f696af28963', 'options': 'recursive,force'} 2024-09-19T08:21:07.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:06 smithi157 ceph-mon[35143]: pgmap v185: 97 pgs: 97 active+clean; 679 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-09-19T08:21:07.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:06 smithi022 ceph-mon[30963]: pgmap v185: 97 pgs: 97 active+clean; 679 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-09-19T08:21:07.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:06 smithi064 ceph-mon[34900]: pgmap v185: 97 pgs: 97 active+clean; 679 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-09-19T08:21:09.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:08 smithi157 ceph-mon[35143]: pgmap v186: 97 pgs: 97 active+clean; 680 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-09-19T08:21:09.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:08 smithi022 ceph-mon[30963]: pgmap v186: 97 pgs: 97 active+clean; 680 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-09-19T08:21:09.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:08 smithi064 ceph-mon[34900]: pgmap v186: 97 pgs: 97 active+clean; 680 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-09-19T08:21:11.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:10 smithi157 ceph-mon[35143]: pgmap v187: 97 pgs: 97 active+clean; 692 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 467 op/s 2024-09-19T08:21:11.260 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:10 smithi022 ceph-mon[30963]: pgmap v187: 97 pgs: 97 active+clean; 692 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 467 op/s 2024-09-19T08:21:11.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:10 smithi064 ceph-mon[34900]: pgmap v187: 97 pgs: 97 active+clean; 692 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 467 op/s 2024-09-19T08:21:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:12 smithi157 ceph-mon[35143]: pgmap v188: 97 pgs: 97 active+clean; 692 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.5 MiB/s wr, 417 op/s 2024-09-19T08:21:13.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:12 smithi022 ceph-mon[30963]: pgmap v188: 97 pgs: 97 active+clean; 692 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.5 MiB/s wr, 417 op/s 2024-09-19T08:21:13.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:12 smithi064 ceph-mon[34900]: pgmap v188: 97 pgs: 97 active+clean; 692 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.5 MiB/s wr, 417 op/s 2024-09-19T08:21:15.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:14 smithi157 ceph-mon[35143]: pgmap v189: 97 pgs: 97 active+clean; 772 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.4 MiB/s wr, 1.04k op/s 2024-09-19T08:21:15.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:14 smithi022 ceph-mon[30963]: pgmap v189: 97 pgs: 97 active+clean; 772 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.4 MiB/s wr, 1.04k op/s 2024-09-19T08:21:15.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:14 smithi064 ceph-mon[34900]: pgmap v189: 97 pgs: 97 active+clean; 772 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.4 MiB/s wr, 1.04k op/s 2024-09-19T08:21:16.156 INFO:tasks.workunit.client.0.smithi022.stdout: 134 1097655 213 724079 342 626922 786 2024-09-19T08:21:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:15 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:21:16.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:15 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:21:16.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:15 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:21:17.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:16 smithi157 ceph-mon[35143]: pgmap v190: 97 pgs: 97 active+clean; 772 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.1 MiB/s wr, 921 op/s 2024-09-19T08:21:17.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:16 smithi022 ceph-mon[30963]: pgmap v190: 97 pgs: 97 active+clean; 772 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.1 MiB/s wr, 921 op/s 2024-09-19T08:21:17.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:16 smithi064 ceph-mon[34900]: pgmap v190: 97 pgs: 97 active+clean; 772 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.1 MiB/s wr, 921 op/s 2024-09-19T08:21:18.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:17 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:21:18.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:17 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:21:18.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:17 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:21:18.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:17 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:21:18.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:17 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:21:18.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:17 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:21:18.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:17 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:21:18.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:17 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:21:18.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:17 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:21:19.010 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:19 smithi022 ceph-mon[30963]: pgmap v191: 97 pgs: 97 active+clean; 771 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.5 MiB/s wr, 995 op/s 2024-09-19T08:21:19.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:19 smithi064 ceph-mon[34900]: pgmap v191: 97 pgs: 97 active+clean; 771 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.5 MiB/s wr, 995 op/s 2024-09-19T08:21:19.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:19 smithi157 ceph-mon[35143]: pgmap v191: 97 pgs: 97 active+clean; 771 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.5 MiB/s wr, 995 op/s 2024-09-19T08:21:21.305 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:21 smithi157 ceph-mon[35143]: pgmap v192: 97 pgs: 97 active+clean; 759 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.8 MiB/s wr, 1.09k op/s 2024-09-19T08:21:21.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:21 smithi064 ceph-mon[34900]: pgmap v192: 97 pgs: 97 active+clean; 759 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.8 MiB/s wr, 1.09k op/s 2024-09-19T08:21:21.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:21 smithi022 ceph-mon[30963]: pgmap v192: 97 pgs: 97 active+clean; 759 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.8 MiB/s wr, 1.09k op/s 2024-09-19T08:21:23.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:23 smithi064 ceph-mon[34900]: pgmap v193: 97 pgs: 97 active+clean; 759 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.7 MiB/s wr, 878 op/s 2024-09-19T08:21:23.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:23 smithi157 ceph-mon[35143]: pgmap v193: 97 pgs: 97 active+clean; 759 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.7 MiB/s wr, 878 op/s 2024-09-19T08:21:23.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:23 smithi022 ceph-mon[30963]: pgmap v193: 97 pgs: 97 active+clean; 759 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.7 MiB/s wr, 878 op/s 2024-09-19T08:21:25.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:25 smithi064 ceph-mon[34900]: pgmap v194: 97 pgs: 97 active+clean; 775 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 968 op/s 2024-09-19T08:21:25.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:25 smithi157 ceph-mon[35143]: pgmap v194: 97 pgs: 97 active+clean; 775 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 968 op/s 2024-09-19T08:21:25.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:25 smithi022 ceph-mon[30963]: pgmap v194: 97 pgs: 97 active+clean; 775 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 968 op/s 2024-09-19T08:21:26.156 INFO:tasks.workunit.client.0.smithi022.stdout: 140 1987542 582 1315273 358 1141660 1838 2024-09-19T08:21:27.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:27 smithi064 ceph-mon[34900]: pgmap v195: 97 pgs: 97 active+clean; 775 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-09-19T08:21:27.502 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:27 smithi157 ceph-mon[35143]: pgmap v195: 97 pgs: 97 active+clean; 775 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-09-19T08:21:27.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:27 smithi022 ceph-mon[30963]: pgmap v195: 97 pgs: 97 active+clean; 775 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-09-19T08:21:29.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:29 smithi064 ceph-mon[34900]: pgmap v196: 97 pgs: 97 active+clean; 779 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 375 op/s 2024-09-19T08:21:29.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:29 smithi157 ceph-mon[35143]: pgmap v196: 97 pgs: 97 active+clean; 779 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 375 op/s 2024-09-19T08:21:29.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:29 smithi022 ceph-mon[30963]: pgmap v196: 97 pgs: 97 active+clean; 779 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 375 op/s 2024-09-19T08:21:31.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:31 smithi064 ceph-mon[34900]: pgmap v197: 97 pgs: 97 active+clean; 789 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.2 MiB/s wr, 394 op/s 2024-09-19T08:21:31.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:31 smithi157 ceph-mon[35143]: pgmap v197: 97 pgs: 97 active+clean; 789 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.2 MiB/s wr, 394 op/s 2024-09-19T08:21:31.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:31 smithi022 ceph-mon[30963]: pgmap v197: 97 pgs: 97 active+clean; 789 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.2 MiB/s wr, 394 op/s 2024-09-19T08:21:33.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:33 smithi064 ceph-mon[34900]: pgmap v198: 97 pgs: 97 active+clean; 789 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 215 op/s 2024-09-19T08:21:33.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:33 smithi157 ceph-mon[35143]: pgmap v198: 97 pgs: 97 active+clean; 789 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 215 op/s 2024-09-19T08:21:33.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:33 smithi022 ceph-mon[30963]: pgmap v198: 97 pgs: 97 active+clean; 789 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 215 op/s 2024-09-19T08:21:35.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:35 smithi064 ceph-mon[34900]: pgmap v199: 97 pgs: 97 active+clean; 825 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.0 MiB/s wr, 636 op/s 2024-09-19T08:21:35.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:35 smithi157 ceph-mon[35143]: pgmap v199: 97 pgs: 97 active+clean; 825 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.0 MiB/s wr, 636 op/s 2024-09-19T08:21:35.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:35 smithi022 ceph-mon[30963]: pgmap v199: 97 pgs: 97 active+clean; 825 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.0 MiB/s wr, 636 op/s 2024-09-19T08:21:36.156 INFO:tasks.workunit.client.0.smithi022.stdout: 144 1514681 415 990965 283 843345 1059 2024-09-19T08:21:36.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:21:37.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:37 smithi064 ceph-mon[34900]: pgmap v200: 97 pgs: 97 active+clean; 825 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.5 MiB/s wr, 546 op/s 2024-09-19T08:21:37.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:37 smithi157 ceph-mon[35143]: pgmap v200: 97 pgs: 97 active+clean; 825 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.5 MiB/s wr, 546 op/s 2024-09-19T08:21:37.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:37 smithi022 ceph-mon[30963]: pgmap v200: 97 pgs: 97 active+clean; 825 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.5 MiB/s wr, 546 op/s 2024-09-19T08:21:39.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:39 smithi064 ceph-mon[34900]: pgmap v201: 97 pgs: 97 active+clean; 832 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 629 op/s 2024-09-19T08:21:39.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:39 smithi157 ceph-mon[35143]: pgmap v201: 97 pgs: 97 active+clean; 832 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 629 op/s 2024-09-19T08:21:39.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:39 smithi022 ceph-mon[30963]: pgmap v201: 97 pgs: 97 active+clean; 832 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 629 op/s 2024-09-19T08:21:39.896 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:21:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.896+0000 7f39f3ce8640 1 -- 172.21.15.22:0/683535430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39ec112a40 msgr2=0x7f39ec114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.896+0000 7f39f3ce8640 1 --2- 172.21.15.22:0/683535430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39ec112a40 0x7f39ec114e30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009ca0 tx=0x7f39e402a8b0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.898+0000 7f39f3ce8640 1 -- 172.21.15.22:0/683535430 shutdown_connections 2024-09-19T08:21:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.898+0000 7f39f3ce8640 1 --2- 172.21.15.22:0/683535430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39ec112a40 0x7f39ec114e30 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.898+0000 7f39f3ce8640 1 --2- 172.21.15.22:0/683535430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec077040 0x7f39ec112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.898+0000 7f39f3ce8640 1 --2- 172.21.15.22:0/683535430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39ec076720 0x7f39ec076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.898+0000 7f39f3ce8640 1 -- 172.21.15.22:0/683535430 >> 172.21.15.22:0/683535430 conn(0x7f39ec0ffa20 msgr2=0x7f39ec101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:40.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.903+0000 7f39f3ce8640 1 -- 172.21.15.22:0/683535430 shutdown_connections 2024-09-19T08:21:40.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.903+0000 7f39f3ce8640 1 -- 172.21.15.22:0/683535430 wait complete. 2024-09-19T08:21:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.904+0000 7f39f3ce8640 1 Processor -- start 2024-09-19T08:21:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.904+0000 7f39f3ce8640 1 -- start start 2024-09-19T08:21:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.904+0000 7f39f3ce8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec076720 0x7f39ec111d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.904+0000 7f39f3ce8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39ec077040 0x7f39ec1122c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:40.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.904+0000 7f39f3ce8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39ec112a40 0x7f39ec10be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.904+0000 7f39f3ce8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec1042c0 con 0x7f39ec112a40 2024-09-19T08:21:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.904+0000 7f39f3ce8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec104140 con 0x7f39ec076720 2024-09-19T08:21:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.904+0000 7f39f3ce8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec104440 con 0x7f39ec077040 2024-09-19T08:21:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.905+0000 7f39f1a84640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec076720 0x7f39ec111d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.905+0000 7f39f1a84640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec076720 0x7f39ec111d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:35980/0 (socket says 172.21.15.22:35980) 2024-09-19T08:21:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.905+0000 7f39f1a84640 1 -- 172.21.15.22:0/2230900447 learned_addr learned my addr 172.21.15.22:0/2230900447 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.905+0000 7f39f2285640 1 --2- 172.21.15.22:0/2230900447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39ec112a40 0x7f39ec10be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.905+0000 7f39f1a84640 1 -- 172.21.15.22:0/2230900447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39ec077040 msgr2=0x7f39ec1122c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:21:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.905+0000 7f39f1a84640 1 --2- 172.21.15.22:0/2230900447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39ec077040 0x7f39ec1122c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.905+0000 7f39f1a84640 1 -- 172.21.15.22:0/2230900447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39ec112a40 msgr2=0x7f39ec10be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.905+0000 7f39f1a84640 1 --2- 172.21.15.22:0/2230900447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39ec112a40 0x7f39ec10be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.905+0000 7f39f1a84640 1 -- 172.21.15.22:0/2230900447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ec10c730 con 0x7f39ec076720 2024-09-19T08:21:40.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.906+0000 7f39f1a84640 1 --2- 172.21.15.22:0/2230900447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec076720 0x7f39ec111d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f39e800e9f0 tx=0x7f39e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:40.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.913+0000 7f39e2ffd640 1 -- 172.21.15.22:0/2230900447 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e800cd80 con 0x7f39ec076720 2024-09-19T08:21:40.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.913+0000 7f39e2ffd640 1 -- 172.21.15.22:0/2230900447 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e8010910 con 0x7f39ec076720 2024-09-19T08:21:40.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.913+0000 7f39e2ffd640 1 -- 172.21.15.22:0/2230900447 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8018950 con 0x7f39ec076720 2024-09-19T08:21:40.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.914+0000 7f39f3ce8640 1 -- 172.21.15.22:0/2230900447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39ec10ca20 con 0x7f39ec076720 2024-09-19T08:21:40.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.914+0000 7f39f3ce8640 1 -- 172.21.15.22:0/2230900447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39ec11e1c0 con 0x7f39ec076720 2024-09-19T08:21:40.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.921+0000 7f39f60c1740 1 Processor -- start 2024-09-19T08:21:40.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.924+0000 7f39e2ffd640 1 -- 172.21.15.22:0/2230900447 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f39e8010ac0 con 0x7f39ec076720 2024-09-19T08:21:40.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.929+0000 7f39e2ffd640 1 --2- 172.21.15.22:0/2230900447 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f39d80a35c0 0x7f39d80a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:40.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.929+0000 7f39e2ffd640 1 -- 172.21.15.22:0/2230900447 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f39e80d01a0 con 0x7f39ec076720 2024-09-19T08:21:40.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.929+0000 7f39f60c1740 1 -- start start 2024-09-19T08:21:40.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.930+0000 7f39f60c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09aa64a0 0x558f09aa6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:40.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.930+0000 7f39f60c1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa6dc0 0x558f09ab1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:40.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.930+0000 7f39f60c1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09ab1680 0x558f09ab3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:40.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.930+0000 7f39f60c1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558f09a0c2e0 con 0x558f09aa6dc0 2024-09-19T08:21:40.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.930+0000 7f39f60c1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558f099b3ce0 con 0x558f09ab1680 2024-09-19T08:21:40.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.930+0000 7f39f60c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558f09a082a0 con 0x558f09aa64a0 2024-09-19T08:21:40.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.930+0000 7f39f2285640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09ab1680 0x558f09ab3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:40.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.930+0000 7f39f2285640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09ab1680 0x558f09ab3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:35992/0 (socket says 172.21.15.22:35992) 2024-09-19T08:21:40.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.930+0000 7f39f2285640 1 -- 172.21.15.22:0/3039548076 learned_addr learned my addr 172.21.15.22:0/3039548076 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:40.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.931+0000 7f39f2285640 1 --2- 172.21.15.22:0/2230900447 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f39d80a35c0 0x7f39d80a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:40.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.931+0000 7f39f1a84640 1 --2- 172.21.15.22:0/3039548076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09aa64a0 0x558f09aa6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:40.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.931+0000 7f39f1283640 1 --2- 172.21.15.22:0/3039548076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa6dc0 0x558f09ab1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:40.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.931+0000 7f39f2285640 1 -- 172.21.15.22:0/3039548076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09aa64a0 msgr2=0x558f09aa6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:40.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.931+0000 7f39f2285640 1 --2- 172.21.15.22:0/3039548076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09aa64a0 0x558f09aa6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.931+0000 7f39f2285640 1 -- 172.21.15.22:0/3039548076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa6dc0 msgr2=0x558f09ab1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:40.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.931+0000 7f39f2285640 1 --2- 172.21.15.22:0/3039548076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa6dc0 0x558f09ab1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.931+0000 7f39f2285640 1 -- 172.21.15.22:0/3039548076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f09998440 con 0x558f09ab1680 2024-09-19T08:21:40.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.931+0000 7f39f2285640 1 --2- 172.21.15.22:0/3039548076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09ab1680 0x558f09ab3a70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f39ec07aa10 tx=0x7f39e4031570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9c9357baa8ab59b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:40.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.932+0000 7f39f2285640 1 --2- 172.21.15.22:0/2230900447 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f39d80a35c0 0x7f39d80a5a80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f39e4031020 tx=0x7f39e4048000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:40.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.932+0000 7f39d7fff640 1 -- 172.21.15.22:0/3039548076 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4003df0 con 0x558f09ab1680 2024-09-19T08:21:40.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.932+0000 7f39d7fff640 1 -- 172.21.15.22:0/3039548076 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e404e070 con 0x558f09ab1680 2024-09-19T08:21:40.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.932+0000 7f39d7fff640 1 -- 172.21.15.22:0/3039548076 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e40303a0 con 0x558f09ab1680 2024-09-19T08:21:40.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.934+0000 7f39f60c1740 1 -- 172.21.15.22:0/3039548076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09ab1680 msgr2=0x558f09ab3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:40.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.934+0000 7f39f60c1740 1 --2- 172.21.15.22:0/3039548076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09ab1680 0x558f09ab3a70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f39ec07aa10 tx=0x7f39e4031570 comp rx=0 tx=0).stop 2024-09-19T08:21:40.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.939+0000 7f39f60c1740 1 -- 172.21.15.22:0/3039548076 shutdown_connections 2024-09-19T08:21:40.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.939+0000 7f39f60c1740 1 --2- 172.21.15.22:0/3039548076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09ab1680 0x558f09ab3a70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.939+0000 7f39f60c1740 1 --2- 172.21.15.22:0/3039548076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa6dc0 0x558f09ab1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.939+0000 7f39f60c1740 1 --2- 172.21.15.22:0/3039548076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09aa64a0 0x558f09aa6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.939+0000 7f39f60c1740 1 -- 172.21.15.22:0/3039548076 >> 172.21.15.22:0/3039548076 conn(0x558f09aa2a30 msgr2=0x558f09aa2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:40.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.948+0000 7f39f60c1740 1 -- 172.21.15.22:0/3039548076 shutdown_connections 2024-09-19T08:21:40.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.953+0000 7f39f60c1740 1 -- 172.21.15.22:0/3039548076 wait complete. 2024-09-19T08:21:40.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.953+0000 7f39f60c1740 1 -- start start 2024-09-19T08:21:40.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.953+0000 7f39f60c1740 1 Processor -- start 2024-09-19T08:21:40.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.954+0000 7f39f60c1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa64a0 0x558f09ab1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:40.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.954+0000 7f39f60c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09aa6dc0 0x558f09aae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:40.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.954+0000 7f39f60c1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09ab1680 0x558f09aae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:40.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.955+0000 7f39f60c1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558f09ac5260 con 0x558f09ab1680 2024-09-19T08:21:40.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.955+0000 7f39f60c1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558f099c7050 con 0x558f09aa64a0 2024-09-19T08:21:40.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.955+0000 7f39f60c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558f09ac53e0 con 0x558f09aa6dc0 2024-09-19T08:21:40.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.955+0000 7f39f60c1740 10 client.0 _ll_register_callbacks cb 0x558f09a07160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:21:40.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.955+0000 7f39f1a84640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa64a0 0x558f09ab1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:40.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.955+0000 7f39f2285640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09ab1680 0x558f09aae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:40.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.955+0000 7f39f2285640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09ab1680 0x558f09aae9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32902/0 (socket says 172.21.15.22:32902) 2024-09-19T08:21:40.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.955+0000 7f39f2285640 1 -- 172.21.15.22:0/533501018 learned_addr learned my addr 172.21.15.22:0/533501018 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:40.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.955+0000 7f39f1283640 1 --2- 172.21.15.22:0/533501018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09aa6dc0 0x558f09aae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:40.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.956+0000 7f39f2285640 1 -- 172.21.15.22:0/533501018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09aa6dc0 msgr2=0x558f09aae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:40.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.956+0000 7f39f2285640 1 --2- 172.21.15.22:0/533501018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09aa6dc0 0x558f09aae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.956+0000 7f39f2285640 1 -- 172.21.15.22:0/533501018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa64a0 msgr2=0x558f09ab1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:40.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.956+0000 7f39f2285640 1 --2- 172.21.15.22:0/533501018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa64a0 0x558f09ab1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.956+0000 7f39f2285640 1 -- 172.21.15.22:0/533501018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f0977d3c0 con 0x558f09ab1680 2024-09-19T08:21:40.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.956+0000 7f39f2285640 1 --2- 172.21.15.22:0/533501018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09ab1680 0x558f09aae9d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009950 tx=0x7f39e4030f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666ec2acc9a43cad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:40.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.957+0000 7f39d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:21:40.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.957+0000 7f39d57fa640 1 -- 172.21.15.22:0/533501018 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e40303a0 con 0x558f09ab1680 2024-09-19T08:21:40.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.957+0000 7f39f60c1740 1 -- 172.21.15.22:0/533501018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39e404e070 con 0x558f09ab1680 2024-09-19T08:21:40.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.957+0000 7f39d57fa640 1 -- 172.21.15.22:0/533501018 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f39e4004760 con 0x558f09ab1680 2024-09-19T08:21:40.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.958+0000 7f39d57fa640 1 -- 172.21.15.22:0/533501018 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f39e4031c50 con 0x558f09ab1680 2024-09-19T08:21:40.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.958+0000 7f39f60c1740 10 client.16602 fetch_fsmap learned FSMap version 31 2024-09-19T08:21:40.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.958+0000 7f39f60c1740 1 -- 172.21.15.22:0/533501018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558f09ac5260 con 0x558f09ab1680 2024-09-19T08:21:40.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.958+0000 7f39d57fa640 1 -- 172.21.15.22:0/533501018 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e4036040 con 0x558f09ab1680 2024-09-19T08:21:40.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.958+0000 7f39d57fa640 1 -- 172.21.15.22:0/533501018 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f39e403a420 con 0x558f09ab1680 2024-09-19T08:21:40.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.958+0000 7f39f60c1740 10 client.16602 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:21:40.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.958+0000 7f39f60c1740 10 client.16602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:21:40.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.959+0000 7f39f60c1740 20 client.16602 populate_metadata read hostname 'smithi022' 2024-09-19T08:21:40.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.959+0000 7f39f60c1740 1 --2- 172.21.15.22:0/533501018 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f09ad2080 0x558f09ad44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:40.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.959+0000 7f39f60c1740 4 client.16602 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:21:40.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.959+0000 7f39f60c1740 1 -- 172.21.15.22:0/533501018 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f39e404e070 con 0x558f09ad2080 2024-09-19T08:21:40.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.963+0000 7f39f1a84640 1 --2- 172.21.15.22:0/533501018 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f09ad2080 0x558f09ad44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:40.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.973+0000 7f39f1a84640 1 --2- 172.21.15.22:0/533501018 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f09ad2080 0x558f09ad44a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5433a7f2c609c6b server_cookie=3761e7bb9be2aace in_seq=0 out_seq=0 2024-09-19T08:21:40.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.973+0000 7f39d57fa640 10 client.16602 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:21:40.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.973+0000 7f39d57fa640 1 -- 172.21.15.22:0/533501018 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f39e404e070 con 0x558f09ad2080 2024-09-19T08:21:40.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.973+0000 7f39d57fa640 10 client.16602 handle_command_reply: tid=0 2024-09-19T08:21:40.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.973+0000 7f39f60c1740 1 client.16602 shutdown 2024-09-19T08:21:40.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.974+0000 7f39f60c1740 1 -- 172.21.15.22:0/533501018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09ab1680 msgr2=0x558f09aae9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:40.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.974+0000 7f39f60c1740 1 --2- 172.21.15.22:0/533501018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09ab1680 0x558f09aae9d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009950 tx=0x7f39e4030f90 comp rx=0 tx=0).stop 2024-09-19T08:21:40.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.974+0000 7f39f60c1740 1 -- 172.21.15.22:0/533501018 shutdown_connections 2024-09-19T08:21:40.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.974+0000 7f39f60c1740 1 --2- 172.21.15.22:0/533501018 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f09ad2080 0x558f09ad44a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.974+0000 7f39f60c1740 1 --2- 172.21.15.22:0/533501018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09ab1680 0x558f09aae9d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.974+0000 7f39f60c1740 1 --2- 172.21.15.22:0/533501018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09aa6dc0 0x558f09aae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.974+0000 7f39f60c1740 1 --2- 172.21.15.22:0/533501018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa64a0 0x558f09ab1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:40.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.974+0000 7f39d57fa640 0 client.16602 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:21:40.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.974+0000 7f39f60c1740 1 -- 172.21.15.22:0/533501018 >> 172.21.15.22:0/533501018 conn(0x558f09aa2a30 msgr2=0x558f09aa8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:40.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.974+0000 7f39f60c1740 1 -- 172.21.15.22:0/533501018 shutdown_connections 2024-09-19T08:21:40.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.974+0000 7f39f60c1740 1 -- 172.21.15.22:0/533501018 wait complete. 2024-09-19T08:21:40.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:40.975+0000 7f39f60c1740 20 client.16602 trim_cache size 0 max 16384 2024-09-19T08:21:41.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.018+0000 7f39f60c1740 1 Processor -- start 2024-09-19T08:21:41.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.018+0000 7f39f60c1740 1 -- start start 2024-09-19T08:21:41.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.019+0000 7f39f60c1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa64a0 0x558f09aa1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:41.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.019+0000 7f39f60c1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa6dc0 0x558f09ace4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:41.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.019+0000 7f39f60c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09ab1680 0x558f09ad9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:41.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.019+0000 7f39f1a84640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa64a0 0x558f09aa1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:41.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.019+0000 7f39f1a84640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa64a0 0x558f09aa1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:36002/0 (socket says 172.21.15.22:36002) 2024-09-19T08:21:41.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.019+0000 7f39f1a84640 1 -- 172.21.15.22:0/1663009270 learned_addr learned my addr 172.21.15.22:0/1663009270 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:41.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.019+0000 7f39f60c1740 1 -- 172.21.15.22:0/1663009270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec120c70 con 0x558f09aa6dc0 2024-09-19T08:21:41.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.019+0000 7f39f60c1740 1 -- 172.21.15.22:0/1663009270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec11e4e0 con 0x558f09aa64a0 2024-09-19T08:21:41.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.019+0000 7f39f60c1740 1 -- 172.21.15.22:0/1663009270 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec0721f0 con 0x558f09ab1680 2024-09-19T08:21:41.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.020+0000 7f39f1283640 1 --2- 172.21.15.22:0/1663009270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa6dc0 0x558f09ace4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:41.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.020+0000 7f39f1a84640 1 -- 172.21.15.22:0/1663009270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09ab1680 msgr2=0x558f09ad9d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:21:41.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.020+0000 7f39f1a84640 1 --2- 172.21.15.22:0/1663009270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09ab1680 0x558f09ad9d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.020+0000 7f39f1a84640 1 -- 172.21.15.22:0/1663009270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa6dc0 msgr2=0x558f09ace4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:41.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.020+0000 7f39f1a84640 1 --2- 172.21.15.22:0/1663009270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa6dc0 0x558f09ace4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.020+0000 7f39f1a84640 1 -- 172.21.15.22:0/1663009270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f09ad4b20 con 0x558f09aa64a0 2024-09-19T08:21:41.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.020+0000 7f39f1a84640 1 --2- 172.21.15.22:0/1663009270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa64a0 0x558f09aa1e20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f39e800c6a0 tx=0x7f39e8009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d86e97ad27b378f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:41.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.025+0000 7f39d77fe640 1 -- 172.21.15.22:0/1663009270 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e809a730 con 0x558f09aa64a0 2024-09-19T08:21:41.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.025+0000 7f39d77fe640 1 -- 172.21.15.22:0/1663009270 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e8098aa0 con 0x558f09aa64a0 2024-09-19T08:21:41.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.026+0000 7f39f60c1740 1 -- 172.21.15.22:0/1663009270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa64a0 msgr2=0x558f09aa1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:41.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.026+0000 7f39f60c1740 1 --2- 172.21.15.22:0/1663009270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa64a0 0x558f09aa1e20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f39e800c6a0 tx=0x7f39e8009760 comp rx=0 tx=0).stop 2024-09-19T08:21:41.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.027+0000 7f39f60c1740 1 -- 172.21.15.22:0/1663009270 shutdown_connections 2024-09-19T08:21:41.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.027+0000 7f39f60c1740 1 --2- 172.21.15.22:0/1663009270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09ab1680 0x558f09ad9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.027+0000 7f39f60c1740 1 --2- 172.21.15.22:0/1663009270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa6dc0 0x558f09ace4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.027+0000 7f39f60c1740 1 --2- 172.21.15.22:0/1663009270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa64a0 0x558f09aa1e20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.027+0000 7f39f60c1740 1 -- 172.21.15.22:0/1663009270 >> 172.21.15.22:0/1663009270 conn(0x558f09aa2a30 msgr2=0x558f09ab03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:41.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.029+0000 7f39f60c1740 1 -- 172.21.15.22:0/1663009270 shutdown_connections 2024-09-19T08:21:41.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.029+0000 7f39f60c1740 1 -- 172.21.15.22:0/1663009270 wait complete. 2024-09-19T08:21:41.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.030+0000 7f39f60c1740 1 -- start start 2024-09-19T08:21:41.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.030+0000 7f39f60c1740 1 Processor -- start 2024-09-19T08:21:41.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.031+0000 7f39f60c1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa64a0 0x558f09acab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.031+0000 7f39f60c1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa6dc0 0x558f09ac6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.031+0000 7f39f60c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09ab1680 0x558f09ac7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.031+0000 7f39f60c1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec072860 con 0x558f09aa64a0 2024-09-19T08:21:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.031+0000 7f39f60c1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec073cd0 con 0x558f09aa6dc0 2024-09-19T08:21:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.031+0000 7f39f60c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec072f30 con 0x558f09ab1680 2024-09-19T08:21:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.031+0000 7f39f60c1740 10 client.0 _ll_register_callbacks cb 0x7f39e4031440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:21:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.031+0000 7f39f2285640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09ab1680 0x558f09ac7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.031+0000 7f39f2285640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09ab1680 0x558f09ac7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:39676/0 (socket says 172.21.15.22:39676) 2024-09-19T08:21:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.031+0000 7f39f2285640 1 -- 172.21.15.22:0/3668521873 learned_addr learned my addr 172.21.15.22:0/3668521873 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:21:41.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.032+0000 7f39f1a84640 1 --2- 172.21.15.22:0/3668521873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa64a0 0x558f09acab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:41.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.033+0000 7f39f1a84640 1 -- 172.21.15.22:0/3668521873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09ab1680 msgr2=0x558f09ac7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:41.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.033+0000 7f39f1a84640 1 --2- 172.21.15.22:0/3668521873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09ab1680 0x558f09ac7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.033+0000 7f39f1a84640 1 -- 172.21.15.22:0/3668521873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa6dc0 msgr2=0x558f09ac6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:21:41.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.033+0000 7f39f1a84640 1 --2- 172.21.15.22:0/3668521873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa6dc0 0x558f09ac6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.033+0000 7f39f1a84640 1 -- 172.21.15.22:0/3668521873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f09ac79f0 con 0x558f09aa64a0 2024-09-19T08:21:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.033+0000 7f39f1a84640 1 --2- 172.21.15.22:0/3668521873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa64a0 0x558f09acab80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f39e809c130 tx=0x7f39e80a0960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258bd60f68f63725 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:21:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.034+0000 7f39d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:21:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.034+0000 7f39d57fa640 1 -- 172.21.15.22:0/3668521873 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e80cdb20 con 0x558f09aa64a0 2024-09-19T08:21:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.034+0000 7f39f60c1740 1 -- 172.21.15.22:0/3668521873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558f09a19850 con 0x558f09aa64a0 2024-09-19T08:21:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.034+0000 7f39d57fa640 1 -- 172.21.15.22:0/3668521873 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f39e80cdcc0 con 0x558f09aa64a0 2024-09-19T08:21:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.035+0000 7f39d57fa640 1 -- 172.21.15.22:0/3668521873 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f39e80b52b0 con 0x558f09aa64a0 2024-09-19T08:21:41.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.040+0000 7f39f60c1740 10 client.16608 fetch_fsmap learned FSMap version 31 2024-09-19T08:21:41.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.040+0000 7f39f60c1740 1 -- 172.21.15.22:0/3668521873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558f09b3dbf0 con 0x558f09aa64a0 2024-09-19T08:21:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.041+0000 7f39d57fa640 1 -- 172.21.15.22:0/3668521873 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558f09a19850 con 0x558f09aa64a0 2024-09-19T08:21:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.041+0000 7f39d57fa640 1 -- 172.21.15.22:0/3668521873 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f39e80b0070 con 0x558f09aa64a0 2024-09-19T08:21:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.041+0000 7f39f60c1740 10 client.16608 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:21:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.041+0000 7f39f60c1740 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:21:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.041+0000 7f39f60c1740 20 client.16608 populate_metadata read hostname 'smithi022' 2024-09-19T08:21:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.041+0000 7f39f60c1740 1 --2- 172.21.15.22:0/3668521873 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f09b3e210 0x558f09b40600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:21:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.041+0000 7f39f60c1740 4 client.16608 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:21:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.041+0000 7f39f60c1740 1 -- 172.21.15.22:0/3668521873 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558f09a13d60 con 0x558f09b3e210 2024-09-19T08:21:41.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.047+0000 7f39f1283640 1 --2- 172.21.15.22:0/3668521873 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f09b3e210 0x558f09b40600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:21:41.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.048+0000 7f39f1283640 1 --2- 172.21.15.22:0/3668521873 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f09b3e210 0x558f09b40600 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b91dd623c74dae9 server_cookie=7b487b553a122deb in_seq=0 out_seq=0 2024-09-19T08:21:41.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.048+0000 7f39d57fa640 10 client.16608 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:21:41.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.048+0000 7f39d57fa640 1 -- 172.21.15.22:0/3668521873 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558f09a13d60 con 0x558f09b3e210 2024-09-19T08:21:41.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.048+0000 7f39d57fa640 10 client.16608 handle_command_reply: tid=0 2024-09-19T08:21:41.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.048+0000 7f39f60c1740 1 client.16608 shutdown 2024-09-19T08:21:41.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.049+0000 7f39f60c1740 1 -- 172.21.15.22:0/3668521873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa64a0 msgr2=0x558f09acab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:41.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.049+0000 7f39f60c1740 1 --2- 172.21.15.22:0/3668521873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa64a0 0x558f09acab80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f39e809c130 tx=0x7f39e80a0960 comp rx=0 tx=0).stop 2024-09-19T08:21:41.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.051+0000 7f39f60c1740 1 -- 172.21.15.22:0/3668521873 shutdown_connections 2024-09-19T08:21:41.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.051+0000 7f39f60c1740 1 --2- 172.21.15.22:0/3668521873 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f09b3e210 0x558f09b40600 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.051+0000 7f39f60c1740 1 --2- 172.21.15.22:0/3668521873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f09ab1680 0x558f09ac7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.051+0000 7f39f60c1740 1 --2- 172.21.15.22:0/3668521873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f09aa6dc0 0x558f09ac6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.051+0000 7f39f60c1740 1 --2- 172.21.15.22:0/3668521873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f09aa64a0 0x558f09acab80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.051+0000 7f39f60c1740 1 -- 172.21.15.22:0/3668521873 >> 172.21.15.22:0/3668521873 conn(0x558f09aa2a30 msgr2=0x558f09aa9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:41.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.051+0000 7f39d57fa640 0 client.16608 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:21:41.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.058+0000 7f39f60c1740 1 -- 172.21.15.22:0/3668521873 shutdown_connections 2024-09-19T08:21:41.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.060+0000 7f39f60c1740 1 -- 172.21.15.22:0/3668521873 wait complete. 2024-09-19T08:21:41.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.060+0000 7f39f60c1740 20 client.16608 trim_cache size 0 max 16384 2024-09-19T08:21:41.060 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:21:41.060 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:21:41.060 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-19T08:21:41.060 INFO:teuthology.orchestra.run.smithi022.stdout: "a4ff4855-83f3-407a-9efb-8f696af28963": { 2024-09-19T08:21:41.061 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-19T08:21:41.061 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "a4ff4855-83f3-407a-9efb-8f696af28963", 2024-09-19T08:21:41.061 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-09-19T08:21:41.061 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:21:41.061 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:21:41.061 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:21:41.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.064+0000 7f39f3ce8640 1 -- 172.21.15.22:0/2230900447 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f39d80a35c0 msgr2=0x7f39d80a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:41.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.064+0000 7f39f3ce8640 1 --2- 172.21.15.22:0/2230900447 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f39d80a35c0 0x7f39d80a5a80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f39e4031020 tx=0x7f39e4048000 comp rx=0 tx=0).stop 2024-09-19T08:21:41.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.064+0000 7f39f3ce8640 1 -- 172.21.15.22:0/2230900447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec076720 msgr2=0x7f39ec111d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:21:41.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.064+0000 7f39f3ce8640 1 --2- 172.21.15.22:0/2230900447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec076720 0x7f39ec111d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f39e800e9f0 tx=0x7f39e800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.065+0000 7f39f3ce8640 1 -- 172.21.15.22:0/2230900447 shutdown_connections 2024-09-19T08:21:41.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.065+0000 7f39f3ce8640 1 --2- 172.21.15.22:0/2230900447 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f39d80a35c0 0x7f39d80a5a80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.065+0000 7f39f3ce8640 1 --2- 172.21.15.22:0/2230900447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39ec112a40 0x7f39ec10be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.065+0000 7f39f3ce8640 1 --2- 172.21.15.22:0/2230900447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39ec077040 0x7f39ec1122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.065+0000 7f39f3ce8640 1 --2- 172.21.15.22:0/2230900447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec076720 0x7f39ec111d80 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f39e800e9f0 tx=0x7f39e800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:21:41.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.065+0000 7f39f3ce8640 1 -- 172.21.15.22:0/2230900447 >> 172.21.15.22:0/2230900447 conn(0x7f39ec0ffa20 msgr2=0x7f39ec112fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:21:41.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.065+0000 7f39f3ce8640 1 -- 172.21.15.22:0/2230900447 shutdown_connections 2024-09-19T08:21:41.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:21:41.065+0000 7f39f3ce8640 1 -- 172.21.15.22:0/2230900447 wait complete. 2024-09-19T08:21:41.288 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:41 smithi022 ceph-mon[30963]: pgmap v202: 97 pgs: 97 active+clean; 839 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.7 MiB/s wr, 745 op/s 2024-09-19T08:21:41.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:41 smithi064 ceph-mon[34900]: pgmap v202: 97 pgs: 97 active+clean; 839 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.7 MiB/s wr, 745 op/s 2024-09-19T08:21:41.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:41 smithi157 ceph-mon[35143]: pgmap v202: 97 pgs: 97 active+clean; 839 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.7 MiB/s wr, 745 op/s 2024-09-19T08:21:42.565 INFO:tasks.cephfs.filesystem:scrub status for tag:a4ff4855-83f3-407a-9efb-8f696af28963 - {'path': '/', 'tag': 'a4ff4855-83f3-407a-9efb-8f696af28963', 'options': 'recursive,force'} 2024-09-19T08:21:43.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:43 smithi157 ceph-mon[35143]: pgmap v203: 97 pgs: 97 active+clean; 839 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.7 MiB/s wr, 653 op/s 2024-09-19T08:21:43.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:43 smithi022 ceph-mon[30963]: pgmap v203: 97 pgs: 97 active+clean; 839 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.7 MiB/s wr, 653 op/s 2024-09-19T08:21:43.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:43 smithi064 ceph-mon[34900]: pgmap v203: 97 pgs: 97 active+clean; 839 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.7 MiB/s wr, 653 op/s 2024-09-19T08:21:44.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:44 smithi157 ceph-mon[35143]: pgmap v204: 97 pgs: 97 active+clean; 824 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.0 MiB/s wr, 844 op/s 2024-09-19T08:21:44.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:44 smithi022 ceph-mon[30963]: pgmap v204: 97 pgs: 97 active+clean; 824 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.0 MiB/s wr, 844 op/s 2024-09-19T08:21:44.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:44 smithi064 ceph-mon[34900]: pgmap v204: 97 pgs: 97 active+clean; 824 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.0 MiB/s wr, 844 op/s 2024-09-19T08:21:46.157 INFO:tasks.workunit.client.0.smithi022.stdout: 153 2069183 633 1360447 484 1149958 1661 2024-09-19T08:21:46.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:46 smithi157 ceph-mon[35143]: pgmap v205: 97 pgs: 97 active+clean; 824 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 423 op/s 2024-09-19T08:21:46.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:46 smithi022 ceph-mon[30963]: pgmap v205: 97 pgs: 97 active+clean; 824 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 423 op/s 2024-09-19T08:21:46.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:46 smithi064 ceph-mon[34900]: pgmap v205: 97 pgs: 97 active+clean; 824 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 423 op/s 2024-09-19T08:21:48.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:48 smithi157 ceph-mon[35143]: pgmap v206: 97 pgs: 97 active+clean; 835 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 489 op/s 2024-09-19T08:21:48.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:48 smithi022 ceph-mon[30963]: pgmap v206: 97 pgs: 97 active+clean; 835 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 489 op/s 2024-09-19T08:21:48.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:48 smithi064 ceph-mon[34900]: pgmap v206: 97 pgs: 97 active+clean; 835 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 489 op/s 2024-09-19T08:21:50.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:50 smithi157 ceph-mon[35143]: pgmap v207: 97 pgs: 97 active+clean; 839 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 512 op/s 2024-09-19T08:21:50.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:50 smithi022 ceph-mon[30963]: pgmap v207: 97 pgs: 97 active+clean; 839 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 512 op/s 2024-09-19T08:21:50.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:50 smithi064 ceph-mon[34900]: pgmap v207: 97 pgs: 97 active+clean; 839 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 512 op/s 2024-09-19T08:21:52.511 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:52 smithi022 ceph-mon[30963]: pgmap v208: 97 pgs: 97 active+clean; 839 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 363 op/s 2024-09-19T08:21:52.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:52 smithi157 ceph-mon[35143]: pgmap v208: 97 pgs: 97 active+clean; 839 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 363 op/s 2024-09-19T08:21:52.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:52 smithi064 ceph-mon[34900]: pgmap v208: 97 pgs: 97 active+clean; 839 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 363 op/s 2024-09-19T08:21:54.640 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:54 smithi064 ceph-mon[34900]: pgmap v209: 97 pgs: 97 active+clean; 902 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.3 MiB/s wr, 890 op/s 2024-09-19T08:21:54.752 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:54 smithi157 ceph-mon[35143]: pgmap v209: 97 pgs: 97 active+clean; 902 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.3 MiB/s wr, 890 op/s 2024-09-19T08:21:54.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:54 smithi022 ceph-mon[30963]: pgmap v209: 97 pgs: 97 active+clean; 902 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.3 MiB/s wr, 890 op/s 2024-09-19T08:21:56.158 INFO:tasks.workunit.client.0.smithi022.stdout: 158 1479212 294 979291 276 812785 899 2024-09-19T08:21:56.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:56 smithi157 ceph-mon[35143]: pgmap v210: 97 pgs: 97 active+clean; 902 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 699 op/s 2024-09-19T08:21:56.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:56 smithi022 ceph-mon[30963]: pgmap v210: 97 pgs: 97 active+clean; 902 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 699 op/s 2024-09-19T08:21:56.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:56 smithi064 ceph-mon[34900]: pgmap v210: 97 pgs: 97 active+clean; 902 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 699 op/s 2024-09-19T08:21:58.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:21:58 smithi157 ceph-mon[35143]: pgmap v211: 97 pgs: 97 active+clean; 910 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.8 MiB/s wr, 823 op/s 2024-09-19T08:21:58.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:21:58 smithi022 ceph-mon[30963]: pgmap v211: 97 pgs: 97 active+clean; 910 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.8 MiB/s wr, 823 op/s 2024-09-19T08:21:58.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:21:58 smithi064 ceph-mon[34900]: pgmap v211: 97 pgs: 97 active+clean; 910 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.8 MiB/s wr, 823 op/s 2024-09-19T08:22:00.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:00 smithi157 ceph-mon[35143]: pgmap v212: 97 pgs: 97 active+clean; 928 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.5 MiB/s wr, 1.05k op/s 2024-09-19T08:22:00.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:00 smithi022 ceph-mon[30963]: pgmap v212: 97 pgs: 97 active+clean; 928 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.5 MiB/s wr, 1.05k op/s 2024-09-19T08:22:00.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:00 smithi064 ceph-mon[34900]: pgmap v212: 97 pgs: 97 active+clean; 928 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.5 MiB/s wr, 1.05k op/s 2024-09-19T08:22:02.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:02 smithi157 ceph-mon[35143]: pgmap v213: 97 pgs: 97 active+clean; 928 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.1 MiB/s wr, 944 op/s 2024-09-19T08:22:02.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:02 smithi022 ceph-mon[30963]: pgmap v213: 97 pgs: 97 active+clean; 928 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.1 MiB/s wr, 944 op/s 2024-09-19T08:22:02.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:02 smithi064 ceph-mon[34900]: pgmap v213: 97 pgs: 97 active+clean; 928 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.1 MiB/s wr, 944 op/s 2024-09-19T08:22:03.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:03 smithi157 ceph-mon[35143]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T08:22:03.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:03 smithi022 ceph-mon[30963]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T08:22:03.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:03 smithi064 ceph-mon[34900]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T08:22:04.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:04 smithi157 ceph-mon[35143]: pgmap v215: 97 pgs: 97 active+clean; 905 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 853 op/s 2024-09-19T08:22:04.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:04 smithi022 ceph-mon[30963]: pgmap v215: 97 pgs: 97 active+clean; 905 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 853 op/s 2024-09-19T08:22:04.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:04 smithi064 ceph-mon[34900]: pgmap v215: 97 pgs: 97 active+clean; 905 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 853 op/s 2024-09-19T08:22:06.158 INFO:tasks.workunit.client.0.smithi022.stdout: 165 2131685 482 1401022 542 1197858 1682 2024-09-19T08:22:06.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:06 smithi157 ceph-mon[35143]: pgmap v216: 97 pgs: 97 active+clean; 905 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 853 op/s 2024-09-19T08:22:06.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:06 smithi022 ceph-mon[30963]: pgmap v216: 97 pgs: 97 active+clean; 905 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 853 op/s 2024-09-19T08:22:06.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:06 smithi064 ceph-mon[34900]: pgmap v216: 97 pgs: 97 active+clean; 905 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 853 op/s 2024-09-19T08:22:08.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:08 smithi157 ceph-mon[35143]: pgmap v217: 97 pgs: 97 active+clean; 909 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 752 op/s 2024-09-19T08:22:08.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:08 smithi022 ceph-mon[30963]: pgmap v217: 97 pgs: 97 active+clean; 909 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 752 op/s 2024-09-19T08:22:08.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:08 smithi064 ceph-mon[34900]: pgmap v217: 97 pgs: 97 active+clean; 909 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 752 op/s 2024-09-19T08:22:10.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:10 smithi157 ceph-mon[35143]: pgmap v218: 97 pgs: 97 active+clean; 921 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 537 op/s 2024-09-19T08:22:10.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:10 smithi022 ceph-mon[30963]: pgmap v218: 97 pgs: 97 active+clean; 921 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 537 op/s 2024-09-19T08:22:10.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:10 smithi064 ceph-mon[34900]: pgmap v218: 97 pgs: 97 active+clean; 921 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 537 op/s 2024-09-19T08:22:11.755 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:11 smithi157 ceph-mon[35143]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T08:22:11.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:11 smithi022 ceph-mon[30963]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T08:22:11.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:11 smithi064 ceph-mon[34900]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T08:22:12.567 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:22:12.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:12 smithi157 ceph-mon[35143]: pgmap v220: 97 pgs: 97 active+clean; 921 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 404 op/s 2024-09-19T08:22:12.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:12 smithi022 ceph-mon[30963]: pgmap v220: 97 pgs: 97 active+clean; 921 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 404 op/s 2024-09-19T08:22:12.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:12 smithi064 ceph-mon[34900]: pgmap v220: 97 pgs: 97 active+clean; 921 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 404 op/s 2024-09-19T08:22:13.039 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:22:13.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.809+0000 7f0a3f8a8640 1 -- 172.21.15.22:0/4084433322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a38102ef0 msgr2=0x7f0a3810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:13.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.809+0000 7f0a3f8a8640 1 --2- 172.21.15.22:0/4084433322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a38102ef0 0x7f0a3810da40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009900 tx=0x7f0a3002a480 comp rx=0 tx=0).stop 2024-09-19T08:22:13.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.809+0000 7f0a3f8a8640 1 -- 172.21.15.22:0/4084433322 shutdown_connections 2024-09-19T08:22:13.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.809+0000 7f0a3f8a8640 1 --2- 172.21.15.22:0/4084433322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a38102ef0 0x7f0a3810da40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.809+0000 7f0a3f8a8640 1 --2- 172.21.15.22:0/4084433322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a38102570 0x7f0a381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.809+0000 7f0a3f8a8640 1 --2- 172.21.15.22:0/4084433322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a38100e20 0x7f0a38101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.809+0000 7f0a3f8a8640 1 -- 172.21.15.22:0/4084433322 >> 172.21.15.22:0/4084433322 conn(0x7f0a380fc7d0 msgr2=0x7f0a380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:13.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.809+0000 7f0a3f8a8640 1 -- 172.21.15.22:0/4084433322 shutdown_connections 2024-09-19T08:22:13.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.809+0000 7f0a3f8a8640 1 -- 172.21.15.22:0/4084433322 wait complete. 2024-09-19T08:22:13.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.810+0000 7f0a3f8a8640 1 Processor -- start 2024-09-19T08:22:13.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.810+0000 7f0a3f8a8640 1 -- start start 2024-09-19T08:22:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.810+0000 7f0a3f8a8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a38100e20 0x7f0a380fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.810+0000 7f0a3f8a8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a38102570 0x7f0a380fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.810+0000 7f0a3f8a8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a38102ef0 0x7f0a380fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.810+0000 7f0a3f8a8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0a38110f80 con 0x7f0a38102570 2024-09-19T08:22:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.810+0000 7f0a3f8a8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a38110e00 con 0x7f0a38100e20 2024-09-19T08:22:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.810+0000 7f0a3f8a8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a38111100 con 0x7f0a38102ef0 2024-09-19T08:22:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.810+0000 7f0a3ce43640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a38102570 0x7f0a380fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.810+0000 7f0a3d644640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a38100e20 0x7f0a380fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.810+0000 7f0a3d644640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a38100e20 0x7f0a380fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:60672/0 (socket says 172.21.15.22:60672) 2024-09-19T08:22:13.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.810+0000 7f0a3ce43640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a38102570 0x7f0a380fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50812/0 (socket says 172.21.15.22:50812) 2024-09-19T08:22:13.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.810+0000 7f0a3d644640 1 -- 172.21.15.22:0/483355764 learned_addr learned my addr 172.21.15.22:0/483355764 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:13.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.810+0000 7f0a3de45640 1 --2- 172.21.15.22:0/483355764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a38102ef0 0x7f0a380fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.811+0000 7f0a3d644640 1 -- 172.21.15.22:0/483355764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a38102ef0 msgr2=0x7f0a380fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:13.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.811+0000 7f0a3d644640 1 --2- 172.21.15.22:0/483355764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a38102ef0 0x7f0a380fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.811+0000 7f0a3d644640 1 -- 172.21.15.22:0/483355764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a38102570 msgr2=0x7f0a380fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:13.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.811+0000 7f0a3d644640 1 --2- 172.21.15.22:0/483355764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a38102570 0x7f0a380fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.811+0000 7f0a3d644640 1 -- 172.21.15.22:0/483355764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a381a6910 con 0x7f0a38100e20 2024-09-19T08:22:13.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.811+0000 7f0a3ce43640 1 --2- 172.21.15.22:0/483355764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a38102570 0x7f0a380fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:22:13.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.811+0000 7f0a3d644640 1 --2- 172.21.15.22:0/483355764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a38100e20 0x7f0a380fcc80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400cce0 tx=0x7f0a3400a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:13.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.811+0000 7f0a2e7fc640 1 -- 172.21.15.22:0/483355764 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34004b20 con 0x7f0a38100e20 2024-09-19T08:22:13.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.811+0000 7f0a3f8a8640 1 -- 172.21.15.22:0/483355764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a381a6c00 con 0x7f0a38100e20 2024-09-19T08:22:13.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.811+0000 7f0a2e7fc640 1 -- 172.21.15.22:0/483355764 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a34004cc0 con 0x7f0a38100e20 2024-09-19T08:22:13.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.811+0000 7f0a2e7fc640 1 -- 172.21.15.22:0/483355764 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3400f7d0 con 0x7f0a38100e20 2024-09-19T08:22:13.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.811+0000 7f0a3f8a8640 1 -- 172.21.15.22:0/483355764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a381a7190 con 0x7f0a38100e20 2024-09-19T08:22:13.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.814+0000 7f0a2e7fc640 1 -- 172.21.15.22:0/483355764 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0a3400f970 con 0x7f0a38100e20 2024-09-19T08:22:13.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.814+0000 7f0a41c81740 1 Processor -- start 2024-09-19T08:22:13.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.814+0000 7f0a41c81740 1 -- start start 2024-09-19T08:22:13.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a41c81740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c83d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a41c81740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c83ddc0 0x55fe0c848140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a41c81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c848680 0x55fe0c84aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a41c81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fe0c7a32e0 con 0x55fe0c848680 2024-09-19T08:22:13.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a41c81740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fe0c74ace0 con 0x55fe0c83d4a0 2024-09-19T08:22:13.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a41c81740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fe0c79f2a0 con 0x55fe0c83ddc0 2024-09-19T08:22:13.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a3de45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c83d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a3d644640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c83ddc0 0x55fe0c848140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a3de45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c83d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:60676/0 (socket says 172.21.15.22:60676) 2024-09-19T08:22:13.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a3de45640 1 -- 172.21.15.22:0/3910022109 learned_addr learned my addr 172.21.15.22:0/3910022109 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:13.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a3ce43640 1 --2- 172.21.15.22:0/3910022109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c848680 0x55fe0c84aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a2e7fc640 1 --2- 172.21.15.22:0/483355764 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0a240a33b0 0x7f0a240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a3de45640 1 --2- 172.21.15.22:0/483355764 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0a240a33b0 0x7f0a240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a2e7fc640 1 -- 172.21.15.22:0/483355764 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f0a340cfd30 con 0x7f0a38100e20 2024-09-19T08:22:13.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a3ce43640 1 -- 172.21.15.22:0/3910022109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c83ddc0 msgr2=0x55fe0c848140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:13.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a3ce43640 1 --2- 172.21.15.22:0/3910022109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c83ddc0 0x55fe0c848140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a3ce43640 1 -- 172.21.15.22:0/3910022109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 msgr2=0x55fe0c83d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:13.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a3ce43640 1 --2- 172.21.15.22:0/3910022109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c83d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.815+0000 7f0a3ce43640 1 -- 172.21.15.22:0/3910022109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe0c72f440 con 0x55fe0c848680 2024-09-19T08:22:13.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.816+0000 7f0a3ce43640 1 --2- 172.21.15.22:0/3910022109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c848680 0x55fe0c84aa70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800bf30 tx=0x7f0a28010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1504888a652eaebf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:13.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.816+0000 7f0a237fe640 1 -- 172.21.15.22:0/3910022109 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28004220 con 0x55fe0c848680 2024-09-19T08:22:13.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.816+0000 7f0a237fe640 1 -- 172.21.15.22:0/3910022109 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a280043c0 con 0x55fe0c848680 2024-09-19T08:22:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.816+0000 7f0a3de45640 1 --2- 172.21.15.22:0/483355764 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0a240a33b0 0x7f0a240a5870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009900 tx=0x7f0a3002a480 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.816+0000 7f0a237fe640 1 -- 172.21.15.22:0/3910022109 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a280119d0 con 0x55fe0c848680 2024-09-19T08:22:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.817+0000 7f0a41c81740 1 -- 172.21.15.22:0/3910022109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c848680 msgr2=0x55fe0c84aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.817+0000 7f0a41c81740 1 --2- 172.21.15.22:0/3910022109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c848680 0x55fe0c84aa70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800bf30 tx=0x7f0a28010860 comp rx=0 tx=0).stop 2024-09-19T08:22:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.817+0000 7f0a41c81740 1 -- 172.21.15.22:0/3910022109 shutdown_connections 2024-09-19T08:22:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.817+0000 7f0a41c81740 1 --2- 172.21.15.22:0/3910022109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c848680 0x55fe0c84aa70 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.817+0000 7f0a41c81740 1 --2- 172.21.15.22:0/3910022109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c83ddc0 0x55fe0c848140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.817+0000 7f0a41c81740 1 --2- 172.21.15.22:0/3910022109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c83d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.817+0000 7f0a41c81740 1 -- 172.21.15.22:0/3910022109 >> 172.21.15.22:0/3910022109 conn(0x55fe0c839a30 msgr2=0x55fe0c839e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.817+0000 7f0a41c81740 1 -- 172.21.15.22:0/3910022109 shutdown_connections 2024-09-19T08:22:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.817+0000 7f0a41c81740 1 -- 172.21.15.22:0/3910022109 wait complete. 2024-09-19T08:22:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.818+0000 7f0a41c81740 1 -- start start 2024-09-19T08:22:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.818+0000 7f0a41c81740 1 Processor -- start 2024-09-19T08:22:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.818+0000 7f0a41c81740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c847f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.818+0000 7f0a41c81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c83ddc0 0x55fe0c845210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.818+0000 7f0a41c81740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c848680 0x55fe0c845750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.818+0000 7f0a41c81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fe0c84b250 con 0x55fe0c83ddc0 2024-09-19T08:22:13.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.818+0000 7f0a41c81740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fe0c75e050 con 0x55fe0c83d4a0 2024-09-19T08:22:13.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.818+0000 7f0a41c81740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fe0c84b3d0 con 0x55fe0c848680 2024-09-19T08:22:13.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.818+0000 7f0a41c81740 10 client.0 _ll_register_callbacks cb 0x55fe0c79e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:22:13.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.818+0000 7f0a3ce43640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c83ddc0 0x55fe0c845210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.818+0000 7f0a3ce43640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c83ddc0 0x55fe0c845210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50828/0 (socket says 172.21.15.22:50828) 2024-09-19T08:22:13.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.818+0000 7f0a3d644640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c847f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.818+0000 7f0a3ce43640 1 -- 172.21.15.22:0/319953127 learned_addr learned my addr 172.21.15.22:0/319953127 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:13.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.818+0000 7f0a3de45640 1 --2- 172.21.15.22:0/319953127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c848680 0x55fe0c845750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.819+0000 7f0a3d644640 1 -- 172.21.15.22:0/319953127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c848680 msgr2=0x55fe0c845750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.819+0000 7f0a3d644640 1 --2- 172.21.15.22:0/319953127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c848680 0x55fe0c845750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.819+0000 7f0a3d644640 1 -- 172.21.15.22:0/319953127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c83ddc0 msgr2=0x55fe0c845210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.819+0000 7f0a3d644640 1 --2- 172.21.15.22:0/319953127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c83ddc0 0x55fe0c845210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.819+0000 7f0a3d644640 1 -- 172.21.15.22:0/319953127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe0c5143c0 con 0x55fe0c83d4a0 2024-09-19T08:22:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.819+0000 7f0a3ce43640 1 --2- 172.21.15.22:0/319953127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c83ddc0 0x55fe0c845210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:22:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.819+0000 7f0a3d644640 1 --2- 172.21.15.22:0/319953127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c847f60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0a340cd470 tx=0x7f0a340d3390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f518639a5408a7b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.819+0000 7f0a20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:22:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.819+0000 7f0a20ff9640 1 -- 172.21.15.22:0/319953127 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3409ba10 con 0x55fe0c83d4a0 2024-09-19T08:22:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.819+0000 7f0a20ff9640 1 -- 172.21.15.22:0/319953127 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f0a34098720 con 0x55fe0c83d4a0 2024-09-19T08:22:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.819+0000 7f0a41c81740 1 -- 172.21.15.22:0/319953127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a280043c0 con 0x55fe0c83d4a0 2024-09-19T08:22:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.819+0000 7f0a20ff9640 1 -- 172.21.15.22:0/319953127 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0a3409c0b0 con 0x55fe0c83d4a0 2024-09-19T08:22:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.820+0000 7f0a41c81740 10 client.26138 fetch_fsmap learned FSMap version 31 2024-09-19T08:22:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.820+0000 7f0a41c81740 1 -- 172.21.15.22:0/319953127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fe0c856c50 con 0x55fe0c83d4a0 2024-09-19T08:22:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.820+0000 7f0a20ff9640 1 -- 172.21.15.22:0/319953127 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a280043c0 con 0x55fe0c83d4a0 2024-09-19T08:22:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.820+0000 7f0a20ff9640 1 -- 172.21.15.22:0/319953127 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f0a340b8050 con 0x55fe0c83d4a0 2024-09-19T08:22:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.820+0000 7f0a41c81740 10 client.26138 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:22:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.820+0000 7f0a41c81740 10 client.26138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:22:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.820+0000 7f0a41c81740 20 client.26138 populate_metadata read hostname 'smithi022' 2024-09-19T08:22:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.820+0000 7f0a41c81740 1 --2- 172.21.15.22:0/319953127 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fe0c858110 0x55fe0c85a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.820+0000 7f0a41c81740 4 client.26138 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:22:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.820+0000 7f0a41c81740 1 -- 172.21.15.22:0/319953127 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe0c7b0850 con 0x55fe0c858110 2024-09-19T08:22:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.821+0000 7f0a3ce43640 1 --2- 172.21.15.22:0/319953127 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fe0c858110 0x55fe0c85a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.821+0000 7f0a20ff9640 10 client.26138 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:13.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.821+0000 7f0a3ce43640 1 --2- 172.21.15.22:0/319953127 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fe0c858110 0x55fe0c85a560 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ad28980bd43c3e server_cookie=fc5ef22e0963604d in_seq=0 out_seq=0 2024-09-19T08:22:13.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.822+0000 7f0a20ff9640 1 -- 172.21.15.22:0/319953127 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe0c7b0850 con 0x55fe0c858110 2024-09-19T08:22:13.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.822+0000 7f0a20ff9640 10 client.26138 handle_command_reply: tid=0 2024-09-19T08:22:13.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.822+0000 7f0a41c81740 1 client.26138 shutdown 2024-09-19T08:22:13.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.822+0000 7f0a41c81740 1 -- 172.21.15.22:0/319953127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 msgr2=0x55fe0c847f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:13.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.822+0000 7f0a41c81740 1 --2- 172.21.15.22:0/319953127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c847f60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0a340cd470 tx=0x7f0a340d3390 comp rx=0 tx=0).stop 2024-09-19T08:22:13.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.822+0000 7f0a41c81740 1 -- 172.21.15.22:0/319953127 shutdown_connections 2024-09-19T08:22:13.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.822+0000 7f0a41c81740 1 --2- 172.21.15.22:0/319953127 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fe0c858110 0x55fe0c85a560 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.822+0000 7f0a41c81740 1 --2- 172.21.15.22:0/319953127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c848680 0x55fe0c845750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.822+0000 7f0a41c81740 1 --2- 172.21.15.22:0/319953127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c83ddc0 0x55fe0c845210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.822+0000 7f0a20ff9640 0 client.26138 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:13.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.822+0000 7f0a41c81740 1 --2- 172.21.15.22:0/319953127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c847f60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.822+0000 7f0a41c81740 1 -- 172.21.15.22:0/319953127 >> 172.21.15.22:0/319953127 conn(0x55fe0c839a30 msgr2=0x55fe0c83f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:13.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.822+0000 7f0a41c81740 1 -- 172.21.15.22:0/319953127 shutdown_connections 2024-09-19T08:22:13.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.822+0000 7f0a41c81740 1 -- 172.21.15.22:0/319953127 wait complete. 2024-09-19T08:22:13.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.823+0000 7f0a41c81740 20 client.26138 trim_cache size 0 max 16384 2024-09-19T08:22:13.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.844+0000 7f0a41c81740 1 Processor -- start 2024-09-19T08:22:13.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.844+0000 7f0a41c81740 1 -- start start 2024-09-19T08:22:13.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.844+0000 7f0a41c81740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c83d4a0 0x55fe0c8544e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.844+0000 7f0a41c81740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83ddc0 0x55fe0c85fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.844+0000 7f0a41c81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c848680 0x55fe0c860370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.844+0000 7f0a41c81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381b2370 con 0x55fe0c848680 2024-09-19T08:22:13.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.844+0000 7f0a41c81740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381afbe0 con 0x55fe0c83ddc0 2024-09-19T08:22:13.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.844+0000 7f0a41c81740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381ada50 con 0x55fe0c83d4a0 2024-09-19T08:22:13.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.845+0000 7f0a3d644640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c83d4a0 0x55fe0c8544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.845+0000 7f0a3de45640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c848680 0x55fe0c860370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.845+0000 7f0a3d644640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c83d4a0 0x55fe0c8544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:46338/0 (socket says 172.21.15.22:46338) 2024-09-19T08:22:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.845+0000 7f0a3d644640 1 -- 172.21.15.22:0/2368530027 learned_addr learned my addr 172.21.15.22:0/2368530027 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.845+0000 7f0a3ce43640 1 --2- 172.21.15.22:0/2368530027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83ddc0 0x55fe0c85fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.845+0000 7f0a3d644640 1 -- 172.21.15.22:0/2368530027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83ddc0 msgr2=0x55fe0c85fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.845+0000 7f0a3d644640 1 --2- 172.21.15.22:0/2368530027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83ddc0 0x55fe0c85fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.845+0000 7f0a3d644640 1 -- 172.21.15.22:0/2368530027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c848680 msgr2=0x55fe0c860370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.845+0000 7f0a3d644640 1 --2- 172.21.15.22:0/2368530027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c848680 0x55fe0c860370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.845+0000 7f0a3d644640 1 -- 172.21.15.22:0/2368530027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe0c839860 con 0x55fe0c83d4a0 2024-09-19T08:22:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.845+0000 7f0a3d644640 1 --2- 172.21.15.22:0/2368530027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c83d4a0 0x55fe0c8544e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a340cd520 tx=0x7f0a34098230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e366c72753b476c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.845+0000 7f0a22ffd640 1 -- 172.21.15.22:0/2368530027 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3409ba10 con 0x55fe0c83d4a0 2024-09-19T08:22:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.846+0000 7f0a22ffd640 1 -- 172.21.15.22:0/2368530027 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a340bc0c0 con 0x55fe0c83d4a0 2024-09-19T08:22:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.846+0000 7f0a22ffd640 1 -- 172.21.15.22:0/2368530027 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3409be80 con 0x55fe0c83d4a0 2024-09-19T08:22:13.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.846+0000 7f0a41c81740 1 -- 172.21.15.22:0/2368530027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c83d4a0 msgr2=0x55fe0c8544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:13.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.846+0000 7f0a41c81740 1 --2- 172.21.15.22:0/2368530027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c83d4a0 0x55fe0c8544e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a340cd520 tx=0x7f0a34098230 comp rx=0 tx=0).stop 2024-09-19T08:22:13.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.846+0000 7f0a41c81740 1 -- 172.21.15.22:0/2368530027 shutdown_connections 2024-09-19T08:22:13.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.846+0000 7f0a41c81740 1 --2- 172.21.15.22:0/2368530027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c848680 0x55fe0c860370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.846+0000 7f0a41c81740 1 --2- 172.21.15.22:0/2368530027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83ddc0 0x55fe0c85fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.846+0000 7f0a41c81740 1 --2- 172.21.15.22:0/2368530027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c83d4a0 0x55fe0c8544e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.846+0000 7f0a41c81740 1 -- 172.21.15.22:0/2368530027 >> 172.21.15.22:0/2368530027 conn(0x55fe0c839a30 msgr2=0x55fe0c843ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:13.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.847+0000 7f0a41c81740 1 -- 172.21.15.22:0/2368530027 shutdown_connections 2024-09-19T08:22:13.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.847+0000 7f0a41c81740 1 -- 172.21.15.22:0/2368530027 wait complete. 2024-09-19T08:22:13.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.847+0000 7f0a41c81740 1 -- start start 2024-09-19T08:22:13.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.847+0000 7f0a41c81740 1 Processor -- start 2024-09-19T08:22:13.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.847+0000 7f0a41c81740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c84c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.847+0000 7f0a41c81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c83ddc0 0x55fe0c84c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.847+0000 7f0a41c81740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c848680 0x55fe0c84ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.847+0000 7f0a41c81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381ae0a0 con 0x55fe0c83ddc0 2024-09-19T08:22:13.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.847+0000 7f0a41c81740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381af510 con 0x55fe0c83d4a0 2024-09-19T08:22:13.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.847+0000 7f0a41c81740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381ae770 con 0x55fe0c848680 2024-09-19T08:22:13.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.848+0000 7f0a41c81740 10 client.0 _ll_register_callbacks cb 0x7f0a34007850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:22:13.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.848+0000 7f0a3ce43640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c83ddc0 0x55fe0c84c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.848+0000 7f0a3ce43640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c83ddc0 0x55fe0c84c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50838/0 (socket says 172.21.15.22:50838) 2024-09-19T08:22:13.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.848+0000 7f0a3d644640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c84c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.848+0000 7f0a3ce43640 1 -- 172.21.15.22:0/4108414933 learned_addr learned my addr 172.21.15.22:0/4108414933 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:13.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.848+0000 7f0a3de45640 1 --2- 172.21.15.22:0/4108414933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c848680 0x55fe0c84ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.848+0000 7f0a3d644640 1 -- 172.21.15.22:0/4108414933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c848680 msgr2=0x55fe0c84ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.848+0000 7f0a3d644640 1 --2- 172.21.15.22:0/4108414933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c848680 0x55fe0c84ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.848+0000 7f0a3d644640 1 -- 172.21.15.22:0/4108414933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c83ddc0 msgr2=0x55fe0c84c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.848+0000 7f0a3d644640 1 --2- 172.21.15.22:0/4108414933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c83ddc0 0x55fe0c84c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.848+0000 7f0a3d644640 1 -- 172.21.15.22:0/4108414933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe0c83d2d0 con 0x55fe0c83d4a0 2024-09-19T08:22:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.848+0000 7f0a3d644640 1 --2- 172.21.15.22:0/4108414933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c84c410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a3409a360 tx=0x7f0a340d3370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=201eb86d8141fc66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.848+0000 7f0a20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:22:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.849+0000 7f0a20ff9640 1 -- 172.21.15.22:0/4108414933 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3409be80 con 0x55fe0c83d4a0 2024-09-19T08:22:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.849+0000 7f0a41c81740 1 -- 172.21.15.22:0/4108414933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a340bc0c0 con 0x55fe0c83d4a0 2024-09-19T08:22:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.849+0000 7f0a20ff9640 1 -- 172.21.15.22:0/4108414933 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f0a3409af70 con 0x55fe0c83d4a0 2024-09-19T08:22:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.849+0000 7f0a20ff9640 1 -- 172.21.15.22:0/4108414933 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0a340a9050 con 0x55fe0c83d4a0 2024-09-19T08:22:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.849+0000 7f0a41c81740 10 client.26144 fetch_fsmap learned FSMap version 31 2024-09-19T08:22:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.849+0000 7f0a41c81740 1 -- 172.21.15.22:0/4108414933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fe0c8c2f80 con 0x55fe0c83d4a0 2024-09-19T08:22:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.849+0000 7f0a20ff9640 1 -- 172.21.15.22:0/4108414933 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a340bc0c0 con 0x55fe0c83d4a0 2024-09-19T08:22:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.849+0000 7f0a20ff9640 1 -- 172.21.15.22:0/4108414933 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f0a340b8070 con 0x55fe0c83d4a0 2024-09-19T08:22:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.849+0000 7f0a41c81740 10 client.26144 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:22:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.849+0000 7f0a41c81740 10 client.26144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:22:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.849+0000 7f0a41c81740 20 client.26144 populate_metadata read hostname 'smithi022' 2024-09-19T08:22:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.849+0000 7f0a41c81740 1 --2- 172.21.15.22:0/4108414933 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fe0c8c33b0 0x55fe0c8c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.849+0000 7f0a41c81740 4 client.26144 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:22:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.849+0000 7f0a41c81740 1 -- 172.21.15.22:0/4108414933 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fe0c7aad60 con 0x55fe0c8c33b0 2024-09-19T08:22:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a3ce43640 1 --2- 172.21.15.22:0/4108414933 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fe0c8c33b0 0x55fe0c8c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a3ce43640 1 --2- 172.21.15.22:0/4108414933 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fe0c8c33b0 0x55fe0c8c57a0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e25bbe066220b1 server_cookie=e8b7c1d24d503d8c in_seq=0 out_seq=0 2024-09-19T08:22:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a20ff9640 10 client.26144 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a20ff9640 1 -- 172.21.15.22:0/4108414933 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fe0c7aad60 con 0x55fe0c8c33b0 2024-09-19T08:22:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a20ff9640 10 client.26144 handle_command_reply: tid=0 2024-09-19T08:22:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a41c81740 1 client.26144 shutdown 2024-09-19T08:22:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a41c81740 1 -- 172.21.15.22:0/4108414933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 msgr2=0x55fe0c84c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a41c81740 1 --2- 172.21.15.22:0/4108414933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c84c410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a3409a360 tx=0x7f0a340d3370 comp rx=0 tx=0).stop 2024-09-19T08:22:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a41c81740 1 -- 172.21.15.22:0/4108414933 shutdown_connections 2024-09-19T08:22:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a41c81740 1 --2- 172.21.15.22:0/4108414933 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fe0c8c33b0 0x55fe0c8c57a0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a41c81740 1 --2- 172.21.15.22:0/4108414933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe0c848680 0x55fe0c84ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a41c81740 1 --2- 172.21.15.22:0/4108414933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe0c83ddc0 0x55fe0c84c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a41c81740 1 --2- 172.21.15.22:0/4108414933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fe0c83d4a0 0x55fe0c84c410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a20ff9640 0 client.26144 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.850+0000 7f0a41c81740 1 -- 172.21.15.22:0/4108414933 >> 172.21.15.22:0/4108414933 conn(0x55fe0c839a30 msgr2=0x55fe0c838350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.851+0000 7f0a41c81740 1 -- 172.21.15.22:0/4108414933 shutdown_connections 2024-09-19T08:22:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.851+0000 7f0a41c81740 1 -- 172.21.15.22:0/4108414933 wait complete. 2024-09-19T08:22:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.851+0000 7f0a41c81740 20 client.26144 trim_cache size 0 max 16384 2024-09-19T08:22:13.858 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:22:13.858 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:22:13.858 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-19T08:22:13.858 INFO:teuthology.orchestra.run.smithi022.stdout: "a4ff4855-83f3-407a-9efb-8f696af28963": { 2024-09-19T08:22:13.858 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-19T08:22:13.858 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "a4ff4855-83f3-407a-9efb-8f696af28963", 2024-09-19T08:22:13.858 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-09-19T08:22:13.858 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:22:13.858 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:22:13.858 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:22:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.851+0000 7f0a3f8a8640 1 -- 172.21.15.22:0/483355764 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0a240a33b0 msgr2=0x7f0a240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.851+0000 7f0a3f8a8640 1 --2- 172.21.15.22:0/483355764 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0a240a33b0 0x7f0a240a5870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009900 tx=0x7f0a3002a480 comp rx=0 tx=0).stop 2024-09-19T08:22:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.851+0000 7f0a3f8a8640 1 -- 172.21.15.22:0/483355764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a38100e20 msgr2=0x7f0a380fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.851+0000 7f0a3f8a8640 1 --2- 172.21.15.22:0/483355764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a38100e20 0x7f0a380fcc80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400cce0 tx=0x7f0a3400a620 comp rx=0 tx=0).stop 2024-09-19T08:22:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.852+0000 7f0a3f8a8640 1 -- 172.21.15.22:0/483355764 shutdown_connections 2024-09-19T08:22:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.852+0000 7f0a3f8a8640 1 --2- 172.21.15.22:0/483355764 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0a240a33b0 0x7f0a240a5870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.852+0000 7f0a3f8a8640 1 --2- 172.21.15.22:0/483355764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a38102ef0 0x7f0a380fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.852+0000 7f0a3f8a8640 1 --2- 172.21.15.22:0/483355764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a38102570 0x7f0a380fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.852+0000 7f0a3f8a8640 1 --2- 172.21.15.22:0/483355764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a38100e20 0x7f0a380fcc80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:13.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.852+0000 7f0a3f8a8640 1 -- 172.21.15.22:0/483355764 >> 172.21.15.22:0/483355764 conn(0x7f0a380fc7d0 msgr2=0x7f0a3806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:13.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.852+0000 7f0a3f8a8640 1 -- 172.21.15.22:0/483355764 shutdown_connections 2024-09-19T08:22:13.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:13.852+0000 7f0a3f8a8640 1 -- 172.21.15.22:0/483355764 wait complete. 2024-09-19T08:22:14.688 INFO:tasks.cephfs.filesystem:scrub status for tag:a4ff4855-83f3-407a-9efb-8f696af28963 - {'path': '/', 'tag': 'a4ff4855-83f3-407a-9efb-8f696af28963', 'options': 'recursive,force'} 2024-09-19T08:22:14.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:14 smithi157 ceph-mon[35143]: pgmap v221: 97 pgs: 97 active+clean; 986 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 738 op/s 2024-09-19T08:22:14.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:14 smithi022 ceph-mon[30963]: pgmap v221: 97 pgs: 97 active+clean; 986 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 738 op/s 2024-09-19T08:22:14.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:14 smithi064 ceph-mon[34900]: pgmap v221: 97 pgs: 97 active+clean; 986 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 738 op/s 2024-09-19T08:22:16.158 INFO:tasks.workunit.client.0.smithi022.stdout: 170 1259705 403 825952 256 711556 825 2024-09-19T08:22:16.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:16 smithi157 ceph-mon[35143]: pgmap v222: 97 pgs: 97 active+clean; 986 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 738 op/s 2024-09-19T08:22:16.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:16 smithi022 ceph-mon[30963]: pgmap v222: 97 pgs: 97 active+clean; 986 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 738 op/s 2024-09-19T08:22:16.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:16 smithi064 ceph-mon[34900]: pgmap v222: 97 pgs: 97 active+clean; 986 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 738 op/s 2024-09-19T08:22:18.752 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:18 smithi157 ceph-mon[35143]: pgmap v223: 97 pgs: 97 active+clean; 981 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.5 MiB/s wr, 722 op/s 2024-09-19T08:22:18.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:18 smithi022 ceph-mon[30963]: pgmap v223: 97 pgs: 97 active+clean; 981 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.5 MiB/s wr, 722 op/s 2024-09-19T08:22:18.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:18 smithi064 ceph-mon[34900]: pgmap v223: 97 pgs: 97 active+clean; 981 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.5 MiB/s wr, 722 op/s 2024-09-19T08:22:18.908 INFO:tasks.workunit.client.0.smithi022.stdout: 2024-09-19T08:22:18.908 INFO:tasks.workunit.client.0.smithi022.stdout:Final score for writes: 170 2024-09-19T08:22:18.908 INFO:tasks.workunit.client.0.smithi022.stdout:Final score for reads : 549380 2024-09-19T08:22:18.909 INFO:tasks.workunit.client.0.smithi022.stdout: 2024-09-19T08:22:18.909 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T08:22:18.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T08:22:19.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:19 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:22:19.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:19 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:22:19.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:19 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:22:19.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:19 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:22:19.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:19 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:22:19.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:19 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:22:20.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:20 smithi157 ceph-mon[35143]: pgmap v224: 97 pgs: 97 active+clean; 986 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.9 MiB/s wr, 883 op/s 2024-09-19T08:22:20.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:20 smithi022 ceph-mon[30963]: pgmap v224: 97 pgs: 97 active+clean; 986 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.9 MiB/s wr, 883 op/s 2024-09-19T08:22:20.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:20 smithi064 ceph-mon[34900]: pgmap v224: 97 pgs: 97 active+clean; 986 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.9 MiB/s wr, 883 op/s 2024-09-19T08:22:22.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:22 smithi157 ceph-mon[35143]: pgmap v225: 97 pgs: 97 active+clean; 986 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 814 op/s 2024-09-19T08:22:22.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:22 smithi022 ceph-mon[30963]: pgmap v225: 97 pgs: 97 active+clean; 986 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 814 op/s 2024-09-19T08:22:22.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:22 smithi064 ceph-mon[34900]: pgmap v225: 97 pgs: 97 active+clean; 986 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 814 op/s 2024-09-19T08:22:23.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:23 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:22:23.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:23 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:22:23.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:23 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:22:23.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:23 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:22:23.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:23 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:22:23.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:23 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:22:23.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:23 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:22:23.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:23 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:22:23.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:23 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:22:23.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:23 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:22:23.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:23 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:22:23.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:23 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:22:24.024 INFO:tasks.quiescer.fs.[cephfs]:Total duration (51.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T08:22:24.025 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T08:22:24.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:22:24.305 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:22:24.715 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:24 smithi022 ceph-mon[30963]: pgmap v226: 97 pgs: 97 active+clean; 1000 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 970 op/s 2024-09-19T08:22:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.804+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3664420137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0102570 msgr2=0x7ff0f01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.804+0000 7ff0f7cbf640 1 --2- 172.21.15.22:0/3664420137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0102570 0x7ff0f01029b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009900 tx=0x7ff0e002a480 comp rx=0 tx=0).stop 2024-09-19T08:22:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.804+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3664420137 shutdown_connections 2024-09-19T08:22:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.804+0000 7ff0f7cbf640 1 --2- 172.21.15.22:0/3664420137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f0102ef0 0x7ff0f010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.804+0000 7ff0f7cbf640 1 --2- 172.21.15.22:0/3664420137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0102570 0x7ff0f01029b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.804+0000 7ff0f7cbf640 1 --2- 172.21.15.22:0/3664420137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f0100e20 0x7ff0f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.804+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3664420137 >> 172.21.15.22:0/3664420137 conn(0x7ff0f00fc7d0 msgr2=0x7ff0f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.804+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3664420137 shutdown_connections 2024-09-19T08:22:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.804+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3664420137 wait complete. 2024-09-19T08:22:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.805+0000 7ff0f7cbf640 1 Processor -- start 2024-09-19T08:22:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.805+0000 7ff0f7cbf640 1 -- start start 2024-09-19T08:22:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.805+0000 7ff0f7cbf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f0100e20 0x7ff0f010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.805+0000 7ff0f7cbf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f0102570 0x7ff0f010ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.805+0000 7ff0f7cbf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0102ef0 0x7ff0f010ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.805+0000 7ff0f7cbf640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0112060 con 0x7ff0f0102ef0 2024-09-19T08:22:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.805+0000 7ff0f7cbf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0111ee0 con 0x7ff0f0100e20 2024-09-19T08:22:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.805+0000 7ff0f7cbf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01121e0 con 0x7ff0f0102570 2024-09-19T08:22:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.805+0000 7ff0f625c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0102ef0 0x7ff0f010ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.805+0000 7ff0f625c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0102ef0 0x7ff0f010ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37422/0 (socket says 172.21.15.22:37422) 2024-09-19T08:22:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.805+0000 7ff0f625c640 1 -- 172.21.15.22:0/3741390831 learned_addr learned my addr 172.21.15.22:0/3741390831 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.805+0000 7ff0f525a640 1 --2- 172.21.15.22:0/3741390831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f0102570 0x7ff0f010ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.805+0000 7ff0f5a5b640 1 --2- 172.21.15.22:0/3741390831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f0100e20 0x7ff0f010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.806+0000 7ff0f625c640 1 -- 172.21.15.22:0/3741390831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f0102570 msgr2=0x7ff0f010ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.806+0000 7ff0f625c640 1 --2- 172.21.15.22:0/3741390831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f0102570 0x7ff0f010ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.806+0000 7ff0f625c640 1 -- 172.21.15.22:0/3741390831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f0100e20 msgr2=0x7ff0f010e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.806+0000 7ff0f625c640 1 --2- 172.21.15.22:0/3741390831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f0100e20 0x7ff0f010e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.806+0000 7ff0f625c640 1 -- 172.21.15.22:0/3741390831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f01a6980 con 0x7ff0f0102ef0 2024-09-19T08:22:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.806+0000 7ff0f625c640 1 --2- 172.21.15.22:0/3741390831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0102ef0 0x7ff0f010ef50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800eb10 tx=0x7ff0e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.806+0000 7ff0e6ffd640 1 -- 172.21.15.22:0/3741390831 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8004b20 con 0x7ff0f0102ef0 2024-09-19T08:22:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.806+0000 7ff0e6ffd640 1 -- 172.21.15.22:0/3741390831 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e8004cc0 con 0x7ff0f0102ef0 2024-09-19T08:22:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.806+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3741390831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f01a6c70 con 0x7ff0f0102ef0 2024-09-19T08:22:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.806+0000 7ff0e6ffd640 1 -- 172.21.15.22:0/3741390831 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8010760 con 0x7ff0f0102ef0 2024-09-19T08:22:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.806+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3741390831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f01a7180 con 0x7ff0f0102ef0 2024-09-19T08:22:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.807+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3741390831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f0101200 con 0x7ff0f0102ef0 2024-09-19T08:22:24.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.808+0000 7ff0e6ffd640 1 -- 172.21.15.22:0/3741390831 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff0e8010900 con 0x7ff0f0102ef0 2024-09-19T08:22:24.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.809+0000 7ff0e6ffd640 1 --2- 172.21.15.22:0/3741390831 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff0dc0a3430 0x7ff0dc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:24.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.809+0000 7ff0f5a5b640 1 --2- 172.21.15.22:0/3741390831 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff0dc0a3430 0x7ff0dc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:24.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.810+0000 7ff0e6ffd640 1 -- 172.21.15.22:0/3741390831 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7ff0e80d0410 con 0x7ff0f0102ef0 2024-09-19T08:22:24.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.810+0000 7ff0f5a5b640 1 --2- 172.21.15.22:0/3741390831 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff0dc0a3430 0x7ff0dc0a58f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00bbe0 tx=0x7ff0ec00b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:24.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:24.813+0000 7ff0e6ffd640 1 -- 172.21.15.22:0/3741390831 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff0e8054210 con 0x7ff0f0102ef0 2024-09-19T08:22:24.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:24 smithi064 ceph-mon[34900]: pgmap v226: 97 pgs: 97 active+clean; 1000 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 970 op/s 2024-09-19T08:22:25.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:24 smithi157 ceph-mon[35143]: pgmap v226: 97 pgs: 97 active+clean; 1000 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 970 op/s 2024-09-19T08:22:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:25.030+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3741390831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0f010fa00 con 0x7ff0f0102ef0 2024-09-19T08:22:25.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:25.030+0000 7ff0e6ffd640 1 -- 172.21.15.22:0/3741390831 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11910 (secure 0 0 0) 0x7ff0f010fa00 con 0x7ff0f0102ef0 2024-09-19T08:22:25.030 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:22:25.031 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":32,"btime":"2024-09-19T08:22:22:467137+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:22:21.873871+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:22:25.031 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 32 2024-09-19T08:22:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:25.032+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3741390831 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff0dc0a3430 msgr2=0x7ff0dc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:25.032+0000 7ff0f7cbf640 1 --2- 172.21.15.22:0/3741390831 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff0dc0a3430 0x7ff0dc0a58f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00bbe0 tx=0x7ff0ec00b390 comp rx=0 tx=0).stop 2024-09-19T08:22:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:25.032+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3741390831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0102ef0 msgr2=0x7ff0f010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:25.032+0000 7ff0f7cbf640 1 --2- 172.21.15.22:0/3741390831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0102ef0 0x7ff0f010ef50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800eb10 tx=0x7ff0e800c2d0 comp rx=0 tx=0).stop 2024-09-19T08:22:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:25.033+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3741390831 shutdown_connections 2024-09-19T08:22:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:25.033+0000 7ff0f7cbf640 1 --2- 172.21.15.22:0/3741390831 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff0dc0a3430 0x7ff0dc0a58f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:25.033+0000 7ff0f7cbf640 1 --2- 172.21.15.22:0/3741390831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0f0102ef0 0x7ff0f010ef50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:25.033+0000 7ff0f7cbf640 1 --2- 172.21.15.22:0/3741390831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f0102570 0x7ff0f010ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:25.033+0000 7ff0f7cbf640 1 --2- 172.21.15.22:0/3741390831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f0100e20 0x7ff0f010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:25.033+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3741390831 >> 172.21.15.22:0/3741390831 conn(0x7ff0f00fc7d0 msgr2=0x7ff0f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:25.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:25.033+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3741390831 shutdown_connections 2024-09-19T08:22:25.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:25.033+0000 7ff0f7cbf640 1 -- 172.21.15.22:0/3741390831 wait complete. 2024-09-19T08:22:25.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:25 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3741390831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:22:25.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:25 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3741390831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:22:25.949 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T08:22:25.949 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T08:22:26.002 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:25 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3741390831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:22:26.209 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:22:26.625 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:26 smithi022 ceph-mon[30963]: pgmap v227: 97 pgs: 97 active+clean; 1000 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 509 op/s 2024-09-19T08:22:26.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.718+0000 7ff5db305640 1 -- 172.21.15.22:0/1097221927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5d41036c0 msgr2=0x7ff5d410bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:26.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.718+0000 7ff5db305640 1 --2- 172.21.15.22:0/1097221927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5d41036c0 0x7ff5d410bbb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009900 tx=0x7ff5c402a480 comp rx=0 tx=0).stop 2024-09-19T08:22:26.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.718+0000 7ff5db305640 1 -- 172.21.15.22:0/1097221927 shutdown_connections 2024-09-19T08:22:26.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.718+0000 7ff5db305640 1 --2- 172.21.15.22:0/1097221927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d410c180 0x7ff5d410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.718+0000 7ff5db305640 1 --2- 172.21.15.22:0/1097221927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5d41036c0 0x7ff5d410bbb0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.718+0000 7ff5db305640 1 --2- 172.21.15.22:0/1097221927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d4102da0 0x7ff5d4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.718+0000 7ff5db305640 1 -- 172.21.15.22:0/1097221927 >> 172.21.15.22:0/1097221927 conn(0x7ff5d40fc7a0 msgr2=0x7ff5d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:26.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.718+0000 7ff5db305640 1 -- 172.21.15.22:0/1097221927 shutdown_connections 2024-09-19T08:22:26.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.718+0000 7ff5db305640 1 -- 172.21.15.22:0/1097221927 wait complete. 2024-09-19T08:22:26.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5db305640 1 Processor -- start 2024-09-19T08:22:26.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5db305640 1 -- start start 2024-09-19T08:22:26.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5db305640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5d4102da0 0x7ff5d4100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5db305640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d41036c0 0x7ff5d4100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5db305640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d410c180 0x7ff5d4101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5db305640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d4111fb0 con 0x7ff5d4102da0 2024-09-19T08:22:26.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5db305640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d4111e30 con 0x7ff5d41036c0 2024-09-19T08:22:26.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5db305640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d4112130 con 0x7ff5d410c180 2024-09-19T08:22:26.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5d90a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5d4102da0 0x7ff5d4100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5d90a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5d4102da0 0x7ff5d4100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37438/0 (socket says 172.21.15.22:37438) 2024-09-19T08:22:26.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5d90a1640 1 -- 172.21.15.22:0/3606147714 learned_addr learned my addr 172.21.15.22:0/3606147714 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:26.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5d88a0640 1 --2- 172.21.15.22:0/3606147714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d41036c0 0x7ff5d4100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5d98a2640 1 --2- 172.21.15.22:0/3606147714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d410c180 0x7ff5d4101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5d90a1640 1 -- 172.21.15.22:0/3606147714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d410c180 msgr2=0x7ff5d4101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:26.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5d90a1640 1 --2- 172.21.15.22:0/3606147714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d410c180 0x7ff5d4101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5d90a1640 1 -- 172.21.15.22:0/3606147714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d41036c0 msgr2=0x7ff5d4100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:26.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5d90a1640 1 --2- 172.21.15.22:0/3606147714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d41036c0 0x7ff5d4100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.719+0000 7ff5d90a1640 1 -- 172.21.15.22:0/3606147714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d41a68f0 con 0x7ff5d4102da0 2024-09-19T08:22:26.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.720+0000 7ff5d90a1640 1 --2- 172.21.15.22:0/3606147714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5d4102da0 0x7ff5d4100a00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000d950 tx=0x7ff5d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:26.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.720+0000 7ff5ca7fc640 1 -- 172.21.15.22:0/3606147714 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0017030 con 0x7ff5d4102da0 2024-09-19T08:22:26.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.720+0000 7ff5ca7fc640 1 -- 172.21.15.22:0/3606147714 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5d0004020 con 0x7ff5d4102da0 2024-09-19T08:22:26.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.720+0000 7ff5db305640 1 -- 172.21.15.22:0/3606147714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d41a6be0 con 0x7ff5d4102da0 2024-09-19T08:22:26.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.720+0000 7ff5ca7fc640 1 -- 172.21.15.22:0/3606147714 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00047a0 con 0x7ff5d4102da0 2024-09-19T08:22:26.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.720+0000 7ff5db305640 1 -- 172.21.15.22:0/3606147714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d41a7140 con 0x7ff5d4102da0 2024-09-19T08:22:26.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.722+0000 7ff5ca7fc640 1 -- 172.21.15.22:0/3606147714 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff5d00041d0 con 0x7ff5d4102da0 2024-09-19T08:22:26.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5dd6de740 1 Processor -- start 2024-09-19T08:22:26.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5dd6de740 1 -- start start 2024-09-19T08:22:26.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5dd6de740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ad850 0x55858a3adc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5dd6de740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ae170 0x55858a3b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5dd6de740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 0x55858a3bae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5dd6de740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55858a312160 con 0x55858a3ad850 2024-09-19T08:22:26.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5dd6de740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55858a2b9720 con 0x55858a3b8a30 2024-09-19T08:22:26.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5dd6de740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55858a30e120 con 0x55858a3ae170 2024-09-19T08:22:26.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5d98a2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ad850 0x55858a3adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5d90a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ae170 0x55858a3b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5d98a2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ad850 0x55858a3adc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37442/0 (socket says 172.21.15.22:37442) 2024-09-19T08:22:26.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5d98a2640 1 -- 172.21.15.22:0/3099989075 learned_addr learned my addr 172.21.15.22:0/3099989075 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:26.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5ca7fc640 1 --2- 172.21.15.22:0/3606147714 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff5c00a3480 0x7ff5c00a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5d88a0640 1 --2- 172.21.15.22:0/3099989075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 0x55858a3bae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5d98a2640 1 -- 172.21.15.22:0/3099989075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ae170 msgr2=0x55858a3b84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:26.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5d98a2640 1 --2- 172.21.15.22:0/3099989075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ae170 0x55858a3b84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5d98a2640 1 -- 172.21.15.22:0/3099989075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 msgr2=0x55858a3bae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:26.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.723+0000 7ff5d98a2640 1 --2- 172.21.15.22:0/3099989075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 0x55858a3bae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.724+0000 7ff5d98a2640 1 -- 172.21.15.22:0/3099989075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858a083ba0 con 0x55858a3ad850 2024-09-19T08:22:26.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.724+0000 7ff5d88a0640 1 --2- 172.21.15.22:0/3099989075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 0x55858a3bae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:22:26.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.724+0000 7ff5d98a2640 1 --2- 172.21.15.22:0/3606147714 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff5c00a3480 0x7ff5c00a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.724+0000 7ff5ca7fc640 1 -- 172.21.15.22:0/3606147714 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7ff5d00cf640 con 0x7ff5d4102da0 2024-09-19T08:22:26.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.724+0000 7ff5d98a2640 1 --2- 172.21.15.22:0/3099989075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ad850 0x55858a3adc30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc011590 tx=0x7ff5cc011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae9f967a244caec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:26.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.724+0000 7ff5bf7fe640 1 -- 172.21.15.22:0/3099989075 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc017050 con 0x55858a3ad850 2024-09-19T08:22:26.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.724+0000 7ff5bf7fe640 1 -- 172.21.15.22:0/3099989075 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5cc01bb90 con 0x55858a3ad850 2024-09-19T08:22:26.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.724+0000 7ff5bf7fe640 1 -- 172.21.15.22:0/3099989075 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00e3b0 con 0x55858a3ad850 2024-09-19T08:22:26.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.725+0000 7ff5dd6de740 1 -- 172.21.15.22:0/3099989075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ad850 msgr2=0x55858a3adc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:26.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.725+0000 7ff5d98a2640 1 --2- 172.21.15.22:0/3606147714 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff5c00a3480 0x7ff5c00a5940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc022000 tx=0x7ff5cc023000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:26.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.725+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/3099989075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ad850 0x55858a3adc30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc011590 tx=0x7ff5cc011a50 comp rx=0 tx=0).stop 2024-09-19T08:22:26.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.725+0000 7ff5dd6de740 1 -- 172.21.15.22:0/3099989075 shutdown_connections 2024-09-19T08:22:26.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.725+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/3099989075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 0x55858a3bae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.725+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/3099989075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ae170 0x55858a3b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.725+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/3099989075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ad850 0x55858a3adc30 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.725+0000 7ff5dd6de740 1 -- 172.21.15.22:0/3099989075 >> 172.21.15.22:0/3099989075 conn(0x55858a3a9de0 msgr2=0x55858a3aa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:26.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.725+0000 7ff5dd6de740 1 -- 172.21.15.22:0/3099989075 shutdown_connections 2024-09-19T08:22:26.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.725+0000 7ff5dd6de740 1 -- 172.21.15.22:0/3099989075 wait complete. 2024-09-19T08:22:26.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.725+0000 7ff5dd6de740 1 -- start start 2024-09-19T08:22:26.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.725+0000 7ff5dd6de740 1 Processor -- start 2024-09-19T08:22:26.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5dd6de740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5dd6de740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ae170 0x55858a3b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5dd6de740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 0x55858a3b6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5dd6de740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55858a3bb600 con 0x55858a3ae170 2024-09-19T08:22:26.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5dd6de740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55858a29a900 con 0x55858a3b8a30 2024-09-19T08:22:26.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5dd6de740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55858a3bb780 con 0x55858a3ad850 2024-09-19T08:22:26.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5dd6de740 10 client.0 _ll_register_callbacks cb 0x55858a298780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:22:26.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5d88a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ae170 0x55858a3b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5d90a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5d88a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ae170 0x55858a3b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37458/0 (socket says 172.21.15.22:37458) 2024-09-19T08:22:26.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5d88a0640 1 -- 172.21.15.22:0/1560732864 learned_addr learned my addr 172.21.15.22:0/1560732864 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:26.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5d98a2640 1 --2- 172.21.15.22:0/1560732864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 0x55858a3b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5d88a0640 1 -- 172.21.15.22:0/1560732864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 msgr2=0x55858a3b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:26.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5d88a0640 1 --2- 172.21.15.22:0/1560732864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5d88a0640 1 -- 172.21.15.22:0/1560732864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 msgr2=0x55858a3b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:26.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5d88a0640 1 --2- 172.21.15.22:0/1560732864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 0x55858a3b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5d88a0640 1 -- 172.21.15.22:0/1560732864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858a2dac50 con 0x55858a3ae170 2024-09-19T08:22:26.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5d90a1640 1 --2- 172.21.15.22:0/1560732864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:22:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5d88a0640 1 --2- 172.21.15.22:0/1560732864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ae170 0x55858a3b5df0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009230 tx=0x7ff5c40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403def764371e29a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.726+0000 7ff5bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:22:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5bcff9640 1 -- 172.21.15.22:0/1560732864 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4035040 con 0x55858a3ae170 2024-09-19T08:22:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1560732864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5cc01bb90 con 0x55858a3ae170 2024-09-19T08:22:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5bcff9640 1 -- 172.21.15.22:0/1560732864 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7ff5c4005450 con 0x55858a3ae170 2024-09-19T08:22:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5bcff9640 1 -- 172.21.15.22:0/1560732864 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff5c4005650 con 0x55858a3ae170 2024-09-19T08:22:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5dd6de740 10 client.16644 fetch_fsmap learned FSMap version 33 2024-09-19T08:22:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1560732864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55858a3c7240 con 0x55858a3ae170 2024-09-19T08:22:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5bcff9640 1 -- 172.21.15.22:0/1560732864 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5cc01bb90 con 0x55858a3ae170 2024-09-19T08:22:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5bcff9640 1 -- 172.21.15.22:0/1560732864 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff5c4031920 con 0x55858a3ae170 2024-09-19T08:22:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5dd6de740 10 client.16644 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:22:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5dd6de740 10 client.16644 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:22:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5dd6de740 20 client.16644 populate_metadata read hostname 'smithi022' 2024-09-19T08:22:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/1560732864 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55858a3c8700 0x55858a3cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5dd6de740 4 client.16644 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1560732864 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55858a31f890 con 0x55858a3c8700 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.727+0000 7ff5d90a1640 1 --2- 172.21.15.22:0/1560732864 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55858a3c8700 0x55858a3cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.728+0000 7ff5bcff9640 10 client.16644 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.728+0000 7ff5d90a1640 1 --2- 172.21.15.22:0/1560732864 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55858a3c8700 0x55858a3cab30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916f54c38290ebbc server_cookie=4ae45c04811a3a38 in_seq=0 out_seq=0 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.729+0000 7ff5bcff9640 1 -- 172.21.15.22:0/1560732864 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55858a31f890 con 0x55858a3c8700 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.729+0000 7ff5bcff9640 10 client.16644 handle_command_reply: tid=0 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.729+0000 7ff5dd6de740 1 client.16644 shutdown 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.729+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1560732864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ae170 msgr2=0x55858a3b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.729+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/1560732864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ae170 0x55858a3b5df0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009230 tx=0x7ff5c40043b0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.729+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1560732864 shutdown_connections 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.729+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/1560732864 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55858a3c8700 0x55858a3cab30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.729+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/1560732864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 0x55858a3b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.729+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/1560732864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ae170 0x55858a3b5df0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.729+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/1560732864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.729+0000 7ff5bcff9640 0 client.16644 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:26.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.729+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1560732864 >> 172.21.15.22:0/1560732864 conn(0x55858a3a9de0 msgr2=0x55858a3a8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:26.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.729+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1560732864 shutdown_connections 2024-09-19T08:22:26.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.729+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1560732864 wait complete. 2024-09-19T08:22:26.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.730+0000 7ff5dd6de740 20 client.16644 trim_cache size 0 max 16384 2024-09-19T08:22:26.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.748+0000 7ff5dd6de740 1 Processor -- start 2024-09-19T08:22:26.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.748+0000 7ff5dd6de740 1 -- start start 2024-09-19T08:22:26.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.748+0000 7ff5dd6de740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.748+0000 7ff5dd6de740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3ae170 0x55858a3d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.748+0000 7ff5dd6de740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3b8a30 0x55858a3b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.748+0000 7ff5dd6de740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41b24f0 con 0x55858a3b8a30 2024-09-19T08:22:26.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.748+0000 7ff5dd6de740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41afd30 con 0x55858a3ae170 2024-09-19T08:22:26.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.748+0000 7ff5dd6de740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41adba0 con 0x55858a3ad850 2024-09-19T08:22:26.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.748+0000 7ff5d98a2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3b8a30 0x55858a3b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.748+0000 7ff5d98a2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3b8a30 0x55858a3b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37470/0 (socket says 172.21.15.22:37470) 2024-09-19T08:22:26.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.749+0000 7ff5d98a2640 1 -- 172.21.15.22:0/2114460655 learned_addr learned my addr 172.21.15.22:0/2114460655 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:26.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.749+0000 7ff5d88a0640 1 --2- 172.21.15.22:0/2114460655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3ae170 0x55858a3d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.749+0000 7ff5d90a1640 1 --2- 172.21.15.22:0/2114460655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.749+0000 7ff5d98a2640 1 -- 172.21.15.22:0/2114460655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 msgr2=0x55858a3d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:26.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.749+0000 7ff5d98a2640 1 --2- 172.21.15.22:0/2114460655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.749+0000 7ff5d98a2640 1 -- 172.21.15.22:0/2114460655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3ae170 msgr2=0x55858a3d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:26.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.749+0000 7ff5d98a2640 1 --2- 172.21.15.22:0/2114460655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3ae170 0x55858a3d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.749+0000 7ff5d98a2640 1 -- 172.21.15.22:0/2114460655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858a3a53f0 con 0x55858a3b8a30 2024-09-19T08:22:26.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.749+0000 7ff5d98a2640 1 --2- 172.21.15.22:0/2114460655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3b8a30 0x55858a3b2b10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc025be0 tx=0x7ff5cc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4ca9403ec86ffe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:26.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.749+0000 7ff5beffd640 1 -- 172.21.15.22:0/2114460655 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc02a040 con 0x55858a3b8a30 2024-09-19T08:22:26.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.749+0000 7ff5beffd640 1 -- 172.21.15.22:0/2114460655 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5cc034070 con 0x55858a3b8a30 2024-09-19T08:22:26.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.749+0000 7ff5beffd640 1 -- 172.21.15.22:0/2114460655 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc01bbf0 con 0x55858a3b8a30 2024-09-19T08:22:26.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.750+0000 7ff5dd6de740 1 -- 172.21.15.22:0/2114460655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3b8a30 msgr2=0x55858a3b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:26.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.750+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/2114460655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3b8a30 0x55858a3b2b10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc025be0 tx=0x7ff5cc014000 comp rx=0 tx=0).stop 2024-09-19T08:22:26.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.750+0000 7ff5dd6de740 1 -- 172.21.15.22:0/2114460655 shutdown_connections 2024-09-19T08:22:26.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.750+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/2114460655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3b8a30 0x55858a3b2b10 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.750+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/2114460655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3ae170 0x55858a3d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.750+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/2114460655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.750+0000 7ff5dd6de740 1 -- 172.21.15.22:0/2114460655 >> 172.21.15.22:0/2114460655 conn(0x55858a3a9de0 msgr2=0x55858a3a86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:26.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.750+0000 7ff5dd6de740 1 -- 172.21.15.22:0/2114460655 shutdown_connections 2024-09-19T08:22:26.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.750+0000 7ff5dd6de740 1 -- 172.21.15.22:0/2114460655 wait complete. 2024-09-19T08:22:26.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.750+0000 7ff5dd6de740 1 -- start start 2024-09-19T08:22:26.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.750+0000 7ff5dd6de740 1 Processor -- start 2024-09-19T08:22:26.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5dd6de740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3c0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5dd6de740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ae170 0x55858a3c0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5dd6de740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 0x55858a3bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5dd6de740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41ae1f0 con 0x55858a3ae170 2024-09-19T08:22:26.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5dd6de740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41af660 con 0x55858a3b8a30 2024-09-19T08:22:26.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5dd6de740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41ae8c0 con 0x55858a3ad850 2024-09-19T08:22:26.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5dd6de740 10 client.0 _ll_register_callbacks cb 0x7ff5c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:22:26.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5d90a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3c0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5d88a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ae170 0x55858a3c0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5d90a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3c0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:47132/0 (socket says 172.21.15.22:47132) 2024-09-19T08:22:26.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5d90a1640 1 -- 172.21.15.22:0/1957477493 learned_addr learned my addr 172.21.15.22:0/1957477493 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:26.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5d98a2640 1 --2- 172.21.15.22:0/1957477493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 0x55858a3bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5d88a0640 1 -- 172.21.15.22:0/1957477493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 msgr2=0x55858a3c0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:26.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5d88a0640 1 --2- 172.21.15.22:0/1957477493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3c0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5d88a0640 1 -- 172.21.15.22:0/1957477493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 msgr2=0x55858a3bcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:26.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5d88a0640 1 --2- 172.21.15.22:0/1957477493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 0x55858a3bcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:26.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5d88a0640 1 -- 172.21.15.22:0/1957477493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858a3bd750 con 0x55858a3ae170 2024-09-19T08:22:26.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.751+0000 7ff5d90a1640 1 --2- 172.21.15.22:0/1957477493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3c0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:22:26.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.752+0000 7ff5d88a0640 1 --2- 172.21.15.22:0/1957477493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ae170 0x55858a3c0fd0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4031f30 tx=0x7ff5c4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0af6637b405ab26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:26.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.752+0000 7ff5bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:22:26.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.752+0000 7ff5bcff9640 1 -- 172.21.15.22:0/1957477493 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4035070 con 0x55858a3ae170 2024-09-19T08:22:26.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.752+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1957477493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5cc034070 con 0x55858a3ae170 2024-09-19T08:22:26.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.752+0000 7ff5bcff9640 1 -- 172.21.15.22:0/1957477493 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7ff5c40317b0 con 0x55858a3ae170 2024-09-19T08:22:26.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.752+0000 7ff5bcff9640 1 -- 172.21.15.22:0/1957477493 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff5c4052040 con 0x55858a3ae170 2024-09-19T08:22:26.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.752+0000 7ff5dd6de740 10 client.16650 fetch_fsmap learned FSMap version 33 2024-09-19T08:22:26.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.752+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1957477493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55858a433ee0 con 0x55858a3ae170 2024-09-19T08:22:26.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.752+0000 7ff5bcff9640 1 -- 172.21.15.22:0/1957477493 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5cc034070 con 0x55858a3ae170 2024-09-19T08:22:26.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.753+0000 7ff5bcff9640 1 -- 172.21.15.22:0/1957477493 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff5c4055070 con 0x55858a3ae170 2024-09-19T08:22:26.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.753+0000 7ff5dd6de740 10 client.16650 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:22:26.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.753+0000 7ff5dd6de740 10 client.16650 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:22:26.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.753+0000 7ff5dd6de740 20 client.16650 populate_metadata read hostname 'smithi022' 2024-09-19T08:22:26.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.753+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/1957477493 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55858a4346f0 0x55858a436b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:26.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.753+0000 7ff5dd6de740 4 client.16650 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T08:22:26.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.753+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1957477493 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x55858a319be0 con 0x55858a4346f0 2024-09-19T08:22:26.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.753+0000 7ff5d90a1640 1 --2- 172.21.15.22:0/1957477493 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55858a4346f0 0x55858a436b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:26.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.753+0000 7ff5bcff9640 10 client.16650 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:26.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:26.753+0000 7ff5d90a1640 1 --2- 172.21.15.22:0/1957477493 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55858a4346f0 0x55858a436b10 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3663d517cee79c07 server_cookie=9aa29c2068e82d0e in_seq=0 out_seq=0 2024-09-19T08:22:26.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:26 smithi064 ceph-mon[34900]: pgmap v227: 97 pgs: 97 active+clean; 1000 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 509 op/s 2024-09-19T08:22:27.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:26 smithi157 ceph-mon[35143]: pgmap v227: 97 pgs: 97 active+clean; 1000 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 509 op/s 2024-09-19T08:22:27.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:27 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:22:27.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:27 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:22:28.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:27 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:22:28.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:27 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:22:28.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:27 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:22:28.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:27 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:22:28.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:28 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:22:28.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:28 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:22:28.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:28 smithi064 ceph-mon[34900]: pgmap v228: 97 pgs: 97 active+clean; 988 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 545 op/s 2024-09-19T08:22:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:28 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:22:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:28 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:22:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:28 smithi157 ceph-mon[35143]: pgmap v228: 97 pgs: 97 active+clean; 988 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 545 op/s 2024-09-19T08:22:29.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:28 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:22:29.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:28 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:22:29.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:28 smithi022 ceph-mon[30963]: pgmap v228: 97 pgs: 97 active+clean; 988 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 545 op/s 2024-09-19T08:22:30.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:30 smithi064 ceph-mon[34900]: pgmap v229: 97 pgs: 97 active+clean; 983 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 681 op/s 2024-09-19T08:22:31.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:30 smithi157 ceph-mon[35143]: pgmap v229: 97 pgs: 97 active+clean; 983 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 681 op/s 2024-09-19T08:22:31.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:30 smithi022 ceph-mon[30963]: pgmap v229: 97 pgs: 97 active+clean; 983 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 681 op/s 2024-09-19T08:22:32.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:32 smithi064 ceph-mon[34900]: pgmap v230: 97 pgs: 97 active+clean; 983 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 433 op/s 2024-09-19T08:22:33.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:32 smithi157 ceph-mon[35143]: pgmap v230: 97 pgs: 97 active+clean; 983 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 433 op/s 2024-09-19T08:22:33.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:32 smithi022 ceph-mon[30963]: pgmap v230: 97 pgs: 97 active+clean; 983 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 433 op/s 2024-09-19T08:22:34.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:34 smithi064 ceph-mon[34900]: pgmap v231: 97 pgs: 97 active+clean; 988 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 566 op/s 2024-09-19T08:22:35.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:34 smithi157 ceph-mon[35143]: pgmap v231: 97 pgs: 97 active+clean; 988 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 566 op/s 2024-09-19T08:22:35.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:34 smithi022 ceph-mon[30963]: pgmap v231: 97 pgs: 97 active+clean; 988 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 566 op/s 2024-09-19T08:22:36.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:36 smithi064 ceph-mon[34900]: pgmap v232: 97 pgs: 97 active+clean; 988 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 332 op/s 2024-09-19T08:22:37.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:36 smithi157 ceph-mon[35143]: pgmap v232: 97 pgs: 97 active+clean; 988 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 332 op/s 2024-09-19T08:22:37.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:36 smithi022 ceph-mon[30963]: pgmap v232: 97 pgs: 97 active+clean; 988 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 332 op/s 2024-09-19T08:22:38.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:38 smithi064 ceph-mon[34900]: pgmap v233: 97 pgs: 97 active+clean; 996 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 519 op/s 2024-09-19T08:22:39.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:38 smithi157 ceph-mon[35143]: pgmap v233: 97 pgs: 97 active+clean; 996 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 519 op/s 2024-09-19T08:22:39.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:38 smithi022 ceph-mon[30963]: pgmap v233: 97 pgs: 97 active+clean; 996 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 519 op/s 2024-09-19T08:22:40.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:40 smithi064 ceph-mon[34900]: pgmap v234: 97 pgs: 97 active+clean; 1000 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 601 op/s 2024-09-19T08:22:41.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:40 smithi157 ceph-mon[35143]: pgmap v234: 97 pgs: 97 active+clean; 1000 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 601 op/s 2024-09-19T08:22:41.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:40 smithi022 ceph-mon[30963]: pgmap v234: 97 pgs: 97 active+clean; 1000 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 601 op/s 2024-09-19T08:22:42.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:42 smithi064 ceph-mon[34900]: pgmap v235: 97 pgs: 97 active+clean; 1000 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 438 op/s 2024-09-19T08:22:43.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:42 smithi157 ceph-mon[35143]: pgmap v235: 97 pgs: 97 active+clean; 1000 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 438 op/s 2024-09-19T08:22:43.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:42 smithi022 ceph-mon[30963]: pgmap v235: 97 pgs: 97 active+clean; 1000 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 438 op/s 2024-09-19T08:22:44.689 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:22:44.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:44 smithi064 ceph-mon[34900]: pgmap v236: 97 pgs: 97 active+clean; 1015 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 910 op/s 2024-09-19T08:22:44.973 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:22:45.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:44 smithi157 ceph-mon[35143]: pgmap v236: 97 pgs: 97 active+clean; 1015 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 910 op/s 2024-09-19T08:22:45.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:44 smithi022 ceph-mon[30963]: pgmap v236: 97 pgs: 97 active+clean; 1015 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 910 op/s 2024-09-19T08:22:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.667+0000 7f3b0824f640 1 -- 172.21.15.22:0/1338581181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b00100e20 msgr2=0x7f3b00101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.667+0000 7f3b0824f640 1 --2- 172.21.15.22:0/1338581181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b00100e20 0x7f3b00101200 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009900 tx=0x7f3afc02a480 comp rx=0 tx=0).stop 2024-09-19T08:22:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.667+0000 7f3b0824f640 1 -- 172.21.15.22:0/1338581181 shutdown_connections 2024-09-19T08:22:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.667+0000 7f3b0824f640 1 --2- 172.21.15.22:0/1338581181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b00102ef0 0x7f3b0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.668+0000 7f3b0824f640 1 --2- 172.21.15.22:0/1338581181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b00102570 0x7f3b001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.668+0000 7f3b0824f640 1 --2- 172.21.15.22:0/1338581181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b00100e20 0x7f3b00101200 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.668+0000 7f3b0824f640 1 -- 172.21.15.22:0/1338581181 >> 172.21.15.22:0/1338581181 conn(0x7f3b000fc7d0 msgr2=0x7f3b000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:45.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.668+0000 7f3b0824f640 1 -- 172.21.15.22:0/1338581181 shutdown_connections 2024-09-19T08:22:45.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.668+0000 7f3b0824f640 1 -- 172.21.15.22:0/1338581181 wait complete. 2024-09-19T08:22:45.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.670+0000 7f3b0824f640 1 Processor -- start 2024-09-19T08:22:45.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.670+0000 7f3b0824f640 1 -- start start 2024-09-19T08:22:45.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.670+0000 7f3b0824f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b00100e20 0x7f3b00199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.670+0000 7f3b0824f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b00102570 0x7f3b001998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.670+0000 7f3b0824f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b00102ef0 0x7f3b001a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.670+0000 7f3b0824f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00110f50 con 0x7f3b00100e20 2024-09-19T08:22:45.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.670+0000 7f3b0824f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00110dd0 con 0x7f3b00102570 2024-09-19T08:22:45.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.670+0000 7f3b0824f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001110d0 con 0x7f3b00102ef0 2024-09-19T08:22:45.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.670+0000 7f3b067ec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b00102ef0 0x7f3b001a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.670+0000 7f3b067ec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b00102ef0 0x7f3b001a43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:56036/0 (socket says 172.21.15.22:56036) 2024-09-19T08:22:45.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.670+0000 7f3b067ec640 1 -- 172.21.15.22:0/4279611181 learned_addr learned my addr 172.21.15.22:0/4279611181 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:45.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.671+0000 7f3b057ea640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b00102570 0x7f3b001998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.671+0000 7f3b05feb640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b00100e20 0x7f3b00199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.671+0000 7f3b067ec640 1 -- 172.21.15.22:0/4279611181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b00102570 msgr2=0x7f3b001998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:45.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.671+0000 7f3b067ec640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b00102570 0x7f3b001998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.671+0000 7f3b067ec640 1 -- 172.21.15.22:0/4279611181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b00100e20 msgr2=0x7f3b00199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:45.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.671+0000 7f3b067ec640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b00100e20 0x7f3b00199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.671+0000 7f3b067ec640 1 -- 172.21.15.22:0/4279611181 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b001a4920 con 0x7f3b00102ef0 2024-09-19T08:22:45.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.671+0000 7f3b05feb640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b00100e20 0x7f3b00199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:22:45.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.671+0000 7f3b067ec640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b00102ef0 0x7f3b001a43e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009fd0 tx=0x7f3af800b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:45.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.672+0000 7f3af6ffd640 1 -- 172.21.15.22:0/4279611181 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8018070 con 0x7f3b00102ef0 2024-09-19T08:22:45.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.672+0000 7f3af6ffd640 1 -- 172.21.15.22:0/4279611181 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af8003e50 con 0x7f3b00102ef0 2024-09-19T08:22:45.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.672+0000 7f3af6ffd640 1 -- 172.21.15.22:0/4279611181 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8004280 con 0x7f3b00102ef0 2024-09-19T08:22:45.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.672+0000 7f3b0824f640 1 -- 172.21.15.22:0/4279611181 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b00197150 con 0x7f3b00102ef0 2024-09-19T08:22:45.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.672+0000 7f3b0824f640 1 -- 172.21.15.22:0/4279611181 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b001975e0 con 0x7f3b00102ef0 2024-09-19T08:22:45.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.675+0000 7f3af6ffd640 1 -- 172.21.15.22:0/4279611181 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3af80128f0 con 0x7f3b00102ef0 2024-09-19T08:22:45.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.675+0000 7f3b0a628740 1 Processor -- start 2024-09-19T08:22:45.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.676+0000 7f3b0a628740 1 -- start start 2024-09-19T08:22:45.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.676+0000 7f3b0a628740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14ea4a0 0x555cb14ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.676+0000 7f3b0a628740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14eadc0 0x555cb14f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.676+0000 7f3b0a628740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14f5680 0x555cb14f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.676+0000 7f3b0a628740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555cb14502e0 con 0x555cb14eadc0 2024-09-19T08:22:45.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.676+0000 7f3b0a628740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555cb13f7ce0 con 0x555cb14f5680 2024-09-19T08:22:45.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.676+0000 7f3b0a628740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555cb144c2a0 con 0x555cb14ea4a0 2024-09-19T08:22:45.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.676+0000 7f3b057ea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14ea4a0 0x555cb14ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.676+0000 7f3b057ea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14ea4a0 0x555cb14ea880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:56048/0 (socket says 172.21.15.22:56048) 2024-09-19T08:22:45.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.676+0000 7f3b057ea640 1 -- 172.21.15.22:0/752625500 learned_addr learned my addr 172.21.15.22:0/752625500 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:45.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.676+0000 7f3b05feb640 1 --2- 172.21.15.22:0/752625500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14eadc0 0x555cb14f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.677+0000 7f3af6ffd640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3aec0a3430 0x7f3aec0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.677+0000 7f3b057ea640 1 --2- 172.21.15.22:0/752625500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14f5680 0x555cb14f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.677+0000 7f3b057ea640 1 -- 172.21.15.22:0/752625500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14f5680 msgr2=0x555cb14f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:45.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.677+0000 7f3b057ea640 1 --2- 172.21.15.22:0/752625500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14f5680 0x555cb14f7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.677+0000 7f3b057ea640 1 -- 172.21.15.22:0/752625500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14eadc0 msgr2=0x555cb14f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:45.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.677+0000 7f3b057ea640 1 --2- 172.21.15.22:0/752625500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14eadc0 0x555cb14f5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.677+0000 7f3b057ea640 1 -- 172.21.15.22:0/752625500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cb13dc440 con 0x555cb14ea4a0 2024-09-19T08:22:45.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.677+0000 7f3b067ec640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3aec0a3430 0x7f3aec0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.677+0000 7f3af6ffd640 1 -- 172.21.15.22:0/4279611181 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f3af80d3a80 con 0x7f3b00102ef0 2024-09-19T08:22:45.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.677+0000 7f3b057ea640 1 --2- 172.21.15.22:0/752625500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14ea4a0 0x555cb14ea880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3af00058a0 tx=0x7f3af000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d521b7c45ccc1769 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:45.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.677+0000 7f3b067ec640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3aec0a3430 0x7f3aec0a58f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3af809c830 tx=0x7f3af8012be0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:45.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.678+0000 7f3aebfff640 1 -- 172.21.15.22:0/752625500 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0004030 con 0x555cb14ea4a0 2024-09-19T08:22:45.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.678+0000 7f3aebfff640 1 -- 172.21.15.22:0/752625500 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af00041d0 con 0x555cb14ea4a0 2024-09-19T08:22:45.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.678+0000 7f3aebfff640 1 -- 172.21.15.22:0/752625500 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af00106c0 con 0x555cb14ea4a0 2024-09-19T08:22:45.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.678+0000 7f3b0a628740 1 -- 172.21.15.22:0/752625500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14ea4a0 msgr2=0x555cb14ea880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:45.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.678+0000 7f3b0a628740 1 --2- 172.21.15.22:0/752625500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14ea4a0 0x555cb14ea880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3af00058a0 tx=0x7f3af000b840 comp rx=0 tx=0).stop 2024-09-19T08:22:45.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.679+0000 7f3b0a628740 1 -- 172.21.15.22:0/752625500 shutdown_connections 2024-09-19T08:22:45.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.679+0000 7f3b0a628740 1 --2- 172.21.15.22:0/752625500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14f5680 0x555cb14f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.679+0000 7f3b0a628740 1 --2- 172.21.15.22:0/752625500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14eadc0 0x555cb14f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.679+0000 7f3b0a628740 1 --2- 172.21.15.22:0/752625500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14ea4a0 0x555cb14ea880 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.679+0000 7f3b0a628740 1 -- 172.21.15.22:0/752625500 >> 172.21.15.22:0/752625500 conn(0x555cb14e6a30 msgr2=0x555cb14e6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:45.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.679+0000 7f3b0a628740 1 -- 172.21.15.22:0/752625500 shutdown_connections 2024-09-19T08:22:45.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.679+0000 7f3b0a628740 1 -- 172.21.15.22:0/752625500 wait complete. 2024-09-19T08:22:45.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.679+0000 7f3b0a628740 1 -- start start 2024-09-19T08:22:45.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.679+0000 7f3b0a628740 1 Processor -- start 2024-09-19T08:22:45.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.680+0000 7f3b0a628740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14ea4a0 0x555cb14f5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.680+0000 7f3b0a628740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14eadc0 0x555cb14f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.680+0000 7f3b0a628740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14f5680 0x555cb14f2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.680+0000 7f3b0a628740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555cb14f8250 con 0x555cb14eadc0 2024-09-19T08:22:45.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.680+0000 7f3b0a628740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555cb140b050 con 0x555cb14ea4a0 2024-09-19T08:22:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.680+0000 7f3b0a628740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555cb14f83d0 con 0x555cb14f5680 2024-09-19T08:22:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.680+0000 7f3b0a628740 10 client.0 _ll_register_callbacks cb 0x555cb144b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:22:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.680+0000 7f3b05feb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14eadc0 0x555cb14f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.680+0000 7f3b05feb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14eadc0 0x555cb14f22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53346/0 (socket says 172.21.15.22:53346) 2024-09-19T08:22:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.680+0000 7f3b05feb640 1 -- 172.21.15.22:0/2672219434 learned_addr learned my addr 172.21.15.22:0/2672219434 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.681+0000 7f3b057ea640 1 --2- 172.21.15.22:0/2672219434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14f5680 0x555cb14f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.681+0000 7f3b057ea640 1 --2- 172.21.15.22:0/2672219434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14ea4a0 0x555cb14f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.681+0000 7f3b05feb640 1 -- 172.21.15.22:0/2672219434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14f5680 msgr2=0x555cb14f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.681+0000 7f3b05feb640 1 --2- 172.21.15.22:0/2672219434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14f5680 0x555cb14f2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.681+0000 7f3b05feb640 1 -- 172.21.15.22:0/2672219434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14ea4a0 msgr2=0x555cb14f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.681+0000 7f3b05feb640 1 --2- 172.21.15.22:0/2672219434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14ea4a0 0x555cb14f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.681+0000 7f3b05feb640 1 -- 172.21.15.22:0/2672219434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cb11c13c0 con 0x555cb14eadc0 2024-09-19T08:22:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.681+0000 7f3b057ea640 1 --2- 172.21.15.22:0/2672219434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14ea4a0 0x555cb14f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:22:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.681+0000 7f3b05feb640 1 --2- 172.21.15.22:0/2672219434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14eadc0 0x555cb14f22c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009900 tx=0x7f3afc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682c12fc2a7623f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.681+0000 7f3ae97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:22:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.682+0000 7f3ae97fa640 1 -- 172.21.15.22:0/2672219434 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc035040 con 0x555cb14eadc0 2024-09-19T08:22:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.682+0000 7f3ae97fa640 1 -- 172.21.15.22:0/2672219434 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f3afc0048c0 con 0x555cb14eadc0 2024-09-19T08:22:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.682+0000 7f3b0a628740 1 -- 172.21.15.22:0/2672219434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af00041d0 con 0x555cb14eadc0 2024-09-19T08:22:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.682+0000 7f3ae97fa640 1 -- 172.21.15.22:0/2672219434 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3afc004ac0 con 0x555cb14eadc0 2024-09-19T08:22:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.682+0000 7f3b0a628740 10 client.16662 fetch_fsmap learned FSMap version 33 2024-09-19T08:22:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.682+0000 7f3b0a628740 1 -- 172.21.15.22:0/2672219434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555cb1503c40 con 0x555cb14eadc0 2024-09-19T08:22:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.682+0000 7f3ae97fa640 1 -- 172.21.15.22:0/2672219434 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af00041d0 con 0x555cb14eadc0 2024-09-19T08:22:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.683+0000 7f3ae97fa640 1 -- 172.21.15.22:0/2672219434 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3afc031540 con 0x555cb14eadc0 2024-09-19T08:22:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.683+0000 7f3b0a628740 10 client.16662 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:22:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.683+0000 7f3b0a628740 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:22:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.683+0000 7f3b0a628740 20 client.16662 populate_metadata read hostname 'smithi022' 2024-09-19T08:22:45.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.683+0000 7f3b0a628740 1 --2- 172.21.15.22:0/2672219434 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555cb15050a0 0x555cb15074c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.683+0000 7f3b0a628740 4 client.16662 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:22:45.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.683+0000 7f3b0a628740 1 -- 172.21.15.22:0/2672219434 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cb145d850 con 0x555cb15050a0 2024-09-19T08:22:45.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.683+0000 7f3b057ea640 1 --2- 172.21.15.22:0/2672219434 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555cb15050a0 0x555cb15074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.683+0000 7f3b057ea640 1 --2- 172.21.15.22:0/2672219434 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555cb15050a0 0x555cb15074c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a325cba400247df2 server_cookie=275264ea8775d6eb in_seq=0 out_seq=0 2024-09-19T08:22:45.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.683+0000 7f3ae97fa640 10 client.16662 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:45.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.684+0000 7f3ae97fa640 1 -- 172.21.15.22:0/2672219434 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555cb145d850 con 0x555cb15050a0 2024-09-19T08:22:45.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.684+0000 7f3ae97fa640 10 client.16662 handle_command_reply: tid=0 2024-09-19T08:22:45.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.684+0000 7f3b0a628740 1 client.16662 shutdown 2024-09-19T08:22:45.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.684+0000 7f3b0a628740 1 -- 172.21.15.22:0/2672219434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14eadc0 msgr2=0x555cb14f22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:45.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.684+0000 7f3b0a628740 1 --2- 172.21.15.22:0/2672219434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14eadc0 0x555cb14f22c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009900 tx=0x7f3afc005270 comp rx=0 tx=0).stop 2024-09-19T08:22:45.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.685+0000 7f3b0a628740 1 -- 172.21.15.22:0/2672219434 shutdown_connections 2024-09-19T08:22:45.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.685+0000 7f3b0a628740 1 --2- 172.21.15.22:0/2672219434 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555cb15050a0 0x555cb15074c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.685+0000 7f3b0a628740 1 --2- 172.21.15.22:0/2672219434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14f5680 0x555cb14f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.685+0000 7f3b0a628740 1 --2- 172.21.15.22:0/2672219434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14eadc0 0x555cb14f22c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.685+0000 7f3ae97fa640 0 client.16662 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:45.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.685+0000 7f3b0a628740 1 --2- 172.21.15.22:0/2672219434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14ea4a0 0x555cb14f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.685+0000 7f3b0a628740 1 -- 172.21.15.22:0/2672219434 >> 172.21.15.22:0/2672219434 conn(0x555cb14e6a30 msgr2=0x555cb14ec760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:45.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.685+0000 7f3b0a628740 1 -- 172.21.15.22:0/2672219434 shutdown_connections 2024-09-19T08:22:45.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.685+0000 7f3b0a628740 1 -- 172.21.15.22:0/2672219434 wait complete. 2024-09-19T08:22:45.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.685+0000 7f3b0a628740 20 client.16662 trim_cache size 0 max 16384 2024-09-19T08:22:45.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.710+0000 7f3b0a628740 1 Processor -- start 2024-09-19T08:22:45.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.710+0000 7f3b0a628740 1 -- start start 2024-09-19T08:22:45.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b0a628740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14ea4a0 0x555cb14f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b0a628740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14eadc0 0x555cb150cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b0a628740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14f5680 0x555cb150d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b0a628740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001ae050 con 0x555cb14f5680 2024-09-19T08:22:45.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b0a628740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001ab890 con 0x555cb14eadc0 2024-09-19T08:22:45.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b0a628740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001a9700 con 0x555cb14ea4a0 2024-09-19T08:22:45.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b057ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14f5680 0x555cb150d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b057ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14f5680 0x555cb150d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53350/0 (socket says 172.21.15.22:53350) 2024-09-19T08:22:45.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b057ea640 1 -- 172.21.15.22:0/1915960614 learned_addr learned my addr 172.21.15.22:0/1915960614 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:45.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b057ea640 1 --2- 172.21.15.22:0/1915960614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14ea4a0 0x555cb14f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b05feb640 1 --2- 172.21.15.22:0/1915960614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14eadc0 0x555cb150cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b057ea640 1 -- 172.21.15.22:0/1915960614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14ea4a0 msgr2=0x555cb14f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:45.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b057ea640 1 --2- 172.21.15.22:0/1915960614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14ea4a0 0x555cb14f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b057ea640 1 -- 172.21.15.22:0/1915960614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14eadc0 msgr2=0x555cb150cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:45.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b057ea640 1 --2- 172.21.15.22:0/1915960614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14eadc0 0x555cb150cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.711+0000 7f3b057ea640 1 -- 172.21.15.22:0/1915960614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cb14ea2d0 con 0x555cb14f5680 2024-09-19T08:22:45.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.712+0000 7f3b057ea640 1 --2- 172.21.15.22:0/1915960614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14f5680 0x555cb150d2b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3af0012570 tx=0x7f3af000b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765df0b0c953e7d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:45.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.712+0000 7f3aeb7fe640 1 -- 172.21.15.22:0/1915960614 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af00053f0 con 0x555cb14f5680 2024-09-19T08:22:45.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.712+0000 7f3aeb7fe640 1 -- 172.21.15.22:0/1915960614 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af0005590 con 0x555cb14f5680 2024-09-19T08:22:45.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.712+0000 7f3aeb7fe640 1 -- 172.21.15.22:0/1915960614 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af002bab0 con 0x555cb14f5680 2024-09-19T08:22:45.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.713+0000 7f3b0a628740 1 -- 172.21.15.22:0/1915960614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14f5680 msgr2=0x555cb150d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:45.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.713+0000 7f3b0a628740 1 --2- 172.21.15.22:0/1915960614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14f5680 0x555cb150d2b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3af0012570 tx=0x7f3af000b570 comp rx=0 tx=0).stop 2024-09-19T08:22:45.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.713+0000 7f3b0a628740 1 -- 172.21.15.22:0/1915960614 shutdown_connections 2024-09-19T08:22:45.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.713+0000 7f3b0a628740 1 --2- 172.21.15.22:0/1915960614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14f5680 0x555cb150d2b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.713+0000 7f3b0a628740 1 --2- 172.21.15.22:0/1915960614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14eadc0 0x555cb150cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.713+0000 7f3b0a628740 1 --2- 172.21.15.22:0/1915960614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14ea4a0 0x555cb14f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.713+0000 7f3b0a628740 1 -- 172.21.15.22:0/1915960614 >> 172.21.15.22:0/1915960614 conn(0x555cb14e6a30 msgr2=0x555cb14e5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.713+0000 7f3b0a628740 1 -- 172.21.15.22:0/1915960614 shutdown_connections 2024-09-19T08:22:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.713+0000 7f3b0a628740 1 -- 172.21.15.22:0/1915960614 wait complete. 2024-09-19T08:22:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.714+0000 7f3b0a628740 1 -- start start 2024-09-19T08:22:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.714+0000 7f3b0a628740 1 Processor -- start 2024-09-19T08:22:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.714+0000 7f3b0a628740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14ea4a0 0x555cb14fb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.714+0000 7f3b0a628740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14eadc0 0x555cb14fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b0a628740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14f5680 0x555cb14fbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b0a628740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001a9d50 con 0x555cb14ea4a0 2024-09-19T08:22:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b0a628740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001ab1c0 con 0x555cb14eadc0 2024-09-19T08:22:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b057ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14ea4a0 0x555cb14fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b0a628740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001aa420 con 0x555cb14f5680 2024-09-19T08:22:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b057ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14ea4a0 0x555cb14fb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53356/0 (socket says 172.21.15.22:53356) 2024-09-19T08:22:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b0a628740 10 client.0 _ll_register_callbacks cb 0x7f3afc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:22:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b057ea640 1 -- 172.21.15.22:0/1118477695 learned_addr learned my addr 172.21.15.22:0/1118477695 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b057ea640 1 --2- 172.21.15.22:0/1118477695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14f5680 0x555cb14fbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b057ea640 1 -- 172.21.15.22:0/1118477695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14f5680 msgr2=0x555cb14fbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b057ea640 1 --2- 172.21.15.22:0/1118477695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14f5680 0x555cb14fbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b057ea640 1 -- 172.21.15.22:0/1118477695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14eadc0 msgr2=0x555cb14fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b057ea640 1 --2- 172.21.15.22:0/1118477695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14eadc0 0x555cb14fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b057ea640 1 -- 172.21.15.22:0/1118477695 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cb1577560 con 0x555cb14ea4a0 2024-09-19T08:22:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.715+0000 7f3b057ea640 1 --2- 172.21.15.22:0/1118477695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14ea4a0 0x555cb14fb380 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3af000b620 tx=0x7f3af000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec978c98f0c4b40c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.716+0000 7f3ae97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:22:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.716+0000 7f3ae97fa640 1 -- 172.21.15.22:0/1118477695 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af002b560 con 0x555cb14ea4a0 2024-09-19T08:22:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.716+0000 7f3ae97fa640 1 -- 172.21.15.22:0/1118477695 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f3af002b830 con 0x555cb14ea4a0 2024-09-19T08:22:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.716+0000 7f3b0a628740 1 -- 172.21.15.22:0/1118477695 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af0005590 con 0x555cb14ea4a0 2024-09-19T08:22:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.716+0000 7f3ae97fa640 1 -- 172.21.15.22:0/1118477695 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3af002bc40 con 0x555cb14ea4a0 2024-09-19T08:22:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.716+0000 7f3b0a628740 10 client.16668 fetch_fsmap learned FSMap version 33 2024-09-19T08:22:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.716+0000 7f3b0a628740 1 -- 172.21.15.22:0/1118477695 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555cb15787f0 con 0x555cb14ea4a0 2024-09-19T08:22:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.717+0000 7f3ae97fa640 1 -- 172.21.15.22:0/1118477695 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af0005590 con 0x555cb14ea4a0 2024-09-19T08:22:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.717+0000 7f3ae97fa640 1 -- 172.21.15.22:0/1118477695 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3af00109d0 con 0x555cb14ea4a0 2024-09-19T08:22:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.717+0000 7f3b0a628740 10 client.16668 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:22:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.717+0000 7f3b0a628740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:22:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.717+0000 7f3b0a628740 20 client.16668 populate_metadata read hostname 'smithi022' 2024-09-19T08:22:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.717+0000 7f3b0a628740 1 --2- 172.21.15.22:0/1118477695 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555cb1578bd0 0x555cb157afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.717+0000 7f3b0a628740 4 client.16668 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:22:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.717+0000 7f3b0a628740 1 -- 172.21.15.22:0/1118477695 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555cb1457d60 con 0x555cb1578bd0 2024-09-19T08:22:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.717+0000 7f3b05feb640 1 --2- 172.21.15.22:0/1118477695 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555cb1578bd0 0x555cb157afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.718+0000 7f3b05feb640 1 --2- 172.21.15.22:0/1118477695 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555cb1578bd0 0x555cb157afc0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcc321a36aa13293 server_cookie=2b47e35a4c22064 in_seq=0 out_seq=0 2024-09-19T08:22:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.718+0000 7f3ae97fa640 10 client.16668 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.718+0000 7f3ae97fa640 1 -- 172.21.15.22:0/1118477695 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555cb1457d60 con 0x555cb1578bd0 2024-09-19T08:22:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.718+0000 7f3ae97fa640 10 client.16668 handle_command_reply: tid=0 2024-09-19T08:22:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.718+0000 7f3b0a628740 1 client.16668 shutdown 2024-09-19T08:22:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.718+0000 7f3b0a628740 1 -- 172.21.15.22:0/1118477695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14ea4a0 msgr2=0x555cb14fb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.719+0000 7f3b0a628740 1 --2- 172.21.15.22:0/1118477695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14ea4a0 0x555cb14fb380 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3af000b620 tx=0x7f3af000b8b0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.719+0000 7f3b0a628740 1 -- 172.21.15.22:0/1118477695 shutdown_connections 2024-09-19T08:22:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.719+0000 7f3b0a628740 1 --2- 172.21.15.22:0/1118477695 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555cb1578bd0 0x555cb157afc0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.719+0000 7f3b0a628740 1 --2- 172.21.15.22:0/1118477695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cb14f5680 0x555cb14fbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.719+0000 7f3b0a628740 1 --2- 172.21.15.22:0/1118477695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555cb14eadc0 0x555cb14fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.719+0000 7f3b0a628740 1 --2- 172.21.15.22:0/1118477695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555cb14ea4a0 0x555cb14fb380 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.719+0000 7f3b0a628740 1 -- 172.21.15.22:0/1118477695 >> 172.21.15.22:0/1118477695 conn(0x555cb14e6a30 msgr2=0x555cb14e5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.719+0000 7f3ae97fa640 0 client.16668 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.719+0000 7f3b0a628740 1 -- 172.21.15.22:0/1118477695 shutdown_connections 2024-09-19T08:22:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.719+0000 7f3b0a628740 1 -- 172.21.15.22:0/1118477695 wait complete. 2024-09-19T08:22:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.719+0000 7f3b0a628740 20 client.16668 trim_cache size 0 max 16384 2024-09-19T08:22:45.722 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:22:45.723 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-19T08:22:45.723 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-09-19T08:22:45.723 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:22:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.720+0000 7f3b0824f640 1 -- 172.21.15.22:0/4279611181 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3aec0a3430 msgr2=0x7f3aec0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.720+0000 7f3b0824f640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3aec0a3430 0x7f3aec0a58f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3af809c830 tx=0x7f3af8012be0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.720+0000 7f3b0824f640 1 -- 172.21.15.22:0/4279611181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b00102ef0 msgr2=0x7f3b001a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.720+0000 7f3b0824f640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b00102ef0 0x7f3b001a43e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009fd0 tx=0x7f3af800b260 comp rx=0 tx=0).stop 2024-09-19T08:22:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.720+0000 7f3b0824f640 1 -- 172.21.15.22:0/4279611181 shutdown_connections 2024-09-19T08:22:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.720+0000 7f3b0824f640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f3aec0a3430 0x7f3aec0a58f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.720+0000 7f3b0824f640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b00102ef0 0x7f3b001a43e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.720+0000 7f3b0824f640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b00102570 0x7f3b001998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.720+0000 7f3b0824f640 1 --2- 172.21.15.22:0/4279611181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b00100e20 0x7f3b00199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.720+0000 7f3b0824f640 1 -- 172.21.15.22:0/4279611181 >> 172.21.15.22:0/4279611181 conn(0x7f3b000fc7d0 msgr2=0x7f3b000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.720+0000 7f3b0824f640 1 -- 172.21.15.22:0/4279611181 shutdown_connections 2024-09-19T08:22:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:45.720+0000 7f3b0824f640 1 -- 172.21.15.22:0/4279611181 wait complete. 2024-09-19T08:22:46.628 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:46 smithi022 ceph-mon[30963]: pgmap v237: 97 pgs: 97 active+clean; 1015 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 777 op/s 2024-09-19T08:22:46.663 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:22:46.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:22:46.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:46 smithi064 ceph-mon[34900]: pgmap v237: 97 pgs: 97 active+clean; 1015 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 777 op/s 2024-09-19T08:22:46.946 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:22:47.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:46 smithi157 ceph-mon[35143]: pgmap v237: 97 pgs: 97 active+clean; 1015 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 777 op/s 2024-09-19T08:22:47.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.482+0000 7faccdc3f640 1 -- 172.21.15.22:0/3333144364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facc8101de0 msgr2=0x7facc81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:47.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.482+0000 7faccdc3f640 1 --2- 172.21.15.22:0/3333144364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facc8101de0 0x7facc81021c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7facb8009900 tx=0x7facb802a410 comp rx=0 tx=0).stop 2024-09-19T08:22:47.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.482+0000 7faccdc3f640 1 -- 172.21.15.22:0/3333144364 shutdown_connections 2024-09-19T08:22:47.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.482+0000 7faccdc3f640 1 --2- 172.21.15.22:0/3333144364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facc8106f10 0x7facc810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:47.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.482+0000 7faccdc3f640 1 --2- 172.21.15.22:0/3333144364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc8102790 0x7facc81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:47.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.482+0000 7faccdc3f640 1 --2- 172.21.15.22:0/3333144364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facc8101de0 0x7facc81021c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:47.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.482+0000 7faccdc3f640 1 -- 172.21.15.22:0/3333144364 >> 172.21.15.22:0/3333144364 conn(0x7facc80fd8c0 msgr2=0x7facc80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:47.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.482+0000 7faccdc3f640 1 -- 172.21.15.22:0/3333144364 shutdown_connections 2024-09-19T08:22:47.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.482+0000 7faccdc3f640 1 -- 172.21.15.22:0/3333144364 wait complete. 2024-09-19T08:22:47.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.483+0000 7faccdc3f640 1 Processor -- start 2024-09-19T08:22:47.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.483+0000 7faccdc3f640 1 -- start start 2024-09-19T08:22:47.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.483+0000 7faccdc3f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc8101de0 0x7facc8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:47.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.483+0000 7faccdc3f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facc8102790 0x7facc8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:47.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.483+0000 7faccdc3f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facc8106f10 0x7facc819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:47.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.483+0000 7faccdc3f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7facc8111f70 con 0x7facc8102790 2024-09-19T08:22:47.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.483+0000 7faccdc3f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7facc8111df0 con 0x7facc8101de0 2024-09-19T08:22:47.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.483+0000 7faccdc3f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7facc81120f0 con 0x7facc8106f10 2024-09-19T08:22:47.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.483+0000 7facc6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facc8102790 0x7facc8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:47.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.483+0000 7facc6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facc8102790 0x7facc8198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34638/0 (socket says 172.21.15.22:34638) 2024-09-19T08:22:47.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.483+0000 7facc6ffd640 1 -- 172.21.15.22:0/227461624 learned_addr learned my addr 172.21.15.22:0/227461624 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:47.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.483+0000 7facc7fff640 1 --2- 172.21.15.22:0/227461624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facc8106f10 0x7facc819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:47.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.483+0000 7facc77fe640 1 --2- 172.21.15.22:0/227461624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc8101de0 0x7facc8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:47.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.484+0000 7facc6ffd640 1 -- 172.21.15.22:0/227461624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facc8106f10 msgr2=0x7facc819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:47.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.484+0000 7facc6ffd640 1 --2- 172.21.15.22:0/227461624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facc8106f10 0x7facc819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:47.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.484+0000 7facc6ffd640 1 -- 172.21.15.22:0/227461624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc8101de0 msgr2=0x7facc8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:47.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.484+0000 7facc6ffd640 1 --2- 172.21.15.22:0/227461624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc8101de0 0x7facc8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:47.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.484+0000 7facc6ffd640 1 -- 172.21.15.22:0/227461624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc819d3e0 con 0x7facc8102790 2024-09-19T08:22:47.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.484+0000 7facc6ffd640 1 --2- 172.21.15.22:0/227461624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facc8102790 0x7facc8198970 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7facbc00c890 tx=0x7facbc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:47.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.484+0000 7facc4ff9640 1 -- 172.21.15.22:0/227461624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc004220 con 0x7facc8102790 2024-09-19T08:22:47.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.484+0000 7facc4ff9640 1 -- 172.21.15.22:0/227461624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facbc0043c0 con 0x7facc8102790 2024-09-19T08:22:47.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.484+0000 7faccdc3f640 1 -- 172.21.15.22:0/227461624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc819d6d0 con 0x7facc8102790 2024-09-19T08:22:47.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.484+0000 7facc4ff9640 1 -- 172.21.15.22:0/227461624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc0049e0 con 0x7facc8102790 2024-09-19T08:22:47.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.484+0000 7faccdc3f640 1 -- 172.21.15.22:0/227461624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc81a4fb0 con 0x7facc8102790 2024-09-19T08:22:47.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.485+0000 7faccdc3f640 1 -- 172.21.15.22:0/227461624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc81021c0 con 0x7facc8102790 2024-09-19T08:22:47.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.487+0000 7facc4ff9640 1 -- 172.21.15.22:0/227461624 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7facbc0079b0 con 0x7facc8102790 2024-09-19T08:22:47.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.487+0000 7facc4ff9640 1 --2- 172.21.15.22:0/227461624 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7facb00a33b0 0x7facb00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:47.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.487+0000 7facc77fe640 1 --2- 172.21.15.22:0/227461624 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7facb00a33b0 0x7facb00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:47.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.487+0000 7facc4ff9640 1 -- 172.21.15.22:0/227461624 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7facbc0cf930 con 0x7facc8102790 2024-09-19T08:22:47.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.488+0000 7facc77fe640 1 --2- 172.21.15.22:0/227461624 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7facb00a33b0 0x7facb00a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7facb8009230 tx=0x7facb80058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:47.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.493+0000 7facc4ff9640 1 -- 172.21.15.22:0/227461624 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7facbc014030 con 0x7facc8102790 2024-09-19T08:22:47.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.733+0000 7faccdc3f640 1 -- 172.21.15.22:0/227461624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facc802d7c0 con 0x7facc8102790 2024-09-19T08:22:47.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.734+0000 7facc4ff9640 1 -- 172.21.15.22:0/227461624 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11911 (secure 0 0 0) 0x7facbc09bcb0 con 0x7facc8102790 2024-09-19T08:22:47.734 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:22:47.734 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"btime":"2024-09-19T08:22:26:539605+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:22:25.667214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:22:47.734 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 33 2024-09-19T08:22:47.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.736+0000 7faccdc3f640 1 -- 172.21.15.22:0/227461624 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7facb00a33b0 msgr2=0x7facb00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:47.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.736+0000 7faccdc3f640 1 --2- 172.21.15.22:0/227461624 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7facb00a33b0 0x7facb00a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7facb8009230 tx=0x7facb80058d0 comp rx=0 tx=0).stop 2024-09-19T08:22:47.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.736+0000 7faccdc3f640 1 -- 172.21.15.22:0/227461624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facc8102790 msgr2=0x7facc8198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:47.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.736+0000 7faccdc3f640 1 --2- 172.21.15.22:0/227461624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facc8102790 0x7facc8198970 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7facbc00c890 tx=0x7facbc00cd50 comp rx=0 tx=0).stop 2024-09-19T08:22:47.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.736+0000 7faccdc3f640 1 -- 172.21.15.22:0/227461624 shutdown_connections 2024-09-19T08:22:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.736+0000 7faccdc3f640 1 --2- 172.21.15.22:0/227461624 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7facb00a33b0 0x7facb00a5870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.736+0000 7faccdc3f640 1 --2- 172.21.15.22:0/227461624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facc8106f10 0x7facc819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.736+0000 7faccdc3f640 1 --2- 172.21.15.22:0/227461624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7facc8102790 0x7facc8198970 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.736+0000 7faccdc3f640 1 --2- 172.21.15.22:0/227461624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc8101de0 0x7facc8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.736+0000 7faccdc3f640 1 -- 172.21.15.22:0/227461624 >> 172.21.15.22:0/227461624 conn(0x7facc80fd8c0 msgr2=0x7facc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.736+0000 7faccdc3f640 1 -- 172.21.15.22:0/227461624 shutdown_connections 2024-09-19T08:22:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:47.736+0000 7faccdc3f640 1 -- 172.21.15.22:0/227461624 wait complete. 2024-09-19T08:22:48.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:22:48.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:48 smithi064 ceph-mon[34900]: pgmap v238: 97 pgs: 97 active+clean; 1012 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 942 op/s 2024-09-19T08:22:48.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:48 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/227461624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:22:48.927 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:22:49.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:48 smithi157 ceph-mon[35143]: pgmap v238: 97 pgs: 97 active+clean; 1012 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 942 op/s 2024-09-19T08:22:49.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:48 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/227461624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:22:49.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:48 smithi022 ceph-mon[30963]: pgmap v238: 97 pgs: 97 active+clean; 1012 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 942 op/s 2024-09-19T08:22:49.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:48 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/227461624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:22:49.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.405+0000 7f8c8f5e5640 1 -- 172.21.15.22:0/303516227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c8810e480 msgr2=0x7f8c88110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:49.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.405+0000 7f8c8f5e5640 1 --2- 172.21.15.22:0/303516227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c8810e480 0x7f8c88110870 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009ca0 tx=0x7f8c8002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.406+0000 7f8c8f5e5640 1 -- 172.21.15.22:0/303516227 shutdown_connections 2024-09-19T08:22:49.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.406+0000 7f8c8f5e5640 1 --2- 172.21.15.22:0/303516227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c8810e480 0x7f8c88110870 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.406+0000 7f8c8f5e5640 1 --2- 172.21.15.22:0/303516227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c88101280 0x7f8c8810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.406+0000 7f8c8f5e5640 1 --2- 172.21.15.22:0/303516227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c881008d0 0x7f8c88100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.406+0000 7f8c8f5e5640 1 -- 172.21.15.22:0/303516227 >> 172.21.15.22:0/303516227 conn(0x7f8c880fc7a0 msgr2=0x7f8c880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:49.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.406+0000 7f8c8f5e5640 1 -- 172.21.15.22:0/303516227 shutdown_connections 2024-09-19T08:22:49.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.406+0000 7f8c8f5e5640 1 -- 172.21.15.22:0/303516227 wait complete. 2024-09-19T08:22:49.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.406+0000 7f8c8f5e5640 1 Processor -- start 2024-09-19T08:22:49.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.406+0000 7f8c8f5e5640 1 -- start start 2024-09-19T08:22:49.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8f5e5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c881008d0 0x7f8c8819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8f5e5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c88101280 0x7f8c8819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8f5e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8810e480 0x7f8c8819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8f5e5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8c88114180 con 0x7f8c88101280 2024-09-19T08:22:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8f5e5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8c88114000 con 0x7f8c881008d0 2024-09-19T08:22:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8f5e5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c88114300 con 0x7f8c8810e480 2024-09-19T08:22:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8cb80640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c88101280 0x7f8c8819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8cb80640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c88101280 0x7f8c8819abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34652/0 (socket says 172.21.15.22:34652) 2024-09-19T08:22:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8cb80640 1 -- 172.21.15.22:0/2812803122 learned_addr learned my addr 172.21.15.22:0/2812803122 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8d381640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c881008d0 0x7f8c8819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8db82640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8810e480 0x7f8c8819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8cb80640 1 -- 172.21.15.22:0/2812803122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8810e480 msgr2=0x7f8c8819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8cb80640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8810e480 0x7f8c8819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8cb80640 1 -- 172.21.15.22:0/2812803122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c881008d0 msgr2=0x7f8c8819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8cb80640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c881008d0 0x7f8c8819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8cb80640 1 -- 172.21.15.22:0/2812803122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8819f650 con 0x7f8c88101280 2024-09-19T08:22:49.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8d381640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c881008d0 0x7f8c8819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:22:49.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8cb80640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c88101280 0x7f8c8819abe0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00e9f0 tx=0x7f8c7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:49.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c7a7fc640 1 -- 172.21.15.22:0/2812803122 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c00cd80 con 0x7f8c88101280 2024-09-19T08:22:49.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c7a7fc640 1 -- 172.21.15.22:0/2812803122 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c005500 con 0x7f8c88101280 2024-09-19T08:22:49.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c8f5e5640 1 -- 172.21.15.22:0/2812803122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8819f940 con 0x7f8c88101280 2024-09-19T08:22:49.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.407+0000 7f8c7a7fc640 1 -- 172.21.15.22:0/2812803122 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c010690 con 0x7f8c88101280 2024-09-19T08:22:49.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.408+0000 7f8c8f5e5640 1 -- 172.21.15.22:0/2812803122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c881a7220 con 0x7f8c88101280 2024-09-19T08:22:49.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.410+0000 7f8c7a7fc640 1 -- 172.21.15.22:0/2812803122 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8c7c0046c0 con 0x7f8c88101280 2024-09-19T08:22:49.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.410+0000 7f8c919be740 1 Processor -- start 2024-09-19T08:22:49.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.410+0000 7f8c919be740 1 -- start start 2024-09-19T08:22:49.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.410+0000 7f8c919be740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d5b40 0x55e53f2d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.410+0000 7f8c919be740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d45d0 0x55e53f2d4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c919be740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2df370 0x55e53f2e1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c919be740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e53f23a2f0 con 0x55e53f2d45d0 2024-09-19T08:22:49.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c7a7fc640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f8c740a33b0 0x7f8c740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c919be740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e53f1e1cf0 con 0x55e53f2d5b40 2024-09-19T08:22:49.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c919be740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e53f2362b0 con 0x55e53f2df370 2024-09-19T08:22:49.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c8d381640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d45d0 0x55e53f2d4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c8d381640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d45d0 0x55e53f2d4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34662/0 (socket says 172.21.15.22:34662) 2024-09-19T08:22:49.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c8d381640 1 -- 172.21.15.22:0/2990418468 learned_addr learned my addr 172.21.15.22:0/2990418468 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:49.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c8cb80640 1 --2- 172.21.15.22:0/2990418468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2df370 0x55e53f2e1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c8db82640 1 --2- 172.21.15.22:0/2990418468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d5b40 0x55e53f2d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c8db82640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f8c740a33b0 0x7f8c740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c7a7fc640 1 -- 172.21.15.22:0/2812803122 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f8c7c0d0130 con 0x7f8c88101280 2024-09-19T08:22:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c8d381640 1 -- 172.21.15.22:0/2990418468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2df370 msgr2=0x55e53f2e1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c8d381640 1 --2- 172.21.15.22:0/2990418468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2df370 0x55e53f2e1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c8d381640 1 -- 172.21.15.22:0/2990418468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d5b40 msgr2=0x55e53f2d4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c8d381640 1 --2- 172.21.15.22:0/2990418468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d5b40 0x55e53f2d4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c8d381640 1 -- 172.21.15.22:0/2990418468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e53f1c63a0 con 0x55e53f2d45d0 2024-09-19T08:22:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c8d381640 1 --2- 172.21.15.22:0/2990418468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d45d0 0x55e53f2d4a10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8c840115f0 tx=0x7f8c84011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99997ee1f24ebc6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c8db82640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f8c740a33b0 0x7f8c740a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8c80006010 tx=0x7f8c80004a80 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.411+0000 7f8c737fe640 1 -- 172.21.15.22:0/2990418468 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8401e070 con 0x55e53f2d45d0 2024-09-19T08:22:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.412+0000 7f8c737fe640 1 -- 172.21.15.22:0/2990418468 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8400fb20 con 0x55e53f2d45d0 2024-09-19T08:22:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.412+0000 7f8c737fe640 1 -- 172.21.15.22:0/2990418468 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84019bf0 con 0x55e53f2d45d0 2024-09-19T08:22:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.412+0000 7f8c919be740 1 -- 172.21.15.22:0/2990418468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d45d0 msgr2=0x55e53f2d4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.412+0000 7f8c919be740 1 --2- 172.21.15.22:0/2990418468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d45d0 0x55e53f2d4a10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8c840115f0 tx=0x7f8c84011ab0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.412+0000 7f8c919be740 1 -- 172.21.15.22:0/2990418468 shutdown_connections 2024-09-19T08:22:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.412+0000 7f8c919be740 1 --2- 172.21.15.22:0/2990418468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2df370 0x55e53f2e1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.412+0000 7f8c919be740 1 --2- 172.21.15.22:0/2990418468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d45d0 0x55e53f2d4a10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.412+0000 7f8c919be740 1 --2- 172.21.15.22:0/2990418468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d5b40 0x55e53f2d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.412+0000 7f8c919be740 1 -- 172.21.15.22:0/2990418468 >> 172.21.15.22:0/2990418468 conn(0x55e53f29d310 msgr2=0x55e53f2d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.412+0000 7f8c919be740 1 -- 172.21.15.22:0/2990418468 shutdown_connections 2024-09-19T08:22:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.412+0000 7f8c919be740 1 -- 172.21.15.22:0/2990418468 wait complete. 2024-09-19T08:22:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.413+0000 7f8c919be740 1 -- start start 2024-09-19T08:22:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.413+0000 7f8c919be740 1 Processor -- start 2024-09-19T08:22:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.413+0000 7f8c919be740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2d45d0 0x55e53f2ded40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.413+0000 7f8c919be740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d5b40 0x55e53f2dbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.413+0000 7f8c919be740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2df370 0x55e53f2dc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.413+0000 7f8c919be740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e53f2e2070 con 0x55e53f2d5b40 2024-09-19T08:22:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.413+0000 7f8c919be740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e53f1f5060 con 0x55e53f2df370 2024-09-19T08:22:49.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.413+0000 7f8c919be740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e53f2e21f0 con 0x55e53f2d45d0 2024-09-19T08:22:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.413+0000 7f8c919be740 10 client.0 _ll_register_callbacks cb 0x55e53ef488c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:22:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.413+0000 7f8c8cb80640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d5b40 0x55e53f2dbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.413+0000 7f8c8cb80640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d5b40 0x55e53f2dbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34672/0 (socket says 172.21.15.22:34672) 2024-09-19T08:22:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.413+0000 7f8c8d381640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2d45d0 0x55e53f2ded40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.413+0000 7f8c8cb80640 1 -- 172.21.15.22:0/3076577739 learned_addr learned my addr 172.21.15.22:0/3076577739 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.413+0000 7f8c8db82640 1 --2- 172.21.15.22:0/3076577739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2df370 0x55e53f2dc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c8d381640 1 -- 172.21.15.22:0/3076577739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2df370 msgr2=0x55e53f2dc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c8d381640 1 --2- 172.21.15.22:0/3076577739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2df370 0x55e53f2dc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c8d381640 1 -- 172.21.15.22:0/3076577739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d5b40 msgr2=0x55e53f2dbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c8d381640 1 --2- 172.21.15.22:0/3076577739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d5b40 0x55e53f2dbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c8d381640 1 -- 172.21.15.22:0/3076577739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e53efab3c0 con 0x55e53f2d45d0 2024-09-19T08:22:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c8cb80640 1 --2- 172.21.15.22:0/3076577739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d5b40 0x55e53f2dbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:22:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c8d381640 1 --2- 172.21.15.22:0/3076577739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2d45d0 0x55e53f2ded40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8c8401c040 tx=0x7f8c8400f8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2afd5fc89541f8da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:22:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c70ff9640 1 -- 172.21.15.22:0/3076577739 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84029060 con 0x55e53f2d45d0 2024-09-19T08:22:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c70ff9640 1 -- 172.21.15.22:0/3076577739 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f8c84004040 con 0x55e53f2d45d0 2024-09-19T08:22:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c919be740 1 -- 172.21.15.22:0/3076577739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c8400fb20 con 0x55e53f2d45d0 2024-09-19T08:22:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c70ff9640 1 -- 172.21.15.22:0/3076577739 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8c8401e040 con 0x55e53f2d45d0 2024-09-19T08:22:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c919be740 10 client.26173 fetch_fsmap learned FSMap version 33 2024-09-19T08:22:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c919be740 1 -- 172.21.15.22:0/3076577739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e53f2edae0 con 0x55e53f2d45d0 2024-09-19T08:22:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.414+0000 7f8c70ff9640 1 -- 172.21.15.22:0/3076577739 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8400fb20 con 0x55e53f2d45d0 2024-09-19T08:22:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.415+0000 7f8c70ff9640 1 -- 172.21.15.22:0/3076577739 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f8c840035f0 con 0x55e53f2d45d0 2024-09-19T08:22:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.415+0000 7f8c919be740 10 client.26173 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:22:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.415+0000 7f8c919be740 10 client.26173 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:22:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.415+0000 7f8c919be740 20 client.26173 populate_metadata read hostname 'smithi022' 2024-09-19T08:22:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.415+0000 7f8c919be740 1 --2- 172.21.15.22:0/3076577739 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55e53f2eeee0 0x55e53f2f12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.415+0000 7f8c919be740 4 client.26173 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:22:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.415+0000 7f8c919be740 1 -- 172.21.15.22:0/3076577739 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e53f247860 con 0x55e53f2eeee0 2024-09-19T08:22:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.415+0000 7f8c8cb80640 1 --2- 172.21.15.22:0/3076577739 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55e53f2eeee0 0x55e53f2f12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.415+0000 7f8c70ff9640 10 client.26173 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:22:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.415+0000 7f8c8cb80640 1 --2- 172.21.15.22:0/3076577739 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55e53f2eeee0 0x55e53f2f12d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47df6b3e9a774be3 server_cookie=f5044dfd072acf98 in_seq=0 out_seq=0 2024-09-19T08:22:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.427+0000 7f8c70ff9640 1 -- 172.21.15.22:0/3076577739 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e53f247860 con 0x55e53f2eeee0 2024-09-19T08:22:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.427+0000 7f8c70ff9640 10 client.26173 handle_command_reply: tid=0 2024-09-19T08:22:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.427+0000 7f8c919be740 1 client.26173 shutdown 2024-09-19T08:22:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.428+0000 7f8c919be740 1 -- 172.21.15.22:0/3076577739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2d45d0 msgr2=0x55e53f2ded40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.428+0000 7f8c919be740 1 --2- 172.21.15.22:0/3076577739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2d45d0 0x55e53f2ded40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8c8401c040 tx=0x7f8c8400f8f0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.428+0000 7f8c919be740 1 -- 172.21.15.22:0/3076577739 shutdown_connections 2024-09-19T08:22:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.428+0000 7f8c919be740 1 --2- 172.21.15.22:0/3076577739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2df370 0x55e53f2dc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.428+0000 7f8c919be740 1 --2- 172.21.15.22:0/3076577739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d5b40 0x55e53f2dbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.428+0000 7f8c919be740 1 --2- 172.21.15.22:0/3076577739 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55e53f2eeee0 0x55e53f2f12d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.428+0000 7f8c919be740 1 --2- 172.21.15.22:0/3076577739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2d45d0 0x55e53f2ded40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.428+0000 7f8c70ff9640 0 client.26173 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:22:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.428+0000 7f8c919be740 1 -- 172.21.15.22:0/3076577739 >> 172.21.15.22:0/3076577739 conn(0x55e53f29d310 msgr2=0x55e53f2d6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.428+0000 7f8c919be740 1 -- 172.21.15.22:0/3076577739 shutdown_connections 2024-09-19T08:22:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.428+0000 7f8c919be740 1 -- 172.21.15.22:0/3076577739 wait complete. 2024-09-19T08:22:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.428+0000 7f8c919be740 20 client.26173 trim_cache size 0 max 16384 2024-09-19T08:22:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.441+0000 7f8c919be740 1 Processor -- start 2024-09-19T08:22:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.441+0000 7f8c919be740 1 -- start start 2024-09-19T08:22:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.441+0000 7f8c919be740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d45d0 0x55e53f2eb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.441+0000 7f8c919be740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2d5b40 0x55e53f2db8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.441+0000 7f8c919be740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2df370 0x55e53f2d7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.441+0000 7f8c919be740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8c881b2490 con 0x55e53f2df370 2024-09-19T08:22:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.441+0000 7f8c919be740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8c881afcd0 con 0x55e53f2d45d0 2024-09-19T08:22:49.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.441+0000 7f8c919be740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c881adb40 con 0x55e53f2d5b40 2024-09-19T08:22:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.441+0000 7f8c8db82640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2df370 0x55e53f2d7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.441+0000 7f8c8db82640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2df370 0x55e53f2d7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34684/0 (socket says 172.21.15.22:34684) 2024-09-19T08:22:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.441+0000 7f8c8db82640 1 -- 172.21.15.22:0/1005765113 learned_addr learned my addr 172.21.15.22:0/1005765113 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.441+0000 7f8c8d381640 1 --2- 172.21.15.22:0/1005765113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d45d0 0x55e53f2eb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.441+0000 7f8c8cb80640 1 --2- 172.21.15.22:0/1005765113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2d5b40 0x55e53f2db8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.442+0000 7f8c8d381640 1 -- 172.21.15.22:0/1005765113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2d5b40 msgr2=0x55e53f2db8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.442+0000 7f8c8d381640 1 --2- 172.21.15.22:0/1005765113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2d5b40 0x55e53f2db8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.442+0000 7f8c8d381640 1 -- 172.21.15.22:0/1005765113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2df370 msgr2=0x55e53f2d7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.442+0000 7f8c8d381640 1 --2- 172.21.15.22:0/1005765113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2df370 0x55e53f2d7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.442+0000 7f8c8d381640 1 -- 172.21.15.22:0/1005765113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e53f2dc950 con 0x55e53f2d45d0 2024-09-19T08:22:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.442+0000 7f8c8db82640 1 --2- 172.21.15.22:0/1005765113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2df370 0x55e53f2d7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:22:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.442+0000 7f8c8d381640 1 --2- 172.21.15.22:0/1005765113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d45d0 0x55e53f2eb310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8c84011510 tx=0x7f8c84011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eba726b427230fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.442+0000 7f8c72ffd640 1 -- 172.21.15.22:0/1005765113 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84015070 con 0x55e53f2d45d0 2024-09-19T08:22:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.442+0000 7f8c72ffd640 1 -- 172.21.15.22:0/1005765113 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c84019d20 con 0x55e53f2d45d0 2024-09-19T08:22:49.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.442+0000 7f8c72ffd640 1 -- 172.21.15.22:0/1005765113 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84029030 con 0x55e53f2d45d0 2024-09-19T08:22:49.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.443+0000 7f8c919be740 1 -- 172.21.15.22:0/1005765113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d45d0 msgr2=0x55e53f2eb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.443+0000 7f8c919be740 1 --2- 172.21.15.22:0/1005765113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d45d0 0x55e53f2eb310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8c84011510 tx=0x7f8c84011ab0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.443+0000 7f8c919be740 1 -- 172.21.15.22:0/1005765113 shutdown_connections 2024-09-19T08:22:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.443+0000 7f8c919be740 1 --2- 172.21.15.22:0/1005765113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2df370 0x55e53f2d7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.443+0000 7f8c919be740 1 --2- 172.21.15.22:0/1005765113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2d5b40 0x55e53f2db8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.443+0000 7f8c919be740 1 --2- 172.21.15.22:0/1005765113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d45d0 0x55e53f2eb310 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.443+0000 7f8c919be740 1 -- 172.21.15.22:0/1005765113 >> 172.21.15.22:0/1005765113 conn(0x55e53f29d310 msgr2=0x55e53f2cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.443+0000 7f8c919be740 1 -- 172.21.15.22:0/1005765113 shutdown_connections 2024-09-19T08:22:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.443+0000 7f8c919be740 1 -- 172.21.15.22:0/1005765113 wait complete. 2024-09-19T08:22:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.443+0000 7f8c919be740 1 -- start start 2024-09-19T08:22:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.443+0000 7f8c919be740 1 Processor -- start 2024-09-19T08:22:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c919be740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d45d0 0x55e53f2e7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c919be740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d5b40 0x55e53f2e7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c919be740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2df370 0x55e53f2d8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c919be740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8c881ae190 con 0x55e53f2d45d0 2024-09-19T08:22:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c919be740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8c881af600 con 0x55e53f2d5b40 2024-09-19T08:22:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c919be740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c881ae860 con 0x55e53f2df370 2024-09-19T08:22:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c919be740 10 client.0 _ll_register_callbacks cb 0x7f8c8401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:22:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c8d381640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d45d0 0x55e53f2e7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c8d381640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d45d0 0x55e53f2e7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34696/0 (socket says 172.21.15.22:34696) 2024-09-19T08:22:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c8d381640 1 -- 172.21.15.22:0/1609922023 learned_addr learned my addr 172.21.15.22:0/1609922023 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c8db82640 1 --2- 172.21.15.22:0/1609922023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2df370 0x55e53f2d8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c8cb80640 1 --2- 172.21.15.22:0/1609922023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d5b40 0x55e53f2e7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c8d381640 1 -- 172.21.15.22:0/1609922023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2df370 msgr2=0x55e53f2d8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c8d381640 1 --2- 172.21.15.22:0/1609922023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2df370 0x55e53f2d8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c8d381640 1 -- 172.21.15.22:0/1609922023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d5b40 msgr2=0x55e53f2e7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c8d381640 1 --2- 172.21.15.22:0/1609922023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d5b40 0x55e53f2e7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c8d381640 1 -- 172.21.15.22:0/1609922023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e53f2d8bd0 con 0x55e53f2d45d0 2024-09-19T08:22:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c8d381640 1 --2- 172.21.15.22:0/1609922023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d45d0 0x55e53f2e7450 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400fc90 tx=0x7f8c8400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578267010668a9a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:22:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c70ff9640 1 -- 172.21.15.22:0/1609922023 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84029070 con 0x55e53f2d45d0 2024-09-19T08:22:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c70ff9640 1 -- 172.21.15.22:0/1609922023 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f8c84005070 con 0x55e53f2d45d0 2024-09-19T08:22:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.444+0000 7f8c919be740 1 -- 172.21.15.22:0/1609922023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c84019d20 con 0x55e53f2d45d0 2024-09-19T08:22:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.445+0000 7f8c70ff9640 1 -- 172.21.15.22:0/1609922023 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8c84046050 con 0x55e53f2d45d0 2024-09-19T08:22:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.445+0000 7f8c919be740 10 client.16692 fetch_fsmap learned FSMap version 33 2024-09-19T08:22:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.445+0000 7f8c919be740 1 -- 172.21.15.22:0/1609922023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e53f359180 con 0x55e53f2d45d0 2024-09-19T08:22:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.445+0000 7f8c70ff9640 1 -- 172.21.15.22:0/1609922023 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c84019d20 con 0x55e53f2d45d0 2024-09-19T08:22:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.445+0000 7f8c70ff9640 1 -- 172.21.15.22:0/1609922023 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f8c84024760 con 0x55e53f2d45d0 2024-09-19T08:22:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.445+0000 7f8c919be740 10 client.16692 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:22:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.445+0000 7f8c919be740 10 client.16692 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:22:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.445+0000 7f8c919be740 20 client.16692 populate_metadata read hostname 'smithi022' 2024-09-19T08:22:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.445+0000 7f8c919be740 1 --2- 172.21.15.22:0/1609922023 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55e53f359760 0x55e53f35bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.445+0000 7f8c919be740 4 client.16692 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:22:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.445+0000 7f8c919be740 1 -- 172.21.15.22:0/1609922023 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e53f241d70 con 0x55e53f359760 2024-09-19T08:22:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.445+0000 7f8c8cb80640 1 --2- 172.21.15.22:0/1609922023 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55e53f359760 0x55e53f35bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.446+0000 7f8c8cb80640 1 --2- 172.21.15.22:0/1609922023 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55e53f359760 0x55e53f35bb80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=920702bbcc6fa16 server_cookie=81fc1fa217135bb4 in_seq=0 out_seq=0 2024-09-19T08:22:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.446+0000 7f8c70ff9640 10 client.16692 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:22:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.452+0000 7f8c70ff9640 1 -- 172.21.15.22:0/1609922023 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e53f241d70 con 0x55e53f359760 2024-09-19T08:22:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.452+0000 7f8c70ff9640 10 client.16692 handle_command_reply: tid=0 2024-09-19T08:22:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.452+0000 7f8c919be740 1 client.16692 shutdown 2024-09-19T08:22:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.452+0000 7f8c919be740 1 -- 172.21.15.22:0/1609922023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d45d0 msgr2=0x55e53f2e7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.452+0000 7f8c919be740 1 --2- 172.21.15.22:0/1609922023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d45d0 0x55e53f2e7450 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400fc90 tx=0x7f8c8400f3c0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.452+0000 7f8c919be740 1 -- 172.21.15.22:0/1609922023 shutdown_connections 2024-09-19T08:22:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.452+0000 7f8c919be740 1 --2- 172.21.15.22:0/1609922023 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55e53f359760 0x55e53f35bb80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.452+0000 7f8c919be740 1 --2- 172.21.15.22:0/1609922023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e53f2df370 0x55e53f2d8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.452+0000 7f8c919be740 1 --2- 172.21.15.22:0/1609922023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e53f2d5b40 0x55e53f2e7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.452+0000 7f8c919be740 1 --2- 172.21.15.22:0/1609922023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e53f2d45d0 0x55e53f2e7450 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.453+0000 7f8c919be740 1 -- 172.21.15.22:0/1609922023 >> 172.21.15.22:0/1609922023 conn(0x55e53f29d310 msgr2=0x55e53f2cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:49.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.453+0000 7f8c70ff9640 0 client.16692 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:22:49.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.453+0000 7f8c919be740 1 -- 172.21.15.22:0/1609922023 shutdown_connections 2024-09-19T08:22:49.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.453+0000 7f8c919be740 1 -- 172.21.15.22:0/1609922023 wait complete. 2024-09-19T08:22:49.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.453+0000 7f8c919be740 20 client.16692 trim_cache size 0 max 16384 2024-09-19T08:22:49.455 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:22:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.453+0000 7f8c8f5e5640 1 -- 172.21.15.22:0/2812803122 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f8c740a33b0 msgr2=0x7f8c740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.453+0000 7f8c8f5e5640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f8c740a33b0 0x7f8c740a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8c80006010 tx=0x7f8c80004a80 comp rx=0 tx=0).stop 2024-09-19T08:22:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.453+0000 7f8c8f5e5640 1 -- 172.21.15.22:0/2812803122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c88101280 msgr2=0x7f8c8819abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.453+0000 7f8c8f5e5640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c88101280 0x7f8c8819abe0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00e9f0 tx=0x7f8c7c00edf0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.453+0000 7f8c8f5e5640 1 -- 172.21.15.22:0/2812803122 shutdown_connections 2024-09-19T08:22:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.454+0000 7f8c8f5e5640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f8c740a33b0 0x7f8c740a5870 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.454+0000 7f8c8f5e5640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c8810e480 0x7f8c8819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.454+0000 7f8c8f5e5640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c88101280 0x7f8c8819abe0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.454+0000 7f8c8f5e5640 1 --2- 172.21.15.22:0/2812803122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c881008d0 0x7f8c8819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.454+0000 7f8c8f5e5640 1 -- 172.21.15.22:0/2812803122 >> 172.21.15.22:0/2812803122 conn(0x7f8c880fc7a0 msgr2=0x7f8c880feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:49.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.454+0000 7f8c8f5e5640 1 -- 172.21.15.22:0/2812803122 shutdown_connections 2024-09-19T08:22:49.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.454+0000 7f8c8f5e5640 1 -- 172.21.15.22:0/2812803122 wait complete. 2024-09-19T08:22:49.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.782+0000 7ff5bcff9640 1 -- 172.21.15.22:0/1957477493 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55858a319be0 con 0x55858a4346f0 2024-09-19T08:22:49.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.782+0000 7ff5bcff9640 10 client.16650 handle_command_reply: tid=0 2024-09-19T08:22:49.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.782+0000 7ff5dd6de740 1 client.16650 shutdown 2024-09-19T08:22:49.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.782+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1957477493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ae170 msgr2=0x55858a3c0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:49.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.782+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/1957477493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ae170 0x55858a3c0fd0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4031f30 tx=0x7ff5c4009260 comp rx=0 tx=0).stop 2024-09-19T08:22:49.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.782+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1957477493 shutdown_connections 2024-09-19T08:22:49.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.782+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/1957477493 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55858a4346f0 0x55858a436b10 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.782+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/1957477493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55858a3b8a30 0x55858a3bcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.782+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/1957477493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858a3ae170 0x55858a3c0fd0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.782+0000 7ff5bcff9640 0 client.16650 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:49.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.782+0000 7ff5dd6de740 1 --2- 172.21.15.22:0/1957477493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55858a3ad850 0x55858a3c0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.782+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1957477493 >> 172.21.15.22:0/1957477493 conn(0x55858a3a9de0 msgr2=0x55858a3b7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:49.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.782+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1957477493 shutdown_connections 2024-09-19T08:22:49.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.783+0000 7ff5dd6de740 1 -- 172.21.15.22:0/1957477493 wait complete. 2024-09-19T08:22:49.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.783+0000 7ff5dd6de740 20 client.16650 trim_cache size 0 max 16384 2024-09-19T08:22:49.784 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:22:49.784 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 33, 2024-09-19T08:22:49.784 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:22:49.785 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 26, 2024-09-19T08:22:49.785 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:22:49.785 INFO:teuthology.orchestra.run.smithi022.stdout: "25e0fcbe": { 2024-09-19T08:22:49.785 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 26, 2024-09-19T08:22:49.785 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:22:49.785 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:22:49.785 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:22:49.785 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:22:49.785 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:22:49.785 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:22:49.785 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 145.8, 2024-09-19T08:22:49.785 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:22:49.785 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:22:49.786 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:22:49.786 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:22:49.786 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:22:49.786 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:22:49.786 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:22:49.786 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:22:49.786 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:22:49.786 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:22:49.786 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:22:49.786 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:22:49.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.783+0000 7ff5db305640 1 -- 172.21.15.22:0/3606147714 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff5c00a3480 msgr2=0x7ff5c00a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:49.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.783+0000 7ff5db305640 1 --2- 172.21.15.22:0/3606147714 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff5c00a3480 0x7ff5c00a5940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc022000 tx=0x7ff5cc023000 comp rx=0 tx=0).stop 2024-09-19T08:22:49.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.783+0000 7ff5db305640 1 -- 172.21.15.22:0/3606147714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5d4102da0 msgr2=0x7ff5d4100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:49.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.783+0000 7ff5db305640 1 --2- 172.21.15.22:0/3606147714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5d4102da0 0x7ff5d4100a00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000d950 tx=0x7ff5d000de10 comp rx=0 tx=0).stop 2024-09-19T08:22:49.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.784+0000 7ff5db305640 1 -- 172.21.15.22:0/3606147714 shutdown_connections 2024-09-19T08:22:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.784+0000 7ff5db305640 1 --2- 172.21.15.22:0/3606147714 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff5c00a3480 0x7ff5c00a5940 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.784+0000 7ff5db305640 1 --2- 172.21.15.22:0/3606147714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d410c180 0x7ff5d4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.784+0000 7ff5db305640 1 --2- 172.21.15.22:0/3606147714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5d41036c0 0x7ff5d4100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.784+0000 7ff5db305640 1 --2- 172.21.15.22:0/3606147714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5d4102da0 0x7ff5d4100a00 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.784+0000 7ff5db305640 1 -- 172.21.15.22:0/3606147714 >> 172.21.15.22:0/3606147714 conn(0x7ff5d40fc7a0 msgr2=0x7ff5d40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.784+0000 7ff5db305640 1 -- 172.21.15.22:0/3606147714 shutdown_connections 2024-09-19T08:22:49.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:49.784+0000 7ff5db305640 1 -- 172.21.15.22:0/3606147714 wait complete. 2024-09-19T08:22:50.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:50 smithi064 ceph-mon[34900]: pgmap v239: 97 pgs: 97 active+clean; 993 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 1.00k op/s 2024-09-19T08:22:51.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:50 smithi157 ceph-mon[35143]: pgmap v239: 97 pgs: 97 active+clean; 993 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 1.00k op/s 2024-09-19T08:22:51.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:50 smithi022 ceph-mon[30963]: pgmap v239: 97 pgs: 97 active+clean; 993 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 1.00k op/s 2024-09-19T08:22:51.979 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '25e0fcbe', quiesce took 28.0 seconds. Will release after: -2.25 2024-09-19T08:22:51.980 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:22:52.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:22:52.784 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:22:52.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:52 smithi064 ceph-mon[34900]: pgmap v240: 97 pgs: 97 active+clean; 993 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 883 op/s 2024-09-19T08:22:52.909 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:52 smithi022 ceph-mon[30963]: pgmap v240: 97 pgs: 97 active+clean; 993 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 883 op/s 2024-09-19T08:22:53.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:52 smithi157 ceph-mon[35143]: pgmap v240: 97 pgs: 97 active+clean; 993 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 883 op/s 2024-09-19T08:22:53.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.279+0000 7f53a0c39640 1 -- 172.21.15.22:0/110360883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f539c100a80 msgr2=0x7f539c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.279+0000 7f53a0c39640 1 --2- 172.21.15.22:0/110360883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f539c100a80 0x7f539c100ee0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5390009930 tx=0x7f539002a420 comp rx=0 tx=0).stop 2024-09-19T08:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.279+0000 7f53a0c39640 1 -- 172.21.15.22:0/110360883 shutdown_connections 2024-09-19T08:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.279+0000 7f53a0c39640 1 --2- 172.21.15.22:0/110360883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c101420 0x7f539c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.279+0000 7f53a0c39640 1 --2- 172.21.15.22:0/110360883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f539c100a80 0x7f539c100ee0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.279+0000 7f53a0c39640 1 --2- 172.21.15.22:0/110360883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f539c106a80 0x7f539c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.279+0000 7f53a0c39640 1 -- 172.21.15.22:0/110360883 >> 172.21.15.22:0/110360883 conn(0x7f539c0fc7a0 msgr2=0x7f539c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.279+0000 7f53a0c39640 1 -- 172.21.15.22:0/110360883 shutdown_connections 2024-09-19T08:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.279+0000 7f53a0c39640 1 -- 172.21.15.22:0/110360883 wait complete. 2024-09-19T08:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.280+0000 7f53a0c39640 1 Processor -- start 2024-09-19T08:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.280+0000 7f53a0c39640 1 -- start start 2024-09-19T08:22:53.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.280+0000 7f53a0c39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c100a80 0x7f539c1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:53.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.280+0000 7f53a0c39640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f539c101420 0x7f539c197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:53.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.280+0000 7f53a0c39640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f539c106a80 0x7f539c19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:53.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.280+0000 7f53a0c39640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f539c110e80 con 0x7f539c101420 2024-09-19T08:22:53.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.280+0000 7f53a0c39640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f539c110d00 con 0x7f539c100a80 2024-09-19T08:22:53.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.280+0000 7f53a0c39640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f539c111000 con 0x7f539c106a80 2024-09-19T08:22:53.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f5399d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f539c101420 0x7f539c197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:53.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f5399d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f539c101420 0x7f539c197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34718/0 (socket says 172.21.15.22:34718) 2024-09-19T08:22:53.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f5399d9b640 1 -- 172.21.15.22:0/50060924 learned_addr learned my addr 172.21.15.22:0/50060924 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:53.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f539ad9d640 1 --2- 172.21.15.22:0/50060924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f539c106a80 0x7f539c19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:53.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f539a59c640 1 --2- 172.21.15.22:0/50060924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c100a80 0x7f539c1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:53.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f5399d9b640 1 -- 172.21.15.22:0/50060924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f539c106a80 msgr2=0x7f539c19bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:53.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f5399d9b640 1 --2- 172.21.15.22:0/50060924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f539c106a80 0x7f539c19bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:53.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f5399d9b640 1 -- 172.21.15.22:0/50060924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c100a80 msgr2=0x7f539c1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:53.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f5399d9b640 1 --2- 172.21.15.22:0/50060924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c100a80 0x7f539c1973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:53.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f5399d9b640 1 -- 172.21.15.22:0/50060924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c19c370 con 0x7f539c101420 2024-09-19T08:22:53.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f5399d9b640 1 --2- 172.21.15.22:0/50060924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f539c101420 0x7f539c197900 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5390009230 tx=0x7f5390004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:53.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f538b7fe640 1 -- 172.21.15.22:0/50060924 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539003f030 con 0x7f539c101420 2024-09-19T08:22:53.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f538b7fe640 1 -- 172.21.15.22:0/50060924 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5390003ad0 con 0x7f539c101420 2024-09-19T08:22:53.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f53a0c39640 1 -- 172.21.15.22:0/50060924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c19c600 con 0x7f539c101420 2024-09-19T08:22:53.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f538b7fe640 1 -- 172.21.15.22:0/50060924 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390035070 con 0x7f539c101420 2024-09-19T08:22:53.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.281+0000 7f53a0c39640 1 -- 172.21.15.22:0/50060924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c1a3f40 con 0x7f539c101420 2024-09-19T08:22:53.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.282+0000 7f53a0c39640 1 -- 172.21.15.22:0/50060924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c1021c0 con 0x7f539c101420 2024-09-19T08:22:53.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.284+0000 7f538b7fe640 1 -- 172.21.15.22:0/50060924 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5390042030 con 0x7f539c101420 2024-09-19T08:22:53.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.284+0000 7f538b7fe640 1 --2- 172.21.15.22:0/50060924 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f53840a33b0 0x7f53840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:53.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.284+0000 7f538b7fe640 1 -- 172.21.15.22:0/50060924 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f53900eff60 con 0x7f539c101420 2024-09-19T08:22:53.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.284+0000 7f539a59c640 1 --2- 172.21.15.22:0/50060924 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f53840a33b0 0x7f53840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:53.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.285+0000 7f539a59c640 1 --2- 172.21.15.22:0/50060924 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f53840a33b0 0x7f53840a5870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f538c005fd0 tx=0x7f538c00b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:53.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.289+0000 7f538b7fe640 1 -- 172.21.15.22:0/50060924 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f53900bc2e0 con 0x7f539c101420 2024-09-19T08:22:53.396 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:22:53.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.540+0000 7f53a0c39640 1 -- 172.21.15.22:0/50060924 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f539c109020 con 0x7f539c101420 2024-09-19T08:22:53.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.541+0000 7f538b7fe640 1 -- 172.21.15.22:0/50060924 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11911 (secure 0 0 0) 0x7f53900c05e0 con 0x7f539c101420 2024-09-19T08:22:53.541 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:22:53.541 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"btime":"2024-09-19T08:22:26:539605+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:22:25.667214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:22:53.542 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 33 2024-09-19T08:22:53.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.543+0000 7f53a0c39640 1 -- 172.21.15.22:0/50060924 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f53840a33b0 msgr2=0x7f53840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:53.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.543+0000 7f53a0c39640 1 --2- 172.21.15.22:0/50060924 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f53840a33b0 0x7f53840a5870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f538c005fd0 tx=0x7f538c00b390 comp rx=0 tx=0).stop 2024-09-19T08:22:53.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.543+0000 7f53a0c39640 1 -- 172.21.15.22:0/50060924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f539c101420 msgr2=0x7f539c197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:53.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.543+0000 7f53a0c39640 1 --2- 172.21.15.22:0/50060924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f539c101420 0x7f539c197900 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5390009230 tx=0x7f5390004270 comp rx=0 tx=0).stop 2024-09-19T08:22:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.543+0000 7f53a0c39640 1 -- 172.21.15.22:0/50060924 shutdown_connections 2024-09-19T08:22:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.543+0000 7f53a0c39640 1 --2- 172.21.15.22:0/50060924 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f53840a33b0 0x7f53840a5870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.543+0000 7f53a0c39640 1 --2- 172.21.15.22:0/50060924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f539c106a80 0x7f539c19bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.543+0000 7f53a0c39640 1 --2- 172.21.15.22:0/50060924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f539c101420 0x7f539c197900 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.543+0000 7f53a0c39640 1 --2- 172.21.15.22:0/50060924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f539c100a80 0x7f539c1973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.543+0000 7f53a0c39640 1 -- 172.21.15.22:0/50060924 >> 172.21.15.22:0/50060924 conn(0x7f539c0fc7a0 msgr2=0x7f539c0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.543+0000 7f53a0c39640 1 -- 172.21.15.22:0/50060924 shutdown_connections 2024-09-19T08:22:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:53.543+0000 7f53a0c39640 1 -- 172.21.15.22:0/50060924 wait complete. 2024-09-19T08:22:53.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:53 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/50060924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:22:53.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:53 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/50060924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:22:54.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:53 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/50060924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:22:54.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.407+0000 7f8709416640 1 -- 172.21.15.22:0/488187454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87041009b0 msgr2=0x7f8704100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.407+0000 7f8709416640 1 --2- 172.21.15.22:0/488187454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87041009b0 0x7f8704100d90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009900 tx=0x7f86f402a410 comp rx=0 tx=0).stop 2024-09-19T08:22:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.407+0000 7f8709416640 1 -- 172.21.15.22:0/488187454 shutdown_connections 2024-09-19T08:22:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.407+0000 7f8709416640 1 --2- 172.21.15.22:0/488187454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f870410e4d0 0x7f87041108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.407+0000 7f8709416640 1 --2- 172.21.15.22:0/488187454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8704101360 0x7f870410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.407+0000 7f8709416640 1 --2- 172.21.15.22:0/488187454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87041009b0 0x7f8704100d90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.407+0000 7f8709416640 1 -- 172.21.15.22:0/488187454 >> 172.21.15.22:0/488187454 conn(0x7f87040fc7d0 msgr2=0x7f87040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.408+0000 7f8709416640 1 -- 172.21.15.22:0/488187454 shutdown_connections 2024-09-19T08:22:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.408+0000 7f8709416640 1 -- 172.21.15.22:0/488187454 wait complete. 2024-09-19T08:22:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.408+0000 7f8709416640 1 Processor -- start 2024-09-19T08:22:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.408+0000 7f8709416640 1 -- start start 2024-09-19T08:22:54.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f8709416640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87041009b0 0x7f87041a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f8709416640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8704101360 0x7f87041a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f8709416640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870410e4d0 0x7f870419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f8709416640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8704114230 con 0x7f8704101360 2024-09-19T08:22:54.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f8709416640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f87041140b0 con 0x7f870410e4d0 2024-09-19T08:22:54.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f8709416640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87041143b0 con 0x7f87041009b0 2024-09-19T08:22:54.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f87027fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8704101360 0x7f87041a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f87027fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8704101360 0x7f87041a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34744/0 (socket says 172.21.15.22:34744) 2024-09-19T08:22:54.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f87027fc640 1 -- 172.21.15.22:0/1440521438 learned_addr learned my addr 172.21.15.22:0/1440521438 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:54.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f8702ffd640 1 --2- 172.21.15.22:0/1440521438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87041009b0 0x7f87041a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f87037fe640 1 --2- 172.21.15.22:0/1440521438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870410e4d0 0x7f870419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f87027fc640 1 -- 172.21.15.22:0/1440521438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87041009b0 msgr2=0x7f87041a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f87027fc640 1 --2- 172.21.15.22:0/1440521438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87041009b0 0x7f87041a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f87027fc640 1 -- 172.21.15.22:0/1440521438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870410e4d0 msgr2=0x7f870419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f87027fc640 1 --2- 172.21.15.22:0/1440521438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870410e4d0 0x7f870419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f87027fc640 1 -- 172.21.15.22:0/1440521438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870419acf0 con 0x7f8704101360 2024-09-19T08:22:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f87027fc640 1 --2- 172.21.15.22:0/1440521438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8704101360 0x7f87041a0940 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f86f800ea60 tx=0x7f86f800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.409+0000 7f86ebfff640 1 -- 172.21.15.22:0/1440521438 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f800cde0 con 0x7f8704101360 2024-09-19T08:22:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.410+0000 7f86ebfff640 1 -- 172.21.15.22:0/1440521438 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f80035f0 con 0x7f8704101360 2024-09-19T08:22:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.410+0000 7f8709416640 1 -- 172.21.15.22:0/1440521438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870419afe0 con 0x7f8704101360 2024-09-19T08:22:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.410+0000 7f86ebfff640 1 -- 172.21.15.22:0/1440521438 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8010670 con 0x7f8704101360 2024-09-19T08:22:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.410+0000 7f8709416640 1 -- 172.21.15.22:0/1440521438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87041a72d0 con 0x7f8704101360 2024-09-19T08:22:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.412+0000 7f86ebfff640 1 -- 172.21.15.22:0/1440521438 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f86f80037a0 con 0x7f8704101360 2024-09-19T08:22:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.412+0000 7f870b7ef740 1 Processor -- start 2024-09-19T08:22:54.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.412+0000 7f870b7ef740 1 -- start start 2024-09-19T08:22:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.412+0000 7f870b7ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b52063b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.412+0000 7f870b7ef740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b52063ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.412+0000 7f870b7ef740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b520646370 0x55b520648760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.412+0000 7f870b7ef740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b5205a12f0 con 0x55b52063b5d0 2024-09-19T08:22:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.412+0000 7f870b7ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b520548cf0 con 0x55b52063cb40 2024-09-19T08:22:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f870b7ef740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b52059d2b0 con 0x55b520646370 2024-09-19T08:22:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f87037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b52063b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f8702ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b52063ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f87037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b52063b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:45514/0 (socket says 172.21.15.22:45514) 2024-09-19T08:22:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f87037fe640 1 -- 172.21.15.22:0/3531035856 learned_addr learned my addr 172.21.15.22:0/3531035856 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f87027fc640 1 --2- 172.21.15.22:0/3531035856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b520646370 0x55b520648760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f86ebfff640 1 --2- 172.21.15.22:0/1440521438 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f86ec0a33b0 0x7f86ec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f87037fe640 1 -- 172.21.15.22:0/3531035856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b520646370 msgr2=0x55b520648760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:54.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f87037fe640 1 --2- 172.21.15.22:0/3531035856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b520646370 0x55b520648760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f87037fe640 1 -- 172.21.15.22:0/3531035856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 msgr2=0x55b52063ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:54.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f87037fe640 1 --2- 172.21.15.22:0/3531035856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b52063ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f87037fe640 1 -- 172.21.15.22:0/3531035856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b52052d3a0 con 0x55b52063cb40 2024-09-19T08:22:54.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f8702ffd640 1 --2- 172.21.15.22:0/3531035856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b52063ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:22:54.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f86ebfff640 1 -- 172.21.15.22:0/1440521438 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f86f80cffe0 con 0x7f8704101360 2024-09-19T08:22:54.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f87037fe640 1 --2- 172.21.15.22:0/1440521438 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f86ec0a33b0 0x7f86ec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.413+0000 7f87037fe640 1 --2- 172.21.15.22:0/3531035856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b52063b090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009f50 tx=0x7f86fc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aa967d4a51bc8a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:54.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.414+0000 7f86e97fa640 1 -- 172.21.15.22:0/3531035856 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc00f680 con 0x55b52063cb40 2024-09-19T08:22:54.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.414+0000 7f86e97fa640 1 -- 172.21.15.22:0/3531035856 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86fc003cb0 con 0x55b52063cb40 2024-09-19T08:22:54.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.414+0000 7f86e97fa640 1 -- 172.21.15.22:0/3531035856 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc01bb70 con 0x55b52063cb40 2024-09-19T08:22:54.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.414+0000 7f870b7ef740 1 -- 172.21.15.22:0/3531035856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 msgr2=0x55b52063b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:54.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.414+0000 7f87037fe640 1 --2- 172.21.15.22:0/1440521438 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f86ec0a33b0 0x7f86ec0a5870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f86fc012fd0 tx=0x7f86fc021000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:54.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.414+0000 7f870b7ef740 1 --2- 172.21.15.22:0/3531035856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b52063b090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009f50 tx=0x7f86fc0118d0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 -- 172.21.15.22:0/3531035856 shutdown_connections 2024-09-19T08:22:54.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 --2- 172.21.15.22:0/3531035856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b520646370 0x55b520648760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 --2- 172.21.15.22:0/3531035856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b52063ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 --2- 172.21.15.22:0/3531035856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b52063b090 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 -- 172.21.15.22:0/3531035856 >> 172.21.15.22:0/3531035856 conn(0x55b520604310 msgr2=0x55b520637c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:54.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 -- 172.21.15.22:0/3531035856 shutdown_connections 2024-09-19T08:22:54.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 -- 172.21.15.22:0/3531035856 wait complete. 2024-09-19T08:22:54.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 -- start start 2024-09-19T08:22:54.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 Processor -- start 2024-09-19T08:22:54.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b52063b5d0 0x55b520645d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b520642fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b520646370 0x55b520643570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b520649070 con 0x55b520646370 2024-09-19T08:22:54.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b52055c060 con 0x55b52063cb40 2024-09-19T08:22:54.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b5206491f0 con 0x55b52063b5d0 2024-09-19T08:22:54.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.415+0000 7f870b7ef740 10 client.0 _ll_register_callbacks cb 0x55b5202af8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:22:54.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f87037fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b520646370 0x55b520643570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f87027fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b520642fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f87037fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b520646370 0x55b520643570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34760/0 (socket says 172.21.15.22:34760) 2024-09-19T08:22:54.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f87037fe640 1 -- 172.21.15.22:0/460986015 learned_addr learned my addr 172.21.15.22:0/460986015 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:54.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f8702ffd640 1 --2- 172.21.15.22:0/460986015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b52063b5d0 0x55b520645d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f87037fe640 1 -- 172.21.15.22:0/460986015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b52063b5d0 msgr2=0x55b520645d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:54.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f87037fe640 1 --2- 172.21.15.22:0/460986015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b52063b5d0 0x55b520645d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f87037fe640 1 -- 172.21.15.22:0/460986015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 msgr2=0x55b520642fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f87037fe640 1 --2- 172.21.15.22:0/460986015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b520642fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f87037fe640 1 -- 172.21.15.22:0/460986015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5203123c0 con 0x55b520646370 2024-09-19T08:22:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f8702ffd640 1 --2- 172.21.15.22:0/460986015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b52063b5d0 0x55b520645d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:22:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f87027fc640 1 --2- 172.21.15.22:0/460986015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b520642fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:22:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f87037fe640 1 --2- 172.21.15.22:0/460986015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b520646370 0x55b520643570 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f86fc01b470 tx=0x7f86fc00fad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32af43aad64ac7b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f86e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:22:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f86e6ff5640 1 -- 172.21.15.22:0/460986015 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc009be0 con 0x55b520646370 2024-09-19T08:22:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f86e6ff5640 1 -- 172.21.15.22:0/460986015 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f86fc01bac0 con 0x55b520646370 2024-09-19T08:22:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.416+0000 7f870b7ef740 1 -- 172.21.15.22:0/460986015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b5205ae860 con 0x55b520646370 2024-09-19T08:22:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.417+0000 7f86e6ff5640 1 -- 172.21.15.22:0/460986015 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f86fc03d050 con 0x55b520646370 2024-09-19T08:22:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.417+0000 7f870b7ef740 10 client.16710 fetch_fsmap learned FSMap version 33 2024-09-19T08:22:54.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.417+0000 7f870b7ef740 1 -- 172.21.15.22:0/460986015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b520654ac0 con 0x55b520646370 2024-09-19T08:22:54.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.417+0000 7f86e6ff5640 1 -- 172.21.15.22:0/460986015 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b5205ae860 con 0x55b520646370 2024-09-19T08:22:54.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.417+0000 7f86e6ff5640 1 -- 172.21.15.22:0/460986015 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f86fc025c20 con 0x55b520646370 2024-09-19T08:22:54.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.417+0000 7f870b7ef740 10 client.16710 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:22:54.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.417+0000 7f870b7ef740 10 client.16710 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:22:54.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.417+0000 7f870b7ef740 20 client.16710 populate_metadata read hostname 'smithi022' 2024-09-19T08:22:54.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.417+0000 7f870b7ef740 1 --2- 172.21.15.22:0/460986015 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b520655e10 0x55b520658200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.417+0000 7f870b7ef740 4 client.16710 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:22:54.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.417+0000 7f870b7ef740 1 -- 172.21.15.22:0/460986015 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5205a8d70 con 0x55b520655e10 2024-09-19T08:22:54.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.418+0000 7f8702ffd640 1 --2- 172.21.15.22:0/460986015 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b520655e10 0x55b520658200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.418+0000 7f8702ffd640 1 --2- 172.21.15.22:0/460986015 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b520655e10 0x55b520658200 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f30f5bdb9e75ab67 server_cookie=5eaea1fabb5fab46 in_seq=0 out_seq=0 2024-09-19T08:22:54.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.418+0000 7f86e6ff5640 10 client.16710 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:22:54.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.419+0000 7f86e6ff5640 1 -- 172.21.15.22:0/460986015 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5205a8d70 con 0x55b520655e10 2024-09-19T08:22:54.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.419+0000 7f86e6ff5640 10 client.16710 handle_command_reply: tid=0 2024-09-19T08:22:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.419+0000 7f870b7ef740 1 client.16710 shutdown 2024-09-19T08:22:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.419+0000 7f870b7ef740 1 -- 172.21.15.22:0/460986015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b520646370 msgr2=0x55b520643570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.419+0000 7f870b7ef740 1 --2- 172.21.15.22:0/460986015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b520646370 0x55b520643570 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f86fc01b470 tx=0x7f86fc00fad0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.419+0000 7f870b7ef740 1 -- 172.21.15.22:0/460986015 shutdown_connections 2024-09-19T08:22:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.419+0000 7f870b7ef740 1 --2- 172.21.15.22:0/460986015 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b520655e10 0x55b520658200 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.419+0000 7f870b7ef740 1 --2- 172.21.15.22:0/460986015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b520646370 0x55b520643570 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.419+0000 7f870b7ef740 1 --2- 172.21.15.22:0/460986015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b520642fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.419+0000 7f86e6ff5640 0 client.16710 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:22:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.419+0000 7f870b7ef740 1 --2- 172.21.15.22:0/460986015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b52063b5d0 0x55b520645d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.419+0000 7f870b7ef740 1 -- 172.21.15.22:0/460986015 >> 172.21.15.22:0/460986015 conn(0x55b520604310 msgr2=0x55b52063d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.420+0000 7f870b7ef740 1 -- 172.21.15.22:0/460986015 shutdown_connections 2024-09-19T08:22:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.420+0000 7f870b7ef740 1 -- 172.21.15.22:0/460986015 wait complete. 2024-09-19T08:22:54.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.420+0000 7f870b7ef740 20 client.16710 trim_cache size 0 max 16384 2024-09-19T08:22:54.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f870b7ef740 1 Processor -- start 2024-09-19T08:22:54.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f870b7ef740 1 -- start start 2024-09-19T08:22:54.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f870b7ef740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b520652310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f870b7ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b5206428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f870b7ef740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b520646370 0x55b52065d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f870b7ef740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f87041b24b0 con 0x55b52063b5d0 2024-09-19T08:22:54.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f870b7ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f87041afcf0 con 0x55b52063cb40 2024-09-19T08:22:54.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f870b7ef740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87041adb60 con 0x55b520646370 2024-09-19T08:22:54.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f8702ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b520652310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f8702ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b520652310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34762/0 (socket says 172.21.15.22:34762) 2024-09-19T08:22:54.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f8702ffd640 1 -- 172.21.15.22:0/1221081184 learned_addr learned my addr 172.21.15.22:0/1221081184 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:54.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f87037fe640 1 --2- 172.21.15.22:0/1221081184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b520646370 0x55b52065d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f87027fc640 1 --2- 172.21.15.22:0/1221081184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b5206428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f8702ffd640 1 -- 172.21.15.22:0/1221081184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b520646370 msgr2=0x55b52065d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:54.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f8702ffd640 1 --2- 172.21.15.22:0/1221081184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b520646370 0x55b52065d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f8702ffd640 1 -- 172.21.15.22:0/1221081184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 msgr2=0x55b5206428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:54.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.433+0000 7f8702ffd640 1 --2- 172.21.15.22:0/1221081184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b5206428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.434+0000 7f8702ffd640 1 -- 172.21.15.22:0/1221081184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b520646090 con 0x55b52063b5d0 2024-09-19T08:22:54.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.434+0000 7f87027fc640 1 --2- 172.21.15.22:0/1221081184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b5206428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:22:54.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.434+0000 7f8702ffd640 1 --2- 172.21.15.22:0/1221081184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b520652310 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009230 tx=0x7f86f40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bbc20aaa922bd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:54.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.434+0000 7f86e8ff9640 1 -- 172.21.15.22:0/1221081184 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f404a070 con 0x55b52063b5d0 2024-09-19T08:22:54.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.434+0000 7f86e8ff9640 1 -- 172.21.15.22:0/1221081184 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f4005500 con 0x55b52063b5d0 2024-09-19T08:22:54.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.434+0000 7f86e8ff9640 1 -- 172.21.15.22:0/1221081184 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4035040 con 0x55b52063b5d0 2024-09-19T08:22:54.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.435+0000 7f870b7ef740 1 -- 172.21.15.22:0/1221081184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 msgr2=0x55b520652310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:54.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.435+0000 7f870b7ef740 1 --2- 172.21.15.22:0/1221081184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b520652310 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009230 tx=0x7f86f40058d0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.435+0000 7f870b7ef740 1 -- 172.21.15.22:0/1221081184 shutdown_connections 2024-09-19T08:22:54.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.435+0000 7f870b7ef740 1 --2- 172.21.15.22:0/1221081184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b520646370 0x55b52065d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.435+0000 7f870b7ef740 1 --2- 172.21.15.22:0/1221081184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b52063cb40 0x55b5206428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.435+0000 7f870b7ef740 1 --2- 172.21.15.22:0/1221081184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b520652310 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.435+0000 7f870b7ef740 1 -- 172.21.15.22:0/1221081184 >> 172.21.15.22:0/1221081184 conn(0x55b520604310 msgr2=0x55b520645050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:54.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.435+0000 7f870b7ef740 1 -- 172.21.15.22:0/1221081184 shutdown_connections 2024-09-19T08:22:54.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.435+0000 7f870b7ef740 1 -- 172.21.15.22:0/1221081184 wait complete. 2024-09-19T08:22:54.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.435+0000 7f870b7ef740 1 -- start start 2024-09-19T08:22:54.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.435+0000 7f870b7ef740 1 Processor -- start 2024-09-19T08:22:54.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.435+0000 7f870b7ef740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b520649560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.435+0000 7f870b7ef740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b52063cb40 0x55b520649aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.435+0000 7f870b7ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b520646370 0x55b5206be9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f870b7ef740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f87041ae1b0 con 0x55b52063b5d0 2024-09-19T08:22:54.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f870b7ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f87041af620 con 0x55b520646370 2024-09-19T08:22:54.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f870b7ef740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87041ae880 con 0x55b52063cb40 2024-09-19T08:22:54.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f870b7ef740 10 client.0 _ll_register_callbacks cb 0x7f86fc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:22:54.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f8702ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b520649560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f8702ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b520649560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34772/0 (socket says 172.21.15.22:34772) 2024-09-19T08:22:54.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f8702ffd640 1 -- 172.21.15.22:0/1946821565 learned_addr learned my addr 172.21.15.22:0/1946821565 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:54.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f87037fe640 1 --2- 172.21.15.22:0/1946821565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b520646370 0x55b5206be9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f87027fc640 1 --2- 172.21.15.22:0/1946821565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b52063cb40 0x55b520649aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f8702ffd640 1 -- 172.21.15.22:0/1946821565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b52063cb40 msgr2=0x55b520649aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:54.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f8702ffd640 1 --2- 172.21.15.22:0/1946821565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b52063cb40 0x55b520649aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f8702ffd640 1 -- 172.21.15.22:0/1946821565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b520646370 msgr2=0x55b5206be9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:54.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f8702ffd640 1 --2- 172.21.15.22:0/1946821565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b520646370 0x55b5206be9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f8702ffd640 1 -- 172.21.15.22:0/1946821565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b52064a390 con 0x55b52063b5d0 2024-09-19T08:22:54.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f8702ffd640 1 --2- 172.21.15.22:0/1946821565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b520649560 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f86f4004910 tx=0x7f86f40056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede1286efacc0ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:54.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f86e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:22:54.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f86e6ff5640 1 -- 172.21.15.22:0/1946821565 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f404a070 con 0x55b52063b5d0 2024-09-19T08:22:54.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f86e6ff5640 1 -- 172.21.15.22:0/1946821565 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f86f4004040 con 0x55b52063b5d0 2024-09-19T08:22:54.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.436+0000 7f870b7ef740 1 -- 172.21.15.22:0/1946821565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f4005500 con 0x55b52063b5d0 2024-09-19T08:22:54.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.437+0000 7f86e6ff5640 1 -- 172.21.15.22:0/1946821565 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f86f4035040 con 0x55b52063b5d0 2024-09-19T08:22:54.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.437+0000 7f870b7ef740 10 client.16716 fetch_fsmap learned FSMap version 33 2024-09-19T08:22:54.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.437+0000 7f870b7ef740 1 -- 172.21.15.22:0/1946821565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b5206bff80 con 0x55b52063b5d0 2024-09-19T08:22:54.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.437+0000 7f86e6ff5640 1 -- 172.21.15.22:0/1946821565 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f4005500 con 0x55b52063b5d0 2024-09-19T08:22:54.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.437+0000 7f86e6ff5640 1 -- 172.21.15.22:0/1946821565 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f86f40038f0 con 0x55b52063b5d0 2024-09-19T08:22:54.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.437+0000 7f870b7ef740 10 client.16716 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:22:54.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.437+0000 7f870b7ef740 10 client.16716 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:22:54.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.437+0000 7f870b7ef740 20 client.16716 populate_metadata read hostname 'smithi022' 2024-09-19T08:22:54.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.437+0000 7f870b7ef740 1 --2- 172.21.15.22:0/1946821565 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b5206c0670 0x55b5206c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:54.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.437+0000 7f870b7ef740 4 client.16716 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:22:54.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.437+0000 7f870b7ef740 1 -- 172.21.15.22:0/1946821565 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b520525170 con 0x55b5206c0670 2024-09-19T08:22:54.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.437+0000 7f87027fc640 1 --2- 172.21.15.22:0/1946821565 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b5206c0670 0x55b5206c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:54.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f87027fc640 1 --2- 172.21.15.22:0/1946821565 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b5206c0670 0x55b5206c2a90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da377bc778644e7e server_cookie=e4ebf717ebaa303e in_seq=0 out_seq=0 2024-09-19T08:22:54.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f86e6ff5640 10 client.16716 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:22:54.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f86e6ff5640 1 -- 172.21.15.22:0/1946821565 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b520525170 con 0x55b5206c0670 2024-09-19T08:22:54.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f86e6ff5640 10 client.16716 handle_command_reply: tid=0 2024-09-19T08:22:54.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f870b7ef740 1 client.16716 shutdown 2024-09-19T08:22:54.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f870b7ef740 1 -- 172.21.15.22:0/1946821565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 msgr2=0x55b520649560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:54.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f870b7ef740 1 --2- 172.21.15.22:0/1946821565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b520649560 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f86f4004910 tx=0x7f86f40056b0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f870b7ef740 1 -- 172.21.15.22:0/1946821565 shutdown_connections 2024-09-19T08:22:54.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f870b7ef740 1 --2- 172.21.15.22:0/1946821565 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b5206c0670 0x55b5206c2a90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f870b7ef740 1 --2- 172.21.15.22:0/1946821565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b520646370 0x55b5206be9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f870b7ef740 1 --2- 172.21.15.22:0/1946821565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b52063cb40 0x55b520649aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f870b7ef740 1 --2- 172.21.15.22:0/1946821565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52063b5d0 0x55b520649560 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f870b7ef740 1 -- 172.21.15.22:0/1946821565 >> 172.21.15.22:0/1946821565 conn(0x55b520604310 msgr2=0x55b52063d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:54.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f86e6ff5640 0 client.16716 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:22:54.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f870b7ef740 1 -- 172.21.15.22:0/1946821565 shutdown_connections 2024-09-19T08:22:54.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f870b7ef740 1 -- 172.21.15.22:0/1946821565 wait complete. 2024-09-19T08:22:54.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.438+0000 7f870b7ef740 20 client.16716 trim_cache size 0 max 16384 2024-09-19T08:22:54.445 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:22:54.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.439+0000 7f8709416640 1 -- 172.21.15.22:0/1440521438 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f86ec0a33b0 msgr2=0x7f86ec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:54.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.439+0000 7f8709416640 1 --2- 172.21.15.22:0/1440521438 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f86ec0a33b0 0x7f86ec0a5870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f86fc012fd0 tx=0x7f86fc021000 comp rx=0 tx=0).stop 2024-09-19T08:22:54.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.439+0000 7f8709416640 1 -- 172.21.15.22:0/1440521438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8704101360 msgr2=0x7f87041a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:54.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.439+0000 7f8709416640 1 --2- 172.21.15.22:0/1440521438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8704101360 0x7f87041a0940 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f86f800ea60 tx=0x7f86f800ef20 comp rx=0 tx=0).stop 2024-09-19T08:22:54.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.439+0000 7f8709416640 1 -- 172.21.15.22:0/1440521438 shutdown_connections 2024-09-19T08:22:54.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.439+0000 7f8709416640 1 --2- 172.21.15.22:0/1440521438 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f86ec0a33b0 0x7f86ec0a5870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.439+0000 7f8709416640 1 --2- 172.21.15.22:0/1440521438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870410e4d0 0x7f870419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.439+0000 7f8709416640 1 --2- 172.21.15.22:0/1440521438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8704101360 0x7f87041a0940 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.439+0000 7f8709416640 1 --2- 172.21.15.22:0/1440521438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87041009b0 0x7f87041a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:54.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.439+0000 7f8709416640 1 -- 172.21.15.22:0/1440521438 >> 172.21.15.22:0/1440521438 conn(0x7f87040fc7d0 msgr2=0x7f87040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:54.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.439+0000 7f8709416640 1 -- 172.21.15.22:0/1440521438 shutdown_connections 2024-09-19T08:22:54.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:54.439+0000 7f8709416640 1 -- 172.21.15.22:0/1440521438 wait complete. 2024-09-19T08:22:54.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:54 smithi064 ceph-mon[34900]: pgmap v241: 97 pgs: 97 active+clean; 998 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 1.03k op/s 2024-09-19T08:22:55.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:54 smithi157 ceph-mon[35143]: pgmap v241: 97 pgs: 97 active+clean; 998 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 1.03k op/s 2024-09-19T08:22:55.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:54 smithi022 ceph-mon[30963]: pgmap v241: 97 pgs: 97 active+clean; 998 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 1.03k op/s 2024-09-19T08:22:55.321 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id 25e0fcbe --release --await' 2024-09-19T08:22:55.321 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id 25e0fcbe --release --await 2024-09-19T08:22:55.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:22:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:22:56.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.425+0000 7f03326b2640 1 -- 172.21.15.22:0/3870441579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f032c106a80 msgr2=0x7f032c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.425+0000 7f03326b2640 1 --2- 172.21.15.22:0/3870441579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f032c106a80 0x7f032c106e60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f031c009930 tx=0x7f031c02a420 comp rx=0 tx=0).stop 2024-09-19T08:22:56.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.425+0000 7f03326b2640 1 -- 172.21.15.22:0/3870441579 shutdown_connections 2024-09-19T08:22:56.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.425+0000 7f03326b2640 1 --2- 172.21.15.22:0/3870441579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c101420 0x7f032c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.425+0000 7f03326b2640 1 --2- 172.21.15.22:0/3870441579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c100a80 0x7f032c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.425+0000 7f03326b2640 1 --2- 172.21.15.22:0/3870441579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f032c106a80 0x7f032c106e60 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.425+0000 7f03326b2640 1 -- 172.21.15.22:0/3870441579 >> 172.21.15.22:0/3870441579 conn(0x7f032c0fc7a0 msgr2=0x7f032c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:56.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.425+0000 7f03326b2640 1 -- 172.21.15.22:0/3870441579 shutdown_connections 2024-09-19T08:22:56.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.425+0000 7f03326b2640 1 -- 172.21.15.22:0/3870441579 wait complete. 2024-09-19T08:22:56.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.426+0000 7f03326b2640 1 Processor -- start 2024-09-19T08:22:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.426+0000 7f03326b2640 1 -- start start 2024-09-19T08:22:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.426+0000 7f03326b2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c100a80 0x7f032c109380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.426+0000 7f03326b2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c101420 0x7f032c1098c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.426+0000 7f03326b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f032c106a80 0x7f032c1a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.426+0000 7f03326b2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f032c10fd70 con 0x7f032c106a80 2024-09-19T08:22:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.426+0000 7f03326b2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f032c10fbf0 con 0x7f032c100a80 2024-09-19T08:22:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.426+0000 7f03326b2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f032c10fef0 con 0x7f032c101420 2024-09-19T08:22:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.426+0000 7f0330c4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f032c106a80 0x7f032c1a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.426+0000 7f0330c4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f032c106a80 0x7f032c1a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34796/0 (socket says 172.21.15.22:34796) 2024-09-19T08:22:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.426+0000 7f0330c4f640 1 -- 172.21.15.22:0/2202651669 learned_addr learned my addr 172.21.15.22:0/2202651669 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.426+0000 7f032bfff640 1 --2- 172.21.15.22:0/2202651669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c100a80 0x7f032c109380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.427+0000 7f032b7fe640 1 --2- 172.21.15.22:0/2202651669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c101420 0x7f032c1098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.427+0000 7f0330c4f640 1 -- 172.21.15.22:0/2202651669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c101420 msgr2=0x7f032c1098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.427+0000 7f0330c4f640 1 --2- 172.21.15.22:0/2202651669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c101420 0x7f032c1098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.427+0000 7f0330c4f640 1 -- 172.21.15.22:0/2202651669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c100a80 msgr2=0x7f032c109380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.427+0000 7f0330c4f640 1 --2- 172.21.15.22:0/2202651669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c100a80 0x7f032c109380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.427+0000 7f0330c4f640 1 -- 172.21.15.22:0/2202651669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c1a4cf0 con 0x7f032c106a80 2024-09-19T08:22:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.427+0000 7f0330c4f640 1 --2- 172.21.15.22:0/2202651669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f032c106a80 0x7f032c1a45f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f032400cad0 tx=0x7f032400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.427+0000 7f03297fa640 1 -- 172.21.15.22:0/2202651669 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032400ae20 con 0x7f032c106a80 2024-09-19T08:22:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.427+0000 7f03326b2640 1 -- 172.21.15.22:0/2202651669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032c1a4fe0 con 0x7f032c106a80 2024-09-19T08:22:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.427+0000 7f03297fa640 1 -- 172.21.15.22:0/2202651669 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0324005020 con 0x7f032c106a80 2024-09-19T08:22:56.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.427+0000 7f03297fa640 1 -- 172.21.15.22:0/2202651669 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032400f6b0 con 0x7f032c106a80 2024-09-19T08:22:56.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.427+0000 7f03326b2640 1 -- 172.21.15.22:0/2202651669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032c1ab6a0 con 0x7f032c106a80 2024-09-19T08:22:56.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.430+0000 7f0334a8b740 1 Processor -- start 2024-09-19T08:22:56.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.430+0000 7f03297fa640 1 -- 172.21.15.22:0/2202651669 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f03240051d0 con 0x7f032c106a80 2024-09-19T08:22:56.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.430+0000 7f0334a8b740 1 -- start start 2024-09-19T08:22:56.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f0334a8b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff060 0x5655355ff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f0334a8b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff980 0x565535609d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f0334a8b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56553560a240 0x56553560c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f0334a8b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565535564620 con 0x5655355ff060 2024-09-19T08:22:56.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f0334a8b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56553550be50 con 0x5655355ff980 2024-09-19T08:22:56.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f0334a8b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5655355605e0 con 0x56553560a240 2024-09-19T08:22:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff060 0x5655355ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff060 0x5655355ff440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34812/0 (socket says 172.21.15.22:34812) 2024-09-19T08:22:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f032b7fe640 1 -- 172.21.15.22:0/296158532 learned_addr learned my addr 172.21.15.22:0/296158532 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f032bfff640 1 --2- 172.21.15.22:0/296158532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff980 0x565535609d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f032b7fe640 1 --2- 172.21.15.22:0/296158532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56553560a240 0x56553560c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f03297fa640 1 --2- 172.21.15.22:0/2202651669 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f03140a33b0 0x7f03140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f032b7fe640 1 -- 172.21.15.22:0/296158532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56553560a240 msgr2=0x56553560c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f032b7fe640 1 --2- 172.21.15.22:0/296158532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56553560a240 0x56553560c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f032b7fe640 1 -- 172.21.15.22:0/296158532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff980 msgr2=0x565535609d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f032b7fe640 1 --2- 172.21.15.22:0/296158532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff980 0x565535609d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f032b7fe640 1 -- 172.21.15.22:0/296158532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655354fcd40 con 0x5655355ff060 2024-09-19T08:22:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f0330c4f640 1 --2- 172.21.15.22:0/2202651669 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f03140a33b0 0x7f03140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.431+0000 7f03297fa640 1 -- 172.21.15.22:0/2202651669 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f03240d0010 con 0x7f032c106a80 2024-09-19T08:22:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.432+0000 7f032b7fe640 1 --2- 172.21.15.22:0/296158532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff060 0x5655355ff440 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0320004650 tx=0x7f03200098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840b76a48908cc0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.432+0000 7f03127fc640 1 -- 172.21.15.22:0/296158532 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320012650 con 0x5655355ff060 2024-09-19T08:22:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.432+0000 7f03127fc640 1 -- 172.21.15.22:0/296158532 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0320019030 con 0x5655355ff060 2024-09-19T08:22:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.432+0000 7f03127fc640 1 -- 172.21.15.22:0/296158532 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032000fa80 con 0x5655355ff060 2024-09-19T08:22:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.433+0000 7f0330c4f640 1 --2- 172.21.15.22:0/2202651669 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f03140a33b0 0x7f03140a5870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f03240cd800 tx=0x7f0324005b40 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.433+0000 7f0334a8b740 1 -- 172.21.15.22:0/296158532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff060 msgr2=0x5655355ff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.433+0000 7f0334a8b740 1 --2- 172.21.15.22:0/296158532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff060 0x5655355ff440 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0320004650 tx=0x7f03200098d0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.433+0000 7f0334a8b740 1 -- 172.21.15.22:0/296158532 shutdown_connections 2024-09-19T08:22:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.433+0000 7f0334a8b740 1 --2- 172.21.15.22:0/296158532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56553560a240 0x56553560c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.433+0000 7f0334a8b740 1 --2- 172.21.15.22:0/296158532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff980 0x565535609d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.433+0000 7f0334a8b740 1 --2- 172.21.15.22:0/296158532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff060 0x5655355ff440 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.433+0000 7f0334a8b740 1 -- 172.21.15.22:0/296158532 >> 172.21.15.22:0/296158532 conn(0x5655355fb5f0 msgr2=0x5655355fb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.433+0000 7f0334a8b740 1 -- 172.21.15.22:0/296158532 shutdown_connections 2024-09-19T08:22:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.433+0000 7f0334a8b740 1 -- 172.21.15.22:0/296158532 wait complete. 2024-09-19T08:22:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.433+0000 7f0334a8b740 1 -- start start 2024-09-19T08:22:56.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.433+0000 7f0334a8b740 1 Processor -- start 2024-09-19T08:22:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f0334a8b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff060 0x565535607170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f0334a8b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff980 0x5655356076b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f0334a8b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56553560a240 0x565535607bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f0334a8b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56553560ce10 con 0x5655355ff980 2024-09-19T08:22:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f0334a8b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5655354ecf10 con 0x5655355ff060 2024-09-19T08:22:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f0334a8b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56553560cf90 con 0x56553560a240 2024-09-19T08:22:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f0334a8b740 10 client.0 _ll_register_callbacks cb 0x565535510db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:22:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f032bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff980 0x5655356076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f032bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff980 0x5655356076b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34824/0 (socket says 172.21.15.22:34824) 2024-09-19T08:22:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f032bfff640 1 -- 172.21.15.22:0/3036957525 learned_addr learned my addr 172.21.15.22:0/3036957525 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f032b7fe640 1 --2- 172.21.15.22:0/3036957525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff060 0x565535607170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f032b7fe640 1 --2- 172.21.15.22:0/3036957525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56553560a240 0x565535607bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f032bfff640 1 -- 172.21.15.22:0/3036957525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56553560a240 msgr2=0x565535607bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f032bfff640 1 --2- 172.21.15.22:0/3036957525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56553560a240 0x565535607bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f032bfff640 1 -- 172.21.15.22:0/3036957525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff060 msgr2=0x565535607170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f032bfff640 1 --2- 172.21.15.22:0/3036957525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff060 0x565535607170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f032bfff640 1 -- 172.21.15.22:0/3036957525 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565535273580 con 0x5655355ff980 2024-09-19T08:22:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f032b7fe640 1 --2- 172.21.15.22:0/3036957525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56553560a240 0x565535607bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:22:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f032bfff640 1 --2- 172.21.15.22:0/3036957525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff980 0x5655356076b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f031c033000 tx=0x7f031c004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4f5b688ab8ad76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.434+0000 7f030b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:22:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.435+0000 7f030b7fe640 1 -- 172.21.15.22:0/3036957525 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c035020 con 0x5655355ff980 2024-09-19T08:22:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.435+0000 7f030b7fe640 1 -- 172.21.15.22:0/3036957525 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f031c005020 con 0x5655355ff980 2024-09-19T08:22:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.435+0000 7f0334a8b740 1 -- 172.21.15.22:0/3036957525 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0320019030 con 0x5655355ff980 2024-09-19T08:22:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.435+0000 7f030b7fe640 1 -- 172.21.15.22:0/3036957525 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f031c0314d0 con 0x5655355ff980 2024-09-19T08:22:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.435+0000 7f0334a8b740 10 client.16728 fetch_fsmap learned FSMap version 33 2024-09-19T08:22:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.435+0000 7f0334a8b740 1 -- 172.21.15.22:0/3036957525 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56553533d960 con 0x5655355ff980 2024-09-19T08:22:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.435+0000 7f030b7fe640 1 -- 172.21.15.22:0/3036957525 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0320019030 con 0x5655355ff980 2024-09-19T08:22:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.435+0000 7f030b7fe640 1 -- 172.21.15.22:0/3036957525 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f031c03a040 con 0x5655355ff980 2024-09-19T08:22:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.435+0000 7f0334a8b740 10 client.16728 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:22:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.435+0000 7f0334a8b740 10 client.16728 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:22:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.435+0000 7f0334a8b740 20 client.16728 populate_metadata read hostname 'smithi022' 2024-09-19T08:22:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.435+0000 7f0334a8b740 1 --2- 172.21.15.22:0/3036957525 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x565535619e00 0x56553561c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.435+0000 7f0334a8b740 4 client.16728 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:22:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.435+0000 7f0334a8b740 1 -- 172.21.15.22:0/3036957525 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565535571b90 con 0x565535619e00 2024-09-19T08:22:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.436+0000 7f032b7fe640 1 --2- 172.21.15.22:0/3036957525 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x565535619e00 0x56553561c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.436+0000 7f032b7fe640 1 --2- 172.21.15.22:0/3036957525 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x565535619e00 0x56553561c260 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d74e8812ec587bd5 server_cookie=4a70841f9823a28f in_seq=0 out_seq=0 2024-09-19T08:22:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.436+0000 7f030b7fe640 10 client.16728 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.437+0000 7f030b7fe640 1 -- 172.21.15.22:0/3036957525 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565535571b90 con 0x565535619e00 2024-09-19T08:22:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.437+0000 7f030b7fe640 10 client.16728 handle_command_reply: tid=0 2024-09-19T08:22:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.437+0000 7f0334a8b740 1 client.16728 shutdown 2024-09-19T08:22:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.437+0000 7f0334a8b740 1 -- 172.21.15.22:0/3036957525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff980 msgr2=0x5655356076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.437+0000 7f0334a8b740 1 --2- 172.21.15.22:0/3036957525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff980 0x5655356076b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f031c033000 tx=0x7f031c004250 comp rx=0 tx=0).stop 2024-09-19T08:22:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.438+0000 7f0334a8b740 1 -- 172.21.15.22:0/3036957525 shutdown_connections 2024-09-19T08:22:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.438+0000 7f0334a8b740 1 --2- 172.21.15.22:0/3036957525 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x565535619e00 0x56553561c260 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.438+0000 7f0334a8b740 1 --2- 172.21.15.22:0/3036957525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56553560a240 0x565535607bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.438+0000 7f0334a8b740 1 --2- 172.21.15.22:0/3036957525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff980 0x5655356076b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.438+0000 7f0334a8b740 1 --2- 172.21.15.22:0/3036957525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff060 0x565535607170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.438+0000 7f0334a8b740 1 -- 172.21.15.22:0/3036957525 >> 172.21.15.22:0/3036957525 conn(0x5655355fb5f0 msgr2=0x565535601320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.438+0000 7f030b7fe640 0 client.16728 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.438+0000 7f0334a8b740 1 -- 172.21.15.22:0/3036957525 shutdown_connections 2024-09-19T08:22:56.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.438+0000 7f0334a8b740 1 -- 172.21.15.22:0/3036957525 wait complete. 2024-09-19T08:22:56.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.438+0000 7f0334a8b740 20 client.16728 trim_cache size 0 max 16384 2024-09-19T08:22:56.446 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:22:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.455+0000 7f0334a8b740 1 Processor -- start 2024-09-19T08:22:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.455+0000 7f0334a8b740 1 -- start start 2024-09-19T08:22:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f0334a8b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655355ff060 0x5655355fb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f0334a8b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff980 0x565535601ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f0334a8b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56553560a240 0x5655355fa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f0334a8b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b6940 con 0x56553560a240 2024-09-19T08:22:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f0334a8b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b4180 con 0x5655355ff980 2024-09-19T08:22:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f0334a8b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b1ff0 con 0x5655355ff060 2024-09-19T08:22:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56553560a240 0x5655355fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56553560a240 0x5655355fa640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34828/0 (socket says 172.21.15.22:34828) 2024-09-19T08:22:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f032bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff980 0x565535601ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f032b7fe640 1 -- 172.21.15.22:0/1009770311 learned_addr learned my addr 172.21.15.22:0/1009770311 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f032b7fe640 1 --2- 172.21.15.22:0/1009770311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655355ff060 0x5655355fb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f032bfff640 1 -- 172.21.15.22:0/1009770311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655355ff060 msgr2=0x5655355fb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f032bfff640 1 --2- 172.21.15.22:0/1009770311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655355ff060 0x5655355fb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f032bfff640 1 -- 172.21.15.22:0/1009770311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56553560a240 msgr2=0x5655355fa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f032bfff640 1 --2- 172.21.15.22:0/1009770311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56553560a240 0x5655355fa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f032bfff640 1 -- 172.21.15.22:0/1009770311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655356090b0 con 0x5655355ff980 2024-09-19T08:22:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.456+0000 7f032b7fe640 1 --2- 172.21.15.22:0/1009770311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56553560a240 0x5655355fa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:22:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.457+0000 7f032bfff640 1 --2- 172.21.15.22:0/1009770311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff980 0x565535601ac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f031c005ec0 tx=0x7f031c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25928dc85aeafda6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.457+0000 7f0311ffb640 1 -- 172.21.15.22:0/1009770311 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c054030 con 0x5655355ff980 2024-09-19T08:22:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.457+0000 7f0311ffb640 1 -- 172.21.15.22:0/1009770311 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f031c0038f0 con 0x5655355ff980 2024-09-19T08:22:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.457+0000 7f0311ffb640 1 -- 172.21.15.22:0/1009770311 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c035040 con 0x5655355ff980 2024-09-19T08:22:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.458+0000 7f0334a8b740 1 -- 172.21.15.22:0/1009770311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff980 msgr2=0x565535601ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.458+0000 7f0334a8b740 1 --2- 172.21.15.22:0/1009770311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff980 0x565535601ac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f031c005ec0 tx=0x7f031c005fd0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.458+0000 7f0334a8b740 1 -- 172.21.15.22:0/1009770311 shutdown_connections 2024-09-19T08:22:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.458+0000 7f0334a8b740 1 --2- 172.21.15.22:0/1009770311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56553560a240 0x5655355fa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.458+0000 7f0334a8b740 1 --2- 172.21.15.22:0/1009770311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5655355ff980 0x565535601ac0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.458+0000 7f0334a8b740 1 --2- 172.21.15.22:0/1009770311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655355ff060 0x5655355fb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.458+0000 7f0334a8b740 1 -- 172.21.15.22:0/1009770311 >> 172.21.15.22:0/1009770311 conn(0x5655355fb5f0 msgr2=0x5655355f9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.458+0000 7f0334a8b740 1 -- 172.21.15.22:0/1009770311 shutdown_connections 2024-09-19T08:22:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.458+0000 7f0334a8b740 1 -- 172.21.15.22:0/1009770311 wait complete. 2024-09-19T08:22:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.458+0000 7f0334a8b740 1 -- start start 2024-09-19T08:22:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.458+0000 7f0334a8b740 1 Processor -- start 2024-09-19T08:22:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f0334a8b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff060 0x5655356121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f0334a8b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655355ff980 0x565535612720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f0334a8b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56553560a240 0x56553560f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f0334a8b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b2640 con 0x5655355ff060 2024-09-19T08:22:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f0334a8b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b3ab0 con 0x56553560a240 2024-09-19T08:22:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f0334a8b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b2d10 con 0x5655355ff980 2024-09-19T08:22:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f0334a8b740 10 client.0 _ll_register_callbacks cb 0x7f031c005700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:22:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff060 0x5655356121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff060 0x5655356121e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34836/0 (socket says 172.21.15.22:34836) 2024-09-19T08:22:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f032bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655355ff980 0x565535612720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f032b7fe640 1 -- 172.21.15.22:0/4121703888 learned_addr learned my addr 172.21.15.22:0/4121703888 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f032b7fe640 1 --2- 172.21.15.22:0/4121703888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56553560a240 0x56553560f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f032bfff640 1 -- 172.21.15.22:0/4121703888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56553560a240 msgr2=0x56553560f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f032bfff640 1 --2- 172.21.15.22:0/4121703888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56553560a240 0x56553560f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f032bfff640 1 -- 172.21.15.22:0/4121703888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff060 msgr2=0x5655356121e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f032bfff640 1 --2- 172.21.15.22:0/4121703888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff060 0x5655356121e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f032bfff640 1 -- 172.21.15.22:0/4121703888 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565535612d90 con 0x5655355ff980 2024-09-19T08:22:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.459+0000 7f032b7fe640 1 --2- 172.21.15.22:0/4121703888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff060 0x5655356121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:22:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.460+0000 7f032bfff640 1 --2- 172.21.15.22:0/4121703888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655355ff980 0x565535612720 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x5655355070e0 tx=0x7f031c003d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe2e2e471b0bce08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.460+0000 7f030b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:22:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.460+0000 7f030b7fe640 1 -- 172.21.15.22:0/4121703888 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c05e060 con 0x5655355ff980 2024-09-19T08:22:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.460+0000 7f030b7fe640 1 -- 172.21.15.22:0/4121703888 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f031c0035f0 con 0x5655355ff980 2024-09-19T08:22:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.460+0000 7f0334a8b740 1 -- 172.21.15.22:0/4121703888 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f031c0038f0 con 0x5655355ff980 2024-09-19T08:22:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.460+0000 7f030b7fe640 1 -- 172.21.15.22:0/4121703888 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f031c054070 con 0x5655355ff980 2024-09-19T08:22:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.460+0000 7f0334a8b740 10 client.26212 fetch_fsmap learned FSMap version 33 2024-09-19T08:22:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.460+0000 7f0334a8b740 1 -- 172.21.15.22:0/4121703888 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5655356850a0 con 0x5655355ff980 2024-09-19T08:22:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.460+0000 7f030b7fe640 1 -- 172.21.15.22:0/4121703888 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f031c0038f0 con 0x5655355ff980 2024-09-19T08:22:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.461+0000 7f030b7fe640 1 -- 172.21.15.22:0/4121703888 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f031c047580 con 0x5655355ff980 2024-09-19T08:22:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.461+0000 7f0334a8b740 10 client.26212 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:22:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.461+0000 7f0334a8b740 10 client.26212 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:22:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.461+0000 7f0334a8b740 20 client.26212 populate_metadata read hostname 'smithi022' 2024-09-19T08:22:56.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.461+0000 7f0334a8b740 1 --2- 172.21.15.22:0/4121703888 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5655356856b0 0x565535687ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.461+0000 7f0334a8b740 4 client.26212 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "25e0fcbe", "release": true, "await": true}] 2024-09-19T08:22:56.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.461+0000 7f0334a8b740 1 -- 172.21.15.22:0/4121703888 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "set_id": "25e0fcbe", "release": true, "await": true}) -- 0x56553556c0a0 con 0x5655356856b0 2024-09-19T08:22:56.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.461+0000 7f032b7fe640 1 --2- 172.21.15.22:0/4121703888 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5655356856b0 0x565535687ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.461+0000 7f032b7fe640 1 --2- 172.21.15.22:0/4121703888 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5655356856b0 0x565535687ad0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=706e47b16e924961 server_cookie=ee5836da09952fa2 in_seq=0 out_seq=0 2024-09-19T08:22:56.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.461+0000 7f030b7fe640 10 client.26212 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:56.710 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:56 smithi022 ceph-mon[30963]: pgmap v242: 97 pgs: 97 active+clean; 998 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 557 op/s 2024-09-19T08:22:56.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.938+0000 7f82e2424640 1 -- 172.21.15.22:0/3629666160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82dc102790 msgr2=0x7f82dc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.938+0000 7f82e2424640 1 --2- 172.21.15.22:0/3629666160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82dc102790 0x7f82dc1067e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009900 tx=0x7f82d002a410 comp rx=0 tx=0).stop 2024-09-19T08:22:56.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.938+0000 7f82e2424640 1 -- 172.21.15.22:0/3629666160 shutdown_connections 2024-09-19T08:22:56.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.938+0000 7f82e2424640 1 --2- 172.21.15.22:0/3629666160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82dc106f10 0x7f82dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.938+0000 7f82e2424640 1 --2- 172.21.15.22:0/3629666160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82dc102790 0x7f82dc1067e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.938+0000 7f82e2424640 1 --2- 172.21.15.22:0/3629666160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82dc101de0 0x7f82dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.938+0000 7f82e2424640 1 -- 172.21.15.22:0/3629666160 >> 172.21.15.22:0/3629666160 conn(0x7f82dc0fd8c0 msgr2=0x7f82dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.938+0000 7f82e2424640 1 -- 172.21.15.22:0/3629666160 shutdown_connections 2024-09-19T08:22:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.938+0000 7f82e2424640 1 -- 172.21.15.22:0/3629666160 wait complete. 2024-09-19T08:22:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.939+0000 7f82e2424640 1 Processor -- start 2024-09-19T08:22:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.939+0000 7f82e2424640 1 -- start start 2024-09-19T08:22:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.939+0000 7f82e2424640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82dc101de0 0x7f82dc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.939+0000 7f82e2424640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82dc102790 0x7f82dc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.939+0000 7f82e2424640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82dc106f10 0x7f82dc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.939+0000 7f82e2424640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc111f70 con 0x7f82dc106f10 2024-09-19T08:22:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.939+0000 7f82e2424640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc111df0 con 0x7f82dc102790 2024-09-19T08:22:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.939+0000 7f82e2424640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1120f0 con 0x7f82dc101de0 2024-09-19T08:22:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.939+0000 7f82e09c1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82dc106f10 0x7f82dc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.939+0000 7f82e09c1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82dc106f10 0x7f82dc19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34850/0 (socket says 172.21.15.22:34850) 2024-09-19T08:22:56.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.939+0000 7f82e09c1640 1 -- 172.21.15.22:0/2143128070 learned_addr learned my addr 172.21.15.22:0/2143128070 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:56.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.939+0000 7f82dbfff640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82dc101de0 0x7f82dc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.939+0000 7f82db7fe640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82dc102790 0x7f82dc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.940+0000 7f82e09c1640 1 -- 172.21.15.22:0/2143128070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82dc101de0 msgr2=0x7f82dc1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.940+0000 7f82e09c1640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82dc101de0 0x7f82dc1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.940+0000 7f82e09c1640 1 -- 172.21.15.22:0/2143128070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82dc102790 msgr2=0x7f82dc198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.940+0000 7f82e09c1640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82dc102790 0x7f82dc198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.940+0000 7f82e09c1640 1 -- 172.21.15.22:0/2143128070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82dc19d480 con 0x7f82dc106f10 2024-09-19T08:22:56.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.940+0000 7f82dbfff640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82dc101de0 0x7f82dc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:22:56.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.940+0000 7f82db7fe640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82dc102790 0x7f82dc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:22:56.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.940+0000 7f82e09c1640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82dc106f10 0x7f82dc19cda0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f82d4009fd0 tx=0x7f82d400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:56.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.940+0000 7f82d97fa640 1 -- 172.21.15.22:0/2143128070 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4017070 con 0x7f82dc106f10 2024-09-19T08:22:56.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.940+0000 7f82e2424640 1 -- 172.21.15.22:0/2143128070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82dc19d770 con 0x7f82dc106f10 2024-09-19T08:22:56.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.940+0000 7f82d97fa640 1 -- 172.21.15.22:0/2143128070 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d4003e50 con 0x7f82dc106f10 2024-09-19T08:22:56.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.940+0000 7f82d97fa640 1 -- 172.21.15.22:0/2143128070 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d40106c0 con 0x7f82dc106f10 2024-09-19T08:22:56.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.940+0000 7f82e2424640 1 -- 172.21.15.22:0/2143128070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82dc1a5050 con 0x7f82dc106f10 2024-09-19T08:22:56.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.942+0000 7f82e47fd740 1 Processor -- start 2024-09-19T08:22:56.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82d97fa640 1 -- 172.21.15.22:0/2143128070 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f82d4003970 con 0x7f82dc106f10 2024-09-19T08:22:56.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82e47fd740 1 -- start start 2024-09-19T08:22:56.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82e47fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e441090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82e47fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e4415d0 0x56174e441a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82e47fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 0x56174e44e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82e47fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56174e3a72f0 con 0x56174e442b40 2024-09-19T08:22:56.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82e47fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56174e34ecf0 con 0x56174e4415d0 2024-09-19T08:22:56.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82e47fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56174e3a32b0 con 0x56174e44c370 2024-09-19T08:22:56.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82db7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e441090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82db7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e441090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34858/0 (socket says 172.21.15.22:34858) 2024-09-19T08:22:56.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82db7fe640 1 -- 172.21.15.22:0/4279229153 learned_addr learned my addr 172.21.15.22:0/4279229153 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:56.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82dbfff640 1 --2- 172.21.15.22:0/4279229153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e4415d0 0x56174e441a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82db7fe640 1 --2- 172.21.15.22:0/4279229153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 0x56174e44e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82d97fa640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f82c40a33b0 0x7f82c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82db7fe640 1 -- 172.21.15.22:0/4279229153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 msgr2=0x56174e44e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82db7fe640 1 --2- 172.21.15.22:0/4279229153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 0x56174e44e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82db7fe640 1 -- 172.21.15.22:0/4279229153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e4415d0 msgr2=0x56174e441a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82db7fe640 1 --2- 172.21.15.22:0/4279229153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e4415d0 0x56174e441a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.943+0000 7f82db7fe640 1 -- 172.21.15.22:0/4279229153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56174e3333a0 con 0x56174e442b40 2024-09-19T08:22:56.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.944+0000 7f82e09c1640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f82c40a33b0 0x7f82c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.944+0000 7f82d97fa640 1 -- 172.21.15.22:0/2143128070 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f82d40cfaa0 con 0x7f82dc106f10 2024-09-19T08:22:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.944+0000 7f82db7fe640 1 --2- 172.21.15.22:0/4279229153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e441090 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x56174e442070 tx=0x7f82d0031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1e3d30a60f369d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.944+0000 7f82c27fc640 1 -- 172.21.15.22:0/4279229153 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0035050 con 0x56174e442b40 2024-09-19T08:22:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.944+0000 7f82c27fc640 1 -- 172.21.15.22:0/4279229153 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d0031890 con 0x56174e442b40 2024-09-19T08:22:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.944+0000 7f82c27fc640 1 -- 172.21.15.22:0/4279229153 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d00046c0 con 0x56174e442b40 2024-09-19T08:22:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.945+0000 7f82e47fd740 1 -- 172.21.15.22:0/4279229153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 msgr2=0x56174e441090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.945+0000 7f82e09c1640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f82c40a33b0 0x7f82c40a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f82d4005c40 tx=0x7f82d4004020 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.945+0000 7f82e47fd740 1 --2- 172.21.15.22:0/4279229153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e441090 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x56174e442070 tx=0x7f82d0031480 comp rx=0 tx=0).stop 2024-09-19T08:22:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.945+0000 7f82e47fd740 1 -- 172.21.15.22:0/4279229153 shutdown_connections 2024-09-19T08:22:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.945+0000 7f82e47fd740 1 --2- 172.21.15.22:0/4279229153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 0x56174e44e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.945+0000 7f82e47fd740 1 --2- 172.21.15.22:0/4279229153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e4415d0 0x56174e441a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.945+0000 7f82e47fd740 1 --2- 172.21.15.22:0/4279229153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e441090 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.945+0000 7f82e47fd740 1 -- 172.21.15.22:0/4279229153 >> 172.21.15.22:0/4279229153 conn(0x56174e40a310 msgr2=0x56174e43dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.945+0000 7f82e47fd740 1 -- 172.21.15.22:0/4279229153 shutdown_connections 2024-09-19T08:22:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.945+0000 7f82e47fd740 1 -- 172.21.15.22:0/4279229153 wait complete. 2024-09-19T08:22:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.945+0000 7f82e47fd740 1 -- start start 2024-09-19T08:22:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.945+0000 7f82e47fd740 1 Processor -- start 2024-09-19T08:22:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82e47fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e4415d0 0x56174e44bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82e47fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e448ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82e47fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e44c370 0x56174e4495e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82e47fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56174e44f010 con 0x56174e442b40 2024-09-19T08:22:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82e47fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56174e362060 con 0x56174e44c370 2024-09-19T08:22:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82e47fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56174e44f190 con 0x56174e4415d0 2024-09-19T08:22:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82e47fd740 10 client.0 _ll_register_callbacks cb 0x56174e0b58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:22:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82dbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e448ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82dbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e448ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34870/0 (socket says 172.21.15.22:34870) 2024-09-19T08:22:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82dbfff640 1 -- 172.21.15.22:0/3659784499 learned_addr learned my addr 172.21.15.22:0/3659784499 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82db7fe640 1 --2- 172.21.15.22:0/3659784499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e4415d0 0x56174e44bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82db7fe640 1 --2- 172.21.15.22:0/3659784499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e44c370 0x56174e4495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82dbfff640 1 -- 172.21.15.22:0/3659784499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e4415d0 msgr2=0x56174e44bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82dbfff640 1 --2- 172.21.15.22:0/3659784499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e4415d0 0x56174e44bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82dbfff640 1 -- 172.21.15.22:0/3659784499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e44c370 msgr2=0x56174e4495e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82dbfff640 1 --2- 172.21.15.22:0/3659784499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e44c370 0x56174e4495e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82dbfff640 1 -- 172.21.15.22:0/3659784499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56174e1183c0 con 0x56174e442b40 2024-09-19T08:22:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82db7fe640 1 --2- 172.21.15.22:0/3659784499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e44c370 0x56174e4495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:22:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82dbfff640 1 --2- 172.21.15.22:0/3659784499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e448ff0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f82cc010f10 tx=0x7f82cc00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18f91146ac082d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:22:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82bb7fe640 1 -- 172.21.15.22:0/3659784499 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc005020 con 0x56174e442b40 2024-09-19T08:22:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82bb7fe640 1 -- 172.21.15.22:0/3659784499 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f82cc0051c0 con 0x56174e442b40 2024-09-19T08:22:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82e47fd740 1 -- 172.21.15.22:0/3659784499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82d0031890 con 0x56174e442b40 2024-09-19T08:22:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.946+0000 7f82bb7fe640 1 -- 172.21.15.22:0/3659784499 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f82cc005400 con 0x56174e442b40 2024-09-19T08:22:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.947+0000 7f82e47fd740 10 client.16746 fetch_fsmap learned FSMap version 33 2024-09-19T08:22:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.947+0000 7f82e47fd740 1 -- 172.21.15.22:0/3659784499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56174e45ab30 con 0x56174e442b40 2024-09-19T08:22:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.947+0000 7f82bb7fe640 1 -- 172.21.15.22:0/3659784499 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d0031890 con 0x56174e442b40 2024-09-19T08:22:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.947+0000 7f82bb7fe640 1 -- 172.21.15.22:0/3659784499 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f82cc019740 con 0x56174e442b40 2024-09-19T08:22:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.947+0000 7f82e47fd740 10 client.16746 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:22:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.947+0000 7f82e47fd740 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:22:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.947+0000 7f82e47fd740 20 client.16746 populate_metadata read hostname 'smithi022' 2024-09-19T08:22:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.947+0000 7f82e47fd740 1 --2- 172.21.15.22:0/3659784499 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56174e45c000 0x56174e45e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.947+0000 7f82e47fd740 4 client.16746 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:22:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.947+0000 7f82e47fd740 1 -- 172.21.15.22:0/3659784499 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56174e3b4860 con 0x56174e45c000 2024-09-19T08:22:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.947+0000 7f82db7fe640 1 --2- 172.21.15.22:0/3659784499 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56174e45c000 0x56174e45e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.947+0000 7f82db7fe640 1 --2- 172.21.15.22:0/3659784499 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56174e45c000 0x56174e45e420 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e94b890518acaa server_cookie=4b815a9304bebb8a in_seq=0 out_seq=0 2024-09-19T08:22:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.947+0000 7f82bb7fe640 10 client.16746 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.948+0000 7f82bb7fe640 1 -- 172.21.15.22:0/3659784499 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56174e3b4860 con 0x56174e45c000 2024-09-19T08:22:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.948+0000 7f82bb7fe640 10 client.16746 handle_command_reply: tid=0 2024-09-19T08:22:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.948+0000 7f82e47fd740 1 client.16746 shutdown 2024-09-19T08:22:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.948+0000 7f82e47fd740 1 -- 172.21.15.22:0/3659784499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 msgr2=0x56174e448ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.948+0000 7f82e47fd740 1 --2- 172.21.15.22:0/3659784499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e448ff0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f82cc010f10 tx=0x7f82cc00f7f0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.948+0000 7f82e47fd740 1 -- 172.21.15.22:0/3659784499 shutdown_connections 2024-09-19T08:22:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.948+0000 7f82e47fd740 1 --2- 172.21.15.22:0/3659784499 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56174e45c000 0x56174e45e420 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.948+0000 7f82e47fd740 1 --2- 172.21.15.22:0/3659784499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e44c370 0x56174e4495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.948+0000 7f82e47fd740 1 --2- 172.21.15.22:0/3659784499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e448ff0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.948+0000 7f82e47fd740 1 --2- 172.21.15.22:0/3659784499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e4415d0 0x56174e44bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.948+0000 7f82e47fd740 1 -- 172.21.15.22:0/3659784499 >> 172.21.15.22:0/3659784499 conn(0x56174e40a310 msgr2=0x56174e443930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.948+0000 7f82bb7fe640 0 client.16746 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.949+0000 7f82e47fd740 1 -- 172.21.15.22:0/3659784499 shutdown_connections 2024-09-19T08:22:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.949+0000 7f82e47fd740 1 -- 172.21.15.22:0/3659784499 wait complete. 2024-09-19T08:22:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.949+0000 7f82e47fd740 20 client.16746 trim_cache size 0 max 16384 2024-09-19T08:22:56.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82e47fd740 1 Processor -- start 2024-09-19T08:22:56.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82e47fd740 1 -- start start 2024-09-19T08:22:56.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82e47fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e4415d0 0x56174e4582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82e47fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e442b40 0x56174e447fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82e47fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 0x56174e463cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82e47fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1b0120 con 0x56174e4415d0 2024-09-19T08:22:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82e47fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1ad960 con 0x56174e442b40 2024-09-19T08:22:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82e47fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1ab7d0 con 0x56174e44c370 2024-09-19T08:22:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82db7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e4415d0 0x56174e4582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82db7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e4415d0 0x56174e4582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34884/0 (socket says 172.21.15.22:34884) 2024-09-19T08:22:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82db7fe640 1 -- 172.21.15.22:0/420198505 learned_addr learned my addr 172.21.15.22:0/420198505 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82dbfff640 1 --2- 172.21.15.22:0/420198505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e442b40 0x56174e447fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82db7fe640 1 --2- 172.21.15.22:0/420198505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 0x56174e463cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82db7fe640 1 -- 172.21.15.22:0/420198505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 msgr2=0x56174e463cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82db7fe640 1 --2- 172.21.15.22:0/420198505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 0x56174e463cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82db7fe640 1 -- 172.21.15.22:0/420198505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e442b40 msgr2=0x56174e447fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82db7fe640 1 --2- 172.21.15.22:0/420198505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e442b40 0x56174e447fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82db7fe640 1 -- 172.21.15.22:0/420198505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56174e43ded0 con 0x56174e4415d0 2024-09-19T08:22:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.962+0000 7f82dbfff640 1 --2- 172.21.15.22:0/420198505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e442b40 0x56174e447fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:22:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.963+0000 7f82db7fe640 1 --2- 172.21.15.22:0/420198505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e4415d0 0x56174e4582b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x56174e442070 tx=0x7f82d0031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8095e46bdf395903 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.963+0000 7f82c1ffb640 1 -- 172.21.15.22:0/420198505 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0035040 con 0x56174e4415d0 2024-09-19T08:22:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.963+0000 7f82c1ffb640 1 -- 172.21.15.22:0/420198505 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d0031d70 con 0x56174e4415d0 2024-09-19T08:22:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.963+0000 7f82c1ffb640 1 -- 172.21.15.22:0/420198505 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d00037f0 con 0x56174e4415d0 2024-09-19T08:22:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.964+0000 7f82e47fd740 1 -- 172.21.15.22:0/420198505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e4415d0 msgr2=0x56174e4582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.964+0000 7f82e47fd740 1 --2- 172.21.15.22:0/420198505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e4415d0 0x56174e4582b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x56174e442070 tx=0x7f82d0031480 comp rx=0 tx=0).stop 2024-09-19T08:22:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.964+0000 7f82e47fd740 1 -- 172.21.15.22:0/420198505 shutdown_connections 2024-09-19T08:22:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.964+0000 7f82e47fd740 1 --2- 172.21.15.22:0/420198505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 0x56174e463cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.964+0000 7f82e47fd740 1 --2- 172.21.15.22:0/420198505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e442b40 0x56174e447fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.964+0000 7f82e47fd740 1 --2- 172.21.15.22:0/420198505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e4415d0 0x56174e4582b0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.964+0000 7f82e47fd740 1 -- 172.21.15.22:0/420198505 >> 172.21.15.22:0/420198505 conn(0x56174e40a310 msgr2=0x56174e4478c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.964+0000 7f82e47fd740 1 -- 172.21.15.22:0/420198505 shutdown_connections 2024-09-19T08:22:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.964+0000 7f82e47fd740 1 -- 172.21.15.22:0/420198505 wait complete. 2024-09-19T08:22:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.964+0000 7f82e47fd740 1 -- start start 2024-09-19T08:22:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.964+0000 7f82e47fd740 1 Processor -- start 2024-09-19T08:22:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82e47fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e4415d0 0x56174e454a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82e47fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e44fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82e47fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 0x56174e450300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82e47fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1abe20 con 0x56174e442b40 2024-09-19T08:22:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82e47fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1ad290 con 0x56174e4415d0 2024-09-19T08:22:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82e47fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1ac4f0 con 0x56174e44c370 2024-09-19T08:22:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82e47fd740 10 client.0 _ll_register_callbacks cb 0x7f82cc00ab90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:22:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82dbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e44fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82dbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e44fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34896/0 (socket says 172.21.15.22:34896) 2024-09-19T08:22:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82dbfff640 1 -- 172.21.15.22:0/526532145 learned_addr learned my addr 172.21.15.22:0/526532145 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82db7fe640 1 --2- 172.21.15.22:0/526532145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e4415d0 0x56174e454a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82db7fe640 1 --2- 172.21.15.22:0/526532145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 0x56174e450300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82dbfff640 1 -- 172.21.15.22:0/526532145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 msgr2=0x56174e450300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82dbfff640 1 --2- 172.21.15.22:0/526532145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 0x56174e450300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82dbfff640 1 -- 172.21.15.22:0/526532145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e4415d0 msgr2=0x56174e454a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:22:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82dbfff640 1 --2- 172.21.15.22:0/526532145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e4415d0 0x56174e454a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82dbfff640 1 -- 172.21.15.22:0/526532145 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56174e450b30 con 0x56174e442b40 2024-09-19T08:22:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82db7fe640 1 --2- 172.21.15.22:0/526532145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e4415d0 0x56174e454a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:22:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82dbfff640 1 --2- 172.21.15.22:0/526532145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e44fdc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0058c0 tx=0x7f82cc0107d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4fa73b1b9ceb24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.965+0000 7f82bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:22:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.966+0000 7f82bb7fe640 1 -- 172.21.15.22:0/526532145 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0198d0 con 0x56174e442b40 2024-09-19T08:22:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.966+0000 7f82bb7fe640 1 -- 172.21.15.22:0/526532145 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f82cc022680 con 0x56174e442b40 2024-09-19T08:22:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.966+0000 7f82e47fd740 1 -- 172.21.15.22:0/526532145 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82d0031d70 con 0x56174e442b40 2024-09-19T08:22:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.966+0000 7f82bb7fe640 1 -- 172.21.15.22:0/526532145 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f82cc022880 con 0x56174e442b40 2024-09-19T08:22:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.966+0000 7f82e47fd740 10 client.16752 fetch_fsmap learned FSMap version 33 2024-09-19T08:22:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.966+0000 7f82e47fd740 1 -- 172.21.15.22:0/526532145 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56174e4c6bf0 con 0x56174e442b40 2024-09-19T08:22:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.966+0000 7f82bb7fe640 1 -- 172.21.15.22:0/526532145 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82cc02c430 con 0x56174e442b40 2024-09-19T08:22:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.966+0000 7f82bb7fe640 1 -- 172.21.15.22:0/526532145 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f82cc00e420 con 0x56174e442b40 2024-09-19T08:22:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.966+0000 7f82e47fd740 10 client.16752 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:22:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.966+0000 7f82e47fd740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:22:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.966+0000 7f82e47fd740 20 client.16752 populate_metadata read hostname 'smithi022' 2024-09-19T08:22:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.967+0000 7f82e47fd740 1 --2- 172.21.15.22:0/526532145 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56174e4c7370 0x56174e4c9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.967+0000 7f82e47fd740 4 client.16752 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:22:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.967+0000 7f82e47fd740 1 -- 172.21.15.22:0/526532145 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f82d0031d70 con 0x56174e4c7370 2024-09-19T08:22:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.967+0000 7f82db7fe640 1 --2- 172.21.15.22:0/526532145 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56174e4c7370 0x56174e4c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.967+0000 7f82bb7fe640 10 client.16752 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.967+0000 7f82db7fe640 1 --2- 172.21.15.22:0/526532145 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56174e4c7370 0x56174e4c9790 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c29ec72b81da132 server_cookie=a22f8e88b7b1c082 in_seq=0 out_seq=0 2024-09-19T08:22:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.967+0000 7f82bb7fe640 1 -- 172.21.15.22:0/526532145 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f82d0031d70 con 0x56174e4c7370 2024-09-19T08:22:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.967+0000 7f82bb7fe640 10 client.16752 handle_command_reply: tid=0 2024-09-19T08:22:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.967+0000 7f82e47fd740 1 client.16752 shutdown 2024-09-19T08:22:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.967+0000 7f82e47fd740 1 -- 172.21.15.22:0/526532145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 msgr2=0x56174e44fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.968+0000 7f82e47fd740 1 --2- 172.21.15.22:0/526532145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e44fdc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0058c0 tx=0x7f82cc0107d0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.968+0000 7f82e47fd740 1 -- 172.21.15.22:0/526532145 shutdown_connections 2024-09-19T08:22:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.968+0000 7f82e47fd740 1 --2- 172.21.15.22:0/526532145 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56174e4c7370 0x56174e4c9790 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.968+0000 7f82e47fd740 1 --2- 172.21.15.22:0/526532145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56174e44c370 0x56174e450300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.968+0000 7f82e47fd740 1 --2- 172.21.15.22:0/526532145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56174e442b40 0x56174e44fdc0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.968+0000 7f82e47fd740 1 --2- 172.21.15.22:0/526532145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56174e4415d0 0x56174e454a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.968+0000 7f82e47fd740 1 -- 172.21.15.22:0/526532145 >> 172.21.15.22:0/526532145 conn(0x56174e40a310 msgr2=0x56174e4480e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.968+0000 7f82bb7fe640 0 client.16752 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.968+0000 7f82e47fd740 1 -- 172.21.15.22:0/526532145 shutdown_connections 2024-09-19T08:22:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.968+0000 7f82e47fd740 1 -- 172.21.15.22:0/526532145 wait complete. 2024-09-19T08:22:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.968+0000 7f82e47fd740 20 client.16752 trim_cache size 0 max 16384 2024-09-19T08:22:56.978 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:22:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.968+0000 7f82e2424640 1 -- 172.21.15.22:0/2143128070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f82c40a33b0 msgr2=0x7f82c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.968+0000 7f82e2424640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f82c40a33b0 0x7f82c40a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f82d4005c40 tx=0x7f82d4004020 comp rx=0 tx=0).stop 2024-09-19T08:22:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.969+0000 7f82e2424640 1 -- 172.21.15.22:0/2143128070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82dc106f10 msgr2=0x7f82dc19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.969+0000 7f82e2424640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82dc106f10 0x7f82dc19cda0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f82d4009fd0 tx=0x7f82d400b260 comp rx=0 tx=0).stop 2024-09-19T08:22:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.969+0000 7f82e2424640 1 -- 172.21.15.22:0/2143128070 shutdown_connections 2024-09-19T08:22:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.969+0000 7f82e2424640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f82c40a33b0 0x7f82c40a5870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.969+0000 7f82e2424640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82dc106f10 0x7f82dc19cda0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.969+0000 7f82e2424640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82dc102790 0x7f82dc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.969+0000 7f82e2424640 1 --2- 172.21.15.22:0/2143128070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82dc101de0 0x7f82dc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.969+0000 7f82e2424640 1 -- 172.21.15.22:0/2143128070 >> 172.21.15.22:0/2143128070 conn(0x7f82dc0fd8c0 msgr2=0x7f82dc0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.969+0000 7f82e2424640 1 -- 172.21.15.22:0/2143128070 shutdown_connections 2024-09-19T08:22:56.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:56.969+0000 7f82e2424640 1 -- 172.21.15.22:0/2143128070 wait complete. 2024-09-19T08:22:57.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:56 smithi157 ceph-mon[35143]: pgmap v242: 97 pgs: 97 active+clean; 998 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 557 op/s 2024-09-19T08:22:57.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:56 smithi064 ceph-mon[34900]: pgmap v242: 97 pgs: 97 active+clean; 998 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 557 op/s 2024-09-19T08:22:57.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f030b7fe640 1 -- 172.21.15.22:0/4121703888 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56553556c0a0 con 0x5655356856b0 2024-09-19T08:22:57.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f030b7fe640 10 client.26212 handle_command_reply: tid=0 2024-09-19T08:22:57.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f0334a8b740 1 client.26212 shutdown 2024-09-19T08:22:57.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f0334a8b740 1 -- 172.21.15.22:0/4121703888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655355ff980 msgr2=0x565535612720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:57.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f0334a8b740 1 --2- 172.21.15.22:0/4121703888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655355ff980 0x565535612720 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x5655355070e0 tx=0x7f031c003d50 comp rx=0 tx=0).stop 2024-09-19T08:22:57.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f0334a8b740 1 -- 172.21.15.22:0/4121703888 shutdown_connections 2024-09-19T08:22:57.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f0334a8b740 1 --2- 172.21.15.22:0/4121703888 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5655356856b0 0x565535687ad0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:57.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f0334a8b740 1 --2- 172.21.15.22:0/4121703888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56553560a240 0x56553560f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:57.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f0334a8b740 1 --2- 172.21.15.22:0/4121703888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5655355ff980 0x565535612720 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:57.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f030b7fe640 0 client.26212 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:22:57.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f0334a8b740 1 --2- 172.21.15.22:0/4121703888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5655355ff060 0x5655356121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:57.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f0334a8b740 1 -- 172.21.15.22:0/4121703888 >> 172.21.15.22:0/4121703888 conn(0x5655355fb5f0 msgr2=0x5655355fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:57.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f0334a8b740 1 -- 172.21.15.22:0/4121703888 shutdown_connections 2024-09-19T08:22:57.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f0334a8b740 1 -- 172.21.15.22:0/4121703888 wait complete. 2024-09-19T08:22:57.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.504+0000 7f0334a8b740 20 client.26212 trim_cache size 0 max 16384 2024-09-19T08:22:57.506 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:22:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 33, 2024-09-19T08:22:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:22:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 28, 2024-09-19T08:22:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:22:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "25e0fcbe": { 2024-09-19T08:22:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 28, 2024-09-19T08:22:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:22:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:22:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:22:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:22:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:22:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:22:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 145.8, 2024-09-19T08:22:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:22:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:22:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:22:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:22:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:22:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:22:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:22:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:22:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:22:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:22:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:22:57.508 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:22:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.505+0000 7f03326b2640 1 -- 172.21.15.22:0/2202651669 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f03140a33b0 msgr2=0x7f03140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.505+0000 7f03326b2640 1 --2- 172.21.15.22:0/2202651669 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f03140a33b0 0x7f03140a5870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f03240cd800 tx=0x7f0324005b40 comp rx=0 tx=0).stop 2024-09-19T08:22:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.505+0000 7f03326b2640 1 -- 172.21.15.22:0/2202651669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f032c106a80 msgr2=0x7f032c1a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.505+0000 7f03326b2640 1 --2- 172.21.15.22:0/2202651669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f032c106a80 0x7f032c1a45f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f032400cad0 tx=0x7f032400cf90 comp rx=0 tx=0).stop 2024-09-19T08:22:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.505+0000 7f03326b2640 1 -- 172.21.15.22:0/2202651669 shutdown_connections 2024-09-19T08:22:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.505+0000 7f03326b2640 1 --2- 172.21.15.22:0/2202651669 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f03140a33b0 0x7f03140a5870 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.505+0000 7f03326b2640 1 --2- 172.21.15.22:0/2202651669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f032c106a80 0x7f032c1a45f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.505+0000 7f03326b2640 1 --2- 172.21.15.22:0/2202651669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c101420 0x7f032c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.505+0000 7f03326b2640 1 --2- 172.21.15.22:0/2202651669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f032c100a80 0x7f032c109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.505+0000 7f03326b2640 1 -- 172.21.15.22:0/2202651669 >> 172.21.15.22:0/2202651669 conn(0x7f032c0fc7a0 msgr2=0x7f032c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.505+0000 7f03326b2640 1 -- 172.21.15.22:0/2202651669 shutdown_connections 2024-09-19T08:22:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:57.505+0000 7f03326b2640 1 -- 172.21.15.22:0/2202651669 wait complete. 2024-09-19T08:22:58.661 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '25e0fcbe', total seconds elapsed: 34.6 2024-09-19T08:22:58.661 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T08:22:58.661 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:22:58.973 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:22:59.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:22:58 smithi157 ceph-mon[35143]: pgmap v243: 97 pgs: 97 active+clean; 998 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 570 op/s 2024-09-19T08:22:59.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:22:58 smithi022 ceph-mon[30963]: pgmap v243: 97 pgs: 97 active+clean; 998 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 570 op/s 2024-09-19T08:22:59.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:22:58 smithi064 ceph-mon[34900]: pgmap v243: 97 pgs: 97 active+clean; 998 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 570 op/s 2024-09-19T08:22:59.195 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:22:59.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:22:59.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.482+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3459501297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0be410c170 msgr2=0x7f0be410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:59.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.482+0000 7f0be8f5f640 1 --2- 172.21.15.22:0/3459501297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0be410c170 0x7f0be410e610 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009ca0 tx=0x7f0bdc02a950 comp rx=0 tx=0).stop 2024-09-19T08:22:59.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.482+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3459501297 shutdown_connections 2024-09-19T08:22:59.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.482+0000 7f0be8f5f640 1 --2- 172.21.15.22:0/3459501297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0be410c170 0x7f0be410e610 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:59.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.482+0000 7f0be8f5f640 1 --2- 172.21.15.22:0/3459501297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0be4103660 0x7f0be410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:59.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.482+0000 7f0be8f5f640 1 --2- 172.21.15.22:0/3459501297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0be4102d40 0x7f0be4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.482+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3459501297 >> 172.21.15.22:0/3459501297 conn(0x7f0be40fc7a0 msgr2=0x7f0be40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.482+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3459501297 shutdown_connections 2024-09-19T08:22:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.482+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3459501297 wait complete. 2024-09-19T08:22:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.483+0000 7f0be8f5f640 1 Processor -- start 2024-09-19T08:22:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.483+0000 7f0be8f5f640 1 -- start start 2024-09-19T08:22:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be8f5f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0be4102d40 0x7f0be41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be8f5f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0be4103660 0x7f0be4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be8f5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0be410c170 0x7f0be419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be8f5f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0be4111fc0 con 0x7f0be4103660 2024-09-19T08:22:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be8f5f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0be4111e40 con 0x7f0be410c170 2024-09-19T08:22:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be8f5f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0be4112140 con 0x7f0be4102d40 2024-09-19T08:22:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0be4103660 0x7f0be4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0be4103660 0x7f0be4198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35258/0 (socket says 172.21.15.22:35258) 2024-09-19T08:22:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0be4102d40 0x7f0be41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be1d9b640 1 -- 172.21.15.22:0/3500189865 learned_addr learned my addr 172.21.15.22:0/3500189865 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:22:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be2d9d640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0be410c170 0x7f0be419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be1d9b640 1 -- 172.21.15.22:0/3500189865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0be4102d40 msgr2=0x7f0be41983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be1d9b640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0be4102d40 0x7f0be41983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be1d9b640 1 -- 172.21.15.22:0/3500189865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0be410c170 msgr2=0x7f0be419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be1d9b640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0be410c170 0x7f0be419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be1d9b640 1 -- 172.21.15.22:0/3500189865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be419d370 con 0x7f0be4103660 2024-09-19T08:22:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be2d9d640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0be410c170 0x7f0be419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:22:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be259c640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0be4102d40 0x7f0be41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:22:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be1d9b640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0be4103660 0x7f0be4198900 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800d950 tx=0x7f0bd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0bd37fe640 1 -- 172.21.15.22:0/3500189865 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8017030 con 0x7f0be4103660 2024-09-19T08:22:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3500189865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be419d660 con 0x7f0be4103660 2024-09-19T08:22:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0bd37fe640 1 -- 172.21.15.22:0/3500189865 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bd8004020 con 0x7f0be4103660 2024-09-19T08:22:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.484+0000 7f0bd37fe640 1 -- 172.21.15.22:0/3500189865 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80047a0 con 0x7f0be4103660 2024-09-19T08:22:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.485+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3500189865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be41a4f40 con 0x7f0be4103660 2024-09-19T08:22:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.486+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3500189865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be4103120 con 0x7f0be4103660 2024-09-19T08:22:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.487+0000 7f0bd37fe640 1 -- 172.21.15.22:0/3500189865 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0bd80041d0 con 0x7f0be4103660 2024-09-19T08:22:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.488+0000 7f0bd37fe640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0bcc0a33b0 0x7f0bcc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:22:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.488+0000 7f0be259c640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0bcc0a33b0 0x7f0bcc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:22:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.488+0000 7f0bd37fe640 1 -- 172.21.15.22:0/3500189865 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f0bd80cf640 con 0x7f0be4103660 2024-09-19T08:22:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.488+0000 7f0be259c640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0bcc0a33b0 0x7f0bcc0a5870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009fc0 tx=0x7f0bd4005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:22:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.492+0000 7f0bd37fe640 1 -- 172.21.15.22:0/3500189865 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0bd809c010 con 0x7f0be4103660 2024-09-19T08:22:59.595 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:22:59.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.811+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3500189865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0be41a5230 con 0x7f0be4103660 2024-09-19T08:22:59.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.812+0000 7f0bd37fe640 1 -- 172.21.15.22:0/3500189865 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11911 (secure 0 0 0) 0x7f0bd809b910 con 0x7f0be4103660 2024-09-19T08:22:59.813 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:22:59.813 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"btime":"2024-09-19T08:22:26:539605+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:22:25.667214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:22:59.813 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 33 2024-09-19T08:22:59.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.815+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3500189865 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0bcc0a33b0 msgr2=0x7f0bcc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.815+0000 7f0be8f5f640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0bcc0a33b0 0x7f0bcc0a5870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009fc0 tx=0x7f0bd4005f90 comp rx=0 tx=0).stop 2024-09-19T08:22:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.815+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3500189865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0be4103660 msgr2=0x7f0be4198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:22:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.815+0000 7f0be8f5f640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0be4103660 0x7f0be4198900 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800d950 tx=0x7f0bd800de10 comp rx=0 tx=0).stop 2024-09-19T08:22:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.815+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3500189865 shutdown_connections 2024-09-19T08:22:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.815+0000 7f0be8f5f640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0bcc0a33b0 0x7f0bcc0a5870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.815+0000 7f0be8f5f640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0be410c170 0x7f0be419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.815+0000 7f0be8f5f640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0be4103660 0x7f0be4198900 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.815+0000 7f0be8f5f640 1 --2- 172.21.15.22:0/3500189865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0be4102d40 0x7f0be41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:22:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.815+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3500189865 >> 172.21.15.22:0/3500189865 conn(0x7f0be40fc7a0 msgr2=0x7f0be40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:22:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.816+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3500189865 shutdown_connections 2024-09-19T08:22:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:22:59.816+0000 7f0be8f5f640 1 -- 172.21.15.22:0/3500189865 wait complete. 2024-09-19T08:23:00.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.286+0000 7fe6dc9a0640 1 -- 172.21.15.22:0/3588195617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d8102370 msgr2=0x7fe6d810e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:00.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.286+0000 7fe6dc9a0640 1 --2- 172.21.15.22:0/3588195617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d8102370 0x7fe6d810e8a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009cd0 tx=0x7fe6d002a8e0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.286+0000 7fe6dc9a0640 1 -- 172.21.15.22:0/3588195617 shutdown_connections 2024-09-19T08:23:00.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.286+0000 7fe6dc9a0640 1 --2- 172.21.15.22:0/3588195617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d8102370 0x7fe6d810e8a0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.286+0000 7fe6dc9a0640 1 --2- 172.21.15.22:0/3588195617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d81019d0 0x7fe6d8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.286+0000 7fe6dc9a0640 1 --2- 172.21.15.22:0/3588195617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6d81079d0 0x7fe6d8107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.286+0000 7fe6dc9a0640 1 -- 172.21.15.22:0/3588195617 >> 172.21.15.22:0/3588195617 conn(0x7fe6d80fd710 msgr2=0x7fe6d80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:00.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.286+0000 7fe6dc9a0640 1 -- 172.21.15.22:0/3588195617 shutdown_connections 2024-09-19T08:23:00.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.286+0000 7fe6dc9a0640 1 -- 172.21.15.22:0/3588195617 wait complete. 2024-09-19T08:23:00.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.286+0000 7fe6dc9a0640 1 Processor -- start 2024-09-19T08:23:00.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.286+0000 7fe6dc9a0640 1 -- start start 2024-09-19T08:23:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6dc9a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d81019d0 0x7fe6d81982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6dc9a0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6d8102370 0x7fe6d8198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6dc9a0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d81079d0 0x7fe6d819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6dc9a0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8111dd0 con 0x7fe6d81019d0 2024-09-19T08:23:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6dc9a0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8111c50 con 0x7fe6d81079d0 2024-09-19T08:23:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6dc9a0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8111f50 con 0x7fe6d8102370 2024-09-19T08:23:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d81019d0 0x7fe6d81982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d81019d0 0x7fe6d81982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35282/0 (socket says 172.21.15.22:35282) 2024-09-19T08:23:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6d77fe640 1 -- 172.21.15.22:0/2560672155 learned_addr learned my addr 172.21.15.22:0/2560672155 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6cffff640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6d8102370 0x7fe6d8198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6d7fff640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d81079d0 0x7fe6d819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6d77fe640 1 -- 172.21.15.22:0/2560672155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6d8102370 msgr2=0x7fe6d8198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6d77fe640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6d8102370 0x7fe6d8198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6d77fe640 1 -- 172.21.15.22:0/2560672155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d81079d0 msgr2=0x7fe6d819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6d77fe640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d81079d0 0x7fe6d819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6d77fe640 1 -- 172.21.15.22:0/2560672155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d819d280 con 0x7fe6d81019d0 2024-09-19T08:23:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6d7fff640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d81079d0 0x7fe6d819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:23:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6d77fe640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d81019d0 0x7fe6d81982d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e9f0 tx=0x7fe6c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6d57fa640 1 -- 172.21.15.22:0/2560672155 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c800cd80 con 0x7fe6d81019d0 2024-09-19T08:23:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6d57fa640 1 -- 172.21.15.22:0/2560672155 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8005500 con 0x7fe6d81019d0 2024-09-19T08:23:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6dc9a0640 1 -- 172.21.15.22:0/2560672155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d819d570 con 0x7fe6d81019d0 2024-09-19T08:23:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.287+0000 7fe6d57fa640 1 -- 172.21.15.22:0/2560672155 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8010690 con 0x7fe6d81019d0 2024-09-19T08:23:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.288+0000 7fe6dc9a0640 1 -- 172.21.15.22:0/2560672155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d81a4e50 con 0x7fe6d81019d0 2024-09-19T08:23:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.290+0000 7fe6d57fa640 1 -- 172.21.15.22:0/2560672155 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe6c80046c0 con 0x7fe6d81019d0 2024-09-19T08:23:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.290+0000 7fe6dffdb740 1 Processor -- start 2024-09-19T08:23:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6dffdb740 1 -- start start 2024-09-19T08:23:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6d57fa640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe6c40a33b0 0x7fe6c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6dffdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f4dd70 0x562d48f4e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6dffdb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f58a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6dffdb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f58f50 0x562d48f5b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6d7fff640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe6c40a33b0 0x7fe6c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6dffdb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562d48eb3280 con 0x562d48f4e690 2024-09-19T08:23:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6dffdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562d48e5ab10 con 0x562d48f4dd70 2024-09-19T08:23:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6dffdb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562d48eaf240 con 0x562d48f58f50 2024-09-19T08:23:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6d57fa640 1 -- 172.21.15.22:0/2560672155 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fe6c80d00f0 con 0x7fe6d81019d0 2024-09-19T08:23:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f58a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f58a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35298/0 (socket says 172.21.15.22:35298) 2024-09-19T08:23:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6cffff640 1 -- 172.21.15.22:0/2097534866 learned_addr learned my addr 172.21.15.22:0/2097534866 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6d7fff640 1 --2- 172.21.15.22:0/2097534866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f58f50 0x562d48f5b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6d77fe640 1 --2- 172.21.15.22:0/2097534866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f4dd70 0x562d48f4e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6cffff640 1 -- 172.21.15.22:0/2097534866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f58f50 msgr2=0x562d48f5b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6cffff640 1 --2- 172.21.15.22:0/2097534866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f58f50 0x562d48f5b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6cffff640 1 -- 172.21.15.22:0/2097534866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f4dd70 msgr2=0x562d48f4e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6cffff640 1 --2- 172.21.15.22:0/2097534866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f4dd70 0x562d48f4e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6cffff640 1 -- 172.21.15.22:0/2097534866 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d48e671f0 con 0x562d48f4e690 2024-09-19T08:23:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.291+0000 7fe6d7fff640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe6c40a33b0 0x7fe6c40a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009ca0 tx=0x7fe6d0031620 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.292+0000 7fe6cffff640 1 --2- 172.21.15.22:0/2097534866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f58a10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009c10 tx=0x7fe6c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7a59c96b53858 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.292+0000 7fe6cdffb640 1 -- 172.21.15.22:0/2097534866 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c001e070 con 0x562d48f4e690 2024-09-19T08:23:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.292+0000 7fe6cdffb640 1 -- 172.21.15.22:0/2097534866 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c0019b10 con 0x562d48f4e690 2024-09-19T08:23:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.292+0000 7fe6cdffb640 1 -- 172.21.15.22:0/2097534866 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000e9f0 con 0x562d48f4e690 2024-09-19T08:23:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.292+0000 7fe6dffdb740 1 -- 172.21.15.22:0/2097534866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 msgr2=0x562d48f58a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.292+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/2097534866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f58a10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009c10 tx=0x7fe6c0011810 comp rx=0 tx=0).stop 2024-09-19T08:23:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 -- 172.21.15.22:0/2097534866 shutdown_connections 2024-09-19T08:23:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/2097534866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f58f50 0x562d48f5b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/2097534866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f58a10 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/2097534866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f4dd70 0x562d48f4e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 -- 172.21.15.22:0/2097534866 >> 172.21.15.22:0/2097534866 conn(0x562d48f4a300 msgr2=0x562d48f4a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 -- 172.21.15.22:0/2097534866 shutdown_connections 2024-09-19T08:23:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 -- 172.21.15.22:0/2097534866 wait complete. 2024-09-19T08:23:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 -- start start 2024-09-19T08:23:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 Processor -- start 2024-09-19T08:23:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f4dd70 0x562d48f55e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f56350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f58f50 0x562d48f56890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562d48f5bb20 con 0x562d48f4e690 2024-09-19T08:23:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562d48e6df10 con 0x562d48f58f50 2024-09-19T08:23:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562d48f5bca0 con 0x562d48f4dd70 2024-09-19T08:23:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.293+0000 7fe6dffdb740 10 client.0 _ll_register_callbacks cb 0x562d48e36a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f4dd70 0x562d48f55e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f56350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f4dd70 0x562d48f55e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:59474/0 (socket says 172.21.15.22:59474) 2024-09-19T08:23:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6d77fe640 1 -- 172.21.15.22:0/3450688785 learned_addr learned my addr 172.21.15.22:0/3450688785 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6cffff640 1 --2- 172.21.15.22:0/3450688785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f56350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35308/0 (socket says 172.21.15.22:35308) 2024-09-19T08:23:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6d7fff640 1 --2- 172.21.15.22:0/3450688785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f58f50 0x562d48f56890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6cffff640 1 -- 172.21.15.22:0/3450688785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f4dd70 msgr2=0x562d48f55e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6cffff640 1 --2- 172.21.15.22:0/3450688785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f4dd70 0x562d48f55e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6cffff640 1 -- 172.21.15.22:0/3450688785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f58f50 msgr2=0x562d48f56890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:00.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6cffff640 1 --2- 172.21.15.22:0/3450688785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f58f50 0x562d48f56890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6cffff640 1 -- 172.21.15.22:0/3450688785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d48c246a0 con 0x562d48f4e690 2024-09-19T08:23:00.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6d77fe640 1 --2- 172.21.15.22:0/3450688785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f4dd70 0x562d48f55e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:00.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6cffff640 1 --2- 172.21.15.22:0/3450688785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f56350 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe6c001c040 tx=0x7fe6c0019aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d64e29ebf214f2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:00.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:00.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6b6ffd640 1 -- 172.21.15.22:0/3450688785 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c001e050 con 0x562d48f4e690 2024-09-19T08:23:00.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6b6ffd640 1 -- 172.21.15.22:0/3450688785 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fe6c0005450 con 0x562d48f4e690 2024-09-19T08:23:00.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6dffdb740 1 -- 172.21.15.22:0/3450688785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c0019b10 con 0x562d48f4e690 2024-09-19T08:23:00.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6b6ffd640 1 -- 172.21.15.22:0/3450688785 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe6c0032050 con 0x562d48f4e690 2024-09-19T08:23:00.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6dffdb740 10 client.16770 fetch_fsmap learned FSMap version 33 2024-09-19T08:23:00.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.294+0000 7fe6dffdb740 1 -- 172.21.15.22:0/3450688785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562d48f67700 con 0x562d48f4e690 2024-09-19T08:23:00.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.295+0000 7fe6b6ffd640 1 -- 172.21.15.22:0/3450688785 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c0019b10 con 0x562d48f4e690 2024-09-19T08:23:00.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.295+0000 7fe6b6ffd640 1 -- 172.21.15.22:0/3450688785 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe6c002d030 con 0x562d48f4e690 2024-09-19T08:23:00.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.295+0000 7fe6dffdb740 10 client.16770 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:23:00.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.295+0000 7fe6dffdb740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:23:00.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.295+0000 7fe6dffdb740 20 client.16770 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:00.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.295+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/3450688785 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562d48f68bc0 0x562d48f6afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.295+0000 7fe6dffdb740 4 client.16770 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:23:00.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.295+0000 7fe6dffdb740 1 -- 172.21.15.22:0/3450688785 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d48ec07f0 con 0x562d48f68bc0 2024-09-19T08:23:00.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.295+0000 7fe6d77fe640 1 --2- 172.21.15.22:0/3450688785 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562d48f68bc0 0x562d48f6afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.295+0000 7fe6b6ffd640 10 client.16770 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:00.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.295+0000 7fe6d77fe640 1 --2- 172.21.15.22:0/3450688785 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562d48f68bc0 0x562d48f6afb0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=312e67e47694ab4c server_cookie=5491fb1225ae268c in_seq=0 out_seq=0 2024-09-19T08:23:00.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.296+0000 7fe6b6ffd640 1 -- 172.21.15.22:0/3450688785 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d48ec07f0 con 0x562d48f68bc0 2024-09-19T08:23:00.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.296+0000 7fe6b6ffd640 10 client.16770 handle_command_reply: tid=0 2024-09-19T08:23:00.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.296+0000 7fe6dffdb740 1 client.16770 shutdown 2024-09-19T08:23:00.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.297+0000 7fe6dffdb740 1 -- 172.21.15.22:0/3450688785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 msgr2=0x562d48f56350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:00.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.297+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/3450688785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f56350 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe6c001c040 tx=0x7fe6c0019aa0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.297+0000 7fe6dffdb740 1 -- 172.21.15.22:0/3450688785 shutdown_connections 2024-09-19T08:23:00.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.297+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/3450688785 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562d48f68bc0 0x562d48f6afb0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.297+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/3450688785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f58f50 0x562d48f56890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.297+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/3450688785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f56350 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.297+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/3450688785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f4dd70 0x562d48f55e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.297+0000 7fe6dffdb740 1 -- 172.21.15.22:0/3450688785 >> 172.21.15.22:0/3450688785 conn(0x562d48f4a300 msgr2=0x562d48f50030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:00.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.297+0000 7fe6b6ffd640 0 client.16770 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:00.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.297+0000 7fe6dffdb740 1 -- 172.21.15.22:0/3450688785 shutdown_connections 2024-09-19T08:23:00.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.297+0000 7fe6dffdb740 1 -- 172.21.15.22:0/3450688785 wait complete. 2024-09-19T08:23:00.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.297+0000 7fe6dffdb740 20 client.16770 trim_cache size 0 max 16384 2024-09-19T08:23:00.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.316+0000 7fe6dffdb740 1 Processor -- start 2024-09-19T08:23:00.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.316+0000 7fe6dffdb740 1 -- start start 2024-09-19T08:23:00.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.316+0000 7fe6dffdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f4dd70 0x562d48f507d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.316+0000 7fe6dffdb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f49300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.316+0000 7fe6dffdb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f58f50 0x562d48f70720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.316+0000 7fe6dffdb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81adf40 con 0x562d48f4e690 2024-09-19T08:23:00.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.316+0000 7fe6dffdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81ab780 con 0x562d48f4dd70 2024-09-19T08:23:00.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.316+0000 7fe6dffdb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81a95f0 con 0x562d48f58f50 2024-09-19T08:23:00.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.316+0000 7fe6d7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f58f50 0x562d48f70720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.316+0000 7fe6d7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f58f50 0x562d48f70720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:59476/0 (socket says 172.21.15.22:59476) 2024-09-19T08:23:00.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.316+0000 7fe6d7fff640 1 -- 172.21.15.22:0/300368895 learned_addr learned my addr 172.21.15.22:0/300368895 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:00.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.317+0000 7fe6d77fe640 1 --2- 172.21.15.22:0/300368895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f4dd70 0x562d48f507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.317+0000 7fe6cffff640 1 --2- 172.21.15.22:0/300368895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f49300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.317+0000 7fe6d7fff640 1 -- 172.21.15.22:0/300368895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f4dd70 msgr2=0x562d48f507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:00.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.317+0000 7fe6d7fff640 1 --2- 172.21.15.22:0/300368895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f4dd70 0x562d48f507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.317+0000 7fe6d7fff640 1 -- 172.21.15.22:0/300368895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 msgr2=0x562d48f49300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:00.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.317+0000 7fe6d7fff640 1 --2- 172.21.15.22:0/300368895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f49300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.317+0000 7fe6d7fff640 1 -- 172.21.15.22:0/300368895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d48f5b6a0 con 0x562d48f58f50 2024-09-19T08:23:00.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.317+0000 7fe6cffff640 1 --2- 172.21.15.22:0/300368895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f49300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:23:00.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.317+0000 7fe6d77fe640 1 --2- 172.21.15.22:0/300368895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f4dd70 0x562d48f507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:23:00.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.317+0000 7fe6d7fff640 1 --2- 172.21.15.22:0/300368895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f58f50 0x562d48f70720 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0005db0 tx=0x7fe6d0047000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=928a465f1dfd5651 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:00.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.317+0000 7fe6cd7fa640 1 -- 172.21.15.22:0/300368895 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d004f070 con 0x562d48f58f50 2024-09-19T08:23:00.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.318+0000 7fe6cd7fa640 1 -- 172.21.15.22:0/300368895 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d0036040 con 0x562d48f58f50 2024-09-19T08:23:00.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.318+0000 7fe6cd7fa640 1 -- 172.21.15.22:0/300368895 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0030800 con 0x562d48f58f50 2024-09-19T08:23:00.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.318+0000 7fe6dffdb740 1 -- 172.21.15.22:0/300368895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f58f50 msgr2=0x562d48f70720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:00.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.318+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/300368895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f58f50 0x562d48f70720 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0005db0 tx=0x7fe6d0047000 comp rx=0 tx=0).stop 2024-09-19T08:23:00.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.318+0000 7fe6dffdb740 1 -- 172.21.15.22:0/300368895 shutdown_connections 2024-09-19T08:23:00.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.318+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/300368895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f58f50 0x562d48f70720 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.318+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/300368895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f49300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.318+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/300368895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f4dd70 0x562d48f507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.318+0000 7fe6dffdb740 1 -- 172.21.15.22:0/300368895 >> 172.21.15.22:0/300368895 conn(0x562d48f4a300 msgr2=0x562d48f48bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:00.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.318+0000 7fe6dffdb740 1 -- 172.21.15.22:0/300368895 shutdown_connections 2024-09-19T08:23:00.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.318+0000 7fe6dffdb740 1 -- 172.21.15.22:0/300368895 wait complete. 2024-09-19T08:23:00.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.319+0000 7fe6dffdb740 1 -- start start 2024-09-19T08:23:00.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.319+0000 7fe6dffdb740 1 Processor -- start 2024-09-19T08:23:00.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.319+0000 7fe6dffdb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f4dd70 0x562d48f61460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.319+0000 7fe6dffdb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f5c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.319+0000 7fe6dffdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f58f50 0x562d48f5ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.319+0000 7fe6dffdb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81a9c40 con 0x562d48f4e690 2024-09-19T08:23:00.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.319+0000 7fe6dffdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81ab0b0 con 0x562d48f58f50 2024-09-19T08:23:00.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.319+0000 7fe6dffdb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81aa310 con 0x562d48f4dd70 2024-09-19T08:23:00.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f5c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f4dd70 0x562d48f61460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f5c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35328/0 (socket says 172.21.15.22:35328) 2024-09-19T08:23:00.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6cffff640 1 -- 172.21.15.22:0/1897072476 learned_addr learned my addr 172.21.15.22:0/1897072476 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:00.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6d7fff640 1 --2- 172.21.15.22:0/1897072476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f58f50 0x562d48f5ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6dffdb740 10 client.0 _ll_register_callbacks cb 0x7fe6c001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:00.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6d7fff640 1 -- 172.21.15.22:0/1897072476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f4dd70 msgr2=0x562d48f61460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:00.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6d7fff640 1 --2- 172.21.15.22:0/1897072476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f4dd70 0x562d48f61460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6d7fff640 1 -- 172.21.15.22:0/1897072476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 msgr2=0x562d48f5c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:00.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6d7fff640 1 --2- 172.21.15.22:0/1897072476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f5c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6d7fff640 1 -- 172.21.15.22:0/1897072476 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d48f5d310 con 0x562d48f58f50 2024-09-19T08:23:00.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6cffff640 1 --2- 172.21.15.22:0/1897072476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f5c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:00.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6d7fff640 1 --2- 172.21.15.22:0/1897072476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f58f50 0x562d48f5ccd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0006010 tx=0x7fe6d00095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7242ea54c542f905 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:00.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:23:00.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.320+0000 7fe6b6ffd640 1 -- 172.21.15.22:0/1897072476 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0058020 con 0x562d48f58f50 2024-09-19T08:23:00.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.321+0000 7fe6dffdb740 1 -- 172.21.15.22:0/1897072476 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6d0036040 con 0x562d48f58f50 2024-09-19T08:23:00.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.321+0000 7fe6b6ffd640 1 -- 172.21.15.22:0/1897072476 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fe6d0004240 con 0x562d48f58f50 2024-09-19T08:23:00.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.321+0000 7fe6b6ffd640 1 -- 172.21.15.22:0/1897072476 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe6d0063050 con 0x562d48f58f50 2024-09-19T08:23:00.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.321+0000 7fe6dffdb740 10 client.26273 fetch_fsmap learned FSMap version 33 2024-09-19T08:23:00.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.321+0000 7fe6dffdb740 1 -- 172.21.15.22:0/1897072476 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562d48fdbd30 con 0x562d48f58f50 2024-09-19T08:23:00.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.321+0000 7fe6b6ffd640 1 -- 172.21.15.22:0/1897072476 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d0036040 con 0x562d48f58f50 2024-09-19T08:23:00.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.321+0000 7fe6b6ffd640 1 -- 172.21.15.22:0/1897072476 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe6d0030070 con 0x562d48f58f50 2024-09-19T08:23:00.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.321+0000 7fe6dffdb740 10 client.26273 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:23:00.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.321+0000 7fe6dffdb740 10 client.26273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:23:00.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.321+0000 7fe6dffdb740 20 client.26273 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:00.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.321+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/1897072476 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562d48fdc4d0 0x562d48fde8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:00.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.321+0000 7fe6dffdb740 4 client.26273 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:23:00.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.321+0000 7fe6dffdb740 1 -- 172.21.15.22:0/1897072476 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562d48ebad00 con 0x562d48fdc4d0 2024-09-19T08:23:00.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.321+0000 7fe6d77fe640 1 --2- 172.21.15.22:0/1897072476 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562d48fdc4d0 0x562d48fde8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:00.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.322+0000 7fe6b6ffd640 10 client.26273 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:00.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.322+0000 7fe6d77fe640 1 --2- 172.21.15.22:0/1897072476 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562d48fdc4d0 0x562d48fde8f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44e06340bba7ff64 server_cookie=bfa21b2658c35e63 in_seq=0 out_seq=0 2024-09-19T08:23:00.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6b6ffd640 1 -- 172.21.15.22:0/1897072476 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562d48ebad00 con 0x562d48fdc4d0 2024-09-19T08:23:00.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6b6ffd640 10 client.26273 handle_command_reply: tid=0 2024-09-19T08:23:00.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6dffdb740 1 client.26273 shutdown 2024-09-19T08:23:00.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6dffdb740 1 -- 172.21.15.22:0/1897072476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f58f50 msgr2=0x562d48f5ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:00.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/1897072476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f58f50 0x562d48f5ccd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0006010 tx=0x7fe6d00095c0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6dffdb740 1 -- 172.21.15.22:0/1897072476 shutdown_connections 2024-09-19T08:23:00.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/1897072476 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562d48fdc4d0 0x562d48fde8f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/1897072476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d48f58f50 0x562d48f5ccd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/1897072476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d48f4e690 0x562d48f5c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6dffdb740 1 --2- 172.21.15.22:0/1897072476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d48f4dd70 0x562d48f61460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6dffdb740 1 -- 172.21.15.22:0/1897072476 >> 172.21.15.22:0/1897072476 conn(0x562d48f4a300 msgr2=0x562d48f49410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:00.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6b6ffd640 0 client.26273 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:00.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6dffdb740 1 -- 172.21.15.22:0/1897072476 shutdown_connections 2024-09-19T08:23:00.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6dffdb740 1 -- 172.21.15.22:0/1897072476 wait complete. 2024-09-19T08:23:00.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.323+0000 7fe6dffdb740 20 client.26273 trim_cache size 0 max 16384 2024-09-19T08:23:00.329 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:23:00.329 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-19T08:23:00.329 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "775f4f07-dc6d-46c4-b861-4aa631c480d4", 2024-09-19T08:23:00.329 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-19T08:23:00.329 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:23:00.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.324+0000 7fe6dc9a0640 1 -- 172.21.15.22:0/2560672155 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe6c40a33b0 msgr2=0x7fe6c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:00.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.324+0000 7fe6dc9a0640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe6c40a33b0 0x7fe6c40a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009ca0 tx=0x7fe6d0031620 comp rx=0 tx=0).stop 2024-09-19T08:23:00.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.324+0000 7fe6dc9a0640 1 -- 172.21.15.22:0/2560672155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d81019d0 msgr2=0x7fe6d81982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:00.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.324+0000 7fe6dc9a0640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d81019d0 0x7fe6d81982d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e9f0 tx=0x7fe6c800edf0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.324+0000 7fe6dc9a0640 1 -- 172.21.15.22:0/2560672155 shutdown_connections 2024-09-19T08:23:00.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.324+0000 7fe6dc9a0640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe6c40a33b0 0x7fe6c40a5870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.324+0000 7fe6dc9a0640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6d81079d0 0x7fe6d819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.324+0000 7fe6dc9a0640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6d8102370 0x7fe6d8198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.324+0000 7fe6dc9a0640 1 --2- 172.21.15.22:0/2560672155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d81019d0 0x7fe6d81982d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:00.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.324+0000 7fe6dc9a0640 1 -- 172.21.15.22:0/2560672155 >> 172.21.15.22:0/2560672155 conn(0x7fe6d80fd710 msgr2=0x7fe6d80feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:00.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.324+0000 7fe6dc9a0640 1 -- 172.21.15.22:0/2560672155 shutdown_connections 2024-09-19T08:23:00.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:00.324+0000 7fe6dc9a0640 1 -- 172.21.15.22:0/2560672155 wait complete. 2024-09-19T08:23:00.804 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T08:23:00.804 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T08:23:00.864 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:00 smithi022 ceph-mon[30963]: pgmap v244: 97 pgs: 97 active+clean; 999 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 452 op/s 2024-09-19T08:23:00.864 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:00 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3500189865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:23:01.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:00 smithi157 ceph-mon[35143]: pgmap v244: 97 pgs: 97 active+clean; 999 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 452 op/s 2024-09-19T08:23:01.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:00 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3500189865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:23:01.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:00 smithi064 ceph-mon[34900]: pgmap v244: 97 pgs: 97 active+clean; 999 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 452 op/s 2024-09-19T08:23:01.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:00 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3500189865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:23:01.544 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:23:01.590 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:23:01.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:01 smithi022 ceph-mon[30963]: scrub queued for path: / 2024-09-19T08:23:01.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:01 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:01.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:01 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:23:01.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:01 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:23:01.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:01 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:01.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:01 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:23:01.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:01 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:02.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:01 smithi157 ceph-mon[35143]: scrub queued for path: / 2024-09-19T08:23:02.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:01 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:02.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:01 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:23:02.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:01 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:23:02.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:01 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:02.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:01 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:23:02.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:01 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:02.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:01 smithi064 ceph-mon[34900]: scrub queued for path: / 2024-09-19T08:23:02.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:01 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:02.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:01 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:23:02.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:01 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:23:02.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:01 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:02.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:01 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:23:02.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:01 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:02.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.131+0000 7f0c895fe640 1 -- 172.21.15.22:0/393051544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c84102570 msgr2=0x7f0c841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.131+0000 7f0c895fe640 1 --2- 172.21.15.22:0/393051544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c84102570 0x7f0c841029b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009900 tx=0x7f0c6c02a480 comp rx=0 tx=0).stop 2024-09-19T08:23:02.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.131+0000 7f0c895fe640 1 -- 172.21.15.22:0/393051544 shutdown_connections 2024-09-19T08:23:02.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.132+0000 7f0c895fe640 1 --2- 172.21.15.22:0/393051544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84102ef0 0x7f0c8410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.132+0000 7f0c895fe640 1 --2- 172.21.15.22:0/393051544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c84102570 0x7f0c841029b0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.132+0000 7f0c895fe640 1 --2- 172.21.15.22:0/393051544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c84100e20 0x7f0c84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.132+0000 7f0c895fe640 1 -- 172.21.15.22:0/393051544 >> 172.21.15.22:0/393051544 conn(0x7f0c840fc7d0 msgr2=0x7f0c840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.132+0000 7f0c895fe640 1 -- 172.21.15.22:0/393051544 shutdown_connections 2024-09-19T08:23:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.132+0000 7f0c895fe640 1 -- 172.21.15.22:0/393051544 wait complete. 2024-09-19T08:23:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.132+0000 7f0c895fe640 1 Processor -- start 2024-09-19T08:23:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.132+0000 7f0c895fe640 1 -- start start 2024-09-19T08:23:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.132+0000 7f0c895fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c84100e20 0x7f0c841985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.132+0000 7f0c895fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c84102570 0x7f0c84198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.132+0000 7f0c895fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84102ef0 0x7f0c8419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.132+0000 7f0c895fe640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84110f80 con 0x7f0c84100e20 2024-09-19T08:23:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.132+0000 7f0c895fe640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84110e00 con 0x7f0c84102570 2024-09-19T08:23:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.132+0000 7f0c895fe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84111100 con 0x7f0c84102ef0 2024-09-19T08:23:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.133+0000 7f0c82ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c84100e20 0x7f0c841985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.133+0000 7f0c82ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c84100e20 0x7f0c841985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35356/0 (socket says 172.21.15.22:35356) 2024-09-19T08:23:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.133+0000 7f0c7bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c84102570 0x7f0c84198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.133+0000 7f0c837fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84102ef0 0x7f0c8419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.133+0000 7f0c82ffd640 1 -- 172.21.15.22:0/69073036 learned_addr learned my addr 172.21.15.22:0/69073036 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.133+0000 7f0c7bfff640 1 -- 172.21.15.22:0/69073036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84102ef0 msgr2=0x7f0c8419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.133+0000 7f0c7bfff640 1 --2- 172.21.15.22:0/69073036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84102ef0 0x7f0c8419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.133+0000 7f0c7bfff640 1 -- 172.21.15.22:0/69073036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c84100e20 msgr2=0x7f0c841985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.133+0000 7f0c7bfff640 1 --2- 172.21.15.22:0/69073036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c84100e20 0x7f0c841985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.133+0000 7f0c7bfff640 1 -- 172.21.15.22:0/69073036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8419d570 con 0x7f0c84102570 2024-09-19T08:23:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.133+0000 7f0c7bfff640 1 --2- 172.21.15.22:0/69073036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c84102570 0x7f0c84198b00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c005940 tx=0x7f0c6c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.134+0000 7f0c80ff9640 1 -- 172.21.15.22:0/69073036 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c03f070 con 0x7f0c84102570 2024-09-19T08:23:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.134+0000 7f0c895fe640 1 -- 172.21.15.22:0/69073036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8419d800 con 0x7f0c84102570 2024-09-19T08:23:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.134+0000 7f0c895fe640 1 -- 172.21.15.22:0/69073036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c840fcbb0 con 0x7f0c84102570 2024-09-19T08:23:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.134+0000 7f0c80ff9640 1 -- 172.21.15.22:0/69073036 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c6c031c30 con 0x7f0c84102570 2024-09-19T08:23:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.134+0000 7f0c80ff9640 1 -- 172.21.15.22:0/69073036 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c049060 con 0x7f0c84102570 2024-09-19T08:23:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.136+0000 7f0c80ff9640 1 -- 172.21.15.22:0/69073036 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0c6c042070 con 0x7f0c84102570 2024-09-19T08:23:02.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.136+0000 7f0c8b9d7740 1 Processor -- start 2024-09-19T08:23:02.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.136+0000 7f0c8b9d7740 1 -- start start 2024-09-19T08:23:02.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.137+0000 7f0c8b9d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb18850 0x55d23bb18c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.137+0000 7f0c8b9d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb19170 0x55d23bb234f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.137+0000 7f0c8b9d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb25e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.137+0000 7f0c8b9d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d23ba7d160 con 0x55d23bb18850 2024-09-19T08:23:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.137+0000 7f0c8b9d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d23ba24720 con 0x55d23bb23a30 2024-09-19T08:23:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.137+0000 7f0c8b9d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d23ba79120 con 0x55d23bb19170 2024-09-19T08:23:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.137+0000 7f0c82ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb19170 0x55d23bb234f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.137+0000 7f0c837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb18850 0x55d23bb18c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.137+0000 7f0c82ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb19170 0x55d23bb234f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:59508/0 (socket says 172.21.15.22:59508) 2024-09-19T08:23:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.137+0000 7f0c82ffd640 1 -- 172.21.15.22:0/3748290970 learned_addr learned my addr 172.21.15.22:0/3748290970 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.137+0000 7f0c80ff9640 1 --2- 172.21.15.22:0/69073036 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0c700a33b0 0x7f0c700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.137+0000 7f0c7bfff640 1 --2- 172.21.15.22:0/3748290970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb25e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.137+0000 7f0c837fe640 1 --2- 172.21.15.22:0/69073036 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0c700a33b0 0x7f0c700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.137+0000 7f0c82ffd640 1 -- 172.21.15.22:0/3748290970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 msgr2=0x55d23bb25e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.138+0000 7f0c82ffd640 1 --2- 172.21.15.22:0/3748290970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb25e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.138+0000 7f0c82ffd640 1 -- 172.21.15.22:0/3748290970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb18850 msgr2=0x55d23bb18c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.138+0000 7f0c82ffd640 1 --2- 172.21.15.22:0/3748290970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb18850 0x55d23bb18c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.138+0000 7f0c82ffd640 1 -- 172.21.15.22:0/3748290970 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d23b7eeba0 con 0x55d23bb19170 2024-09-19T08:23:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.138+0000 7f0c837fe640 1 --2- 172.21.15.22:0/3748290970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb18850 0x55d23bb18c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.138+0000 7f0c80ff9640 1 -- 172.21.15.22:0/69073036 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f0c6c0eff40 con 0x7f0c84102570 2024-09-19T08:23:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.138+0000 7f0c82ffd640 1 --2- 172.21.15.22:0/3748290970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb19170 0x55d23bb234f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009dc0 tx=0x7f0c74011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3988bc5499e6ade3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.138+0000 7f0c797fa640 1 -- 172.21.15.22:0/3748290970 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74005220 con 0x55d23bb19170 2024-09-19T08:23:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.138+0000 7f0c837fe640 1 --2- 172.21.15.22:0/69073036 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0c700a33b0 0x7f0c700a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009fd0 tx=0x7f0c7c004ab0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.138+0000 7f0c797fa640 1 -- 172.21.15.22:0/3748290970 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c74004c80 con 0x55d23bb19170 2024-09-19T08:23:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.138+0000 7f0c797fa640 1 -- 172.21.15.22:0/3748290970 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74019930 con 0x55d23bb19170 2024-09-19T08:23:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.139+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3748290970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb19170 msgr2=0x55d23bb234f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.139+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3748290970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb19170 0x55d23bb234f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009dc0 tx=0x7f0c74011810 comp rx=0 tx=0).stop 2024-09-19T08:23:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.139+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3748290970 shutdown_connections 2024-09-19T08:23:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.139+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3748290970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb25e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.139+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3748290970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb19170 0x55d23bb234f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.139+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3748290970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb18850 0x55d23bb18c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.139+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3748290970 >> 172.21.15.22:0/3748290970 conn(0x55d23bb14de0 msgr2=0x55d23bb151c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:02.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.139+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3748290970 shutdown_connections 2024-09-19T08:23:02.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.139+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3748290970 wait complete. 2024-09-19T08:23:02.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.139+0000 7f0c8b9d7740 1 -- start start 2024-09-19T08:23:02.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c8b9d7740 1 Processor -- start 2024-09-19T08:23:02.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c8b9d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb18850 0x55d23bb208b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c8b9d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb19170 0x55d23bb20df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c8b9d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb21330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c8b9d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d23bb26600 con 0x55d23bb18850 2024-09-19T08:23:02.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c8b9d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d23ba05900 con 0x55d23bb23a30 2024-09-19T08:23:02.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c8b9d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d23bb26780 con 0x55d23bb19170 2024-09-19T08:23:02.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c8b9d7740 10 client.0 _ll_register_callbacks cb 0x55d23ba03780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:02.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c82ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb18850 0x55d23bb208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c7bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb19170 0x55d23bb20df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c82ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb18850 0x55d23bb208b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35366/0 (socket says 172.21.15.22:35366) 2024-09-19T08:23:02.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c7bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb19170 0x55d23bb20df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:59514/0 (socket says 172.21.15.22:59514) 2024-09-19T08:23:02.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c82ffd640 1 -- 172.21.15.22:0/3795713868 learned_addr learned my addr 172.21.15.22:0/3795713868 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:02.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c837fe640 1 --2- 172.21.15.22:0/3795713868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb21330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c82ffd640 1 -- 172.21.15.22:0/3795713868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb19170 msgr2=0x55d23bb20df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c82ffd640 1 --2- 172.21.15.22:0/3795713868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb19170 0x55d23bb20df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c82ffd640 1 -- 172.21.15.22:0/3795713868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 msgr2=0x55d23bb21330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c82ffd640 1 --2- 172.21.15.22:0/3795713868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb21330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c82ffd640 1 -- 172.21.15.22:0/3795713868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d23ba45c50 con 0x55d23bb18850 2024-09-19T08:23:02.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.140+0000 7f0c7bfff640 1 --2- 172.21.15.22:0/3795713868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb19170 0x55d23bb20df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:02.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.141+0000 7f0c82ffd640 1 --2- 172.21.15.22:0/3795713868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb18850 0x55d23bb208b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0c74019510 tx=0x7f0c740048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6846b4f9d30dafd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:02.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.141+0000 7f0c6a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:02.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.141+0000 7f0c6a7fc640 1 -- 172.21.15.22:0/3795713868 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74027070 con 0x55d23bb18850 2024-09-19T08:23:02.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.141+0000 7f0c6a7fc640 1 -- 172.21.15.22:0/3795713868 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f0c74004450 con 0x55d23bb18850 2024-09-19T08:23:02.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.141+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3795713868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c74004c80 con 0x55d23bb18850 2024-09-19T08:23:02.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.141+0000 7f0c6a7fc640 1 -- 172.21.15.22:0/3795713868 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0c74033050 con 0x55d23bb18850 2024-09-19T08:23:02.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.141+0000 7f0c8b9d7740 10 client.16788 fetch_fsmap learned FSMap version 33 2024-09-19T08:23:02.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.141+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3795713868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d23bb32240 con 0x55d23bb18850 2024-09-19T08:23:02.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.141+0000 7f0c6a7fc640 1 -- 172.21.15.22:0/3795713868 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c74004c80 con 0x55d23bb18850 2024-09-19T08:23:02.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.142+0000 7f0c6a7fc640 1 -- 172.21.15.22:0/3795713868 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f0c74019be0 con 0x55d23bb18850 2024-09-19T08:23:02.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.142+0000 7f0c8b9d7740 10 client.16788 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:23:02.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.142+0000 7f0c8b9d7740 10 client.16788 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:23:02.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.142+0000 7f0c8b9d7740 20 client.16788 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:02.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.142+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3795713868 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d23bb33650 0x55d23bb35a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.142+0000 7f0c8b9d7740 4 client.16788 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:23:02.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.142+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3795713868 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d23ba8a890 con 0x55d23bb33650 2024-09-19T08:23:02.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.142+0000 7f0c7bfff640 1 --2- 172.21.15.22:0/3795713868 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d23bb33650 0x55d23bb35a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.142+0000 7f0c7bfff640 1 --2- 172.21.15.22:0/3795713868 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d23bb33650 0x55d23bb35a40 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fed65bf0db06be33 server_cookie=46d7a5eed44051b4 in_seq=0 out_seq=0 2024-09-19T08:23:02.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.142+0000 7f0c6a7fc640 10 client.16788 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:02.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c6a7fc640 1 -- 172.21.15.22:0/3795713868 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d23ba8a890 con 0x55d23bb33650 2024-09-19T08:23:02.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c6a7fc640 10 client.16788 handle_command_reply: tid=0 2024-09-19T08:23:02.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c8b9d7740 1 client.16788 shutdown 2024-09-19T08:23:02.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3795713868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb18850 msgr2=0x55d23bb208b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3795713868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb18850 0x55d23bb208b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0c74019510 tx=0x7f0c740048f0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3795713868 shutdown_connections 2024-09-19T08:23:02.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3795713868 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d23bb33650 0x55d23bb35a40 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3795713868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb21330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3795713868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb19170 0x55d23bb20df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c6a7fc640 0 client.16788 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:02.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3795713868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb18850 0x55d23bb208b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3795713868 >> 172.21.15.22:0/3795713868 conn(0x55d23bb14de0 msgr2=0x55d23bb1ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:02.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3795713868 shutdown_connections 2024-09-19T08:23:02.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3795713868 wait complete. 2024-09-19T08:23:02.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.143+0000 7f0c8b9d7740 20 client.16788 trim_cache size 0 max 16384 2024-09-19T08:23:02.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.163+0000 7f0c8b9d7740 1 Processor -- start 2024-09-19T08:23:02.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.163+0000 7f0c8b9d7740 1 -- start start 2024-09-19T08:23:02.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.164+0000 7f0c8b9d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb18850 0x55d23bb3b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.164+0000 7f0c8b9d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb19170 0x55d23bb3b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.164+0000 7f0c8b9d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb1db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.164+0000 7f0c8b9d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841b23f0 con 0x55d23bb19170 2024-09-19T08:23:02.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.164+0000 7f0c8b9d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841afc30 con 0x55d23bb23a30 2024-09-19T08:23:02.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.164+0000 7f0c8b9d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841adaa0 con 0x55d23bb18850 2024-09-19T08:23:02.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.164+0000 7f0c82ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb18850 0x55d23bb3b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.164+0000 7f0c7bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb19170 0x55d23bb3b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.164+0000 7f0c82ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb18850 0x55d23bb3b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:59524/0 (socket says 172.21.15.22:59524) 2024-09-19T08:23:02.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.164+0000 7f0c82ffd640 1 -- 172.21.15.22:0/2363543415 learned_addr learned my addr 172.21.15.22:0/2363543415 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:02.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.164+0000 7f0c837fe640 1 --2- 172.21.15.22:0/2363543415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb1db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.165+0000 7f0c7bfff640 1 -- 172.21.15.22:0/2363543415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb18850 msgr2=0x55d23bb3b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.165+0000 7f0c7bfff640 1 --2- 172.21.15.22:0/2363543415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb18850 0x55d23bb3b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.165+0000 7f0c7bfff640 1 -- 172.21.15.22:0/2363543415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 msgr2=0x55d23bb1db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.165+0000 7f0c7bfff640 1 --2- 172.21.15.22:0/2363543415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb1db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.165+0000 7f0c7bfff640 1 -- 172.21.15.22:0/2363543415 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d23bb18fa0 con 0x55d23bb19170 2024-09-19T08:23:02.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.165+0000 7f0c82ffd640 1 --2- 172.21.15.22:0/2363543415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb18850 0x55d23bb3b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:02.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.165+0000 7f0c837fe640 1 --2- 172.21.15.22:0/2363543415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb1db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:02.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.165+0000 7f0c7bfff640 1 --2- 172.21.15.22:0/2363543415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb19170 0x55d23bb3b7b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0ed680 tx=0x7f0c6c0b8610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375b4b5037eeb2da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:02.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.165+0000 7f0c78ff9640 1 -- 172.21.15.22:0/2363543415 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c0bc2c0 con 0x55d23bb19170 2024-09-19T08:23:02.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.165+0000 7f0c78ff9640 1 -- 172.21.15.22:0/2363543415 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c6c0dae50 con 0x55d23bb19170 2024-09-19T08:23:02.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.165+0000 7f0c78ff9640 1 -- 172.21.15.22:0/2363543415 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c0ba650 con 0x55d23bb19170 2024-09-19T08:23:02.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.166+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/2363543415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb19170 msgr2=0x55d23bb3b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.166+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/2363543415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb19170 0x55d23bb3b7b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0ed680 tx=0x7f0c6c0b8610 comp rx=0 tx=0).stop 2024-09-19T08:23:02.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.166+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/2363543415 shutdown_connections 2024-09-19T08:23:02.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.166+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/2363543415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb1db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.166+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/2363543415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb19170 0x55d23bb3b7b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.166+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/2363543415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb18850 0x55d23bb3b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.166+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/2363543415 >> 172.21.15.22:0/2363543415 conn(0x55d23bb14de0 msgr2=0x55d23bb22110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:02.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.166+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/2363543415 shutdown_connections 2024-09-19T08:23:02.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.166+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/2363543415 wait complete. 2024-09-19T08:23:02.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.167+0000 7f0c8b9d7740 1 -- start start 2024-09-19T08:23:02.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.167+0000 7f0c8b9d7740 1 Processor -- start 2024-09-19T08:23:02.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.167+0000 7f0c8b9d7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb18850 0x55d23bb1d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.167+0000 7f0c8b9d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb1def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.167+0000 7f0c8b9d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb1e430 0x55d23bb9d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.167+0000 7f0c8b9d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841ae0f0 con 0x55d23bb1e430 2024-09-19T08:23:02.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.167+0000 7f0c8b9d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841af560 con 0x55d23bb23a30 2024-09-19T08:23:02.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.167+0000 7f0c8b9d7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841ae7c0 con 0x55d23bb18850 2024-09-19T08:23:02.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.167+0000 7f0c8b9d7740 10 client.0 _ll_register_callbacks cb 0x7f0c7c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:02.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.167+0000 7f0c7bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb1def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.167+0000 7f0c82ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb18850 0x55d23bb1d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.167+0000 7f0c837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb1e430 0x55d23bb9d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.168+0000 7f0c82ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb18850 0x55d23bb1d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:59528/0 (socket says 172.21.15.22:59528) 2024-09-19T08:23:02.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.168+0000 7f0c837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb1e430 0x55d23bb9d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35388/0 (socket says 172.21.15.22:35388) 2024-09-19T08:23:02.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.168+0000 7f0c82ffd640 1 -- 172.21.15.22:0/3116291486 learned_addr learned my addr 172.21.15.22:0/3116291486 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.168+0000 7f0c7bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb1def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:50652/0 (socket says 172.21.15.22:50652) 2024-09-19T08:23:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.168+0000 7f0c7bfff640 1 -- 172.21.15.22:0/3116291486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb18850 msgr2=0x55d23bb1d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.168+0000 7f0c7bfff640 1 --2- 172.21.15.22:0/3116291486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb18850 0x55d23bb1d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.168+0000 7f0c7bfff640 1 -- 172.21.15.22:0/3116291486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb1e430 msgr2=0x55d23bb9d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.168+0000 7f0c7bfff640 1 --2- 172.21.15.22:0/3116291486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb1e430 0x55d23bb9d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.168+0000 7f0c7bfff640 1 -- 172.21.15.22:0/3116291486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d23bb9d7d0 con 0x55d23bb23a30 2024-09-19T08:23:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.168+0000 7f0c7bfff640 1 --2- 172.21.15.22:0/3116291486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb1def0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0ed680 tx=0x7f0c6c02a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777e2e081c092920 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.168+0000 7f0c6a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:23:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.168+0000 7f0c6a7fc640 1 -- 172.21.15.22:0/3116291486 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c0bc2c0 con 0x55d23bb23a30 2024-09-19T08:23:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.168+0000 7f0c6a7fc640 1 -- 172.21.15.22:0/3116291486 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f0c6c0ba360 con 0x55d23bb23a30 2024-09-19T08:23:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.168+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3116291486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c6c0dae50 con 0x55d23bb23a30 2024-09-19T08:23:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.169+0000 7f0c6a7fc640 1 -- 172.21.15.22:0/3116291486 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0c6c0ed900 con 0x55d23bb23a30 2024-09-19T08:23:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.169+0000 7f0c8b9d7740 10 client.26291 fetch_fsmap learned FSMap version 33 2024-09-19T08:23:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.169+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3116291486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d23bb9ea70 con 0x55d23bb23a30 2024-09-19T08:23:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.169+0000 7f0c6a7fc640 1 -- 172.21.15.22:0/3116291486 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c6c0dae50 con 0x55d23bb23a30 2024-09-19T08:23:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.169+0000 7f0c6a7fc640 1 -- 172.21.15.22:0/3116291486 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f0c6c04c060 con 0x55d23bb23a30 2024-09-19T08:23:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.169+0000 7f0c8b9d7740 10 client.26291 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:23:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.169+0000 7f0c8b9d7740 10 client.26291 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:23:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.169+0000 7f0c8b9d7740 20 client.26291 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.169+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3116291486 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d23bb9ef70 0x55d23bba1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.169+0000 7f0c8b9d7740 4 client.26291 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T08:23:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.169+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3116291486 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x55d23ba84be0 con 0x55d23bb9ef70 2024-09-19T08:23:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.169+0000 7f0c82ffd640 1 --2- 172.21.15.22:0/3116291486 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d23bb9ef70 0x55d23bba1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.170+0000 7f0c82ffd640 1 --2- 172.21.15.22:0/3116291486 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d23bb9ef70 0x55d23bba1390 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc540485eea732b server_cookie=74bc234704714782 in_seq=0 out_seq=0 2024-09-19T08:23:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.170+0000 7f0c6a7fc640 10 client.26291 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:02.226 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:23:02.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.707+0000 7f433b7f6640 1 -- 172.21.15.22:0/671109777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4334101de0 msgr2=0x7f43341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.707+0000 7f433b7f6640 1 --2- 172.21.15.22:0/671109777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4334101de0 0x7f43341021c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4330009900 tx=0x7f433002a410 comp rx=0 tx=0).stop 2024-09-19T08:23:02.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.707+0000 7f433b7f6640 1 -- 172.21.15.22:0/671109777 shutdown_connections 2024-09-19T08:23:02.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.707+0000 7f433b7f6640 1 --2- 172.21.15.22:0/671109777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4334106f10 0x7f433410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.707+0000 7f433b7f6640 1 --2- 172.21.15.22:0/671109777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4334102790 0x7f43341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.707+0000 7f433b7f6640 1 --2- 172.21.15.22:0/671109777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4334101de0 0x7f43341021c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.707+0000 7f433b7f6640 1 -- 172.21.15.22:0/671109777 >> 172.21.15.22:0/671109777 conn(0x7f43340fd8c0 msgr2=0x7f43340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.708+0000 7f433b7f6640 1 -- 172.21.15.22:0/671109777 shutdown_connections 2024-09-19T08:23:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.708+0000 7f433b7f6640 1 -- 172.21.15.22:0/671109777 wait complete. 2024-09-19T08:23:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.708+0000 7f433b7f6640 1 Processor -- start 2024-09-19T08:23:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.708+0000 7f433b7f6640 1 -- start start 2024-09-19T08:23:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f433b7f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4334101de0 0x7f43341984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f433b7f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4334102790 0x7f4334198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f433b7f6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4334106f10 0x7f433419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f433b7f6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4334111f70 con 0x7f4334102790 2024-09-19T08:23:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f433b7f6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4334111df0 con 0x7f4334101de0 2024-09-19T08:23:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f433b7f6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f43341120f0 con 0x7f4334106f10 2024-09-19T08:23:02.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f4338d91640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4334102790 0x7f4334198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f4339592640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4334101de0 0x7f43341984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f4338d91640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4334102790 0x7f4334198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35414/0 (socket says 172.21.15.22:35414) 2024-09-19T08:23:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f4338d91640 1 -- 172.21.15.22:0/1026226605 learned_addr learned my addr 172.21.15.22:0/1026226605 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f4339d93640 1 --2- 172.21.15.22:0/1026226605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4334106f10 0x7f433419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f4338d91640 1 -- 172.21.15.22:0/1026226605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4334106f10 msgr2=0x7f433419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f4338d91640 1 --2- 172.21.15.22:0/1026226605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4334106f10 0x7f433419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f4338d91640 1 -- 172.21.15.22:0/1026226605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4334101de0 msgr2=0x7f43341984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f4338d91640 1 --2- 172.21.15.22:0/1026226605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4334101de0 0x7f43341984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f4338d91640 1 -- 172.21.15.22:0/1026226605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433419d480 con 0x7f4334102790 2024-09-19T08:23:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f4338d91640 1 --2- 172.21.15.22:0/1026226605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4334102790 0x7f4334198a10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f432400d950 tx=0x7f432400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f432a7fc640 1 -- 172.21.15.22:0/1026226605 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324017030 con 0x7f4334102790 2024-09-19T08:23:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f432a7fc640 1 -- 172.21.15.22:0/1026226605 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4324004020 con 0x7f4334102790 2024-09-19T08:23:02.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f433b7f6640 1 -- 172.21.15.22:0/1026226605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433419d770 con 0x7f4334102790 2024-09-19T08:23:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f432a7fc640 1 -- 172.21.15.22:0/1026226605 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43240047a0 con 0x7f4334102790 2024-09-19T08:23:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.709+0000 7f433b7f6640 1 -- 172.21.15.22:0/1026226605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43341a5050 con 0x7f4334102790 2024-09-19T08:23:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f433dbcf740 1 Processor -- start 2024-09-19T08:23:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f433dbcf740 1 -- start start 2024-09-19T08:23:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f432a7fc640 1 -- 172.21.15.22:0/1026226605 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f43240041d0 con 0x7f4334102790 2024-09-19T08:23:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f433dbcf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad32880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f433dbcf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad32dc0 0x55f39ad3d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f433dbcf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad3d680 0x55f39ad3fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f433dbcf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f39ac982e0 con 0x55f39ad3d680 2024-09-19T08:23:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f433dbcf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f39ac3fce0 con 0x55f39ad32dc0 2024-09-19T08:23:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f433dbcf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f39ac942a0 con 0x55f39ad324a0 2024-09-19T08:23:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f4338d91640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad3d680 0x55f39ad3fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f4339d93640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad32880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f4339d93640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad32880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:59556/0 (socket says 172.21.15.22:59556) 2024-09-19T08:23:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f4339d93640 1 -- 172.21.15.22:0/3606038697 learned_addr learned my addr 172.21.15.22:0/3606038697 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f4339592640 1 --2- 172.21.15.22:0/3606038697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad32dc0 0x55f39ad3d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f4339d93640 1 -- 172.21.15.22:0/3606038697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad32dc0 msgr2=0x55f39ad3d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f4339d93640 1 --2- 172.21.15.22:0/3606038697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad32dc0 0x55f39ad3d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f4339d93640 1 -- 172.21.15.22:0/3606038697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad3d680 msgr2=0x55f39ad3fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f4339d93640 1 --2- 172.21.15.22:0/3606038697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad3d680 0x55f39ad3fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f4339d93640 1 -- 172.21.15.22:0/3606038697 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f39ac24440 con 0x55f39ad324a0 2024-09-19T08:23:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f4339592640 1 --2- 172.21.15.22:0/3606038697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad32dc0 0x55f39ad3d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:23:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.713+0000 7f4338d91640 1 --2- 172.21.15.22:0/3606038697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad3d680 0x55f39ad3fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.714+0000 7f432a7fc640 1 --2- 172.21.15.22:0/1026226605 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f43200a3430 0x7f43200a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.714+0000 7f4339d93640 1 --2- 172.21.15.22:0/3606038697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad32880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f432c009f50 tx=0x7f432c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=408d8a21169a195a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.714+0000 7f4339592640 1 --2- 172.21.15.22:0/1026226605 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f43200a3430 0x7f43200a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.714+0000 7f431f7fe640 1 -- 172.21.15.22:0/3606038697 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c004d20 con 0x55f39ad324a0 2024-09-19T08:23:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.714+0000 7f432a7fc640 1 -- 172.21.15.22:0/1026226605 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f43240cf610 con 0x7f4334102790 2024-09-19T08:23:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.714+0000 7f431f7fe640 1 -- 172.21.15.22:0/3606038697 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f432c004050 con 0x55f39ad324a0 2024-09-19T08:23:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.714+0000 7f431f7fe640 1 -- 172.21.15.22:0/3606038697 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c005220 con 0x55f39ad324a0 2024-09-19T08:23:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.715+0000 7f433dbcf740 1 -- 172.21.15.22:0/3606038697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 msgr2=0x55f39ad32880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.715+0000 7f433dbcf740 1 --2- 172.21.15.22:0/3606038697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad32880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f432c009f50 tx=0x7f432c0118d0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.715+0000 7f4339592640 1 --2- 172.21.15.22:0/1026226605 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f43200a3430 0x7f43200a58f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4330009230 tx=0x7f43300058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.715+0000 7f433dbcf740 1 -- 172.21.15.22:0/3606038697 shutdown_connections 2024-09-19T08:23:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.715+0000 7f433dbcf740 1 --2- 172.21.15.22:0/3606038697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad3d680 0x55f39ad3fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.715+0000 7f433dbcf740 1 --2- 172.21.15.22:0/3606038697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad32dc0 0x55f39ad3d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.715+0000 7f433dbcf740 1 --2- 172.21.15.22:0/3606038697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad32880 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.715+0000 7f433dbcf740 1 -- 172.21.15.22:0/3606038697 >> 172.21.15.22:0/3606038697 conn(0x55f39ad2ea30 msgr2=0x55f39ad2ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.715+0000 7f433dbcf740 1 -- 172.21.15.22:0/3606038697 shutdown_connections 2024-09-19T08:23:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.715+0000 7f433dbcf740 1 -- 172.21.15.22:0/3606038697 wait complete. 2024-09-19T08:23:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.715+0000 7f433dbcf740 1 -- start start 2024-09-19T08:23:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.715+0000 7f433dbcf740 1 Processor -- start 2024-09-19T08:23:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f433dbcf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad324a0 0x55f39ad3cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f433dbcf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad32dc0 0x55f39ad3a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f433dbcf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad3d680 0x55f39ad3a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f433dbcf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f39ad40250 con 0x55f39ad324a0 2024-09-19T08:23:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f433dbcf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f39ac53050 con 0x55f39ad3d680 2024-09-19T08:23:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f433dbcf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f39ad403d0 con 0x55f39ad32dc0 2024-09-19T08:23:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f433dbcf740 10 client.0 _ll_register_callbacks cb 0x55f39ac93160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f4339592640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad324a0 0x55f39ad3cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f4339592640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad324a0 0x55f39ad3cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35434/0 (socket says 172.21.15.22:35434) 2024-09-19T08:23:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f4339592640 1 -- 172.21.15.22:0/2636308548 learned_addr learned my addr 172.21.15.22:0/2636308548 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f4338d91640 1 --2- 172.21.15.22:0/2636308548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad32dc0 0x55f39ad3a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f4339d93640 1 --2- 172.21.15.22:0/2636308548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad3d680 0x55f39ad3a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f4339592640 1 -- 172.21.15.22:0/2636308548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad32dc0 msgr2=0x55f39ad3a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f4339592640 1 --2- 172.21.15.22:0/2636308548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad32dc0 0x55f39ad3a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f4339592640 1 -- 172.21.15.22:0/2636308548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad3d680 msgr2=0x55f39ad3a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f4339592640 1 --2- 172.21.15.22:0/2636308548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad3d680 0x55f39ad3a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f4339592640 1 -- 172.21.15.22:0/2636308548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f39aa093c0 con 0x55f39ad324a0 2024-09-19T08:23:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f4338d91640 1 --2- 172.21.15.22:0/2636308548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad32dc0 0x55f39ad3a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f4339592640 1 --2- 172.21.15.22:0/2636308548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad324a0 0x55f39ad3cf90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4330009900 tx=0x7f4330009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1093a46657faeaa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f431cff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.716+0000 7f431cff9640 1 -- 172.21.15.22:0/2636308548 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433004d070 con 0x55f39ad324a0 2024-09-19T08:23:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.717+0000 7f431cff9640 1 -- 172.21.15.22:0/2636308548 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f43300037f0 con 0x55f39ad324a0 2024-09-19T08:23:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.717+0000 7f433dbcf740 1 -- 172.21.15.22:0/2636308548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f432c004050 con 0x55f39ad324a0 2024-09-19T08:23:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.717+0000 7f431cff9640 1 -- 172.21.15.22:0/2636308548 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4330059050 con 0x55f39ad324a0 2024-09-19T08:23:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.717+0000 7f433dbcf740 10 client.16806 fetch_fsmap learned FSMap version 33 2024-09-19T08:23:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.717+0000 7f433dbcf740 1 -- 172.21.15.22:0/2636308548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f39ad4bbd0 con 0x55f39ad324a0 2024-09-19T08:23:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.717+0000 7f431cff9640 1 -- 172.21.15.22:0/2636308548 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f432c004050 con 0x55f39ad324a0 2024-09-19T08:23:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.717+0000 7f431cff9640 1 -- 172.21.15.22:0/2636308548 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f4330049a50 con 0x55f39ad324a0 2024-09-19T08:23:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.717+0000 7f433dbcf740 10 client.16806 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:23:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.717+0000 7f433dbcf740 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:23:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.717+0000 7f433dbcf740 20 client.16806 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.717+0000 7f433dbcf740 1 --2- 172.21.15.22:0/2636308548 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f39ad4cfe0 0x55f39ad4f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.717+0000 7f433dbcf740 4 client.16806 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:23:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.717+0000 7f433dbcf740 1 -- 172.21.15.22:0/2636308548 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f39aca5850 con 0x55f39ad4cfe0 2024-09-19T08:23:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.717+0000 7f4338d91640 1 --2- 172.21.15.22:0/2636308548 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f39ad4cfe0 0x55f39ad4f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.718+0000 7f431cff9640 10 client.16806 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.718+0000 7f4338d91640 1 --2- 172.21.15.22:0/2636308548 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f39ad4cfe0 0x55f39ad4f460 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55f09b7af71fcb4 server_cookie=fcee685a9c5490e3 in_seq=0 out_seq=0 2024-09-19T08:23:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.718+0000 7f431cff9640 1 -- 172.21.15.22:0/2636308548 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f39aca5850 con 0x55f39ad4cfe0 2024-09-19T08:23:02.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.718+0000 7f431cff9640 10 client.16806 handle_command_reply: tid=0 2024-09-19T08:23:02.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.718+0000 7f433dbcf740 1 client.16806 shutdown 2024-09-19T08:23:02.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.719+0000 7f433dbcf740 1 -- 172.21.15.22:0/2636308548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad324a0 msgr2=0x55f39ad3cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.719+0000 7f433dbcf740 1 --2- 172.21.15.22:0/2636308548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad324a0 0x55f39ad3cf90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4330009900 tx=0x7f4330009260 comp rx=0 tx=0).stop 2024-09-19T08:23:02.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.719+0000 7f433dbcf740 1 -- 172.21.15.22:0/2636308548 shutdown_connections 2024-09-19T08:23:02.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.719+0000 7f433dbcf740 1 --2- 172.21.15.22:0/2636308548 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f39ad4cfe0 0x55f39ad4f460 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.719+0000 7f433dbcf740 1 --2- 172.21.15.22:0/2636308548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad3d680 0x55f39ad3a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.719+0000 7f431cff9640 0 client.16806 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:02.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.719+0000 7f433dbcf740 1 --2- 172.21.15.22:0/2636308548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad32dc0 0x55f39ad3a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.719+0000 7f433dbcf740 1 --2- 172.21.15.22:0/2636308548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad324a0 0x55f39ad3cf90 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.719+0000 7f433dbcf740 1 -- 172.21.15.22:0/2636308548 >> 172.21.15.22:0/2636308548 conn(0x55f39ad2ea30 msgr2=0x55f39ad34710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:02.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.719+0000 7f433dbcf740 1 -- 172.21.15.22:0/2636308548 shutdown_connections 2024-09-19T08:23:02.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.719+0000 7f433dbcf740 1 -- 172.21.15.22:0/2636308548 wait complete. 2024-09-19T08:23:02.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.719+0000 7f433dbcf740 20 client.16806 trim_cache size 0 max 16384 2024-09-19T08:23:02.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:02 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:23:02.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:02 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:02.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:02 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:23:02.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:02 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:02.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:02 smithi022 ceph-mon[30963]: pgmap v245: 97 pgs: 97 active+clean; 999 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 679 KiB/s wr, 205 op/s 2024-09-19T08:23:02.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:02 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:23:02.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:02 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:02.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.740+0000 7f433dbcf740 1 Processor -- start 2024-09-19T08:23:02.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.740+0000 7f433dbcf740 1 -- start start 2024-09-19T08:23:02.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.740+0000 7f433dbcf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad2ea30 0x55f39ad2da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.740+0000 7f433dbcf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad54d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.740+0000 7f433dbcf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad32dc0 0x55f39ad552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.740+0000 7f433dbcf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f43341b0280 con 0x55f39ad32dc0 2024-09-19T08:23:02.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.740+0000 7f433dbcf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43341adac0 con 0x55f39ad2ea30 2024-09-19T08:23:02.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.740+0000 7f433dbcf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f43341ab930 con 0x55f39ad324a0 2024-09-19T08:23:02.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.740+0000 7f4339d93640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad32dc0 0x55f39ad552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.740+0000 7f4338d91640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad54d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.740+0000 7f4339d93640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad32dc0 0x55f39ad552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35450/0 (socket says 172.21.15.22:35450) 2024-09-19T08:23:02.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.740+0000 7f4339d93640 1 -- 172.21.15.22:0/656806973 learned_addr learned my addr 172.21.15.22:0/656806973 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:02.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.740+0000 7f4339592640 1 --2- 172.21.15.22:0/656806973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad2ea30 0x55f39ad2da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.741+0000 7f4338d91640 1 -- 172.21.15.22:0/656806973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad2ea30 msgr2=0x55f39ad2da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.741+0000 7f4338d91640 1 --2- 172.21.15.22:0/656806973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad2ea30 0x55f39ad2da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.741+0000 7f4338d91640 1 -- 172.21.15.22:0/656806973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad32dc0 msgr2=0x55f39ad552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.741+0000 7f4338d91640 1 --2- 172.21.15.22:0/656806973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad32dc0 0x55f39ad552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.741+0000 7f4338d91640 1 -- 172.21.15.22:0/656806973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f39ad3d300 con 0x55f39ad324a0 2024-09-19T08:23:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.741+0000 7f4339d93640 1 --2- 172.21.15.22:0/656806973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad32dc0 0x55f39ad552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.741+0000 7f4338d91640 1 --2- 172.21.15.22:0/656806973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad54d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f43240ccd20 tx=0x7f43240d5550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30e093890c6a7609 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.741+0000 7f431effd640 1 -- 172.21.15.22:0/656806973 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432409b6b0 con 0x55f39ad324a0 2024-09-19T08:23:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.741+0000 7f431effd640 1 -- 172.21.15.22:0/656806973 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f432409bfe0 con 0x55f39ad324a0 2024-09-19T08:23:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.741+0000 7f431effd640 1 -- 172.21.15.22:0/656806973 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432409b240 con 0x55f39ad324a0 2024-09-19T08:23:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.742+0000 7f433dbcf740 1 -- 172.21.15.22:0/656806973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 msgr2=0x55f39ad54d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.742+0000 7f433dbcf740 1 --2- 172.21.15.22:0/656806973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad54d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f43240ccd20 tx=0x7f43240d5550 comp rx=0 tx=0).stop 2024-09-19T08:23:02.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.742+0000 7f433dbcf740 1 -- 172.21.15.22:0/656806973 shutdown_connections 2024-09-19T08:23:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.742+0000 7f433dbcf740 1 --2- 172.21.15.22:0/656806973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad32dc0 0x55f39ad552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.742+0000 7f433dbcf740 1 --2- 172.21.15.22:0/656806973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad54d80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.742+0000 7f433dbcf740 1 --2- 172.21.15.22:0/656806973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad2ea30 0x55f39ad2da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.742+0000 7f433dbcf740 1 -- 172.21.15.22:0/656806973 >> 172.21.15.22:0/656806973 conn(0x55f39ad3a8e0 msgr2=0x55f39ad2d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.742+0000 7f433dbcf740 1 -- 172.21.15.22:0/656806973 shutdown_connections 2024-09-19T08:23:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.742+0000 7f433dbcf740 1 -- 172.21.15.22:0/656806973 wait complete. 2024-09-19T08:23:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.742+0000 7f433dbcf740 1 -- start start 2024-09-19T08:23:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.743+0000 7f433dbcf740 1 Processor -- start 2024-09-19T08:23:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.743+0000 7f433dbcf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad2ea30 0x55f39ad45e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.743+0000 7f433dbcf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad43190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.743+0000 7f433dbcf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad32dc0 0x55f39ad43730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.743+0000 7f433dbcf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f43341abf80 con 0x55f39ad2ea30 2024-09-19T08:23:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.743+0000 7f433dbcf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43341ad3f0 con 0x55f39ad32dc0 2024-09-19T08:23:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.743+0000 7f433dbcf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f43341ac650 con 0x55f39ad324a0 2024-09-19T08:23:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.743+0000 7f4339592640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad2ea30 0x55f39ad45e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.743+0000 7f4339592640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad2ea30 0x55f39ad45e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35456/0 (socket says 172.21.15.22:35456) 2024-09-19T08:23:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.743+0000 7f4339592640 1 -- 172.21.15.22:0/2452320992 learned_addr learned my addr 172.21.15.22:0/2452320992 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.743+0000 7f4338d91640 1 --2- 172.21.15.22:0/2452320992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad43190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.743+0000 7f433dbcf740 10 client.0 _ll_register_callbacks cb 0x7f4330009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.743+0000 7f4339d93640 1 --2- 172.21.15.22:0/2452320992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad32dc0 0x55f39ad43730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.744+0000 7f4339592640 1 -- 172.21.15.22:0/2452320992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 msgr2=0x55f39ad43190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.744+0000 7f4339592640 1 --2- 172.21.15.22:0/2452320992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad43190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.744+0000 7f4339592640 1 -- 172.21.15.22:0/2452320992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad32dc0 msgr2=0x55f39ad43730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.744+0000 7f4339592640 1 --2- 172.21.15.22:0/2452320992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad32dc0 0x55f39ad43730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.744+0000 7f4339592640 1 -- 172.21.15.22:0/2452320992 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f39ad43f60 con 0x55f39ad2ea30 2024-09-19T08:23:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.744+0000 7f4339d93640 1 --2- 172.21.15.22:0/2452320992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad32dc0 0x55f39ad43730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:23:02.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.744+0000 7f4339592640 1 --2- 172.21.15.22:0/2452320992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad2ea30 0x55f39ad45e40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f43300096c0 tx=0x7f433003c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e2a12f79fe499d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:02.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.744+0000 7f431cff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:02.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.744+0000 7f431cff9640 1 -- 172.21.15.22:0/2452320992 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433004d040 con 0x55f39ad2ea30 2024-09-19T08:23:02.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.744+0000 7f431cff9640 1 -- 172.21.15.22:0/2452320992 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f433003cd40 con 0x55f39ad2ea30 2024-09-19T08:23:02.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.744+0000 7f433dbcf740 1 -- 172.21.15.22:0/2452320992 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f432409bfe0 con 0x55f39ad2ea30 2024-09-19T08:23:02.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.744+0000 7f431cff9640 1 -- 172.21.15.22:0/2452320992 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f433006c050 con 0x55f39ad2ea30 2024-09-19T08:23:02.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.744+0000 7f433dbcf740 10 client.16812 fetch_fsmap learned FSMap version 33 2024-09-19T08:23:02.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.744+0000 7f433dbcf740 1 -- 172.21.15.22:0/2452320992 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f39adbc230 con 0x55f39ad2ea30 2024-09-19T08:23:02.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.745+0000 7f431cff9640 1 -- 172.21.15.22:0/2452320992 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f432409bfe0 con 0x55f39ad2ea30 2024-09-19T08:23:02.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.745+0000 7f431cff9640 1 -- 172.21.15.22:0/2452320992 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f4330003710 con 0x55f39ad2ea30 2024-09-19T08:23:02.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.745+0000 7f433dbcf740 10 client.16812 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:23:02.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.745+0000 7f433dbcf740 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:23:02.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.745+0000 7f433dbcf740 20 client.16812 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:02.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.745+0000 7f433dbcf740 1 --2- 172.21.15.22:0/2452320992 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f39ad3d680 0x55f39adbea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:02.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.745+0000 7f433dbcf740 4 client.16812 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:23:02.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.745+0000 7f433dbcf740 1 -- 172.21.15.22:0/2452320992 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f39ac9fd60 con 0x55f39ad3d680 2024-09-19T08:23:02.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.745+0000 7f4338d91640 1 --2- 172.21.15.22:0/2452320992 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f39ad3d680 0x55f39adbea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:02.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f431cff9640 10 client.16812 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:02.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f4338d91640 1 --2- 172.21.15.22:0/2452320992 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f39ad3d680 0x55f39adbea40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85068623ca8944c1 server_cookie=2050907b91399aa5 in_seq=0 out_seq=0 2024-09-19T08:23:02.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f431cff9640 1 -- 172.21.15.22:0/2452320992 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f39ac9fd60 con 0x55f39ad3d680 2024-09-19T08:23:02.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f431cff9640 10 client.16812 handle_command_reply: tid=0 2024-09-19T08:23:02.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f433dbcf740 1 client.16812 shutdown 2024-09-19T08:23:02.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f433dbcf740 1 -- 172.21.15.22:0/2452320992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad2ea30 msgr2=0x55f39ad45e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f433dbcf740 1 --2- 172.21.15.22:0/2452320992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad2ea30 0x55f39ad45e40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f43300096c0 tx=0x7f433003c6a0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f433dbcf740 1 -- 172.21.15.22:0/2452320992 shutdown_connections 2024-09-19T08:23:02.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f433dbcf740 1 --2- 172.21.15.22:0/2452320992 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55f39ad3d680 0x55f39adbea40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f433dbcf740 1 --2- 172.21.15.22:0/2452320992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f39ad32dc0 0x55f39ad43730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f433dbcf740 1 --2- 172.21.15.22:0/2452320992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f39ad324a0 0x55f39ad43190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f431cff9640 0 client.16812 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:02.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f433dbcf740 1 --2- 172.21.15.22:0/2452320992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f39ad2ea30 0x55f39ad45e40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f433dbcf740 1 -- 172.21.15.22:0/2452320992 >> 172.21.15.22:0/2452320992 conn(0x55f39ad3a8e0 msgr2=0x55f39ad2d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:02.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f433dbcf740 1 -- 172.21.15.22:0/2452320992 shutdown_connections 2024-09-19T08:23:02.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f433dbcf740 1 -- 172.21.15.22:0/2452320992 wait complete. 2024-09-19T08:23:02.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.746+0000 7f433dbcf740 20 client.16812 trim_cache size 0 max 16384 2024-09-19T08:23:02.761 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:23:02.761 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:23:02.761 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-19T08:23:02.761 INFO:teuthology.orchestra.run.smithi022.stdout: "775f4f07-dc6d-46c4-b861-4aa631c480d4": { 2024-09-19T08:23:02.761 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-19T08:23:02.761 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "775f4f07-dc6d-46c4-b861-4aa631c480d4", 2024-09-19T08:23:02.761 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-09-19T08:23:02.761 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:23:02.762 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:23:02.762 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:23:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.747+0000 7f433b7f6640 1 -- 172.21.15.22:0/1026226605 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f43200a3430 msgr2=0x7f43200a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.747+0000 7f433b7f6640 1 --2- 172.21.15.22:0/1026226605 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f43200a3430 0x7f43200a58f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4330009230 tx=0x7f43300058d0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.747+0000 7f433b7f6640 1 -- 172.21.15.22:0/1026226605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4334102790 msgr2=0x7f4334198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.747+0000 7f433b7f6640 1 --2- 172.21.15.22:0/1026226605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4334102790 0x7f4334198a10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f432400d950 tx=0x7f432400de10 comp rx=0 tx=0).stop 2024-09-19T08:23:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.747+0000 7f433b7f6640 1 -- 172.21.15.22:0/1026226605 shutdown_connections 2024-09-19T08:23:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.747+0000 7f433b7f6640 1 --2- 172.21.15.22:0/1026226605 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f43200a3430 0x7f43200a58f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.747+0000 7f433b7f6640 1 --2- 172.21.15.22:0/1026226605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4334106f10 0x7f433419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.747+0000 7f433b7f6640 1 --2- 172.21.15.22:0/1026226605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4334102790 0x7f4334198a10 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.747+0000 7f433b7f6640 1 --2- 172.21.15.22:0/1026226605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4334101de0 0x7f43341984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:02.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.747+0000 7f433b7f6640 1 -- 172.21.15.22:0/1026226605 >> 172.21.15.22:0/1026226605 conn(0x7f43340fd8c0 msgr2=0x7f43340fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:02.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.747+0000 7f433b7f6640 1 -- 172.21.15.22:0/1026226605 shutdown_connections 2024-09-19T08:23:02.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:02.747+0000 7f433b7f6640 1 -- 172.21.15.22:0/1026226605 wait complete. 2024-09-19T08:23:03.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:02 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:23:03.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:02 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:03.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:02 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:23:03.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:02 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:03.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:02 smithi157 ceph-mon[35143]: pgmap v245: 97 pgs: 97 active+clean; 999 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 679 KiB/s wr, 205 op/s 2024-09-19T08:23:03.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:02 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:23:03.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:02 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:03.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:02 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:23:03.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:02 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:03.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:02 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:23:03.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:02 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:03.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:02 smithi064 ceph-mon[34900]: pgmap v245: 97 pgs: 97 active+clean; 999 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 679 KiB/s wr, 205 op/s 2024-09-19T08:23:03.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:02 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:23:03.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:02 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:03.565 INFO:tasks.cephfs.filesystem:scrub status for tag:775f4f07-dc6d-46c4-b861-4aa631c480d4 - {'path': '/', 'tag': '775f4f07-dc6d-46c4-b861-4aa631c480d4', 'options': 'recursive,force'} 2024-09-19T08:23:05.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:04 smithi157 ceph-mon[35143]: pgmap v246: 97 pgs: 97 active+clean; 1013 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 584 op/s 2024-09-19T08:23:05.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:04 smithi022 ceph-mon[30963]: pgmap v246: 97 pgs: 97 active+clean; 1013 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 584 op/s 2024-09-19T08:23:05.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:04 smithi064 ceph-mon[34900]: pgmap v246: 97 pgs: 97 active+clean; 1013 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 584 op/s 2024-09-19T08:23:07.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:06 smithi157 ceph-mon[35143]: pgmap v247: 97 pgs: 97 active+clean; 1013 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 438 op/s 2024-09-19T08:23:07.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:06 smithi022 ceph-mon[30963]: pgmap v247: 97 pgs: 97 active+clean; 1013 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 438 op/s 2024-09-19T08:23:07.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:06 smithi064 ceph-mon[34900]: pgmap v247: 97 pgs: 97 active+clean; 1013 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 438 op/s 2024-09-19T08:23:07.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.325+0000 7f0c6a7fc640 1 -- 172.21.15.22:0/3116291486 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d23ba84be0 con 0x55d23bb9ef70 2024-09-19T08:23:07.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.325+0000 7f0c6a7fc640 10 client.26291 handle_command_reply: tid=0 2024-09-19T08:23:07.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.325+0000 7f0c8b9d7740 1 client.26291 shutdown 2024-09-19T08:23:07.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.326+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3116291486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 msgr2=0x55d23bb1def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:07.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.326+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3116291486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb1def0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0ed680 tx=0x7f0c6c02a480 comp rx=0 tx=0).stop 2024-09-19T08:23:07.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.326+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3116291486 shutdown_connections 2024-09-19T08:23:07.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.326+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3116291486 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d23bb9ef70 0x55d23bba1390 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:07.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.326+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3116291486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d23bb1e430 0x55d23bb9d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:07.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.326+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3116291486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d23bb23a30 0x55d23bb1def0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:07.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.326+0000 7f0c8b9d7740 1 --2- 172.21.15.22:0/3116291486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d23bb18850 0x55d23bb1d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:07.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.326+0000 7f0c6a7fc640 0 client.26291 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:07.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.326+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3116291486 >> 172.21.15.22:0/3116291486 conn(0x55d23bb14de0 msgr2=0x55d23bb20850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:07.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.326+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3116291486 shutdown_connections 2024-09-19T08:23:07.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.326+0000 7f0c8b9d7740 1 -- 172.21.15.22:0/3116291486 wait complete. 2024-09-19T08:23:07.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.326+0000 7f0c8b9d7740 20 client.26291 trim_cache size 0 max 16384 2024-09-19T08:23:07.328 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:23:07.328 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 33, 2024-09-19T08:23:07.328 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:23:07.328 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 30, 2024-09-19T08:23:07.328 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:23:07.328 INFO:teuthology.orchestra.run.smithi022.stdout: "ee82e705": { 2024-09-19T08:23:07.328 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 30, 2024-09-19T08:23:07.328 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:23:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:23:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:23:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:23:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:23:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:23:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 145.8, 2024-09-19T08:23:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:23:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:23:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:23:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:23:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "QUIESCED", 2024-09-19T08:23:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:23:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:23:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:23:07.330 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:23:07.330 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:23:07.330 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:23:07.330 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:23:07.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.327+0000 7f0c895fe640 1 -- 172.21.15.22:0/69073036 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0c700a33b0 msgr2=0x7f0c700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:07.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.327+0000 7f0c895fe640 1 --2- 172.21.15.22:0/69073036 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0c700a33b0 0x7f0c700a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009fd0 tx=0x7f0c7c004ab0 comp rx=0 tx=0).stop 2024-09-19T08:23:07.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.327+0000 7f0c895fe640 1 -- 172.21.15.22:0/69073036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c84102570 msgr2=0x7f0c84198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:07.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.327+0000 7f0c895fe640 1 --2- 172.21.15.22:0/69073036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c84102570 0x7f0c84198b00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c005940 tx=0x7f0c6c003c10 comp rx=0 tx=0).stop 2024-09-19T08:23:07.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.327+0000 7f0c895fe640 1 -- 172.21.15.22:0/69073036 shutdown_connections 2024-09-19T08:23:07.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.327+0000 7f0c895fe640 1 --2- 172.21.15.22:0/69073036 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0c700a33b0 0x7f0c700a5870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:07.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.327+0000 7f0c895fe640 1 --2- 172.21.15.22:0/69073036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c84102ef0 0x7f0c8419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:07.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.327+0000 7f0c895fe640 1 --2- 172.21.15.22:0/69073036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c84102570 0x7f0c84198b00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:07.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.327+0000 7f0c895fe640 1 --2- 172.21.15.22:0/69073036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c84100e20 0x7f0c841985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:07.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.327+0000 7f0c895fe640 1 -- 172.21.15.22:0/69073036 >> 172.21.15.22:0/69073036 conn(0x7f0c840fc7d0 msgr2=0x7f0c8406a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:07.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.328+0000 7f0c895fe640 1 -- 172.21.15.22:0/69073036 shutdown_connections 2024-09-19T08:23:07.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:07.329+0000 7f0c895fe640 1 -- 172.21.15.22:0/69073036 wait complete. 2024-09-19T08:23:08.111 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ee82e705', quiesce took 9.4 seconds. Will release after: 16.35 2024-09-19T08:23:09.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:08 smithi157 ceph-mon[35143]: pgmap v248: 97 pgs: 97 active+clean; 1014 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 459 op/s 2024-09-19T08:23:09.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:08 smithi157 ceph-mon[35143]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T08:23:09.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:08 smithi022 ceph-mon[30963]: pgmap v248: 97 pgs: 97 active+clean; 1014 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 459 op/s 2024-09-19T08:23:09.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:08 smithi022 ceph-mon[30963]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T08:23:09.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:08 smithi064 ceph-mon[34900]: pgmap v248: 97 pgs: 97 active+clean; 1014 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 459 op/s 2024-09-19T08:23:09.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:08 smithi064 ceph-mon[34900]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T08:23:11.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:10 smithi157 ceph-mon[35143]: pgmap v250: 97 pgs: 97 active+clean; 1014 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 638 op/s 2024-09-19T08:23:11.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:10 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:11.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:10 smithi022 ceph-mon[30963]: pgmap v250: 97 pgs: 97 active+clean; 1014 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 638 op/s 2024-09-19T08:23:11.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:10 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:11.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:10 smithi064 ceph-mon[34900]: pgmap v250: 97 pgs: 97 active+clean; 1014 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 638 op/s 2024-09-19T08:23:11.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:10 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:13.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:12 smithi157 ceph-mon[35143]: pgmap v251: 97 pgs: 97 active+clean; 1014 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 638 op/s 2024-09-19T08:23:13.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:12 smithi022 ceph-mon[30963]: pgmap v251: 97 pgs: 97 active+clean; 1014 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 638 op/s 2024-09-19T08:23:13.090 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:12 smithi064 ceph-mon[34900]: pgmap v251: 97 pgs: 97 active+clean; 1014 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 638 op/s 2024-09-19T08:23:15.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:14 smithi157 ceph-mon[35143]: pgmap v252: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 223 KiB/s wr, 1.26k op/s 2024-09-19T08:23:15.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:14 smithi022 ceph-mon[30963]: pgmap v252: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 223 KiB/s wr, 1.26k op/s 2024-09-19T08:23:15.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:14 smithi064 ceph-mon[34900]: pgmap v252: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 223 KiB/s wr, 1.26k op/s 2024-09-19T08:23:16.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:16 smithi157 ceph-mon[35143]: pgmap v253: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 223 KiB/s wr, 1.26k op/s 2024-09-19T08:23:16.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:16 smithi157 ceph-mon[35143]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T08:23:16.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:16 smithi022 ceph-mon[30963]: pgmap v253: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 223 KiB/s wr, 1.26k op/s 2024-09-19T08:23:16.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:16 smithi022 ceph-mon[30963]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T08:23:16.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:16 smithi064 ceph-mon[34900]: pgmap v253: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 223 KiB/s wr, 1.26k op/s 2024-09-19T08:23:16.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:16 smithi064 ceph-mon[34900]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T08:23:17.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:17 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:23:17.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:17 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:23:17.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:17 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:23:18.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:18 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:23:18.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:18 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:23:18.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:18 smithi157 ceph-mon[35143]: pgmap v255: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 47 KiB/s wr, 1.21k op/s 2024-09-19T08:23:18.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:18 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:23:18.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:18 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:23:18.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:18 smithi022 ceph-mon[30963]: pgmap v255: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 47 KiB/s wr, 1.21k op/s 2024-09-19T08:23:18.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:18 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:23:18.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:18 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:23:18.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:18 smithi064 ceph-mon[34900]: pgmap v255: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 47 KiB/s wr, 1.21k op/s 2024-09-19T08:23:20.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:20 smithi157 ceph-mon[35143]: pgmap v256: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 46 KiB/s wr, 1.08k op/s 2024-09-19T08:23:20.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:20 smithi022 ceph-mon[30963]: pgmap v256: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 46 KiB/s wr, 1.08k op/s 2024-09-19T08:23:20.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:20 smithi064 ceph-mon[34900]: pgmap v256: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 46 KiB/s wr, 1.08k op/s 2024-09-19T08:23:22.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:22 smithi157 ceph-mon[35143]: pgmap v257: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 46 KiB/s wr, 1.08k op/s 2024-09-19T08:23:22.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:22 smithi022 ceph-mon[30963]: pgmap v257: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 46 KiB/s wr, 1.08k op/s 2024-09-19T08:23:22.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:22 smithi064 ceph-mon[34900]: pgmap v257: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 46 KiB/s wr, 1.08k op/s 2024-09-19T08:23:23.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:23 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:23:23.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:23 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:23:23.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:23 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:23:23.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:23 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:23:23.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:23 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:23:23.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:23 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:23:23.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:23 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:23:23.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:23 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:23:23.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:23 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:23:23.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:23 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:23:23.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:23 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:23:23.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:23 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:23:24.462 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:23:24.733 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:23:24.741 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:24 smithi064 ceph-mon[34900]: pgmap v258: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-09-19T08:23:24.741 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:24 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:23:24.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:24 smithi157 ceph-mon[35143]: pgmap v258: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-09-19T08:23:24.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:24 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:23:24.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:24 smithi022 ceph-mon[30963]: pgmap v258: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-09-19T08:23:24.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:24 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:23:25.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.621+0000 7f462d696640 1 -- 172.21.15.22:0/2670642731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f462810c170 msgr2=0x7f462810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:25.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.621+0000 7f462d696640 1 --2- 172.21.15.22:0/2670642731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f462810c170 0x7f462810e610 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4620009ca0 tx=0x7f462002a950 comp rx=0 tx=0).stop 2024-09-19T08:23:25.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.621+0000 7f462d696640 1 -- 172.21.15.22:0/2670642731 shutdown_connections 2024-09-19T08:23:25.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.621+0000 7f462d696640 1 --2- 172.21.15.22:0/2670642731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f462810c170 0x7f462810e610 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:25.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.621+0000 7f462d696640 1 --2- 172.21.15.22:0/2670642731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4628103660 0x7f462810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:25.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.621+0000 7f462d696640 1 --2- 172.21.15.22:0/2670642731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4628102d40 0x7f4628103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:25.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.621+0000 7f462d696640 1 -- 172.21.15.22:0/2670642731 >> 172.21.15.22:0/2670642731 conn(0x7f46280fc7a0 msgr2=0x7f46280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:25.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.621+0000 7f462d696640 1 -- 172.21.15.22:0/2670642731 shutdown_connections 2024-09-19T08:23:25.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.621+0000 7f462d696640 1 -- 172.21.15.22:0/2670642731 wait complete. 2024-09-19T08:23:25.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.622+0000 7f462d696640 1 Processor -- start 2024-09-19T08:23:25.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.622+0000 7f462d696640 1 -- start start 2024-09-19T08:23:25.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.622+0000 7f462d696640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4628102d40 0x7f462819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:25.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.622+0000 7f462d696640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4628103660 0x7f462819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:25.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.622+0000 7f462d696640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f462810c170 0x7f462819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:25.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f462d696640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4628111f50 con 0x7f4628103660 2024-09-19T08:23:25.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f462d696640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4628111dd0 con 0x7f462810c170 2024-09-19T08:23:25.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f462d696640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46281120d0 con 0x7f4628102d40 2024-09-19T08:23:25.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f461e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4628103660 0x7f462819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:25.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f461e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4628103660 0x7f462819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33334/0 (socket says 172.21.15.22:33334) 2024-09-19T08:23:25.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f461e7fc640 1 -- 172.21.15.22:0/1436720752 learned_addr learned my addr 172.21.15.22:0/1436720752 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:25.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f4626ffd640 1 --2- 172.21.15.22:0/1436720752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4628102d40 0x7f462819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:25.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f46277fe640 1 --2- 172.21.15.22:0/1436720752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f462810c170 0x7f462819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:25.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f461e7fc640 1 -- 172.21.15.22:0/1436720752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4628102d40 msgr2=0x7f462819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:25.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f461e7fc640 1 --2- 172.21.15.22:0/1436720752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4628102d40 0x7f462819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:25.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f461e7fc640 1 -- 172.21.15.22:0/1436720752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f462810c170 msgr2=0x7f462819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:25.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f461e7fc640 1 --2- 172.21.15.22:0/1436720752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f462810c170 0x7f462819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:25.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f461e7fc640 1 -- 172.21.15.22:0/1436720752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462819f570 con 0x7f4628103660 2024-09-19T08:23:25.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f461e7fc640 1 --2- 172.21.15.22:0/1436720752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4628103660 0x7f462819ab00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f461000d750 tx=0x7f461000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:25.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f4624ff9640 1 -- 172.21.15.22:0/1436720752 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610017070 con 0x7f4628103660 2024-09-19T08:23:25.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f462d696640 1 -- 172.21.15.22:0/1436720752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462819f860 con 0x7f4628103660 2024-09-19T08:23:25.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f4624ff9640 1 -- 172.21.15.22:0/1436720752 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4610005020 con 0x7f4628103660 2024-09-19T08:23:25.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f4624ff9640 1 -- 172.21.15.22:0/1436720752 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610004d00 con 0x7f4628103660 2024-09-19T08:23:25.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.623+0000 7f462d696640 1 -- 172.21.15.22:0/1436720752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46281a7140 con 0x7f4628103660 2024-09-19T08:23:25.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.624+0000 7f462d696640 1 -- 172.21.15.22:0/1436720752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4628103120 con 0x7f4628103660 2024-09-19T08:23:25.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.626+0000 7f4624ff9640 1 -- 172.21.15.22:0/1436720752 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4610009a30 con 0x7f4628103660 2024-09-19T08:23:25.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.626+0000 7f4624ff9640 1 --2- 172.21.15.22:0/1436720752 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f46140a33b0 0x7f46140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:25.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.627+0000 7f4626ffd640 1 --2- 172.21.15.22:0/1436720752 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f46140a33b0 0x7f46140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:25.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.627+0000 7f4624ff9640 1 -- 172.21.15.22:0/1436720752 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f46100cfb90 con 0x7f4628103660 2024-09-19T08:23:25.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.627+0000 7f4626ffd640 1 --2- 172.21.15.22:0/1436720752 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f46140a33b0 0x7f46140a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f461800bbe0 tx=0x7f461800b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:25.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.630+0000 7f4624ff9640 1 -- 172.21.15.22:0/1436720752 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f461009c500 con 0x7f4628103660 2024-09-19T08:23:25.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.841+0000 7f462d696640 1 -- 172.21.15.22:0/1436720752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46281a73a0 con 0x7f4628103660 2024-09-19T08:23:25.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.842+0000 7f4624ff9640 1 -- 172.21.15.22:0/1436720752 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11909 (secure 0 0 0) 0x7f461009be00 con 0x7f4628103660 2024-09-19T08:23:25.843 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:23:25.843 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":34,"btime":"2024-09-19T08:23:23:475035+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:23:22.677085+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:23:25.843 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 34 2024-09-19T08:23:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.844+0000 7f462d696640 1 -- 172.21.15.22:0/1436720752 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f46140a33b0 msgr2=0x7f46140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.844+0000 7f462d696640 1 --2- 172.21.15.22:0/1436720752 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f46140a33b0 0x7f46140a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f461800bbe0 tx=0x7f461800b390 comp rx=0 tx=0).stop 2024-09-19T08:23:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.844+0000 7f462d696640 1 -- 172.21.15.22:0/1436720752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4628103660 msgr2=0x7f462819ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.844+0000 7f462d696640 1 --2- 172.21.15.22:0/1436720752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4628103660 0x7f462819ab00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f461000d750 tx=0x7f461000dc10 comp rx=0 tx=0).stop 2024-09-19T08:23:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.844+0000 7f462d696640 1 -- 172.21.15.22:0/1436720752 shutdown_connections 2024-09-19T08:23:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.844+0000 7f462d696640 1 --2- 172.21.15.22:0/1436720752 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f46140a33b0 0x7f46140a5870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.844+0000 7f462d696640 1 --2- 172.21.15.22:0/1436720752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f462810c170 0x7f462819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.844+0000 7f462d696640 1 --2- 172.21.15.22:0/1436720752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4628103660 0x7f462819ab00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.844+0000 7f462d696640 1 --2- 172.21.15.22:0/1436720752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4628102d40 0x7f462819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.844+0000 7f462d696640 1 -- 172.21.15.22:0/1436720752 >> 172.21.15.22:0/1436720752 conn(0x7f46280fc7a0 msgr2=0x7f46280fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:25.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.844+0000 7f462d696640 1 -- 172.21.15.22:0/1436720752 shutdown_connections 2024-09-19T08:23:25.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:25.844+0000 7f462d696640 1 -- 172.21.15.22:0/1436720752 wait complete. 2024-09-19T08:23:26.692 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id ee82e705 --release --await' 2024-09-19T08:23:26.692 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id ee82e705 --release --await 2024-09-19T08:23:26.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:26 smithi157 ceph-mon[35143]: pgmap v259: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-09-19T08:23:26.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:26 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1436720752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:23:26.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:26 smithi022 ceph-mon[30963]: pgmap v259: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-09-19T08:23:26.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:26 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1436720752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:23:26.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:26 smithi064 ceph-mon[34900]: pgmap v259: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-09-19T08:23:26.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:26 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1436720752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:23:26.960 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:23:27.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.450+0000 7ffbe9248640 1 -- 172.21.15.22:0/3112589031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbe4101de0 msgr2=0x7ffbe41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:27.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.450+0000 7ffbe9248640 1 --2- 172.21.15.22:0/3112589031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbe4101de0 0x7ffbe41021c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4009900 tx=0x7ffbd402a410 comp rx=0 tx=0).stop 2024-09-19T08:23:27.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.450+0000 7ffbe9248640 1 -- 172.21.15.22:0/3112589031 shutdown_connections 2024-09-19T08:23:27.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.450+0000 7ffbe9248640 1 --2- 172.21.15.22:0/3112589031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbe4106f10 0x7ffbe410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.450+0000 7ffbe9248640 1 --2- 172.21.15.22:0/3112589031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbe4102790 0x7ffbe41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.450+0000 7ffbe9248640 1 --2- 172.21.15.22:0/3112589031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbe4101de0 0x7ffbe41021c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.450+0000 7ffbe9248640 1 -- 172.21.15.22:0/3112589031 >> 172.21.15.22:0/3112589031 conn(0x7ffbe40fd8c0 msgr2=0x7ffbe40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:27.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.450+0000 7ffbe9248640 1 -- 172.21.15.22:0/3112589031 shutdown_connections 2024-09-19T08:23:27.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.450+0000 7ffbe9248640 1 -- 172.21.15.22:0/3112589031 wait complete. 2024-09-19T08:23:27.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.451+0000 7ffbe9248640 1 Processor -- start 2024-09-19T08:23:27.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.451+0000 7ffbe9248640 1 -- start start 2024-09-19T08:23:27.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.451+0000 7ffbe9248640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbe4101de0 0x7ffbe41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.451+0000 7ffbe9248640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbe4102790 0x7ffbe41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.451+0000 7ffbe9248640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbe4106f10 0x7ffbe419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.451+0000 7ffbe9248640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe4111f70 con 0x7ffbe4106f10 2024-09-19T08:23:27.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.451+0000 7ffbe9248640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe4111df0 con 0x7ffbe4101de0 2024-09-19T08:23:27.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.451+0000 7ffbe9248640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41120f0 con 0x7ffbe4102790 2024-09-19T08:23:27.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.451+0000 7ffbe359e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbe4106f10 0x7ffbe419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.451+0000 7ffbe359e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbe4106f10 0x7ffbe419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41386/0 (socket says 172.21.15.22:41386) 2024-09-19T08:23:27.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.451+0000 7ffbe359e640 1 -- 172.21.15.22:0/3402985673 learned_addr learned my addr 172.21.15.22:0/3402985673 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:27.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.452+0000 7ffbe2d9d640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbe4101de0 0x7ffbe41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.452+0000 7ffbe259c640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbe4102790 0x7ffbe41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.452+0000 7ffbe359e640 1 -- 172.21.15.22:0/3402985673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbe4102790 msgr2=0x7ffbe41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:27.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.452+0000 7ffbe359e640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbe4102790 0x7ffbe41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.452+0000 7ffbe359e640 1 -- 172.21.15.22:0/3402985673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbe4101de0 msgr2=0x7ffbe41983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:27.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.452+0000 7ffbe359e640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbe4101de0 0x7ffbe41983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.452+0000 7ffbe359e640 1 -- 172.21.15.22:0/3402985673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe419d350 con 0x7ffbe4106f10 2024-09-19T08:23:27.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.452+0000 7ffbe259c640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbe4102790 0x7ffbe41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:23:27.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.452+0000 7ffbe359e640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbe4106f10 0x7ffbe419cc70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00cce0 tx=0x7ffbdc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:27.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.452+0000 7ffbcbfff640 1 -- 172.21.15.22:0/3402985673 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc00f450 con 0x7ffbe4106f10 2024-09-19T08:23:27.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.452+0000 7ffbe9248640 1 -- 172.21.15.22:0/3402985673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe419d640 con 0x7ffbe4106f10 2024-09-19T08:23:27.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.452+0000 7ffbcbfff640 1 -- 172.21.15.22:0/3402985673 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbdc00fad0 con 0x7ffbe4106f10 2024-09-19T08:23:27.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.452+0000 7ffbcbfff640 1 -- 172.21.15.22:0/3402985673 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc018c10 con 0x7ffbe4106f10 2024-09-19T08:23:27.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.452+0000 7ffbe9248640 1 -- 172.21.15.22:0/3402985673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe4074420 con 0x7ffbe4106f10 2024-09-19T08:23:27.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.455+0000 7ffbcbfff640 1 -- 172.21.15.22:0/3402985673 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffbdc00f5f0 con 0x7ffbe4106f10 2024-09-19T08:23:27.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.455+0000 7ffbeb621740 1 Processor -- start 2024-09-19T08:23:27.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.455+0000 7ffbeb621740 1 -- start start 2024-09-19T08:23:27.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.455+0000 7ffbeb621740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea060 0x562f346ea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.455+0000 7ffbeb621740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea980 0x562f346f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.455+0000 7ffbeb621740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f346f7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.455+0000 7ffbeb621740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562f3464f620 con 0x562f346f5240 2024-09-19T08:23:27.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.455+0000 7ffbeb621740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562f345f6e50 con 0x562f346ea060 2024-09-19T08:23:27.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.455+0000 7ffbeb621740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562f3464b5e0 con 0x562f346ea980 2024-09-19T08:23:27.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbe259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f346f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbe2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea980 0x562f346f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbe259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f346f7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41390/0 (socket says 172.21.15.22:41390) 2024-09-19T08:23:27.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbe259c640 1 -- 172.21.15.22:0/920932269 learned_addr learned my addr 172.21.15.22:0/920932269 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:27.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbcbfff640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ffbcc0a33b0 0x7ffbcc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbe259c640 1 --2- 172.21.15.22:0/920932269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea060 0x562f346ea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbcbfff640 1 -- 172.21.15.22:0/3402985673 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7ffbdc0d0360 con 0x7ffbe4106f10 2024-09-19T08:23:27.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbe359e640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ffbcc0a33b0 0x7ffbcc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbe2d9d640 1 -- 172.21.15.22:0/920932269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea060 msgr2=0x562f346ea440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:27.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbe2d9d640 1 --2- 172.21.15.22:0/920932269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea060 0x562f346ea440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbe2d9d640 1 -- 172.21.15.22:0/920932269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 msgr2=0x562f346f7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:27.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbe2d9d640 1 --2- 172.21.15.22:0/920932269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f346f7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbe2d9d640 1 -- 172.21.15.22:0/920932269 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f345e7d40 con 0x562f346ea980 2024-09-19T08:23:27.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbe259c640 1 --2- 172.21.15.22:0/920932269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f346f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:27.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbe2d9d640 1 --2- 172.21.15.22:0/920932269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea980 0x562f346f4d00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4005940 tx=0x7ffbd4005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3cb612b0b86e3a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:27.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.456+0000 7ffbe359e640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ffbcc0a33b0 0x7ffbcc0a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc09c5c0 tx=0x7ffbdc0d0660 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:27.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.457+0000 7ffbc97fa640 1 -- 172.21.15.22:0/920932269 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4040070 con 0x562f346ea980 2024-09-19T08:23:27.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.457+0000 7ffbc97fa640 1 -- 172.21.15.22:0/920932269 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd40319f0 con 0x562f346ea980 2024-09-19T08:23:27.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.457+0000 7ffbc97fa640 1 -- 172.21.15.22:0/920932269 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4035040 con 0x562f346ea980 2024-09-19T08:23:27.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.457+0000 7ffbeb621740 1 -- 172.21.15.22:0/920932269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea980 msgr2=0x562f346f4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:27.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.457+0000 7ffbeb621740 1 --2- 172.21.15.22:0/920932269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea980 0x562f346f4d00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4005940 tx=0x7ffbd4005e80 comp rx=0 tx=0).stop 2024-09-19T08:23:27.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.457+0000 7ffbeb621740 1 -- 172.21.15.22:0/920932269 shutdown_connections 2024-09-19T08:23:27.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.457+0000 7ffbeb621740 1 --2- 172.21.15.22:0/920932269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f346f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.457+0000 7ffbeb621740 1 --2- 172.21.15.22:0/920932269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea980 0x562f346f4d00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.457+0000 7ffbeb621740 1 --2- 172.21.15.22:0/920932269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea060 0x562f346ea440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.457+0000 7ffbeb621740 1 -- 172.21.15.22:0/920932269 >> 172.21.15.22:0/920932269 conn(0x562f346e65f0 msgr2=0x562f346e69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:27.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.457+0000 7ffbeb621740 1 -- 172.21.15.22:0/920932269 shutdown_connections 2024-09-19T08:23:27.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.457+0000 7ffbeb621740 1 -- 172.21.15.22:0/920932269 wait complete. 2024-09-19T08:23:27.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.458+0000 7ffbeb621740 1 -- start start 2024-09-19T08:23:27.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.458+0000 7ffbeb621740 1 Processor -- start 2024-09-19T08:23:27.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.458+0000 7ffbeb621740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea060 0x562f346f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.458+0000 7ffbeb621740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346ea980 0x562f346f2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.458+0000 7ffbeb621740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346f5240 0x562f346f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.458+0000 7ffbeb621740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562f346f7e10 con 0x562f346ea980 2024-09-19T08:23:27.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.458+0000 7ffbeb621740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562f345d7f10 con 0x562f346f5240 2024-09-19T08:23:27.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.458+0000 7ffbeb621740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562f346f7f90 con 0x562f346ea060 2024-09-19T08:23:27.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.458+0000 7ffbeb621740 10 client.0 _ll_register_callbacks cb 0x562f345fbdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:27.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.458+0000 7ffbe2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346ea980 0x562f346f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.458+0000 7ffbe259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea060 0x562f346f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.458+0000 7ffbe2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346ea980 0x562f346f2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41396/0 (socket says 172.21.15.22:41396) 2024-09-19T08:23:27.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.458+0000 7ffbe2d9d640 1 -- 172.21.15.22:0/1081189691 learned_addr learned my addr 172.21.15.22:0/1081189691 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:27.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.458+0000 7ffbe259c640 1 --2- 172.21.15.22:0/1081189691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346f5240 0x562f346f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbe2d9d640 1 -- 172.21.15.22:0/1081189691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea060 msgr2=0x562f346f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:27.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbe2d9d640 1 --2- 172.21.15.22:0/1081189691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea060 0x562f346f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbe2d9d640 1 -- 172.21.15.22:0/1081189691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346f5240 msgr2=0x562f346f2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:27.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbe2d9d640 1 --2- 172.21.15.22:0/1081189691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346f5240 0x562f346f2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbe2d9d640 1 -- 172.21.15.22:0/1081189691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3435e580 con 0x562f346ea980 2024-09-19T08:23:27.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbe259c640 1 --2- 172.21.15.22:0/1081189691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea060 0x562f346f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:27.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbe2d9d640 1 --2- 172.21.15.22:0/1081189691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346ea980 0x562f346f2610 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4005560 tx=0x7ffbd4005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1884cd810d4200 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:27.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:27.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbc6ff5640 1 -- 172.21.15.22:0/1081189691 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4040030 con 0x562f346ea980 2024-09-19T08:23:27.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbc6ff5640 1 -- 172.21.15.22:0/1081189691 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7ffbd40317f0 con 0x562f346ea980 2024-09-19T08:23:27.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbeb621740 1 -- 172.21.15.22:0/1081189691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbd40319f0 con 0x562f346ea980 2024-09-19T08:23:27.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbc6ff5640 1 -- 172.21.15.22:0/1081189691 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ffbd4035070 con 0x562f346ea980 2024-09-19T08:23:27.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbeb621740 10 client.16830 fetch_fsmap learned FSMap version 34 2024-09-19T08:23:27.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbeb621740 1 -- 172.21.15.22:0/1081189691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562f34428960 con 0x562f346ea980 2024-09-19T08:23:27.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.459+0000 7ffbc6ff5640 1 -- 172.21.15.22:0/1081189691 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd40319f0 con 0x562f346ea980 2024-09-19T08:23:27.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.460+0000 7ffbc6ff5640 1 -- 172.21.15.22:0/1081189691 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7ffbd40035f0 con 0x562f346ea980 2024-09-19T08:23:27.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.460+0000 7ffbeb621740 10 client.16830 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T08:23:27.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.460+0000 7ffbeb621740 10 client.16830 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:23:27.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.460+0000 7ffbeb621740 20 client.16830 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:27.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.460+0000 7ffbeb621740 1 --2- 172.21.15.22:0/1081189691 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562f34704bd0 0x562f34706fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.460+0000 7ffbeb621740 4 client.16830 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:23:27.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.460+0000 7ffbeb621740 1 -- 172.21.15.22:0/1081189691 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f3465cb90 con 0x562f34704bd0 2024-09-19T08:23:27.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.460+0000 7ffbe259c640 1 --2- 172.21.15.22:0/1081189691 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562f34704bd0 0x562f34706fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.460+0000 7ffbc6ff5640 10 client.16830 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:27.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.460+0000 7ffbe259c640 1 --2- 172.21.15.22:0/1081189691 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562f34704bd0 0x562f34706fc0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e81fafabd0114a4f server_cookie=16c86ab7248ad6eb in_seq=0 out_seq=0 2024-09-19T08:23:27.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.461+0000 7ffbc6ff5640 1 -- 172.21.15.22:0/1081189691 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f3465cb90 con 0x562f34704bd0 2024-09-19T08:23:27.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.461+0000 7ffbc6ff5640 10 client.16830 handle_command_reply: tid=0 2024-09-19T08:23:27.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.461+0000 7ffbeb621740 1 client.16830 shutdown 2024-09-19T08:23:27.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.461+0000 7ffbeb621740 1 -- 172.21.15.22:0/1081189691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346ea980 msgr2=0x562f346f2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:27.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.461+0000 7ffbeb621740 1 --2- 172.21.15.22:0/1081189691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346ea980 0x562f346f2610 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4005560 tx=0x7ffbd4005250 comp rx=0 tx=0).stop 2024-09-19T08:23:27.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.461+0000 7ffbeb621740 1 -- 172.21.15.22:0/1081189691 shutdown_connections 2024-09-19T08:23:27.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.461+0000 7ffbeb621740 1 --2- 172.21.15.22:0/1081189691 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562f34704bd0 0x562f34706fc0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.461+0000 7ffbeb621740 1 --2- 172.21.15.22:0/1081189691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346f5240 0x562f346f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.461+0000 7ffbeb621740 1 --2- 172.21.15.22:0/1081189691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346ea980 0x562f346f2610 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.461+0000 7ffbeb621740 1 --2- 172.21.15.22:0/1081189691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea060 0x562f346f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.461+0000 7ffbeb621740 1 -- 172.21.15.22:0/1081189691 >> 172.21.15.22:0/1081189691 conn(0x562f346e65f0 msgr2=0x562f346ec2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:27.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.461+0000 7ffbc6ff5640 0 client.16830 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:27.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.462+0000 7ffbeb621740 1 -- 172.21.15.22:0/1081189691 shutdown_connections 2024-09-19T08:23:27.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.462+0000 7ffbeb621740 1 -- 172.21.15.22:0/1081189691 wait complete. 2024-09-19T08:23:27.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.462+0000 7ffbeb621740 20 client.16830 trim_cache size 0 max 16384 2024-09-19T08:23:27.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbeb621740 1 Processor -- start 2024-09-19T08:23:27.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbeb621740 1 -- start start 2024-09-19T08:23:27.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbeb621740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea060 0x562f346e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbeb621740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea980 0x562f346e5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbeb621740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f3470c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbeb621740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41adec0 con 0x562f346f5240 2024-09-19T08:23:27.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbeb621740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41ab730 con 0x562f346ea060 2024-09-19T08:23:27.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbeb621740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41a95a0 con 0x562f346ea980 2024-09-19T08:23:27.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbe259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f3470c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbe259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f3470c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41398/0 (socket says 172.21.15.22:41398) 2024-09-19T08:23:27.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbe259c640 1 -- 172.21.15.22:0/2145624430 learned_addr learned my addr 172.21.15.22:0/2145624430 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:27.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbe2d9d640 1 --2- 172.21.15.22:0/2145624430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea980 0x562f346e5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbe259c640 1 --2- 172.21.15.22:0/2145624430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea060 0x562f346e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbe259c640 1 -- 172.21.15.22:0/2145624430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea980 msgr2=0x562f346e5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:27.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbe259c640 1 --2- 172.21.15.22:0/2145624430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea980 0x562f346e5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbe259c640 1 -- 172.21.15.22:0/2145624430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea060 msgr2=0x562f346e6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:27.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbe259c640 1 --2- 172.21.15.22:0/2145624430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea060 0x562f346e6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.479+0000 7ffbe259c640 1 -- 172.21.15.22:0/2145624430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f346f4010 con 0x562f346f5240 2024-09-19T08:23:27.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.480+0000 7ffbe259c640 1 --2- 172.21.15.22:0/2145624430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f3470c5c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8010710 tx=0x7ffbd8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15758ed95c493f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:27.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.480+0000 7ffbc8ff9640 1 -- 172.21.15.22:0/2145624430 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8009da0 con 0x562f346f5240 2024-09-19T08:23:27.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.480+0000 7ffbc8ff9640 1 -- 172.21.15.22:0/2145624430 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd8005030 con 0x562f346f5240 2024-09-19T08:23:27.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.480+0000 7ffbc8ff9640 1 -- 172.21.15.22:0/2145624430 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8004890 con 0x562f346f5240 2024-09-19T08:23:27.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.480+0000 7ffbeb621740 1 -- 172.21.15.22:0/2145624430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 msgr2=0x562f3470c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:27.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.480+0000 7ffbeb621740 1 --2- 172.21.15.22:0/2145624430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f3470c5c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8010710 tx=0x7ffbd8005f90 comp rx=0 tx=0).stop 2024-09-19T08:23:27.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.480+0000 7ffbeb621740 1 -- 172.21.15.22:0/2145624430 shutdown_connections 2024-09-19T08:23:27.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.480+0000 7ffbeb621740 1 --2- 172.21.15.22:0/2145624430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f3470c5c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.480+0000 7ffbeb621740 1 --2- 172.21.15.22:0/2145624430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea980 0x562f346e5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.480+0000 7ffbeb621740 1 --2- 172.21.15.22:0/2145624430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea060 0x562f346e6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.480+0000 7ffbeb621740 1 -- 172.21.15.22:0/2145624430 >> 172.21.15.22:0/2145624430 conn(0x562f346e65f0 msgr2=0x562f346e4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:27.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.480+0000 7ffbeb621740 1 -- 172.21.15.22:0/2145624430 shutdown_connections 2024-09-19T08:23:27.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.480+0000 7ffbeb621740 1 -- 172.21.15.22:0/2145624430 wait complete. 2024-09-19T08:23:27.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.481+0000 7ffbeb621740 1 -- start start 2024-09-19T08:23:27.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.481+0000 7ffbeb621740 1 Processor -- start 2024-09-19T08:23:27.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.481+0000 7ffbeb621740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea060 0x562f346f9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.481+0000 7ffbeb621740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea980 0x562f346f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.481+0000 7ffbeb621740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f3476e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.481+0000 7ffbeb621740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41a9bf0 con 0x562f346f5240 2024-09-19T08:23:27.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.481+0000 7ffbeb621740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41ab060 con 0x562f346ea980 2024-09-19T08:23:27.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.481+0000 7ffbeb621740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41aa2c0 con 0x562f346ea060 2024-09-19T08:23:27.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.481+0000 7ffbeb621740 10 client.0 _ll_register_callbacks cb 0x7ffbd4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:27.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.481+0000 7ffbe259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f3476e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.481+0000 7ffbe259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f3476e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41414/0 (socket says 172.21.15.22:41414) 2024-09-19T08:23:27.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.481+0000 7ffbe259c640 1 -- 172.21.15.22:0/1366699259 learned_addr learned my addr 172.21.15.22:0/1366699259 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:27.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.481+0000 7ffbe2d9d640 1 --2- 172.21.15.22:0/1366699259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea980 0x562f346f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.481+0000 7ffbe259c640 1 --2- 172.21.15.22:0/1366699259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea060 0x562f346f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.482+0000 7ffbe259c640 1 -- 172.21.15.22:0/1366699259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea060 msgr2=0x562f346f9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:27.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.482+0000 7ffbe259c640 1 --2- 172.21.15.22:0/1366699259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea060 0x562f346f9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.482+0000 7ffbe259c640 1 -- 172.21.15.22:0/1366699259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea980 msgr2=0x562f346f9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:27.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.482+0000 7ffbe259c640 1 --2- 172.21.15.22:0/1366699259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea980 0x562f346f9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:27.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.482+0000 7ffbe259c640 1 -- 172.21.15.22:0/1366699259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3476ead0 con 0x562f346f5240 2024-09-19T08:23:27.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.482+0000 7ffbe259c640 1 --2- 172.21.15.22:0/1366699259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f3476e590 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009250 tx=0x7ffbd8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2295d83cad2aa29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:27.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.482+0000 7ffbc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:27.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.482+0000 7ffbc6ff5640 1 -- 172.21.15.22:0/1366699259 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8003af0 con 0x562f346f5240 2024-09-19T08:23:27.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.482+0000 7ffbc6ff5640 1 -- 172.21.15.22:0/1366699259 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7ffbd8003c90 con 0x562f346f5240 2024-09-19T08:23:27.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.482+0000 7ffbeb621740 1 -- 172.21.15.22:0/1366699259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbd8005030 con 0x562f346f5240 2024-09-19T08:23:27.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.482+0000 7ffbc6ff5640 1 -- 172.21.15.22:0/1366699259 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ffbd80051e0 con 0x562f346f5240 2024-09-19T08:23:27.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.482+0000 7ffbeb621740 10 client.16836 fetch_fsmap learned FSMap version 34 2024-09-19T08:23:27.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.482+0000 7ffbeb621740 1 -- 172.21.15.22:0/1366699259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562f3476fdb0 con 0x562f346f5240 2024-09-19T08:23:27.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.485+0000 7ffbc6ff5640 1 -- 172.21.15.22:0/1366699259 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd8005030 con 0x562f346f5240 2024-09-19T08:23:27.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.485+0000 7ffbc6ff5640 1 -- 172.21.15.22:0/1366699259 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7ffbd8039040 con 0x562f346f5240 2024-09-19T08:23:27.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.485+0000 7ffbeb621740 10 client.16836 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T08:23:27.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.485+0000 7ffbeb621740 10 client.16836 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:23:27.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.485+0000 7ffbeb621740 20 client.16836 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:27.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.485+0000 7ffbeb621740 1 --2- 172.21.15.22:0/1366699259 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562f34770400 0x562f34772820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:27.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.485+0000 7ffbeb621740 4 client.16836 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ee82e705", "release": true, "await": true}] 2024-09-19T08:23:27.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.485+0000 7ffbeb621740 1 -- 172.21.15.22:0/1366699259 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ee82e705", "release": true, "await": true}) -- 0x562f346edcf0 con 0x562f34770400 2024-09-19T08:23:27.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.485+0000 7ffbe2d9d640 1 --2- 172.21.15.22:0/1366699259 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562f34770400 0x562f34772820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:27.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.486+0000 7ffbe2d9d640 1 --2- 172.21.15.22:0/1366699259 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562f34770400 0x562f34772820 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=608ad8cc7514ba3d server_cookie=21f940e0de40488a in_seq=0 out_seq=0 2024-09-19T08:23:27.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:27.486+0000 7ffbc6ff5640 10 client.16836 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbc6ff5640 1 -- 172.21.15.22:0/1366699259 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562f346edcf0 con 0x562f34770400 2024-09-19T08:23:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbc6ff5640 10 client.16836 handle_command_reply: tid=0 2024-09-19T08:23:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbeb621740 1 client.16836 shutdown 2024-09-19T08:23:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbeb621740 1 -- 172.21.15.22:0/1366699259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 msgr2=0x562f3476e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbeb621740 1 --2- 172.21.15.22:0/1366699259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f3476e590 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009250 tx=0x7ffbd8013040 comp rx=0 tx=0).stop 2024-09-19T08:23:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbeb621740 1 -- 172.21.15.22:0/1366699259 shutdown_connections 2024-09-19T08:23:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbeb621740 1 --2- 172.21.15.22:0/1366699259 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x562f34770400 0x562f34772820 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbeb621740 1 --2- 172.21.15.22:0/1366699259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f346f5240 0x562f3476e590 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:28.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbeb621740 1 --2- 172.21.15.22:0/1366699259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f346ea980 0x562f346f9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:28.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbeb621740 1 --2- 172.21.15.22:0/1366699259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562f346ea060 0x562f346f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:28.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbc6ff5640 0 client.16836 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:28.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbeb621740 1 -- 172.21.15.22:0/1366699259 >> 172.21.15.22:0/1366699259 conn(0x562f346e65f0 msgr2=0x562f346ec2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:28.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbeb621740 1 -- 172.21.15.22:0/1366699259 shutdown_connections 2024-09-19T08:23:28.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbeb621740 1 -- 172.21.15.22:0/1366699259 wait complete. 2024-09-19T08:23:28.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.441+0000 7ffbeb621740 20 client.16836 trim_cache size 0 max 16384 2024-09-19T08:23:28.443 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:23:28.444 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 34, 2024-09-19T08:23:28.444 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:23:28.444 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 32, 2024-09-19T08:23:28.444 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": { 2024-09-19T08:23:28.444 INFO:teuthology.orchestra.run.smithi022.stdout: "ee82e705": { 2024-09-19T08:23:28.444 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 32, 2024-09-19T08:23:28.444 INFO:teuthology.orchestra.run.smithi022.stdout: "age_ref": 0.0, 2024-09-19T08:23:28.444 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:23:28.444 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:23:28.444 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:23:28.444 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:23:28.444 INFO:teuthology.orchestra.run.smithi022.stdout: "timeout": 90.0, 2024-09-19T08:23:28.444 INFO:teuthology.orchestra.run.smithi022.stdout: "expiration": 145.8, 2024-09-19T08:23:28.444 INFO:teuthology.orchestra.run.smithi022.stdout: "members": { 2024-09-19T08:23:28.445 INFO:teuthology.orchestra.run.smithi022.stdout: "file:/": { 2024-09-19T08:23:28.445 INFO:teuthology.orchestra.run.smithi022.stdout: "excluded": false, 2024-09-19T08:23:28.445 INFO:teuthology.orchestra.run.smithi022.stdout: "state": { 2024-09-19T08:23:28.445 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RELEASED", 2024-09-19T08:23:28.445 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0 2024-09-19T08:23:28.445 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:23:28.445 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:23:28.445 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:23:28.445 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:23:28.445 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:23:28.445 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:23:28.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.442+0000 7ffbe9248640 1 -- 172.21.15.22:0/3402985673 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ffbcc0a33b0 msgr2=0x7ffbcc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:28.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.442+0000 7ffbe9248640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ffbcc0a33b0 0x7ffbcc0a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc09c5c0 tx=0x7ffbdc0d0660 comp rx=0 tx=0).stop 2024-09-19T08:23:28.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.442+0000 7ffbe9248640 1 -- 172.21.15.22:0/3402985673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbe4106f10 msgr2=0x7ffbe419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:28.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.442+0000 7ffbe9248640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbe4106f10 0x7ffbe419cc70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00cce0 tx=0x7ffbdc00a620 comp rx=0 tx=0).stop 2024-09-19T08:23:28.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.442+0000 7ffbe9248640 1 -- 172.21.15.22:0/3402985673 shutdown_connections 2024-09-19T08:23:28.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.442+0000 7ffbe9248640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ffbcc0a33b0 0x7ffbcc0a5870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:28.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.442+0000 7ffbe9248640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffbe4106f10 0x7ffbe419cc70 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:28.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.442+0000 7ffbe9248640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbe4102790 0x7ffbe41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:28.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.442+0000 7ffbe9248640 1 --2- 172.21.15.22:0/3402985673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbe4101de0 0x7ffbe41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:28.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.442+0000 7ffbe9248640 1 -- 172.21.15.22:0/3402985673 >> 172.21.15.22:0/3402985673 conn(0x7ffbe40fd8c0 msgr2=0x7ffbe40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:28.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.442+0000 7ffbe9248640 1 -- 172.21.15.22:0/3402985673 shutdown_connections 2024-09-19T08:23:28.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:28.442+0000 7ffbe9248640 1 -- 172.21.15.22:0/3402985673 wait complete. 2024-09-19T08:23:28.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:28 smithi157 ceph-mon[35143]: pgmap v260: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-09-19T08:23:28.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:28 smithi022 ceph-mon[30963]: pgmap v260: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-09-19T08:23:28.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:28 smithi064 ceph-mon[34900]: pgmap v260: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-09-19T08:23:29.275 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ee82e705', total seconds elapsed: 30.6 2024-09-19T08:23:30.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:30 smithi157 ceph-mon[35143]: pgmap v261: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 8 op/s 2024-09-19T08:23:30.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:30 smithi022 ceph-mon[30963]: pgmap v261: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 8 op/s 2024-09-19T08:23:30.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:30 smithi064 ceph-mon[34900]: pgmap v261: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 8 op/s 2024-09-19T08:23:32.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:32 smithi157 ceph-mon[35143]: pgmap v262: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-19T08:23:32.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:32 smithi022 ceph-mon[30963]: pgmap v262: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-19T08:23:32.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:32 smithi064 ceph-mon[34900]: pgmap v262: 97 pgs: 97 active+clean; 1015 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-19T08:23:33.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:23:33.848 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:23:34.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.379+0000 7f514c16c640 1 -- 172.21.15.22:0/1563597892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51441027a0 msgr2=0x7f51441067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:34.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.379+0000 7f514c16c640 1 --2- 172.21.15.22:0/1563597892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51441027a0 0x7f51441067f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5138009900 tx=0x7f513802a410 comp rx=0 tx=0).stop 2024-09-19T08:23:34.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.379+0000 7f514c16c640 1 -- 172.21.15.22:0/1563597892 shutdown_connections 2024-09-19T08:23:34.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.379+0000 7f514c16c640 1 --2- 172.21.15.22:0/1563597892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5144106f20 0x7f514410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.379+0000 7f514c16c640 1 --2- 172.21.15.22:0/1563597892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51441027a0 0x7f51441067f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.379+0000 7f514c16c640 1 --2- 172.21.15.22:0/1563597892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5144101df0 0x7f51441021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.379+0000 7f514c16c640 1 -- 172.21.15.22:0/1563597892 >> 172.21.15.22:0/1563597892 conn(0x7f51440fd8f0 msgr2=0x7f51440ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:34.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.379+0000 7f514c16c640 1 -- 172.21.15.22:0/1563597892 shutdown_connections 2024-09-19T08:23:34.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.379+0000 7f514c16c640 1 -- 172.21.15.22:0/1563597892 wait complete. 2024-09-19T08:23:34.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.380+0000 7f514c16c640 1 Processor -- start 2024-09-19T08:23:34.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.380+0000 7f514c16c640 1 -- start start 2024-09-19T08:23:34.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.380+0000 7f514c16c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5144101df0 0x7f51440744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.380+0000 7f514c16c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51441027a0 0x7f5144074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.380+0000 7f514c16c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5144106f20 0x7f5144074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.380+0000 7f514c16c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5144111f80 con 0x7f5144101df0 2024-09-19T08:23:34.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.380+0000 7f514c16c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5144111e00 con 0x7f51441027a0 2024-09-19T08:23:34.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.380+0000 7f514c16c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5144112100 con 0x7f5144106f20 2024-09-19T08:23:34.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.380+0000 7f5149f08640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5144101df0 0x7f51440744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.380+0000 7f5149707640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51441027a0 0x7f5144074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.380+0000 7f5149f08640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5144101df0 0x7f51440744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41422/0 (socket says 172.21.15.22:41422) 2024-09-19T08:23:34.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.380+0000 7f5149707640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51441027a0 0x7f5144074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:41026/0 (socket says 172.21.15.22:41026) 2024-09-19T08:23:34.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.380+0000 7f5149f08640 1 -- 172.21.15.22:0/2984272686 learned_addr learned my addr 172.21.15.22:0/2984272686 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:34.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.380+0000 7f514a709640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5144106f20 0x7f5144074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.381+0000 7f5149707640 1 -- 172.21.15.22:0/2984272686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5144106f20 msgr2=0x7f5144074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:34.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.381+0000 7f5149707640 1 --2- 172.21.15.22:0/2984272686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5144106f20 0x7f5144074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.381+0000 7f5149707640 1 -- 172.21.15.22:0/2984272686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5144101df0 msgr2=0x7f51440744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:34.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.381+0000 7f5149707640 1 --2- 172.21.15.22:0/2984272686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5144101df0 0x7f51440744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.381+0000 7f5149707640 1 -- 172.21.15.22:0/2984272686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51441a6980 con 0x7f51441027a0 2024-09-19T08:23:34.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.381+0000 7f5149f08640 1 --2- 172.21.15.22:0/2984272686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5144101df0 0x7f51440744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:34.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.381+0000 7f5149707640 1 --2- 172.21.15.22:0/2984272686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51441027a0 0x7f5144074a30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5138005940 tx=0x7f5138004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:34.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.381+0000 7f5142ffd640 1 -- 172.21.15.22:0/2984272686 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513803f030 con 0x7f51441027a0 2024-09-19T08:23:34.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.381+0000 7f5142ffd640 1 -- 172.21.15.22:0/2984272686 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5138003ad0 con 0x7f51441027a0 2024-09-19T08:23:34.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.381+0000 7f514c16c640 1 -- 172.21.15.22:0/2984272686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51441a6c10 con 0x7f51441027a0 2024-09-19T08:23:34.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.381+0000 7f5142ffd640 1 -- 172.21.15.22:0/2984272686 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138035070 con 0x7f51441027a0 2024-09-19T08:23:34.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.381+0000 7f514c16c640 1 -- 172.21.15.22:0/2984272686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51441a7140 con 0x7f51441027a0 2024-09-19T08:23:34.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.384+0000 7f5142ffd640 1 -- 172.21.15.22:0/2984272686 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5138042030 con 0x7f51441027a0 2024-09-19T08:23:34.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.384+0000 7f514e545740 1 Processor -- start 2024-09-19T08:23:34.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.384+0000 7f514e545740 1 -- start start 2024-09-19T08:23:34.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.384+0000 7f514e545740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 0x56107bce1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.384+0000 7f514e545740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bcec140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.384+0000 7f514e545740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcec680 0x56107bceea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.384+0000 7f514e545740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56107bc472e0 con 0x56107bcec680 2024-09-19T08:23:34.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.384+0000 7f514e545740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56107bbeece0 con 0x56107bce1dc0 2024-09-19T08:23:34.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.384+0000 7f514e545740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56107bc432a0 con 0x56107bce14a0 2024-09-19T08:23:34.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f514a709640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 0x56107bce1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f5149707640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcec680 0x56107bceea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f514a709640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 0x56107bce1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:54124/0 (socket says 172.21.15.22:54124) 2024-09-19T08:23:34.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f514a709640 1 -- 172.21.15.22:0/454737710 learned_addr learned my addr 172.21.15.22:0/454737710 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:34.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f5149f08640 1 --2- 172.21.15.22:0/454737710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bcec140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f5142ffd640 1 --2- 172.21.15.22:0/2984272686 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f512c0a3430 0x7f512c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f514a709640 1 -- 172.21.15.22:0/454737710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 msgr2=0x56107bcec140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:34.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f514a709640 1 --2- 172.21.15.22:0/454737710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bcec140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f514a709640 1 -- 172.21.15.22:0/454737710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcec680 msgr2=0x56107bceea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:34.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f5142ffd640 1 -- 172.21.15.22:0/2984272686 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f51380f01c0 con 0x7f51441027a0 2024-09-19T08:23:34.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f514a709640 1 --2- 172.21.15.22:0/454737710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcec680 0x56107bceea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f514a709640 1 -- 172.21.15.22:0/454737710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56107bbd3440 con 0x56107bce14a0 2024-09-19T08:23:34.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f5149707640 1 --2- 172.21.15.22:0/454737710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcec680 0x56107bceea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:34.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f514a709640 1 --2- 172.21.15.22:0/2984272686 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f512c0a3430 0x7f512c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.385+0000 7f514a709640 1 --2- 172.21.15.22:0/454737710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 0x56107bce1880 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f513c00b7a0 tx=0x7f513c00bba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67c31f34d760955a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:34.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.386+0000 7f512bfff640 1 -- 172.21.15.22:0/454737710 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c024070 con 0x56107bce14a0 2024-09-19T08:23:34.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.386+0000 7f512bfff640 1 -- 172.21.15.22:0/454737710 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f513c01e960 con 0x56107bce14a0 2024-09-19T08:23:34.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.386+0000 7f514a709640 1 --2- 172.21.15.22:0/2984272686 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f512c0a3430 0x7f512c0a58f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f513c021000 tx=0x7f513c01e000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:34.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.386+0000 7f512bfff640 1 -- 172.21.15.22:0/454737710 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c011900 con 0x56107bce14a0 2024-09-19T08:23:34.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.386+0000 7f514e545740 1 -- 172.21.15.22:0/454737710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 msgr2=0x56107bce1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:34.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.386+0000 7f514e545740 1 --2- 172.21.15.22:0/454737710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 0x56107bce1880 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f513c00b7a0 tx=0x7f513c00bba0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.386+0000 7f514e545740 1 -- 172.21.15.22:0/454737710 shutdown_connections 2024-09-19T08:23:34.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.386+0000 7f514e545740 1 --2- 172.21.15.22:0/454737710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcec680 0x56107bceea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.386+0000 7f514e545740 1 --2- 172.21.15.22:0/454737710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bcec140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.386+0000 7f514e545740 1 --2- 172.21.15.22:0/454737710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 0x56107bce1880 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.386+0000 7f514e545740 1 -- 172.21.15.22:0/454737710 >> 172.21.15.22:0/454737710 conn(0x56107bcdda30 msgr2=0x56107bcdde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:34.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.387+0000 7f514e545740 1 -- 172.21.15.22:0/454737710 shutdown_connections 2024-09-19T08:23:34.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.387+0000 7f514e545740 1 -- 172.21.15.22:0/454737710 wait complete. 2024-09-19T08:23:34.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.387+0000 7f514e545740 1 -- start start 2024-09-19T08:23:34.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.387+0000 7f514e545740 1 Processor -- start 2024-09-19T08:23:34.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.387+0000 7f514e545740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 0x56107bcebf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.387+0000 7f514e545740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bce9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.387+0000 7f514e545740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcec680 0x56107bce9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.387+0000 7f514e545740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56107bcef250 con 0x56107bcec680 2024-09-19T08:23:34.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.387+0000 7f514e545740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56107bc02050 con 0x56107bce1dc0 2024-09-19T08:23:34.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.387+0000 7f514e545740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56107bcef3d0 con 0x56107bce14a0 2024-09-19T08:23:34.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.387+0000 7f514e545740 10 client.0 _ll_register_callbacks cb 0x56107bc42160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:34.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.387+0000 7f514a709640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcec680 0x56107bce9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f5149f08640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 0x56107bcebf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f514a709640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcec680 0x56107bce9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41440/0 (socket says 172.21.15.22:41440) 2024-09-19T08:23:34.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f514a709640 1 -- 172.21.15.22:0/741203242 learned_addr learned my addr 172.21.15.22:0/741203242 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:34.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f5149707640 1 --2- 172.21.15.22:0/741203242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bce9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f514a709640 1 -- 172.21.15.22:0/741203242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 msgr2=0x56107bcebf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:34.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f514a709640 1 --2- 172.21.15.22:0/741203242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 0x56107bcebf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f514a709640 1 -- 172.21.15.22:0/741203242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 msgr2=0x56107bce9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:34.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f514a709640 1 --2- 172.21.15.22:0/741203242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bce9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f514a709640 1 -- 172.21.15.22:0/741203242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56107b9b83c0 con 0x56107bcec680 2024-09-19T08:23:34.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f514a709640 1 --2- 172.21.15.22:0/741203242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcec680 0x56107bce9760 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f513c00bfd0 tx=0x7f513c013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc55abb347a4515b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:34.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f51297fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:34.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f51297fa640 1 -- 172.21.15.22:0/741203242 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c024040 con 0x56107bcec680 2024-09-19T08:23:34.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f51297fa640 1 -- 172.21.15.22:0/741203242 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f513c011680 con 0x56107bcec680 2024-09-19T08:23:34.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f514e545740 1 -- 172.21.15.22:0/741203242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f513c01e960 con 0x56107bcec680 2024-09-19T08:23:34.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f51297fa640 1 -- 172.21.15.22:0/741203242 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f513c039050 con 0x56107bcec680 2024-09-19T08:23:34.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f514e545740 10 client.16848 fetch_fsmap learned FSMap version 35 2024-09-19T08:23:34.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.388+0000 7f514e545740 1 -- 172.21.15.22:0/741203242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56107bcfabc0 con 0x56107bcec680 2024-09-19T08:23:34.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.389+0000 7f51297fa640 1 -- 172.21.15.22:0/741203242 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f513c01e960 con 0x56107bcec680 2024-09-19T08:23:34.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.389+0000 7f51297fa640 1 -- 172.21.15.22:0/741203242 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f513c005050 con 0x56107bcec680 2024-09-19T08:23:34.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.389+0000 7f514e545740 10 client.16848 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:23:34.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.389+0000 7f514e545740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:23:34.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.389+0000 7f514e545740 20 client.16848 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:34.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.389+0000 7f514e545740 1 --2- 172.21.15.22:0/741203242 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56107bcfbf10 0x56107bcfe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.389+0000 7f514e545740 4 client.16848 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:23:34.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.389+0000 7f514e545740 1 -- 172.21.15.22:0/741203242 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56107bc54850 con 0x56107bcfbf10 2024-09-19T08:23:34.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.389+0000 7f5149f08640 1 --2- 172.21.15.22:0/741203242 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56107bcfbf10 0x56107bcfe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.390+0000 7f5149f08640 1 --2- 172.21.15.22:0/741203242 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56107bcfbf10 0x56107bcfe300 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfebe5c350c6dac7 server_cookie=4ebdb20754f0a409 in_seq=0 out_seq=0 2024-09-19T08:23:34.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.390+0000 7f51297fa640 10 client.16848 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:34.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.390+0000 7f51297fa640 1 -- 172.21.15.22:0/741203242 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56107bc54850 con 0x56107bcfbf10 2024-09-19T08:23:34.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.390+0000 7f51297fa640 10 client.16848 handle_command_reply: tid=0 2024-09-19T08:23:34.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.390+0000 7f514e545740 1 client.16848 shutdown 2024-09-19T08:23:34.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.390+0000 7f514e545740 1 -- 172.21.15.22:0/741203242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcec680 msgr2=0x56107bce9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:34.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.390+0000 7f514e545740 1 --2- 172.21.15.22:0/741203242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcec680 0x56107bce9760 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f513c00bfd0 tx=0x7f513c013f50 comp rx=0 tx=0).stop 2024-09-19T08:23:34.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.390+0000 7f514e545740 1 -- 172.21.15.22:0/741203242 shutdown_connections 2024-09-19T08:23:34.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.390+0000 7f514e545740 1 --2- 172.21.15.22:0/741203242 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56107bcfbf10 0x56107bcfe300 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.390+0000 7f514e545740 1 --2- 172.21.15.22:0/741203242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcec680 0x56107bce9760 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.390+0000 7f514e545740 1 --2- 172.21.15.22:0/741203242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bce9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.390+0000 7f51297fa640 0 client.16848 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:34.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.390+0000 7f514e545740 1 --2- 172.21.15.22:0/741203242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 0x56107bcebf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.390+0000 7f514e545740 1 -- 172.21.15.22:0/741203242 >> 172.21.15.22:0/741203242 conn(0x56107bcdda30 msgr2=0x56107bce3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:34.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.391+0000 7f514e545740 1 -- 172.21.15.22:0/741203242 shutdown_connections 2024-09-19T08:23:34.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.391+0000 7f514e545740 1 -- 172.21.15.22:0/741203242 wait complete. 2024-09-19T08:23:34.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.391+0000 7f514e545740 20 client.16848 trim_cache size 0 max 16384 2024-09-19T08:23:34.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.409+0000 7f514e545740 1 Processor -- start 2024-09-19T08:23:34.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.409+0000 7f514e545740 1 -- start start 2024-09-19T08:23:34.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.409+0000 7f514e545740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcdda30 0x56107bce7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.409+0000 7f514e545740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 0x56107bd03990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.409+0000 7f514e545740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bd03ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.409+0000 7f514e545740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f51441b23b0 con 0x56107bcdda30 2024-09-19T08:23:34.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.409+0000 7f514e545740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f51441afc50 con 0x56107bce1dc0 2024-09-19T08:23:34.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.409+0000 7f514e545740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51441adac0 con 0x56107bce14a0 2024-09-19T08:23:34.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.410+0000 7f5149f08640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcdda30 0x56107bce7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.410+0000 7f5149f08640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcdda30 0x56107bce7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41442/0 (socket says 172.21.15.22:41442) 2024-09-19T08:23:34.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.410+0000 7f5149f08640 1 -- 172.21.15.22:0/1418507444 learned_addr learned my addr 172.21.15.22:0/1418507444 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:34.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.410+0000 7f5149707640 1 --2- 172.21.15.22:0/1418507444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 0x56107bd03990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.410+0000 7f514a709640 1 --2- 172.21.15.22:0/1418507444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bd03ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.410+0000 7f5149f08640 1 -- 172.21.15.22:0/1418507444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 msgr2=0x56107bd03990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:34.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.410+0000 7f5149f08640 1 --2- 172.21.15.22:0/1418507444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 0x56107bd03990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.410+0000 7f5149f08640 1 -- 172.21.15.22:0/1418507444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 msgr2=0x56107bd03ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:34.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.410+0000 7f5149f08640 1 --2- 172.21.15.22:0/1418507444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bd03ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.410+0000 7f5149f08640 1 -- 172.21.15.22:0/1418507444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56107bcdd4d0 con 0x56107bcdda30 2024-09-19T08:23:34.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.410+0000 7f5149f08640 1 --2- 172.21.15.22:0/1418507444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcdda30 0x56107bce7af0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x56107bcfe930 tx=0x7f5134009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9031fe54c39b6306 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:34.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.410+0000 7f512b7fe640 1 -- 172.21.15.22:0/1418507444 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513400fea0 con 0x56107bcdda30 2024-09-19T08:23:34.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.410+0000 7f512b7fe640 1 -- 172.21.15.22:0/1418507444 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5134005020 con 0x56107bcdda30 2024-09-19T08:23:34.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.410+0000 7f512b7fe640 1 -- 172.21.15.22:0/1418507444 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513401e650 con 0x56107bcdda30 2024-09-19T08:23:34.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.411+0000 7f514e545740 1 -- 172.21.15.22:0/1418507444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcdda30 msgr2=0x56107bce7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:34.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.411+0000 7f514e545740 1 --2- 172.21.15.22:0/1418507444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcdda30 0x56107bce7af0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x56107bcfe930 tx=0x7f5134009500 comp rx=0 tx=0).stop 2024-09-19T08:23:34.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.411+0000 7f514e545740 1 -- 172.21.15.22:0/1418507444 shutdown_connections 2024-09-19T08:23:34.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.411+0000 7f514e545740 1 --2- 172.21.15.22:0/1418507444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bd03ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.411+0000 7f514e545740 1 --2- 172.21.15.22:0/1418507444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bce14a0 0x56107bd03990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.411+0000 7f514e545740 1 --2- 172.21.15.22:0/1418507444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bcdda30 0x56107bce7af0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.411+0000 7f514e545740 1 -- 172.21.15.22:0/1418507444 >> 172.21.15.22:0/1418507444 conn(0x56107bce98c0 msgr2=0x56107bceb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:34.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.411+0000 7f514e545740 1 -- 172.21.15.22:0/1418507444 shutdown_connections 2024-09-19T08:23:34.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.411+0000 7f514e545740 1 -- 172.21.15.22:0/1418507444 wait complete. 2024-09-19T08:23:34.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.411+0000 7f514e545740 1 -- start start 2024-09-19T08:23:34.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f514e545740 1 Processor -- start 2024-09-19T08:23:34.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f514e545740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bcdda30 0x56107bcf2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f514e545740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bce14a0 0x56107bcf2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f514e545740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bd65970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f514e545740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ae110 con 0x56107bce14a0 2024-09-19T08:23:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f514e545740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f51441af580 con 0x56107bce1dc0 2024-09-19T08:23:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f514e545740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ae7e0 con 0x56107bcdda30 2024-09-19T08:23:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f514e545740 10 client.0 _ll_register_callbacks cb 0x7f513c022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f5149707640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bce14a0 0x56107bcf2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f5149707640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bce14a0 0x56107bcf2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41444/0 (socket says 172.21.15.22:41444) 2024-09-19T08:23:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f5149707640 1 -- 172.21.15.22:0/2570470872 learned_addr learned my addr 172.21.15.22:0/2570470872 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f5149f08640 1 --2- 172.21.15.22:0/2570470872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bcdda30 0x56107bcf2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f514a709640 1 --2- 172.21.15.22:0/2570470872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bd65970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f5149707640 1 -- 172.21.15.22:0/2570470872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bcdda30 msgr2=0x56107bcf2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f5149707640 1 --2- 172.21.15.22:0/2570470872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bcdda30 0x56107bcf2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f5149707640 1 -- 172.21.15.22:0/2570470872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 msgr2=0x56107bd65970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:34.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f5149707640 1 --2- 172.21.15.22:0/2570470872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bd65970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f5149707640 1 -- 172.21.15.22:0/2570470872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56107bcf3580 con 0x56107bce14a0 2024-09-19T08:23:34.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.412+0000 7f514a709640 1 --2- 172.21.15.22:0/2570470872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bd65970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:34.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.413+0000 7f5149707640 1 --2- 172.21.15.22:0/2570470872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bce14a0 0x56107bcf2c80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f51380b89a0 tx=0x7f5138005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888e13d6ad3ad71e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:34.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.413+0000 7f51297fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:34.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.413+0000 7f51297fa640 1 -- 172.21.15.22:0/2570470872 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138049040 con 0x56107bce14a0 2024-09-19T08:23:34.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.413+0000 7f51297fa640 1 -- 172.21.15.22:0/2570470872 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f5138003c80 con 0x56107bce14a0 2024-09-19T08:23:34.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.413+0000 7f514e545740 1 -- 172.21.15.22:0/2570470872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5134005020 con 0x56107bce14a0 2024-09-19T08:23:34.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.413+0000 7f51297fa640 1 -- 172.21.15.22:0/2570470872 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f51380bc310 con 0x56107bce14a0 2024-09-19T08:23:34.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.413+0000 7f514e545740 10 client.16854 fetch_fsmap learned FSMap version 35 2024-09-19T08:23:34.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.413+0000 7f514e545740 1 -- 172.21.15.22:0/2570470872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56107bd67050 con 0x56107bce14a0 2024-09-19T08:23:34.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.413+0000 7f51297fa640 1 -- 172.21.15.22:0/2570470872 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5134005020 con 0x56107bce14a0 2024-09-19T08:23:34.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.413+0000 7f51297fa640 1 -- 172.21.15.22:0/2570470872 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f51380b8bb0 con 0x56107bce14a0 2024-09-19T08:23:34.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.414+0000 7f514e545740 10 client.16854 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:23:34.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.414+0000 7f514e545740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:23:34.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.414+0000 7f514e545740 20 client.16854 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.414+0000 7f514e545740 1 --2- 172.21.15.22:0/2570470872 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56107bcec680 0x56107bd69800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.414+0000 7f514e545740 4 client.16854 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:23:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.414+0000 7f514e545740 1 -- 172.21.15.22:0/2570470872 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56107bc4ed60 con 0x56107bcec680 2024-09-19T08:23:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.414+0000 7f5149f08640 1 --2- 172.21.15.22:0/2570470872 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56107bcec680 0x56107bd69800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.414+0000 7f51297fa640 10 client.16854 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.414+0000 7f5149f08640 1 --2- 172.21.15.22:0/2570470872 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56107bcec680 0x56107bd69800 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f9eceb287c6ecd server_cookie=42b40f1bebc8bb77 in_seq=0 out_seq=0 2024-09-19T08:23:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.414+0000 7f51297fa640 1 -- 172.21.15.22:0/2570470872 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56107bc4ed60 con 0x56107bcec680 2024-09-19T08:23:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.414+0000 7f51297fa640 10 client.16854 handle_command_reply: tid=0 2024-09-19T08:23:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.414+0000 7f514e545740 1 client.16854 shutdown 2024-09-19T08:23:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.415+0000 7f514e545740 1 -- 172.21.15.22:0/2570470872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bce14a0 msgr2=0x56107bcf2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.415+0000 7f514e545740 1 --2- 172.21.15.22:0/2570470872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bce14a0 0x56107bcf2c80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f51380b89a0 tx=0x7f5138005a20 comp rx=0 tx=0).stop 2024-09-19T08:23:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.415+0000 7f514e545740 1 -- 172.21.15.22:0/2570470872 shutdown_connections 2024-09-19T08:23:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.415+0000 7f514e545740 1 --2- 172.21.15.22:0/2570470872 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x56107bcec680 0x56107bd69800 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.415+0000 7f514e545740 1 --2- 172.21.15.22:0/2570470872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56107bce1dc0 0x56107bd65970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.415+0000 7f514e545740 1 --2- 172.21.15.22:0/2570470872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56107bce14a0 0x56107bcf2c80 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.415+0000 7f51297fa640 0 client.16854 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:34.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.415+0000 7f514e545740 1 --2- 172.21.15.22:0/2570470872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56107bcdda30 0x56107bcf2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.415+0000 7f514e545740 1 -- 172.21.15.22:0/2570470872 >> 172.21.15.22:0/2570470872 conn(0x56107bce98c0 msgr2=0x56107bcdc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:34.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.415+0000 7f514e545740 1 -- 172.21.15.22:0/2570470872 shutdown_connections 2024-09-19T08:23:34.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.415+0000 7f514e545740 1 -- 172.21.15.22:0/2570470872 wait complete. 2024-09-19T08:23:34.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.415+0000 7f514e545740 20 client.16854 trim_cache size 0 max 16384 2024-09-19T08:23:34.422 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:23:34.423 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-19T08:23:34.423 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-09-19T08:23:34.423 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:23:34.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.416+0000 7f514c16c640 1 -- 172.21.15.22:0/2984272686 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f512c0a3430 msgr2=0x7f512c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.416+0000 7f514c16c640 1 --2- 172.21.15.22:0/2984272686 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f512c0a3430 0x7f512c0a58f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f513c021000 tx=0x7f513c01e000 comp rx=0 tx=0).stop 2024-09-19T08:23:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.416+0000 7f514c16c640 1 -- 172.21.15.22:0/2984272686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51441027a0 msgr2=0x7f5144074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.416+0000 7f514c16c640 1 --2- 172.21.15.22:0/2984272686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51441027a0 0x7f5144074a30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5138005940 tx=0x7f5138004270 comp rx=0 tx=0).stop 2024-09-19T08:23:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.416+0000 7f514c16c640 1 -- 172.21.15.22:0/2984272686 shutdown_connections 2024-09-19T08:23:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.416+0000 7f514c16c640 1 --2- 172.21.15.22:0/2984272686 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f512c0a3430 0x7f512c0a58f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.416+0000 7f514c16c640 1 --2- 172.21.15.22:0/2984272686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5144106f20 0x7f5144074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.416+0000 7f514c16c640 1 --2- 172.21.15.22:0/2984272686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51441027a0 0x7f5144074a30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.416+0000 7f514c16c640 1 --2- 172.21.15.22:0/2984272686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5144101df0 0x7f51440744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.416+0000 7f514c16c640 1 -- 172.21.15.22:0/2984272686 >> 172.21.15.22:0/2984272686 conn(0x7f51440fd8f0 msgr2=0x7f51440fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.416+0000 7f514c16c640 1 -- 172.21.15.22:0/2984272686 shutdown_connections 2024-09-19T08:23:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:34.416+0000 7f514c16c640 1 -- 172.21.15.22:0/2984272686 wait complete. 2024-09-19T08:23:34.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:34 smithi022 ceph-mon[30963]: pgmap v263: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 227 op/s 2024-09-19T08:23:34.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:34 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:23:34.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:34 smithi064 ceph-mon[34900]: pgmap v263: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 227 op/s 2024-09-19T08:23:34.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:34 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:23:35.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:34 smithi157 ceph-mon[35143]: pgmap v263: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 227 op/s 2024-09-19T08:23:35.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:34 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:23:35.273 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:23:35.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:23:35.531 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:23:36.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.089+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/4182004036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb8100a80 msgr2=0x7fcdb8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:36.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.089+0000 7fcdbe5ac640 1 --2- 172.21.15.22:0/4182004036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb8100a80 0x7fcdb8100ee0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009930 tx=0x7fcdac02a420 comp rx=0 tx=0).stop 2024-09-19T08:23:36.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.089+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/4182004036 shutdown_connections 2024-09-19T08:23:36.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.089+0000 7fcdbe5ac640 1 --2- 172.21.15.22:0/4182004036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdb8101420 0x7fcdb810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:36.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.089+0000 7fcdbe5ac640 1 --2- 172.21.15.22:0/4182004036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb8100a80 0x7fcdb8100ee0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.089+0000 7fcdbe5ac640 1 --2- 172.21.15.22:0/4182004036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdb8106a80 0x7fcdb8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.089+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/4182004036 >> 172.21.15.22:0/4182004036 conn(0x7fcdb80fc7a0 msgr2=0x7fcdb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.089+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/4182004036 shutdown_connections 2024-09-19T08:23:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.089+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/4182004036 wait complete. 2024-09-19T08:23:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.090+0000 7fcdbe5ac640 1 Processor -- start 2024-09-19T08:23:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.090+0000 7fcdbe5ac640 1 -- start start 2024-09-19T08:23:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.090+0000 7fcdbe5ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdb8100a80 0x7fcdb8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.090+0000 7fcdbe5ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdb8101420 0x7fcdb8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.090+0000 7fcdbe5ac640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb8106a80 0x7fcdb819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.090+0000 7fcdbe5ac640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8110e80 con 0x7fcdb8106a80 2024-09-19T08:23:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.090+0000 7fcdbe5ac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8110d00 con 0x7fcdb8101420 2024-09-19T08:23:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.090+0000 7fcdbe5ac640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8111000 con 0x7fcdb8100a80 2024-09-19T08:23:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdbcb49640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb8106a80 0x7fcdb819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:36.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdb7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdb8100a80 0x7fcdb8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:36.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdbcb49640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb8106a80 0x7fcdb819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41464/0 (socket says 172.21.15.22:41464) 2024-09-19T08:23:36.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdbcb49640 1 -- 172.21.15.22:0/43568250 learned_addr learned my addr 172.21.15.22:0/43568250 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:36.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdb77fe640 1 --2- 172.21.15.22:0/43568250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdb8101420 0x7fcdb8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:36.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdbcb49640 1 -- 172.21.15.22:0/43568250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdb8100a80 msgr2=0x7fcdb8198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:36.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdbcb49640 1 --2- 172.21.15.22:0/43568250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdb8100a80 0x7fcdb8198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:36.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdbcb49640 1 -- 172.21.15.22:0/43568250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdb8101420 msgr2=0x7fcdb8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:36.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdbcb49640 1 --2- 172.21.15.22:0/43568250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdb8101420 0x7fcdb8198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:36.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdbcb49640 1 -- 172.21.15.22:0/43568250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb819d3d0 con 0x7fcdb8106a80 2024-09-19T08:23:36.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdb7fff640 1 --2- 172.21.15.22:0/43568250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdb8100a80 0x7fcdb8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:36.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdbcb49640 1 --2- 172.21.15.22:0/43568250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb8106a80 0x7fcdb819ccf0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000eb10 tx=0x7fcdb000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:36.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdb57fa640 1 -- 172.21.15.22:0/43568250 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0004b20 con 0x7fcdb8106a80 2024-09-19T08:23:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/43568250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb819d6c0 con 0x7fcdb8106a80 2024-09-19T08:23:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdb57fa640 1 -- 172.21.15.22:0/43568250 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb0004cc0 con 0x7fcdb8106a80 2024-09-19T08:23:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdb57fa640 1 -- 172.21.15.22:0/43568250 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0010760 con 0x7fcdb8106a80 2024-09-19T08:23:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.091+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/43568250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb81a4fa0 con 0x7fcdb8106a80 2024-09-19T08:23:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.093+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/43568250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb8100ee0 con 0x7fcdb8106a80 2024-09-19T08:23:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.094+0000 7fcdb57fa640 1 -- 172.21.15.22:0/43568250 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcdb0005020 con 0x7fcdb8106a80 2024-09-19T08:23:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.094+0000 7fcdb57fa640 1 --2- 172.21.15.22:0/43568250 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcda00a33b0 0x7fcda00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.095+0000 7fcdb7fff640 1 --2- 172.21.15.22:0/43568250 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcda00a33b0 0x7fcda00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.095+0000 7fcdb57fa640 1 -- 172.21.15.22:0/43568250 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fcdb00d03d0 con 0x7fcdb8106a80 2024-09-19T08:23:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.095+0000 7fcdb7fff640 1 --2- 172.21.15.22:0/43568250 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcda00a33b0 0x7fcda00a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009f40 tx=0x7fcda8005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.099+0000 7fcdb57fa640 1 -- 172.21.15.22:0/43568250 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcdb009cda0 con 0x7fcdb8106a80 2024-09-19T08:23:36.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.314+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/43568250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdb802d7c0 con 0x7fcdb8106a80 2024-09-19T08:23:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.315+0000 7fcdb57fa640 1 -- 172.21.15.22:0/43568250 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11908 (secure 0 0 0) 0x7fcdb009c6a0 con 0x7fcdb8106a80 2024-09-19T08:23:36.315 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:23:36.315 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"btime":"2024-09-19T08:23:33:502329+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:23:33.442439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:23:36.316 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-09-19T08:23:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.317+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/43568250 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcda00a33b0 msgr2=0x7fcda00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.317+0000 7fcdbe5ac640 1 --2- 172.21.15.22:0/43568250 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcda00a33b0 0x7fcda00a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009f40 tx=0x7fcda8005f90 comp rx=0 tx=0).stop 2024-09-19T08:23:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.317+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/43568250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb8106a80 msgr2=0x7fcdb819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.317+0000 7fcdbe5ac640 1 --2- 172.21.15.22:0/43568250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb8106a80 0x7fcdb819ccf0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000eb10 tx=0x7fcdb000c2d0 comp rx=0 tx=0).stop 2024-09-19T08:23:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.317+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/43568250 shutdown_connections 2024-09-19T08:23:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.317+0000 7fcdbe5ac640 1 --2- 172.21.15.22:0/43568250 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fcda00a33b0 0x7fcda00a5870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.317+0000 7fcdbe5ac640 1 --2- 172.21.15.22:0/43568250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb8106a80 0x7fcdb819ccf0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.317+0000 7fcdbe5ac640 1 --2- 172.21.15.22:0/43568250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdb8101420 0x7fcdb8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.317+0000 7fcdbe5ac640 1 --2- 172.21.15.22:0/43568250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdb8100a80 0x7fcdb8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.317+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/43568250 >> 172.21.15.22:0/43568250 conn(0x7fcdb80fc7a0 msgr2=0x7fcdb806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.317+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/43568250 shutdown_connections 2024-09-19T08:23:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:36.317+0000 7fcdbe5ac640 1 -- 172.21.15.22:0/43568250 wait complete. 2024-09-19T08:23:36.687 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:36 smithi022 ceph-mon[30963]: pgmap v264: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 223 op/s 2024-09-19T08:23:36.687 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:36 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/43568250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:23:36.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:36 smithi064 ceph-mon[34900]: pgmap v264: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 223 op/s 2024-09-19T08:23:36.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:36 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/43568250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:23:37.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:36 smithi157 ceph-mon[35143]: pgmap v264: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 223 op/s 2024-09-19T08:23:37.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:36 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/43568250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:23:37.052 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:23:37.317 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:23:37.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.862+0000 7f486606e640 1 -- 172.21.15.22:0/749483987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f486010c170 msgr2=0x7f486010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:37.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.862+0000 7f486606e640 1 --2- 172.21.15.22:0/749483987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f486010c170 0x7f486010e610 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4858009ca0 tx=0x7f485802a950 comp rx=0 tx=0).stop 2024-09-19T08:23:37.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.862+0000 7f486606e640 1 -- 172.21.15.22:0/749483987 shutdown_connections 2024-09-19T08:23:37.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.862+0000 7f486606e640 1 --2- 172.21.15.22:0/749483987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f486010c170 0x7f486010e610 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.862+0000 7f486606e640 1 --2- 172.21.15.22:0/749483987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4860103660 0x7f486010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.862+0000 7f486606e640 1 --2- 172.21.15.22:0/749483987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4860102d40 0x7f4860103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.862+0000 7f486606e640 1 -- 172.21.15.22:0/749483987 >> 172.21.15.22:0/749483987 conn(0x7f48600fc7a0 msgr2=0x7f48600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:37.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.862+0000 7f486606e640 1 -- 172.21.15.22:0/749483987 shutdown_connections 2024-09-19T08:23:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.862+0000 7f486606e640 1 -- 172.21.15.22:0/749483987 wait complete. 2024-09-19T08:23:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.863+0000 7f486606e640 1 Processor -- start 2024-09-19T08:23:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.863+0000 7f486606e640 1 -- start start 2024-09-19T08:23:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.864+0000 7f486606e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4860102d40 0x7f48601009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.864+0000 7f486606e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4860103660 0x7f4860100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.864+0000 7f486606e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f486010c170 0x7f4860101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.864+0000 7f486606e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4860111fa0 con 0x7f4860103660 2024-09-19T08:23:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.864+0000 7f486606e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4860111e20 con 0x7f486010c170 2024-09-19T08:23:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.864+0000 7f486606e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4860112120 con 0x7f4860102d40 2024-09-19T08:23:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.864+0000 7f485ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f486010c170 0x7f4860101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.864+0000 7f485f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4860102d40 0x7f48601009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.864+0000 7f485ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f486010c170 0x7f4860101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:41140/0 (socket says 172.21.15.22:41140) 2024-09-19T08:23:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.864+0000 7f485f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4860102d40 0x7f48601009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:43952/0 (socket says 172.21.15.22:43952) 2024-09-19T08:23:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.864+0000 7f485ffff640 1 -- 172.21.15.22:0/4125530573 learned_addr learned my addr 172.21.15.22:0/4125530573 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.865+0000 7f485f7fe640 1 -- 172.21.15.22:0/4125530573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f486010c170 msgr2=0x7f4860101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.865+0000 7f485f7fe640 1 --2- 172.21.15.22:0/4125530573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f486010c170 0x7f4860101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.865+0000 7f485f7fe640 1 -- 172.21.15.22:0/4125530573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4860103660 msgr2=0x7f4860100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.865+0000 7f485effd640 1 --2- 172.21.15.22:0/4125530573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4860103660 0x7f4860100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.865+0000 7f485f7fe640 1 --2- 172.21.15.22:0/4125530573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4860103660 0x7f4860100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.865+0000 7f485f7fe640 1 -- 172.21.15.22:0/4125530573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48601aae30 con 0x7f4860102d40 2024-09-19T08:23:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.865+0000 7f485effd640 1 --2- 172.21.15.22:0/4125530573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4860103660 0x7f4860100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:23:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.867+0000 7f485f7fe640 1 --2- 172.21.15.22:0/4125530573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4860102d40 0x7f48601009a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f485000d6d0 tx=0x7f485000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:37.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.867+0000 7f485cff9640 1 -- 172.21.15.22:0/4125530573 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850017070 con 0x7f4860102d40 2024-09-19T08:23:37.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.867+0000 7f485cff9640 1 -- 172.21.15.22:0/4125530573 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4850009820 con 0x7f4860102d40 2024-09-19T08:23:37.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.867+0000 7f485cff9640 1 -- 172.21.15.22:0/4125530573 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850004d00 con 0x7f4860102d40 2024-09-19T08:23:37.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.867+0000 7f486606e640 1 -- 172.21.15.22:0/4125530573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48601ab060 con 0x7f4860102d40 2024-09-19T08:23:37.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.868+0000 7f486606e640 1 -- 172.21.15.22:0/4125530573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48601ab550 con 0x7f4860102d40 2024-09-19T08:23:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.871+0000 7f4868447740 1 Processor -- start 2024-09-19T08:23:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.871+0000 7f4868447740 1 -- start start 2024-09-19T08:23:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.871+0000 7f485cff9640 1 -- 172.21.15.22:0/4125530573 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f48500099d0 con 0x7f4860102d40 2024-09-19T08:23:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.871+0000 7f4868447740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 0x55bdfe68a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.871+0000 7f4868447740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe68a5d0 0x55bdfe68aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.872+0000 7f4868447740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe695370 0x55bdfe697760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.872+0000 7f4868447740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bdfe5f02f0 con 0x55bdfe68a5d0 2024-09-19T08:23:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.872+0000 7f4868447740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bdfe597cf0 con 0x55bdfe695370 2024-09-19T08:23:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.872+0000 7f4868447740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bdfe5ec2b0 con 0x55bdfe68bb40 2024-09-19T08:23:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.872+0000 7f485f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe68a5d0 0x55bdfe68aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.872+0000 7f485effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe695370 0x55bdfe697760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.872+0000 7f485effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe695370 0x55bdfe697760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:41154/0 (socket says 172.21.15.22:41154) 2024-09-19T08:23:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.872+0000 7f485effd640 1 -- 172.21.15.22:0/494446804 learned_addr learned my addr 172.21.15.22:0/494446804 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.872+0000 7f485ffff640 1 --2- 172.21.15.22:0/494446804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 0x55bdfe68a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.873+0000 7f485effd640 1 -- 172.21.15.22:0/494446804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 msgr2=0x55bdfe68a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.873+0000 7f485effd640 1 --2- 172.21.15.22:0/494446804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 0x55bdfe68a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.873+0000 7f485effd640 1 -- 172.21.15.22:0/494446804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe68a5d0 msgr2=0x55bdfe68aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.873+0000 7f485effd640 1 --2- 172.21.15.22:0/494446804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe68a5d0 0x55bdfe68aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.873+0000 7f485effd640 1 -- 172.21.15.22:0/494446804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdfe57c3a0 con 0x55bdfe695370 2024-09-19T08:23:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.873+0000 7f485cff9640 1 --2- 172.21.15.22:0/4125530573 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f48480a3430 0x7f48480a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.873+0000 7f485f7fe640 1 --2- 172.21.15.22:0/494446804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe68a5d0 0x55bdfe68aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:23:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.873+0000 7f485cff9640 1 -- 172.21.15.22:0/4125530573 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f48500cf860 con 0x7f4860102d40 2024-09-19T08:23:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.873+0000 7f485ffff640 1 --2- 172.21.15.22:0/4125530573 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f48480a3430 0x7f48480a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.874+0000 7f485effd640 1 --2- 172.21.15.22:0/494446804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe695370 0x55bdfe697760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4854009c00 tx=0x7f4854010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b19f1d5ec408ddd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.874+0000 7f4845ffb640 1 -- 172.21.15.22:0/494446804 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854004020 con 0x55bdfe695370 2024-09-19T08:23:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.874+0000 7f4845ffb640 1 -- 172.21.15.22:0/494446804 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48540041c0 con 0x55bdfe695370 2024-09-19T08:23:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.874+0000 7f4845ffb640 1 -- 172.21.15.22:0/494446804 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48540126b0 con 0x55bdfe695370 2024-09-19T08:23:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.875+0000 7f485ffff640 1 --2- 172.21.15.22:0/4125530573 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f48480a3430 0x7f48480a58f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f48580098d0 tx=0x7f4858009900 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.875+0000 7f4868447740 1 -- 172.21.15.22:0/494446804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe695370 msgr2=0x55bdfe697760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.875+0000 7f4868447740 1 --2- 172.21.15.22:0/494446804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe695370 0x55bdfe697760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4854009c00 tx=0x7f4854010740 comp rx=0 tx=0).stop 2024-09-19T08:23:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.876+0000 7f4868447740 1 -- 172.21.15.22:0/494446804 shutdown_connections 2024-09-19T08:23:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.876+0000 7f4868447740 1 --2- 172.21.15.22:0/494446804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe695370 0x55bdfe697760 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.876+0000 7f4868447740 1 --2- 172.21.15.22:0/494446804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe68a5d0 0x55bdfe68aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.876+0000 7f4868447740 1 --2- 172.21.15.22:0/494446804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 0x55bdfe68a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.876+0000 7f4868447740 1 -- 172.21.15.22:0/494446804 >> 172.21.15.22:0/494446804 conn(0x55bdfe653310 msgr2=0x55bdfe686c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.876+0000 7f4868447740 1 -- 172.21.15.22:0/494446804 shutdown_connections 2024-09-19T08:23:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.876+0000 7f4868447740 1 -- 172.21.15.22:0/494446804 wait complete. 2024-09-19T08:23:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.877+0000 7f4868447740 1 -- start start 2024-09-19T08:23:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.877+0000 7f4868447740 1 Processor -- start 2024-09-19T08:23:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.877+0000 7f4868447740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68a5d0 0x55bdfe694d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.877+0000 7f4868447740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe68bb40 0x55bdfe692010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.877+0000 7f4868447740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe695370 0x55bdfe692600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.877+0000 7f4868447740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bdfe698070 con 0x55bdfe695370 2024-09-19T08:23:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.877+0000 7f4868447740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bdfe5ab060 con 0x55bdfe68bb40 2024-09-19T08:23:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.877+0000 7f4868447740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bdfe6981f0 con 0x55bdfe68a5d0 2024-09-19T08:23:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.877+0000 7f485ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe695370 0x55bdfe692600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.877+0000 7f485ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe695370 0x55bdfe692600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45376/0 (socket says 172.21.15.22:45376) 2024-09-19T08:23:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.877+0000 7f485ffff640 1 -- 172.21.15.22:0/1813070083 learned_addr learned my addr 172.21.15.22:0/1813070083 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.877+0000 7f485f7fe640 1 --2- 172.21.15.22:0/1813070083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68a5d0 0x55bdfe694d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.878+0000 7f4868447740 10 client.0 _ll_register_callbacks cb 0x55bdfe2fe8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.878+0000 7f485ffff640 1 -- 172.21.15.22:0/1813070083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68a5d0 msgr2=0x55bdfe694d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.878+0000 7f485ffff640 1 --2- 172.21.15.22:0/1813070083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68a5d0 0x55bdfe694d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.878+0000 7f485ffff640 1 -- 172.21.15.22:0/1813070083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe68bb40 msgr2=0x55bdfe692010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:23:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.878+0000 7f485ffff640 1 --2- 172.21.15.22:0/1813070083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe68bb40 0x55bdfe692010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.878+0000 7f485ffff640 1 -- 172.21.15.22:0/1813070083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdfe3613c0 con 0x55bdfe695370 2024-09-19T08:23:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.879+0000 7f485ffff640 1 --2- 172.21.15.22:0/1813070083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe695370 0x55bdfe692600 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f485802f8b0 tx=0x7f48580095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390e9333738aa536 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.879+0000 7f483effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.880+0000 7f483effd640 1 -- 172.21.15.22:0/1813070083 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858034040 con 0x55bdfe695370 2024-09-19T08:23:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.880+0000 7f483effd640 1 -- 172.21.15.22:0/1813070083 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f4858004450 con 0x55bdfe695370 2024-09-19T08:23:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.880+0000 7f4868447740 1 -- 172.21.15.22:0/1813070083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48540041c0 con 0x55bdfe695370 2024-09-19T08:23:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.880+0000 7f483effd640 1 -- 172.21.15.22:0/1813070083 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4858004650 con 0x55bdfe695370 2024-09-19T08:23:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.881+0000 7f4868447740 10 client.16869 fetch_fsmap learned FSMap version 35 2024-09-19T08:23:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.881+0000 7f4868447740 1 -- 172.21.15.22:0/1813070083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bdfe6a3b10 con 0x55bdfe695370 2024-09-19T08:23:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.881+0000 7f483effd640 1 -- 172.21.15.22:0/1813070083 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48540041c0 con 0x55bdfe695370 2024-09-19T08:23:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.881+0000 7f483effd640 1 -- 172.21.15.22:0/1813070083 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f4858050070 con 0x55bdfe695370 2024-09-19T08:23:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.881+0000 7f4868447740 10 client.16869 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:23:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.881+0000 7f4868447740 10 client.16869 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:23:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.881+0000 7f4868447740 20 client.16869 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.881+0000 7f4868447740 1 --2- 172.21.15.22:0/1813070083 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55bdfe6a4f90 0x55bdfe6a7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.882+0000 7f4868447740 4 client.16869 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:23:37.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.882+0000 7f4868447740 1 -- 172.21.15.22:0/1813070083 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdfe5fd860 con 0x55bdfe6a4f90 2024-09-19T08:23:37.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.882+0000 7f485f7fe640 1 --2- 172.21.15.22:0/1813070083 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55bdfe6a4f90 0x55bdfe6a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.883+0000 7f485f7fe640 1 --2- 172.21.15.22:0/1813070083 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55bdfe6a4f90 0x55bdfe6a7450 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29b5b33c093e316d server_cookie=a58022fde8f48cba in_seq=0 out_seq=0 2024-09-19T08:23:37.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.883+0000 7f483effd640 10 client.16869 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:23:37.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.886+0000 7f483effd640 1 -- 172.21.15.22:0/1813070083 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdfe5fd860 con 0x55bdfe6a4f90 2024-09-19T08:23:37.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.886+0000 7f483effd640 10 client.16869 handle_command_reply: tid=0 2024-09-19T08:23:37.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.886+0000 7f4868447740 1 client.16869 shutdown 2024-09-19T08:23:37.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.887+0000 7f4868447740 1 -- 172.21.15.22:0/1813070083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe695370 msgr2=0x55bdfe692600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:37.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.887+0000 7f4868447740 1 --2- 172.21.15.22:0/1813070083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe695370 0x55bdfe692600 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f485802f8b0 tx=0x7f48580095c0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.887+0000 7f4868447740 1 -- 172.21.15.22:0/1813070083 shutdown_connections 2024-09-19T08:23:37.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.887+0000 7f4868447740 1 --2- 172.21.15.22:0/1813070083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe695370 0x55bdfe692600 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.887+0000 7f4868447740 1 --2- 172.21.15.22:0/1813070083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe68bb40 0x55bdfe692010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.887+0000 7f4868447740 1 --2- 172.21.15.22:0/1813070083 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55bdfe6a4f90 0x55bdfe6a7450 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.887+0000 7f4868447740 1 --2- 172.21.15.22:0/1813070083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68a5d0 0x55bdfe694d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.887+0000 7f4868447740 1 -- 172.21.15.22:0/1813070083 >> 172.21.15.22:0/1813070083 conn(0x55bdfe653310 msgr2=0x55bdfe68c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:37.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.887+0000 7f483effd640 0 client.16869 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:23:37.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.888+0000 7f4868447740 1 -- 172.21.15.22:0/1813070083 shutdown_connections 2024-09-19T08:23:37.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.888+0000 7f4868447740 1 -- 172.21.15.22:0/1813070083 wait complete. 2024-09-19T08:23:37.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.888+0000 7f4868447740 20 client.16869 trim_cache size 0 max 16384 2024-09-19T08:23:37.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.910+0000 7f4868447740 1 Processor -- start 2024-09-19T08:23:37.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.910+0000 7f4868447740 1 -- start start 2024-09-19T08:23:37.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.910+0000 7f4868447740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe68a5d0 0x55bdfe693f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.910+0000 7f4868447740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 0x55bdfe68d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f4868447740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe695370 0x55bdfe68d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f4868447740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f48601b68f0 con 0x55bdfe695370 2024-09-19T08:23:37.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f4868447740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f48601b4190 con 0x55bdfe68a5d0 2024-09-19T08:23:37.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f4868447740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f48601b2000 con 0x55bdfe68bb40 2024-09-19T08:23:37.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f485ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe695370 0x55bdfe68d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f485ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe695370 0x55bdfe68d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45392/0 (socket says 172.21.15.22:45392) 2024-09-19T08:23:37.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f485ffff640 1 -- 172.21.15.22:0/1966519698 learned_addr learned my addr 172.21.15.22:0/1966519698 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:37.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f485f7fe640 1 --2- 172.21.15.22:0/1966519698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe68a5d0 0x55bdfe693f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f485effd640 1 --2- 172.21.15.22:0/1966519698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 0x55bdfe68d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f485ffff640 1 -- 172.21.15.22:0/1966519698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 msgr2=0x55bdfe68d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:37.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f485ffff640 1 --2- 172.21.15.22:0/1966519698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 0x55bdfe68d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f485ffff640 1 -- 172.21.15.22:0/1966519698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe68a5d0 msgr2=0x55bdfe693f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:37.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f485ffff640 1 --2- 172.21.15.22:0/1966519698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe68a5d0 0x55bdfe693f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f485ffff640 1 -- 172.21.15.22:0/1966519698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdfe690560 con 0x55bdfe695370 2024-09-19T08:23:37.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.911+0000 7f485ffff640 1 --2- 172.21.15.22:0/1966519698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe695370 0x55bdfe68d660 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f485802af40 tx=0x7f4858003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba76f466c576273b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:37.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.912+0000 7f48457fa640 1 -- 172.21.15.22:0/1966519698 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858030830 con 0x55bdfe695370 2024-09-19T08:23:37.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.912+0000 7f48457fa640 1 -- 172.21.15.22:0/1966519698 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4858004250 con 0x55bdfe695370 2024-09-19T08:23:37.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.912+0000 7f48457fa640 1 -- 172.21.15.22:0/1966519698 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858052b20 con 0x55bdfe695370 2024-09-19T08:23:37.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.912+0000 7f4868447740 1 -- 172.21.15.22:0/1966519698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe695370 msgr2=0x55bdfe68d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.912+0000 7f4868447740 1 --2- 172.21.15.22:0/1966519698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe695370 0x55bdfe68d660 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f485802af40 tx=0x7f4858003f70 comp rx=0 tx=0).stop 2024-09-19T08:23:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.912+0000 7f4868447740 1 -- 172.21.15.22:0/1966519698 shutdown_connections 2024-09-19T08:23:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.912+0000 7f4868447740 1 --2- 172.21.15.22:0/1966519698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe695370 0x55bdfe68d660 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.912+0000 7f4868447740 1 --2- 172.21.15.22:0/1966519698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 0x55bdfe68d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.912+0000 7f4868447740 1 --2- 172.21.15.22:0/1966519698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe68a5d0 0x55bdfe693f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.912+0000 7f4868447740 1 -- 172.21.15.22:0/1966519698 >> 172.21.15.22:0/1966519698 conn(0x55bdfe653310 msgr2=0x55bdfe693870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.912+0000 7f4868447740 1 -- 172.21.15.22:0/1966519698 shutdown_connections 2024-09-19T08:23:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.913+0000 7f4868447740 1 -- 172.21.15.22:0/1966519698 wait complete. 2024-09-19T08:23:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.913+0000 7f4868447740 1 -- start start 2024-09-19T08:23:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.913+0000 7f4868447740 1 Processor -- start 2024-09-19T08:23:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.913+0000 7f4868447740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe68a5d0 0x55bdfe699620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.913+0000 7f4868447740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 0x55bdfe699b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.913+0000 7f4868447740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe695370 0x55bdfe70ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.913+0000 7f4868447740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f48601b2650 con 0x55bdfe68a5d0 2024-09-19T08:23:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.913+0000 7f4868447740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f48601b3ac0 con 0x55bdfe695370 2024-09-19T08:23:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.913+0000 7f4868447740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f48601b2d20 con 0x55bdfe68bb40 2024-09-19T08:23:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.913+0000 7f4868447740 10 client.0 _ll_register_callbacks cb 0x7f4858009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.913+0000 7f485f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe68a5d0 0x55bdfe699620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.913+0000 7f485f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe68a5d0 0x55bdfe699620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45400/0 (socket says 172.21.15.22:45400) 2024-09-19T08:23:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.913+0000 7f485f7fe640 1 -- 172.21.15.22:0/194782479 learned_addr learned my addr 172.21.15.22:0/194782479 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.914+0000 7f485ffff640 1 --2- 172.21.15.22:0/194782479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe695370 0x55bdfe70ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.914+0000 7f485effd640 1 --2- 172.21.15.22:0/194782479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 0x55bdfe699b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.914+0000 7f485f7fe640 1 -- 172.21.15.22:0/194782479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 msgr2=0x55bdfe699b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.914+0000 7f485f7fe640 1 --2- 172.21.15.22:0/194782479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 0x55bdfe699b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.914+0000 7f485f7fe640 1 -- 172.21.15.22:0/194782479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe695370 msgr2=0x55bdfe70ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.914+0000 7f485f7fe640 1 --2- 172.21.15.22:0/194782479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe695370 0x55bdfe70ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.914+0000 7f485f7fe640 1 -- 172.21.15.22:0/194782479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdfe70f230 con 0x55bdfe68a5d0 2024-09-19T08:23:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.914+0000 7f485f7fe640 1 --2- 172.21.15.22:0/194782479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe68a5d0 0x55bdfe699620 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f4850098500 tx=0x7f4850086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ca1bd74cacb371 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.914+0000 7f483effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.914+0000 7f483effd640 1 -- 172.21.15.22:0/194782479 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485009ce60 con 0x55bdfe68a5d0 2024-09-19T08:23:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.914+0000 7f483effd640 1 -- 172.21.15.22:0/194782479 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f4850009d40 con 0x55bdfe68a5d0 2024-09-19T08:23:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.914+0000 7f4868447740 1 -- 172.21.15.22:0/194782479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4858004250 con 0x55bdfe68a5d0 2024-09-19T08:23:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.915+0000 7f483effd640 1 -- 172.21.15.22:0/194782479 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4850020040 con 0x55bdfe68a5d0 2024-09-19T08:23:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.915+0000 7f4868447740 10 client.16875 fetch_fsmap learned FSMap version 35 2024-09-19T08:23:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.915+0000 7f4868447740 1 -- 172.21.15.22:0/194782479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bdfe710470 con 0x55bdfe68a5d0 2024-09-19T08:23:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.915+0000 7f483effd640 1 -- 172.21.15.22:0/194782479 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4858004250 con 0x55bdfe68a5d0 2024-09-19T08:23:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.915+0000 7f483effd640 1 -- 172.21.15.22:0/194782479 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f48500a4310 con 0x55bdfe68a5d0 2024-09-19T08:23:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.915+0000 7f4868447740 10 client.16875 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:23:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.915+0000 7f4868447740 10 client.16875 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:23:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.915+0000 7f4868447740 20 client.16875 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.915+0000 7f4868447740 1 --2- 172.21.15.22:0/194782479 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55bdfe710b90 0x55bdfe712f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.915+0000 7f4868447740 4 client.16875 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:23:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.915+0000 7f4868447740 1 -- 172.21.15.22:0/194782479 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bdfe5f7d70 con 0x55bdfe710b90 2024-09-19T08:23:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.915+0000 7f485effd640 1 --2- 172.21.15.22:0/194782479 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55bdfe710b90 0x55bdfe712f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.916+0000 7f483effd640 10 client.16875 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:23:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.916+0000 7f485effd640 1 --2- 172.21.15.22:0/194782479 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55bdfe710b90 0x55bdfe712f80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed8c7f1a0faad151 server_cookie=3442c534c637d93a in_seq=0 out_seq=0 2024-09-19T08:23:37.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f483effd640 1 -- 172.21.15.22:0/194782479 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bdfe5f7d70 con 0x55bdfe710b90 2024-09-19T08:23:37.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f483effd640 10 client.16875 handle_command_reply: tid=0 2024-09-19T08:23:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f4868447740 1 client.16875 shutdown 2024-09-19T08:23:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f4868447740 1 -- 172.21.15.22:0/194782479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe68a5d0 msgr2=0x55bdfe699620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f4868447740 1 --2- 172.21.15.22:0/194782479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe68a5d0 0x55bdfe699620 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f4850098500 tx=0x7f4850086040 comp rx=0 tx=0).stop 2024-09-19T08:23:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f4868447740 1 -- 172.21.15.22:0/194782479 shutdown_connections 2024-09-19T08:23:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f4868447740 1 --2- 172.21.15.22:0/194782479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdfe695370 0x55bdfe70ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f4868447740 1 --2- 172.21.15.22:0/194782479 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55bdfe710b90 0x55bdfe712f80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f4868447740 1 --2- 172.21.15.22:0/194782479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bdfe68bb40 0x55bdfe699b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f4868447740 1 --2- 172.21.15.22:0/194782479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bdfe68a5d0 0x55bdfe699620 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f4868447740 1 -- 172.21.15.22:0/194782479 >> 172.21.15.22:0/194782479 conn(0x55bdfe653310 msgr2=0x55bdfe694090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f483effd640 0 client.16875 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:23:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f4868447740 1 -- 172.21.15.22:0/194782479 shutdown_connections 2024-09-19T08:23:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f4868447740 1 -- 172.21.15.22:0/194782479 wait complete. 2024-09-19T08:23:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.920+0000 7f4868447740 20 client.16875 trim_cache size 0 max 16384 2024-09-19T08:23:37.924 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:23:37.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.921+0000 7f486606e640 1 -- 172.21.15.22:0/4125530573 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f48480a3430 msgr2=0x7f48480a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.921+0000 7f486606e640 1 --2- 172.21.15.22:0/4125530573 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f48480a3430 0x7f48480a58f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f48580098d0 tx=0x7f4858009900 comp rx=0 tx=0).stop 2024-09-19T08:23:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.921+0000 7f486606e640 1 -- 172.21.15.22:0/4125530573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4860102d40 msgr2=0x7f48601009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.921+0000 7f486606e640 1 --2- 172.21.15.22:0/4125530573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4860102d40 0x7f48601009a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f485000d6d0 tx=0x7f485000db90 comp rx=0 tx=0).stop 2024-09-19T08:23:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.921+0000 7f486606e640 1 -- 172.21.15.22:0/4125530573 shutdown_connections 2024-09-19T08:23:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.921+0000 7f486606e640 1 --2- 172.21.15.22:0/4125530573 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f48480a3430 0x7f48480a58f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.921+0000 7f486606e640 1 --2- 172.21.15.22:0/4125530573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f486010c170 0x7f4860101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.921+0000 7f486606e640 1 --2- 172.21.15.22:0/4125530573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4860103660 0x7f4860100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.921+0000 7f486606e640 1 --2- 172.21.15.22:0/4125530573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4860102d40 0x7f48601009a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.921+0000 7f486606e640 1 -- 172.21.15.22:0/4125530573 >> 172.21.15.22:0/4125530573 conn(0x7f48600fc7a0 msgr2=0x7f48600fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.921+0000 7f486606e640 1 -- 172.21.15.22:0/4125530573 shutdown_connections 2024-09-19T08:23:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:37.921+0000 7f486606e640 1 -- 172.21.15.22:0/4125530573 wait complete. 2024-09-19T08:23:38.675 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:38 smithi022 ceph-mon[30963]: pgmap v265: 97 pgs: 97 active+clean; 1007 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 283 op/s 2024-09-19T08:23:38.702 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:23:38.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:38 smithi064 ceph-mon[34900]: pgmap v265: 97 pgs: 97 active+clean; 1007 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 283 op/s 2024-09-19T08:23:39.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:38 smithi157 ceph-mon[35143]: pgmap v265: 97 pgs: 97 active+clean; 1007 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 283 op/s 2024-09-19T08:23:39.013 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:23:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.495+0000 7f7659bbb640 1 -- 172.21.15.22:0/2102372503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76541037a0 msgr2=0x7f765410bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.495+0000 7f7659bbb640 1 --2- 172.21.15.22:0/2102372503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76541037a0 0x7f765410bc00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7648009900 tx=0x7f764802a410 comp rx=0 tx=0).stop 2024-09-19T08:23:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.495+0000 7f7659bbb640 1 -- 172.21.15.22:0/2102372503 shutdown_connections 2024-09-19T08:23:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.495+0000 7f7659bbb640 1 --2- 172.21.15.22:0/2102372503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f765410c2f0 0x7f765410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.495+0000 7f7659bbb640 1 --2- 172.21.15.22:0/2102372503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76541037a0 0x7f765410bc00 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.495+0000 7f7659bbb640 1 --2- 172.21.15.22:0/2102372503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7654102df0 0x7f76541031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.495+0000 7f7659bbb640 1 -- 172.21.15.22:0/2102372503 >> 172.21.15.22:0/2102372503 conn(0x7f76540fc7d0 msgr2=0x7f76540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.495+0000 7f7659bbb640 1 -- 172.21.15.22:0/2102372503 shutdown_connections 2024-09-19T08:23:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.495+0000 7f7659bbb640 1 -- 172.21.15.22:0/2102372503 wait complete. 2024-09-19T08:23:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.496+0000 7f7659bbb640 1 Processor -- start 2024-09-19T08:23:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.496+0000 7f7659bbb640 1 -- start start 2024-09-19T08:23:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.496+0000 7f7659bbb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7654102df0 0x7f7654198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.496+0000 7f7659bbb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76541037a0 0x7f7654198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.496+0000 7f7659bbb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f765410c2f0 0x7f765419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.496+0000 7f7659bbb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f76541120d0 con 0x7f7654102df0 2024-09-19T08:23:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.496+0000 7f7659bbb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7654111f50 con 0x7f765410c2f0 2024-09-19T08:23:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.496+0000 7f7659bbb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7654112250 con 0x7f76541037a0 2024-09-19T08:23:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.496+0000 7f76537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7654102df0 0x7f7654198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.496+0000 7f76537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7654102df0 0x7f7654198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45414/0 (socket says 172.21.15.22:45414) 2024-09-19T08:23:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.496+0000 7f76537fe640 1 -- 172.21.15.22:0/2510867151 learned_addr learned my addr 172.21.15.22:0/2510867151 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.497+0000 7f7652ffd640 1 --2- 172.21.15.22:0/2510867151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76541037a0 0x7f7654198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.497+0000 7f7653fff640 1 --2- 172.21.15.22:0/2510867151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f765410c2f0 0x7f765419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.497+0000 7f76537fe640 1 -- 172.21.15.22:0/2510867151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76541037a0 msgr2=0x7f7654198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.497+0000 7f76537fe640 1 --2- 172.21.15.22:0/2510867151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76541037a0 0x7f7654198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.497+0000 7f76537fe640 1 -- 172.21.15.22:0/2510867151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f765410c2f0 msgr2=0x7f765419cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.497+0000 7f76537fe640 1 --2- 172.21.15.22:0/2510867151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f765410c2f0 0x7f765419cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.497+0000 7f76537fe640 1 -- 172.21.15.22:0/2510867151 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765419d5c0 con 0x7f7654102df0 2024-09-19T08:23:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.497+0000 7f76537fe640 1 --2- 172.21.15.22:0/2510867151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7654102df0 0x7f7654198610 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f764400e9f0 tx=0x7f764400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.497+0000 7f7650ff9640 1 -- 172.21.15.22:0/2510867151 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764400cd80 con 0x7f7654102df0 2024-09-19T08:23:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.497+0000 7f7659bbb640 1 -- 172.21.15.22:0/2510867151 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765419d8b0 con 0x7f7654102df0 2024-09-19T08:23:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.497+0000 7f7650ff9640 1 -- 172.21.15.22:0/2510867151 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7644010910 con 0x7f7654102df0 2024-09-19T08:23:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.497+0000 7f7650ff9640 1 -- 172.21.15.22:0/2510867151 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76440188f0 con 0x7f7654102df0 2024-09-19T08:23:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.497+0000 7f7659bbb640 1 -- 172.21.15.22:0/2510867151 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76541a5190 con 0x7f7654102df0 2024-09-19T08:23:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f7650ff9640 1 -- 172.21.15.22:0/2510867151 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7644010ac0 con 0x7f7654102df0 2024-09-19T08:23:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f765bf94740 1 Processor -- start 2024-09-19T08:23:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f765bf94740 1 -- start start 2024-09-19T08:23:39.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f765bf94740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a5b40 0x55b41b1a4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f765bf94740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a45d0 0x55b41b1a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f765bf94740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1af370 0x55b41b1b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f765bf94740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b41b10a2f0 con 0x55b41b1a45d0 2024-09-19T08:23:39.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f765bf94740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b41b0b1cf0 con 0x55b41b1af370 2024-09-19T08:23:39.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f765bf94740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b41b1062b0 con 0x55b41b1a5b40 2024-09-19T08:23:39.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f7650ff9640 1 --2- 172.21.15.22:0/2510867151 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f763c0a33b0 0x7f763c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f7653fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a5b40 0x55b41b1a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f76537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a45d0 0x55b41b1a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f7653fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a5b40 0x55b41b1a4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:44006/0 (socket says 172.21.15.22:44006) 2024-09-19T08:23:39.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f7653fff640 1 -- 172.21.15.22:0/258556072 learned_addr learned my addr 172.21.15.22:0/258556072 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:39.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.500+0000 7f7652ffd640 1 --2- 172.21.15.22:0/258556072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1af370 0x55b41b1b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.501+0000 7f7653fff640 1 --2- 172.21.15.22:0/2510867151 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f763c0a33b0 0x7f763c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.501+0000 7f7653fff640 1 -- 172.21.15.22:0/258556072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1af370 msgr2=0x55b41b1b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:39.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.501+0000 7f7650ff9640 1 -- 172.21.15.22:0/2510867151 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f76440d00d0 con 0x7f7654102df0 2024-09-19T08:23:39.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.501+0000 7f7653fff640 1 --2- 172.21.15.22:0/258556072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1af370 0x55b41b1b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.501+0000 7f7653fff640 1 -- 172.21.15.22:0/258556072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a45d0 msgr2=0x55b41b1a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:39.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.501+0000 7f7653fff640 1 --2- 172.21.15.22:0/258556072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a45d0 0x55b41b1a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.501+0000 7f7653fff640 1 -- 172.21.15.22:0/258556072 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b41b0963a0 con 0x55b41b1a5b40 2024-09-19T08:23:39.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.501+0000 7f7652ffd640 1 --2- 172.21.15.22:0/258556072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1af370 0x55b41b1b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:23:39.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.501+0000 7f7653fff640 1 --2- 172.21.15.22:0/258556072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a5b40 0x55b41b1a4090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f764c0129f0 tx=0x7f764c012eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41a008f55cd93fbb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:39.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.501+0000 7f7653fff640 1 --2- 172.21.15.22:0/2510867151 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f763c0a33b0 0x7f763c0a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f764c01c000 tx=0x7f764c020000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:39.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.501+0000 7f7639ffb640 1 -- 172.21.15.22:0/258556072 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c0120d0 con 0x55b41b1a5b40 2024-09-19T08:23:39.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.501+0000 7f7639ffb640 1 -- 172.21.15.22:0/258556072 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f764c023020 con 0x55b41b1a5b40 2024-09-19T08:23:39.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.501+0000 7f7639ffb640 1 -- 172.21.15.22:0/258556072 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c004030 con 0x55b41b1a5b40 2024-09-19T08:23:39.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.502+0000 7f765bf94740 1 -- 172.21.15.22:0/258556072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a5b40 msgr2=0x55b41b1a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.502+0000 7f765bf94740 1 --2- 172.21.15.22:0/258556072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a5b40 0x55b41b1a4090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f764c0129f0 tx=0x7f764c012eb0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.502+0000 7f765bf94740 1 -- 172.21.15.22:0/258556072 shutdown_connections 2024-09-19T08:23:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.502+0000 7f765bf94740 1 --2- 172.21.15.22:0/258556072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1af370 0x55b41b1b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.502+0000 7f765bf94740 1 --2- 172.21.15.22:0/258556072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a45d0 0x55b41b1a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.502+0000 7f765bf94740 1 --2- 172.21.15.22:0/258556072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a5b40 0x55b41b1a4090 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.502+0000 7f765bf94740 1 -- 172.21.15.22:0/258556072 >> 172.21.15.22:0/258556072 conn(0x55b41b16d310 msgr2=0x55b41b1a0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.502+0000 7f765bf94740 1 -- 172.21.15.22:0/258556072 shutdown_connections 2024-09-19T08:23:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.502+0000 7f765bf94740 1 -- 172.21.15.22:0/258556072 wait complete. 2024-09-19T08:23:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.502+0000 7f765bf94740 1 -- start start 2024-09-19T08:23:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.502+0000 7f765bf94740 1 Processor -- start 2024-09-19T08:23:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f765bf94740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1a45d0 0x55b41b1aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f765bf94740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a5b40 0x55b41b1ac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f765bf94740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1af370 0x55b41b1ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f765bf94740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b41b1b1fb0 con 0x55b41b1af370 2024-09-19T08:23:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f765bf94740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b41b0c5060 con 0x55b41b1a45d0 2024-09-19T08:23:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f765bf94740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b41b1b2130 con 0x55b41b1a5b40 2024-09-19T08:23:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f765bf94740 10 client.0 _ll_register_callbacks cb 0x55b41ae188c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f7653fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1af370 0x55b41b1ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f7652ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a5b40 0x55b41b1ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f7653fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1af370 0x55b41b1ac7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45424/0 (socket says 172.21.15.22:45424) 2024-09-19T08:23:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f7653fff640 1 -- 172.21.15.22:0/2022804960 learned_addr learned my addr 172.21.15.22:0/2022804960 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f76537fe640 1 --2- 172.21.15.22:0/2022804960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1a45d0 0x55b41b1aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f7653fff640 1 -- 172.21.15.22:0/2022804960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a5b40 msgr2=0x55b41b1ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f7653fff640 1 --2- 172.21.15.22:0/2022804960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a5b40 0x55b41b1ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f7653fff640 1 -- 172.21.15.22:0/2022804960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1a45d0 msgr2=0x55b41b1aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f7653fff640 1 --2- 172.21.15.22:0/2022804960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1a45d0 0x55b41b1aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f7653fff640 1 -- 172.21.15.22:0/2022804960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b41ae7b3c0 con 0x55b41b1af370 2024-09-19T08:23:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f7653fff640 1 --2- 172.21.15.22:0/2022804960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1af370 0x55b41b1ac7e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f764c006010 tx=0x7f764c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d1dc92e88c6648 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f7632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.503+0000 7f7632ffd640 1 -- 172.21.15.22:0/2022804960 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c01f980 con 0x55b41b1af370 2024-09-19T08:23:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.504+0000 7f7632ffd640 1 -- 172.21.15.22:0/2022804960 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f764c01fb20 con 0x55b41b1af370 2024-09-19T08:23:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.504+0000 7f765bf94740 1 -- 172.21.15.22:0/2022804960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f764c023020 con 0x55b41b1af370 2024-09-19T08:23:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.504+0000 7f7632ffd640 1 -- 172.21.15.22:0/2022804960 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f764c01fd20 con 0x55b41b1af370 2024-09-19T08:23:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.504+0000 7f765bf94740 10 client.16887 fetch_fsmap learned FSMap version 35 2024-09-19T08:23:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.504+0000 7f765bf94740 1 -- 172.21.15.22:0/2022804960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b41b1bdae0 con 0x55b41b1af370 2024-09-19T08:23:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.504+0000 7f7632ffd640 1 -- 172.21.15.22:0/2022804960 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f764c023020 con 0x55b41b1af370 2024-09-19T08:23:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.504+0000 7f7632ffd640 1 -- 172.21.15.22:0/2022804960 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f764c01e070 con 0x55b41b1af370 2024-09-19T08:23:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.504+0000 7f765bf94740 10 client.16887 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:23:39.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.504+0000 7f765bf94740 10 client.16887 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:23:39.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.504+0000 7f765bf94740 20 client.16887 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:39.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.504+0000 7f765bf94740 1 --2- 172.21.15.22:0/2022804960 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b41b1beef0 0x55b41b1c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.504+0000 7f765bf94740 4 client.16887 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:23:39.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.504+0000 7f765bf94740 1 -- 172.21.15.22:0/2022804960 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b41b117860 con 0x55b41b1beef0 2024-09-19T08:23:39.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.505+0000 7f76537fe640 1 --2- 172.21.15.22:0/2022804960 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b41b1beef0 0x55b41b1c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.505+0000 7f76537fe640 1 --2- 172.21.15.22:0/2022804960 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b41b1beef0 0x55b41b1c12e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b25f1e80bc1d9872 server_cookie=67fc7d85c3394bff in_seq=0 out_seq=0 2024-09-19T08:23:39.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.505+0000 7f7632ffd640 10 client.16887 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:23:39.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f7632ffd640 1 -- 172.21.15.22:0/2022804960 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b41b117860 con 0x55b41b1beef0 2024-09-19T08:23:39.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f7632ffd640 10 client.16887 handle_command_reply: tid=0 2024-09-19T08:23:39.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f765bf94740 1 client.16887 shutdown 2024-09-19T08:23:39.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f765bf94740 1 -- 172.21.15.22:0/2022804960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1af370 msgr2=0x55b41b1ac7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:39.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f765bf94740 1 --2- 172.21.15.22:0/2022804960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1af370 0x55b41b1ac7e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f764c006010 tx=0x7f764c009500 comp rx=0 tx=0).stop 2024-09-19T08:23:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f765bf94740 1 -- 172.21.15.22:0/2022804960 shutdown_connections 2024-09-19T08:23:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f765bf94740 1 --2- 172.21.15.22:0/2022804960 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b41b1beef0 0x55b41b1c12e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f765bf94740 1 --2- 172.21.15.22:0/2022804960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1af370 0x55b41b1ac7e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f765bf94740 1 --2- 172.21.15.22:0/2022804960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a5b40 0x55b41b1ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f765bf94740 1 --2- 172.21.15.22:0/2022804960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1a45d0 0x55b41b1aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f765bf94740 1 -- 172.21.15.22:0/2022804960 >> 172.21.15.22:0/2022804960 conn(0x55b41b16d310 msgr2=0x55b41b1a6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f7632ffd640 0 client.16887 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:23:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f765bf94740 1 -- 172.21.15.22:0/2022804960 shutdown_connections 2024-09-19T08:23:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f765bf94740 1 -- 172.21.15.22:0/2022804960 wait complete. 2024-09-19T08:23:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.506+0000 7f765bf94740 20 client.16887 trim_cache size 0 max 16384 2024-09-19T08:23:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.519+0000 7f765bf94740 1 Processor -- start 2024-09-19T08:23:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f765bf94740 1 -- start start 2024-09-19T08:23:39.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f765bf94740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a45d0 0x55b41b1ab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f765bf94740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a5b40 0x55b41b1a7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f765bf94740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1af370 0x55b41b1a7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f765bf94740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f76541b0290 con 0x55b41b1a5b40 2024-09-19T08:23:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f765bf94740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f76541adb30 con 0x55b41b1af370 2024-09-19T08:23:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f765bf94740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f76541ab9a0 con 0x55b41b1a45d0 2024-09-19T08:23:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f7652ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a5b40 0x55b41b1a7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f7652ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a5b40 0x55b41b1a7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45438/0 (socket says 172.21.15.22:45438) 2024-09-19T08:23:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f7652ffd640 1 -- 172.21.15.22:0/58627716 learned_addr learned my addr 172.21.15.22:0/58627716 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f76537fe640 1 --2- 172.21.15.22:0/58627716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a45d0 0x55b41b1ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f7653fff640 1 --2- 172.21.15.22:0/58627716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1af370 0x55b41b1a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f7652ffd640 1 -- 172.21.15.22:0/58627716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a45d0 msgr2=0x55b41b1ab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f7652ffd640 1 --2- 172.21.15.22:0/58627716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a45d0 0x55b41b1ab970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f7652ffd640 1 -- 172.21.15.22:0/58627716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1af370 msgr2=0x55b41b1a7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f7652ffd640 1 --2- 172.21.15.22:0/58627716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1af370 0x55b41b1a7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f7652ffd640 1 -- 172.21.15.22:0/58627716 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b41b1aa5e0 con 0x55b41b1a5b40 2024-09-19T08:23:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.520+0000 7f7652ffd640 1 --2- 172.21.15.22:0/58627716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a5b40 0x55b41b1a7120 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f76540697d0 tx=0x7f7648031420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8eab0f2cc6d15d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.521+0000 7f76397fa640 1 -- 172.21.15.22:0/58627716 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648035040 con 0x55b41b1a5b40 2024-09-19T08:23:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.521+0000 7f76397fa640 1 -- 172.21.15.22:0/58627716 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76480317b0 con 0x55b41b1a5b40 2024-09-19T08:23:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.521+0000 7f76397fa640 1 -- 172.21.15.22:0/58627716 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648004750 con 0x55b41b1a5b40 2024-09-19T08:23:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.522+0000 7f765bf94740 1 -- 172.21.15.22:0/58627716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a5b40 msgr2=0x55b41b1a7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.522+0000 7f765bf94740 1 --2- 172.21.15.22:0/58627716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a5b40 0x55b41b1a7120 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f76540697d0 tx=0x7f7648031420 comp rx=0 tx=0).stop 2024-09-19T08:23:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.522+0000 7f765bf94740 1 -- 172.21.15.22:0/58627716 shutdown_connections 2024-09-19T08:23:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.522+0000 7f765bf94740 1 --2- 172.21.15.22:0/58627716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1af370 0x55b41b1a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.522+0000 7f765bf94740 1 --2- 172.21.15.22:0/58627716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a5b40 0x55b41b1a7120 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.522+0000 7f765bf94740 1 --2- 172.21.15.22:0/58627716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1a45d0 0x55b41b1ab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.522+0000 7f765bf94740 1 -- 172.21.15.22:0/58627716 >> 172.21.15.22:0/58627716 conn(0x55b41b16d310 msgr2=0x55b41b1ae0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.522+0000 7f765bf94740 1 -- 172.21.15.22:0/58627716 shutdown_connections 2024-09-19T08:23:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.522+0000 7f765bf94740 1 -- 172.21.15.22:0/58627716 wait complete. 2024-09-19T08:23:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f765bf94740 1 -- start start 2024-09-19T08:23:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f765bf94740 1 Processor -- start 2024-09-19T08:23:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f765bf94740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1a45d0 0x55b41b1b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f765bf94740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a5b40 0x55b41b1b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f765bf94740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1af370 0x55b41b1a8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f765bf94740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f76541abff0 con 0x55b41b1a5b40 2024-09-19T08:23:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f765bf94740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f76541ad460 con 0x55b41b1a45d0 2024-09-19T08:23:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f765bf94740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f76541ac6c0 con 0x55b41b1af370 2024-09-19T08:23:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f765bf94740 10 client.0 _ll_register_callbacks cb 0x7f764409c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f76537fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1a45d0 0x55b41b1b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f7653fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1af370 0x55b41b1a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f7652ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a5b40 0x55b41b1b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f7653fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1af370 0x55b41b1a8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:44030/0 (socket says 172.21.15.22:44030) 2024-09-19T08:23:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f76537fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1a45d0 0x55b41b1b75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:41228/0 (socket says 172.21.15.22:41228) 2024-09-19T08:23:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f7653fff640 1 -- 172.21.15.22:0/1512058217 learned_addr learned my addr 172.21.15.22:0/1512058217 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.523+0000 7f7652ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a5b40 0x55b41b1b7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45442/0 (socket says 172.21.15.22:45442) 2024-09-19T08:23:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.524+0000 7f7652ffd640 1 -- 172.21.15.22:0/1512058217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1af370 msgr2=0x55b41b1a8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.524+0000 7f7652ffd640 1 --2- 172.21.15.22:0/1512058217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1af370 0x55b41b1a8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.524+0000 7f7652ffd640 1 -- 172.21.15.22:0/1512058217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1a45d0 msgr2=0x55b41b1b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.524+0000 7f7652ffd640 1 --2- 172.21.15.22:0/1512058217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1a45d0 0x55b41b1b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.524+0000 7f7652ffd640 1 -- 172.21.15.22:0/1512058217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b41b1a9130 con 0x55b41b1a5b40 2024-09-19T08:23:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.524+0000 7f7653fff640 1 --2- 172.21.15.22:0/1512058217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1af370 0x55b41b1a8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.524+0000 7f7652ffd640 1 --2- 172.21.15.22:0/1512058217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a5b40 0x55b41b1b7ae0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x55b41b1ad600 tx=0x7f7648005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcea99a0a879529 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:39.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.524+0000 7f7632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:39.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.524+0000 7f7632ffd640 1 -- 172.21.15.22:0/1512058217 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648035040 con 0x55b41b1a5b40 2024-09-19T08:23:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.524+0000 7f7632ffd640 1 -- 172.21.15.22:0/1512058217 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f7648004040 con 0x55b41b1a5b40 2024-09-19T08:23:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.524+0000 7f765bf94740 1 -- 172.21.15.22:0/1512058217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76480317b0 con 0x55b41b1a5b40 2024-09-19T08:23:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.524+0000 7f7632ffd640 1 -- 172.21.15.22:0/1512058217 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7648058050 con 0x55b41b1a5b40 2024-09-19T08:23:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.524+0000 7f765bf94740 10 client.16893 fetch_fsmap learned FSMap version 35 2024-09-19T08:23:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.524+0000 7f765bf94740 1 -- 172.21.15.22:0/1512058217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b41b229570 con 0x55b41b1a5b40 2024-09-19T08:23:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.525+0000 7f7632ffd640 1 -- 172.21.15.22:0/1512058217 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76480317b0 con 0x55b41b1a5b40 2024-09-19T08:23:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.525+0000 7f7632ffd640 1 -- 172.21.15.22:0/1512058217 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f7648004c30 con 0x55b41b1a5b40 2024-09-19T08:23:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.525+0000 7f765bf94740 10 client.16893 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:23:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.525+0000 7f765bf94740 10 client.16893 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:23:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.525+0000 7f765bf94740 20 client.16893 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.525+0000 7f765bf94740 1 --2- 172.21.15.22:0/1512058217 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b41b229c70 0x55b41b22c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.525+0000 7f765bf94740 4 client.16893 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:23:39.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.525+0000 7f765bf94740 1 -- 172.21.15.22:0/1512058217 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b41b111d70 con 0x55b41b229c70 2024-09-19T08:23:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.525+0000 7f76537fe640 1 --2- 172.21.15.22:0/1512058217 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b41b229c70 0x55b41b22c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f7632ffd640 10 client.16893 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:23:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.525+0000 7f76537fe640 1 --2- 172.21.15.22:0/1512058217 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b41b229c70 0x55b41b22c090 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e932c14061b57a server_cookie=54285dd2fbaf74ea in_seq=0 out_seq=0 2024-09-19T08:23:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f7632ffd640 1 -- 172.21.15.22:0/1512058217 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b41b111d70 con 0x55b41b229c70 2024-09-19T08:23:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f7632ffd640 10 client.16893 handle_command_reply: tid=0 2024-09-19T08:23:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f765bf94740 1 client.16893 shutdown 2024-09-19T08:23:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f765bf94740 1 -- 172.21.15.22:0/1512058217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a5b40 msgr2=0x55b41b1b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f765bf94740 1 --2- 172.21.15.22:0/1512058217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a5b40 0x55b41b1b7ae0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x55b41b1ad600 tx=0x7f7648005fd0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f765bf94740 1 -- 172.21.15.22:0/1512058217 shutdown_connections 2024-09-19T08:23:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f765bf94740 1 --2- 172.21.15.22:0/1512058217 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55b41b229c70 0x55b41b22c090 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f7632ffd640 0 client.16893 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:23:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f765bf94740 1 --2- 172.21.15.22:0/1512058217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b41b1af370 0x55b41b1a8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f765bf94740 1 --2- 172.21.15.22:0/1512058217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b41b1a5b40 0x55b41b1b7ae0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f765bf94740 1 --2- 172.21.15.22:0/1512058217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b41b1a45d0 0x55b41b1b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f765bf94740 1 -- 172.21.15.22:0/1512058217 >> 172.21.15.22:0/1512058217 conn(0x55b41b16d310 msgr2=0x55b41b19f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:39.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f765bf94740 1 -- 172.21.15.22:0/1512058217 shutdown_connections 2024-09-19T08:23:39.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f765bf94740 1 -- 172.21.15.22:0/1512058217 wait complete. 2024-09-19T08:23:39.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.526+0000 7f765bf94740 20 client.16893 trim_cache size 0 max 16384 2024-09-19T08:23:39.542 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:23:39.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.527+0000 7f7659bbb640 1 -- 172.21.15.22:0/2510867151 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f763c0a33b0 msgr2=0x7f763c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:39.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.527+0000 7f7659bbb640 1 --2- 172.21.15.22:0/2510867151 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f763c0a33b0 0x7f763c0a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f764c01c000 tx=0x7f764c020000 comp rx=0 tx=0).stop 2024-09-19T08:23:39.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.527+0000 7f7659bbb640 1 -- 172.21.15.22:0/2510867151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7654102df0 msgr2=0x7f7654198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:39.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.527+0000 7f7659bbb640 1 --2- 172.21.15.22:0/2510867151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7654102df0 0x7f7654198610 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f764400e9f0 tx=0x7f764400eeb0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.527+0000 7f7659bbb640 1 -- 172.21.15.22:0/2510867151 shutdown_connections 2024-09-19T08:23:39.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.527+0000 7f7659bbb640 1 --2- 172.21.15.22:0/2510867151 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f763c0a33b0 0x7f763c0a5870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.527+0000 7f7659bbb640 1 --2- 172.21.15.22:0/2510867151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f765410c2f0 0x7f765419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.527+0000 7f7659bbb640 1 --2- 172.21.15.22:0/2510867151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76541037a0 0x7f7654198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.527+0000 7f7659bbb640 1 --2- 172.21.15.22:0/2510867151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7654102df0 0x7f7654198610 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:39.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.527+0000 7f7659bbb640 1 -- 172.21.15.22:0/2510867151 >> 172.21.15.22:0/2510867151 conn(0x7f76540fc7d0 msgr2=0x7f76540fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:39.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.527+0000 7f7659bbb640 1 -- 172.21.15.22:0/2510867151 shutdown_connections 2024-09-19T08:23:39.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:39.527+0000 7f7659bbb640 1 -- 172.21.15.22:0/2510867151 wait complete. 2024-09-19T08:23:40.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:23:40.504 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:23:40.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:40 smithi022 ceph-mon[30963]: pgmap v266: 97 pgs: 97 active+clean; 998 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 535 op/s 2024-09-19T08:23:40.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:40 smithi064 ceph-mon[34900]: pgmap v266: 97 pgs: 97 active+clean; 998 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 535 op/s 2024-09-19T08:23:41.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:40 smithi157 ceph-mon[35143]: pgmap v266: 97 pgs: 97 active+clean; 998 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 535 op/s 2024-09-19T08:23:41.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.044+0000 7f9b29b97640 1 -- 172.21.15.22:0/1580899302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b24109450 msgr2=0x7f9b24110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:41.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.044+0000 7f9b29b97640 1 --2- 172.21.15.22:0/1580899302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b24109450 0x7f9b24110250 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009900 tx=0x7f9b0c02a3f0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.044+0000 7f9b29b97640 1 -- 172.21.15.22:0/1580899302 shutdown_connections 2024-09-19T08:23:41.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.044+0000 7f9b29b97640 1 --2- 172.21.15.22:0/1580899302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b2411b140 0x7f9b2411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.044+0000 7f9b29b97640 1 --2- 172.21.15.22:0/1580899302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b24109830 0x7f9b24109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.044+0000 7f9b29b97640 1 --2- 172.21.15.22:0/1580899302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b24109450 0x7f9b24110250 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.044+0000 7f9b29b97640 1 -- 172.21.15.22:0/1580899302 >> 172.21.15.22:0/1580899302 conn(0x7f9b2406e140 msgr2=0x7f9b2406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:41.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.044+0000 7f9b29b97640 1 -- 172.21.15.22:0/1580899302 shutdown_connections 2024-09-19T08:23:41.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.044+0000 7f9b29b97640 1 -- 172.21.15.22:0/1580899302 wait complete. 2024-09-19T08:23:41.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.045+0000 7f9b29b97640 1 Processor -- start 2024-09-19T08:23:41.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.046+0000 7f9b29b97640 1 -- start start 2024-09-19T08:23:41.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.047+0000 7f9b29b97640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b24109450 0x7f9b241a7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.047+0000 7f9b29b97640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b24109830 0x7f9b241a7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.047+0000 7f9b29b97640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b2411b140 0x7f9b241abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.047+0000 7f9b29b97640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24120df0 con 0x7f9b2411b140 2024-09-19T08:23:41.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.047+0000 7f9b29b97640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24120c70 con 0x7f9b24109450 2024-09-19T08:23:41.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.047+0000 7f9b29b97640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24120f70 con 0x7f9b24109830 2024-09-19T08:23:41.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.047+0000 7f9b1b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b24109450 0x7f9b241a7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.047+0000 7f9b1b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b24109450 0x7f9b241a7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:41252/0 (socket says 172.21.15.22:41252) 2024-09-19T08:23:41.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.047+0000 7f9b1b7fe640 1 -- 172.21.15.22:0/4057799025 learned_addr learned my addr 172.21.15.22:0/4057799025 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:41.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.047+0000 7f9b237fe640 1 --2- 172.21.15.22:0/4057799025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b24109830 0x7f9b241a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.047+0000 7f9b23fff640 1 --2- 172.21.15.22:0/4057799025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b2411b140 0x7f9b241abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.048+0000 7f9b1b7fe640 1 -- 172.21.15.22:0/4057799025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b24109830 msgr2=0x7f9b241a7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:41.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.048+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/4057799025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b24109830 0x7f9b241a7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.048+0000 7f9b1b7fe640 1 -- 172.21.15.22:0/4057799025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b2411b140 msgr2=0x7f9b241abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:41.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.048+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/4057799025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b2411b140 0x7f9b241abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.048+0000 7f9b1b7fe640 1 -- 172.21.15.22:0/4057799025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b241ac200 con 0x7f9b24109450 2024-09-19T08:23:41.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.048+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/4057799025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b24109450 0x7f9b241a7230 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c02a960 tx=0x7f9b0c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:41.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.049+0000 7f9b217fa640 1 -- 172.21.15.22:0/4057799025 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c03f070 con 0x7f9b24109450 2024-09-19T08:23:41.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.049+0000 7f9b217fa640 1 -- 172.21.15.22:0/4057799025 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b0c031c90 con 0x7f9b24109450 2024-09-19T08:23:41.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.049+0000 7f9b29b97640 1 -- 172.21.15.22:0/4057799025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b241ac490 con 0x7f9b24109450 2024-09-19T08:23:41.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.049+0000 7f9b29b97640 1 -- 172.21.15.22:0/4057799025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b24110920 con 0x7f9b24109450 2024-09-19T08:23:41.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.049+0000 7f9b217fa640 1 -- 172.21.15.22:0/4057799025 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c035030 con 0x7f9b24109450 2024-09-19T08:23:41.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.053+0000 7f9b2bf70740 1 Processor -- start 2024-09-19T08:23:41.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.053+0000 7f9b217fa640 1 -- 172.21.15.22:0/4057799025 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9b0c042070 con 0x7f9b24109450 2024-09-19T08:23:41.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.053+0000 7f9b2bf70740 1 -- start start 2024-09-19T08:23:41.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.053+0000 7f9b2bf70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a0f9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.053+0000 7f9b2bf70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a0f95d0 0x560a2a0f9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.053+0000 7f9b2bf70740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a104370 0x560a2a106760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.053+0000 7f9b2bf70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560a2a05f2f0 con 0x560a2a0fab40 2024-09-19T08:23:41.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.053+0000 7f9b2bf70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560a2a006cf0 con 0x560a2a0f95d0 2024-09-19T08:23:41.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.053+0000 7f9b2bf70740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560a2a05b2b0 con 0x560a2a104370 2024-09-19T08:23:41.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.054+0000 7f9b237fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a104370 0x560a2a106760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.054+0000 7f9b23fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a0f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.054+0000 7f9b23fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a0f9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45466/0 (socket says 172.21.15.22:45466) 2024-09-19T08:23:41.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.054+0000 7f9b23fff640 1 -- 172.21.15.22:0/4192099864 learned_addr learned my addr 172.21.15.22:0/4192099864 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:41.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.054+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/4192099864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a0f95d0 0x560a2a0f9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.054+0000 7f9b237fe640 1 -- 172.21.15.22:0/4192099864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a0f95d0 msgr2=0x560a2a0f9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:41.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.054+0000 7f9b237fe640 1 --2- 172.21.15.22:0/4192099864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a0f95d0 0x560a2a0f9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.054+0000 7f9b237fe640 1 -- 172.21.15.22:0/4192099864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 msgr2=0x560a2a0f9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:41.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.054+0000 7f9b237fe640 1 --2- 172.21.15.22:0/4192099864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a0f9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.054+0000 7f9b237fe640 1 -- 172.21.15.22:0/4192099864 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a29feb3a0 con 0x560a2a104370 2024-09-19T08:23:41.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.054+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/4192099864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a0f95d0 0x560a2a0f9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:41.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.054+0000 7f9b23fff640 1 --2- 172.21.15.22:0/4192099864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a0f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:41.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.055+0000 7f9b217fa640 1 --2- 172.21.15.22:0/4057799025 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9b100a3480 0x7f9b100a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.055+0000 7f9b237fe640 1 --2- 172.21.15.22:0/4192099864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a104370 0x560a2a106760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009bf0 tx=0x7f9b14011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ff51300259228ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:41.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.055+0000 7f9b23fff640 1 --2- 172.21.15.22:0/4057799025 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9b100a3480 0x7f9b100a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.055+0000 7f9b217fa640 1 -- 172.21.15.22:0/4057799025 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f9b0c0f0650 con 0x7f9b24109450 2024-09-19T08:23:41.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.055+0000 7f9b19ffb640 1 -- 172.21.15.22:0/4192099864 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14004050 con 0x560a2a104370 2024-09-19T08:23:41.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.055+0000 7f9b19ffb640 1 -- 172.21.15.22:0/4192099864 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b14005500 con 0x560a2a104370 2024-09-19T08:23:41.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.055+0000 7f9b19ffb640 1 -- 172.21.15.22:0/4192099864 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14004e40 con 0x560a2a104370 2024-09-19T08:23:41.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.056+0000 7f9b2bf70740 1 -- 172.21.15.22:0/4192099864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a104370 msgr2=0x560a2a106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:41.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.056+0000 7f9b23fff640 1 --2- 172.21.15.22:0/4057799025 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9b100a3480 0x7f9b100a5940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0117f0 tx=0x7f9b1c009500 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:41.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.056+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/4192099864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a104370 0x560a2a106760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009bf0 tx=0x7f9b14011810 comp rx=0 tx=0).stop 2024-09-19T08:23:41.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.056+0000 7f9b2bf70740 1 -- 172.21.15.22:0/4192099864 shutdown_connections 2024-09-19T08:23:41.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.056+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/4192099864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a104370 0x560a2a106760 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.056+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/4192099864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a0f95d0 0x560a2a0f9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.056+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/4192099864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a0f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.056+0000 7f9b2bf70740 1 -- 172.21.15.22:0/4192099864 >> 172.21.15.22:0/4192099864 conn(0x560a2a0c2310 msgr2=0x560a2a0f5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:41.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.056+0000 7f9b2bf70740 1 -- 172.21.15.22:0/4192099864 shutdown_connections 2024-09-19T08:23:41.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.056+0000 7f9b2bf70740 1 -- 172.21.15.22:0/4192099864 wait complete. 2024-09-19T08:23:41.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.057+0000 7f9b2bf70740 1 -- start start 2024-09-19T08:23:41.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.057+0000 7f9b2bf70740 1 Processor -- start 2024-09-19T08:23:41.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.057+0000 7f9b2bf70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a0f95d0 0x560a2a103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.057+0000 7f9b2bf70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a100f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.057+0000 7f9b2bf70740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a104370 0x560a2a1014d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.057+0000 7f9b2bf70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560a2a107070 con 0x560a2a0fab40 2024-09-19T08:23:41.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.057+0000 7f9b2bf70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560a2a01a060 con 0x560a2a0f95d0 2024-09-19T08:23:41.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.057+0000 7f9b2bf70740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560a2a1071f0 con 0x560a2a104370 2024-09-19T08:23:41.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.057+0000 7f9b2bf70740 10 client.0 _ll_register_callbacks cb 0x560a29d6d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:41.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b1b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a0f95d0 0x560a2a103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b1b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a0f95d0 0x560a2a103cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:41274/0 (socket says 172.21.15.22:41274) 2024-09-19T08:23:41.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b1b7fe640 1 -- 172.21.15.22:0/2934820728 learned_addr learned my addr 172.21.15.22:0/2934820728 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:41.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b23fff640 1 --2- 172.21.15.22:0/2934820728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a104370 0x560a2a1014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b1b7fe640 1 -- 172.21.15.22:0/2934820728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a104370 msgr2=0x560a2a1014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:41.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/2934820728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a104370 0x560a2a1014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b1b7fe640 1 -- 172.21.15.22:0/2934820728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 msgr2=0x560a2a100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:41.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/2934820728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b1b7fe640 1 -- 172.21.15.22:0/2934820728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a29dd03c0 con 0x560a2a0f95d0 2024-09-19T08:23:41.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b237fe640 1 --2- 172.21.15.22:0/2934820728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:41.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/2934820728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a0f95d0 0x560a2a103cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0edd90 tx=0x7f9b0c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54b3501dbdd5e027 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:41.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:23:41.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b02ffd640 1 -- 172.21.15.22:0/2934820728 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c049070 con 0x560a2a0f95d0 2024-09-19T08:23:41.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b2bf70740 1 -- 172.21.15.22:0/2934820728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b14005500 con 0x560a2a0f95d0 2024-09-19T08:23:41.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.058+0000 7f9b02ffd640 1 -- 172.21.15.22:0/2934820728 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f9b0c0edfb0 con 0x560a2a0f95d0 2024-09-19T08:23:41.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.059+0000 7f9b02ffd640 1 -- 172.21.15.22:0/2934820728 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9b0c0bc4b0 con 0x560a2a0f95d0 2024-09-19T08:23:41.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.059+0000 7f9b2bf70740 10 client.26402 fetch_fsmap learned FSMap version 35 2024-09-19T08:23:41.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.059+0000 7f9b2bf70740 1 -- 172.21.15.22:0/2934820728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560a2a112ab0 con 0x560a2a0f95d0 2024-09-19T08:23:41.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.059+0000 7f9b02ffd640 1 -- 172.21.15.22:0/2934820728 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b14005500 con 0x560a2a0f95d0 2024-09-19T08:23:41.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.059+0000 7f9b02ffd640 1 -- 172.21.15.22:0/2934820728 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f9b0c03bbe0 con 0x560a2a0f95d0 2024-09-19T08:23:41.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.059+0000 7f9b2bf70740 10 client.26402 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:23:41.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.059+0000 7f9b2bf70740 10 client.26402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:23:41.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.059+0000 7f9b2bf70740 20 client.26402 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:41.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.059+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/2934820728 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560a2a113ef0 0x560a2a116380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.059+0000 7f9b2bf70740 4 client.26402 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:23:41.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.059+0000 7f9b2bf70740 1 -- 172.21.15.22:0/2934820728 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a2a06c860 con 0x560a2a113ef0 2024-09-19T08:23:41.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.059+0000 7f9b237fe640 1 --2- 172.21.15.22:0/2934820728 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560a2a113ef0 0x560a2a116380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.060+0000 7f9b237fe640 1 --2- 172.21.15.22:0/2934820728 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560a2a113ef0 0x560a2a116380 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac34cfadcd8c8e7b server_cookie=9545e4d3ea4f05f1 in_seq=0 out_seq=0 2024-09-19T08:23:41.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.060+0000 7f9b02ffd640 10 client.26402 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:41.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b02ffd640 1 -- 172.21.15.22:0/2934820728 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a2a06c860 con 0x560a2a113ef0 2024-09-19T08:23:41.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b02ffd640 10 client.26402 handle_command_reply: tid=0 2024-09-19T08:23:41.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b2bf70740 1 client.26402 shutdown 2024-09-19T08:23:41.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b2bf70740 1 -- 172.21.15.22:0/2934820728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a0f95d0 msgr2=0x560a2a103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:41.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/2934820728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a0f95d0 0x560a2a103cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0edd90 tx=0x7f9b0c0046c0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b2bf70740 1 -- 172.21.15.22:0/2934820728 shutdown_connections 2024-09-19T08:23:41.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/2934820728 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560a2a113ef0 0x560a2a116380 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b02ffd640 0 client.26402 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:41.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/2934820728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a104370 0x560a2a1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/2934820728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/2934820728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a0f95d0 0x560a2a103cc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b2bf70740 1 -- 172.21.15.22:0/2934820728 >> 172.21.15.22:0/2934820728 conn(0x560a2a0c2310 msgr2=0x560a2a0fb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:41.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b2bf70740 1 -- 172.21.15.22:0/2934820728 shutdown_connections 2024-09-19T08:23:41.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b2bf70740 1 -- 172.21.15.22:0/2934820728 wait complete. 2024-09-19T08:23:41.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.061+0000 7f9b2bf70740 20 client.26402 trim_cache size 0 max 16384 2024-09-19T08:23:41.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.074+0000 7f9b2bf70740 1 Processor -- start 2024-09-19T08:23:41.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.074+0000 7f9b2bf70740 1 -- start start 2024-09-19T08:23:41.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b2bf70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0f95d0 0x560a2a0f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b2bf70740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a0fab40 0x560a2a110340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b2bf70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a104370 0x560a2a11bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b2bf70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9b241c77f0 con 0x560a2a0f95d0 2024-09-19T08:23:41.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b2bf70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b241c5090 con 0x560a2a104370 2024-09-19T08:23:41.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b2bf70740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24117270 con 0x560a2a0fab40 2024-09-19T08:23:41.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b1b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0f95d0 0x560a2a0f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b1b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0f95d0 0x560a2a0f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45476/0 (socket says 172.21.15.22:45476) 2024-09-19T08:23:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b1b7fe640 1 -- 172.21.15.22:0/3169395358 learned_addr learned my addr 172.21.15.22:0/3169395358 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b237fe640 1 --2- 172.21.15.22:0/3169395358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a0fab40 0x560a2a110340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b23fff640 1 --2- 172.21.15.22:0/3169395358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a104370 0x560a2a11bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b1b7fe640 1 -- 172.21.15.22:0/3169395358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a0fab40 msgr2=0x560a2a110340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/3169395358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a0fab40 0x560a2a110340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b1b7fe640 1 -- 172.21.15.22:0/3169395358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a104370 msgr2=0x560a2a11bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/3169395358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a104370 0x560a2a11bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b1b7fe640 1 -- 172.21.15.22:0/3169395358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a2a106f50 con 0x560a2a0f95d0 2024-09-19T08:23:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/3169395358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0f95d0 0x560a2a0f5e10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0bc900 tx=0x7f9b0c0bae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddaa8fe45afd7d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b197fa640 1 -- 172.21.15.22:0/3169395358 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c049050 con 0x560a2a0f95d0 2024-09-19T08:23:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b197fa640 1 -- 172.21.15.22:0/3169395358 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b0c0c0c70 con 0x560a2a0f95d0 2024-09-19T08:23:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.075+0000 7f9b197fa640 1 -- 172.21.15.22:0/3169395358 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0bc2d0 con 0x560a2a0f95d0 2024-09-19T08:23:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.076+0000 7f9b2bf70740 1 -- 172.21.15.22:0/3169395358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0f95d0 msgr2=0x560a2a0f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.076+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/3169395358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0f95d0 0x560a2a0f5e10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0bc900 tx=0x7f9b0c0bae80 comp rx=0 tx=0).stop 2024-09-19T08:23:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.076+0000 7f9b2bf70740 1 -- 172.21.15.22:0/3169395358 shutdown_connections 2024-09-19T08:23:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.076+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/3169395358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a104370 0x560a2a11bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.076+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/3169395358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a0fab40 0x560a2a110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.076+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/3169395358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0f95d0 0x560a2a0f5e10 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.076+0000 7f9b2bf70740 1 -- 172.21.15.22:0/3169395358 >> 172.21.15.22:0/3169395358 conn(0x560a2a0c2310 msgr2=0x560a2a0f5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.076+0000 7f9b2bf70740 1 -- 172.21.15.22:0/3169395358 shutdown_connections 2024-09-19T08:23:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.076+0000 7f9b2bf70740 1 -- 172.21.15.22:0/3169395358 wait complete. 2024-09-19T08:23:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.076+0000 7f9b2bf70740 1 -- start start 2024-09-19T08:23:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b2bf70740 1 Processor -- start 2024-09-19T08:23:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b2bf70740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a0f95d0 0x560a2a10cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b2bf70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a107f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b2bf70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a104370 0x560a2a1084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a107f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a107f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45488/0 (socket says 172.21.15.22:45488) 2024-09-19T08:23:41.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b237fe640 1 -- 172.21.15.22:0/191081946 learned_addr learned my addr 172.21.15.22:0/191081946 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:41.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b2bf70740 1 -- 172.21.15.22:0/191081946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9b241178c0 con 0x560a2a0fab40 2024-09-19T08:23:41.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/191081946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a0f95d0 0x560a2a10cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b2bf70740 1 -- 172.21.15.22:0/191081946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24118d30 con 0x560a2a104370 2024-09-19T08:23:41.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b2bf70740 1 -- 172.21.15.22:0/191081946 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24117f90 con 0x560a2a0f95d0 2024-09-19T08:23:41.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b2bf70740 10 client.0 _ll_register_callbacks cb 0x7f9b0c0f6220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:41.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b237fe640 1 -- 172.21.15.22:0/191081946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a0f95d0 msgr2=0x560a2a10cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:41.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b237fe640 1 --2- 172.21.15.22:0/191081946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a0f95d0 0x560a2a10cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b237fe640 1 -- 172.21.15.22:0/191081946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a104370 msgr2=0x560a2a1084a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:41.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b237fe640 1 --2- 172.21.15.22:0/191081946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a104370 0x560a2a1084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b237fe640 1 -- 172.21.15.22:0/191081946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a2a108ca0 con 0x560a2a0fab40 2024-09-19T08:23:41.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.077+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/191081946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a0f95d0 0x560a2a10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:41.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.078+0000 7f9b237fe640 1 --2- 172.21.15.22:0/191081946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a107f60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9b14011d10 tx=0x7f9b14009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f785341cb6491aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:41.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.078+0000 7f9b02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:41.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.078+0000 7f9b02ffd640 1 -- 172.21.15.22:0/191081946 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b140037f0 con 0x560a2a0fab40 2024-09-19T08:23:41.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.078+0000 7f9b02ffd640 1 -- 172.21.15.22:0/191081946 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f9b14003990 con 0x560a2a0fab40 2024-09-19T08:23:41.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.078+0000 7f9b2bf70740 1 -- 172.21.15.22:0/191081946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b0c0c0c70 con 0x560a2a0fab40 2024-09-19T08:23:41.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.078+0000 7f9b02ffd640 1 -- 172.21.15.22:0/191081946 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9b1403d050 con 0x560a2a0fab40 2024-09-19T08:23:41.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.078+0000 7f9b2bf70740 10 client.16911 fetch_fsmap learned FSMap version 35 2024-09-19T08:23:41.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.078+0000 7f9b2bf70740 1 -- 172.21.15.22:0/191081946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560a2a17ec10 con 0x560a2a0fab40 2024-09-19T08:23:41.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.078+0000 7f9b02ffd640 1 -- 172.21.15.22:0/191081946 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b0c0c0c70 con 0x560a2a0fab40 2024-09-19T08:23:41.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.079+0000 7f9b02ffd640 1 -- 172.21.15.22:0/191081946 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f9b1402e9a0 con 0x560a2a0fab40 2024-09-19T08:23:41.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.079+0000 7f9b2bf70740 10 client.16911 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:23:41.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.079+0000 7f9b2bf70740 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:23:41.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.079+0000 7f9b2bf70740 20 client.16911 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:41.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.079+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/191081946 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560a2a113ef0 0x560a2a181330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:41.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.079+0000 7f9b2bf70740 4 client.16911 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:23:41.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.079+0000 7f9b2bf70740 1 -- 172.21.15.22:0/191081946 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a2a066d70 con 0x560a2a113ef0 2024-09-19T08:23:41.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.079+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/191081946 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560a2a113ef0 0x560a2a181330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:41.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.079+0000 7f9b02ffd640 10 client.16911 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:41.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.079+0000 7f9b1b7fe640 1 --2- 172.21.15.22:0/191081946 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560a2a113ef0 0x560a2a181330 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c45bf891d2e81f7 server_cookie=dedade3e6b01aea6 in_seq=0 out_seq=0 2024-09-19T08:23:41.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b02ffd640 1 -- 172.21.15.22:0/191081946 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a2a066d70 con 0x560a2a113ef0 2024-09-19T08:23:41.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b02ffd640 10 client.16911 handle_command_reply: tid=0 2024-09-19T08:23:41.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b2bf70740 1 client.16911 shutdown 2024-09-19T08:23:41.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b2bf70740 1 -- 172.21.15.22:0/191081946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 msgr2=0x560a2a107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:41.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/191081946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a107f60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9b14011d10 tx=0x7f9b14009830 comp rx=0 tx=0).stop 2024-09-19T08:23:41.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b2bf70740 1 -- 172.21.15.22:0/191081946 shutdown_connections 2024-09-19T08:23:41.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/191081946 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x560a2a113ef0 0x560a2a181330 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/191081946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a2a104370 0x560a2a1084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/191081946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a2a0fab40 0x560a2a107f60 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b2bf70740 1 --2- 172.21.15.22:0/191081946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a2a0f95d0 0x560a2a10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b02ffd640 0 client.16911 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:41.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b2bf70740 1 -- 172.21.15.22:0/191081946 >> 172.21.15.22:0/191081946 conn(0x560a2a0c2310 msgr2=0x560a2a102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:41.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b2bf70740 1 -- 172.21.15.22:0/191081946 shutdown_connections 2024-09-19T08:23:41.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b2bf70740 1 -- 172.21.15.22:0/191081946 wait complete. 2024-09-19T08:23:41.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.080+0000 7f9b2bf70740 20 client.16911 trim_cache size 0 max 16384 2024-09-19T08:23:41.105 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:23:41.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.081+0000 7f9b29b97640 1 -- 172.21.15.22:0/4057799025 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9b100a3480 msgr2=0x7f9b100a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:41.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.081+0000 7f9b29b97640 1 --2- 172.21.15.22:0/4057799025 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9b100a3480 0x7f9b100a5940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0117f0 tx=0x7f9b1c009500 comp rx=0 tx=0).stop 2024-09-19T08:23:41.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.081+0000 7f9b29b97640 1 -- 172.21.15.22:0/4057799025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b24109450 msgr2=0x7f9b241a7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:41.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.081+0000 7f9b29b97640 1 --2- 172.21.15.22:0/4057799025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b24109450 0x7f9b241a7230 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c02a960 tx=0x7f9b0c003b90 comp rx=0 tx=0).stop 2024-09-19T08:23:41.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.081+0000 7f9b29b97640 1 -- 172.21.15.22:0/4057799025 shutdown_connections 2024-09-19T08:23:41.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.081+0000 7f9b29b97640 1 --2- 172.21.15.22:0/4057799025 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f9b100a3480 0x7f9b100a5940 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.081+0000 7f9b29b97640 1 --2- 172.21.15.22:0/4057799025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b2411b140 0x7f9b241abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.081+0000 7f9b29b97640 1 --2- 172.21.15.22:0/4057799025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b24109830 0x7f9b241a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.081+0000 7f9b29b97640 1 --2- 172.21.15.22:0/4057799025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b24109450 0x7f9b241a7230 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:41.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.081+0000 7f9b29b97640 1 -- 172.21.15.22:0/4057799025 >> 172.21.15.22:0/4057799025 conn(0x7f9b2406e140 msgr2=0x7f9b2411be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:41.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.081+0000 7f9b29b97640 1 -- 172.21.15.22:0/4057799025 shutdown_connections 2024-09-19T08:23:41.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:41.081+0000 7f9b29b97640 1 -- 172.21.15.22:0/4057799025 wait complete. 2024-09-19T08:23:42.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:42 smithi064 ceph-mon[34900]: pgmap v267: 97 pgs: 97 active+clean; 998 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 533 op/s 2024-09-19T08:23:42.864 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:23:42.864 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:23:42.897 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:42 smithi022 ceph-mon[30963]: pgmap v267: 97 pgs: 97 active+clean; 998 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 533 op/s 2024-09-19T08:23:43.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:42 smithi157 ceph-mon[35143]: pgmap v267: 97 pgs: 97 active+clean; 998 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 533 op/s 2024-09-19T08:23:43.143 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:23:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.662+0000 7f514a1a5640 1 -- 172.21.15.22:0/3009519860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5144102d30 msgr2=0x7f5144103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.662+0000 7f514a1a5640 1 --2- 172.21.15.22:0/3009519860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5144102d30 0x7f5144103110 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5134009930 tx=0x7f513402a420 comp rx=0 tx=0).stop 2024-09-19T08:23:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.662+0000 7f514a1a5640 1 -- 172.21.15.22:0/3009519860 shutdown_connections 2024-09-19T08:23:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.662+0000 7f514a1a5640 1 --2- 172.21.15.22:0/3009519860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51441082a0 0x7f514410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.662+0000 7f514a1a5640 1 --2- 172.21.15.22:0/3009519860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51441036e0 0x7f5144107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.662+0000 7f514a1a5640 1 --2- 172.21.15.22:0/3009519860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5144102d30 0x7f5144103110 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.662+0000 7f514a1a5640 1 -- 172.21.15.22:0/3009519860 >> 172.21.15.22:0/3009519860 conn(0x7f51440fc7d0 msgr2=0x7f51440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.662+0000 7f514a1a5640 1 -- 172.21.15.22:0/3009519860 shutdown_connections 2024-09-19T08:23:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.662+0000 7f514a1a5640 1 -- 172.21.15.22:0/3009519860 wait complete. 2024-09-19T08:23:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.662+0000 7f514a1a5640 1 Processor -- start 2024-09-19T08:23:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f514a1a5640 1 -- start start 2024-09-19T08:23:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f514a1a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5144102d30 0x7f5144100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f514a1a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51441036e0 0x7f5144100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f514a1a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51441082a0 0x7f5144101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f514a1a5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f51441120a0 con 0x7f51441082a0 2024-09-19T08:23:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f514a1a5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5144111f20 con 0x7f51441036e0 2024-09-19T08:23:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f514a1a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5144112220 con 0x7f5144102d30 2024-09-19T08:23:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f5143fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51441082a0 0x7f5144101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f51437fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5144102d30 0x7f5144100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f5143fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51441082a0 0x7f5144101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45494/0 (socket says 172.21.15.22:45494) 2024-09-19T08:23:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f51437fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5144102d30 0x7f5144100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:44120/0 (socket says 172.21.15.22:44120) 2024-09-19T08:23:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51441036e0 0x7f5144100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f5143fff640 1 -- 172.21.15.22:0/3103607798 learned_addr learned my addr 172.21.15.22:0/3103607798 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f5143fff640 1 -- 172.21.15.22:0/3103607798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5144102d30 msgr2=0x7f5144100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f5143fff640 1 --2- 172.21.15.22:0/3103607798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5144102d30 0x7f5144100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f5143fff640 1 -- 172.21.15.22:0/3103607798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51441036e0 msgr2=0x7f5144100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f5143fff640 1 --2- 172.21.15.22:0/3103607798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51441036e0 0x7f5144100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f5143fff640 1 -- 172.21.15.22:0/3103607798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51441a6a80 con 0x7f51441082a0 2024-09-19T08:23:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f5142ffd640 1 --2- 172.21.15.22:0/3103607798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51441036e0 0x7f5144100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.663+0000 7f51437fe640 1 --2- 172.21.15.22:0/3103607798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5144102d30 0x7f5144100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.664+0000 7f5143fff640 1 --2- 172.21.15.22:0/3103607798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51441082a0 0x7f5144101410 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f513c00eb10 tx=0x7f513c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.664+0000 7f5140ff9640 1 -- 172.21.15.22:0/3103607798 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c004b20 con 0x7f51441082a0 2024-09-19T08:23:43.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.664+0000 7f514a1a5640 1 -- 172.21.15.22:0/3103607798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51441a6d70 con 0x7f51441082a0 2024-09-19T08:23:43.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.664+0000 7f5140ff9640 1 -- 172.21.15.22:0/3103607798 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f513c004cc0 con 0x7f51441082a0 2024-09-19T08:23:43.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.664+0000 7f5140ff9640 1 -- 172.21.15.22:0/3103607798 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c010760 con 0x7f51441082a0 2024-09-19T08:23:43.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.664+0000 7f514a1a5640 1 -- 172.21.15.22:0/3103607798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51441a72b0 con 0x7f51441082a0 2024-09-19T08:23:43.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.667+0000 7f514c57e740 1 Processor -- start 2024-09-19T08:23:43.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.667+0000 7f5140ff9640 1 -- 172.21.15.22:0/3103607798 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f513c005020 con 0x7f51441082a0 2024-09-19T08:23:43.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.667+0000 7f514c57e740 1 -- start start 2024-09-19T08:23:43.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.667+0000 7f514c57e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.667+0000 7f514c57e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736e6480 0x55a0736f0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.667+0000 7f514c57e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736f0d40 0x55a0736f3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.667+0000 7f514c57e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a07364b280 con 0x55a0736e5b60 2024-09-19T08:23:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.667+0000 7f514c57e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a0735f2b10 con 0x55a0736e6480 2024-09-19T08:23:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.667+0000 7f514c57e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a073647240 con 0x55a0736f0d40 2024-09-19T08:23:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.667+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.667+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736e5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45506/0 (socket says 172.21.15.22:45506) 2024-09-19T08:23:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.667+0000 7f5142ffd640 1 -- 172.21.15.22:0/3639495062 learned_addr learned my addr 172.21.15.22:0/3639495062 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.667+0000 7f51437fe640 1 --2- 172.21.15.22:0/3639495062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736e6480 0x55a0736f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.667+0000 7f5142ffd640 1 --2- 172.21.15.22:0/3639495062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736f0d40 0x55a0736f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.668+0000 7f51437fe640 1 -- 172.21.15.22:0/3639495062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736f0d40 msgr2=0x55a0736f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.668+0000 7f51437fe640 1 --2- 172.21.15.22:0/3639495062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736f0d40 0x55a0736f3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.668+0000 7f51437fe640 1 -- 172.21.15.22:0/3639495062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 msgr2=0x55a0736e5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.668+0000 7f51437fe640 1 --2- 172.21.15.22:0/3639495062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736e5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.668+0000 7f51437fe640 1 -- 172.21.15.22:0/3639495062 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0735ff1f0 con 0x55a0736e6480 2024-09-19T08:23:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.668+0000 7f5142ffd640 1 --2- 172.21.15.22:0/3639495062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.668+0000 7f5140ff9640 1 --2- 172.21.15.22:0/3103607798 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f51300a33b0 0x7f51300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.668+0000 7f5142ffd640 1 --2- 172.21.15.22:0/3103607798 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f51300a33b0 0x7f51300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.668+0000 7f51437fe640 1 --2- 172.21.15.22:0/3639495062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736e6480 0x55a0736f0800 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5134005940 tx=0x7f5134005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=750828e38149fc8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.668+0000 7f5140ff9640 1 -- 172.21.15.22:0/3103607798 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f513c0d0220 con 0x7f51441082a0 2024-09-19T08:23:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.668+0000 7f5139ffb640 1 -- 172.21.15.22:0/3639495062 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134035040 con 0x55a0736e6480 2024-09-19T08:23:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.668+0000 7f5139ffb640 1 -- 172.21.15.22:0/3639495062 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51340317b0 con 0x55a0736e6480 2024-09-19T08:23:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.668+0000 7f5139ffb640 1 -- 172.21.15.22:0/3639495062 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51340047a0 con 0x55a0736e6480 2024-09-19T08:23:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.669+0000 7f5142ffd640 1 --2- 172.21.15.22:0/3103607798 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f51300a33b0 0x7f51300a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f512c0058c0 tx=0x7f512c005850 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.669+0000 7f514c57e740 1 -- 172.21.15.22:0/3639495062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736e6480 msgr2=0x55a0736f0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.669+0000 7f514c57e740 1 --2- 172.21.15.22:0/3639495062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736e6480 0x55a0736f0800 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5134005940 tx=0x7f5134005c00 comp rx=0 tx=0).stop 2024-09-19T08:23:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.669+0000 7f514c57e740 1 -- 172.21.15.22:0/3639495062 shutdown_connections 2024-09-19T08:23:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.669+0000 7f514c57e740 1 --2- 172.21.15.22:0/3639495062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736f0d40 0x55a0736f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.669+0000 7f514c57e740 1 --2- 172.21.15.22:0/3639495062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736e6480 0x55a0736f0800 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.669+0000 7f514c57e740 1 --2- 172.21.15.22:0/3639495062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.669+0000 7f514c57e740 1 -- 172.21.15.22:0/3639495062 >> 172.21.15.22:0/3639495062 conn(0x55a0736e20f0 msgr2=0x55a0736e24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:43.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.669+0000 7f514c57e740 1 -- 172.21.15.22:0/3639495062 shutdown_connections 2024-09-19T08:23:43.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.669+0000 7f514c57e740 1 -- 172.21.15.22:0/3639495062 wait complete. 2024-09-19T08:23:43.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.670+0000 7f514c57e740 1 -- start start 2024-09-19T08:23:43.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.670+0000 7f514c57e740 1 Processor -- start 2024-09-19T08:23:43.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.670+0000 7f514c57e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.670+0000 7f514c57e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e6480 0x55a0736ee110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.670+0000 7f514c57e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a0736ee650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.670+0000 7f514c57e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a0736f3910 con 0x55a0736e5b60 2024-09-19T08:23:43.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.670+0000 7f514c57e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a073605f10 con 0x55a0736f0d40 2024-09-19T08:23:43.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.670+0000 7f514c57e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a0736f3a90 con 0x55a0736e6480 2024-09-19T08:23:43.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.670+0000 7f514c57e740 10 client.0 _ll_register_callbacks cb 0x55a0735cea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:43.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.670+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e6480 0x55a0736ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.670+0000 7f51437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.670+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e6480 0x55a0736ee110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:44140/0 (socket says 172.21.15.22:44140) 2024-09-19T08:23:43.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.670+0000 7f5142ffd640 1 -- 172.21.15.22:0/2992924796 learned_addr learned my addr 172.21.15.22:0/2992924796 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:43.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.670+0000 7f5143fff640 1 --2- 172.21.15.22:0/2992924796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a0736ee650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f5142ffd640 1 -- 172.21.15.22:0/2992924796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 msgr2=0x55a0736ee650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:43.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f5142ffd640 1 --2- 172.21.15.22:0/2992924796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a0736ee650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f5142ffd640 1 -- 172.21.15.22:0/2992924796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 msgr2=0x55a0736edbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:43.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f5142ffd640 1 --2- 172.21.15.22:0/2992924796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736edbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f5142ffd640 1 -- 172.21.15.22:0/2992924796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0733bc6a0 con 0x55a0736e6480 2024-09-19T08:23:43.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f5143fff640 1 --2- 172.21.15.22:0/2992924796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a0736ee650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:43.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f51437fe640 1 --2- 172.21.15.22:0/2992924796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736edbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:43.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f5142ffd640 1 --2- 172.21.15.22:0/2992924796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e6480 0x55a0736ee110 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f512c013400 tx=0x7f512c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcf5e60ef3d0fc5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:43.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f5122ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:23:43.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f5122ffd640 1 -- 172.21.15.22:0/2992924796 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c0038c0 con 0x55a0736e6480 2024-09-19T08:23:43.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f5122ffd640 1 -- 172.21.15.22:0/2992924796 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f512c009040 con 0x55a0736e6480 2024-09-19T08:23:43.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f514c57e740 1 -- 172.21.15.22:0/2992924796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51340317b0 con 0x55a0736e6480 2024-09-19T08:23:43.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f5122ffd640 1 -- 172.21.15.22:0/2992924796 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f512c003a60 con 0x55a0736e6480 2024-09-19T08:23:43.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f514c57e740 10 client.26401 fetch_fsmap learned FSMap version 35 2024-09-19T08:23:43.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.671+0000 7f514c57e740 1 -- 172.21.15.22:0/2992924796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a0736ff4c0 con 0x55a0736e6480 2024-09-19T08:23:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.672+0000 7f5122ffd640 1 -- 172.21.15.22:0/2992924796 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51340317b0 con 0x55a0736e6480 2024-09-19T08:23:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.672+0000 7f5122ffd640 1 -- 172.21.15.22:0/2992924796 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f512c029020 con 0x55a0736e6480 2024-09-19T08:23:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.672+0000 7f514c57e740 10 client.26401 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:23:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.672+0000 7f514c57e740 10 client.26401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:23:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.672+0000 7f514c57e740 20 client.26401 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.672+0000 7f514c57e740 1 --2- 172.21.15.22:0/2992924796 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a073700980 0x55a073702e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.672+0000 7f514c57e740 4 client.26401 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:23:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.672+0000 7f514c57e740 1 -- 172.21.15.22:0/2992924796 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0736587f0 con 0x55a073700980 2024-09-19T08:23:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.672+0000 7f51437fe640 1 --2- 172.21.15.22:0/2992924796 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a073700980 0x55a073702e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.672+0000 7f51437fe640 1 --2- 172.21.15.22:0/2992924796 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a073700980 0x55a073702e20 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d837dd069643a5a7 server_cookie=f8031c4948641da in_seq=0 out_seq=0 2024-09-19T08:23:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.672+0000 7f5122ffd640 10 client.26401 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.673+0000 7f5122ffd640 1 -- 172.21.15.22:0/2992924796 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0736587f0 con 0x55a073700980 2024-09-19T08:23:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.673+0000 7f5122ffd640 10 client.26401 handle_command_reply: tid=0 2024-09-19T08:23:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.673+0000 7f514c57e740 1 client.26401 shutdown 2024-09-19T08:23:43.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.673+0000 7f514c57e740 1 -- 172.21.15.22:0/2992924796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e6480 msgr2=0x55a0736ee110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:43.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.673+0000 7f514c57e740 1 --2- 172.21.15.22:0/2992924796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e6480 0x55a0736ee110 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f512c013400 tx=0x7f512c005f90 comp rx=0 tx=0).stop 2024-09-19T08:23:43.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.673+0000 7f514c57e740 1 -- 172.21.15.22:0/2992924796 shutdown_connections 2024-09-19T08:23:43.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.673+0000 7f514c57e740 1 --2- 172.21.15.22:0/2992924796 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a073700980 0x55a073702e20 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.673+0000 7f514c57e740 1 --2- 172.21.15.22:0/2992924796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a0736ee650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.673+0000 7f514c57e740 1 --2- 172.21.15.22:0/2992924796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e6480 0x55a0736ee110 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.673+0000 7f514c57e740 1 --2- 172.21.15.22:0/2992924796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736edbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.673+0000 7f514c57e740 1 -- 172.21.15.22:0/2992924796 >> 172.21.15.22:0/2992924796 conn(0x55a0736e20f0 msgr2=0x55a0736e1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:43.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.673+0000 7f5122ffd640 0 client.26401 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:43.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.673+0000 7f514c57e740 1 -- 172.21.15.22:0/2992924796 shutdown_connections 2024-09-19T08:23:43.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.673+0000 7f514c57e740 1 -- 172.21.15.22:0/2992924796 wait complete. 2024-09-19T08:23:43.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.674+0000 7f514c57e740 20 client.26401 trim_cache size 0 max 16384 2024-09-19T08:23:43.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f514c57e740 1 Processor -- start 2024-09-19T08:23:43.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f514c57e740 1 -- start start 2024-09-19T08:23:43.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f514c57e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736efa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f514c57e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e6480 0x55a073708730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f514c57e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a073708c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f514c57e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f51441b2440 con 0x55a0736e5b60 2024-09-19T08:23:43.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f514c57e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f51441afce0 con 0x55a0736f0d40 2024-09-19T08:23:43.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f514c57e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51441adb50 con 0x55a0736e6480 2024-09-19T08:23:43.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f51437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736efa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f51437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736efa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45534/0 (socket says 172.21.15.22:45534) 2024-09-19T08:23:43.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f51437fe640 1 -- 172.21.15.22:0/42150303 learned_addr learned my addr 172.21.15.22:0/42150303 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:43.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f5143fff640 1 --2- 172.21.15.22:0/42150303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a073708c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f5142ffd640 1 --2- 172.21.15.22:0/42150303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e6480 0x55a073708730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f51437fe640 1 -- 172.21.15.22:0/42150303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e6480 msgr2=0x55a073708730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:43.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f51437fe640 1 --2- 172.21.15.22:0/42150303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e6480 0x55a073708730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f51437fe640 1 -- 172.21.15.22:0/42150303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 msgr2=0x55a073708c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:43.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f51437fe640 1 --2- 172.21.15.22:0/42150303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a073708c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.692+0000 7f51437fe640 1 -- 172.21.15.22:0/42150303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0736e62b0 con 0x55a0736e5b60 2024-09-19T08:23:43.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.693+0000 7f5143fff640 1 --2- 172.21.15.22:0/42150303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a073708c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:43.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.693+0000 7f51437fe640 1 --2- 172.21.15.22:0/42150303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736efa30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5134009930 tx=0x7f5134005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7d48852253a6fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:43.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.693+0000 7f51397fa640 1 -- 172.21.15.22:0/42150303 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134035040 con 0x55a0736e5b60 2024-09-19T08:23:43.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.693+0000 7f51397fa640 1 -- 172.21.15.22:0/42150303 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51340319e0 con 0x55a0736e5b60 2024-09-19T08:23:43.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.693+0000 7f51397fa640 1 -- 172.21.15.22:0/42150303 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134003b50 con 0x55a0736e5b60 2024-09-19T08:23:43.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.694+0000 7f514c57e740 1 -- 172.21.15.22:0/42150303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 msgr2=0x55a0736efa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:43.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.694+0000 7f514c57e740 1 --2- 172.21.15.22:0/42150303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736efa30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5134009930 tx=0x7f5134005fd0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.694+0000 7f514c57e740 1 -- 172.21.15.22:0/42150303 shutdown_connections 2024-09-19T08:23:43.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.694+0000 7f514c57e740 1 --2- 172.21.15.22:0/42150303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a073708c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.694+0000 7f514c57e740 1 --2- 172.21.15.22:0/42150303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e6480 0x55a073708730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.694+0000 7f514c57e740 1 --2- 172.21.15.22:0/42150303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e5b60 0x55a0736efa30 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.694+0000 7f514c57e740 1 -- 172.21.15.22:0/42150303 >> 172.21.15.22:0/42150303 conn(0x55a0736e20f0 msgr2=0x55a0736ef320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:43.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.694+0000 7f514c57e740 1 -- 172.21.15.22:0/42150303 shutdown_connections 2024-09-19T08:23:43.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.694+0000 7f514c57e740 1 -- 172.21.15.22:0/42150303 wait complete. 2024-09-19T08:23:43.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.694+0000 7f514c57e740 1 -- start start 2024-09-19T08:23:43.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.694+0000 7f514c57e740 1 Processor -- start 2024-09-19T08:23:43.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f514c57e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e5b60 0x55a0736f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f514c57e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e6480 0x55a0736f5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f514c57e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a0736f5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f514c57e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ae1a0 con 0x55a0736e6480 2024-09-19T08:23:43.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f514c57e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f51441af610 con 0x55a0736f0d40 2024-09-19T08:23:43.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f514c57e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ae870 con 0x55a0736e5b60 2024-09-19T08:23:43.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f514c57e740 10 client.0 _ll_register_callbacks cb 0x7f512c009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:43.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f51437fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e5b60 0x55a0736f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f51437fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e5b60 0x55a0736f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:44164/0 (socket says 172.21.15.22:44164) 2024-09-19T08:23:43.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e6480 0x55a0736f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f51437fe640 1 -- 172.21.15.22:0/3348832791 learned_addr learned my addr 172.21.15.22:0/3348832791 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:43.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f5143fff640 1 --2- 172.21.15.22:0/3348832791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a0736f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f51437fe640 1 -- 172.21.15.22:0/3348832791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 msgr2=0x55a0736f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:43.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f51437fe640 1 --2- 172.21.15.22:0/3348832791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a0736f5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f51437fe640 1 -- 172.21.15.22:0/3348832791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e6480 msgr2=0x55a0736f5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:43.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f51437fe640 1 --2- 172.21.15.22:0/3348832791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e6480 0x55a0736f5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f51437fe640 1 -- 172.21.15.22:0/3348832791 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a07376a9f0 con 0x55a0736e5b60 2024-09-19T08:23:43.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f5143fff640 1 --2- 172.21.15.22:0/3348832791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a0736f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:23:43.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f5142ffd640 1 --2- 172.21.15.22:0/3348832791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e6480 0x55a0736f5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:43.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f51437fe640 1 --2- 172.21.15.22:0/3348832791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e5b60 0x55a0736f4d50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5134009900 tx=0x7f51340035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b3bc0d172f2553d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:43.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.695+0000 7f5122ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:23:43.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f5122ffd640 1 -- 172.21.15.22:0/3348832791 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513405f070 con 0x55a0736e5b60 2024-09-19T08:23:43.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f5122ffd640 1 -- 172.21.15.22:0/3348832791 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f5134004240 con 0x55a0736e5b60 2024-09-19T08:23:43.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f514c57e740 1 -- 172.21.15.22:0/3348832791 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51340319e0 con 0x55a0736e5b60 2024-09-19T08:23:43.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f5122ffd640 1 -- 172.21.15.22:0/3348832791 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f513406b050 con 0x55a0736e5b60 2024-09-19T08:23:43.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f514c57e740 10 client.26407 fetch_fsmap learned FSMap version 35 2024-09-19T08:23:43.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f514c57e740 1 -- 172.21.15.22:0/3348832791 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a07376bd30 con 0x55a0736e5b60 2024-09-19T08:23:43.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f5122ffd640 1 -- 172.21.15.22:0/3348832791 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51340319e0 con 0x55a0736e5b60 2024-09-19T08:23:43.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f5122ffd640 1 -- 172.21.15.22:0/3348832791 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f51340053d0 con 0x55a0736e5b60 2024-09-19T08:23:43.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f514c57e740 10 client.26407 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:23:43.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f514c57e740 10 client.26407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:23:43.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f514c57e740 20 client.26407 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:43.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f514c57e740 1 --2- 172.21.15.22:0/3348832791 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a07376bf60 0x55a07376e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:43.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f514c57e740 4 client.26407 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:23:43.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f514c57e740 1 -- 172.21.15.22:0/3348832791 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a073652d00 con 0x55a07376bf60 2024-09-19T08:23:43.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.696+0000 7f5142ffd640 1 --2- 172.21.15.22:0/3348832791 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a07376bf60 0x55a07376e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:43.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.697+0000 7f5142ffd640 1 --2- 172.21.15.22:0/3348832791 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a07376bf60 0x55a07376e350 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63b560e0a30290ea server_cookie=44a3ef843af51529 in_seq=0 out_seq=0 2024-09-19T08:23:43.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.697+0000 7f5122ffd640 10 client.26407 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:43.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.698+0000 7f5122ffd640 1 -- 172.21.15.22:0/3348832791 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a073652d00 con 0x55a07376bf60 2024-09-19T08:23:43.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.698+0000 7f5122ffd640 10 client.26407 handle_command_reply: tid=0 2024-09-19T08:23:43.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.698+0000 7f514c57e740 1 client.26407 shutdown 2024-09-19T08:23:43.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.698+0000 7f514c57e740 1 -- 172.21.15.22:0/3348832791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e5b60 msgr2=0x55a0736f4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:43.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.698+0000 7f514c57e740 1 --2- 172.21.15.22:0/3348832791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e5b60 0x55a0736f4d50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5134009900 tx=0x7f51340035f0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.698+0000 7f514c57e740 1 -- 172.21.15.22:0/3348832791 shutdown_connections 2024-09-19T08:23:43.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.698+0000 7f514c57e740 1 --2- 172.21.15.22:0/3348832791 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55a07376bf60 0x55a07376e350 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.698+0000 7f514c57e740 1 --2- 172.21.15.22:0/3348832791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0736f0d40 0x55a0736f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.698+0000 7f514c57e740 1 --2- 172.21.15.22:0/3348832791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0736e6480 0x55a0736f5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.698+0000 7f514c57e740 1 --2- 172.21.15.22:0/3348832791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a0736e5b60 0x55a0736f4d50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.698+0000 7f514c57e740 1 -- 172.21.15.22:0/3348832791 >> 172.21.15.22:0/3348832791 conn(0x55a0736e20f0 msgr2=0x55a0736e7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:43.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.698+0000 7f5122ffd640 0 client.26407 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:43.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.699+0000 7f514c57e740 1 -- 172.21.15.22:0/3348832791 shutdown_connections 2024-09-19T08:23:43.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.699+0000 7f514c57e740 1 -- 172.21.15.22:0/3348832791 wait complete. 2024-09-19T08:23:43.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.699+0000 7f514c57e740 20 client.26407 trim_cache size 0 max 16384 2024-09-19T08:23:43.721 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:23:43.721 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-19T08:23:43.721 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "2d65834f-9d9a-431b-b24b-219b1dbe7fa4", 2024-09-19T08:23:43.721 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-19T08:23:43.722 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:23:43.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.700+0000 7f514a1a5640 1 -- 172.21.15.22:0/3103607798 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f51300a33b0 msgr2=0x7f51300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:43.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.700+0000 7f514a1a5640 1 --2- 172.21.15.22:0/3103607798 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f51300a33b0 0x7f51300a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f512c0058c0 tx=0x7f512c005850 comp rx=0 tx=0).stop 2024-09-19T08:23:43.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.700+0000 7f514a1a5640 1 -- 172.21.15.22:0/3103607798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51441082a0 msgr2=0x7f5144101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:43.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.700+0000 7f514a1a5640 1 --2- 172.21.15.22:0/3103607798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51441082a0 0x7f5144101410 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f513c00eb10 tx=0x7f513c00c2d0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.700+0000 7f514a1a5640 1 -- 172.21.15.22:0/3103607798 shutdown_connections 2024-09-19T08:23:43.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.700+0000 7f514a1a5640 1 --2- 172.21.15.22:0/3103607798 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f51300a33b0 0x7f51300a5870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.700+0000 7f514a1a5640 1 --2- 172.21.15.22:0/3103607798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51441082a0 0x7f5144101410 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.700+0000 7f514a1a5640 1 --2- 172.21.15.22:0/3103607798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51441036e0 0x7f5144100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.700+0000 7f514a1a5640 1 --2- 172.21.15.22:0/3103607798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5144102d30 0x7f5144100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:43.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.700+0000 7f514a1a5640 1 -- 172.21.15.22:0/3103607798 >> 172.21.15.22:0/3103607798 conn(0x7f51440fc7d0 msgr2=0x7f5144105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:43.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.700+0000 7f514a1a5640 1 -- 172.21.15.22:0/3103607798 shutdown_connections 2024-09-19T08:23:43.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:43.700+0000 7f514a1a5640 1 -- 172.21.15.22:0/3103607798 wait complete. 2024-09-19T08:23:44.600 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:44 smithi022 ceph-mon[30963]: pgmap v268: 97 pgs: 97 active+clean; 1018 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 733 op/s 2024-09-19T08:23:44.601 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:44 smithi022 ceph-mon[30963]: scrub queued for path: / 2024-09-19T08:23:44.601 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:44 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:44.601 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:44 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:23:44.601 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:44 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:23:44.601 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:44 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:44.628 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:23:44.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:44 smithi064 ceph-mon[34900]: pgmap v268: 97 pgs: 97 active+clean; 1018 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 733 op/s 2024-09-19T08:23:44.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:44 smithi064 ceph-mon[34900]: scrub queued for path: / 2024-09-19T08:23:44.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:44 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:44.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:44 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:23:44.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:44 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:23:44.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:44 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:23:45.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:44 smithi157 ceph-mon[35143]: pgmap v268: 97 pgs: 97 active+clean; 1018 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 733 op/s 2024-09-19T08:23:45.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:44 smithi157 ceph-mon[35143]: scrub queued for path: / 2024-09-19T08:23:45.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:44 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:45.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:44 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:23:45.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:44 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:23:45.005 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:44 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:45.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.453+0000 7fdfc1572640 1 -- 172.21.15.22:0/728862006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc101df0 msgr2=0x7fdfbc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:45.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.453+0000 7fdfc1572640 1 --2- 172.21.15.22:0/728862006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc101df0 0x7fdfbc1021d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009900 tx=0x7fdfb402a410 comp rx=0 tx=0).stop 2024-09-19T08:23:45.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.453+0000 7fdfc1572640 1 -- 172.21.15.22:0/728862006 shutdown_connections 2024-09-19T08:23:45.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.453+0000 7fdfc1572640 1 --2- 172.21.15.22:0/728862006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfbc106f20 0x7fdfbc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.453+0000 7fdfc1572640 1 --2- 172.21.15.22:0/728862006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfbc1027a0 0x7fdfbc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.453+0000 7fdfc1572640 1 --2- 172.21.15.22:0/728862006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc101df0 0x7fdfbc1021d0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.453+0000 7fdfc1572640 1 -- 172.21.15.22:0/728862006 >> 172.21.15.22:0/728862006 conn(0x7fdfbc0fd8b0 msgr2=0x7fdfbc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:45.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.453+0000 7fdfc1572640 1 -- 172.21.15.22:0/728862006 shutdown_connections 2024-09-19T08:23:45.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.453+0000 7fdfc1572640 1 -- 172.21.15.22:0/728862006 wait complete. 2024-09-19T08:23:45.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.454+0000 7fdfc1572640 1 Processor -- start 2024-09-19T08:23:45.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.454+0000 7fdfc1572640 1 -- start start 2024-09-19T08:23:45.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.454+0000 7fdfc1572640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfbc101df0 0x7fdfbc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.454+0000 7fdfc1572640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfbc1027a0 0x7fdfbc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.454+0000 7fdfc1572640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc106f20 0x7fdfbc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.454+0000 7fdfc1572640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc111f80 con 0x7fdfbc106f20 2024-09-19T08:23:45.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.454+0000 7fdfc1572640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc111e00 con 0x7fdfbc101df0 2024-09-19T08:23:45.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.454+0000 7fdfc1572640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc112100 con 0x7fdfbc1027a0 2024-09-19T08:23:45.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.454+0000 7fdfbb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc106f20 0x7fdfbc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.454+0000 7fdfbb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc106f20 0x7fdfbc19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45562/0 (socket says 172.21.15.22:45562) 2024-09-19T08:23:45.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.454+0000 7fdfbb7fe640 1 -- 172.21.15.22:0/790569475 learned_addr learned my addr 172.21.15.22:0/790569475 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:45.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.454+0000 7fdfba7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfbc1027a0 0x7fdfbc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.454+0000 7fdfbaffd640 1 --2- 172.21.15.22:0/790569475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfbc101df0 0x7fdfbc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.455+0000 7fdfbb7fe640 1 -- 172.21.15.22:0/790569475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfbc1027a0 msgr2=0x7fdfbc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:45.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.455+0000 7fdfbb7fe640 1 --2- 172.21.15.22:0/790569475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfbc1027a0 0x7fdfbc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.455+0000 7fdfbb7fe640 1 -- 172.21.15.22:0/790569475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfbc101df0 msgr2=0x7fdfbc198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:45.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.455+0000 7fdfbb7fe640 1 --2- 172.21.15.22:0/790569475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfbc101df0 0x7fdfbc198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.455+0000 7fdfbb7fe640 1 -- 172.21.15.22:0/790569475 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc19d3b0 con 0x7fdfbc106f20 2024-09-19T08:23:45.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.455+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/790569475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfbc1027a0 0x7fdfbc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:45.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.455+0000 7fdfbb7fe640 1 --2- 172.21.15.22:0/790569475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc106f20 0x7fdfbc19ccd0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00eaa0 tx=0x7fdfac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:45.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.455+0000 7fdfb3fff640 1 -- 172.21.15.22:0/790569475 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00cdb0 con 0x7fdfbc106f20 2024-09-19T08:23:45.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.455+0000 7fdfb3fff640 1 -- 172.21.15.22:0/790569475 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfac005020 con 0x7fdfbc106f20 2024-09-19T08:23:45.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.455+0000 7fdfc1572640 1 -- 172.21.15.22:0/790569475 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfbc19d6a0 con 0x7fdfbc106f20 2024-09-19T08:23:45.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.455+0000 7fdfb3fff640 1 -- 172.21.15.22:0/790569475 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0116b0 con 0x7fdfbc106f20 2024-09-19T08:23:45.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.455+0000 7fdfc1572640 1 -- 172.21.15.22:0/790569475 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfbc1a4f80 con 0x7fdfbc106f20 2024-09-19T08:23:45.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.458+0000 7fdfb3fff640 1 -- 172.21.15.22:0/790569475 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdfac005560 con 0x7fdfbc106f20 2024-09-19T08:23:45.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.458+0000 7fdfc394b740 1 Processor -- start 2024-09-19T08:23:45.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.458+0000 7fdfc394b740 1 -- start start 2024-09-19T08:23:45.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.458+0000 7fdfc394b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf163a0 0x55fbddf16780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.458+0000 7fdfc394b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf16cc0 0x55fbddf25550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.458+0000 7fdfc394b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf25a90 0x55fbddf27e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.458+0000 7fdfc394b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fbdde7c2e0 con 0x55fbddf16cc0 2024-09-19T08:23:45.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.458+0000 7fdfc394b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fbdde23ce0 con 0x55fbddf163a0 2024-09-19T08:23:45.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.458+0000 7fdfc394b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fbdde782a0 con 0x55fbddf25a90 2024-09-19T08:23:45.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfbaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf16cc0 0x55fbddf25550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfba7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf163a0 0x55fbddf16780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfba7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf163a0 0x55fbddf16780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:41368/0 (socket says 172.21.15.22:41368) 2024-09-19T08:23:45.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfbaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf16cc0 0x55fbddf25550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45574/0 (socket says 172.21.15.22:45574) 2024-09-19T08:23:45.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfba7fc640 1 -- 172.21.15.22:0/3880977365 learned_addr learned my addr 172.21.15.22:0/3880977365 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:45.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/3880977365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf25a90 0x55fbddf27e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfb3fff640 1 --2- 172.21.15.22:0/790569475 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdfa80a33b0 0x7fdfa80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfba7fc640 1 -- 172.21.15.22:0/3880977365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf25a90 msgr2=0x55fbddf27e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:45.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/3880977365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf25a90 0x55fbddf27e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfba7fc640 1 -- 172.21.15.22:0/3880977365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf16cc0 msgr2=0x55fbddf25550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:45.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/3880977365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf16cc0 0x55fbddf25550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfba7fc640 1 -- 172.21.15.22:0/3880977365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbdde08440 con 0x55fbddf163a0 2024-09-19T08:23:45.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfbaffd640 1 --2- 172.21.15.22:0/3880977365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf16cc0 0x55fbddf25550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:23:45.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfbb7fe640 1 --2- 172.21.15.22:0/790569475 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdfa80a33b0 0x7fdfa80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/3880977365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf163a0 0x55fbddf16780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4004650 tx=0x7fdfa4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77e95abd92e4f685 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:45.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfb3fff640 1 -- 172.21.15.22:0/790569475 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fdfac0cfe70 con 0x7fdfbc106f20 2024-09-19T08:23:45.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfb17fa640 1 -- 172.21.15.22:0/3880977365 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400fc50 con 0x55fbddf163a0 2024-09-19T08:23:45.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfb17fa640 1 -- 172.21.15.22:0/3880977365 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfa400fdf0 con 0x55fbddf163a0 2024-09-19T08:23:45.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.459+0000 7fdfb17fa640 1 -- 172.21.15.22:0/3880977365 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400e5b0 con 0x55fbddf163a0 2024-09-19T08:23:45.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.460+0000 7fdfc394b740 1 -- 172.21.15.22:0/3880977365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf163a0 msgr2=0x55fbddf16780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:45.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.460+0000 7fdfc394b740 1 --2- 172.21.15.22:0/3880977365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf163a0 0x55fbddf16780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4004650 tx=0x7fdfa4009b80 comp rx=0 tx=0).stop 2024-09-19T08:23:45.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.460+0000 7fdfbb7fe640 1 --2- 172.21.15.22:0/790569475 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdfa80a33b0 0x7fdfa80a5870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdfac098bc0 tx=0x7fdfac0054d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:45.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.460+0000 7fdfc394b740 1 -- 172.21.15.22:0/3880977365 shutdown_connections 2024-09-19T08:23:45.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.460+0000 7fdfc394b740 1 --2- 172.21.15.22:0/3880977365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf25a90 0x55fbddf27e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.460+0000 7fdfc394b740 1 --2- 172.21.15.22:0/3880977365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf16cc0 0x55fbddf25550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.460+0000 7fdfc394b740 1 --2- 172.21.15.22:0/3880977365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf163a0 0x55fbddf16780 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.460+0000 7fdfc394b740 1 -- 172.21.15.22:0/3880977365 >> 172.21.15.22:0/3880977365 conn(0x55fbddf12a30 msgr2=0x55fbddf12e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:45.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.460+0000 7fdfc394b740 1 -- 172.21.15.22:0/3880977365 shutdown_connections 2024-09-19T08:23:45.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.460+0000 7fdfc394b740 1 -- 172.21.15.22:0/3880977365 wait complete. 2024-09-19T08:23:45.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfc394b740 1 -- start start 2024-09-19T08:23:45.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfc394b740 1 Processor -- start 2024-09-19T08:23:45.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfc394b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf163a0 0x55fbddf25400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfc394b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf16cc0 0x55fbddf226b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfc394b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 0x55fbddf22c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfc394b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fbddf286c0 con 0x55fbddf163a0 2024-09-19T08:23:45.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfc394b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fbdde37050 con 0x55fbddf25a90 2024-09-19T08:23:45.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfc394b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fbddf28840 con 0x55fbddf16cc0 2024-09-19T08:23:45.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfc394b740 10 client.0 _ll_register_callbacks cb 0x55fbdde77160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:45.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfba7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf163a0 0x55fbddf25400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfba7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf163a0 0x55fbddf25400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45584/0 (socket says 172.21.15.22:45584) 2024-09-19T08:23:45.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfba7fc640 1 -- 172.21.15.22:0/2625880554 learned_addr learned my addr 172.21.15.22:0/2625880554 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:45.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfbaffd640 1 --2- 172.21.15.22:0/2625880554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf16cc0 0x55fbddf226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/2625880554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 0x55fbddf22c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.461+0000 7fdfba7fc640 1 -- 172.21.15.22:0/2625880554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf16cc0 msgr2=0x55fbddf226b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:45.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.462+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/2625880554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf16cc0 0x55fbddf226b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.462+0000 7fdfba7fc640 1 -- 172.21.15.22:0/2625880554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 msgr2=0x55fbddf22c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:45.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.462+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/2625880554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 0x55fbddf22c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.462+0000 7fdfba7fc640 1 -- 172.21.15.22:0/2625880554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbddbed3c0 con 0x55fbddf163a0 2024-09-19T08:23:45.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.462+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/2625880554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf163a0 0x55fbddf25400 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4011570 tx=0x7fdfa40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6057ddbc4fdef769 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:45.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.462+0000 7fdfa27fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:45.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.462+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2625880554 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4005630 con 0x55fbddf163a0 2024-09-19T08:23:45.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.462+0000 7fdfc394b740 1 -- 172.21.15.22:0/2625880554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfa400fdf0 con 0x55fbddf163a0 2024-09-19T08:23:45.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.462+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2625880554 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fdfa40048d0 con 0x55fbddf163a0 2024-09-19T08:23:45.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.462+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2625880554 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdfa4004ad0 con 0x55fbddf163a0 2024-09-19T08:23:45.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.462+0000 7fdfc394b740 10 client.16941 fetch_fsmap learned FSMap version 35 2024-09-19T08:23:45.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.462+0000 7fdfc394b740 1 -- 172.21.15.22:0/2625880554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fbddf340d0 con 0x55fbddf163a0 2024-09-19T08:23:45.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.462+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2625880554 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfa400fdf0 con 0x55fbddf163a0 2024-09-19T08:23:45.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.462+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2625880554 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fdfa402c030 con 0x55fbddf163a0 2024-09-19T08:23:45.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.463+0000 7fdfc394b740 10 client.16941 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:23:45.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.463+0000 7fdfc394b740 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:23:45.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.463+0000 7fdfc394b740 20 client.16941 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:45.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.463+0000 7fdfc394b740 1 --2- 172.21.15.22:0/2625880554 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fbddf35420 0x55fbddf37810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.463+0000 7fdfc394b740 4 client.16941 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:23:45.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.463+0000 7fdfc394b740 1 -- 172.21.15.22:0/2625880554 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbdde89850 con 0x55fbddf35420 2024-09-19T08:23:45.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.463+0000 7fdfbaffd640 1 --2- 172.21.15.22:0/2625880554 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fbddf35420 0x55fbddf37810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.463+0000 7fdfbaffd640 1 --2- 172.21.15.22:0/2625880554 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fbddf35420 0x55fbddf37810 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0a99af33b54dbd3 server_cookie=808b83478c83d891 in_seq=0 out_seq=0 2024-09-19T08:23:45.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.463+0000 7fdfa27fc640 10 client.16941 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:45.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2625880554 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fbdde89850 con 0x55fbddf35420 2024-09-19T08:23:45.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfa27fc640 10 client.16941 handle_command_reply: tid=0 2024-09-19T08:23:45.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfc394b740 1 client.16941 shutdown 2024-09-19T08:23:45.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfc394b740 1 -- 172.21.15.22:0/2625880554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf163a0 msgr2=0x55fbddf25400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:45.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfc394b740 1 --2- 172.21.15.22:0/2625880554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf163a0 0x55fbddf25400 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4011570 tx=0x7fdfa40096a0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfc394b740 1 -- 172.21.15.22:0/2625880554 shutdown_connections 2024-09-19T08:23:45.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfc394b740 1 --2- 172.21.15.22:0/2625880554 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fbddf35420 0x55fbddf37810 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfc394b740 1 --2- 172.21.15.22:0/2625880554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 0x55fbddf22c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfc394b740 1 --2- 172.21.15.22:0/2625880554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf16cc0 0x55fbddf226b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfc394b740 1 --2- 172.21.15.22:0/2625880554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf163a0 0x55fbddf25400 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfc394b740 1 -- 172.21.15.22:0/2625880554 >> 172.21.15.22:0/2625880554 conn(0x55fbddf12a30 msgr2=0x55fbddf18760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:45.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfa27fc640 0 client.16941 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:45.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfc394b740 1 -- 172.21.15.22:0/2625880554 shutdown_connections 2024-09-19T08:23:45.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfc394b740 1 -- 172.21.15.22:0/2625880554 wait complete. 2024-09-19T08:23:45.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.464+0000 7fdfc394b740 20 client.16941 trim_cache size 0 max 16384 2024-09-19T08:23:45.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.483+0000 7fdfc394b740 1 Processor -- start 2024-09-19T08:23:45.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.483+0000 7fdfc394b740 1 -- start start 2024-09-19T08:23:45.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.483+0000 7fdfc394b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf163a0 0x55fbddf31960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.483+0000 7fdfc394b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf16cc0 0x55fbddf3cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.483+0000 7fdfc394b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 0x55fbddf3d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.483+0000 7fdfc394b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1adfd0 con 0x55fbddf16cc0 2024-09-19T08:23:45.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.483+0000 7fdfc394b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1ab870 con 0x55fbddf25a90 2024-09-19T08:23:45.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.483+0000 7fdfc394b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1a96e0 con 0x55fbddf163a0 2024-09-19T08:23:45.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.483+0000 7fdfbaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf16cc0 0x55fbddf3cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.484+0000 7fdfbaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf16cc0 0x55fbddf3cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45590/0 (socket says 172.21.15.22:45590) 2024-09-19T08:23:45.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.484+0000 7fdfbaffd640 1 -- 172.21.15.22:0/3044199919 learned_addr learned my addr 172.21.15.22:0/3044199919 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:45.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.484+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/3044199919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf163a0 0x55fbddf31960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.484+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/3044199919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 0x55fbddf3d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.484+0000 7fdfbaffd640 1 -- 172.21.15.22:0/3044199919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf163a0 msgr2=0x55fbddf31960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:45.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.484+0000 7fdfbaffd640 1 --2- 172.21.15.22:0/3044199919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf163a0 0x55fbddf31960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.484+0000 7fdfbaffd640 1 -- 172.21.15.22:0/3044199919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 msgr2=0x55fbddf3d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:45.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.484+0000 7fdfbaffd640 1 --2- 172.21.15.22:0/3044199919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 0x55fbddf3d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.484+0000 7fdfbaffd640 1 -- 172.21.15.22:0/3044199919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbddf20c00 con 0x55fbddf16cc0 2024-09-19T08:23:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.484+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/3044199919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 0x55fbddf3d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:23:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.484+0000 7fdfbaffd640 1 --2- 172.21.15.22:0/3044199919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf16cc0 0x55fbddf3cde0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fdfb402a930 tx=0x7fdfb40319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1288f42770871e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.484+0000 7fdfb0ff9640 1 -- 172.21.15.22:0/3044199919 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4035040 con 0x55fbddf16cc0 2024-09-19T08:23:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.484+0000 7fdfb0ff9640 1 -- 172.21.15.22:0/3044199919 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb4005500 con 0x55fbddf16cc0 2024-09-19T08:23:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.484+0000 7fdfb0ff9640 1 -- 172.21.15.22:0/3044199919 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4004b70 con 0x55fbddf16cc0 2024-09-19T08:23:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.485+0000 7fdfc394b740 1 -- 172.21.15.22:0/3044199919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf16cc0 msgr2=0x55fbddf3cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.485+0000 7fdfc394b740 1 --2- 172.21.15.22:0/3044199919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf16cc0 0x55fbddf3cde0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fdfb402a930 tx=0x7fdfb40319f0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.485+0000 7fdfc394b740 1 -- 172.21.15.22:0/3044199919 shutdown_connections 2024-09-19T08:23:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.485+0000 7fdfc394b740 1 --2- 172.21.15.22:0/3044199919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 0x55fbddf3d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.485+0000 7fdfc394b740 1 --2- 172.21.15.22:0/3044199919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf16cc0 0x55fbddf3cde0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.485+0000 7fdfc394b740 1 --2- 172.21.15.22:0/3044199919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf163a0 0x55fbddf31960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.485+0000 7fdfc394b740 1 -- 172.21.15.22:0/3044199919 >> 172.21.15.22:0/3044199919 conn(0x55fbddf12a30 msgr2=0x55fbddf1dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.485+0000 7fdfc394b740 1 -- 172.21.15.22:0/3044199919 shutdown_connections 2024-09-19T08:23:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.485+0000 7fdfc394b740 1 -- 172.21.15.22:0/3044199919 wait complete. 2024-09-19T08:23:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.485+0000 7fdfc394b740 1 -- start start 2024-09-19T08:23:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfc394b740 1 Processor -- start 2024-09-19T08:23:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfc394b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf163a0 0x55fbddf2df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfc394b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf16cc0 0x55fbddf1f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfc394b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 0x55fbddf1f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfc394b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1a9d30 con 0x55fbddf163a0 2024-09-19T08:23:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfc394b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1ab1a0 con 0x55fbddf25a90 2024-09-19T08:23:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfc394b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1aa400 con 0x55fbddf16cc0 2024-09-19T08:23:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfc394b740 10 client.0 _ll_register_callbacks cb 0x7fdfb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:23:45.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfba7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf163a0 0x55fbddf2df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfba7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf163a0 0x55fbddf2df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45592/0 (socket says 172.21.15.22:45592) 2024-09-19T08:23:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfba7fc640 1 -- 172.21.15.22:0/2289294519 learned_addr learned my addr 172.21.15.22:0/2289294519 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:23:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfbaffd640 1 --2- 172.21.15.22:0/2289294519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf16cc0 0x55fbddf1f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/2289294519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 0x55fbddf1f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfba7fc640 1 -- 172.21.15.22:0/2289294519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf16cc0 msgr2=0x55fbddf1f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/2289294519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf16cc0 0x55fbddf1f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfba7fc640 1 -- 172.21.15.22:0/2289294519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 msgr2=0x55fbddf1f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:23:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/2289294519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 0x55fbddf1f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfba7fc640 1 -- 172.21.15.22:0/2289294519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbddf1faf0 con 0x55fbddf163a0 2024-09-19T08:23:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfba7fc640 1 --2- 172.21.15.22:0/2289294519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf163a0 0x55fbddf2df40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40115a0 tx=0x7fdfa402f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a669f172528674 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:23:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.486+0000 7fdfa27fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:23:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.487+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2289294519 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4023070 con 0x55fbddf163a0 2024-09-19T08:23:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.487+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2289294519 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fdfa4011240 con 0x55fbddf163a0 2024-09-19T08:23:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.487+0000 7fdfc394b740 1 -- 172.21.15.22:0/2289294519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfb4005500 con 0x55fbddf163a0 2024-09-19T08:23:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.487+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2289294519 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdfa4044050 con 0x55fbddf163a0 2024-09-19T08:23:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.487+0000 7fdfc394b740 10 client.16947 fetch_fsmap learned FSMap version 35 2024-09-19T08:23:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.487+0000 7fdfc394b740 1 -- 172.21.15.22:0/2289294519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fbddf9fad0 con 0x55fbddf163a0 2024-09-19T08:23:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.487+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2289294519 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb4005500 con 0x55fbddf163a0 2024-09-19T08:23:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.487+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2289294519 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fdfa402c040 con 0x55fbddf163a0 2024-09-19T08:23:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.487+0000 7fdfc394b740 10 client.16947 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:23:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.487+0000 7fdfc394b740 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:23:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.487+0000 7fdfc394b740 20 client.16947 populate_metadata read hostname 'smithi022' 2024-09-19T08:23:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.487+0000 7fdfc394b740 1 --2- 172.21.15.22:0/2289294519 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fbddfa0130 0x55fbddfa2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:23:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.487+0000 7fdfc394b740 4 client.16947 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:23:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.487+0000 7fdfc394b740 1 -- 172.21.15.22:0/2289294519 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fbdde83d60 con 0x55fbddfa0130 2024-09-19T08:23:45.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.488+0000 7fdfbaffd640 1 --2- 172.21.15.22:0/2289294519 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fbddfa0130 0x55fbddfa2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:23:45.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.488+0000 7fdfa27fc640 10 client.16947 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:45.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.488+0000 7fdfbaffd640 1 --2- 172.21.15.22:0/2289294519 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fbddfa0130 0x55fbddfa2520 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58d4d62f972199f2 server_cookie=8e36c612a3331644 in_seq=0 out_seq=0 2024-09-19T08:23:45.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2289294519 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fbdde83d60 con 0x55fbddfa0130 2024-09-19T08:23:45.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfa27fc640 10 client.16947 handle_command_reply: tid=0 2024-09-19T08:23:45.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc394b740 1 client.16947 shutdown 2024-09-19T08:23:45.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc394b740 1 -- 172.21.15.22:0/2289294519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf163a0 msgr2=0x55fbddf2df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:45.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc394b740 1 --2- 172.21.15.22:0/2289294519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf163a0 0x55fbddf2df40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40115a0 tx=0x7fdfa402f040 comp rx=0 tx=0).stop 2024-09-19T08:23:45.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc394b740 1 -- 172.21.15.22:0/2289294519 shutdown_connections 2024-09-19T08:23:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc394b740 1 --2- 172.21.15.22:0/2289294519 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55fbddfa0130 0x55fbddfa2520 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc394b740 1 --2- 172.21.15.22:0/2289294519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbddf25a90 0x55fbddf1f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc394b740 1 --2- 172.21.15.22:0/2289294519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbddf16cc0 0x55fbddf1f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc394b740 1 --2- 172.21.15.22:0/2289294519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fbddf163a0 0x55fbddf2df40 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc394b740 1 -- 172.21.15.22:0/2289294519 >> 172.21.15.22:0/2289294519 conn(0x55fbddf12a30 msgr2=0x55fbddf17f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfa27fc640 0 client.16947 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:23:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc394b740 1 -- 172.21.15.22:0/2289294519 shutdown_connections 2024-09-19T08:23:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc394b740 1 -- 172.21.15.22:0/2289294519 wait complete. 2024-09-19T08:23:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc394b740 20 client.16947 trim_cache size 0 max 16384 2024-09-19T08:23:45.496 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:23:45.497 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:23:45.497 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-19T08:23:45.497 INFO:teuthology.orchestra.run.smithi022.stdout: "2d65834f-9d9a-431b-b24b-219b1dbe7fa4": { 2024-09-19T08:23:45.497 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-19T08:23:45.497 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "2d65834f-9d9a-431b-b24b-219b1dbe7fa4", 2024-09-19T08:23:45.497 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-09-19T08:23:45.497 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:23:45.497 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:23:45.497 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:23:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc1572640 1 -- 172.21.15.22:0/790569475 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdfa80a33b0 msgr2=0x7fdfa80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc1572640 1 --2- 172.21.15.22:0/790569475 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdfa80a33b0 0x7fdfa80a5870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdfac098bc0 tx=0x7fdfac0054d0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc1572640 1 -- 172.21.15.22:0/790569475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc106f20 msgr2=0x7fdfbc19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:23:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.489+0000 7fdfc1572640 1 --2- 172.21.15.22:0/790569475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc106f20 0x7fdfbc19ccd0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00eaa0 tx=0x7fdfac00ef60 comp rx=0 tx=0).stop 2024-09-19T08:23:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.490+0000 7fdfc1572640 1 -- 172.21.15.22:0/790569475 shutdown_connections 2024-09-19T08:23:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.490+0000 7fdfc1572640 1 --2- 172.21.15.22:0/790569475 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdfa80a33b0 0x7fdfa80a5870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.490+0000 7fdfc1572640 1 --2- 172.21.15.22:0/790569475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc106f20 0x7fdfbc19ccd0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.490+0000 7fdfc1572640 1 --2- 172.21.15.22:0/790569475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfbc1027a0 0x7fdfbc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.490+0000 7fdfc1572640 1 --2- 172.21.15.22:0/790569475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfbc101df0 0x7fdfbc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:23:45.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.490+0000 7fdfc1572640 1 -- 172.21.15.22:0/790569475 >> 172.21.15.22:0/790569475 conn(0x7fdfbc0fd8b0 msgr2=0x7fdfbc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:23:45.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.490+0000 7fdfc1572640 1 -- 172.21.15.22:0/790569475 shutdown_connections 2024-09-19T08:23:45.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:23:45.490+0000 7fdfc1572640 1 -- 172.21.15.22:0/790569475 wait complete. 2024-09-19T08:23:45.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:45 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:23:45.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:45 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:45.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:45 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:23:45.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:45 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:45.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:45 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:23:45.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:45 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:45 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:23:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:45 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:45 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:23:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:45 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:45 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:23:45.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:45 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:46.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:45 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:23:46.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:45 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:46.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:45 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:23:46.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:45 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:46.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:45 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:23:46.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:45 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:46.268 INFO:tasks.cephfs.filesystem:scrub status for tag:2d65834f-9d9a-431b-b24b-219b1dbe7fa4 - {'path': '/', 'tag': '2d65834f-9d9a-431b-b24b-219b1dbe7fa4', 'options': 'recursive,force'} 2024-09-19T08:23:46.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:46 smithi064 ceph-mon[34900]: pgmap v269: 97 pgs: 97 active+clean; 1018 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 513 op/s 2024-09-19T08:23:47.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:46 smithi157 ceph-mon[35143]: pgmap v269: 97 pgs: 97 active+clean; 1018 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 513 op/s 2024-09-19T08:23:47.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:46 smithi022 ceph-mon[30963]: pgmap v269: 97 pgs: 97 active+clean; 1018 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 513 op/s 2024-09-19T08:23:48.809 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:48 smithi022 ceph-mon[30963]: pgmap v270: 97 pgs: 97 active+clean; 1018 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 581 op/s 2024-09-19T08:23:48.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:48 smithi064 ceph-mon[34900]: pgmap v270: 97 pgs: 97 active+clean; 1018 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 581 op/s 2024-09-19T08:23:49.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:48 smithi157 ceph-mon[35143]: pgmap v270: 97 pgs: 97 active+clean; 1018 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 581 op/s 2024-09-19T08:23:50.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:50 smithi064 ceph-mon[34900]: pgmap v271: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 886 op/s 2024-09-19T08:23:51.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:50 smithi157 ceph-mon[35143]: pgmap v271: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 886 op/s 2024-09-19T08:23:51.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:50 smithi022 ceph-mon[30963]: pgmap v271: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 886 op/s 2024-09-19T08:23:52.805 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:52 smithi157 ceph-mon[35143]: pgmap v272: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 633 op/s 2024-09-19T08:23:52.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:52 smithi064 ceph-mon[34900]: pgmap v272: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 633 op/s 2024-09-19T08:23:53.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:52 smithi022 ceph-mon[30963]: pgmap v272: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 633 op/s 2024-09-19T08:23:54.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:54 smithi064 ceph-mon[34900]: pgmap v273: 97 pgs: 97 active+clean; 1005 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.8 MiB/s wr, 1.32k op/s 2024-09-19T08:23:55.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:54 smithi157 ceph-mon[35143]: pgmap v273: 97 pgs: 97 active+clean; 1005 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.8 MiB/s wr, 1.32k op/s 2024-09-19T08:23:55.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:54 smithi022 ceph-mon[30963]: pgmap v273: 97 pgs: 97 active+clean; 1005 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.8 MiB/s wr, 1.32k op/s 2024-09-19T08:23:56.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:56 smithi064 ceph-mon[34900]: pgmap v274: 97 pgs: 97 active+clean; 1005 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 1.12k op/s 2024-09-19T08:23:56.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:56 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:57.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:56 smithi157 ceph-mon[35143]: pgmap v274: 97 pgs: 97 active+clean; 1005 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 1.12k op/s 2024-09-19T08:23:57.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:56 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:57.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:56 smithi022 ceph-mon[30963]: pgmap v274: 97 pgs: 97 active+clean; 1005 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 1.12k op/s 2024-09-19T08:23:57.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:56 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:23:58.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:23:58 smithi064 ceph-mon[34900]: pgmap v275: 97 pgs: 97 active+clean; 1014 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 1.25k op/s 2024-09-19T08:23:59.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:23:58 smithi157 ceph-mon[35143]: pgmap v275: 97 pgs: 97 active+clean; 1014 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 1.25k op/s 2024-09-19T08:23:59.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:23:58 smithi022 ceph-mon[30963]: pgmap v275: 97 pgs: 97 active+clean; 1014 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 1.25k op/s 2024-09-19T08:24:00.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:00 smithi064 ceph-mon[34900]: pgmap v276: 97 pgs: 97 active+clean; 1023 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 1.75k op/s 2024-09-19T08:24:01.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:00 smithi157 ceph-mon[35143]: pgmap v276: 97 pgs: 97 active+clean; 1023 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 1.75k op/s 2024-09-19T08:24:01.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:00 smithi022 ceph-mon[30963]: pgmap v276: 97 pgs: 97 active+clean; 1023 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 1.75k op/s 2024-09-19T08:24:02.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:02 smithi064 ceph-mon[34900]: pgmap v277: 97 pgs: 97 active+clean; 1023 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 1.38k op/s 2024-09-19T08:24:02.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:02 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:24:03.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:02 smithi157 ceph-mon[35143]: pgmap v277: 97 pgs: 97 active+clean; 1023 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 1.38k op/s 2024-09-19T08:24:03.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:02 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:24:03.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:02 smithi022 ceph-mon[30963]: pgmap v277: 97 pgs: 97 active+clean; 1023 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 1.38k op/s 2024-09-19T08:24:03.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:02 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:24:03.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:03 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:24:03.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:03 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:24:04.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:03 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:24:04.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:03 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:24:04.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:03 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:24:04.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:03 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:24:04.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:04 smithi064 ceph-mon[34900]: pgmap v278: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.8 MiB/s wr, 1.60k op/s 2024-09-19T08:24:05.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:04 smithi157 ceph-mon[35143]: pgmap v278: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.8 MiB/s wr, 1.60k op/s 2024-09-19T08:24:05.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:04 smithi022 ceph-mon[30963]: pgmap v278: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.8 MiB/s wr, 1.60k op/s 2024-09-19T08:24:06.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:06 smithi064 ceph-mon[34900]: pgmap v279: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 914 op/s 2024-09-19T08:24:07.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:06 smithi157 ceph-mon[35143]: pgmap v279: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 914 op/s 2024-09-19T08:24:07.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:06 smithi022 ceph-mon[30963]: pgmap v279: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 914 op/s 2024-09-19T08:24:08.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:08 smithi064 ceph-mon[34900]: pgmap v280: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 948 op/s 2024-09-19T08:24:08.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:08 smithi064 ceph-mon[34900]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T08:24:09.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:08 smithi157 ceph-mon[35143]: pgmap v280: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 948 op/s 2024-09-19T08:24:09.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:08 smithi157 ceph-mon[35143]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T08:24:09.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:08 smithi022 ceph-mon[30963]: pgmap v280: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 948 op/s 2024-09-19T08:24:09.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:08 smithi022 ceph-mon[30963]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T08:24:10.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:10 smithi064 ceph-mon[34900]: pgmap v282: 97 pgs: 16 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 73 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 510 op/s 2024-09-19T08:24:11.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:10 smithi157 ceph-mon[35143]: pgmap v282: 97 pgs: 16 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 73 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 510 op/s 2024-09-19T08:24:11.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:10 smithi022 ceph-mon[30963]: pgmap v282: 97 pgs: 16 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 73 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 510 op/s 2024-09-19T08:24:12.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:12 smithi064 ceph-mon[34900]: pgmap v283: 97 pgs: 16 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 73 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 510 op/s 2024-09-19T08:24:13.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:12 smithi157 ceph-mon[35143]: pgmap v283: 97 pgs: 16 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 73 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 510 op/s 2024-09-19T08:24:13.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:12 smithi022 ceph-mon[30963]: pgmap v283: 97 pgs: 16 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 73 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 510 op/s 2024-09-19T08:24:14.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:14 smithi064 ceph-mon[34900]: pgmap v284: 97 pgs: 97 active+clean; 1005 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 445 op/s 2024-09-19T08:24:15.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:14 smithi157 ceph-mon[35143]: pgmap v284: 97 pgs: 97 active+clean; 1005 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 445 op/s 2024-09-19T08:24:15.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:14 smithi022 ceph-mon[30963]: pgmap v284: 97 pgs: 97 active+clean; 1005 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 445 op/s 2024-09-19T08:24:16.269 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:24:16.627 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:24:16.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:16 smithi022 ceph-mon[30963]: pgmap v285: 97 pgs: 97 active+clean; 1005 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 445 op/s 2024-09-19T08:24:16.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:16 smithi022 ceph-mon[30963]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T08:24:16.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:16 smithi157 ceph-mon[35143]: pgmap v285: 97 pgs: 97 active+clean; 1005 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 445 op/s 2024-09-19T08:24:16.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:16 smithi157 ceph-mon[35143]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T08:24:16.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:16 smithi064 ceph-mon[34900]: pgmap v285: 97 pgs: 97 active+clean; 1005 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 445 op/s 2024-09-19T08:24:16.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:16 smithi064 ceph-mon[34900]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T08:24:17.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.145+0000 7f165b0af640 1 -- 172.21.15.22:0/945718708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1654103780 msgr2=0x7f1654107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:17.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.145+0000 7f165b0af640 1 --2- 172.21.15.22:0/945718708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1654103780 0x7f1654107bc0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1640009930 tx=0x7f164002a420 comp rx=0 tx=0).stop 2024-09-19T08:24:17.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.145+0000 7f165b0af640 1 -- 172.21.15.22:0/945718708 shutdown_connections 2024-09-19T08:24:17.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.145+0000 7f165b0af640 1 --2- 172.21.15.22:0/945718708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1654108300 0x7f165410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.145+0000 7f165b0af640 1 --2- 172.21.15.22:0/945718708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1654103780 0x7f1654107bc0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.145+0000 7f165b0af640 1 --2- 172.21.15.22:0/945718708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1654102dd0 0x7f16541031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.145+0000 7f165b0af640 1 -- 172.21.15.22:0/945718708 >> 172.21.15.22:0/945718708 conn(0x7f16540fc7d0 msgr2=0x7f16540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.145+0000 7f165b0af640 1 -- 172.21.15.22:0/945718708 shutdown_connections 2024-09-19T08:24:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.145+0000 7f165b0af640 1 -- 172.21.15.22:0/945718708 wait complete. 2024-09-19T08:24:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.146+0000 7f165b0af640 1 Processor -- start 2024-09-19T08:24:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.146+0000 7f165b0af640 1 -- start start 2024-09-19T08:24:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.146+0000 7f165b0af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1654102dd0 0x7f16541a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.146+0000 7f165b0af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1654103780 0x7f16541a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.146+0000 7f165b0af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1654108300 0x7f165419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.146+0000 7f165b0af640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1654112050 con 0x7f1654108300 2024-09-19T08:24:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.146+0000 7f165b0af640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1654111ed0 con 0x7f1654102dd0 2024-09-19T08:24:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.146+0000 7f165b0af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16541121d0 con 0x7f1654103780 2024-09-19T08:24:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.146+0000 7f1658e4b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1654102dd0 0x7f16541a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.146+0000 7f165964c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1654108300 0x7f165419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.146+0000 7f1658e4b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1654102dd0 0x7f16541a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:44770/0 (socket says 172.21.15.22:44770) 2024-09-19T08:24:17.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.146+0000 7f1653fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1654103780 0x7f16541a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.146+0000 7f1658e4b640 1 -- 172.21.15.22:0/1043110210 learned_addr learned my addr 172.21.15.22:0/1043110210 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:17.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.147+0000 7f165964c640 1 -- 172.21.15.22:0/1043110210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1654103780 msgr2=0x7f16541a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:17.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.147+0000 7f165964c640 1 --2- 172.21.15.22:0/1043110210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1654103780 0x7f16541a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.147+0000 7f165964c640 1 -- 172.21.15.22:0/1043110210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1654102dd0 msgr2=0x7f16541a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:17.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.147+0000 7f165964c640 1 --2- 172.21.15.22:0/1043110210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1654102dd0 0x7f16541a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.147+0000 7f165964c640 1 -- 172.21.15.22:0/1043110210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165419ad50 con 0x7f1654108300 2024-09-19T08:24:17.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.147+0000 7f1658e4b640 1 --2- 172.21.15.22:0/1043110210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1654102dd0 0x7f16541a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:17.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.147+0000 7f165964c640 1 --2- 172.21.15.22:0/1043110210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1654108300 0x7f165419a620 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f164c009fd0 tx=0x7f164c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:17.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.147+0000 7f1651ffb640 1 -- 172.21.15.22:0/1043110210 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c00ce20 con 0x7f1654108300 2024-09-19T08:24:17.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.147+0000 7f165b0af640 1 -- 172.21.15.22:0/1043110210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165419b040 con 0x7f1654108300 2024-09-19T08:24:17.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.147+0000 7f1651ffb640 1 -- 172.21.15.22:0/1043110210 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f164c005020 con 0x7f1654108300 2024-09-19T08:24:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.147+0000 7f1651ffb640 1 -- 172.21.15.22:0/1043110210 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c010720 con 0x7f1654108300 2024-09-19T08:24:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.147+0000 7f165b0af640 1 -- 172.21.15.22:0/1043110210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16541a7330 con 0x7f1654108300 2024-09-19T08:24:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.149+0000 7f1651ffb640 1 -- 172.21.15.22:0/1043110210 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f164c005540 con 0x7f1654108300 2024-09-19T08:24:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.150+0000 7f1651ffb640 1 --2- 172.21.15.22:0/1043110210 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16440a33b0 0x7f16440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.150+0000 7f1658e4b640 1 --2- 172.21.15.22:0/1043110210 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16440a33b0 0x7f16440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.150+0000 7f1651ffb640 1 -- 172.21.15.22:0/1043110210 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f164c0d03b0 con 0x7f1654108300 2024-09-19T08:24:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.151+0000 7f1658e4b640 1 --2- 172.21.15.22:0/1043110210 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16440a33b0 0x7f16440a5870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1648005fd0 tx=0x7f164800b480 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.151+0000 7f165d488740 1 Processor -- start 2024-09-19T08:24:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.152+0000 7f165d488740 1 -- start start 2024-09-19T08:24:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.152+0000 7f165d488740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b64a0 0x5623a91b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.152+0000 7f165d488740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b6dc0 0x5623a91c1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.152+0000 7f165d488740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.152+0000 7f165d488740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5623a911c2e0 con 0x5623a91b6dc0 2024-09-19T08:24:17.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.152+0000 7f165d488740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5623a90c3ce0 con 0x5623a91c1680 2024-09-19T08:24:17.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.152+0000 7f165d488740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5623a91182a0 con 0x5623a91b64a0 2024-09-19T08:24:17.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.152+0000 7f1658e4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b64a0 0x5623a91b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.152+0000 7f1653fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b6dc0 0x5623a91c1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.152+0000 7f1658e4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b64a0 0x5623a91b6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:57554/0 (socket says 172.21.15.22:57554) 2024-09-19T08:24:17.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.152+0000 7f1658e4b640 1 -- 172.21.15.22:0/4105999299 learned_addr learned my addr 172.21.15.22:0/4105999299 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:17.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.152+0000 7f165964c640 1 --2- 172.21.15.22:0/4105999299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.153+0000 7f1653fff640 1 -- 172.21.15.22:0/4105999299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b64a0 msgr2=0x5623a91b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:17.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.153+0000 7f1653fff640 1 --2- 172.21.15.22:0/4105999299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b64a0 0x5623a91b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.153+0000 7f1653fff640 1 -- 172.21.15.22:0/4105999299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 msgr2=0x5623a91c3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:17.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.153+0000 7f1653fff640 1 --2- 172.21.15.22:0/4105999299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91c3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.153+0000 7f1653fff640 1 -- 172.21.15.22:0/4105999299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623a90a8440 con 0x5623a91b6dc0 2024-09-19T08:24:17.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.153+0000 7f1653fff640 1 --2- 172.21.15.22:0/4105999299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b6dc0 0x5623a91c1140 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f164002a940 tx=0x7f1640005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675322a9e8c6b55f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:17.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.153+0000 7f163effd640 1 -- 172.21.15.22:0/4105999299 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640040070 con 0x5623a91b6dc0 2024-09-19T08:24:17.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.153+0000 7f163effd640 1 -- 172.21.15.22:0/4105999299 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1640004030 con 0x5623a91b6dc0 2024-09-19T08:24:17.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.153+0000 7f163effd640 1 -- 172.21.15.22:0/4105999299 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640035040 con 0x5623a91b6dc0 2024-09-19T08:24:17.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.154+0000 7f165d488740 1 -- 172.21.15.22:0/4105999299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b6dc0 msgr2=0x5623a91c1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:17.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.154+0000 7f165d488740 1 --2- 172.21.15.22:0/4105999299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b6dc0 0x5623a91c1140 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f164002a940 tx=0x7f1640005e20 comp rx=0 tx=0).stop 2024-09-19T08:24:17.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.154+0000 7f165d488740 1 -- 172.21.15.22:0/4105999299 shutdown_connections 2024-09-19T08:24:17.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.154+0000 7f165d488740 1 --2- 172.21.15.22:0/4105999299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91c3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.154+0000 7f165d488740 1 --2- 172.21.15.22:0/4105999299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b6dc0 0x5623a91c1140 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.154+0000 7f165d488740 1 --2- 172.21.15.22:0/4105999299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b64a0 0x5623a91b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.154+0000 7f165d488740 1 -- 172.21.15.22:0/4105999299 >> 172.21.15.22:0/4105999299 conn(0x5623a91b2a30 msgr2=0x5623a91b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:17.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.154+0000 7f165d488740 1 -- 172.21.15.22:0/4105999299 shutdown_connections 2024-09-19T08:24:17.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.154+0000 7f165d488740 1 -- 172.21.15.22:0/4105999299 wait complete. 2024-09-19T08:24:17.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.154+0000 7f165d488740 1 -- start start 2024-09-19T08:24:17.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.154+0000 7f165d488740 1 Processor -- start 2024-09-19T08:24:17.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f165d488740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b64a0 0x5623a91c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f165d488740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b6dc0 0x5623a91be250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f165d488740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91be790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f165d488740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5623a91c4250 con 0x5623a91b6dc0 2024-09-19T08:24:17.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f165d488740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5623a90d7050 con 0x5623a91c1680 2024-09-19T08:24:17.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f165d488740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5623a91c43d0 con 0x5623a91b64a0 2024-09-19T08:24:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f165d488740 10 client.0 _ll_register_callbacks cb 0x5623a9117160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:24:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f1653fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b6dc0 0x5623a91be250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f1653fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b6dc0 0x5623a91be250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48928/0 (socket says 172.21.15.22:48928) 2024-09-19T08:24:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f1653fff640 1 -- 172.21.15.22:0/4121870977 learned_addr learned my addr 172.21.15.22:0/4121870977 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f165964c640 1 --2- 172.21.15.22:0/4121870977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91be790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f1658e4b640 1 --2- 172.21.15.22:0/4121870977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b64a0 0x5623a91c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f1653fff640 1 -- 172.21.15.22:0/4121870977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b64a0 msgr2=0x5623a91c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f1653fff640 1 --2- 172.21.15.22:0/4121870977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b64a0 0x5623a91c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f1653fff640 1 -- 172.21.15.22:0/4121870977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 msgr2=0x5623a91be790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f1653fff640 1 --2- 172.21.15.22:0/4121870977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91be790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.155+0000 7f1653fff640 1 -- 172.21.15.22:0/4121870977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623a8e8d3c0 con 0x5623a91b6dc0 2024-09-19T08:24:17.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.156+0000 7f1653fff640 1 --2- 172.21.15.22:0/4121870977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b6dc0 0x5623a91be250 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1640006010 tx=0x7f1640009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4924b3d7eca4d610 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:17.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.156+0000 7f1633fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:24:17.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.156+0000 7f1633fff640 1 -- 172.21.15.22:0/4121870977 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640040070 con 0x5623a91b6dc0 2024-09-19T08:24:17.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.156+0000 7f1633fff640 1 -- 172.21.15.22:0/4121870977 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f16400043e0 con 0x5623a91b6dc0 2024-09-19T08:24:17.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.156+0000 7f165d488740 1 -- 172.21.15.22:0/4121870977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1640004030 con 0x5623a91b6dc0 2024-09-19T08:24:17.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.156+0000 7f1633fff640 1 -- 172.21.15.22:0/4121870977 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1640056050 con 0x5623a91b6dc0 2024-09-19T08:24:17.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.156+0000 7f165d488740 10 client.16959 fetch_fsmap learned FSMap version 35 2024-09-19T08:24:17.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.156+0000 7f165d488740 1 -- 172.21.15.22:0/4121870977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5623a91cfbd0 con 0x5623a91b6dc0 2024-09-19T08:24:17.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.156+0000 7f1633fff640 1 -- 172.21.15.22:0/4121870977 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1640004030 con 0x5623a91b6dc0 2024-09-19T08:24:17.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.157+0000 7f1633fff640 1 -- 172.21.15.22:0/4121870977 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f1640004770 con 0x5623a91b6dc0 2024-09-19T08:24:17.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.157+0000 7f165d488740 10 client.16959 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:24:17.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.157+0000 7f165d488740 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:24:17.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.157+0000 7f165d488740 20 client.16959 populate_metadata read hostname 'smithi022' 2024-09-19T08:24:17.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.157+0000 7f165d488740 1 --2- 172.21.15.22:0/4121870977 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5623a91d0f20 0x5623a91d3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.157+0000 7f165d488740 4 client.16959 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:24:17.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.157+0000 7f165d488740 1 -- 172.21.15.22:0/4121870977 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623a9129850 con 0x5623a91d0f20 2024-09-19T08:24:17.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.157+0000 7f1658e4b640 1 --2- 172.21.15.22:0/4121870977 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5623a91d0f20 0x5623a91d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.157+0000 7f1658e4b640 1 --2- 172.21.15.22:0/4121870977 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5623a91d0f20 0x5623a91d3310 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=585ba7f0716ffe40 server_cookie=5eb3d78e674ba26e in_seq=0 out_seq=0 2024-09-19T08:24:17.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.157+0000 7f1633fff640 10 client.16959 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:17.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.158+0000 7f1633fff640 1 -- 172.21.15.22:0/4121870977 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623a9129850 con 0x5623a91d0f20 2024-09-19T08:24:17.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.158+0000 7f1633fff640 10 client.16959 handle_command_reply: tid=0 2024-09-19T08:24:17.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.158+0000 7f165d488740 1 client.16959 shutdown 2024-09-19T08:24:17.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.158+0000 7f165d488740 1 -- 172.21.15.22:0/4121870977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b6dc0 msgr2=0x5623a91be250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:17.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.158+0000 7f165d488740 1 --2- 172.21.15.22:0/4121870977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b6dc0 0x5623a91be250 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1640006010 tx=0x7f1640009520 comp rx=0 tx=0).stop 2024-09-19T08:24:17.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.158+0000 7f165d488740 1 -- 172.21.15.22:0/4121870977 shutdown_connections 2024-09-19T08:24:17.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.158+0000 7f165d488740 1 --2- 172.21.15.22:0/4121870977 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5623a91d0f20 0x5623a91d3310 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.158+0000 7f165d488740 1 --2- 172.21.15.22:0/4121870977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91be790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.158+0000 7f165d488740 1 --2- 172.21.15.22:0/4121870977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b6dc0 0x5623a91be250 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.158+0000 7f165d488740 1 --2- 172.21.15.22:0/4121870977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b64a0 0x5623a91c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.158+0000 7f1633fff640 0 client.16959 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:17.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.158+0000 7f165d488740 1 -- 172.21.15.22:0/4121870977 >> 172.21.15.22:0/4121870977 conn(0x5623a91b2a30 msgr2=0x5623a91b8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:17.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.158+0000 7f165d488740 1 -- 172.21.15.22:0/4121870977 shutdown_connections 2024-09-19T08:24:17.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.158+0000 7f165d488740 1 -- 172.21.15.22:0/4121870977 wait complete. 2024-09-19T08:24:17.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.159+0000 7f165d488740 20 client.16959 trim_cache size 0 max 16384 2024-09-19T08:24:17.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.196+0000 7f165d488740 1 Processor -- start 2024-09-19T08:24:17.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f165d488740 1 -- start start 2024-09-19T08:24:17.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f165d488740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b64a0 0x5623a91d6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f165d488740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b6dc0 0x5623a91d6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f165d488740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f165d488740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f16541b2390 con 0x5623a91b64a0 2024-09-19T08:24:17.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f165d488740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16541afc30 con 0x5623a91c1680 2024-09-19T08:24:17.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f165d488740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16541adaa0 con 0x5623a91b6dc0 2024-09-19T08:24:17.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f1658e4b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b64a0 0x5623a91d6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f1658e4b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b64a0 0x5623a91d6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48932/0 (socket says 172.21.15.22:48932) 2024-09-19T08:24:17.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f1658e4b640 1 -- 172.21.15.22:0/2993979480 learned_addr learned my addr 172.21.15.22:0/2993979480 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:17.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f1653fff640 1 --2- 172.21.15.22:0/2993979480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b6dc0 0x5623a91d6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f165964c640 1 --2- 172.21.15.22:0/2993979480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f1658e4b640 1 -- 172.21.15.22:0/2993979480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b6dc0 msgr2=0x5623a91d6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:17.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f1658e4b640 1 --2- 172.21.15.22:0/2993979480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b6dc0 0x5623a91d6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f1658e4b640 1 -- 172.21.15.22:0/2993979480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 msgr2=0x5623a91bb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:17.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f1658e4b640 1 --2- 172.21.15.22:0/2993979480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91bb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.197+0000 7f1658e4b640 1 -- 172.21.15.22:0/2993979480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623a91c40a0 con 0x5623a91b64a0 2024-09-19T08:24:17.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.198+0000 7f1658e4b640 1 --2- 172.21.15.22:0/2993979480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b64a0 0x5623a91d6330 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x5623a91c44e0 tx=0x7f1648013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da77e7e4827b953 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:17.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.198+0000 7f163e7fc640 1 -- 172.21.15.22:0/2993979480 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648011e40 con 0x5623a91b64a0 2024-09-19T08:24:17.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.198+0000 7f163e7fc640 1 -- 172.21.15.22:0/2993979480 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1648005500 con 0x5623a91b64a0 2024-09-19T08:24:17.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.198+0000 7f163e7fc640 1 -- 172.21.15.22:0/2993979480 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648004c00 con 0x5623a91b64a0 2024-09-19T08:24:17.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.199+0000 7f165d488740 1 -- 172.21.15.22:0/2993979480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b64a0 msgr2=0x5623a91d6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:17.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.199+0000 7f165d488740 1 --2- 172.21.15.22:0/2993979480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b64a0 0x5623a91d6330 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x5623a91c44e0 tx=0x7f1648013000 comp rx=0 tx=0).stop 2024-09-19T08:24:17.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.199+0000 7f165d488740 1 -- 172.21.15.22:0/2993979480 shutdown_connections 2024-09-19T08:24:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.199+0000 7f165d488740 1 --2- 172.21.15.22:0/2993979480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91bb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.199+0000 7f165d488740 1 --2- 172.21.15.22:0/2993979480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b6dc0 0x5623a91d6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.199+0000 7f165d488740 1 --2- 172.21.15.22:0/2993979480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b64a0 0x5623a91d6330 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.199+0000 7f165d488740 1 -- 172.21.15.22:0/2993979480 >> 172.21.15.22:0/2993979480 conn(0x5623a91b2a30 msgr2=0x5623a91b9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.199+0000 7f165d488740 1 -- 172.21.15.22:0/2993979480 shutdown_connections 2024-09-19T08:24:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.199+0000 7f165d488740 1 -- 172.21.15.22:0/2993979480 wait complete. 2024-09-19T08:24:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.199+0000 7f165d488740 1 -- start start 2024-09-19T08:24:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.199+0000 7f165d488740 1 Processor -- start 2024-09-19T08:24:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.199+0000 7f165d488740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b64a0 0x5623a91d6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.200+0000 7f165d488740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b6dc0 0x5623a91d3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.200+0000 7f165d488740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91d41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.200+0000 7f165d488740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f16541ae0f0 con 0x5623a91b64a0 2024-09-19T08:24:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.200+0000 7f165d488740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16541af560 con 0x5623a91c1680 2024-09-19T08:24:17.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.200+0000 7f165d488740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16541ae7c0 con 0x5623a91b6dc0 2024-09-19T08:24:17.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.200+0000 7f165d488740 10 client.0 _ll_register_callbacks cb 0x7f164c005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:24:17.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.200+0000 7f1658e4b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b64a0 0x5623a91d6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.200+0000 7f1658e4b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b64a0 0x5623a91d6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48944/0 (socket says 172.21.15.22:48944) 2024-09-19T08:24:17.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.200+0000 7f1658e4b640 1 -- 172.21.15.22:0/3405641638 learned_addr learned my addr 172.21.15.22:0/3405641638 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:17.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.201+0000 7f1653fff640 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b6dc0 0x5623a91d3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.201+0000 7f165964c640 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91d41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.201+0000 7f1653fff640 1 -- 172.21.15.22:0/3405641638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 msgr2=0x5623a91d41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:17.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.201+0000 7f1653fff640 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91d41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.201+0000 7f1653fff640 1 -- 172.21.15.22:0/3405641638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b64a0 msgr2=0x5623a91d6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:17.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.201+0000 7f1653fff640 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b64a0 0x5623a91d6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.201+0000 7f1653fff640 1 -- 172.21.15.22:0/3405641638 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623a91d4850 con 0x5623a91b6dc0 2024-09-19T08:24:17.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.201+0000 7f1658e4b640 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b64a0 0x5623a91d6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:17.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.201+0000 7f165964c640 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91d41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:24:17.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.201+0000 7f1653fff640 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b6dc0 0x5623a91d3c70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1640004bc0 tx=0x7f16400049a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c73db78dacc7d443 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:17.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.201+0000 7f1633fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:24:17.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.202+0000 7f1633fff640 1 -- 172.21.15.22:0/3405641638 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640040070 con 0x5623a91b6dc0 2024-09-19T08:24:17.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.202+0000 7f1633fff640 1 -- 172.21.15.22:0/3405641638 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f16400315b0 con 0x5623a91b6dc0 2024-09-19T08:24:17.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.202+0000 7f165d488740 1 -- 172.21.15.22:0/3405641638 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1648005500 con 0x5623a91b6dc0 2024-09-19T08:24:17.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.202+0000 7f1633fff640 1 -- 172.21.15.22:0/3405641638 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1640067050 con 0x5623a91b6dc0 2024-09-19T08:24:17.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.202+0000 7f165d488740 10 client.26443 fetch_fsmap learned FSMap version 35 2024-09-19T08:24:17.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.202+0000 7f165d488740 1 -- 172.21.15.22:0/3405641638 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5623a923b700 con 0x5623a91b6dc0 2024-09-19T08:24:17.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.202+0000 7f1633fff640 1 -- 172.21.15.22:0/3405641638 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f164004d070 con 0x5623a91b6dc0 2024-09-19T08:24:17.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.202+0000 7f1633fff640 1 -- 172.21.15.22:0/3405641638 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f1640062030 con 0x5623a91b6dc0 2024-09-19T08:24:17.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.202+0000 7f165d488740 10 client.26443 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:24:17.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.202+0000 7f165d488740 10 client.26443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:24:17.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.202+0000 7f165d488740 20 client.26443 populate_metadata read hostname 'smithi022' 2024-09-19T08:24:17.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.202+0000 7f165d488740 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5623a923bea0 0x5623a923e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:17.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.202+0000 7f165d488740 4 client.26443 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:24:17.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.202+0000 7f165d488740 1 -- 172.21.15.22:0/3405641638 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1648005500 con 0x5623a923bea0 2024-09-19T08:24:17.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.203+0000 7f1658e4b640 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5623a923bea0 0x5623a923e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:17.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.203+0000 7f1633fff640 10 client.26443 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:17.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.203+0000 7f1658e4b640 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5623a923bea0 0x5623a923e2c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e269e897a4c9e03 server_cookie=eb21162265e7d0cb in_seq=0 out_seq=0 2024-09-19T08:24:17.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.203+0000 7f1633fff640 1 -- 172.21.15.22:0/3405641638 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f1648005500 con 0x5623a923bea0 2024-09-19T08:24:17.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.203+0000 7f1633fff640 10 client.26443 handle_command_reply: tid=0 2024-09-19T08:24:17.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.203+0000 7f165d488740 1 client.26443 shutdown 2024-09-19T08:24:17.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.204+0000 7f165d488740 1 -- 172.21.15.22:0/3405641638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b6dc0 msgr2=0x5623a91d3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:17.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.204+0000 7f165d488740 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b6dc0 0x5623a91d3c70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1640004bc0 tx=0x7f16400049a0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.204+0000 7f165d488740 1 -- 172.21.15.22:0/3405641638 shutdown_connections 2024-09-19T08:24:17.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.204+0000 7f165d488740 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x5623a923bea0 0x5623a923e2c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.204+0000 7f165d488740 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623a91c1680 0x5623a91d41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.204+0000 7f1633fff640 0 client.26443 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:17.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.204+0000 7f165d488740 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623a91b6dc0 0x5623a91d3c70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.204+0000 7f165d488740 1 --2- 172.21.15.22:0/3405641638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623a91b64a0 0x5623a91d6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.204+0000 7f165d488740 1 -- 172.21.15.22:0/3405641638 >> 172.21.15.22:0/3405641638 conn(0x5623a91b2a30 msgr2=0x5623a91b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:17.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.204+0000 7f165d488740 1 -- 172.21.15.22:0/3405641638 shutdown_connections 2024-09-19T08:24:17.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.204+0000 7f165d488740 1 -- 172.21.15.22:0/3405641638 wait complete. 2024-09-19T08:24:17.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.204+0000 7f165d488740 20 client.26443 trim_cache size 0 max 16384 2024-09-19T08:24:17.229 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:24:17.229 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-19T08:24:17.229 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-09-19T08:24:17.229 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:24:17.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.205+0000 7f165b0af640 1 -- 172.21.15.22:0/1043110210 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16440a33b0 msgr2=0x7f16440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:17.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.205+0000 7f165b0af640 1 --2- 172.21.15.22:0/1043110210 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16440a33b0 0x7f16440a5870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1648005fd0 tx=0x7f164800b480 comp rx=0 tx=0).stop 2024-09-19T08:24:17.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.205+0000 7f165b0af640 1 -- 172.21.15.22:0/1043110210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1654108300 msgr2=0x7f165419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:17.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.205+0000 7f165b0af640 1 --2- 172.21.15.22:0/1043110210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1654108300 0x7f165419a620 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f164c009fd0 tx=0x7f164c00ef10 comp rx=0 tx=0).stop 2024-09-19T08:24:17.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.205+0000 7f165b0af640 1 -- 172.21.15.22:0/1043110210 shutdown_connections 2024-09-19T08:24:17.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.205+0000 7f165b0af640 1 --2- 172.21.15.22:0/1043110210 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16440a33b0 0x7f16440a5870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.205+0000 7f165b0af640 1 --2- 172.21.15.22:0/1043110210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1654108300 0x7f165419a620 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.205+0000 7f165b0af640 1 --2- 172.21.15.22:0/1043110210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1654103780 0x7f16541a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.205+0000 7f165b0af640 1 --2- 172.21.15.22:0/1043110210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1654102dd0 0x7f16541a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:17.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.205+0000 7f165b0af640 1 -- 172.21.15.22:0/1043110210 >> 172.21.15.22:0/1043110210 conn(0x7f16540fc7d0 msgr2=0x7f165410d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:17.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.205+0000 7f165b0af640 1 -- 172.21.15.22:0/1043110210 shutdown_connections 2024-09-19T08:24:17.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:17.205+0000 7f165b0af640 1 -- 172.21.15.22:0/1043110210 wait complete. 2024-09-19T08:24:18.599 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:18 smithi022 ceph-mon[30963]: pgmap v287: 97 pgs: 97 active+clean; 1009 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 347 op/s 2024-09-19T08:24:18.627 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:24:18.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:24:18.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:18 smithi157 ceph-mon[35143]: pgmap v287: 97 pgs: 97 active+clean; 1009 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 347 op/s 2024-09-19T08:24:18.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:18 smithi064 ceph-mon[34900]: pgmap v287: 97 pgs: 97 active+clean; 1009 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 347 op/s 2024-09-19T08:24:18.894 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:24:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.410+0000 7f74498bc640 1 -- 172.21.15.22:0/2397924224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7444108e00 msgr2=0x7f744410ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.410+0000 7f74498bc640 1 --2- 172.21.15.22:0/2397924224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7444108e00 0x7f744410ce00 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7434009ca0 tx=0x7f743402a8c0 comp rx=0 tx=0).stop 2024-09-19T08:24:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.410+0000 7f74498bc640 1 -- 172.21.15.22:0/2397924224 shutdown_connections 2024-09-19T08:24:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.410+0000 7f74498bc640 1 --2- 172.21.15.22:0/2397924224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f744410d530 0x7f74441150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.410+0000 7f74498bc640 1 --2- 172.21.15.22:0/2397924224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7444108e00 0x7f744410ce00 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.410+0000 7f74498bc640 1 --2- 172.21.15.22:0/2397924224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7444108450 0x7f7444108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.410+0000 7f74498bc640 1 -- 172.21.15.22:0/2397924224 >> 172.21.15.22:0/2397924224 conn(0x7f7444077ba0 msgr2=0x7f7444079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.410+0000 7f74498bc640 1 -- 172.21.15.22:0/2397924224 shutdown_connections 2024-09-19T08:24:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.410+0000 7f74498bc640 1 -- 172.21.15.22:0/2397924224 wait complete. 2024-09-19T08:24:19.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.411+0000 7f74498bc640 1 Processor -- start 2024-09-19T08:24:19.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.411+0000 7f74498bc640 1 -- start start 2024-09-19T08:24:19.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.411+0000 7f74498bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7444108450 0x7f74441a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:19.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.411+0000 7f74498bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7444108e00 0x7f74441a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:19.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.411+0000 7f74498bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f744410d530 0x7f744419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:19.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.411+0000 7f7442ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7444108450 0x7f74441a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.411+0000 7f7442ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7444108450 0x7f74441a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54432/0 (socket says 172.21.15.22:54432) 2024-09-19T08:24:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f7442ffd640 1 -- 172.21.15.22:0/1305625106 learned_addr learned my addr 172.21.15.22:0/1305625106 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f74498bc640 1 -- 172.21.15.22:0/1305625106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f74441185a0 con 0x7f7444108450 2024-09-19T08:24:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f74427fc640 1 --2- 172.21.15.22:0/1305625106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7444108e00 0x7f74441a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f74498bc640 1 -- 172.21.15.22:0/1305625106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7444118420 con 0x7f744410d530 2024-09-19T08:24:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f74498bc640 1 -- 172.21.15.22:0/1305625106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7444118720 con 0x7f7444108e00 2024-09-19T08:24:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f74437fe640 1 --2- 172.21.15.22:0/1305625106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f744410d530 0x7f744419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f7442ffd640 1 -- 172.21.15.22:0/1305625106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7444108e00 msgr2=0x7f74441a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f7442ffd640 1 --2- 172.21.15.22:0/1305625106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7444108e00 0x7f74441a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f7442ffd640 1 -- 172.21.15.22:0/1305625106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f744410d530 msgr2=0x7f744419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f7442ffd640 1 --2- 172.21.15.22:0/1305625106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f744410d530 0x7f744419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f7442ffd640 1 -- 172.21.15.22:0/1305625106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744419efe0 con 0x7f7444108450 2024-09-19T08:24:19.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f7442ffd640 1 --2- 172.21.15.22:0/1305625106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7444108450 0x7f74441a46f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f742c00e9f0 tx=0x7f742c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:19.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f74488ba640 1 -- 172.21.15.22:0/1305625106 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c00cd80 con 0x7f7444108450 2024-09-19T08:24:19.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f74488ba640 1 -- 172.21.15.22:0/1305625106 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f742c005500 con 0x7f7444108450 2024-09-19T08:24:19.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.412+0000 7f74498bc640 1 -- 172.21.15.22:0/1305625106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744419f2d0 con 0x7f7444108450 2024-09-19T08:24:19.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.413+0000 7f74488ba640 1 -- 172.21.15.22:0/1305625106 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c010690 con 0x7f7444108450 2024-09-19T08:24:19.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.413+0000 7f74498bc640 1 -- 172.21.15.22:0/1305625106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74441ab5c0 con 0x7f7444108450 2024-09-19T08:24:19.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.414+0000 7f74498bc640 1 -- 172.21.15.22:0/1305625106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7444108830 con 0x7f7444108450 2024-09-19T08:24:19.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.417+0000 7f74488ba640 1 -- 172.21.15.22:0/1305625106 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f742c0046c0 con 0x7f7444108450 2024-09-19T08:24:19.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.418+0000 7f74488ba640 1 --2- 172.21.15.22:0/1305625106 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f74300a3430 0x7f74300a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:19.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.418+0000 7f74488ba640 1 -- 172.21.15.22:0/1305625106 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f742c0d03a0 con 0x7f7444108450 2024-09-19T08:24:19.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.418+0000 7f74427fc640 1 --2- 172.21.15.22:0/1305625106 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f74300a3430 0x7f74300a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.419+0000 7f74427fc640 1 --2- 172.21.15.22:0/1305625106 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f74300a3430 0x7f74300a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7434006010 tx=0x7f7434009820 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:19.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.424+0000 7f74488ba640 1 -- 172.21.15.22:0/1305625106 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f742c054210 con 0x7f7444108450 2024-09-19T08:24:19.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.657+0000 7f74498bc640 1 -- 172.21.15.22:0/1305625106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7444110770 con 0x7f7444108450 2024-09-19T08:24:19.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.658+0000 7f74488ba640 1 -- 172.21.15.22:0/1305625106 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11908 (secure 0 0 0) 0x7f742c09cd10 con 0x7f7444108450 2024-09-19T08:24:19.658 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:24:19.658 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"btime":"2024-09-19T08:23:33:502329+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:23:33.442439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:24:19.658 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-09-19T08:24:19.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.660+0000 7f74498bc640 1 -- 172.21.15.22:0/1305625106 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f74300a3430 msgr2=0x7f74300a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:19.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.660+0000 7f74498bc640 1 --2- 172.21.15.22:0/1305625106 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f74300a3430 0x7f74300a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7434006010 tx=0x7f7434009820 comp rx=0 tx=0).stop 2024-09-19T08:24:19.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.660+0000 7f74498bc640 1 -- 172.21.15.22:0/1305625106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7444108450 msgr2=0x7f74441a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:19.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.660+0000 7f74498bc640 1 --2- 172.21.15.22:0/1305625106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7444108450 0x7f74441a46f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f742c00e9f0 tx=0x7f742c00edf0 comp rx=0 tx=0).stop 2024-09-19T08:24:19.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.660+0000 7f74498bc640 1 -- 172.21.15.22:0/1305625106 shutdown_connections 2024-09-19T08:24:19.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.660+0000 7f74498bc640 1 --2- 172.21.15.22:0/1305625106 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f74300a3430 0x7f74300a58f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:19.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.660+0000 7f74498bc640 1 --2- 172.21.15.22:0/1305625106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f744410d530 0x7f744419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:19.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.660+0000 7f74498bc640 1 --2- 172.21.15.22:0/1305625106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7444108e00 0x7f74441a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:19.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.660+0000 7f74498bc640 1 --2- 172.21.15.22:0/1305625106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7444108450 0x7f74441a46f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:19.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.660+0000 7f74498bc640 1 -- 172.21.15.22:0/1305625106 >> 172.21.15.22:0/1305625106 conn(0x7f7444077ba0 msgr2=0x7f7444078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:19.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.660+0000 7f74498bc640 1 -- 172.21.15.22:0/1305625106 shutdown_connections 2024-09-19T08:24:19.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:19.660+0000 7f74498bc640 1 -- 172.21.15.22:0/1305625106 wait complete. 2024-09-19T08:24:20.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:20 smithi022 ceph-mon[30963]: pgmap v288: 97 pgs: 97 active+clean; 1021 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 499 op/s 2024-09-19T08:24:20.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:20 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1305625106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:24:20.580 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:24:20.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:20 smithi157 ceph-mon[35143]: pgmap v288: 97 pgs: 97 active+clean; 1021 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 499 op/s 2024-09-19T08:24:20.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:20 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1305625106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:24:20.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:20 smithi064 ceph-mon[34900]: pgmap v288: 97 pgs: 97 active+clean; 1021 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 499 op/s 2024-09-19T08:24:20.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:20 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1305625106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:24:20.847 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:24:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.012+0000 7ff799890640 1 -- 172.21.15.22:0/3355386078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff79410c170 msgr2=0x7ff79410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.012+0000 7ff799890640 1 --2- 172.21.15.22:0/3355386078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff79410c170 0x7ff79410e610 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009ca0 tx=0x7ff78c02a950 comp rx=0 tx=0).stop 2024-09-19T08:24:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.012+0000 7ff799890640 1 -- 172.21.15.22:0/3355386078 shutdown_connections 2024-09-19T08:24:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.012+0000 7ff799890640 1 --2- 172.21.15.22:0/3355386078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff79410c170 0x7ff79410e610 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.012+0000 7ff799890640 1 --2- 172.21.15.22:0/3355386078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff794103660 0x7ff79410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.012+0000 7ff799890640 1 --2- 172.21.15.22:0/3355386078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff794102d40 0x7ff794103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.012+0000 7ff799890640 1 -- 172.21.15.22:0/3355386078 >> 172.21.15.22:0/3355386078 conn(0x7ff7940fc7a0 msgr2=0x7ff7940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.012+0000 7ff799890640 1 -- 172.21.15.22:0/3355386078 shutdown_connections 2024-09-19T08:24:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.012+0000 7ff799890640 1 -- 172.21.15.22:0/3355386078 wait complete. 2024-09-19T08:24:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.013+0000 7ff799890640 1 Processor -- start 2024-09-19T08:24:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.013+0000 7ff799890640 1 -- start start 2024-09-19T08:24:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.013+0000 7ff799890640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff794102d40 0x7ff794198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.013+0000 7ff799890640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff794103660 0x7ff794198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.013+0000 7ff799890640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff79410c170 0x7ff79419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.013+0000 7ff799890640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff794111f70 con 0x7ff794103660 2024-09-19T08:24:22.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.013+0000 7ff799890640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff794111df0 con 0x7ff79410c170 2024-09-19T08:24:22.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.013+0000 7ff799890640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941120f0 con 0x7ff794102d40 2024-09-19T08:24:22.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.013+0000 7ff78bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff794103660 0x7ff794198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.013+0000 7ff78bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff794103660 0x7ff794198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54452/0 (socket says 172.21.15.22:54452) 2024-09-19T08:24:22.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.013+0000 7ff78bfff640 1 -- 172.21.15.22:0/2672028755 learned_addr learned my addr 172.21.15.22:0/2672028755 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:22.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.013+0000 7ff792ffd640 1 --2- 172.21.15.22:0/2672028755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff794102d40 0x7ff794198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.013+0000 7ff7937fe640 1 --2- 172.21.15.22:0/2672028755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff79410c170 0x7ff79419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.014+0000 7ff78bfff640 1 -- 172.21.15.22:0/2672028755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff794102d40 msgr2=0x7ff794198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:22.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.014+0000 7ff78bfff640 1 --2- 172.21.15.22:0/2672028755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff794102d40 0x7ff794198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.014+0000 7ff78bfff640 1 -- 172.21.15.22:0/2672028755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff79410c170 msgr2=0x7ff79419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:22.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.014+0000 7ff78bfff640 1 --2- 172.21.15.22:0/2672028755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff79410c170 0x7ff79419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.014+0000 7ff78bfff640 1 -- 172.21.15.22:0/2672028755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79419d3c0 con 0x7ff794103660 2024-09-19T08:24:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.014+0000 7ff78bfff640 1 --2- 172.21.15.22:0/2672028755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff794103660 0x7ff794198950 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00d740 tx=0x7ff77c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.014+0000 7ff79888e640 1 -- 172.21.15.22:0/2672028755 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c017070 con 0x7ff794103660 2024-09-19T08:24:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.014+0000 7ff79888e640 1 -- 172.21.15.22:0/2672028755 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff77c00de50 con 0x7ff794103660 2024-09-19T08:24:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.014+0000 7ff799890640 1 -- 172.21.15.22:0/2672028755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79419d6b0 con 0x7ff794103660 2024-09-19T08:24:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.014+0000 7ff79888e640 1 -- 172.21.15.22:0/2672028755 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00f620 con 0x7ff794103660 2024-09-19T08:24:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.014+0000 7ff799890640 1 -- 172.21.15.22:0/2672028755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7941a4f90 con 0x7ff794103660 2024-09-19T08:24:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.016+0000 7ff79888e640 1 -- 172.21.15.22:0/2672028755 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff77c009c30 con 0x7ff794103660 2024-09-19T08:24:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.017+0000 7ff79bc69740 1 Processor -- start 2024-09-19T08:24:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.017+0000 7ff79bc69740 1 -- start start 2024-09-19T08:24:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.017+0000 7ff79bc69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64fb40 0x55a52a64e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.017+0000 7ff79bc69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a64e5d0 0x55a52a64ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.017+0000 7ff79bc69740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a65d780 0x55a52a65fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.017+0000 7ff79bc69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a52a5b42f0 con 0x55a52a64fb40 2024-09-19T08:24:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.017+0000 7ff79bc69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a52a55bcf0 con 0x55a52a64e5d0 2024-09-19T08:24:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.017+0000 7ff79bc69740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a52a5b02b0 con 0x55a52a65d780 2024-09-19T08:24:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.017+0000 7ff79888e640 1 --2- 172.21.15.22:0/2672028755 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff7800a3430 0x7ff7800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.017+0000 7ff7937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64fb40 0x55a52a64e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.017+0000 7ff7937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64fb40 0x55a52a64e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54468/0 (socket says 172.21.15.22:54468) 2024-09-19T08:24:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.017+0000 7ff7937fe640 1 -- 172.21.15.22:0/1011126883 learned_addr learned my addr 172.21.15.22:0/1011126883 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.017+0000 7ff78bfff640 1 --2- 172.21.15.22:0/1011126883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a65d780 0x55a52a65fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.017+0000 7ff792ffd640 1 --2- 172.21.15.22:0/1011126883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a64e5d0 0x55a52a64ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.018+0000 7ff79888e640 1 -- 172.21.15.22:0/2672028755 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ff77c0cf820 con 0x7ff794103660 2024-09-19T08:24:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.018+0000 7ff7937fe640 1 --2- 172.21.15.22:0/2672028755 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff7800a3430 0x7ff7800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.018+0000 7ff792ffd640 1 -- 172.21.15.22:0/1011126883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a65d780 msgr2=0x55a52a65fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.018+0000 7ff792ffd640 1 --2- 172.21.15.22:0/1011126883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a65d780 0x55a52a65fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.018+0000 7ff792ffd640 1 -- 172.21.15.22:0/1011126883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64fb40 msgr2=0x55a52a64e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.018+0000 7ff792ffd640 1 --2- 172.21.15.22:0/1011126883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64fb40 0x55a52a64e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.018+0000 7ff792ffd640 1 -- 172.21.15.22:0/1011126883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a52a5403a0 con 0x55a52a64e5d0 2024-09-19T08:24:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.018+0000 7ff792ffd640 1 --2- 172.21.15.22:0/1011126883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a64e5d0 0x55a52a64ea10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff784009c00 tx=0x7ff784011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b9043680a327475 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.018+0000 7ff7937fe640 1 --2- 172.21.15.22:0/2672028755 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff7800a3430 0x7ff7800a58f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff78c030040 tx=0x7ff78c02a950 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.018+0000 7ff789ffb640 1 -- 172.21.15.22:0/1011126883 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784004310 con 0x55a52a64e5d0 2024-09-19T08:24:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.018+0000 7ff789ffb640 1 -- 172.21.15.22:0/1011126883 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7840044b0 con 0x55a52a64e5d0 2024-09-19T08:24:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.018+0000 7ff789ffb640 1 -- 172.21.15.22:0/1011126883 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7840035f0 con 0x55a52a64e5d0 2024-09-19T08:24:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.019+0000 7ff79bc69740 1 -- 172.21.15.22:0/1011126883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a64e5d0 msgr2=0x55a52a64ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.019+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1011126883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a64e5d0 0x55a52a64ea10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff784009c00 tx=0x7ff784011440 comp rx=0 tx=0).stop 2024-09-19T08:24:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.019+0000 7ff79bc69740 1 -- 172.21.15.22:0/1011126883 shutdown_connections 2024-09-19T08:24:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.019+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1011126883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a65d780 0x55a52a65fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.019+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1011126883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a64e5d0 0x55a52a64ea10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.019+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1011126883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64fb40 0x55a52a64e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.019+0000 7ff79bc69740 1 -- 172.21.15.22:0/1011126883 >> 172.21.15.22:0/1011126883 conn(0x55a52a617310 msgr2=0x55a52a64ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.019+0000 7ff79bc69740 1 -- 172.21.15.22:0/1011126883 shutdown_connections 2024-09-19T08:24:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.019+0000 7ff79bc69740 1 -- 172.21.15.22:0/1011126883 wait complete. 2024-09-19T08:24:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.019+0000 7ff79bc69740 1 -- start start 2024-09-19T08:24:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff79bc69740 1 Processor -- start 2024-09-19T08:24:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff79bc69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64e5d0 0x55a52a65d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff79bc69740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 0x55a52a65a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff79bc69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a65d780 0x55a52a65a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff79bc69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a52a660480 con 0x55a52a64e5d0 2024-09-19T08:24:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff79bc69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a52a56f060 con 0x55a52a65d780 2024-09-19T08:24:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff79bc69740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a52a660600 con 0x55a52a64fb40 2024-09-19T08:24:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff79bc69740 10 client.0 _ll_register_callbacks cb 0x55a52a2c28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:24:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff792ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64e5d0 0x55a52a65d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff792ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64e5d0 0x55a52a65d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54480/0 (socket says 172.21.15.22:54480) 2024-09-19T08:24:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff78bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 0x55a52a65a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff792ffd640 1 -- 172.21.15.22:0/1499081074 learned_addr learned my addr 172.21.15.22:0/1499081074 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff7937fe640 1 --2- 172.21.15.22:0/1499081074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a65d780 0x55a52a65a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff792ffd640 1 -- 172.21.15.22:0/1499081074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 msgr2=0x55a52a65a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff792ffd640 1 --2- 172.21.15.22:0/1499081074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 0x55a52a65a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff792ffd640 1 -- 172.21.15.22:0/1499081074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a65d780 msgr2=0x55a52a65a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff792ffd640 1 --2- 172.21.15.22:0/1499081074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a65d780 0x55a52a65a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.020+0000 7ff792ffd640 1 -- 172.21.15.22:0/1499081074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a52a3253c0 con 0x55a52a64e5d0 2024-09-19T08:24:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.021+0000 7ff792ffd640 1 --2- 172.21.15.22:0/1499081074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64e5d0 0x55a52a65d140 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff784004c30 tx=0x7ff784009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66de1534e10db06f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.021+0000 7ff772ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:24:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.021+0000 7ff772ffd640 1 -- 172.21.15.22:0/1499081074 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784028020 con 0x55a52a64e5d0 2024-09-19T08:24:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.021+0000 7ff79bc69740 1 -- 172.21.15.22:0/1499081074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7840044b0 con 0x55a52a64e5d0 2024-09-19T08:24:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.021+0000 7ff772ffd640 1 -- 172.21.15.22:0/1499081074 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ff784005200 con 0x55a52a64e5d0 2024-09-19T08:24:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.021+0000 7ff772ffd640 1 -- 172.21.15.22:0/1499081074 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff784033050 con 0x55a52a64e5d0 2024-09-19T08:24:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.021+0000 7ff79bc69740 10 client.16983 fetch_fsmap learned FSMap version 35 2024-09-19T08:24:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.021+0000 7ff79bc69740 1 -- 172.21.15.22:0/1499081074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a52a66bee0 con 0x55a52a64e5d0 2024-09-19T08:24:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.021+0000 7ff772ffd640 1 -- 172.21.15.22:0/1499081074 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7840044b0 con 0x55a52a64e5d0 2024-09-19T08:24:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.021+0000 7ff772ffd640 1 -- 172.21.15.22:0/1499081074 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7ff784018070 con 0x55a52a64e5d0 2024-09-19T08:24:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.021+0000 7ff79bc69740 10 client.16983 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:24:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.021+0000 7ff79bc69740 10 client.16983 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:24:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.022+0000 7ff79bc69740 20 client.16983 populate_metadata read hostname 'smithi022' 2024-09-19T08:24:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.022+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1499081074 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a52a66d2e0 0x55a52a66f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.022+0000 7ff79bc69740 4 client.16983 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:24:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.022+0000 7ff79bc69740 1 -- 172.21.15.22:0/1499081074 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a52a5c1860 con 0x55a52a66d2e0 2024-09-19T08:24:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.022+0000 7ff78bfff640 1 --2- 172.21.15.22:0/1499081074 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a52a66d2e0 0x55a52a66f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.022+0000 7ff772ffd640 10 client.16983 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:24:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.022+0000 7ff78bfff640 1 --2- 172.21.15.22:0/1499081074 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a52a66d2e0 0x55a52a66f6d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68919e5abe21c92a server_cookie=4a3084b4c5f00c5f in_seq=0 out_seq=0 2024-09-19T08:24:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.023+0000 7ff772ffd640 1 -- 172.21.15.22:0/1499081074 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a52a5c1860 con 0x55a52a66d2e0 2024-09-19T08:24:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.023+0000 7ff772ffd640 10 client.16983 handle_command_reply: tid=0 2024-09-19T08:24:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.023+0000 7ff79bc69740 1 client.16983 shutdown 2024-09-19T08:24:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.023+0000 7ff79bc69740 1 -- 172.21.15.22:0/1499081074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64e5d0 msgr2=0x55a52a65d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.023+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1499081074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64e5d0 0x55a52a65d140 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff784004c30 tx=0x7ff784009920 comp rx=0 tx=0).stop 2024-09-19T08:24:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.024+0000 7ff79bc69740 1 -- 172.21.15.22:0/1499081074 shutdown_connections 2024-09-19T08:24:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.024+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1499081074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a65d780 0x55a52a65a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.024+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1499081074 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a52a66d2e0 0x55a52a66f6d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.024+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1499081074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 0x55a52a65a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.024+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1499081074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64e5d0 0x55a52a65d140 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.024+0000 7ff772ffd640 0 client.16983 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:24:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.024+0000 7ff79bc69740 1 -- 172.21.15.22:0/1499081074 >> 172.21.15.22:0/1499081074 conn(0x55a52a617310 msgr2=0x55a52a650960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.024+0000 7ff79bc69740 1 -- 172.21.15.22:0/1499081074 shutdown_connections 2024-09-19T08:24:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.024+0000 7ff79bc69740 1 -- 172.21.15.22:0/1499081074 wait complete. 2024-09-19T08:24:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.024+0000 7ff79bc69740 20 client.16983 trim_cache size 0 max 16384 2024-09-19T08:24:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff79bc69740 1 Processor -- start 2024-09-19T08:24:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff79bc69740 1 -- start start 2024-09-19T08:24:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff79bc69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a64e5d0 0x55a52a64ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff79bc69740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 0x55a52a669720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff79bc69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a65d780 0x55a52a674e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff79bc69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941ae050 con 0x55a52a65d780 2024-09-19T08:24:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff79bc69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941ab8f0 con 0x55a52a64e5d0 2024-09-19T08:24:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff79bc69740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941a9760 con 0x55a52a64fb40 2024-09-19T08:24:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff7937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a65d780 0x55a52a674e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff7937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a65d780 0x55a52a674e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54492/0 (socket says 172.21.15.22:54492) 2024-09-19T08:24:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff7937fe640 1 -- 172.21.15.22:0/296824853 learned_addr learned my addr 172.21.15.22:0/296824853 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff78bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 0x55a52a669720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff792ffd640 1 --2- 172.21.15.22:0/296824853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a64e5d0 0x55a52a64ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff7937fe640 1 -- 172.21.15.22:0/296824853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 msgr2=0x55a52a669720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff7937fe640 1 --2- 172.21.15.22:0/296824853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 0x55a52a669720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff7937fe640 1 -- 172.21.15.22:0/296824853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a64e5d0 msgr2=0x55a52a64ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff7937fe640 1 --2- 172.21.15.22:0/296824853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a64e5d0 0x55a52a64ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.037+0000 7ff7937fe640 1 -- 172.21.15.22:0/296824853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a52a65b4e0 con 0x55a52a65d780 2024-09-19T08:24:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.038+0000 7ff7937fe640 1 --2- 172.21.15.22:0/296824853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a65d780 0x55a52a674e60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009ca0 tx=0x7ff78c004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62dfe14ba5026ffd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.038+0000 7ff7897fa640 1 -- 172.21.15.22:0/296824853 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c043050 con 0x55a52a65d780 2024-09-19T08:24:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.038+0000 7ff7897fa640 1 -- 172.21.15.22:0/296824853 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff78c044040 con 0x55a52a65d780 2024-09-19T08:24:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.038+0000 7ff7897fa640 1 -- 172.21.15.22:0/296824853 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c034040 con 0x55a52a65d780 2024-09-19T08:24:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.038+0000 7ff79bc69740 1 -- 172.21.15.22:0/296824853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a65d780 msgr2=0x55a52a674e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.038+0000 7ff79bc69740 1 --2- 172.21.15.22:0/296824853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a65d780 0x55a52a674e60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009ca0 tx=0x7ff78c004a60 comp rx=0 tx=0).stop 2024-09-19T08:24:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 -- 172.21.15.22:0/296824853 shutdown_connections 2024-09-19T08:24:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 --2- 172.21.15.22:0/296824853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a65d780 0x55a52a674e60 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 --2- 172.21.15.22:0/296824853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 0x55a52a669720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 --2- 172.21.15.22:0/296824853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a64e5d0 0x55a52a64ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 -- 172.21.15.22:0/296824853 >> 172.21.15.22:0/296824853 conn(0x55a52a617310 msgr2=0x55a52a659500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 -- 172.21.15.22:0/296824853 shutdown_connections 2024-09-19T08:24:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 -- 172.21.15.22:0/296824853 wait complete. 2024-09-19T08:24:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 -- start start 2024-09-19T08:24:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 Processor -- start 2024-09-19T08:24:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64e5d0 0x55a52a6575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 0x55a52a657ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a65d780 0x55a52a658020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941a9db0 con 0x55a52a64e5d0 2024-09-19T08:24:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941ab220 con 0x55a52a65d780 2024-09-19T08:24:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.039+0000 7ff79bc69740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941aa480 con 0x55a52a64fb40 2024-09-19T08:24:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff79bc69740 10 client.0 _ll_register_callbacks cb 0x7ff78c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:24:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff792ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64e5d0 0x55a52a6575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff78bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 0x55a52a657ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff78bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 0x55a52a657ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42210/0 (socket says 172.21.15.22:42210) 2024-09-19T08:24:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff78bfff640 1 -- 172.21.15.22:0/1007904070 learned_addr learned my addr 172.21.15.22:0/1007904070 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff7937fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a65d780 0x55a52a658020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff792ffd640 1 --2- 172.21.15.22:0/1007904070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64e5d0 0x55a52a6575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54502/0 (socket says 172.21.15.22:54502) 2024-09-19T08:24:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff7937fe640 1 -- 172.21.15.22:0/1007904070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 msgr2=0x55a52a657ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff7937fe640 1 --2- 172.21.15.22:0/1007904070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 0x55a52a657ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff7937fe640 1 -- 172.21.15.22:0/1007904070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64e5d0 msgr2=0x55a52a6575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff7937fe640 1 --2- 172.21.15.22:0/1007904070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64e5d0 0x55a52a6575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff7937fe640 1 -- 172.21.15.22:0/1007904070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a52a6d6f60 con 0x55a52a65d780 2024-09-19T08:24:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff7937fe640 1 --2- 172.21.15.22:0/1007904070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a65d780 0x55a52a658020 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0049d0 tx=0x7ff78c04bac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3eb5b70d1d780f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff772ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:24:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff772ffd640 1 -- 172.21.15.22:0/1007904070 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c043050 con 0x55a52a65d780 2024-09-19T08:24:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff772ffd640 1 -- 172.21.15.22:0/1007904070 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ff78c030440 con 0x55a52a65d780 2024-09-19T08:24:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.040+0000 7ff79bc69740 1 -- 172.21.15.22:0/1007904070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff78c044040 con 0x55a52a65d780 2024-09-19T08:24:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.041+0000 7ff772ffd640 1 -- 172.21.15.22:0/1007904070 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff78c030680 con 0x55a52a65d780 2024-09-19T08:24:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.041+0000 7ff79bc69740 10 client.26477 fetch_fsmap learned FSMap version 35 2024-09-19T08:24:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.041+0000 7ff79bc69740 1 -- 172.21.15.22:0/1007904070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a52a6d8240 con 0x55a52a65d780 2024-09-19T08:24:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.041+0000 7ff772ffd640 1 -- 172.21.15.22:0/1007904070 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff78c044040 con 0x55a52a65d780 2024-09-19T08:24:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.041+0000 7ff772ffd640 1 -- 172.21.15.22:0/1007904070 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7ff78c060680 con 0x55a52a65d780 2024-09-19T08:24:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.041+0000 7ff79bc69740 10 client.26477 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:24:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.041+0000 7ff79bc69740 10 client.26477 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:24:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.041+0000 7ff79bc69740 20 client.26477 populate_metadata read hostname 'smithi022' 2024-09-19T08:24:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.041+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1007904070 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a52a6d88b0 0x55a52a6dacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.041+0000 7ff79bc69740 4 client.26477 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:24:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.041+0000 7ff79bc69740 1 -- 172.21.15.22:0/1007904070 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a52a5bbd70 con 0x55a52a6d88b0 2024-09-19T08:24:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.041+0000 7ff792ffd640 1 --2- 172.21.15.22:0/1007904070 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a52a6d88b0 0x55a52a6dacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.042+0000 7ff772ffd640 10 client.26477 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:24:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.042+0000 7ff792ffd640 1 --2- 172.21.15.22:0/1007904070 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a52a6d88b0 0x55a52a6dacd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=551a4c3b8e9dd997 server_cookie=23f6f19cc30594ed in_seq=0 out_seq=0 2024-09-19T08:24:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.042+0000 7ff772ffd640 1 -- 172.21.15.22:0/1007904070 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a52a5bbd70 con 0x55a52a6d88b0 2024-09-19T08:24:22.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.042+0000 7ff772ffd640 10 client.26477 handle_command_reply: tid=0 2024-09-19T08:24:22.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.042+0000 7ff79bc69740 1 client.26477 shutdown 2024-09-19T08:24:22.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff79bc69740 1 -- 172.21.15.22:0/1007904070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a65d780 msgr2=0x55a52a658020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:22.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1007904070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a65d780 0x55a52a658020 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0049d0 tx=0x7ff78c04bac0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff79bc69740 1 -- 172.21.15.22:0/1007904070 shutdown_connections 2024-09-19T08:24:22.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1007904070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52a65d780 0x55a52a658020 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1007904070 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55a52a6d88b0 0x55a52a6dacd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1007904070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a52a64fb40 0x55a52a657ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff79bc69740 1 --2- 172.21.15.22:0/1007904070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52a64e5d0 0x55a52a6575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff772ffd640 0 client.26477 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:24:22.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff79bc69740 1 -- 172.21.15.22:0/1007904070 >> 172.21.15.22:0/1007904070 conn(0x55a52a617310 msgr2=0x55a52a6494e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:22.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff79bc69740 1 -- 172.21.15.22:0/1007904070 shutdown_connections 2024-09-19T08:24:22.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff79bc69740 1 -- 172.21.15.22:0/1007904070 wait complete. 2024-09-19T08:24:22.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff79bc69740 20 client.26477 trim_cache size 0 max 16384 2024-09-19T08:24:22.055 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:24:22.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff799890640 1 -- 172.21.15.22:0/2672028755 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff7800a3430 msgr2=0x7ff7800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:22.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff799890640 1 --2- 172.21.15.22:0/2672028755 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff7800a3430 0x7ff7800a58f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff78c030040 tx=0x7ff78c02a950 comp rx=0 tx=0).stop 2024-09-19T08:24:22.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff799890640 1 -- 172.21.15.22:0/2672028755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff794103660 msgr2=0x7ff794198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:22.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff799890640 1 --2- 172.21.15.22:0/2672028755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff794103660 0x7ff794198950 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00d740 tx=0x7ff77c00dc00 comp rx=0 tx=0).stop 2024-09-19T08:24:22.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff799890640 1 -- 172.21.15.22:0/2672028755 shutdown_connections 2024-09-19T08:24:22.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff799890640 1 --2- 172.21.15.22:0/2672028755 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff7800a3430 0x7ff7800a58f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff799890640 1 --2- 172.21.15.22:0/2672028755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff79410c170 0x7ff79419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff799890640 1 --2- 172.21.15.22:0/2672028755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff794103660 0x7ff794198950 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff799890640 1 --2- 172.21.15.22:0/2672028755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff794102d40 0x7ff794198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:22.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff799890640 1 -- 172.21.15.22:0/2672028755 >> 172.21.15.22:0/2672028755 conn(0x7ff7940fc7a0 msgr2=0x7ff7940fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:22.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.043+0000 7ff799890640 1 -- 172.21.15.22:0/2672028755 shutdown_connections 2024-09-19T08:24:22.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:22.044+0000 7ff799890640 1 -- 172.21.15.22:0/2672028755 wait complete. 2024-09-19T08:24:22.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:22 smithi157 ceph-mon[35143]: pgmap v289: 97 pgs: 97 active+clean; 1021 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 499 op/s 2024-09-19T08:24:22.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:22 smithi022 ceph-mon[30963]: pgmap v289: 97 pgs: 97 active+clean; 1021 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 499 op/s 2024-09-19T08:24:22.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:22 smithi064 ceph-mon[34900]: pgmap v289: 97 pgs: 97 active+clean; 1021 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 499 op/s 2024-09-19T08:24:22.874 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:24:23.131 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:24:23.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:23 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:24:23.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:23 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:24:23.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:23 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:24:24.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:24 smithi157 ceph-mon[35143]: pgmap v290: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 503 op/s 2024-09-19T08:24:24.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:24 smithi022 ceph-mon[30963]: pgmap v290: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 503 op/s 2024-09-19T08:24:24.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:24 smithi064 ceph-mon[34900]: pgmap v290: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 503 op/s 2024-09-19T08:24:25.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.276+0000 7f0ec4fc3640 1 -- 172.21.15.22:0/2099913310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ec0106f20 msgr2=0x7f0ec010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:25.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.276+0000 7f0ec4fc3640 1 --2- 172.21.15.22:0/2099913310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ec0106f20 0x7f0ec010eaa0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009ca0 tx=0x7f0eb802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.276+0000 7f0ec4fc3640 1 -- 172.21.15.22:0/2099913310 shutdown_connections 2024-09-19T08:24:25.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.276+0000 7f0ec4fc3640 1 --2- 172.21.15.22:0/2099913310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ec0106f20 0x7f0ec010eaa0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.276+0000 7f0ec4fc3640 1 --2- 172.21.15.22:0/2099913310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ec01027a0 0x7f0ec01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.276+0000 7f0ec4fc3640 1 --2- 172.21.15.22:0/2099913310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ec0101df0 0x7f0ec01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.276+0000 7f0ec4fc3640 1 -- 172.21.15.22:0/2099913310 >> 172.21.15.22:0/2099913310 conn(0x7f0ec00fd8f0 msgr2=0x7f0ec00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.276+0000 7f0ec4fc3640 1 -- 172.21.15.22:0/2099913310 shutdown_connections 2024-09-19T08:24:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.276+0000 7f0ec4fc3640 1 -- 172.21.15.22:0/2099913310 wait complete. 2024-09-19T08:24:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ec4fc3640 1 Processor -- start 2024-09-19T08:24:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ec4fc3640 1 -- start start 2024-09-19T08:24:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ec4fc3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ec0101df0 0x7f0ec019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ec4fc3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ec01027a0 0x7f0ec019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ec4fc3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ec0106f20 0x7f0ec019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ec4fc3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0111f80 con 0x7f0ec01027a0 2024-09-19T08:24:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ec4fc3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0111e00 con 0x7f0ec0101df0 2024-09-19T08:24:25.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ec4fc3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0112100 con 0x7f0ec0106f20 2024-09-19T08:24:25.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ebdd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ec01027a0 0x7f0ec019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ebdd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ec01027a0 0x7f0ec019ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54514/0 (socket says 172.21.15.22:54514) 2024-09-19T08:24:25.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ebdd9b640 1 -- 172.21.15.22:0/1445209353 learned_addr learned my addr 172.21.15.22:0/1445209353 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:25.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ebe59c640 1 --2- 172.21.15.22:0/1445209353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ec0101df0 0x7f0ec019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ebed9d640 1 --2- 172.21.15.22:0/1445209353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ec0106f20 0x7f0ec019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ebdd9b640 1 -- 172.21.15.22:0/1445209353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ec0106f20 msgr2=0x7f0ec019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:25.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.277+0000 7f0ebdd9b640 1 --2- 172.21.15.22:0/1445209353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ec0106f20 0x7f0ec019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.278+0000 7f0ebdd9b640 1 -- 172.21.15.22:0/1445209353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ec0101df0 msgr2=0x7f0ec019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:25.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.278+0000 7f0ebdd9b640 1 --2- 172.21.15.22:0/1445209353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ec0101df0 0x7f0ec019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.278+0000 7f0ebdd9b640 1 -- 172.21.15.22:0/1445209353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec019f5a0 con 0x7f0ec01027a0 2024-09-19T08:24:25.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.278+0000 7f0ebdd9b640 1 --2- 172.21.15.22:0/1445209353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ec01027a0 0x7f0ec019ab30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800d950 tx=0x7f0ea800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.278+0000 7f0eb77fe640 1 -- 172.21.15.22:0/1445209353 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8017030 con 0x7f0ec01027a0 2024-09-19T08:24:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.278+0000 7f0ec4fc3640 1 -- 172.21.15.22:0/1445209353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec019f890 con 0x7f0ec01027a0 2024-09-19T08:24:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.278+0000 7f0eb77fe640 1 -- 172.21.15.22:0/1445209353 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea8004020 con 0x7f0ec01027a0 2024-09-19T08:24:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.278+0000 7f0eb77fe640 1 -- 172.21.15.22:0/1445209353 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80047a0 con 0x7f0ec01027a0 2024-09-19T08:24:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.278+0000 7f0ec4fc3640 1 -- 172.21.15.22:0/1445209353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec01a7170 con 0x7f0ec01027a0 2024-09-19T08:24:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.280+0000 7f0eb77fe640 1 -- 172.21.15.22:0/1445209353 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0ea80041d0 con 0x7f0ec01027a0 2024-09-19T08:24:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.281+0000 7f0ec739c740 1 Processor -- start 2024-09-19T08:24:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.281+0000 7f0ec739c740 1 -- start start 2024-09-19T08:24:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.281+0000 7f0ec739c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d489b40 0x55d22d488090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.281+0000 7f0ec739c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d488a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.281+0000 7f0ec739c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d493370 0x55d22d495760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.281+0000 7f0ec739c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d22d3ee2f0 con 0x55d22d489b40 2024-09-19T08:24:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.281+0000 7f0ec739c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d22d395cf0 con 0x55d22d4885d0 2024-09-19T08:24:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.281+0000 7f0ec739c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d22d3ea2b0 con 0x55d22d493370 2024-09-19T08:24:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0ebdd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d493370 0x55d22d495760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d488a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0ebed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d489b40 0x55d22d488090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d488a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:41438/0 (socket says 172.21.15.22:41438) 2024-09-19T08:24:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0ebdd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d493370 0x55d22d495760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42230/0 (socket says 172.21.15.22:42230) 2024-09-19T08:24:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0ebe59c640 1 -- 172.21.15.22:0/724154286 learned_addr learned my addr 172.21.15.22:0/724154286 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0eb77fe640 1 --2- 172.21.15.22:0/1445209353 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0eac0a33b0 0x7f0eac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0ebdd9b640 1 -- 172.21.15.22:0/724154286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 msgr2=0x55d22d488a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0ebdd9b640 1 --2- 172.21.15.22:0/724154286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d488a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0ebdd9b640 1 -- 172.21.15.22:0/724154286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d489b40 msgr2=0x55d22d488090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0ebdd9b640 1 --2- 172.21.15.22:0/724154286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d489b40 0x55d22d488090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0ebdd9b640 1 -- 172.21.15.22:0/724154286 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d22d37a3a0 con 0x55d22d493370 2024-09-19T08:24:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0ebed9d640 1 --2- 172.21.15.22:0/1445209353 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0eac0a33b0 0x7f0eac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0eb77fe640 1 -- 172.21.15.22:0/1445209353 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f0ea80cf610 con 0x7f0ec01027a0 2024-09-19T08:24:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.282+0000 7f0ebdd9b640 1 --2- 172.21.15.22:0/724154286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d493370 0x55d22d495760 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80ccd50 tx=0x7f0ea80afa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2a3c534d14392b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.283+0000 7f0eb4ff9640 1 -- 172.21.15.22:0/724154286 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea809b420 con 0x55d22d493370 2024-09-19T08:24:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.283+0000 7f0eb4ff9640 1 -- 172.21.15.22:0/724154286 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea809bfe0 con 0x55d22d493370 2024-09-19T08:24:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.283+0000 7f0eb4ff9640 1 -- 172.21.15.22:0/724154286 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea809b240 con 0x55d22d493370 2024-09-19T08:24:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.283+0000 7f0ec739c740 1 -- 172.21.15.22:0/724154286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d493370 msgr2=0x55d22d495760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.283+0000 7f0ebed9d640 1 --2- 172.21.15.22:0/1445209353 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0eac0a33b0 0x7f0eac0a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8004af0 tx=0x7f0eb80095c0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.283+0000 7f0ec739c740 1 --2- 172.21.15.22:0/724154286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d493370 0x55d22d495760 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80ccd50 tx=0x7f0ea80afa40 comp rx=0 tx=0).stop 2024-09-19T08:24:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.283+0000 7f0ec739c740 1 -- 172.21.15.22:0/724154286 shutdown_connections 2024-09-19T08:24:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.283+0000 7f0ec739c740 1 --2- 172.21.15.22:0/724154286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d493370 0x55d22d495760 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.283+0000 7f0ec739c740 1 --2- 172.21.15.22:0/724154286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d488a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.283+0000 7f0ec739c740 1 --2- 172.21.15.22:0/724154286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d489b40 0x55d22d488090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.283+0000 7f0ec739c740 1 -- 172.21.15.22:0/724154286 >> 172.21.15.22:0/724154286 conn(0x55d22d451310 msgr2=0x55d22d484c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ec739c740 1 -- 172.21.15.22:0/724154286 shutdown_connections 2024-09-19T08:24:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ec739c740 1 -- 172.21.15.22:0/724154286 wait complete. 2024-09-19T08:24:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ec739c740 1 -- start start 2024-09-19T08:24:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ec739c740 1 Processor -- start 2024-09-19T08:24:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ec739c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d4885d0 0x55d22d492de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ec739c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 0x55d22d490090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ec739c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d493370 0x55d22d4907d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ec739c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d22d496010 con 0x55d22d4885d0 2024-09-19T08:24:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ec739c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d22d3a9060 con 0x55d22d493370 2024-09-19T08:24:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ec739c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d22d496190 con 0x55d22d489b40 2024-09-19T08:24:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ec739c740 10 client.0 _ll_register_callbacks cb 0x55d22d0fc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:24:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d4885d0 0x55d22d492de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d4885d0 0x55d22d492de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54536/0 (socket says 172.21.15.22:54536) 2024-09-19T08:24:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ebe59c640 1 -- 172.21.15.22:0/2749392609 learned_addr learned my addr 172.21.15.22:0/2749392609 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.284+0000 7f0ebdd9b640 1 --2- 172.21.15.22:0/2749392609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 0x55d22d490090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ebed9d640 1 --2- 172.21.15.22:0/2749392609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d493370 0x55d22d4907d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ebe59c640 1 -- 172.21.15.22:0/2749392609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 msgr2=0x55d22d490090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ebe59c640 1 --2- 172.21.15.22:0/2749392609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 0x55d22d490090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ebe59c640 1 -- 172.21.15.22:0/2749392609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d493370 msgr2=0x55d22d4907d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:25.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ebe59c640 1 --2- 172.21.15.22:0/2749392609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d493370 0x55d22d4907d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ebe59c640 1 -- 172.21.15.22:0/2749392609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d22d15f3c0 con 0x55d22d4885d0 2024-09-19T08:24:25.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ebe59c640 1 --2- 172.21.15.22:0/2749392609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d4885d0 0x55d22d492de0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000bae0 tx=0x7f0eb0004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2689c7a1a53740d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:25.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:24:25.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/2749392609 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0011440 con 0x55d22d4885d0 2024-09-19T08:24:25.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/2749392609 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f0eb00115e0 con 0x55d22d4885d0 2024-09-19T08:24:25.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ec739c740 1 -- 172.21.15.22:0/2749392609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea809bfe0 con 0x55d22d4885d0 2024-09-19T08:24:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/2749392609 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0eb00117e0 con 0x55d22d4885d0 2024-09-19T08:24:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ec739c740 10 client.17001 fetch_fsmap learned FSMap version 35 2024-09-19T08:24:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ec739c740 1 -- 172.21.15.22:0/2749392609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d22d4a1ba0 con 0x55d22d4885d0 2024-09-19T08:24:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.285+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/2749392609 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea809bfe0 con 0x55d22d4885d0 2024-09-19T08:24:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.286+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/2749392609 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f0eb001ba70 con 0x55d22d4885d0 2024-09-19T08:24:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.286+0000 7f0ec739c740 10 client.17001 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:24:25.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.286+0000 7f0ec739c740 10 client.17001 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:24:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.286+0000 7f0ec739c740 20 client.17001 populate_metadata read hostname 'smithi022' 2024-09-19T08:24:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.286+0000 7f0ec739c740 1 --2- 172.21.15.22:0/2749392609 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d22d4a30b0 0x55d22d4a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.286+0000 7f0ec739c740 4 client.17001 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:24:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.286+0000 7f0ec739c740 1 -- 172.21.15.22:0/2749392609 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d22d3fb860 con 0x55d22d4a30b0 2024-09-19T08:24:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.286+0000 7f0ebdd9b640 1 --2- 172.21.15.22:0/2749392609 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d22d4a30b0 0x55d22d4a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.286+0000 7f0ea5ffb640 10 client.17001 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:24:25.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.286+0000 7f0ebdd9b640 1 --2- 172.21.15.22:0/2749392609 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d22d4a30b0 0x55d22d4a5500 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36d3b9f1315b0fd9 server_cookie=e81caf4145fcac03 in_seq=0 out_seq=0 2024-09-19T08:24:25.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.287+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/2749392609 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d22d3fb860 con 0x55d22d4a30b0 2024-09-19T08:24:25.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.287+0000 7f0ea5ffb640 10 client.17001 handle_command_reply: tid=0 2024-09-19T08:24:25.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.287+0000 7f0ec739c740 1 client.17001 shutdown 2024-09-19T08:24:25.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.287+0000 7f0ec739c740 1 -- 172.21.15.22:0/2749392609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d4885d0 msgr2=0x55d22d492de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:25.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.287+0000 7f0ec739c740 1 --2- 172.21.15.22:0/2749392609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d4885d0 0x55d22d492de0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000bae0 tx=0x7f0eb0004560 comp rx=0 tx=0).stop 2024-09-19T08:24:25.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.287+0000 7f0ec739c740 1 -- 172.21.15.22:0/2749392609 shutdown_connections 2024-09-19T08:24:25.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.287+0000 7f0ec739c740 1 --2- 172.21.15.22:0/2749392609 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d22d4a30b0 0x55d22d4a5500 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.287+0000 7f0ec739c740 1 --2- 172.21.15.22:0/2749392609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d493370 0x55d22d4907d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.287+0000 7f0ec739c740 1 --2- 172.21.15.22:0/2749392609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 0x55d22d490090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.287+0000 7f0ec739c740 1 --2- 172.21.15.22:0/2749392609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d4885d0 0x55d22d492de0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.287+0000 7f0ec739c740 1 -- 172.21.15.22:0/2749392609 >> 172.21.15.22:0/2749392609 conn(0x55d22d451310 msgr2=0x55d22d48a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:25.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.287+0000 7f0ea5ffb640 0 client.17001 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:24:25.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.288+0000 7f0ec739c740 1 -- 172.21.15.22:0/2749392609 shutdown_connections 2024-09-19T08:24:25.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.288+0000 7f0ec739c740 1 -- 172.21.15.22:0/2749392609 wait complete. 2024-09-19T08:24:25.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.288+0000 7f0ec739c740 20 client.17001 trim_cache size 0 max 16384 2024-09-19T08:24:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.300+0000 7f0ec739c740 1 Processor -- start 2024-09-19T08:24:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ec739c740 1 -- start start 2024-09-19T08:24:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ec739c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d49f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ec739c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 0x55d22d48b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ec739c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d493370 0x55d22d48b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ec739c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01b2470 con 0x55d22d493370 2024-09-19T08:24:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ec739c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01afd10 con 0x55d22d4885d0 2024-09-19T08:24:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ec739c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01adb80 con 0x55d22d489b40 2024-09-19T08:24:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ebed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d493370 0x55d22d48b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ebdd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 0x55d22d48b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ebed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d493370 0x55d22d48b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54544/0 (socket says 172.21.15.22:54544) 2024-09-19T08:24:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ebed9d640 1 -- 172.21.15.22:0/1782429005 learned_addr learned my addr 172.21.15.22:0/1782429005 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ebe59c640 1 --2- 172.21.15.22:0/1782429005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d49f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ebdd9b640 1 -- 172.21.15.22:0/1782429005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 msgr2=0x55d22d49f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ebdd9b640 1 --2- 172.21.15.22:0/1782429005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d49f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ebdd9b640 1 -- 172.21.15.22:0/1782429005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d493370 msgr2=0x55d22d48b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ebdd9b640 1 --2- 172.21.15.22:0/1782429005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d493370 0x55d22d48b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ebdd9b640 1 -- 172.21.15.22:0/1782429005 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d22d490b40 con 0x55d22d489b40 2024-09-19T08:24:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ebed9d640 1 --2- 172.21.15.22:0/1782429005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d493370 0x55d22d48b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.301+0000 7f0ebdd9b640 1 --2- 172.21.15.22:0/1782429005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 0x55d22d48b120 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8014000 tx=0x7f0ea80d5550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c0e11c04f2b1c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.302+0000 7f0ea7fff640 1 -- 172.21.15.22:0/1782429005 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea809b240 con 0x55d22d489b40 2024-09-19T08:24:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.302+0000 7f0ea7fff640 1 -- 172.21.15.22:0/1782429005 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea809bfe0 con 0x55d22d489b40 2024-09-19T08:24:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.302+0000 7f0ea7fff640 1 -- 172.21.15.22:0/1782429005 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea809b420 con 0x55d22d489b40 2024-09-19T08:24:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.302+0000 7f0ec739c740 1 -- 172.21.15.22:0/1782429005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 msgr2=0x55d22d48b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.302+0000 7f0ec739c740 1 --2- 172.21.15.22:0/1782429005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 0x55d22d48b120 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8014000 tx=0x7f0ea80d5550 comp rx=0 tx=0).stop 2024-09-19T08:24:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.302+0000 7f0ec739c740 1 -- 172.21.15.22:0/1782429005 shutdown_connections 2024-09-19T08:24:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.302+0000 7f0ec739c740 1 --2- 172.21.15.22:0/1782429005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d493370 0x55d22d48b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.302+0000 7f0ec739c740 1 --2- 172.21.15.22:0/1782429005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 0x55d22d48b120 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.302+0000 7f0ec739c740 1 --2- 172.21.15.22:0/1782429005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d49f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.302+0000 7f0ec739c740 1 -- 172.21.15.22:0/1782429005 >> 172.21.15.22:0/1782429005 conn(0x55d22d451310 msgr2=0x55d22d48f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.302+0000 7f0ec739c740 1 -- 172.21.15.22:0/1782429005 shutdown_connections 2024-09-19T08:24:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.302+0000 7f0ec739c740 1 -- 172.21.15.22:0/1782429005 wait complete. 2024-09-19T08:24:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ec739c740 1 -- start start 2024-09-19T08:24:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ec739c740 1 Processor -- start 2024-09-19T08:24:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ec739c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d49b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ec739c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 0x55d22d49b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ec739c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d493370 0x55d22d48c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ec739c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01ae1d0 con 0x55d22d493370 2024-09-19T08:24:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ec739c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01af640 con 0x55d22d4885d0 2024-09-19T08:24:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ec739c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01ae8a0 con 0x55d22d489b40 2024-09-19T08:24:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ec739c740 10 client.0 _ll_register_callbacks cb 0x7f0eb0004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:24:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ebed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d493370 0x55d22d48c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ebdd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 0x55d22d49b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d49b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ebed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d493370 0x55d22d48c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54548/0 (socket says 172.21.15.22:54548) 2024-09-19T08:24:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d49b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:41468/0 (socket says 172.21.15.22:41468) 2024-09-19T08:24:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.303+0000 7f0ebed9d640 1 -- 172.21.15.22:0/4049430467 learned_addr learned my addr 172.21.15.22:0/4049430467 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ebed9d640 1 -- 172.21.15.22:0/4049430467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 msgr2=0x55d22d49b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ebed9d640 1 --2- 172.21.15.22:0/4049430467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 0x55d22d49b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ebed9d640 1 -- 172.21.15.22:0/4049430467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 msgr2=0x55d22d49b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ebed9d640 1 --2- 172.21.15.22:0/4049430467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d49b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ebed9d640 1 -- 172.21.15.22:0/4049430467 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d22d48cc50 con 0x55d22d493370 2024-09-19T08:24:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ebdd9b640 1 --2- 172.21.15.22:0/4049430467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 0x55d22d49b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ebe59c640 1 --2- 172.21.15.22:0/4049430467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d49b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ebed9d640 1 --2- 172.21.15.22:0/4049430467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d493370 0x55d22d48c6e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8035000 tx=0x7f0eb8042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d3aef575b9f69f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:24:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/4049430467 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb802fea0 con 0x55d22d493370 2024-09-19T08:24:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/4049430467 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f0eb80305c0 con 0x55d22d493370 2024-09-19T08:24:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ec739c740 1 -- 172.21.15.22:0/4049430467 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea809bfe0 con 0x55d22d493370 2024-09-19T08:24:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/4049430467 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0eb80307c0 con 0x55d22d493370 2024-09-19T08:24:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ec739c740 10 client.17007 fetch_fsmap learned FSMap version 35 2024-09-19T08:24:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ec739c740 1 -- 172.21.15.22:0/4049430467 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d22d50d600 con 0x55d22d493370 2024-09-19T08:24:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.304+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/4049430467 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea809bfe0 con 0x55d22d493370 2024-09-19T08:24:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.305+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/4049430467 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f0eb8054a60 con 0x55d22d493370 2024-09-19T08:24:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.305+0000 7f0ec739c740 10 client.17007 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:24:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.305+0000 7f0ec739c740 10 client.17007 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:24:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.305+0000 7f0ec739c740 20 client.17007 populate_metadata read hostname 'smithi022' 2024-09-19T08:24:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.305+0000 7f0ec739c740 1 --2- 172.21.15.22:0/4049430467 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d22d50ddb0 0x55d22d5101d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.305+0000 7f0ec739c740 4 client.17007 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:24:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.305+0000 7f0ec739c740 1 -- 172.21.15.22:0/4049430467 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d22d3f5d70 con 0x55d22d50ddb0 2024-09-19T08:24:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.305+0000 7f0ebe59c640 1 --2- 172.21.15.22:0/4049430467 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d22d50ddb0 0x55d22d5101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.305+0000 7f0ea5ffb640 10 client.17007 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:24:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.305+0000 7f0ebe59c640 1 --2- 172.21.15.22:0/4049430467 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d22d50ddb0 0x55d22d5101d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=daf6f16fa8f18b78 server_cookie=70dc06a83703a092 in_seq=0 out_seq=0 2024-09-19T08:24:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ea5ffb640 1 -- 172.21.15.22:0/4049430467 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d22d3f5d70 con 0x55d22d50ddb0 2024-09-19T08:24:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ea5ffb640 10 client.17007 handle_command_reply: tid=0 2024-09-19T08:24:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec739c740 1 client.17007 shutdown 2024-09-19T08:24:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec739c740 1 -- 172.21.15.22:0/4049430467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d493370 msgr2=0x55d22d48c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec739c740 1 --2- 172.21.15.22:0/4049430467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d493370 0x55d22d48c6e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8035000 tx=0x7f0eb8042000 comp rx=0 tx=0).stop 2024-09-19T08:24:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec739c740 1 -- 172.21.15.22:0/4049430467 shutdown_connections 2024-09-19T08:24:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec739c740 1 --2- 172.21.15.22:0/4049430467 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d22d50ddb0 0x55d22d5101d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec739c740 1 --2- 172.21.15.22:0/4049430467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d22d493370 0x55d22d48c6e0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec739c740 1 --2- 172.21.15.22:0/4049430467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d22d489b40 0x55d22d49b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ea5ffb640 0 client.17007 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:24:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec739c740 1 --2- 172.21.15.22:0/4049430467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d22d4885d0 0x55d22d49b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec739c740 1 -- 172.21.15.22:0/4049430467 >> 172.21.15.22:0/4049430467 conn(0x55d22d451310 msgr2=0x55d22d4834e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec739c740 1 -- 172.21.15.22:0/4049430467 shutdown_connections 2024-09-19T08:24:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec739c740 1 -- 172.21.15.22:0/4049430467 wait complete. 2024-09-19T08:24:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec739c740 20 client.17007 trim_cache size 0 max 16384 2024-09-19T08:24:25.323 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:24:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec4fc3640 1 -- 172.21.15.22:0/1445209353 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0eac0a33b0 msgr2=0x7f0eac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec4fc3640 1 --2- 172.21.15.22:0/1445209353 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0eac0a33b0 0x7f0eac0a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8004af0 tx=0x7f0eb80095c0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec4fc3640 1 -- 172.21.15.22:0/1445209353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ec01027a0 msgr2=0x7f0ec019ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.306+0000 7f0ec4fc3640 1 --2- 172.21.15.22:0/1445209353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ec01027a0 0x7f0ec019ab30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800d950 tx=0x7f0ea800de10 comp rx=0 tx=0).stop 2024-09-19T08:24:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.307+0000 7f0ec4fc3640 1 -- 172.21.15.22:0/1445209353 shutdown_connections 2024-09-19T08:24:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.307+0000 7f0ec4fc3640 1 --2- 172.21.15.22:0/1445209353 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0eac0a33b0 0x7f0eac0a5870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.307+0000 7f0ec4fc3640 1 --2- 172.21.15.22:0/1445209353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ec0106f20 0x7f0ec019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.307+0000 7f0ec4fc3640 1 --2- 172.21.15.22:0/1445209353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ec01027a0 0x7f0ec019ab30 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.307+0000 7f0ec4fc3640 1 --2- 172.21.15.22:0/1445209353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ec0101df0 0x7f0ec019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:25.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.307+0000 7f0ec4fc3640 1 -- 172.21.15.22:0/1445209353 >> 172.21.15.22:0/1445209353 conn(0x7f0ec00fd8f0 msgr2=0x7f0ec00ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:25.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.307+0000 7f0ec4fc3640 1 -- 172.21.15.22:0/1445209353 shutdown_connections 2024-09-19T08:24:25.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:25.307+0000 7f0ec4fc3640 1 -- 172.21.15.22:0/1445209353 wait complete. 2024-09-19T08:24:25.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:25 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:24:25.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:25 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:24:25.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:25 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:24:25.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:25 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:24:25.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:25 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:24:25.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:25 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:24:25.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:25 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:24:25.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:25 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:24:25.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:25 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:24:26.515 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:24:26.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:26 smithi022 ceph-mon[30963]: pgmap v291: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 503 op/s 2024-09-19T08:24:26.778 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:24:27.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:26 smithi157 ceph-mon[35143]: pgmap v291: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 503 op/s 2024-09-19T08:24:27.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:26 smithi064 ceph-mon[34900]: pgmap v291: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 503 op/s 2024-09-19T08:24:27.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.276+0000 7fbf86502640 1 -- 172.21.15.22:0/2776328201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf80106f20 msgr2=0x7fbf8010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.276+0000 7fbf86502640 1 --2- 172.21.15.22:0/2776328201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf80106f20 0x7fbf8010eaa0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009ca0 tx=0x7fbf7802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.276+0000 7fbf86502640 1 -- 172.21.15.22:0/2776328201 shutdown_connections 2024-09-19T08:24:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.276+0000 7fbf86502640 1 --2- 172.21.15.22:0/2776328201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf80106f20 0x7fbf8010eaa0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.276+0000 7fbf86502640 1 --2- 172.21.15.22:0/2776328201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf801027a0 0x7fbf801067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.276+0000 7fbf86502640 1 --2- 172.21.15.22:0/2776328201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf80101df0 0x7fbf801021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.276+0000 7fbf86502640 1 -- 172.21.15.22:0/2776328201 >> 172.21.15.22:0/2776328201 conn(0x7fbf800fd8f0 msgr2=0x7fbf800ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.276+0000 7fbf86502640 1 -- 172.21.15.22:0/2776328201 shutdown_connections 2024-09-19T08:24:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.276+0000 7fbf86502640 1 -- 172.21.15.22:0/2776328201 wait complete. 2024-09-19T08:24:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.277+0000 7fbf86502640 1 Processor -- start 2024-09-19T08:24:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.277+0000 7fbf86502640 1 -- start start 2024-09-19T08:24:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.277+0000 7fbf86502640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf80101df0 0x7fbf80198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.277+0000 7fbf86502640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf801027a0 0x7fbf801989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.277+0000 7fbf86502640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf80106f20 0x7fbf8019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.277+0000 7fbf86502640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80111f80 con 0x7fbf801027a0 2024-09-19T08:24:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.277+0000 7fbf86502640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80111e00 con 0x7fbf80101df0 2024-09-19T08:24:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.277+0000 7fbf86502640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80112100 con 0x7fbf80106f20 2024-09-19T08:24:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.277+0000 7fbf77fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf801027a0 0x7fbf801989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.277+0000 7fbf77fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf801027a0 0x7fbf801989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54570/0 (socket says 172.21.15.22:54570) 2024-09-19T08:24:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.277+0000 7fbf77fff640 1 -- 172.21.15.22:0/2748153619 learned_addr learned my addr 172.21.15.22:0/2748153619 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.277+0000 7fbf84a9f640 1 --2- 172.21.15.22:0/2748153619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf80106f20 0x7fbf8019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.277+0000 7fbf7ffff640 1 --2- 172.21.15.22:0/2748153619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf80101df0 0x7fbf80198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.278+0000 7fbf77fff640 1 -- 172.21.15.22:0/2748153619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf80106f20 msgr2=0x7fbf8019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.278+0000 7fbf77fff640 1 --2- 172.21.15.22:0/2748153619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf80106f20 0x7fbf8019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.278+0000 7fbf77fff640 1 -- 172.21.15.22:0/2748153619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf80101df0 msgr2=0x7fbf80198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.278+0000 7fbf77fff640 1 --2- 172.21.15.22:0/2748153619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf80101df0 0x7fbf80198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.278+0000 7fbf77fff640 1 -- 172.21.15.22:0/2748153619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8019d430 con 0x7fbf801027a0 2024-09-19T08:24:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.278+0000 7fbf77fff640 1 --2- 172.21.15.22:0/2748153619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf801027a0 0x7fbf801989c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800e9f0 tx=0x7fbf6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.278+0000 7fbf7dffb640 1 -- 172.21.15.22:0/2748153619 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6800cd80 con 0x7fbf801027a0 2024-09-19T08:24:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.278+0000 7fbf7dffb640 1 -- 172.21.15.22:0/2748153619 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf68010910 con 0x7fbf801027a0 2024-09-19T08:24:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.278+0000 7fbf86502640 1 -- 172.21.15.22:0/2748153619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8019d720 con 0x7fbf801027a0 2024-09-19T08:24:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.278+0000 7fbf7dffb640 1 -- 172.21.15.22:0/2748153619 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68018990 con 0x7fbf801027a0 2024-09-19T08:24:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.278+0000 7fbf86502640 1 -- 172.21.15.22:0/2748153619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf80074420 con 0x7fbf801027a0 2024-09-19T08:24:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.280+0000 7fbf7dffb640 1 -- 172.21.15.22:0/2748153619 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbf68010ac0 con 0x7fbf801027a0 2024-09-19T08:24:27.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf888db740 1 Processor -- start 2024-09-19T08:24:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf888db740 1 -- start start 2024-09-19T08:24:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf888db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553eceb40 0x55b553ecd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf888db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 0x55b553ecda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf888db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553ed8370 0x55b553eda760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf888db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b553e332f0 con 0x55b553eceb40 2024-09-19T08:24:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf888db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b553ddacf0 con 0x55b553ecd5d0 2024-09-19T08:24:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf888db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b553e2f2b0 con 0x55b553ed8370 2024-09-19T08:24:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf84a9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553eceb40 0x55b553ecd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf84a9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553eceb40 0x55b553ecd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54580/0 (socket says 172.21.15.22:54580) 2024-09-19T08:24:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf84a9f640 1 -- 172.21.15.22:0/3328511137 learned_addr learned my addr 172.21.15.22:0/3328511137 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf7ffff640 1 --2- 172.21.15.22:0/3328511137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 0x55b553ecda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf77fff640 1 --2- 172.21.15.22:0/3328511137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553ed8370 0x55b553eda760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf84a9f640 1 -- 172.21.15.22:0/3328511137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553ed8370 msgr2=0x55b553eda760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:27.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf84a9f640 1 --2- 172.21.15.22:0/3328511137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553ed8370 0x55b553eda760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.281+0000 7fbf7dffb640 1 --2- 172.21.15.22:0/2748153619 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbf6c0a33b0 0x7fbf6c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.282+0000 7fbf84a9f640 1 -- 172.21.15.22:0/3328511137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 msgr2=0x55b553ecda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:27.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.282+0000 7fbf84a9f640 1 --2- 172.21.15.22:0/3328511137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 0x55b553ecda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.282+0000 7fbf84a9f640 1 -- 172.21.15.22:0/3328511137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b553dbf3a0 con 0x55b553eceb40 2024-09-19T08:24:27.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.282+0000 7fbf7ffff640 1 --2- 172.21.15.22:0/3328511137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 0x55b553ecda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:27.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.282+0000 7fbf84a9f640 1 --2- 172.21.15.22:0/2748153619 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbf6c0a33b0 0x7fbf6c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.282+0000 7fbf7dffb640 1 -- 172.21.15.22:0/2748153619 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fbf680d00d0 con 0x7fbf801027a0 2024-09-19T08:24:27.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.282+0000 7fbf84a9f640 1 --2- 172.21.15.22:0/3328511137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553eceb40 0x55b553ecd090 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009ca0 tx=0x7fbf78005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931fe2ad5838b11c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:27.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.282+0000 7fbf767fc640 1 -- 172.21.15.22:0/3328511137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78003970 con 0x55b553eceb40 2024-09-19T08:24:27.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.283+0000 7fbf767fc640 1 -- 172.21.15.22:0/3328511137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf78036040 con 0x55b553eceb40 2024-09-19T08:24:27.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.283+0000 7fbf767fc640 1 -- 172.21.15.22:0/3328511137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7802f610 con 0x55b553eceb40 2024-09-19T08:24:27.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.284+0000 7fbf888db740 1 -- 172.21.15.22:0/3328511137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553eceb40 msgr2=0x55b553ecd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:27.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.284+0000 7fbf84a9f640 1 --2- 172.21.15.22:0/2748153619 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbf6c0a33b0 0x7fbf6c0a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbf7804a000 tx=0x7fbf7804b000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:27.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.284+0000 7fbf888db740 1 --2- 172.21.15.22:0/3328511137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553eceb40 0x55b553ecd090 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009ca0 tx=0x7fbf78005f10 comp rx=0 tx=0).stop 2024-09-19T08:24:27.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.284+0000 7fbf888db740 1 -- 172.21.15.22:0/3328511137 shutdown_connections 2024-09-19T08:24:27.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.284+0000 7fbf888db740 1 --2- 172.21.15.22:0/3328511137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553ed8370 0x55b553eda760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.284+0000 7fbf888db740 1 --2- 172.21.15.22:0/3328511137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 0x55b553ecda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.284+0000 7fbf888db740 1 --2- 172.21.15.22:0/3328511137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553eceb40 0x55b553ecd090 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.284+0000 7fbf888db740 1 -- 172.21.15.22:0/3328511137 >> 172.21.15.22:0/3328511137 conn(0x55b553e96310 msgr2=0x55b553ec9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:27.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.284+0000 7fbf888db740 1 -- 172.21.15.22:0/3328511137 shutdown_connections 2024-09-19T08:24:27.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.284+0000 7fbf888db740 1 -- 172.21.15.22:0/3328511137 wait complete. 2024-09-19T08:24:27.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.284+0000 7fbf888db740 1 -- start start 2024-09-19T08:24:27.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf888db740 1 Processor -- start 2024-09-19T08:24:27.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf888db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 0x55b553ed7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf888db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553eceb40 0x55b553ed5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf888db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553ed8370 0x55b553ed5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf888db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b553edaef0 con 0x55b553ed8370 2024-09-19T08:24:27.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf888db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b553dee060 con 0x55b553ecd5d0 2024-09-19T08:24:27.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf888db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b553edb070 con 0x55b553eceb40 2024-09-19T08:24:27.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf84a9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553ed8370 0x55b553ed5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf77fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553eceb40 0x55b553ed5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf84a9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553ed8370 0x55b553ed5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54594/0 (socket says 172.21.15.22:54594) 2024-09-19T08:24:27.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf888db740 10 client.0 _ll_register_callbacks cb 0x55b553b418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:24:27.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf84a9f640 1 -- 172.21.15.22:0/75972644 learned_addr learned my addr 172.21.15.22:0/75972644 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:27.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf77fff640 1 --2- 172.21.15.22:0/75972644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553eceb40 0x55b553ed5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42272/0 (socket says 172.21.15.22:42272) 2024-09-19T08:24:27.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf7ffff640 1 --2- 172.21.15.22:0/75972644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 0x55b553ed7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf84a9f640 1 -- 172.21.15.22:0/75972644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553eceb40 msgr2=0x55b553ed5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:27.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf84a9f640 1 --2- 172.21.15.22:0/75972644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553eceb40 0x55b553ed5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf84a9f640 1 -- 172.21.15.22:0/75972644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 msgr2=0x55b553ed7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:27.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf84a9f640 1 --2- 172.21.15.22:0/75972644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 0x55b553ed7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf84a9f640 1 -- 172.21.15.22:0/75972644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b553ba43c0 con 0x55b553ed8370 2024-09-19T08:24:27.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.285+0000 7fbf77fff640 1 --2- 172.21.15.22:0/75972644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553eceb40 0x55b553ed5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:27.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.286+0000 7fbf84a9f640 1 --2- 172.21.15.22:0/75972644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553ed8370 0x55b553ed5650 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbf7802a900 tx=0x7fbf7802b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad0d36ddc923b35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:27.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.286+0000 7fbf5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:24:27.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.286+0000 7fbf5f7fe640 1 -- 172.21.15.22:0/75972644 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7805e060 con 0x55b553ed8370 2024-09-19T08:24:27.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.286+0000 7fbf5f7fe640 1 -- 172.21.15.22:0/75972644 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fbf78004740 con 0x55b553ed8370 2024-09-19T08:24:27.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.286+0000 7fbf888db740 1 -- 172.21.15.22:0/75972644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf78036040 con 0x55b553ed8370 2024-09-19T08:24:27.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.286+0000 7fbf5f7fe640 1 -- 172.21.15.22:0/75972644 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbf7802f610 con 0x55b553ed8370 2024-09-19T08:24:27.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.286+0000 7fbf888db740 10 client.17019 fetch_fsmap learned FSMap version 35 2024-09-19T08:24:27.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.286+0000 7fbf888db740 1 -- 172.21.15.22:0/75972644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b553edaef0 con 0x55b553ed8370 2024-09-19T08:24:27.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.286+0000 7fbf5f7fe640 1 -- 172.21.15.22:0/75972644 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf78036040 con 0x55b553ed8370 2024-09-19T08:24:27.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.286+0000 7fbf5f7fe640 1 -- 172.21.15.22:0/75972644 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fbf78030780 con 0x55b553ed8370 2024-09-19T08:24:27.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.286+0000 7fbf888db740 10 client.17019 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:24:27.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.286+0000 7fbf888db740 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:24:27.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.286+0000 7fbf888db740 20 client.17019 populate_metadata read hostname 'smithi022' 2024-09-19T08:24:27.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.287+0000 7fbf888db740 1 --2- 172.21.15.22:0/75972644 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b553ee7b90 0x55b553ee9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.287+0000 7fbf888db740 4 client.17019 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:24:27.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.287+0000 7fbf888db740 1 -- 172.21.15.22:0/75972644 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b553e40860 con 0x55b553ee7b90 2024-09-19T08:24:27.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.287+0000 7fbf7ffff640 1 --2- 172.21.15.22:0/75972644 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b553ee7b90 0x55b553ee9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.287+0000 7fbf7ffff640 1 --2- 172.21.15.22:0/75972644 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b553ee7b90 0x55b553ee9f80 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61b79f9935ea632 server_cookie=2890bfd7cc0367b3 in_seq=0 out_seq=0 2024-09-19T08:24:27.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.287+0000 7fbf5f7fe640 10 client.17019 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:27.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf5f7fe640 1 -- 172.21.15.22:0/75972644 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b553e40860 con 0x55b553ee7b90 2024-09-19T08:24:27.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf5f7fe640 10 client.17019 handle_command_reply: tid=0 2024-09-19T08:24:27.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf888db740 1 client.17019 shutdown 2024-09-19T08:24:27.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf888db740 1 -- 172.21.15.22:0/75972644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553ed8370 msgr2=0x55b553ed5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:27.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf888db740 1 --2- 172.21.15.22:0/75972644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553ed8370 0x55b553ed5650 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbf7802a900 tx=0x7fbf7802b040 comp rx=0 tx=0).stop 2024-09-19T08:24:27.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf888db740 1 -- 172.21.15.22:0/75972644 shutdown_connections 2024-09-19T08:24:27.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf888db740 1 --2- 172.21.15.22:0/75972644 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b553ee7b90 0x55b553ee9f80 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf888db740 1 --2- 172.21.15.22:0/75972644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553ed8370 0x55b553ed5650 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf888db740 1 --2- 172.21.15.22:0/75972644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553eceb40 0x55b553ed5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf888db740 1 --2- 172.21.15.22:0/75972644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 0x55b553ed7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf888db740 1 -- 172.21.15.22:0/75972644 >> 172.21.15.22:0/75972644 conn(0x55b553e96310 msgr2=0x55b553ecf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:27.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf5f7fe640 0 client.17019 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:27.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf888db740 1 -- 172.21.15.22:0/75972644 shutdown_connections 2024-09-19T08:24:27.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf888db740 1 -- 172.21.15.22:0/75972644 wait complete. 2024-09-19T08:24:27.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.288+0000 7fbf888db740 20 client.17019 trim_cache size 0 max 16384 2024-09-19T08:24:27.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.302+0000 7fbf888db740 1 Processor -- start 2024-09-19T08:24:27.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.302+0000 7fbf888db740 1 -- start start 2024-09-19T08:24:27.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.302+0000 7fbf888db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 0x55b553ee4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.302+0000 7fbf888db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553eceb40 0x55b553ed4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.302+0000 7fbf888db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553ed8370 0x55b553ed0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.302+0000 7fbf888db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801b22b0 con 0x55b553eceb40 2024-09-19T08:24:27.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.302+0000 7fbf888db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801afb50 con 0x55b553ecd5d0 2024-09-19T08:24:27.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.302+0000 7fbf888db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801ad9c0 con 0x55b553ed8370 2024-09-19T08:24:27.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf77fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553eceb40 0x55b553ed4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf84a9f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553ed8370 0x55b553ed0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf7ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 0x55b553ee4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf84a9f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553ed8370 0x55b553ed0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42278/0 (socket says 172.21.15.22:42278) 2024-09-19T08:24:27.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf7ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 0x55b553ee4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:41538/0 (socket says 172.21.15.22:41538) 2024-09-19T08:24:27.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf77fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553eceb40 0x55b553ed4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54602/0 (socket says 172.21.15.22:54602) 2024-09-19T08:24:27.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf84a9f640 1 -- 172.21.15.22:0/3148586397 learned_addr learned my addr 172.21.15.22:0/3148586397 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:27.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf84a9f640 1 -- 172.21.15.22:0/3148586397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 msgr2=0x55b553ee4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:27.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf84a9f640 1 --2- 172.21.15.22:0/3148586397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 0x55b553ee4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf84a9f640 1 -- 172.21.15.22:0/3148586397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553eceb40 msgr2=0x55b553ed4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:27.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf84a9f640 1 --2- 172.21.15.22:0/3148586397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553eceb40 0x55b553ed4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf84a9f640 1 -- 172.21.15.22:0/3148586397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b553ece3a0 con 0x55b553ed8370 2024-09-19T08:24:27.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf77fff640 1 --2- 172.21.15.22:0/3148586397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553eceb40 0x55b553ed4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:27.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf84a9f640 1 --2- 172.21.15.22:0/3148586397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553ed8370 0x55b553ed0120 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbf78000f80 tx=0x7fbf78004a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54d5a9f85bbea8f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:27.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf75ffb640 1 -- 172.21.15.22:0/3148586397 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7805e040 con 0x55b553ed8370 2024-09-19T08:24:27.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf75ffb640 1 -- 172.21.15.22:0/3148586397 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf78036030 con 0x55b553ed8370 2024-09-19T08:24:27.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.303+0000 7fbf75ffb640 1 -- 172.21.15.22:0/3148586397 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78067080 con 0x55b553ed8370 2024-09-19T08:24:27.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.304+0000 7fbf888db740 1 -- 172.21.15.22:0/3148586397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553ed8370 msgr2=0x55b553ed0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:27.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.304+0000 7fbf888db740 1 --2- 172.21.15.22:0/3148586397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553ed8370 0x55b553ed0120 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbf78000f80 tx=0x7fbf78004a80 comp rx=0 tx=0).stop 2024-09-19T08:24:27.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.304+0000 7fbf888db740 1 -- 172.21.15.22:0/3148586397 shutdown_connections 2024-09-19T08:24:27.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.304+0000 7fbf888db740 1 --2- 172.21.15.22:0/3148586397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553ed8370 0x55b553ed0120 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.304+0000 7fbf888db740 1 --2- 172.21.15.22:0/3148586397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553eceb40 0x55b553ed4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.304+0000 7fbf888db740 1 --2- 172.21.15.22:0/3148586397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ecd5d0 0x55b553ee4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.304+0000 7fbf888db740 1 -- 172.21.15.22:0/3148586397 >> 172.21.15.22:0/3148586397 conn(0x55b553e96310 msgr2=0x55b553ed3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:27.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.304+0000 7fbf888db740 1 -- 172.21.15.22:0/3148586397 shutdown_connections 2024-09-19T08:24:27.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.304+0000 7fbf888db740 1 -- 172.21.15.22:0/3148586397 wait complete. 2024-09-19T08:24:27.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.305+0000 7fbf888db740 1 -- start start 2024-09-19T08:24:27.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.305+0000 7fbf888db740 1 Processor -- start 2024-09-19T08:24:27.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.305+0000 7fbf888db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553ecd5d0 0x55b553ee0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.305+0000 7fbf888db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553eceb40 0x55b553ee0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.305+0000 7fbf888db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ed8370 0x55b553ed1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.305+0000 7fbf888db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801ae010 con 0x55b553ecd5d0 2024-09-19T08:24:27.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.305+0000 7fbf888db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801af480 con 0x55b553ed8370 2024-09-19T08:24:27.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.305+0000 7fbf888db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801ae6e0 con 0x55b553eceb40 2024-09-19T08:24:27.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.305+0000 7fbf888db740 10 client.0 _ll_register_callbacks cb 0x7fbf78049020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:24:27.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.305+0000 7fbf7ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553ecd5d0 0x55b553ee0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.305+0000 7fbf77fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553eceb40 0x55b553ee0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.305+0000 7fbf7ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553ecd5d0 0x55b553ee0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54610/0 (socket says 172.21.15.22:54610) 2024-09-19T08:24:27.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.305+0000 7fbf7ffff640 1 -- 172.21.15.22:0/52579357 learned_addr learned my addr 172.21.15.22:0/52579357 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:27.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.305+0000 7fbf84a9f640 1 --2- 172.21.15.22:0/52579357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ed8370 0x55b553ed1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.306+0000 7fbf7ffff640 1 -- 172.21.15.22:0/52579357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553eceb40 msgr2=0x55b553ee0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:27.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.306+0000 7fbf7ffff640 1 --2- 172.21.15.22:0/52579357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553eceb40 0x55b553ee0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.306+0000 7fbf7ffff640 1 -- 172.21.15.22:0/52579357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ed8370 msgr2=0x55b553ed1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:27.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.306+0000 7fbf7ffff640 1 --2- 172.21.15.22:0/52579357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ed8370 0x55b553ed1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.306+0000 7fbf7ffff640 1 -- 172.21.15.22:0/52579357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b553eca960 con 0x55b553ecd5d0 2024-09-19T08:24:27.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.306+0000 7fbf77fff640 1 --2- 172.21.15.22:0/52579357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553eceb40 0x55b553ee0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:27.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.306+0000 7fbf7ffff640 1 --2- 172.21.15.22:0/52579357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553ecd5d0 0x55b553ee0300 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009b90 tx=0x7fbf70009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0d034d507409b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:27.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.306+0000 7fbf5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:24:27.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.306+0000 7fbf5f7fe640 1 -- 172.21.15.22:0/52579357 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70003b80 con 0x55b553ecd5d0 2024-09-19T08:24:27.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.306+0000 7fbf888db740 1 -- 172.21.15.22:0/52579357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf78036030 con 0x55b553ecd5d0 2024-09-19T08:24:27.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.306+0000 7fbf5f7fe640 1 -- 172.21.15.22:0/52579357 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fbf70003d20 con 0x55b553ecd5d0 2024-09-19T08:24:27.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.306+0000 7fbf5f7fe640 1 -- 172.21.15.22:0/52579357 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbf70034050 con 0x55b553ecd5d0 2024-09-19T08:24:27.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.306+0000 7fbf888db740 10 client.17025 fetch_fsmap learned FSMap version 35 2024-09-19T08:24:27.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.306+0000 7fbf888db740 1 -- 172.21.15.22:0/52579357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b553f52020 con 0x55b553ecd5d0 2024-09-19T08:24:27.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.307+0000 7fbf5f7fe640 1 -- 172.21.15.22:0/52579357 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf78036030 con 0x55b553ecd5d0 2024-09-19T08:24:27.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.307+0000 7fbf5f7fe640 1 -- 172.21.15.22:0/52579357 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fbf7002f030 con 0x55b553ecd5d0 2024-09-19T08:24:27.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.307+0000 7fbf888db740 10 client.17025 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:24:27.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.307+0000 7fbf888db740 10 client.17025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:24:27.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.307+0000 7fbf888db740 20 client.17025 populate_metadata read hostname 'smithi022' 2024-09-19T08:24:27.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.307+0000 7fbf888db740 1 --2- 172.21.15.22:0/52579357 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b553f527a0 0x55b553f54bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:27.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.307+0000 7fbf888db740 4 client.17025 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:24:27.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.307+0000 7fbf888db740 1 -- 172.21.15.22:0/52579357 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b553e3ad70 con 0x55b553f527a0 2024-09-19T08:24:27.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.307+0000 7fbf77fff640 1 --2- 172.21.15.22:0/52579357 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b553f527a0 0x55b553f54bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:27.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.307+0000 7fbf5f7fe640 10 client.17025 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:27.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.307+0000 7fbf77fff640 1 --2- 172.21.15.22:0/52579357 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b553f527a0 0x55b553f54bc0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=532cd5c77c5c3e15 server_cookie=d6025a1a10ccc586 in_seq=0 out_seq=0 2024-09-19T08:24:27.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.308+0000 7fbf5f7fe640 1 -- 172.21.15.22:0/52579357 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b553e3ad70 con 0x55b553f527a0 2024-09-19T08:24:27.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.308+0000 7fbf5f7fe640 10 client.17025 handle_command_reply: tid=0 2024-09-19T08:24:27.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.308+0000 7fbf888db740 1 client.17025 shutdown 2024-09-19T08:24:27.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.308+0000 7fbf888db740 1 -- 172.21.15.22:0/52579357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553ecd5d0 msgr2=0x55b553ee0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:27.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.308+0000 7fbf888db740 1 --2- 172.21.15.22:0/52579357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553ecd5d0 0x55b553ee0300 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009b90 tx=0x7fbf70009c80 comp rx=0 tx=0).stop 2024-09-19T08:24:27.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.308+0000 7fbf888db740 1 -- 172.21.15.22:0/52579357 shutdown_connections 2024-09-19T08:24:27.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.308+0000 7fbf888db740 1 --2- 172.21.15.22:0/52579357 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55b553f527a0 0x55b553f54bc0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.308+0000 7fbf888db740 1 --2- 172.21.15.22:0/52579357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b553ed8370 0x55b553ed1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.308+0000 7fbf5f7fe640 0 client.17025 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:27.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.308+0000 7fbf888db740 1 --2- 172.21.15.22:0/52579357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b553eceb40 0x55b553ee0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.308+0000 7fbf888db740 1 --2- 172.21.15.22:0/52579357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b553ecd5d0 0x55b553ee0300 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.308+0000 7fbf888db740 1 -- 172.21.15.22:0/52579357 >> 172.21.15.22:0/52579357 conn(0x55b553e96310 msgr2=0x55b553ed4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:27.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.308+0000 7fbf888db740 1 -- 172.21.15.22:0/52579357 shutdown_connections 2024-09-19T08:24:27.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.308+0000 7fbf888db740 1 -- 172.21.15.22:0/52579357 wait complete. 2024-09-19T08:24:27.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.309+0000 7fbf888db740 20 client.17025 trim_cache size 0 max 16384 2024-09-19T08:24:27.314 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:24:27.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.309+0000 7fbf86502640 1 -- 172.21.15.22:0/2748153619 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbf6c0a33b0 msgr2=0x7fbf6c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:27.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.309+0000 7fbf86502640 1 --2- 172.21.15.22:0/2748153619 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbf6c0a33b0 0x7fbf6c0a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbf7804a000 tx=0x7fbf7804b000 comp rx=0 tx=0).stop 2024-09-19T08:24:27.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.309+0000 7fbf86502640 1 -- 172.21.15.22:0/2748153619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf801027a0 msgr2=0x7fbf801989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:27.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.309+0000 7fbf86502640 1 --2- 172.21.15.22:0/2748153619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf801027a0 0x7fbf801989c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800e9f0 tx=0x7fbf6800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.309+0000 7fbf86502640 1 -- 172.21.15.22:0/2748153619 shutdown_connections 2024-09-19T08:24:27.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.309+0000 7fbf86502640 1 --2- 172.21.15.22:0/2748153619 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fbf6c0a33b0 0x7fbf6c0a5870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.309+0000 7fbf86502640 1 --2- 172.21.15.22:0/2748153619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf80106f20 0x7fbf8019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.309+0000 7fbf86502640 1 --2- 172.21.15.22:0/2748153619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf801027a0 0x7fbf801989c0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.309+0000 7fbf86502640 1 --2- 172.21.15.22:0/2748153619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf80101df0 0x7fbf80198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:27.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.309+0000 7fbf86502640 1 -- 172.21.15.22:0/2748153619 >> 172.21.15.22:0/2748153619 conn(0x7fbf800fd8f0 msgr2=0x7fbf800fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:27.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.309+0000 7fbf86502640 1 -- 172.21.15.22:0/2748153619 shutdown_connections 2024-09-19T08:24:27.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:27.310+0000 7fbf86502640 1 -- 172.21.15.22:0/2748153619 wait complete. 2024-09-19T08:24:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:28 smithi157 ceph-mon[35143]: pgmap v292: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 452 op/s 2024-09-19T08:24:29.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:28 smithi022 ceph-mon[30963]: pgmap v292: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 452 op/s 2024-09-19T08:24:29.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:28 smithi064 ceph-mon[34900]: pgmap v292: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 452 op/s 2024-09-19T08:24:29.345 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:24:29.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:24:29.645 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:24:30.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.272+0000 7f4d6664d640 1 -- 172.21.15.22:0/275321013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d60102ef0 msgr2=0x7f4d6010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.272+0000 7f4d6664d640 1 --2- 172.21.15.22:0/275321013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d60102ef0 0x7f4d6010da40 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009ca0 tx=0x7f4d5802a950 comp rx=0 tx=0).stop 2024-09-19T08:24:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.272+0000 7f4d6664d640 1 -- 172.21.15.22:0/275321013 shutdown_connections 2024-09-19T08:24:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.272+0000 7f4d6664d640 1 --2- 172.21.15.22:0/275321013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d60102ef0 0x7f4d6010da40 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.272+0000 7f4d6664d640 1 --2- 172.21.15.22:0/275321013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d60102570 0x7f4d601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.272+0000 7f4d6664d640 1 --2- 172.21.15.22:0/275321013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d60100e20 0x7f4d60101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.272+0000 7f4d6664d640 1 -- 172.21.15.22:0/275321013 >> 172.21.15.22:0/275321013 conn(0x7f4d600fc7d0 msgr2=0x7f4d600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.272+0000 7f4d6664d640 1 -- 172.21.15.22:0/275321013 shutdown_connections 2024-09-19T08:24:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.272+0000 7f4d6664d640 1 -- 172.21.15.22:0/275321013 wait complete. 2024-09-19T08:24:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d6664d640 1 Processor -- start 2024-09-19T08:24:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d6664d640 1 -- start start 2024-09-19T08:24:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d6664d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d60100e20 0x7f4d600fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d6664d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d60102570 0x7f4d600fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d6664d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d60102ef0 0x7f4d600fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d6664d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4d60110f80 con 0x7f4d60102570 2024-09-19T08:24:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d6664d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d60110e00 con 0x7f4d60100e20 2024-09-19T08:24:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d6664d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d60111100 con 0x7f4d60102ef0 2024-09-19T08:24:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d5f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d60102570 0x7f4d600fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d5f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d60102570 0x7f4d600fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43854/0 (socket says 172.21.15.22:43854) 2024-09-19T08:24:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d5f7fe640 1 -- 172.21.15.22:0/1958631246 learned_addr learned my addr 172.21.15.22:0/1958631246 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d5ffff640 1 --2- 172.21.15.22:0/1958631246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d60100e20 0x7f4d600fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d64bea640 1 --2- 172.21.15.22:0/1958631246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d60102ef0 0x7f4d600fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d5f7fe640 1 -- 172.21.15.22:0/1958631246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d60102ef0 msgr2=0x7f4d600fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d5f7fe640 1 --2- 172.21.15.22:0/1958631246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d60102ef0 0x7f4d600fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d5f7fe640 1 -- 172.21.15.22:0/1958631246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d60100e20 msgr2=0x7f4d600fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d5f7fe640 1 --2- 172.21.15.22:0/1958631246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d60100e20 0x7f4d600fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.273+0000 7f4d5f7fe640 1 -- 172.21.15.22:0/1958631246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d601a68d0 con 0x7f4d60102570 2024-09-19T08:24:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.274+0000 7f4d5f7fe640 1 --2- 172.21.15.22:0/1958631246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d60102570 0x7f4d600fd1c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400e9f0 tx=0x7f4d5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.274+0000 7f4d5d7fa640 1 -- 172.21.15.22:0/1958631246 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5400cd80 con 0x7f4d60102570 2024-09-19T08:24:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.274+0000 7f4d5d7fa640 1 -- 172.21.15.22:0/1958631246 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d54010910 con 0x7f4d60102570 2024-09-19T08:24:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.274+0000 7f4d5d7fa640 1 -- 172.21.15.22:0/1958631246 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54018990 con 0x7f4d60102570 2024-09-19T08:24:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.274+0000 7f4d6664d640 1 -- 172.21.15.22:0/1958631246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d601a6bc0 con 0x7f4d60102570 2024-09-19T08:24:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.274+0000 7f4d6664d640 1 -- 172.21.15.22:0/1958631246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d601a7100 con 0x7f4d60102570 2024-09-19T08:24:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.276+0000 7f4d5d7fa640 1 -- 172.21.15.22:0/1958631246 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4d54010ac0 con 0x7f4d60102570 2024-09-19T08:24:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d68a26740 1 Processor -- start 2024-09-19T08:24:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d68a26740 1 -- start start 2024-09-19T08:24:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d68a26740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f114b60 0x55cc8f114f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d68a26740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f115480 0x55cc8f11f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d68a26740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f11fd40 0x55cc8f122130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d68a26740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cc8f07a280 con 0x55cc8f115480 2024-09-19T08:24:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d68a26740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cc8f021b10 con 0x55cc8f11fd40 2024-09-19T08:24:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d68a26740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cc8f076240 con 0x55cc8f114b60 2024-09-19T08:24:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d5d7fa640 1 --2- 172.21.15.22:0/1958631246 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f4d480a33b0 0x7f4d480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f115480 0x55cc8f11f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d64bea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f114b60 0x55cc8f114f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f115480 0x55cc8f11f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43868/0 (socket says 172.21.15.22:43868) 2024-09-19T08:24:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d5ffff640 1 -- 172.21.15.22:0/1310881371 learned_addr learned my addr 172.21.15.22:0/1310881371 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d5f7fe640 1 --2- 172.21.15.22:0/1310881371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f11fd40 0x55cc8f122130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d64bea640 1 --2- 172.21.15.22:0/1958631246 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f4d480a33b0 0x7f4d480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d5d7fa640 1 -- 172.21.15.22:0/1958631246 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f4d540cffc0 con 0x7f4d60102570 2024-09-19T08:24:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d64bea640 1 -- 172.21.15.22:0/1310881371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f11fd40 msgr2=0x55cc8f122130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d64bea640 1 --2- 172.21.15.22:0/1310881371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f11fd40 0x55cc8f122130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d64bea640 1 -- 172.21.15.22:0/1310881371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f115480 msgr2=0x55cc8f11f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d64bea640 1 --2- 172.21.15.22:0/1310881371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f115480 0x55cc8f11f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.277+0000 7f4d64bea640 1 -- 172.21.15.22:0/1310881371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc8f02e1f0 con 0x55cc8f114b60 2024-09-19T08:24:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.278+0000 7f4d64bea640 1 --2- 172.21.15.22:0/1310881371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f114b60 0x55cc8f114f40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d58005cc0 tx=0x7f4d5802f9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c267c9544e09862e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.278+0000 7f4d64bea640 1 --2- 172.21.15.22:0/1958631246 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f4d480a33b0 0x7f4d480a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d5803c000 tx=0x7f4d58048000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.278+0000 7f4d467fc640 1 -- 172.21.15.22:0/1310881371 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5804d070 con 0x55cc8f114b60 2024-09-19T08:24:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.278+0000 7f4d467fc640 1 -- 172.21.15.22:0/1310881371 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d58004040 con 0x55cc8f114b60 2024-09-19T08:24:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.278+0000 7f4d467fc640 1 -- 172.21.15.22:0/1310881371 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58034040 con 0x55cc8f114b60 2024-09-19T08:24:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.279+0000 7f4d68a26740 1 -- 172.21.15.22:0/1310881371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f114b60 msgr2=0x55cc8f114f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.279+0000 7f4d68a26740 1 --2- 172.21.15.22:0/1310881371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f114b60 0x55cc8f114f40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d58005cc0 tx=0x7f4d5802f9b0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.279+0000 7f4d68a26740 1 -- 172.21.15.22:0/1310881371 shutdown_connections 2024-09-19T08:24:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.279+0000 7f4d68a26740 1 --2- 172.21.15.22:0/1310881371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f11fd40 0x55cc8f122130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.279+0000 7f4d68a26740 1 --2- 172.21.15.22:0/1310881371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f115480 0x55cc8f11f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.279+0000 7f4d68a26740 1 --2- 172.21.15.22:0/1310881371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f114b60 0x55cc8f114f40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.279+0000 7f4d68a26740 1 -- 172.21.15.22:0/1310881371 >> 172.21.15.22:0/1310881371 conn(0x55cc8f1110f0 msgr2=0x55cc8f1114d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.279+0000 7f4d68a26740 1 -- 172.21.15.22:0/1310881371 shutdown_connections 2024-09-19T08:24:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.279+0000 7f4d68a26740 1 -- 172.21.15.22:0/1310881371 wait complete. 2024-09-19T08:24:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.279+0000 7f4d68a26740 1 -- start start 2024-09-19T08:24:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d68a26740 1 Processor -- start 2024-09-19T08:24:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d68a26740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f114b60 0x55cc8f11cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d68a26740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f115480 0x55cc8f11d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d68a26740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f11fd40 0x55cc8f11d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d68a26740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cc8f122910 con 0x55cc8f11fd40 2024-09-19T08:24:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d68a26740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cc8f034f10 con 0x55cc8f114b60 2024-09-19T08:24:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d68a26740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cc8f122a90 con 0x55cc8f115480 2024-09-19T08:24:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d68a26740 10 client.0 _ll_register_callbacks cb 0x55cc8effda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:24:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d64bea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f11fd40 0x55cc8f11d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d5f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f115480 0x55cc8f11d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d5ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f114b60 0x55cc8f11cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d64bea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f11fd40 0x55cc8f11d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43876/0 (socket says 172.21.15.22:43876) 2024-09-19T08:24:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d64bea640 1 -- 172.21.15.22:0/1645383348 learned_addr learned my addr 172.21.15.22:0/1645383348 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:30.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d5ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f114b60 0x55cc8f11cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:38640/0 (socket says 172.21.15.22:38640) 2024-09-19T08:24:30.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d5f7fe640 1 --2- 172.21.15.22:0/1645383348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f115480 0x55cc8f11d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:60012/0 (socket says 172.21.15.22:60012) 2024-09-19T08:24:30.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d64bea640 1 -- 172.21.15.22:0/1645383348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f115480 msgr2=0x55cc8f11d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:30.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d64bea640 1 --2- 172.21.15.22:0/1645383348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f115480 0x55cc8f11d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d64bea640 1 -- 172.21.15.22:0/1645383348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f114b60 msgr2=0x55cc8f11cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:30.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d64bea640 1 --2- 172.21.15.22:0/1645383348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f114b60 0x55cc8f11cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d64bea640 1 -- 172.21.15.22:0/1645383348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc8edeb6a0 con 0x55cc8f11fd40 2024-09-19T08:24:30.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.280+0000 7f4d5ffff640 1 --2- 172.21.15.22:0/1645383348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f114b60 0x55cc8f11cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:30.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d64bea640 1 --2- 172.21.15.22:0/1645383348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f11fd40 0x55cc8f11d6b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4d58003cf0 tx=0x7f4d58003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa47f6bb2c43d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:30.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:24:30.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d3f7fe640 1 -- 172.21.15.22:0/1645383348 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58054070 con 0x55cc8f11fd40 2024-09-19T08:24:30.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d3f7fe640 1 -- 172.21.15.22:0/1645383348 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f4d58004600 con 0x55cc8f11fd40 2024-09-19T08:24:30.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d68a26740 1 -- 172.21.15.22:0/1645383348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d58004040 con 0x55cc8f11fd40 2024-09-19T08:24:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d3f7fe640 1 -- 172.21.15.22:0/1645383348 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4d58060050 con 0x55cc8f11fd40 2024-09-19T08:24:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d68a26740 10 client.17037 fetch_fsmap learned FSMap version 35 2024-09-19T08:24:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d68a26740 1 -- 172.21.15.22:0/1645383348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cc8f122910 con 0x55cc8f11fd40 2024-09-19T08:24:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d3f7fe640 1 -- 172.21.15.22:0/1645383348 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d58004040 con 0x55cc8f11fd40 2024-09-19T08:24:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d3f7fe640 1 -- 172.21.15.22:0/1645383348 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f4d5803a430 con 0x55cc8f11fd40 2024-09-19T08:24:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d68a26740 10 client.17037 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:24:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d68a26740 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:24:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d68a26740 20 client.17037 populate_metadata read hostname 'smithi022' 2024-09-19T08:24:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d68a26740 1 --2- 172.21.15.22:0/1645383348 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55cc8f12f6a0 0x55cc8f131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d68a26740 4 client.17037 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:24:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.281+0000 7f4d68a26740 1 -- 172.21.15.22:0/1645383348 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc8f0877f0 con 0x55cc8f12f6a0 2024-09-19T08:24:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.282+0000 7f4d5ffff640 1 --2- 172.21.15.22:0/1645383348 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55cc8f12f6a0 0x55cc8f131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.282+0000 7f4d5ffff640 1 --2- 172.21.15.22:0/1645383348 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55cc8f12f6a0 0x55cc8f131a90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b043a0258e823d55 server_cookie=aa69143507b9b24f in_seq=0 out_seq=0 2024-09-19T08:24:30.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.282+0000 7f4d3f7fe640 10 client.17037 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:30.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.283+0000 7f4d3f7fe640 1 -- 172.21.15.22:0/1645383348 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc8f0877f0 con 0x55cc8f12f6a0 2024-09-19T08:24:30.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.283+0000 7f4d3f7fe640 10 client.17037 handle_command_reply: tid=0 2024-09-19T08:24:30.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.283+0000 7f4d68a26740 1 client.17037 shutdown 2024-09-19T08:24:30.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.283+0000 7f4d68a26740 1 -- 172.21.15.22:0/1645383348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f11fd40 msgr2=0x55cc8f11d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:30.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.283+0000 7f4d68a26740 1 --2- 172.21.15.22:0/1645383348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f11fd40 0x55cc8f11d6b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4d58003cf0 tx=0x7f4d58003d20 comp rx=0 tx=0).stop 2024-09-19T08:24:30.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.283+0000 7f4d68a26740 1 -- 172.21.15.22:0/1645383348 shutdown_connections 2024-09-19T08:24:30.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.283+0000 7f4d68a26740 1 --2- 172.21.15.22:0/1645383348 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55cc8f12f6a0 0x55cc8f131a90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.283+0000 7f4d68a26740 1 --2- 172.21.15.22:0/1645383348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f11fd40 0x55cc8f11d6b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.283+0000 7f4d68a26740 1 --2- 172.21.15.22:0/1645383348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f115480 0x55cc8f11d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.283+0000 7f4d68a26740 1 --2- 172.21.15.22:0/1645383348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f114b60 0x55cc8f11cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.283+0000 7f4d3f7fe640 0 client.17037 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:30.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.283+0000 7f4d68a26740 1 -- 172.21.15.22:0/1645383348 >> 172.21.15.22:0/1645383348 conn(0x55cc8f1110f0 msgr2=0x55cc8f110270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:30.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.283+0000 7f4d68a26740 1 -- 172.21.15.22:0/1645383348 shutdown_connections 2024-09-19T08:24:30.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.283+0000 7f4d68a26740 1 -- 172.21.15.22:0/1645383348 wait complete. 2024-09-19T08:24:30.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.284+0000 7f4d68a26740 20 client.17037 trim_cache size 0 max 16384 2024-09-19T08:24:30.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.301+0000 7f4d68a26740 1 Processor -- start 2024-09-19T08:24:30.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.301+0000 7f4d68a26740 1 -- start start 2024-09-19T08:24:30.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.301+0000 7f4d68a26740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f114b60 0x55cc8f11bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.301+0000 7f4d68a26740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f115480 0x55cc8f1371d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.301+0000 7f4d68a26740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f11fd40 0x55cc8f137710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.301+0000 7f4d68a26740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4d601b2360 con 0x55cc8f114b60 2024-09-19T08:24:30.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.301+0000 7f4d68a26740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d601afc00 con 0x55cc8f115480 2024-09-19T08:24:30.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.301+0000 7f4d68a26740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d601ada70 con 0x55cc8f11fd40 2024-09-19T08:24:30.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.301+0000 7f4d5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f114b60 0x55cc8f11bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d5f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f115480 0x55cc8f1371d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f114b60 0x55cc8f11bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43888/0 (socket says 172.21.15.22:43888) 2024-09-19T08:24:30.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d64bea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f11fd40 0x55cc8f137710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d5ffff640 1 -- 172.21.15.22:0/475469999 learned_addr learned my addr 172.21.15.22:0/475469999 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:30.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d5f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f115480 0x55cc8f1371d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:38652/0 (socket says 172.21.15.22:38652) 2024-09-19T08:24:30.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d5f7fe640 1 -- 172.21.15.22:0/475469999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f11fd40 msgr2=0x55cc8f137710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d5f7fe640 1 --2- 172.21.15.22:0/475469999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f11fd40 0x55cc8f137710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d5f7fe640 1 -- 172.21.15.22:0/475469999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f114b60 msgr2=0x55cc8f11bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d5f7fe640 1 --2- 172.21.15.22:0/475469999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f114b60 0x55cc8f11bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d5f7fe640 1 -- 172.21.15.22:0/475469999 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc8f11f500 con 0x55cc8f115480 2024-09-19T08:24:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d5ffff640 1 --2- 172.21.15.22:0/475469999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f114b60 0x55cc8f11bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d5f7fe640 1 --2- 172.21.15.22:0/475469999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f115480 0x55cc8f1371d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4d540cd700 tx=0x7f4d540bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1674a2ebad1634e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d45ffb640 1 -- 172.21.15.22:0/475469999 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54026040 con 0x55cc8f115480 2024-09-19T08:24:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d45ffb640 1 -- 172.21.15.22:0/475469999 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d540cde50 con 0x55cc8f115480 2024-09-19T08:24:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.302+0000 7f4d45ffb640 1 -- 172.21.15.22:0/475469999 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d540bc0b0 con 0x55cc8f115480 2024-09-19T08:24:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.303+0000 7f4d68a26740 1 -- 172.21.15.22:0/475469999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f115480 msgr2=0x55cc8f1371d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.303+0000 7f4d68a26740 1 --2- 172.21.15.22:0/475469999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f115480 0x55cc8f1371d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4d540cd700 tx=0x7f4d540bc040 comp rx=0 tx=0).stop 2024-09-19T08:24:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.303+0000 7f4d68a26740 1 -- 172.21.15.22:0/475469999 shutdown_connections 2024-09-19T08:24:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.303+0000 7f4d68a26740 1 --2- 172.21.15.22:0/475469999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f11fd40 0x55cc8f137710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.303+0000 7f4d68a26740 1 --2- 172.21.15.22:0/475469999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f115480 0x55cc8f1371d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.303+0000 7f4d68a26740 1 --2- 172.21.15.22:0/475469999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f114b60 0x55cc8f11bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.303+0000 7f4d68a26740 1 -- 172.21.15.22:0/475469999 >> 172.21.15.22:0/475469999 conn(0x55cc8f1110f0 msgr2=0x55cc8f11b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.303+0000 7f4d68a26740 1 -- 172.21.15.22:0/475469999 shutdown_connections 2024-09-19T08:24:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.303+0000 7f4d68a26740 1 -- 172.21.15.22:0/475469999 wait complete. 2024-09-19T08:24:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.303+0000 7f4d68a26740 1 -- start start 2024-09-19T08:24:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.303+0000 7f4d68a26740 1 Processor -- start 2024-09-19T08:24:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d68a26740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f114b60 0x55cc8f127ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d68a26740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f115480 0x55cc8f128220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d68a26740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f11fd40 0x55cc8f124cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d68a26740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4d601ae0c0 con 0x55cc8f114b60 2024-09-19T08:24:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d68a26740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d601af530 con 0x55cc8f11fd40 2024-09-19T08:24:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d68a26740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d601ae790 con 0x55cc8f115480 2024-09-19T08:24:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d68a26740 10 client.0 _ll_register_callbacks cb 0x7f4d5804b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:24:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f114b60 0x55cc8f127ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d5f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f115480 0x55cc8f128220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d64bea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f11fd40 0x55cc8f124cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d5f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f115480 0x55cc8f128220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:60036/0 (socket says 172.21.15.22:60036) 2024-09-19T08:24:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d5f7fe640 1 -- 172.21.15.22:0/2169904490 learned_addr learned my addr 172.21.15.22:0/2169904490 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d64bea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f11fd40 0x55cc8f124cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:38660/0 (socket says 172.21.15.22:38660) 2024-09-19T08:24:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f114b60 0x55cc8f127ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43898/0 (socket says 172.21.15.22:43898) 2024-09-19T08:24:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d5f7fe640 1 -- 172.21.15.22:0/2169904490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f11fd40 msgr2=0x55cc8f124cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d5f7fe640 1 --2- 172.21.15.22:0/2169904490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f11fd40 0x55cc8f124cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d5f7fe640 1 -- 172.21.15.22:0/2169904490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f114b60 msgr2=0x55cc8f127ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d5f7fe640 1 --2- 172.21.15.22:0/2169904490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f114b60 0x55cc8f127ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d5f7fe640 1 -- 172.21.15.22:0/2169904490 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc8f128890 con 0x55cc8f115480 2024-09-19T08:24:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d64bea640 1 --2- 172.21.15.22:0/2169904490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f11fd40 0x55cc8f124cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d5ffff640 1 --2- 172.21.15.22:0/2169904490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f114b60 0x55cc8f127ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d5f7fe640 1 --2- 172.21.15.22:0/2169904490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f115480 0x55cc8f128220 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400c6a0 tx=0x7f4d5409a850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5252653e75a19a9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.304+0000 7f4d3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:24:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d3f7fe640 1 -- 172.21.15.22:0/2169904490 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d540a0640 con 0x55cc8f115480 2024-09-19T08:24:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d3f7fe640 1 -- 172.21.15.22:0/2169904490 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f4d540bc0b0 con 0x55cc8f115480 2024-09-19T08:24:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d68a26740 1 -- 172.21.15.22:0/2169904490 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d540cde50 con 0x55cc8f115480 2024-09-19T08:24:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d3f7fe640 1 -- 172.21.15.22:0/2169904490 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4d540bc2b0 con 0x55cc8f115480 2024-09-19T08:24:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d68a26740 10 client.26521 fetch_fsmap learned FSMap version 35 2024-09-19T08:24:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d68a26740 1 -- 172.21.15.22:0/2169904490 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cc8f19a760 con 0x55cc8f115480 2024-09-19T08:24:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d3f7fe640 1 -- 172.21.15.22:0/2169904490 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d540cde50 con 0x55cc8f115480 2024-09-19T08:24:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d3f7fe640 1 -- 172.21.15.22:0/2169904490 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f4d540a00d0 con 0x55cc8f115480 2024-09-19T08:24:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d68a26740 10 client.26521 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:24:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d68a26740 10 client.26521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:24:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d68a26740 20 client.26521 populate_metadata read hostname 'smithi022' 2024-09-19T08:24:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d68a26740 1 --2- 172.21.15.22:0/2169904490 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55cc8f19aee0 0x55cc8f19d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d68a26740 4 client.26521 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:24:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d68a26740 1 -- 172.21.15.22:0/2169904490 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cc8f081d00 con 0x55cc8f19aee0 2024-09-19T08:24:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.305+0000 7f4d5ffff640 1 --2- 172.21.15.22:0/2169904490 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55cc8f19aee0 0x55cc8f19d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.306+0000 7f4d3f7fe640 10 client.26521 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.306+0000 7f4d5ffff640 1 --2- 172.21.15.22:0/2169904490 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55cc8f19aee0 0x55cc8f19d300 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc3b5a68c86fb237 server_cookie=222ac7d4c4a02745 in_seq=0 out_seq=0 2024-09-19T08:24:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d3f7fe640 1 -- 172.21.15.22:0/2169904490 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cc8f081d00 con 0x55cc8f19aee0 2024-09-19T08:24:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d3f7fe640 10 client.26521 handle_command_reply: tid=0 2024-09-19T08:24:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d68a26740 1 client.26521 shutdown 2024-09-19T08:24:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d68a26740 1 -- 172.21.15.22:0/2169904490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f115480 msgr2=0x55cc8f128220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d68a26740 1 --2- 172.21.15.22:0/2169904490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f115480 0x55cc8f128220 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400c6a0 tx=0x7f4d5409a850 comp rx=0 tx=0).stop 2024-09-19T08:24:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d68a26740 1 -- 172.21.15.22:0/2169904490 shutdown_connections 2024-09-19T08:24:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d68a26740 1 --2- 172.21.15.22:0/2169904490 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55cc8f19aee0 0x55cc8f19d300 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d68a26740 1 --2- 172.21.15.22:0/2169904490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc8f11fd40 0x55cc8f124cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d68a26740 1 --2- 172.21.15.22:0/2169904490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc8f115480 0x55cc8f128220 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d68a26740 1 --2- 172.21.15.22:0/2169904490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc8f114b60 0x55cc8f127ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d68a26740 1 -- 172.21.15.22:0/2169904490 >> 172.21.15.22:0/2169904490 conn(0x55cc8f1110f0 msgr2=0x55cc8f110270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d3f7fe640 0 client.26521 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d68a26740 1 -- 172.21.15.22:0/2169904490 shutdown_connections 2024-09-19T08:24:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d68a26740 1 -- 172.21.15.22:0/2169904490 wait complete. 2024-09-19T08:24:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.307+0000 7f4d68a26740 20 client.26521 trim_cache size 0 max 16384 2024-09-19T08:24:30.317 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:24:30.317 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-19T08:24:30.317 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "3217e6b5-00e5-47c1-a277-7e3f13320e37", 2024-09-19T08:24:30.317 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-19T08:24:30.317 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:24:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.308+0000 7f4d6664d640 1 -- 172.21.15.22:0/1958631246 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f4d480a33b0 msgr2=0x7f4d480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.308+0000 7f4d6664d640 1 --2- 172.21.15.22:0/1958631246 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f4d480a33b0 0x7f4d480a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d5803c000 tx=0x7f4d58048000 comp rx=0 tx=0).stop 2024-09-19T08:24:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.308+0000 7f4d6664d640 1 -- 172.21.15.22:0/1958631246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d60102570 msgr2=0x7f4d600fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.308+0000 7f4d6664d640 1 --2- 172.21.15.22:0/1958631246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d60102570 0x7f4d600fd1c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400e9f0 tx=0x7f4d5400eeb0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.308+0000 7f4d6664d640 1 -- 172.21.15.22:0/1958631246 shutdown_connections 2024-09-19T08:24:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.308+0000 7f4d6664d640 1 --2- 172.21.15.22:0/1958631246 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f4d480a33b0 0x7f4d480a5870 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.308+0000 7f4d6664d640 1 --2- 172.21.15.22:0/1958631246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d60102ef0 0x7f4d600fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.308+0000 7f4d6664d640 1 --2- 172.21.15.22:0/1958631246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d60102570 0x7f4d600fd1c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.308+0000 7f4d6664d640 1 --2- 172.21.15.22:0/1958631246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d60100e20 0x7f4d600fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.308+0000 7f4d6664d640 1 -- 172.21.15.22:0/1958631246 >> 172.21.15.22:0/1958631246 conn(0x7f4d600fc7d0 msgr2=0x7f4d6006a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.308+0000 7f4d6664d640 1 -- 172.21.15.22:0/1958631246 shutdown_connections 2024-09-19T08:24:30.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:30.308+0000 7f4d6664d640 1 -- 172.21.15.22:0/1958631246 wait complete. 2024-09-19T08:24:30.975 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:30 smithi022 ceph-mon[30963]: pgmap v293: 97 pgs: 97 active+clean; 1013 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 527 op/s 2024-09-19T08:24:31.003 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:24:31.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:30 smithi157 ceph-mon[35143]: pgmap v293: 97 pgs: 97 active+clean; 1013 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 527 op/s 2024-09-19T08:24:31.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:30 smithi064 ceph-mon[34900]: pgmap v293: 97 pgs: 97 active+clean; 1013 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 527 op/s 2024-09-19T08:24:31.265 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:24:31.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.797+0000 7f7aaecc9640 1 -- 172.21.15.22:0/1965355577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8108290 msgr2=0x7f7aa810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:31.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.797+0000 7f7aaecc9640 1 --2- 172.21.15.22:0/1965355577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8108290 0x7f7aa810ec30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009cd0 tx=0x7f7aa002a8e0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.797+0000 7f7aaecc9640 1 -- 172.21.15.22:0/1965355577 shutdown_connections 2024-09-19T08:24:31.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.797+0000 7f7aaecc9640 1 --2- 172.21.15.22:0/1965355577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8108290 0x7f7aa810ec30 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.797+0000 7f7aaecc9640 1 --2- 172.21.15.22:0/1965355577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aa8103740 0x7f7aa8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.797+0000 7f7aaecc9640 1 --2- 172.21.15.22:0/1965355577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa8102d90 0x7f7aa8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.797+0000 7f7aaecc9640 1 -- 172.21.15.22:0/1965355577 >> 172.21.15.22:0/1965355577 conn(0x7f7aa80fc7d0 msgr2=0x7f7aa80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:31.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.797+0000 7f7aaecc9640 1 -- 172.21.15.22:0/1965355577 shutdown_connections 2024-09-19T08:24:31.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.797+0000 7f7aaecc9640 1 -- 172.21.15.22:0/1965355577 wait complete. 2024-09-19T08:24:31.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.798+0000 7f7aaecc9640 1 Processor -- start 2024-09-19T08:24:31.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.798+0000 7f7aaecc9640 1 -- start start 2024-09-19T08:24:31.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.798+0000 7f7aaecc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aa8102d90 0x7f7aa81009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.798+0000 7f7aaecc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa8103740 0x7f7aa8100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.798+0000 7f7aaecc9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8108290 0x7f7aa8101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.798+0000 7f7aaecc9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8112090 con 0x7f7aa8108290 2024-09-19T08:24:31.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.798+0000 7f7aaecc9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8111f10 con 0x7f7aa8102d90 2024-09-19T08:24:31.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.798+0000 7f7aaecc9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8112210 con 0x7f7aa8103740 2024-09-19T08:24:31.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.798+0000 7f7aaca65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aa8102d90 0x7f7aa81009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.798+0000 7f7aad266640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8108290 0x7f7aa8101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.799+0000 7f7aad266640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8108290 0x7f7aa8101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43928/0 (socket says 172.21.15.22:43928) 2024-09-19T08:24:31.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.799+0000 7f7aad266640 1 -- 172.21.15.22:0/1021514573 learned_addr learned my addr 172.21.15.22:0/1021514573 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:31.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.799+0000 7f7aaca65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aa8102d90 0x7f7aa81009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:38680/0 (socket says 172.21.15.22:38680) 2024-09-19T08:24:31.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.799+0000 7f7aa7fff640 1 --2- 172.21.15.22:0/1021514573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa8103740 0x7f7aa8100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.799+0000 7f7aad266640 1 -- 172.21.15.22:0/1021514573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa8103740 msgr2=0x7f7aa8100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:31.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.799+0000 7f7aad266640 1 --2- 172.21.15.22:0/1021514573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa8103740 0x7f7aa8100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.799+0000 7f7aad266640 1 -- 172.21.15.22:0/1021514573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aa8102d90 msgr2=0x7f7aa81009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:31.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.799+0000 7f7aad266640 1 --2- 172.21.15.22:0/1021514573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aa8102d90 0x7f7aa81009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.799+0000 7f7aad266640 1 -- 172.21.15.22:0/1021514573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa81a6970 con 0x7f7aa8108290 2024-09-19T08:24:31.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.799+0000 7f7aad266640 1 --2- 172.21.15.22:0/1021514573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8108290 0x7f7aa8101470 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0005cc0 tx=0x7f7aa0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:31.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.799+0000 7f7aa5ffb640 1 -- 172.21.15.22:0/1021514573 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0004040 con 0x7f7aa8108290 2024-09-19T08:24:31.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.799+0000 7f7aaecc9640 1 -- 172.21.15.22:0/1021514573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa81a6c00 con 0x7f7aa8108290 2024-09-19T08:24:31.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.799+0000 7f7aa5ffb640 1 -- 172.21.15.22:0/1021514573 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa00046c0 con 0x7f7aa8108290 2024-09-19T08:24:31.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.799+0000 7f7aa5ffb640 1 -- 172.21.15.22:0/1021514573 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0030dc0 con 0x7f7aa8108290 2024-09-19T08:24:31.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.800+0000 7f7aaecc9640 1 -- 172.21.15.22:0/1021514573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa81a7140 con 0x7f7aa8108290 2024-09-19T08:24:31.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.802+0000 7f7aa5ffb640 1 -- 172.21.15.22:0/1021514573 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7aa00041e0 con 0x7f7aa8108290 2024-09-19T08:24:31.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.802+0000 7f7ab10a2740 1 Processor -- start 2024-09-19T08:24:31.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7ab10a2740 1 -- start start 2024-09-19T08:24:31.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7ab10a2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a34a0 0x558f964a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7ab10a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a3dc0 0x558f964ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7aa5ffb640 1 --2- 172.21.15.22:0/1021514573 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7a980a33b0 0x7f7a980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7ab10a2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 0x558f964b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7ab10a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558f964092e0 con 0x558f964a3dc0 2024-09-19T08:24:31.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7ab10a2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558f963b0ce0 con 0x558f964a34a0 2024-09-19T08:24:31.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7ab10a2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558f964052a0 con 0x558f964ae680 2024-09-19T08:24:31.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7aaca65640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a3dc0 0x558f964ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7aaca65640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a3dc0 0x558f964ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43930/0 (socket says 172.21.15.22:43930) 2024-09-19T08:24:31.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7aaca65640 1 -- 172.21.15.22:0/2068539372 learned_addr learned my addr 172.21.15.22:0/2068539372 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:31.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7aa7fff640 1 --2- 172.21.15.22:0/1021514573 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7a980a33b0 0x7f7a980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7aa5ffb640 1 -- 172.21.15.22:0/1021514573 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f7aa002e030 con 0x7f7aa8108290 2024-09-19T08:24:31.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7aad266640 1 --2- 172.21.15.22:0/2068539372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 0x558f964b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7aa7fff640 1 --2- 172.21.15.22:0/2068539372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a34a0 0x558f964a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7aaca65640 1 -- 172.21.15.22:0/2068539372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 msgr2=0x558f964b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:31.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7aaca65640 1 --2- 172.21.15.22:0/2068539372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 0x558f964b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7aaca65640 1 -- 172.21.15.22:0/2068539372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a34a0 msgr2=0x558f964a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:31.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7aaca65640 1 --2- 172.21.15.22:0/2068539372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a34a0 0x558f964a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7aaca65640 1 -- 172.21.15.22:0/2068539372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f96395440 con 0x558f964a3dc0 2024-09-19T08:24:31.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.803+0000 7f7aa7fff640 1 --2- 172.21.15.22:0/2068539372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a34a0 0x558f964a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:24:31.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.804+0000 7f7aaca65640 1 --2- 172.21.15.22:0/2068539372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a3dc0 0x558f964ae140 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009d20 tx=0x7f7a9c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8e330c993bd059 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:31.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.804+0000 7f7aa7fff640 1 --2- 172.21.15.22:0/1021514573 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7a980a33b0 0x7f7a980a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7a94005e10 tx=0x7f7a94005d60 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:31.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.804+0000 7f7a92ffd640 1 -- 172.21.15.22:0/2068539372 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c01e070 con 0x558f964a3dc0 2024-09-19T08:24:31.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.804+0000 7f7a92ffd640 1 -- 172.21.15.22:0/2068539372 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c019b10 con 0x558f964a3dc0 2024-09-19T08:24:31.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.804+0000 7f7a92ffd640 1 -- 172.21.15.22:0/2068539372 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00ebf0 con 0x558f964a3dc0 2024-09-19T08:24:31.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.805+0000 7f7ab10a2740 1 -- 172.21.15.22:0/2068539372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a3dc0 msgr2=0x558f964ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:31.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.805+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/2068539372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a3dc0 0x558f964ae140 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009d20 tx=0x7f7a9c011810 comp rx=0 tx=0).stop 2024-09-19T08:24:31.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.805+0000 7f7ab10a2740 1 -- 172.21.15.22:0/2068539372 shutdown_connections 2024-09-19T08:24:31.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.805+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/2068539372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 0x558f964b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.805+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/2068539372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a3dc0 0x558f964ae140 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.805+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/2068539372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a34a0 0x558f964a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.805+0000 7f7ab10a2740 1 -- 172.21.15.22:0/2068539372 >> 172.21.15.22:0/2068539372 conn(0x558f9649fa30 msgr2=0x558f9649fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:31.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.805+0000 7f7ab10a2740 1 -- 172.21.15.22:0/2068539372 shutdown_connections 2024-09-19T08:24:31.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.805+0000 7f7ab10a2740 1 -- 172.21.15.22:0/2068539372 wait complete. 2024-09-19T08:24:31.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.805+0000 7f7ab10a2740 1 -- start start 2024-09-19T08:24:31.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.805+0000 7f7ab10a2740 1 Processor -- start 2024-09-19T08:24:31.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7ab10a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a34a0 0x558f964adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7ab10a2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964ab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7ab10a2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 0x558f964ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7ab10a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558f964b1250 con 0x558f964a34a0 2024-09-19T08:24:31.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7ab10a2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558f963c4050 con 0x558f964a3dc0 2024-09-19T08:24:31.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7ab10a2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558f964b13d0 con 0x558f964ae680 2024-09-19T08:24:31.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7ab10a2740 10 client.0 _ll_register_callbacks cb 0x558f96404160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:24:31.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7aaca65640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a34a0 0x558f964adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7aa7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7aad266640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 0x558f964ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7aaca65640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a34a0 0x558f964adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43944/0 (socket says 172.21.15.22:43944) 2024-09-19T08:24:31.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7aaca65640 1 -- 172.21.15.22:0/1879359755 learned_addr learned my addr 172.21.15.22:0/1879359755 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:31.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7aa7fff640 1 -- 172.21.15.22:0/1879359755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 msgr2=0x558f964ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:31.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7aa7fff640 1 --2- 172.21.15.22:0/1879359755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 0x558f964ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7aa7fff640 1 -- 172.21.15.22:0/1879359755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a34a0 msgr2=0x558f964adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:31.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7aa7fff640 1 --2- 172.21.15.22:0/1879359755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a34a0 0x558f964adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.806+0000 7f7aa7fff640 1 -- 172.21.15.22:0/1879359755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f9617a3c0 con 0x558f964a3dc0 2024-09-19T08:24:31.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.807+0000 7f7aaca65640 1 --2- 172.21.15.22:0/1879359755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a34a0 0x558f964adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:31.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.807+0000 7f7aad266640 1 --2- 172.21.15.22:0/1879359755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 0x558f964ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:31.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.807+0000 7f7aa7fff640 1 --2- 172.21.15.22:0/1879359755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964ab270 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80697d0 tx=0x7f7a94004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ca0c8078cd814df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:31.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.807+0000 7f7a87fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T08:24:31.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.807+0000 7f7a87fff640 1 -- 172.21.15.22:0/1879359755 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a94024070 con 0x558f964a3dc0 2024-09-19T08:24:31.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.807+0000 7f7ab10a2740 1 -- 172.21.15.22:0/1879359755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a9c019b10 con 0x558f964a3dc0 2024-09-19T08:24:31.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.807+0000 7f7a87fff640 1 -- 172.21.15.22:0/1879359755 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f7a9400fdc0 con 0x558f964a3dc0 2024-09-19T08:24:31.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.807+0000 7f7a87fff640 1 -- 172.21.15.22:0/1879359755 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7a94031470 con 0x558f964a3dc0 2024-09-19T08:24:31.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.807+0000 7f7ab10a2740 10 client.26540 fetch_fsmap learned FSMap version 35 2024-09-19T08:24:31.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.807+0000 7f7ab10a2740 1 -- 172.21.15.22:0/1879359755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558f964c5470 con 0x558f964a3dc0 2024-09-19T08:24:31.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.807+0000 7f7a87fff640 1 -- 172.21.15.22:0/1879359755 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c019b10 con 0x558f964a3dc0 2024-09-19T08:24:31.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.807+0000 7f7a87fff640 1 -- 172.21.15.22:0/1879359755 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f7a94028760 con 0x558f964a3dc0 2024-09-19T08:24:31.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.808+0000 7f7ab10a2740 10 client.26540 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:24:31.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.808+0000 7f7ab10a2740 10 client.26540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:24:31.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.808+0000 7f7ab10a2740 20 client.26540 populate_metadata read hostname 'smithi022' 2024-09-19T08:24:31.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.808+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/1879359755 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f964c69d0 0x558f964c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.808+0000 7f7ab10a2740 4 client.26540 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:24:31.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.808+0000 7f7ab10a2740 1 -- 172.21.15.22:0/1879359755 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f96416850 con 0x558f964c69d0 2024-09-19T08:24:31.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.808+0000 7f7aaca65640 1 --2- 172.21.15.22:0/1879359755 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f964c69d0 0x558f964c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.808+0000 7f7a87fff640 10 client.26540 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:31.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.808+0000 7f7aaca65640 1 --2- 172.21.15.22:0/1879359755 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f964c69d0 0x558f964c8e10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1a0d5268593039c server_cookie=37f673254a2a360b in_seq=0 out_seq=0 2024-09-19T08:24:31.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7a87fff640 1 -- 172.21.15.22:0/1879359755 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f96416850 con 0x558f964c69d0 2024-09-19T08:24:31.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7a87fff640 10 client.26540 handle_command_reply: tid=0 2024-09-19T08:24:31.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7ab10a2740 1 client.26540 shutdown 2024-09-19T08:24:31.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7ab10a2740 1 -- 172.21.15.22:0/1879359755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 msgr2=0x558f964ab270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:31.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/1879359755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964ab270 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80697d0 tx=0x7f7a94004730 comp rx=0 tx=0).stop 2024-09-19T08:24:31.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7ab10a2740 1 -- 172.21.15.22:0/1879359755 shutdown_connections 2024-09-19T08:24:31.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/1879359755 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f964c69d0 0x558f964c8e10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/1879359755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 0x558f964ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/1879359755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964ab270 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7a87fff640 0 client.26540 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:31.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/1879359755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a34a0 0x558f964adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7ab10a2740 1 -- 172.21.15.22:0/1879359755 >> 172.21.15.22:0/1879359755 conn(0x558f9649fa30 msgr2=0x558f964a5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:31.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7ab10a2740 1 -- 172.21.15.22:0/1879359755 shutdown_connections 2024-09-19T08:24:31.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7ab10a2740 1 -- 172.21.15.22:0/1879359755 wait complete. 2024-09-19T08:24:31.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.809+0000 7f7ab10a2740 20 client.26540 trim_cache size 0 max 16384 2024-09-19T08:24:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7ab10a2740 1 Processor -- start 2024-09-19T08:24:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7ab10a2740 1 -- start start 2024-09-19T08:24:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7ab10a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a34a0 0x558f964c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7ab10a2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7ab10a2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 0x558f964cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7ab10a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81adfa0 con 0x558f964a34a0 2024-09-19T08:24:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7ab10a2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81ab840 con 0x558f964a3dc0 2024-09-19T08:24:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7ab10a2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81a96d0 con 0x558f964ae680 2024-09-19T08:24:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7aaca65640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a34a0 0x558f964c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7aaca65640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a34a0 0x558f964c2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43960/0 (socket says 172.21.15.22:43960) 2024-09-19T08:24:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7aaca65640 1 -- 172.21.15.22:0/1473051518 learned_addr learned my addr 172.21.15.22:0/1473051518 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:31.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7aa7fff640 1 --2- 172.21.15.22:0/1473051518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7aad266640 1 --2- 172.21.15.22:0/1473051518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 0x558f964cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7aaca65640 1 -- 172.21.15.22:0/1473051518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 msgr2=0x558f964cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:31.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7aaca65640 1 --2- 172.21.15.22:0/1473051518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 0x558f964cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7aaca65640 1 -- 172.21.15.22:0/1473051518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 msgr2=0x558f964ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:31.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7aaca65640 1 --2- 172.21.15.22:0/1473051518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7aaca65640 1 -- 172.21.15.22:0/1473051518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f964ae330 con 0x558f964a34a0 2024-09-19T08:24:31.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.827+0000 7f7aa7fff640 1 --2- 172.21.15.22:0/1473051518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:31.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.828+0000 7f7aaca65640 1 --2- 172.21.15.22:0/1473051518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a34a0 0x558f964c2d40 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e720 tx=0x7f7a9c005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cfc43e9d7b62ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:31.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.828+0000 7f7a927fc640 1 -- 172.21.15.22:0/1473051518 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c01e040 con 0x558f964a34a0 2024-09-19T08:24:31.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.828+0000 7f7a927fc640 1 -- 172.21.15.22:0/1473051518 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c01f040 con 0x558f964a34a0 2024-09-19T08:24:31.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.828+0000 7f7a927fc640 1 -- 172.21.15.22:0/1473051518 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0048a0 con 0x558f964a34a0 2024-09-19T08:24:31.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.828+0000 7f7ab10a2740 1 -- 172.21.15.22:0/1473051518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a34a0 msgr2=0x558f964c2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:31.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.828+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/1473051518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a34a0 0x558f964c2d40 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e720 tx=0x7f7a9c005210 comp rx=0 tx=0).stop 2024-09-19T08:24:31.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 -- 172.21.15.22:0/1473051518 shutdown_connections 2024-09-19T08:24:31.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/1473051518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964ae680 0x558f964cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/1473051518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/1473051518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964a34a0 0x558f964c2d40 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 -- 172.21.15.22:0/1473051518 >> 172.21.15.22:0/1473051518 conn(0x558f9649fa30 msgr2=0x558f964ad2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:31.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 -- 172.21.15.22:0/1473051518 shutdown_connections 2024-09-19T08:24:31.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 -- 172.21.15.22:0/1473051518 wait complete. 2024-09-19T08:24:31.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 -- start start 2024-09-19T08:24:31.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 Processor -- start 2024-09-19T08:24:31.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964a34a0 0x558f964bbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964bc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964ae680 0x558f9652f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81a9d20 con 0x558f964ae680 2024-09-19T08:24:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81ab170 con 0x558f964a3dc0 2024-09-19T08:24:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81aa3d0 con 0x558f964a34a0 2024-09-19T08:24:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.829+0000 7f7ab10a2740 10 client.0 _ll_register_callbacks cb 0x7f7a9400fa70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:24:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7aad266640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964ae680 0x558f9652f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7aaca65640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964a34a0 0x558f964bbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7aa7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964bc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7aaca65640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964a34a0 0x558f964bbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:60076/0 (socket says 172.21.15.22:60076) 2024-09-19T08:24:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7aad266640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964ae680 0x558f9652f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43968/0 (socket says 172.21.15.22:43968) 2024-09-19T08:24:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7aaca65640 1 -- 172.21.15.22:0/718635772 learned_addr learned my addr 172.21.15.22:0/718635772 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7aaca65640 1 -- 172.21.15.22:0/718635772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 msgr2=0x558f964bc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7aaca65640 1 --2- 172.21.15.22:0/718635772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964bc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7aaca65640 1 -- 172.21.15.22:0/718635772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964ae680 msgr2=0x558f9652f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7aaca65640 1 --2- 172.21.15.22:0/718635772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964ae680 0x558f9652f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7aaca65640 1 -- 172.21.15.22:0/718635772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f9652fbf0 con 0x558f964a34a0 2024-09-19T08:24:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7aad266640 1 --2- 172.21.15.22:0/718635772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964ae680 0x558f9652f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7aa7fff640 1 --2- 172.21.15.22:0/718635772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964bc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:24:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7aaca65640 1 --2- 172.21.15.22:0/718635772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964a34a0 0x558f964bbfd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009250 tx=0x7f7a9c005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23dc0707753386c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7a87fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:24:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7a87fff640 1 -- 172.21.15.22:0/718635772 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c035070 con 0x558f964a34a0 2024-09-19T08:24:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7ab10a2740 1 -- 172.21.15.22:0/718635772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a9c01f040 con 0x558f964a34a0 2024-09-19T08:24:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.830+0000 7f7a87fff640 1 -- 172.21.15.22:0/718635772 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f7a9c00e860 con 0x558f964a34a0 2024-09-19T08:24:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.831+0000 7f7a87fff640 1 -- 172.21.15.22:0/718635772 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7a9c041050 con 0x558f964a34a0 2024-09-19T08:24:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.831+0000 7f7ab10a2740 10 client.26539 fetch_fsmap learned FSMap version 35 2024-09-19T08:24:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.831+0000 7f7ab10a2740 1 -- 172.21.15.22:0/718635772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558f96530d50 con 0x558f964a34a0 2024-09-19T08:24:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.831+0000 7f7a87fff640 1 -- 172.21.15.22:0/718635772 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c01f040 con 0x558f964a34a0 2024-09-19T08:24:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.831+0000 7f7a87fff640 1 -- 172.21.15.22:0/718635772 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f7a9c025040 con 0x558f964a34a0 2024-09-19T08:24:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.831+0000 7f7ab10a2740 10 client.26539 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T08:24:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.831+0000 7f7ab10a2740 10 client.26539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:24:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.831+0000 7f7ab10a2740 20 client.26539 populate_metadata read hostname 'smithi022' 2024-09-19T08:24:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.831+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/718635772 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f96531170 0x558f96533560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.831+0000 7f7ab10a2740 4 client.26539 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:24:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.831+0000 7f7ab10a2740 1 -- 172.21.15.22:0/718635772 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558f96410d60 con 0x558f96531170 2024-09-19T08:24:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.831+0000 7f7aa7fff640 1 --2- 172.21.15.22:0/718635772 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f96531170 0x558f96533560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7aa7fff640 1 --2- 172.21.15.22:0/718635772 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f96531170 0x558f96533560 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a4f955e54923bd server_cookie=1532fef53935aadd in_seq=0 out_seq=0 2024-09-19T08:24:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7a87fff640 10 client.26539 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7a87fff640 1 -- 172.21.15.22:0/718635772 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558f96410d60 con 0x558f96531170 2024-09-19T08:24:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7a87fff640 10 client.26539 handle_command_reply: tid=0 2024-09-19T08:24:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7ab10a2740 1 client.26539 shutdown 2024-09-19T08:24:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7ab10a2740 1 -- 172.21.15.22:0/718635772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964a34a0 msgr2=0x558f964bbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/718635772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964a34a0 0x558f964bbfd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009250 tx=0x7f7a9c005d30 comp rx=0 tx=0).stop 2024-09-19T08:24:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7ab10a2740 1 -- 172.21.15.22:0/718635772 shutdown_connections 2024-09-19T08:24:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/718635772 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558f96531170 0x558f96533560 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/718635772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558f964ae680 0x558f9652f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/718635772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f964a3dc0 0x558f964bc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7ab10a2740 1 --2- 172.21.15.22:0/718635772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f964a34a0 0x558f964bbfd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7a87fff640 0 client.26539 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7ab10a2740 1 -- 172.21.15.22:0/718635772 >> 172.21.15.22:0/718635772 conn(0x558f9649fa30 msgr2=0x558f9649e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7ab10a2740 1 -- 172.21.15.22:0/718635772 shutdown_connections 2024-09-19T08:24:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7ab10a2740 1 -- 172.21.15.22:0/718635772 wait complete. 2024-09-19T08:24:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.832+0000 7f7ab10a2740 20 client.26539 trim_cache size 0 max 16384 2024-09-19T08:24:31.839 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:24:31.839 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:24:31.839 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-19T08:24:31.839 INFO:teuthology.orchestra.run.smithi022.stdout: "3217e6b5-00e5-47c1-a277-7e3f13320e37": { 2024-09-19T08:24:31.839 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-19T08:24:31.839 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "3217e6b5-00e5-47c1-a277-7e3f13320e37", 2024-09-19T08:24:31.840 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-09-19T08:24:31.840 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:24:31.840 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:24:31.840 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:24:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.833+0000 7f7aaecc9640 1 -- 172.21.15.22:0/1021514573 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7a980a33b0 msgr2=0x7f7a980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.833+0000 7f7aaecc9640 1 --2- 172.21.15.22:0/1021514573 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7a980a33b0 0x7f7a980a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7a94005e10 tx=0x7f7a94005d60 comp rx=0 tx=0).stop 2024-09-19T08:24:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.833+0000 7f7aaecc9640 1 -- 172.21.15.22:0/1021514573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8108290 msgr2=0x7f7aa8101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.833+0000 7f7aaecc9640 1 --2- 172.21.15.22:0/1021514573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8108290 0x7f7aa8101470 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0005cc0 tx=0x7f7aa0032f40 comp rx=0 tx=0).stop 2024-09-19T08:24:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.833+0000 7f7aaecc9640 1 -- 172.21.15.22:0/1021514573 shutdown_connections 2024-09-19T08:24:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.833+0000 7f7aaecc9640 1 --2- 172.21.15.22:0/1021514573 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f7a980a33b0 0x7f7a980a5870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.833+0000 7f7aaecc9640 1 --2- 172.21.15.22:0/1021514573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa8108290 0x7f7aa8101470 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.833+0000 7f7aaecc9640 1 --2- 172.21.15.22:0/1021514573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa8103740 0x7f7aa8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.833+0000 7f7aaecc9640 1 --2- 172.21.15.22:0/1021514573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7aa8102d90 0x7f7aa81009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.833+0000 7f7aaecc9640 1 -- 172.21.15.22:0/1021514573 >> 172.21.15.22:0/1021514573 conn(0x7f7aa80fc7d0 msgr2=0x7f7aa80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.833+0000 7f7aaecc9640 1 -- 172.21.15.22:0/1021514573 shutdown_connections 2024-09-19T08:24:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:31.833+0000 7f7aaecc9640 1 -- 172.21.15.22:0/1021514573 wait complete. 2024-09-19T08:24:32.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:31 smithi157 ceph-mon[35143]: scrub queued for path: / 2024-09-19T08:24:32.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:31 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:24:32.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:31 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:24:32.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:31 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:24:32.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:31 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:24:32.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:31 smithi022 ceph-mon[30963]: scrub queued for path: / 2024-09-19T08:24:32.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:31 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:24:32.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:31 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:24:32.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:31 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:24:32.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:31 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:24:32.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:31 smithi064 ceph-mon[34900]: scrub queued for path: / 2024-09-19T08:24:32.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:31 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:24:32.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:31 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:24:32.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:31 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:24:32.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:31 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:24:32.681 INFO:tasks.cephfs.filesystem:scrub status for tag:3217e6b5-00e5-47c1-a277-7e3f13320e37 - {'path': '/', 'tag': '3217e6b5-00e5-47c1-a277-7e3f13320e37', 'options': 'recursive,force'} 2024-09-19T08:24:33.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:32 smithi157 ceph-mon[35143]: pgmap v294: 97 pgs: 97 active+clean; 1013 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 322 op/s 2024-09-19T08:24:33.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:32 smithi022 ceph-mon[30963]: pgmap v294: 97 pgs: 97 active+clean; 1013 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 322 op/s 2024-09-19T08:24:33.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:32 smithi064 ceph-mon[34900]: pgmap v294: 97 pgs: 97 active+clean; 1013 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 322 op/s 2024-09-19T08:24:34.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:33 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:24:34.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:33 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:24:34.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:33 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:24:35.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:34 smithi157 ceph-mon[35143]: pgmap v295: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 648 op/s 2024-09-19T08:24:35.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:34 smithi022 ceph-mon[30963]: pgmap v295: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 648 op/s 2024-09-19T08:24:35.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:34 smithi064 ceph-mon[34900]: pgmap v295: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 648 op/s 2024-09-19T08:24:36.672 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-09-19T08:24:36.673 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T08:24:36.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:36 smithi022 ceph-mon[30963]: pgmap v296: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 476 op/s 2024-09-19T08:24:36.988 DEBUG:teuthology.parallel:result is None 2024-09-19T08:24:36.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T08:24:37.002 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:36 smithi157 ceph-mon[35143]: pgmap v296: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 476 op/s 2024-09-19T08:24:37.020 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T08:24:37.020 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-19T08:24:37.032 INFO:tasks.quiescer:joining Quiescers 2024-09-19T08:24:37.033 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-19T08:24:37.033 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:24:37.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:36 smithi064 ceph-mon[34900]: pgmap v296: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 476 op/s 2024-09-19T08:24:37.316 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:24:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.836+0000 7fe821f7a640 1 -- 172.21.15.22:0/1638930743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe81c106a80 msgr2=0x7fe81c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.836+0000 7fe821f7a640 1 --2- 172.21.15.22:0/1638930743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe81c106a80 0x7fe81c106e60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009930 tx=0x7fe80c02a420 comp rx=0 tx=0).stop 2024-09-19T08:24:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.836+0000 7fe821f7a640 1 -- 172.21.15.22:0/1638930743 shutdown_connections 2024-09-19T08:24:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.836+0000 7fe821f7a640 1 --2- 172.21.15.22:0/1638930743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe81c101420 0x7fe81c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.836+0000 7fe821f7a640 1 --2- 172.21.15.22:0/1638930743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81c100a80 0x7fe81c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.836+0000 7fe821f7a640 1 --2- 172.21.15.22:0/1638930743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe81c106a80 0x7fe81c106e60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.836+0000 7fe821f7a640 1 -- 172.21.15.22:0/1638930743 >> 172.21.15.22:0/1638930743 conn(0x7fe81c0fc7a0 msgr2=0x7fe81c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.836+0000 7fe821f7a640 1 -- 172.21.15.22:0/1638930743 shutdown_connections 2024-09-19T08:24:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.836+0000 7fe821f7a640 1 -- 172.21.15.22:0/1638930743 wait complete. 2024-09-19T08:24:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.836+0000 7fe821f7a640 1 Processor -- start 2024-09-19T08:24:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe821f7a640 1 -- start start 2024-09-19T08:24:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe821f7a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81c100a80 0x7fe81c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe821f7a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe81c101420 0x7fe81c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe821f7a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe81c106a80 0x7fe81c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe821f7a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c110e80 con 0x7fe81c106a80 2024-09-19T08:24:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe821f7a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c110d00 con 0x7fe81c101420 2024-09-19T08:24:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe821f7a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c111000 con 0x7fe81c100a80 2024-09-19T08:24:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe81bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe81c106a80 0x7fe81c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe81bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe81c106a80 0x7fe81c19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50916/0 (socket says 172.21.15.22:50916) 2024-09-19T08:24:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe81bfff640 1 -- 172.21.15.22:0/3979382188 learned_addr learned my addr 172.21.15.22:0/3979382188 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe81affd640 1 --2- 172.21.15.22:0/3979382188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe81c101420 0x7fe81c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe81b7fe640 1 --2- 172.21.15.22:0/3979382188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81c100a80 0x7fe81c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe81bfff640 1 -- 172.21.15.22:0/3979382188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81c100a80 msgr2=0x7fe81c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe81bfff640 1 --2- 172.21.15.22:0/3979382188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81c100a80 0x7fe81c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe81bfff640 1 -- 172.21.15.22:0/3979382188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe81c101420 msgr2=0x7fe81c198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe81bfff640 1 --2- 172.21.15.22:0/3979382188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe81c101420 0x7fe81c198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.837+0000 7fe81bfff640 1 -- 172.21.15.22:0/3979382188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81c19d4a0 con 0x7fe81c106a80 2024-09-19T08:24:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.838+0000 7fe81bfff640 1 --2- 172.21.15.22:0/3979382188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe81c106a80 0x7fe81c19cdc0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe81400efd0 tx=0x7fe81400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.838+0000 7fe818ff9640 1 -- 172.21.15.22:0/3979382188 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814003b70 con 0x7fe81c106a80 2024-09-19T08:24:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.838+0000 7fe818ff9640 1 -- 172.21.15.22:0/3979382188 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe814003d10 con 0x7fe81c106a80 2024-09-19T08:24:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.838+0000 7fe818ff9640 1 -- 172.21.15.22:0/3979382188 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8140109f0 con 0x7fe81c106a80 2024-09-19T08:24:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.838+0000 7fe821f7a640 1 -- 172.21.15.22:0/3979382188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81c19d790 con 0x7fe81c106a80 2024-09-19T08:24:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.838+0000 7fe821f7a640 1 -- 172.21.15.22:0/3979382188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe81c1a5070 con 0x7fe81c106a80 2024-09-19T08:24:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.839+0000 7fe821f7a640 1 -- 172.21.15.22:0/3979382188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe81c100ee0 con 0x7fe81c106a80 2024-09-19T08:24:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.840+0000 7fe818ff9640 1 -- 172.21.15.22:0/3979382188 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe814004020 con 0x7fe81c106a80 2024-09-19T08:24:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.841+0000 7fe818ff9640 1 --2- 172.21.15.22:0/3979382188 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe8080a3430 0x7fe8080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.841+0000 7fe81b7fe640 1 --2- 172.21.15.22:0/3979382188 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe8080a3430 0x7fe8080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.841+0000 7fe818ff9640 1 -- 172.21.15.22:0/3979382188 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fe8140d0070 con 0x7fe81c106a80 2024-09-19T08:24:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.842+0000 7fe81b7fe640 1 --2- 172.21.15.22:0/3979382188 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe8080a3430 0x7fe8080a58f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009900 tx=0x7fe80c0058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:37.845+0000 7fe818ff9640 1 -- 172.21.15.22:0/3979382188 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe81409ca40 con 0x7fe81c106a80 2024-09-19T08:24:38.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:38.058+0000 7fe821f7a640 1 -- 172.21.15.22:0/3979382188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe81c02d7c0 con 0x7fe81c106a80 2024-09-19T08:24:38.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:38.058+0000 7fe818ff9640 1 -- 172.21.15.22:0/3979382188 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11908 (secure 0 0 0) 0x7fe81409c340 con 0x7fe81c106a80 2024-09-19T08:24:38.060 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:24:38.060 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"btime":"2024-09-19T08:23:33:502329+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:23:33.442439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24479,24460]},"id":1}]} 2024-09-19T08:24:38.060 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-09-19T08:24:38.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:38.061+0000 7fe821f7a640 1 -- 172.21.15.22:0/3979382188 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe8080a3430 msgr2=0x7fe8080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:38.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:38.061+0000 7fe821f7a640 1 --2- 172.21.15.22:0/3979382188 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe8080a3430 0x7fe8080a58f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009900 tx=0x7fe80c0058d0 comp rx=0 tx=0).stop 2024-09-19T08:24:38.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:38.061+0000 7fe821f7a640 1 -- 172.21.15.22:0/3979382188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe81c106a80 msgr2=0x7fe81c19cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:38.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:38.061+0000 7fe821f7a640 1 --2- 172.21.15.22:0/3979382188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe81c106a80 0x7fe81c19cdc0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe81400efd0 tx=0x7fe81400c6a0 comp rx=0 tx=0).stop 2024-09-19T08:24:38.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:38.061+0000 7fe821f7a640 1 -- 172.21.15.22:0/3979382188 shutdown_connections 2024-09-19T08:24:38.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:38.061+0000 7fe821f7a640 1 --2- 172.21.15.22:0/3979382188 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe8080a3430 0x7fe8080a58f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:38.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:38.061+0000 7fe821f7a640 1 --2- 172.21.15.22:0/3979382188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe81c106a80 0x7fe81c19cdc0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:38.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:38.061+0000 7fe821f7a640 1 --2- 172.21.15.22:0/3979382188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe81c101420 0x7fe81c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:38.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:38.061+0000 7fe821f7a640 1 --2- 172.21.15.22:0/3979382188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81c100a80 0x7fe81c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:38.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:38.061+0000 7fe821f7a640 1 -- 172.21.15.22:0/3979382188 >> 172.21.15.22:0/3979382188 conn(0x7fe81c0fc7a0 msgr2=0x7fe81c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:38.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:38.061+0000 7fe821f7a640 1 -- 172.21.15.22:0/3979382188 shutdown_connections 2024-09-19T08:24:38.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:38.061+0000 7fe821f7a640 1 -- 172.21.15.22:0/3979382188 wait complete. 2024-09-19T08:24:38.877 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:38 smithi022 ceph-mon[30963]: pgmap v297: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 510 op/s 2024-09-19T08:24:38.877 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:38 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3979382188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:24:38.911 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --cancel --all' 2024-09-19T08:24:38.911 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --cancel --all 2024-09-19T08:24:39.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:38 smithi157 ceph-mon[35143]: pgmap v297: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 510 op/s 2024-09-19T08:24:39.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:38 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3979382188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:24:39.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:38 smithi064 ceph-mon[34900]: pgmap v297: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 510 op/s 2024-09-19T08:24:39.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:38 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3979382188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:24:39.175 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:24:39.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.666+0000 7fc69b3e9640 1 -- 172.21.15.22:0/4151963253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc694106a80 msgr2=0x7fc694106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.666+0000 7fc69b3e9640 1 --2- 172.21.15.22:0/4151963253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc694106a80 0x7fc694106e60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc690009930 tx=0x7fc69002a420 comp rx=0 tx=0).stop 2024-09-19T08:24:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.666+0000 7fc69b3e9640 1 -- 172.21.15.22:0/4151963253 shutdown_connections 2024-09-19T08:24:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.666+0000 7fc69b3e9640 1 --2- 172.21.15.22:0/4151963253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc694101420 0x7fc69410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.666+0000 7fc69b3e9640 1 --2- 172.21.15.22:0/4151963253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc694100a80 0x7fc694100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.666+0000 7fc69b3e9640 1 --2- 172.21.15.22:0/4151963253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc694106a80 0x7fc694106e60 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.666+0000 7fc69b3e9640 1 -- 172.21.15.22:0/4151963253 >> 172.21.15.22:0/4151963253 conn(0x7fc6940fc7a0 msgr2=0x7fc6940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.666+0000 7fc69b3e9640 1 -- 172.21.15.22:0/4151963253 shutdown_connections 2024-09-19T08:24:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.666+0000 7fc69b3e9640 1 -- 172.21.15.22:0/4151963253 wait complete. 2024-09-19T08:24:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc69b3e9640 1 Processor -- start 2024-09-19T08:24:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc69b3e9640 1 -- start start 2024-09-19T08:24:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc69b3e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc694100a80 0x7fc6941983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc69b3e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc694101420 0x7fc694198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc69b3e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc694106a80 0x7fc69419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc69b3e9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc694110e80 con 0x7fc694106a80 2024-09-19T08:24:39.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc69b3e9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc694110d00 con 0x7fc694100a80 2024-09-19T08:24:39.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc69b3e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc694111000 con 0x7fc694101420 2024-09-19T08:24:39.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc699986640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc694106a80 0x7fc69419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc699986640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc694106a80 0x7fc69419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50920/0 (socket says 172.21.15.22:50920) 2024-09-19T08:24:39.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc699986640 1 -- 172.21.15.22:0/1598686850 learned_addr learned my addr 172.21.15.22:0/1598686850 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:39.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc699185640 1 --2- 172.21.15.22:0/1598686850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc694100a80 0x7fc6941983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc698984640 1 --2- 172.21.15.22:0/1598686850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc694101420 0x7fc694198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc699986640 1 -- 172.21.15.22:0/1598686850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc694101420 msgr2=0x7fc694198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:39.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc699986640 1 --2- 172.21.15.22:0/1598686850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc694101420 0x7fc694198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc699986640 1 -- 172.21.15.22:0/1598686850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc694100a80 msgr2=0x7fc6941983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:39.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.667+0000 7fc699986640 1 --2- 172.21.15.22:0/1598686850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc694100a80 0x7fc6941983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.668+0000 7fc699986640 1 -- 172.21.15.22:0/1598686850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69419d380 con 0x7fc694106a80 2024-09-19T08:24:39.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.668+0000 7fc699986640 1 --2- 172.21.15.22:0/1598686850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc694106a80 0x7fc69419cca0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00cce0 tx=0x7fc68c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:39.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.668+0000 7fc68a7fc640 1 -- 172.21.15.22:0/1598686850 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c00f450 con 0x7fc694106a80 2024-09-19T08:24:39.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.668+0000 7fc68a7fc640 1 -- 172.21.15.22:0/1598686850 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc68c00fad0 con 0x7fc694106a80 2024-09-19T08:24:39.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.668+0000 7fc69b3e9640 1 -- 172.21.15.22:0/1598686850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69419d670 con 0x7fc694106a80 2024-09-19T08:24:39.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.668+0000 7fc68a7fc640 1 -- 172.21.15.22:0/1598686850 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c018c10 con 0x7fc694106a80 2024-09-19T08:24:39.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.668+0000 7fc69b3e9640 1 -- 172.21.15.22:0/1598686850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6941a4f50 con 0x7fc694106a80 2024-09-19T08:24:39.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.670+0000 7fc68a7fc640 1 -- 172.21.15.22:0/1598686850 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc68c00f5f0 con 0x7fc694106a80 2024-09-19T08:24:39.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc69d7c2740 1 Processor -- start 2024-09-19T08:24:39.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc69d7c2740 1 -- start start 2024-09-19T08:24:39.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc69d7c2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc363f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc69d7c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 0x55bc363fc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc69d7c2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 0x55bc363fefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc69d7c2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bc36356200 con 0x55bc363f1a00 2024-09-19T08:24:39.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc69d7c2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bc362fcdc0 con 0x55bc363fcbe0 2024-09-19T08:24:39.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc69d7c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bc363521c0 con 0x55bc363f2320 2024-09-19T08:24:39.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc698984640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc363f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc698984640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc363f1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50926/0 (socket says 172.21.15.22:50926) 2024-09-19T08:24:39.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc698984640 1 -- 172.21.15.22:0/2899044351 learned_addr learned my addr 172.21.15.22:0/2899044351 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:39.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc68a7fc640 1 --2- 172.21.15.22:0/1598686850 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc6800a3430 0x7fc6800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc699185640 1 --2- 172.21.15.22:0/2899044351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 0x55bc363fc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc698984640 1 --2- 172.21.15.22:0/2899044351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 0x55bc363fefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.671+0000 7fc699986640 1 --2- 172.21.15.22:0/1598686850 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc6800a3430 0x7fc6800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.672+0000 7fc68a7fc640 1 -- 172.21.15.22:0/1598686850 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fc68c0d0340 con 0x7fc694106a80 2024-09-19T08:24:39.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.672+0000 7fc698984640 1 -- 172.21.15.22:0/2899044351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 msgr2=0x55bc363fc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:39.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.672+0000 7fc698984640 1 --2- 172.21.15.22:0/2899044351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 0x55bc363fc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.672+0000 7fc698984640 1 -- 172.21.15.22:0/2899044351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 msgr2=0x55bc363fefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:39.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.672+0000 7fc698984640 1 --2- 172.21.15.22:0/2899044351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 0x55bc363fefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.672+0000 7fc698984640 1 -- 172.21.15.22:0/2899044351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc360c7620 con 0x55bc363f1a00 2024-09-19T08:24:39.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.672+0000 7fc698984640 1 --2- 172.21.15.22:0/2899044351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc363f1de0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc684005d90 tx=0x7fc68400c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f26b786d1519f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:39.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.672+0000 7fc67f7fe640 1 -- 172.21.15.22:0/2899044351 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684018030 con 0x55bc363f1a00 2024-09-19T08:24:39.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.672+0000 7fc699986640 1 --2- 172.21.15.22:0/1598686850 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc6800a3430 0x7fc6800a58f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc68c09c2b0 tx=0x7fc68c06c1d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:39.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.672+0000 7fc67f7fe640 1 -- 172.21.15.22:0/2899044351 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc684005500 con 0x55bc363f1a00 2024-09-19T08:24:39.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.672+0000 7fc67f7fe640 1 -- 172.21.15.22:0/2899044351 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68400d7b0 con 0x55bc363f1a00 2024-09-19T08:24:39.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.673+0000 7fc69d7c2740 1 -- 172.21.15.22:0/2899044351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 msgr2=0x55bc363f1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:39.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.673+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/2899044351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc363f1de0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc684005d90 tx=0x7fc68400c5e0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.673+0000 7fc69d7c2740 1 -- 172.21.15.22:0/2899044351 shutdown_connections 2024-09-19T08:24:39.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.673+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/2899044351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 0x55bc363fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.673+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/2899044351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 0x55bc363fc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.673+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/2899044351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc363f1de0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.673+0000 7fc69d7c2740 1 -- 172.21.15.22:0/2899044351 >> 172.21.15.22:0/2899044351 conn(0x55bc363edf90 msgr2=0x55bc363ee370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:39.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.673+0000 7fc69d7c2740 1 -- 172.21.15.22:0/2899044351 shutdown_connections 2024-09-19T08:24:39.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.673+0000 7fc69d7c2740 1 -- 172.21.15.22:0/2899044351 wait complete. 2024-09-19T08:24:39.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.673+0000 7fc69d7c2740 1 -- start start 2024-09-19T08:24:39.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.673+0000 7fc69d7c2740 1 Processor -- start 2024-09-19T08:24:39.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc69d7c2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc363fc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc69d7c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 0x55bc363f9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc69d7c2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 0x55bc363f9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc69d7c2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bc363ff7b0 con 0x55bc363f1a00 2024-09-19T08:24:39.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc69d7c2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bc362f0c50 con 0x55bc363fcbe0 2024-09-19T08:24:39.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc69d7c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bc363ff930 con 0x55bc363f2320 2024-09-19T08:24:39.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc69d7c2740 10 client.0 _ll_register_callbacks cb 0x55bc36302ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:24:39.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc698984640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc363fc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc698984640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc363fc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50940/0 (socket says 172.21.15.22:50940) 2024-09-19T08:24:39.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc698984640 1 -- 172.21.15.22:0/1252815025 learned_addr learned my addr 172.21.15.22:0/1252815025 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:39.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc699185640 1 --2- 172.21.15.22:0/1252815025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 0x55bc363f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc698984640 1 -- 172.21.15.22:0/1252815025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 msgr2=0x55bc363f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:39.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc698984640 1 --2- 172.21.15.22:0/1252815025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 0x55bc363f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc698984640 1 -- 172.21.15.22:0/1252815025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 msgr2=0x55bc363f9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:39.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc698984640 1 --2- 172.21.15.22:0/1252815025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 0x55bc363f9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc698984640 1 -- 172.21.15.22:0/1252815025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc363fa440 con 0x55bc363f1a00 2024-09-19T08:24:39.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc699185640 1 --2- 172.21.15.22:0/1252815025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 0x55bc363f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:24:39.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc698984640 1 --2- 172.21.15.22:0/1252815025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc363fc470 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc684004e20 tx=0x7fc684004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e015b1125fe51d31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:39.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc67cff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:24:39.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc67cff9640 1 -- 172.21.15.22:0/1252815025 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68400d750 con 0x55bc363f1a00 2024-09-19T08:24:39.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc69d7c2740 1 -- 172.21.15.22:0/1252815025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc684005500 con 0x55bc363f1a00 2024-09-19T08:24:39.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.674+0000 7fc67cff9640 1 -- 172.21.15.22:0/1252815025 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fc68400d8f0 con 0x55bc363f1a00 2024-09-19T08:24:39.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.675+0000 7fc67cff9640 1 -- 172.21.15.22:0/1252815025 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc68400daf0 con 0x55bc363f1a00 2024-09-19T08:24:39.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.675+0000 7fc69d7c2740 10 client.17079 fetch_fsmap learned FSMap version 36 2024-09-19T08:24:39.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.675+0000 7fc69d7c2740 1 -- 172.21.15.22:0/1252815025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bc3640b280 con 0x55bc363f1a00 2024-09-19T08:24:39.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.675+0000 7fc67cff9640 1 -- 172.21.15.22:0/1252815025 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc684005500 con 0x55bc363f1a00 2024-09-19T08:24:39.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.675+0000 7fc67cff9640 1 -- 172.21.15.22:0/1252815025 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fc684027b80 con 0x55bc363f1a00 2024-09-19T08:24:39.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.675+0000 7fc69d7c2740 10 client.17079 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:24:39.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.675+0000 7fc69d7c2740 10 client.17079 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:24:39.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.675+0000 7fc69d7c2740 20 client.17079 populate_metadata read hostname 'smithi022' 2024-09-19T08:24:39.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.675+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/1252815025 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55bc3640c6e0 0x55bc3640ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.675+0000 7fc69d7c2740 4 client.17079 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:24:39.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.675+0000 7fc69d7c2740 1 -- 172.21.15.22:0/1252815025 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc36363770 con 0x55bc3640c6e0 2024-09-19T08:24:39.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.675+0000 7fc699185640 1 --2- 172.21.15.22:0/1252815025 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55bc3640c6e0 0x55bc3640ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.676+0000 7fc699185640 1 --2- 172.21.15.22:0/1252815025 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55bc3640c6e0 0x55bc3640ead0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db97daf42ef3d9ea server_cookie=d6611471a58a2eb in_seq=0 out_seq=0 2024-09-19T08:24:39.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.676+0000 7fc67cff9640 10 client.17079 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:39.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.676+0000 7fc67cff9640 1 -- 172.21.15.22:0/1252815025 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc36363770 con 0x55bc3640c6e0 2024-09-19T08:24:39.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.676+0000 7fc67cff9640 10 client.17079 handle_command_reply: tid=0 2024-09-19T08:24:39.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.676+0000 7fc69d7c2740 1 client.17079 shutdown 2024-09-19T08:24:39.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.677+0000 7fc69d7c2740 1 -- 172.21.15.22:0/1252815025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 msgr2=0x55bc363fc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:39.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.677+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/1252815025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc363fc470 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc684004e20 tx=0x7fc684004ec0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.677+0000 7fc69d7c2740 1 -- 172.21.15.22:0/1252815025 shutdown_connections 2024-09-19T08:24:39.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.677+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/1252815025 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55bc3640c6e0 0x55bc3640ead0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.677+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/1252815025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 0x55bc363f9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.677+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/1252815025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 0x55bc363f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.677+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/1252815025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc363fc470 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.677+0000 7fc69d7c2740 1 -- 172.21.15.22:0/1252815025 >> 172.21.15.22:0/1252815025 conn(0x55bc363edf90 msgr2=0x55bc363ed1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:39.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.677+0000 7fc67cff9640 0 client.17079 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:39.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.677+0000 7fc69d7c2740 1 -- 172.21.15.22:0/1252815025 shutdown_connections 2024-09-19T08:24:39.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.677+0000 7fc69d7c2740 1 -- 172.21.15.22:0/1252815025 wait complete. 2024-09-19T08:24:39.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.677+0000 7fc69d7c2740 20 client.17079 trim_cache size 0 max 16384 2024-09-19T08:24:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.694+0000 7fc69d7c2740 1 Processor -- start 2024-09-19T08:24:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.694+0000 7fc69d7c2740 1 -- start start 2024-09-19T08:24:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.694+0000 7fc69d7c2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc36408b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.694+0000 7fc69d7c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 0x55bc363f9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.694+0000 7fc69d7c2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 0x55bc363f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.694+0000 7fc69d7c2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941b0210 con 0x55bc363f1a00 2024-09-19T08:24:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.694+0000 7fc69d7c2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941ada80 con 0x55bc363fcbe0 2024-09-19T08:24:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.694+0000 7fc69d7c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941ab8f0 con 0x55bc363f2320 2024-09-19T08:24:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.695+0000 7fc698984640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc36408b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.695+0000 7fc698984640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc36408b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50952/0 (socket says 172.21.15.22:50952) 2024-09-19T08:24:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.695+0000 7fc698984640 1 -- 172.21.15.22:0/1751177812 learned_addr learned my addr 172.21.15.22:0/1751177812 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.695+0000 7fc698984640 1 -- 172.21.15.22:0/1751177812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 msgr2=0x55bc363f9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.695+0000 7fc698984640 1 --2- 172.21.15.22:0/1751177812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 0x55bc363f9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.695+0000 7fc698984640 1 -- 172.21.15.22:0/1751177812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 msgr2=0x55bc363f4d50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:24:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.695+0000 7fc698984640 1 --2- 172.21.15.22:0/1751177812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 0x55bc363f4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.695+0000 7fc698984640 1 -- 172.21.15.22:0/1751177812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc363f86d0 con 0x55bc363f1a00 2024-09-19T08:24:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.695+0000 7fc698984640 1 --2- 172.21.15.22:0/1751177812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc36408b90 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc684012de0 tx=0x7fc684004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4eb2007dd88504 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.695+0000 7fc67effd640 1 -- 172.21.15.22:0/1751177812 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684033070 con 0x55bc363f1a00 2024-09-19T08:24:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.695+0000 7fc67effd640 1 -- 172.21.15.22:0/1751177812 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc684022070 con 0x55bc363f1a00 2024-09-19T08:24:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.695+0000 7fc67effd640 1 -- 172.21.15.22:0/1751177812 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684018050 con 0x55bc363f1a00 2024-09-19T08:24:39.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.696+0000 7fc69d7c2740 1 -- 172.21.15.22:0/1751177812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 msgr2=0x55bc36408b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.696+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/1751177812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc36408b90 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc684012de0 tx=0x7fc684004d90 comp rx=0 tx=0).stop 2024-09-19T08:24:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.696+0000 7fc69d7c2740 1 -- 172.21.15.22:0/1751177812 shutdown_connections 2024-09-19T08:24:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.696+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/1751177812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 0x55bc363f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.696+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/1751177812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f2320 0x55bc363f9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.696+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/1751177812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f1a00 0x55bc36408b90 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.696+0000 7fc69d7c2740 1 -- 172.21.15.22:0/1751177812 >> 172.21.15.22:0/1751177812 conn(0x55bc363edf90 msgr2=0x55bc363f4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.696+0000 7fc69d7c2740 1 -- 172.21.15.22:0/1751177812 shutdown_connections 2024-09-19T08:24:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.696+0000 7fc69d7c2740 1 -- 172.21.15.22:0/1751177812 wait complete. 2024-09-19T08:24:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.696+0000 7fc69d7c2740 1 -- start start 2024-09-19T08:24:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.696+0000 7fc69d7c2740 1 Processor -- start 2024-09-19T08:24:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc69d7c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f1a00 0x55bc3640ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc69d7c2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f2320 0x55bc3640f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc69d7c2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 0x55bc36475010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc69d7c2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941abf40 con 0x55bc363f2320 2024-09-19T08:24:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc69d7c2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941ad3b0 con 0x55bc363fcbe0 2024-09-19T08:24:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc69d7c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941ac610 con 0x55bc363f1a00 2024-09-19T08:24:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc69d7c2740 10 client.0 _ll_register_callbacks cb 0x7fc68400c820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:24:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc699185640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f2320 0x55bc3640f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc699185640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f2320 0x55bc3640f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50954/0 (socket says 172.21.15.22:50954) 2024-09-19T08:24:39.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc699185640 1 -- 172.21.15.22:0/3270560677 learned_addr learned my addr 172.21.15.22:0/3270560677 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:24:39.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc698984640 1 --2- 172.21.15.22:0/3270560677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 0x55bc36475010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc698984640 1 --2- 172.21.15.22:0/3270560677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f1a00 0x55bc3640ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc699185640 1 -- 172.21.15.22:0/3270560677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f1a00 msgr2=0x55bc3640ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:39.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc699185640 1 --2- 172.21.15.22:0/3270560677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f1a00 0x55bc3640ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc699185640 1 -- 172.21.15.22:0/3270560677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 msgr2=0x55bc36475010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:24:39.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc699185640 1 --2- 172.21.15.22:0/3270560677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 0x55bc36475010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc699185640 1 -- 172.21.15.22:0/3270560677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc36475600 con 0x55bc363f2320 2024-09-19T08:24:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.697+0000 7fc698984640 1 --2- 172.21.15.22:0/3270560677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f1a00 0x55bc3640ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:24:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc699185640 1 --2- 172.21.15.22:0/3270560677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f2320 0x55bc3640f1b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc69002a940 tx=0x7fc690031ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea07ef83b61b4a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:24:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc67cff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:24:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc67cff9640 1 -- 172.21.15.22:0/3270560677 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69004a070 con 0x55bc363f2320 2024-09-19T08:24:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc67cff9640 1 -- 172.21.15.22:0/3270560677 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fc690005020 con 0x55bc363f2320 2024-09-19T08:24:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc69d7c2740 1 -- 172.21.15.22:0/3270560677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc684022070 con 0x55bc363f2320 2024-09-19T08:24:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc67cff9640 1 -- 172.21.15.22:0/3270560677 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc690055050 con 0x55bc363f2320 2024-09-19T08:24:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc69d7c2740 10 client.17085 fetch_fsmap learned FSMap version 36 2024-09-19T08:24:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc69d7c2740 1 -- 172.21.15.22:0/3270560677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bc36476990 con 0x55bc363f2320 2024-09-19T08:24:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc67cff9640 1 -- 172.21.15.22:0/3270560677 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc684022070 con 0x55bc363f2320 2024-09-19T08:24:39.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc67cff9640 1 -- 172.21.15.22:0/3270560677 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fc6900035f0 con 0x55bc363f2320 2024-09-19T08:24:39.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc69d7c2740 10 client.17085 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:24:39.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc69d7c2740 10 client.17085 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-19T08:24:39.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc69d7c2740 20 client.17085 populate_metadata read hostname 'smithi022' 2024-09-19T08:24:39.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/3270560677 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55bc36477120 0x55bc36479540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:24:39.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc69d7c2740 4 client.17085 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-19T08:24:39.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.698+0000 7fc69d7c2740 1 -- 172.21.15.22:0/3270560677 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55bc3635dc80 con 0x55bc36477120 2024-09-19T08:24:39.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.699+0000 7fc698984640 1 --2- 172.21.15.22:0/3270560677 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55bc36477120 0x55bc36479540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:24:39.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.699+0000 7fc698984640 1 --2- 172.21.15.22:0/3270560677 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55bc36477120 0x55bc36479540 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc2573900cf8e5ce server_cookie=e35cf68ecb0dec0b in_seq=0 out_seq=0 2024-09-19T08:24:39.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.699+0000 7fc67cff9640 10 client.17085 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:39.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.699+0000 7fc67cff9640 1 -- 172.21.15.22:0/3270560677 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55bc3635dc80 con 0x55bc36477120 2024-09-19T08:24:39.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.699+0000 7fc67cff9640 10 client.17085 handle_command_reply: tid=0 2024-09-19T08:24:39.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.699+0000 7fc69d7c2740 1 client.17085 shutdown 2024-09-19T08:24:39.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.699+0000 7fc69d7c2740 1 -- 172.21.15.22:0/3270560677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f2320 msgr2=0x55bc3640f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:39.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.699+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/3270560677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f2320 0x55bc3640f1b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc69002a940 tx=0x7fc690031ab0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.699+0000 7fc69d7c2740 1 -- 172.21.15.22:0/3270560677 shutdown_connections 2024-09-19T08:24:39.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.699+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/3270560677 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55bc36477120 0x55bc36479540 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.699+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/3270560677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc363fcbe0 0x55bc36475010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.699+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/3270560677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc363f2320 0x55bc3640f1b0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69d7c2740 1 --2- 172.21.15.22:0/3270560677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bc363f1a00 0x55bc3640ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.699+0000 7fc67cff9640 0 client.17085 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:24:39.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69d7c2740 1 -- 172.21.15.22:0/3270560677 >> 172.21.15.22:0/3270560677 conn(0x55bc363edf90 msgr2=0x55bc363f3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:39.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69d7c2740 1 -- 172.21.15.22:0/3270560677 shutdown_connections 2024-09-19T08:24:39.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69d7c2740 1 -- 172.21.15.22:0/3270560677 wait complete. 2024-09-19T08:24:39.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69d7c2740 20 client.17085 trim_cache size 0 max 16384 2024-09-19T08:24:39.717 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:24:39.717 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 36, 2024-09-19T08:24:39.718 INFO:teuthology.orchestra.run.smithi022.stdout: "leader": 24481, 2024-09-19T08:24:39.718 INFO:teuthology.orchestra.run.smithi022.stdout: "set_version": 32, 2024-09-19T08:24:39.718 INFO:teuthology.orchestra.run.smithi022.stdout: "sets": {} 2024-09-19T08:24:39.718 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:24:39.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69b3e9640 1 -- 172.21.15.22:0/1598686850 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc6800a3430 msgr2=0x7fc6800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:39.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69b3e9640 1 --2- 172.21.15.22:0/1598686850 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc6800a3430 0x7fc6800a58f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc68c09c2b0 tx=0x7fc68c06c1d0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69b3e9640 1 -- 172.21.15.22:0/1598686850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc694106a80 msgr2=0x7fc69419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:24:39.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69b3e9640 1 --2- 172.21.15.22:0/1598686850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc694106a80 0x7fc69419cca0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00cce0 tx=0x7fc68c00a620 comp rx=0 tx=0).stop 2024-09-19T08:24:39.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69b3e9640 1 -- 172.21.15.22:0/1598686850 shutdown_connections 2024-09-19T08:24:39.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69b3e9640 1 --2- 172.21.15.22:0/1598686850 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc6800a3430 0x7fc6800a58f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69b3e9640 1 --2- 172.21.15.22:0/1598686850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc694106a80 0x7fc69419cca0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69b3e9640 1 --2- 172.21.15.22:0/1598686850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc694101420 0x7fc694198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69b3e9640 1 --2- 172.21.15.22:0/1598686850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc694100a80 0x7fc6941983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:24:39.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.700+0000 7fc69b3e9640 1 -- 172.21.15.22:0/1598686850 >> 172.21.15.22:0/1598686850 conn(0x7fc6940fc7a0 msgr2=0x7fc69406a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:24:39.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.701+0000 7fc69b3e9640 1 -- 172.21.15.22:0/1598686850 shutdown_connections 2024-09-19T08:24:39.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:24:39.701+0000 7fc69b3e9640 1 -- 172.21.15.22:0/1598686850 wait complete. 2024-09-19T08:24:40.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:39 smithi157 ceph-mon[35143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:24:40.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:39 smithi022 ceph-mon[30963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:24:40.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:39 smithi064 ceph-mon[34900]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T08:24:40.596 INFO:tasks.quiescer:done joining Quiescers 2024-09-19T08:24:40.596 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-19T08:24:40.606 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-19T08:24:40.607 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T08:24:40.607 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-09-19T08:24:40.607 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-19T08:24:40.637 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-19T08:24:41.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:40 smithi157 ceph-mon[35143]: pgmap v298: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 635 op/s 2024-09-19T08:24:41.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:40 smithi022 ceph-mon[30963]: pgmap v298: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 635 op/s 2024-09-19T08:24:41.090 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:40 smithi064 ceph-mon[34900]: pgmap v298: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 635 op/s 2024-09-19T08:24:43.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:42 smithi157 ceph-mon[35143]: pgmap v299: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 511 op/s 2024-09-19T08:24:43.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:42 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:24:43.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:42 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:24:43.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:42 smithi022 ceph-mon[30963]: pgmap v299: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 511 op/s 2024-09-19T08:24:43.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:42 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:24:43.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:42 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:24:43.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:42 smithi064 ceph-mon[34900]: pgmap v299: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 511 op/s 2024-09-19T08:24:43.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:42 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:24:43.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:42 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:24:45.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:44 smithi157 ceph-mon[35143]: pgmap v300: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 534 op/s 2024-09-19T08:24:45.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:44 smithi022 ceph-mon[30963]: pgmap v300: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 534 op/s 2024-09-19T08:24:45.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:44 smithi064 ceph-mon[34900]: pgmap v300: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 534 op/s 2024-09-19T08:24:47.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:46 smithi157 ceph-mon[35143]: pgmap v301: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2024-09-19T08:24:47.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:46 smithi022 ceph-mon[30963]: pgmap v301: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2024-09-19T08:24:47.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:46 smithi064 ceph-mon[34900]: pgmap v301: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2024-09-19T08:24:49.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:48 smithi022 ceph-mon[30963]: pgmap v302: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-09-19T08:24:49.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:48 smithi064 ceph-mon[34900]: pgmap v302: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-09-19T08:24:49.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:48 smithi157 ceph-mon[35143]: pgmap v302: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-09-19T08:24:51.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:50 smithi022 ceph-mon[30963]: pgmap v303: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-09-19T08:24:51.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:50 smithi064 ceph-mon[34900]: pgmap v303: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-09-19T08:24:51.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:50 smithi157 ceph-mon[35143]: pgmap v303: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-09-19T08:24:53.055 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:52 smithi157 ceph-mon[35143]: pgmap v304: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 27 op/s 2024-09-19T08:24:53.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:52 smithi064 ceph-mon[34900]: pgmap v304: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 27 op/s 2024-09-19T08:24:53.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:52 smithi022 ceph-mon[30963]: pgmap v304: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 27 op/s 2024-09-19T08:24:55.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:54 smithi064 ceph-mon[34900]: pgmap v305: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 30 op/s 2024-09-19T08:24:55.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:54 smithi157 ceph-mon[35143]: pgmap v305: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 30 op/s 2024-09-19T08:24:55.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:54 smithi022 ceph-mon[30963]: pgmap v305: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 30 op/s 2024-09-19T08:24:57.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:56 smithi064 ceph-mon[34900]: pgmap v306: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 KiB/s wr, 6 op/s 2024-09-19T08:24:57.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:56 smithi157 ceph-mon[35143]: pgmap v306: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 KiB/s wr, 6 op/s 2024-09-19T08:24:57.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:56 smithi022 ceph-mon[30963]: pgmap v306: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 KiB/s wr, 6 op/s 2024-09-19T08:24:59.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:24:58 smithi064 ceph-mon[34900]: pgmap v307: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-19T08:24:59.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:24:58 smithi157 ceph-mon[35143]: pgmap v307: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-19T08:24:59.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:24:58 smithi022 ceph-mon[30963]: pgmap v307: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-19T08:25:01.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:00 smithi064 ceph-mon[34900]: pgmap v308: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-19T08:25:01.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:00 smithi157 ceph-mon[35143]: pgmap v308: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-19T08:25:01.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:00 smithi022 ceph-mon[30963]: pgmap v308: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-19T08:25:02.682 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:25:02.958 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:25:03.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:02 smithi022 ceph-mon[30963]: pgmap v309: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-19T08:25:03.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:02 smithi064 ceph-mon[34900]: pgmap v309: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-19T08:25:03.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:02 smithi157 ceph-mon[35143]: pgmap v309: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-19T08:25:03.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.477+0000 7f05e388d640 1 -- 172.21.15.22:0/2509451659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc101420 msgr2=0x7f05dc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:03.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.477+0000 7f05e388d640 1 --2- 172.21.15.22:0/2509451659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc101420 0x7f05dc10d950 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009930 tx=0x7f05d402a420 comp rx=0 tx=0).stop 2024-09-19T08:25:03.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.477+0000 7f05e388d640 1 -- 172.21.15.22:0/2509451659 shutdown_connections 2024-09-19T08:25:03.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.477+0000 7f05e388d640 1 --2- 172.21.15.22:0/2509451659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc101420 0x7f05dc10d950 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.477+0000 7f05e388d640 1 --2- 172.21.15.22:0/2509451659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05dc100a80 0x7f05dc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.477+0000 7f05e388d640 1 --2- 172.21.15.22:0/2509451659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc106a80 0x7f05dc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.477+0000 7f05e388d640 1 -- 172.21.15.22:0/2509451659 >> 172.21.15.22:0/2509451659 conn(0x7f05dc0fc7a0 msgr2=0x7f05dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:03.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.477+0000 7f05e388d640 1 -- 172.21.15.22:0/2509451659 shutdown_connections 2024-09-19T08:25:03.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.477+0000 7f05e388d640 1 -- 172.21.15.22:0/2509451659 wait complete. 2024-09-19T08:25:03.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.478+0000 7f05e388d640 1 Processor -- start 2024-09-19T08:25:03.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.478+0000 7f05e388d640 1 -- start start 2024-09-19T08:25:03.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.478+0000 7f05e388d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc100a80 0x7f05dc1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.478+0000 7f05e388d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05dc101420 0x7f05dc199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.478+0000 7f05e388d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc106a80 0x7f05dc1a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.478+0000 7f05e388d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc110e80 con 0x7f05dc100a80 2024-09-19T08:25:03.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.478+0000 7f05e388d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc110d00 con 0x7f05dc106a80 2024-09-19T08:25:03.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.478+0000 7f05e388d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc111000 con 0x7f05dc101420 2024-09-19T08:25:03.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.478+0000 7f05e1629640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc100a80 0x7f05dc1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.478+0000 7f05e1629640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc100a80 0x7f05dc1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56272/0 (socket says 172.21.15.22:56272) 2024-09-19T08:25:03.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.478+0000 7f05e1629640 1 -- 172.21.15.22:0/1761677502 learned_addr learned my addr 172.21.15.22:0/1761677502 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:03.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.478+0000 7f05e0e28640 1 --2- 172.21.15.22:0/1761677502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05dc101420 0x7f05dc199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.478+0000 7f05e1e2a640 1 --2- 172.21.15.22:0/1761677502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc106a80 0x7f05dc1a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.479+0000 7f05e1629640 1 -- 172.21.15.22:0/1761677502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05dc101420 msgr2=0x7f05dc199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:03.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.479+0000 7f05e1629640 1 --2- 172.21.15.22:0/1761677502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05dc101420 0x7f05dc199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.479+0000 7f05e1629640 1 -- 172.21.15.22:0/1761677502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc106a80 msgr2=0x7f05dc1a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:03.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.479+0000 7f05e1629640 1 --2- 172.21.15.22:0/1761677502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc106a80 0x7f05dc1a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.479+0000 7f05e1629640 1 -- 172.21.15.22:0/1761677502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05dc1a4780 con 0x7f05dc100a80 2024-09-19T08:25:03.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.479+0000 7f05e1629640 1 --2- 172.21.15.22:0/1761677502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc100a80 0x7f05dc1991c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00cce0 tx=0x7f05cc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:03.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.479+0000 7f05da7fc640 1 -- 172.21.15.22:0/1761677502 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc00f650 con 0x7f05dc100a80 2024-09-19T08:25:03.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.479+0000 7f05da7fc640 1 -- 172.21.15.22:0/1761677502 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05cc00fcd0 con 0x7f05dc100a80 2024-09-19T08:25:03.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.479+0000 7f05e388d640 1 -- 172.21.15.22:0/1761677502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05dc1a4950 con 0x7f05dc100a80 2024-09-19T08:25:03.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.479+0000 7f05da7fc640 1 -- 172.21.15.22:0/1761677502 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc018c70 con 0x7f05dc100a80 2024-09-19T08:25:03.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.479+0000 7f05e388d640 1 -- 172.21.15.22:0/1761677502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05dc1a4e90 con 0x7f05dc100a80 2024-09-19T08:25:03.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.481+0000 7f05da7fc640 1 -- 172.21.15.22:0/1761677502 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f05cc00f7f0 con 0x7f05dc100a80 2024-09-19T08:25:03.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05e5c66740 1 Processor -- start 2024-09-19T08:25:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05e5c66740 1 -- start start 2024-09-19T08:25:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05da7fc640 1 --2- 172.21.15.22:0/1761677502 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f05c40a3430 0x7f05c40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05e5c66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf2004a0 0x55d1bf200880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05e5c66740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf200dc0 0x55d1bf20b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05e1e2a640 1 --2- 172.21.15.22:0/1761677502 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f05c40a3430 0x7f05c40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05e5c66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf20b680 0x55d1bf20da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05e5c66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d1bf1662e0 con 0x55d1bf20b680 2024-09-19T08:25:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05e5c66740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d1bf10dce0 con 0x55d1bf200dc0 2024-09-19T08:25:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05e5c66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d1bf1622a0 con 0x55d1bf2004a0 2024-09-19T08:25:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05da7fc640 1 -- 172.21.15.22:0/1761677502 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f05cc0d0180 con 0x7f05dc100a80 2024-09-19T08:25:03.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05e1e2a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf20b680 0x55d1bf20da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05e1e2a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf20b680 0x55d1bf20da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56280/0 (socket says 172.21.15.22:56280) 2024-09-19T08:25:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05e0e28640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf200dc0 0x55d1bf20b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.482+0000 7f05e1e2a640 1 -- 172.21.15.22:0/2457732586 learned_addr learned my addr 172.21.15.22:0/2457732586 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.483+0000 7f05e1629640 1 --2- 172.21.15.22:0/2457732586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf2004a0 0x55d1bf200880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.483+0000 7f05e1e2a640 1 -- 172.21.15.22:0/2457732586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf2004a0 msgr2=0x55d1bf200880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.483+0000 7f05e1e2a640 1 --2- 172.21.15.22:0/2457732586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf2004a0 0x55d1bf200880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.483+0000 7f05e1e2a640 1 -- 172.21.15.22:0/2457732586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf200dc0 msgr2=0x55d1bf20b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.483+0000 7f05e1e2a640 1 --2- 172.21.15.22:0/2457732586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf200dc0 0x55d1bf20b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.483+0000 7f05e1e2a640 1 -- 172.21.15.22:0/2457732586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1bf0f2440 con 0x55d1bf20b680 2024-09-19T08:25:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.483+0000 7f05e1629640 1 --2- 172.21.15.22:0/2457732586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf2004a0 0x55d1bf200880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:25:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.483+0000 7f05e1e2a640 1 --2- 172.21.15.22:0/1761677502 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f05c40a3430 0x7f05c40a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009230 tx=0x7f05d4032cb0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.483+0000 7f05e1e2a640 1 --2- 172.21.15.22:0/2457732586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf20b680 0x55d1bf20da70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f05d4033000 tx=0x7f05d4045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986031444506b325 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.483+0000 7f05c37fe640 1 -- 172.21.15.22:0/2457732586 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4035070 con 0x55d1bf20b680 2024-09-19T08:25:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.483+0000 7f05c37fe640 1 -- 172.21.15.22:0/2457732586 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05d404f070 con 0x55d1bf20b680 2024-09-19T08:25:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.483+0000 7f05c37fe640 1 -- 172.21.15.22:0/2457732586 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d40035f0 con 0x55d1bf20b680 2024-09-19T08:25:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.484+0000 7f05e5c66740 1 -- 172.21.15.22:0/2457732586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf20b680 msgr2=0x55d1bf20da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.484+0000 7f05e5c66740 1 --2- 172.21.15.22:0/2457732586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf20b680 0x55d1bf20da70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f05d4033000 tx=0x7f05d4045000 comp rx=0 tx=0).stop 2024-09-19T08:25:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.484+0000 7f05e5c66740 1 -- 172.21.15.22:0/2457732586 shutdown_connections 2024-09-19T08:25:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.484+0000 7f05e5c66740 1 --2- 172.21.15.22:0/2457732586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf20b680 0x55d1bf20da70 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.484+0000 7f05e5c66740 1 --2- 172.21.15.22:0/2457732586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf200dc0 0x55d1bf20b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.484+0000 7f05e5c66740 1 --2- 172.21.15.22:0/2457732586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf2004a0 0x55d1bf200880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.484+0000 7f05e5c66740 1 -- 172.21.15.22:0/2457732586 >> 172.21.15.22:0/2457732586 conn(0x55d1bf1fca30 msgr2=0x55d1bf1fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.484+0000 7f05e5c66740 1 -- 172.21.15.22:0/2457732586 shutdown_connections 2024-09-19T08:25:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.484+0000 7f05e5c66740 1 -- 172.21.15.22:0/2457732586 wait complete. 2024-09-19T08:25:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.484+0000 7f05e5c66740 1 -- start start 2024-09-19T08:25:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e5c66740 1 Processor -- start 2024-09-19T08:25:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e5c66740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf2004a0 0x55d1bf20afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e5c66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf200dc0 0x55d1bf208250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e5c66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf20b680 0x55d1bf208790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e5c66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d1bf20e250 con 0x55d1bf200dc0 2024-09-19T08:25:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e5c66740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d1bf121050 con 0x55d1bf2004a0 2024-09-19T08:25:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e5c66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d1bf20e3d0 con 0x55d1bf20b680 2024-09-19T08:25:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e5c66740 10 client.0 _ll_register_callbacks cb 0x55d1bf161160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e0e28640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf200dc0 0x55d1bf208250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e0e28640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf200dc0 0x55d1bf208250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56286/0 (socket says 172.21.15.22:56286) 2024-09-19T08:25:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e0e28640 1 -- 172.21.15.22:0/3154480896 learned_addr learned my addr 172.21.15.22:0/3154480896 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e1e2a640 1 --2- 172.21.15.22:0/3154480896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf20b680 0x55d1bf208790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e1629640 1 --2- 172.21.15.22:0/3154480896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf2004a0 0x55d1bf20afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e0e28640 1 -- 172.21.15.22:0/3154480896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf20b680 msgr2=0x55d1bf208790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:03.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e0e28640 1 --2- 172.21.15.22:0/3154480896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf20b680 0x55d1bf208790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e0e28640 1 -- 172.21.15.22:0/3154480896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf2004a0 msgr2=0x55d1bf20afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:03.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e0e28640 1 --2- 172.21.15.22:0/3154480896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf2004a0 0x55d1bf20afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e0e28640 1 -- 172.21.15.22:0/3154480896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1beed73c0 con 0x55d1bf200dc0 2024-09-19T08:25:03.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.485+0000 7f05e1629640 1 --2- 172.21.15.22:0/3154480896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf2004a0 0x55d1bf20afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:25:03.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05e0e28640 1 --2- 172.21.15.22:0/3154480896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf200dc0 0x55d1bf208250 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f05d000da10 tx=0x7f05d000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687cd941974504db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:03.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:03.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05c0ff9640 1 -- 172.21.15.22:0/3154480896 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0011640 con 0x55d1bf200dc0 2024-09-19T08:25:03.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05c0ff9640 1 -- 172.21.15.22:0/3154480896 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f05d00117e0 con 0x55d1bf200dc0 2024-09-19T08:25:03.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05e5c66740 1 -- 172.21.15.22:0/3154480896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05d404f070 con 0x55d1bf200dc0 2024-09-19T08:25:03.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05c0ff9640 1 -- 172.21.15.22:0/3154480896 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f05d00119e0 con 0x55d1bf200dc0 2024-09-19T08:25:03.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05e5c66740 10 client.17097 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:03.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05e5c66740 1 -- 172.21.15.22:0/3154480896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d1bf219cb0 con 0x55d1bf200dc0 2024-09-19T08:25:03.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05c0ff9640 1 -- 172.21.15.22:0/3154480896 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05d404f070 con 0x55d1bf200dc0 2024-09-19T08:25:03.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05c0ff9640 1 -- 172.21.15.22:0/3154480896 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f05d0019030 con 0x55d1bf200dc0 2024-09-19T08:25:03.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05e5c66740 10 client.17097 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:03.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05e5c66740 10 client.17097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:25:03.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05e5c66740 20 client.17097 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:03.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05e5c66740 1 --2- 172.21.15.22:0/3154480896 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d1bf21b210 0x55d1bf21d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05e5c66740 4 client.17097 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:25:03.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.486+0000 7f05e5c66740 1 -- 172.21.15.22:0/3154480896 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1bf173850 con 0x55d1bf21b210 2024-09-19T08:25:03.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.487+0000 7f05e1629640 1 --2- 172.21.15.22:0/3154480896 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d1bf21b210 0x55d1bf21d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.487+0000 7f05e1629640 1 --2- 172.21.15.22:0/3154480896 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d1bf21b210 0x55d1bf21d600 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9657f03df55d91ae server_cookie=5f106e5063347ce1 in_seq=0 out_seq=0 2024-09-19T08:25:03.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.487+0000 7f05c0ff9640 10 client.17097 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:03.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05c0ff9640 1 -- 172.21.15.22:0/3154480896 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1bf173850 con 0x55d1bf21b210 2024-09-19T08:25:03.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05c0ff9640 10 client.17097 handle_command_reply: tid=0 2024-09-19T08:25:03.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05e5c66740 1 client.17097 shutdown 2024-09-19T08:25:03.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05e5c66740 1 -- 172.21.15.22:0/3154480896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf200dc0 msgr2=0x55d1bf208250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:03.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05e5c66740 1 --2- 172.21.15.22:0/3154480896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf200dc0 0x55d1bf208250 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f05d000da10 tx=0x7f05d000ded0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05e5c66740 1 -- 172.21.15.22:0/3154480896 shutdown_connections 2024-09-19T08:25:03.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05e5c66740 1 --2- 172.21.15.22:0/3154480896 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d1bf21b210 0x55d1bf21d600 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05e5c66740 1 --2- 172.21.15.22:0/3154480896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf20b680 0x55d1bf208790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05e5c66740 1 --2- 172.21.15.22:0/3154480896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf200dc0 0x55d1bf208250 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05e5c66740 1 --2- 172.21.15.22:0/3154480896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf2004a0 0x55d1bf20afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05e5c66740 1 -- 172.21.15.22:0/3154480896 >> 172.21.15.22:0/3154480896 conn(0x55d1bf1fca30 msgr2=0x55d1bf202760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:03.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05c0ff9640 0 client.17097 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:03.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05e5c66740 1 -- 172.21.15.22:0/3154480896 shutdown_connections 2024-09-19T08:25:03.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05e5c66740 1 -- 172.21.15.22:0/3154480896 wait complete. 2024-09-19T08:25:03.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.488+0000 7f05e5c66740 20 client.17097 trim_cache size 0 max 16384 2024-09-19T08:25:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e5c66740 1 Processor -- start 2024-09-19T08:25:03.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e5c66740 1 -- start start 2024-09-19T08:25:03.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e5c66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf1fca30 0x55d1bf217530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e5c66740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf2004a0 0x55d1bf2037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e5c66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf200dc0 0x55d1bf222ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e5c66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc1b0190 con 0x55d1bf1fca30 2024-09-19T08:25:03.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e5c66740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc1ada00 con 0x55d1bf2004a0 2024-09-19T08:25:03.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e5c66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc1ab870 con 0x55d1bf200dc0 2024-09-19T08:25:03.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e1629640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf1fca30 0x55d1bf217530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e1629640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf1fca30 0x55d1bf217530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56296/0 (socket says 172.21.15.22:56296) 2024-09-19T08:25:03.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e1629640 1 -- 172.21.15.22:0/3201726003 learned_addr learned my addr 172.21.15.22:0/3201726003 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:03.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e0e28640 1 --2- 172.21.15.22:0/3201726003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf2004a0 0x55d1bf2037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e1e2a640 1 --2- 172.21.15.22:0/3201726003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf200dc0 0x55d1bf222ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e1629640 1 -- 172.21.15.22:0/3201726003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf200dc0 msgr2=0x55d1bf222ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:03.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e1629640 1 --2- 172.21.15.22:0/3201726003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf200dc0 0x55d1bf222ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e1629640 1 -- 172.21.15.22:0/3201726003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf2004a0 msgr2=0x55d1bf2037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:03.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e1629640 1 --2- 172.21.15.22:0/3201726003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf2004a0 0x55d1bf2037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e1629640 1 -- 172.21.15.22:0/3201726003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1bf2002d0 con 0x55d1bf1fca30 2024-09-19T08:25:03.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e1e2a640 1 --2- 172.21.15.22:0/3201726003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf200dc0 0x55d1bf222ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:25:03.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.506+0000 7f05e1629640 1 --2- 172.21.15.22:0/3201726003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf1fca30 0x55d1bf217530 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f05cc09c150 tx=0x7f05cc00ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945a48f595711b2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:03.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.507+0000 7f05c2ffd640 1 -- 172.21.15.22:0/3201726003 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc085070 con 0x55d1bf1fca30 2024-09-19T08:25:03.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.507+0000 7f05c2ffd640 1 -- 172.21.15.22:0/3201726003 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05cc09cb50 con 0x55d1bf1fca30 2024-09-19T08:25:03.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.507+0000 7f05c2ffd640 1 -- 172.21.15.22:0/3201726003 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc0cdb40 con 0x55d1bf1fca30 2024-09-19T08:25:03.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.507+0000 7f05e5c66740 1 -- 172.21.15.22:0/3201726003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf1fca30 msgr2=0x55d1bf217530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:03.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.507+0000 7f05e5c66740 1 --2- 172.21.15.22:0/3201726003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf1fca30 0x55d1bf217530 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f05cc09c150 tx=0x7f05cc00ac50 comp rx=0 tx=0).stop 2024-09-19T08:25:03.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.507+0000 7f05e5c66740 1 -- 172.21.15.22:0/3201726003 shutdown_connections 2024-09-19T08:25:03.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.507+0000 7f05e5c66740 1 --2- 172.21.15.22:0/3201726003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf200dc0 0x55d1bf222ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.507+0000 7f05e5c66740 1 --2- 172.21.15.22:0/3201726003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf2004a0 0x55d1bf2037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.507+0000 7f05e5c66740 1 --2- 172.21.15.22:0/3201726003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf1fca30 0x55d1bf217530 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.507+0000 7f05e5c66740 1 -- 172.21.15.22:0/3201726003 >> 172.21.15.22:0/3201726003 conn(0x55d1bf2088f0 msgr2=0x55d1bf203090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:03.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.507+0000 7f05e5c66740 1 -- 172.21.15.22:0/3201726003 shutdown_connections 2024-09-19T08:25:03.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.507+0000 7f05e5c66740 1 -- 172.21.15.22:0/3201726003 wait complete. 2024-09-19T08:25:03.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.508+0000 7f05e5c66740 1 -- start start 2024-09-19T08:25:03.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.508+0000 7f05e5c66740 1 Processor -- start 2024-09-19T08:25:03.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.508+0000 7f05e5c66740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf1fca30 0x55d1bf213f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.508+0000 7f05e5c66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf2004a0 0x55d1bf20f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.508+0000 7f05e5c66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf200dc0 0x55d1bf20f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.508+0000 7f05e5c66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc1abec0 con 0x55d1bf2004a0 2024-09-19T08:25:03.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.508+0000 7f05e5c66740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc1ad330 con 0x55d1bf1fca30 2024-09-19T08:25:03.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.508+0000 7f05e5c66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc1ac590 con 0x55d1bf200dc0 2024-09-19T08:25:03.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.508+0000 7f05e5c66740 10 client.0 _ll_register_callbacks cb 0x7f05d0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:03.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.508+0000 7f05e0e28640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf2004a0 0x55d1bf20f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.508+0000 7f05e0e28640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf2004a0 0x55d1bf20f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56304/0 (socket says 172.21.15.22:56304) 2024-09-19T08:25:03.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.508+0000 7f05e0e28640 1 -- 172.21.15.22:0/1595475313 learned_addr learned my addr 172.21.15.22:0/1595475313 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:03.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.508+0000 7f05e1629640 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf1fca30 0x55d1bf213f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05e1e2a640 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf200dc0 0x55d1bf20f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05e0e28640 1 -- 172.21.15.22:0/1595475313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf200dc0 msgr2=0x55d1bf20f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:03.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05e0e28640 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf200dc0 0x55d1bf20f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05e0e28640 1 -- 172.21.15.22:0/1595475313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf1fca30 msgr2=0x55d1bf213f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:03.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05e0e28640 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf1fca30 0x55d1bf213f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05e0e28640 1 -- 172.21.15.22:0/1595475313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1bf210050 con 0x55d1bf2004a0 2024-09-19T08:25:03.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05e1e2a640 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf200dc0 0x55d1bf20f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:03.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05e1629640 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf1fca30 0x55d1bf213f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:25:03.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05e0e28640 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf2004a0 0x55d1bf20f230 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x55d1bf209890 tx=0x7f05d0003a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f16be5e1ffdc45c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:03.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:03.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05c0ff9640 1 -- 172.21.15.22:0/1595475313 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0027070 con 0x55d1bf2004a0 2024-09-19T08:25:03.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05c0ff9640 1 -- 172.21.15.22:0/1595475313 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f05d001a3f0 con 0x55d1bf2004a0 2024-09-19T08:25:03.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05e5c66740 1 -- 172.21.15.22:0/1595475313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05cc09cb50 con 0x55d1bf2004a0 2024-09-19T08:25:03.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05c0ff9640 1 -- 172.21.15.22:0/1595475313 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f05d003a050 con 0x55d1bf2004a0 2024-09-19T08:25:03.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05e5c66740 10 client.17103 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:03.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.509+0000 7f05e5c66740 1 -- 172.21.15.22:0/1595475313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d1bf285f20 con 0x55d1bf2004a0 2024-09-19T08:25:03.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.510+0000 7f05c0ff9640 1 -- 172.21.15.22:0/1595475313 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05cc09cb50 con 0x55d1bf2004a0 2024-09-19T08:25:03.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.510+0000 7f05c0ff9640 1 -- 172.21.15.22:0/1595475313 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f05d0019040 con 0x55d1bf2004a0 2024-09-19T08:25:03.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.510+0000 7f05e5c66740 10 client.17103 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:03.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.510+0000 7f05e5c66740 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:25:03.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.510+0000 7f05e5c66740 20 client.17103 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:03.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.510+0000 7f05e5c66740 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d1bf20b680 0x55d1bf288730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:03.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.510+0000 7f05e5c66740 4 client.17103 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:25:03.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.510+0000 7f05e5c66740 1 -- 172.21.15.22:0/1595475313 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1bf16dd60 con 0x55d1bf20b680 2024-09-19T08:25:03.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.510+0000 7f05e1629640 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d1bf20b680 0x55d1bf288730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:03.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05e1629640 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d1bf20b680 0x55d1bf288730 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1afebe4aa903c006 server_cookie=b3db2e84a62c057a in_seq=0 out_seq=0 2024-09-19T08:25:03.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05c0ff9640 10 client.17103 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:03.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05c0ff9640 1 -- 172.21.15.22:0/1595475313 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d1bf16dd60 con 0x55d1bf20b680 2024-09-19T08:25:03.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05c0ff9640 10 client.17103 handle_command_reply: tid=0 2024-09-19T08:25:03.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05e5c66740 1 client.17103 shutdown 2024-09-19T08:25:03.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05e5c66740 1 -- 172.21.15.22:0/1595475313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf2004a0 msgr2=0x55d1bf20f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:03.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05e5c66740 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf2004a0 0x55d1bf20f230 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x55d1bf209890 tx=0x7f05d0003a60 comp rx=0 tx=0).stop 2024-09-19T08:25:03.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05e5c66740 1 -- 172.21.15.22:0/1595475313 shutdown_connections 2024-09-19T08:25:03.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05e5c66740 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55d1bf20b680 0x55d1bf288730 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05e5c66740 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1bf200dc0 0x55d1bf20f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05e5c66740 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1bf2004a0 0x55d1bf20f230 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05e5c66740 1 --2- 172.21.15.22:0/1595475313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1bf1fca30 0x55d1bf213f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05c0ff9640 0 client.17103 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:03.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05e5c66740 1 -- 172.21.15.22:0/1595475313 >> 172.21.15.22:0/1595475313 conn(0x55d1bf2088f0 msgr2=0x55d1bf201f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:03.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05e5c66740 1 -- 172.21.15.22:0/1595475313 shutdown_connections 2024-09-19T08:25:03.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05e5c66740 1 -- 172.21.15.22:0/1595475313 wait complete. 2024-09-19T08:25:03.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.511+0000 7f05e5c66740 20 client.17103 trim_cache size 0 max 16384 2024-09-19T08:25:03.523 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:25:03.523 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-19T08:25:03.523 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-09-19T08:25:03.524 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:25:03.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.512+0000 7f05e388d640 1 -- 172.21.15.22:0/1761677502 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f05c40a3430 msgr2=0x7f05c40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:03.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.512+0000 7f05e388d640 1 --2- 172.21.15.22:0/1761677502 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f05c40a3430 0x7f05c40a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009230 tx=0x7f05d4032cb0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.512+0000 7f05e388d640 1 -- 172.21.15.22:0/1761677502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc100a80 msgr2=0x7f05dc1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:03.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.512+0000 7f05e388d640 1 --2- 172.21.15.22:0/1761677502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc100a80 0x7f05dc1991c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00cce0 tx=0x7f05cc00a620 comp rx=0 tx=0).stop 2024-09-19T08:25:03.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.512+0000 7f05e388d640 1 -- 172.21.15.22:0/1761677502 shutdown_connections 2024-09-19T08:25:03.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.512+0000 7f05e388d640 1 --2- 172.21.15.22:0/1761677502 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f05c40a3430 0x7f05c40a58f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.512+0000 7f05e388d640 1 --2- 172.21.15.22:0/1761677502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05dc106a80 0x7f05dc1a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.512+0000 7f05e388d640 1 --2- 172.21.15.22:0/1761677502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05dc101420 0x7f05dc199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.512+0000 7f05e388d640 1 --2- 172.21.15.22:0/1761677502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc100a80 0x7f05dc1991c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:03.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.512+0000 7f05e388d640 1 -- 172.21.15.22:0/1761677502 >> 172.21.15.22:0/1761677502 conn(0x7f05dc0fc7a0 msgr2=0x7f05dc0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:03.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.512+0000 7f05e388d640 1 -- 172.21.15.22:0/1761677502 shutdown_connections 2024-09-19T08:25:03.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:03.512+0000 7f05e388d640 1 -- 172.21.15.22:0/1761677502 wait complete. 2024-09-19T08:25:04.344 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:25:04.345 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:25:04.607 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:25:05.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:04 smithi022 ceph-mon[30963]: pgmap v310: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-09-19T08:25:05.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:04 smithi064 ceph-mon[34900]: pgmap v310: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-09-19T08:25:05.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.144+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2511109195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ff4100e20 msgr2=0x7f2ff4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:05.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.144+0000 7f2ff92b3640 1 --2- 172.21.15.22:0/2511109195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ff4100e20 0x7f2ff4101200 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009ca0 tx=0x7f2fe402a950 comp rx=0 tx=0).stop 2024-09-19T08:25:05.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.144+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2511109195 shutdown_connections 2024-09-19T08:25:05.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.144+0000 7f2ff92b3640 1 --2- 172.21.15.22:0/2511109195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff4102ef0 0x7f2ff410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:05.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.144+0000 7f2ff92b3640 1 --2- 172.21.15.22:0/2511109195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff4102570 0x7f2ff41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:05.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.144+0000 7f2ff92b3640 1 --2- 172.21.15.22:0/2511109195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ff4100e20 0x7f2ff4101200 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:05.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.144+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2511109195 >> 172.21.15.22:0/2511109195 conn(0x7f2ff40fc7d0 msgr2=0x7f2ff40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:05.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.144+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2511109195 shutdown_connections 2024-09-19T08:25:05.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.144+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2511109195 wait complete. 2024-09-19T08:25:05.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.145+0000 7f2ff92b3640 1 Processor -- start 2024-09-19T08:25:05.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.145+0000 7f2ff92b3640 1 -- start start 2024-09-19T08:25:05.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.145+0000 7f2ff92b3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff4100e20 0x7f2ff419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:05.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.145+0000 7f2ff92b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff4102570 0x7f2ff419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:05.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.145+0000 7f2ff92b3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ff4102ef0 0x7f2ff419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:05.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.145+0000 7f2ff92b3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff4110f80 con 0x7f2ff4102ef0 2024-09-19T08:25:05.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.145+0000 7f2ff92b3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff4110e00 con 0x7f2ff4102570 2024-09-19T08:25:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.145+0000 7f2ff92b3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff4111100 con 0x7f2ff4100e20 2024-09-19T08:25:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.145+0000 7f2ff37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ff4102ef0 0x7f2ff419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.145+0000 7f2ff37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ff4102ef0 0x7f2ff419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56312/0 (socket says 172.21.15.22:56312) 2024-09-19T08:25:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.145+0000 7f2ff27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff4102570 0x7f2ff419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.145+0000 7f2ff37fe640 1 -- 172.21.15.22:0/2336611580 learned_addr learned my addr 172.21.15.22:0/2336611580 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.145+0000 7f2ff2ffd640 1 --2- 172.21.15.22:0/2336611580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff4100e20 0x7f2ff419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.146+0000 7f2ff37fe640 1 -- 172.21.15.22:0/2336611580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff4100e20 msgr2=0x7f2ff419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.146+0000 7f2ff37fe640 1 --2- 172.21.15.22:0/2336611580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff4100e20 0x7f2ff419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.146+0000 7f2ff37fe640 1 -- 172.21.15.22:0/2336611580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff4102570 msgr2=0x7f2ff419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.146+0000 7f2ff37fe640 1 --2- 172.21.15.22:0/2336611580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff4102570 0x7f2ff419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.146+0000 7f2ff37fe640 1 -- 172.21.15.22:0/2336611580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff419f5b0 con 0x7f2ff4102ef0 2024-09-19T08:25:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.146+0000 7f2ff37fe640 1 --2- 172.21.15.22:0/2336611580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ff4102ef0 0x7f2ff419eed0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00c970 tx=0x7f2fec00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.146+0000 7f2fdbfff640 1 -- 172.21.15.22:0/2336611580 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec007bb0 con 0x7f2ff4102ef0 2024-09-19T08:25:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.146+0000 7f2fdbfff640 1 -- 172.21.15.22:0/2336611580 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fec007d50 con 0x7f2ff4102ef0 2024-09-19T08:25:05.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.146+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2336611580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff419f8a0 con 0x7f2ff4102ef0 2024-09-19T08:25:05.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.146+0000 7f2fdbfff640 1 -- 172.21.15.22:0/2336611580 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec018770 con 0x7f2ff4102ef0 2024-09-19T08:25:05.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.146+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2336611580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff41a7180 con 0x7f2ff4102ef0 2024-09-19T08:25:05.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.147+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2336611580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff4101200 con 0x7f2ff4102ef0 2024-09-19T08:25:05.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.148+0000 7f2fdbfff640 1 -- 172.21.15.22:0/2336611580 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2fec00f450 con 0x7f2ff4102ef0 2024-09-19T08:25:05.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.149+0000 7f2fdbfff640 1 --2- 172.21.15.22:0/2336611580 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2fdc0a33b0 0x7f2fdc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:05.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.149+0000 7f2fdbfff640 1 -- 172.21.15.22:0/2336611580 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f2fec0cfd20 con 0x7f2ff4102ef0 2024-09-19T08:25:05.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.149+0000 7f2ff2ffd640 1 --2- 172.21.15.22:0/2336611580 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2fdc0a33b0 0x7f2fdc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:05.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.149+0000 7f2ff2ffd640 1 --2- 172.21.15.22:0/2336611580 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2fdc0a33b0 0x7f2fdc0a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40093d0 tx=0x7f2fe4009360 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:05.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.153+0000 7f2fdbfff640 1 -- 172.21.15.22:0/2336611580 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2fec014030 con 0x7f2ff4102ef0 2024-09-19T08:25:05.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:04 smithi157 ceph-mon[35143]: pgmap v310: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-09-19T08:25:05.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.364+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2336611580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ff41a73e0 con 0x7f2ff4102ef0 2024-09-19T08:25:05.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.365+0000 7f2fdbfff640 1 -- 172.21.15.22:0/2336611580 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11909 (secure 0 0 0) 0x7f2fec09c690 con 0x7f2ff4102ef0 2024-09-19T08:25:05.365 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:25:05.366 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":36,"btime":"2024-09-19T08:24:38:729727+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:24:38.451170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:25:05.366 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 36 2024-09-19T08:25:05.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.368+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2336611580 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2fdc0a33b0 msgr2=0x7f2fdc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:05.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.368+0000 7f2ff92b3640 1 --2- 172.21.15.22:0/2336611580 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2fdc0a33b0 0x7f2fdc0a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40093d0 tx=0x7f2fe4009360 comp rx=0 tx=0).stop 2024-09-19T08:25:05.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.368+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2336611580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ff4102ef0 msgr2=0x7f2ff419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:05.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.368+0000 7f2ff92b3640 1 --2- 172.21.15.22:0/2336611580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ff4102ef0 0x7f2ff419eed0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00c970 tx=0x7f2fec00cd70 comp rx=0 tx=0).stop 2024-09-19T08:25:05.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.368+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2336611580 shutdown_connections 2024-09-19T08:25:05.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.368+0000 7f2ff92b3640 1 --2- 172.21.15.22:0/2336611580 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f2fdc0a33b0 0x7f2fdc0a5870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:05.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.368+0000 7f2ff92b3640 1 --2- 172.21.15.22:0/2336611580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ff4102ef0 0x7f2ff419eed0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:05.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.368+0000 7f2ff92b3640 1 --2- 172.21.15.22:0/2336611580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff4102570 0x7f2ff419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:05.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.368+0000 7f2ff92b3640 1 --2- 172.21.15.22:0/2336611580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff4100e20 0x7f2ff419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:05.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.368+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2336611580 >> 172.21.15.22:0/2336611580 conn(0x7f2ff40fc7d0 msgr2=0x7f2ff4069fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:05.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.368+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2336611580 shutdown_connections 2024-09-19T08:25:05.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:05.368+0000 7f2ff92b3640 1 -- 172.21.15.22:0/2336611580 wait complete. 2024-09-19T08:25:06.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:05 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2336611580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:25:06.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:05 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2336611580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:25:06.150 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:25:06.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:05 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2336611580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:25:06.412 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:25:06.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.907+0000 7ff7c3351640 1 -- 172.21.15.22:0/3960196769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7bc101420 msgr2=0x7ff7bc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:06.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.907+0000 7ff7c3351640 1 --2- 172.21.15.22:0/3960196769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7bc101420 0x7ff7bc10d950 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009cd0 tx=0x7ff7b402a8e0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.907+0000 7ff7c3351640 1 -- 172.21.15.22:0/3960196769 shutdown_connections 2024-09-19T08:25:06.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.907+0000 7ff7c3351640 1 --2- 172.21.15.22:0/3960196769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7bc101420 0x7ff7bc10d950 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.907+0000 7ff7c3351640 1 --2- 172.21.15.22:0/3960196769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7bc100a80 0x7ff7bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.907+0000 7ff7c3351640 1 --2- 172.21.15.22:0/3960196769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7bc106a80 0x7ff7bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.907+0000 7ff7c3351640 1 -- 172.21.15.22:0/3960196769 >> 172.21.15.22:0/3960196769 conn(0x7ff7bc0fc7a0 msgr2=0x7ff7bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.907+0000 7ff7c3351640 1 -- 172.21.15.22:0/3960196769 shutdown_connections 2024-09-19T08:25:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.907+0000 7ff7c3351640 1 -- 172.21.15.22:0/3960196769 wait complete. 2024-09-19T08:25:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.907+0000 7ff7c3351640 1 Processor -- start 2024-09-19T08:25:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.907+0000 7ff7c3351640 1 -- start start 2024-09-19T08:25:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c3351640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7bc100a80 0x7ff7bc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c3351640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7bc101420 0x7ff7bc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c3351640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7bc106a80 0x7ff7bc19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c3351640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc110e80 con 0x7ff7bc101420 2024-09-19T08:25:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c3351640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc110d00 con 0x7ff7bc100a80 2024-09-19T08:25:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c3351640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc111000 con 0x7ff7bc106a80 2024-09-19T08:25:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c08ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7bc101420 0x7ff7bc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c08ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7bc101420 0x7ff7bc1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56336/0 (socket says 172.21.15.22:56336) 2024-09-19T08:25:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c08ec640 1 -- 172.21.15.22:0/2758047684 learned_addr learned my addr 172.21.15.22:0/2758047684 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:06.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c10ed640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7bc100a80 0x7ff7bc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c18ee640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7bc106a80 0x7ff7bc19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c08ec640 1 -- 172.21.15.22:0/2758047684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7bc106a80 msgr2=0x7ff7bc19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:06.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c08ec640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7bc106a80 0x7ff7bc19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c08ec640 1 -- 172.21.15.22:0/2758047684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7bc100a80 msgr2=0x7ff7bc198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:06.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c08ec640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7bc100a80 0x7ff7bc198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c08ec640 1 -- 172.21.15.22:0/2758047684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7bc19d420 con 0x7ff7bc101420 2024-09-19T08:25:06.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c18ee640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7bc106a80 0x7ff7bc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:06.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.908+0000 7ff7c08ec640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7bc101420 0x7ff7bc1989b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00d770 tx=0x7ff7ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:06.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.909+0000 7ff7b27fc640 1 -- 172.21.15.22:0/2758047684 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac017070 con 0x7ff7bc101420 2024-09-19T08:25:06.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.909+0000 7ff7b27fc640 1 -- 172.21.15.22:0/2758047684 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7ac005500 con 0x7ff7bc101420 2024-09-19T08:25:06.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.909+0000 7ff7c3351640 1 -- 172.21.15.22:0/2758047684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7bc19d710 con 0x7ff7bc101420 2024-09-19T08:25:06.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.909+0000 7ff7b27fc640 1 -- 172.21.15.22:0/2758047684 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac004e00 con 0x7ff7bc101420 2024-09-19T08:25:06.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.909+0000 7ff7c3351640 1 -- 172.21.15.22:0/2758047684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7bc1a4ff0 con 0x7ff7bc101420 2024-09-19T08:25:06.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.911+0000 7ff7b27fc640 1 -- 172.21.15.22:0/2758047684 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff7ac004fa0 con 0x7ff7bc101420 2024-09-19T08:25:06.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.911+0000 7ff7c572a740 1 Processor -- start 2024-09-19T08:25:06.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.911+0000 7ff7c572a740 1 -- start start 2024-09-19T08:25:06.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c572a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290066b40 0x559290065090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c572a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592900655d0 0x559290065a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c572a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559290070370 0x559290072760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c572a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55928ffcb2f0 con 0x559290066b40 2024-09-19T08:25:06.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c572a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55928ff72cf0 con 0x559290070370 2024-09-19T08:25:06.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c572a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55928ffc72b0 con 0x5592900655d0 2024-09-19T08:25:06.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c18ee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290066b40 0x559290065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c18ee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290066b40 0x559290065090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56346/0 (socket says 172.21.15.22:56346) 2024-09-19T08:25:06.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c18ee640 1 -- 172.21.15.22:0/3431391216 learned_addr learned my addr 172.21.15.22:0/3431391216 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:06.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c08ec640 1 --2- 172.21.15.22:0/3431391216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559290070370 0x559290072760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c10ed640 1 --2- 172.21.15.22:0/3431391216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592900655d0 0x559290065a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c18ee640 1 -- 172.21.15.22:0/3431391216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592900655d0 msgr2=0x559290065a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:06.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c18ee640 1 --2- 172.21.15.22:0/3431391216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592900655d0 0x559290065a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c18ee640 1 -- 172.21.15.22:0/3431391216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559290070370 msgr2=0x559290072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:06.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c18ee640 1 --2- 172.21.15.22:0/3431391216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559290070370 0x559290072760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c18ee640 1 -- 172.21.15.22:0/3431391216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928ff573a0 con 0x559290066b40 2024-09-19T08:25:06.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c08ec640 1 --2- 172.21.15.22:0/3431391216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559290070370 0x559290072760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:06.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c10ed640 1 --2- 172.21.15.22:0/3431391216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592900655d0 0x559290065a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:06.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7b27fc640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff7a80a3430 0x7ff7a80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c18ee640 1 --2- 172.21.15.22:0/3431391216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290066b40 0x559290065090 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40315f0 tx=0x7ff7b4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4545e842cc393e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:06.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.912+0000 7ff7c10ed640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff7a80a3430 0x7ff7a80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.913+0000 7ff7b27fc640 1 -- 172.21.15.22:0/2758047684 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ff7ac0cf700 con 0x7ff7bc101420 2024-09-19T08:25:06.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.913+0000 7ff7a77fe640 1 -- 172.21.15.22:0/3431391216 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4031ac0 con 0x559290066b40 2024-09-19T08:25:06.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.913+0000 7ff7a77fe640 1 -- 172.21.15.22:0/3431391216 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b4036040 con 0x559290066b40 2024-09-19T08:25:06.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.913+0000 7ff7a77fe640 1 -- 172.21.15.22:0/3431391216 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4004860 con 0x559290066b40 2024-09-19T08:25:06.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.913+0000 7ff7c572a740 1 -- 172.21.15.22:0/3431391216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290066b40 msgr2=0x559290065090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:06.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.913+0000 7ff7c10ed640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff7a80a3430 0x7ff7a80a58f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x559290062ac0 tx=0x7ff7b800a2d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:06.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.913+0000 7ff7c572a740 1 --2- 172.21.15.22:0/3431391216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290066b40 0x559290065090 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40315f0 tx=0x7ff7b4009a20 comp rx=0 tx=0).stop 2024-09-19T08:25:06.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 -- 172.21.15.22:0/3431391216 shutdown_connections 2024-09-19T08:25:06.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 --2- 172.21.15.22:0/3431391216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559290070370 0x559290072760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 --2- 172.21.15.22:0/3431391216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592900655d0 0x559290065a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 --2- 172.21.15.22:0/3431391216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290066b40 0x559290065090 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 -- 172.21.15.22:0/3431391216 >> 172.21.15.22:0/3431391216 conn(0x55929002e310 msgr2=0x559290061c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:06.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 -- 172.21.15.22:0/3431391216 shutdown_connections 2024-09-19T08:25:06.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 -- 172.21.15.22:0/3431391216 wait complete. 2024-09-19T08:25:06.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 -- start start 2024-09-19T08:25:06.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 Processor -- start 2024-09-19T08:25:06.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592900655d0 0x55929006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559290066b40 0x55929006d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290070370 0x55929006d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559290073010 con 0x559290070370 2024-09-19T08:25:06.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55928ff86060 con 0x5592900655d0 2024-09-19T08:25:06.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.914+0000 7ff7c572a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559290073190 con 0x559290066b40 2024-09-19T08:25:06.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7c572a740 10 client.0 _ll_register_callbacks cb 0x55928fcd98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:06.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7c18ee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290070370 0x55929006d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7c18ee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290070370 0x55929006d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56354/0 (socket says 172.21.15.22:56354) 2024-09-19T08:25:06.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7c18ee640 1 -- 172.21.15.22:0/3515850514 learned_addr learned my addr 172.21.15.22:0/3515850514 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:06.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7c10ed640 1 --2- 172.21.15.22:0/3515850514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592900655d0 0x55929006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7c08ec640 1 --2- 172.21.15.22:0/3515850514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559290066b40 0x55929006d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7c18ee640 1 -- 172.21.15.22:0/3515850514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559290066b40 msgr2=0x55929006d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:06.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7c18ee640 1 --2- 172.21.15.22:0/3515850514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559290066b40 0x55929006d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7c18ee640 1 -- 172.21.15.22:0/3515850514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592900655d0 msgr2=0x55929006fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:06.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7c18ee640 1 --2- 172.21.15.22:0/3515850514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592900655d0 0x55929006fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7c18ee640 1 -- 172.21.15.22:0/3515850514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928fd3c3c0 con 0x559290070370 2024-09-19T08:25:06.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7c18ee640 1 --2- 172.21.15.22:0/3515850514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290070370 0x55929006d7f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4005fb0 tx=0x7ff7b4031b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7594ebbf31b0e4b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:06.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:06.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7a4ff9640 1 -- 172.21.15.22:0/3515850514 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4003d80 con 0x559290070370 2024-09-19T08:25:06.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7a4ff9640 1 -- 172.21.15.22:0/3515850514 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ff7b4003f20 con 0x559290070370 2024-09-19T08:25:06.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.915+0000 7ff7c572a740 1 -- 172.21.15.22:0/3515850514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7b4036040 con 0x559290070370 2024-09-19T08:25:06.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.916+0000 7ff7a4ff9640 1 -- 172.21.15.22:0/3515850514 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff7b4004120 con 0x559290070370 2024-09-19T08:25:06.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.916+0000 7ff7c572a740 10 client.17121 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:06.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.916+0000 7ff7c572a740 1 -- 172.21.15.22:0/3515850514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55929007eb00 con 0x559290070370 2024-09-19T08:25:06.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.916+0000 7ff7a4ff9640 1 -- 172.21.15.22:0/3515850514 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b4036040 con 0x559290070370 2024-09-19T08:25:06.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.916+0000 7ff7a4ff9640 1 -- 172.21.15.22:0/3515850514 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7ff7b4030070 con 0x559290070370 2024-09-19T08:25:06.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.916+0000 7ff7c572a740 10 client.17121 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:06.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.916+0000 7ff7c572a740 10 client.17121 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:25:06.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.916+0000 7ff7c572a740 20 client.17121 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:06.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.916+0000 7ff7c572a740 1 --2- 172.21.15.22:0/3515850514 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55929007fe50 0x559290082240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.916+0000 7ff7c572a740 4 client.17121 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:25:06.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.916+0000 7ff7c572a740 1 -- 172.21.15.22:0/3515850514 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55928ffd8860 con 0x55929007fe50 2024-09-19T08:25:06.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.916+0000 7ff7c10ed640 1 --2- 172.21.15.22:0/3515850514 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55929007fe50 0x559290082240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.917+0000 7ff7c10ed640 1 --2- 172.21.15.22:0/3515850514 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55929007fe50 0x559290082240 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40d389237c53d635 server_cookie=896bf15175b2ce90 in_seq=0 out_seq=0 2024-09-19T08:25:06.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.917+0000 7ff7a4ff9640 10 client.17121 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:25:06.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7a4ff9640 1 -- 172.21.15.22:0/3515850514 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55928ffd8860 con 0x55929007fe50 2024-09-19T08:25:06.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7a4ff9640 10 client.17121 handle_command_reply: tid=0 2024-09-19T08:25:06.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7c572a740 1 client.17121 shutdown 2024-09-19T08:25:06.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7c572a740 1 -- 172.21.15.22:0/3515850514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290070370 msgr2=0x55929006d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:06.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7c572a740 1 --2- 172.21.15.22:0/3515850514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290070370 0x55929006d7f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4005fb0 tx=0x7ff7b4031b70 comp rx=0 tx=0).stop 2024-09-19T08:25:06.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7c572a740 1 -- 172.21.15.22:0/3515850514 shutdown_connections 2024-09-19T08:25:06.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7c572a740 1 --2- 172.21.15.22:0/3515850514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290070370 0x55929006d7f0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7c572a740 1 --2- 172.21.15.22:0/3515850514 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55929007fe50 0x559290082240 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7c572a740 1 --2- 172.21.15.22:0/3515850514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559290066b40 0x55929006d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7c572a740 1 --2- 172.21.15.22:0/3515850514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592900655d0 0x55929006fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7c572a740 1 -- 172.21.15.22:0/3515850514 >> 172.21.15.22:0/3515850514 conn(0x55929002e310 msgr2=0x559290067960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:06.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7a4ff9640 0 client.17121 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:25:06.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7c572a740 1 -- 172.21.15.22:0/3515850514 shutdown_connections 2024-09-19T08:25:06.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7c572a740 1 -- 172.21.15.22:0/3515850514 wait complete. 2024-09-19T08:25:06.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.918+0000 7ff7c572a740 20 client.17121 trim_cache size 0 max 16384 2024-09-19T08:25:06.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.931+0000 7ff7c572a740 1 Processor -- start 2024-09-19T08:25:06.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.931+0000 7ff7c572a740 1 -- start start 2024-09-19T08:25:06.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.931+0000 7ff7c572a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592900655d0 0x559290068120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.931+0000 7ff7c572a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290066b40 0x559290068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.931+0000 7ff7c572a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559290070370 0x559290084e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.931+0000 7ff7c572a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc1b0180 con 0x559290066b40 2024-09-19T08:25:06.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.931+0000 7ff7c572a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc1ad9f0 con 0x5592900655d0 2024-09-19T08:25:06.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.931+0000 7ff7c572a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc1ab860 con 0x559290070370 2024-09-19T08:25:06.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.931+0000 7ff7c08ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290066b40 0x559290068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.931+0000 7ff7c08ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290066b40 0x559290068660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56368/0 (socket says 172.21.15.22:56368) 2024-09-19T08:25:06.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.931+0000 7ff7c08ec640 1 -- 172.21.15.22:0/2429460975 learned_addr learned my addr 172.21.15.22:0/2429460975 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:06.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.931+0000 7ff7c18ee640 1 --2- 172.21.15.22:0/2429460975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559290070370 0x559290084e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.931+0000 7ff7c10ed640 1 --2- 172.21.15.22:0/2429460975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592900655d0 0x559290068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.932+0000 7ff7c08ec640 1 -- 172.21.15.22:0/2429460975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559290070370 msgr2=0x559290084e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:06.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.932+0000 7ff7c08ec640 1 --2- 172.21.15.22:0/2429460975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559290070370 0x559290084e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.932+0000 7ff7c08ec640 1 -- 172.21.15.22:0/2429460975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592900655d0 msgr2=0x559290068120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.932+0000 7ff7c08ec640 1 --2- 172.21.15.22:0/2429460975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592900655d0 0x559290068120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.932+0000 7ff7c08ec640 1 -- 172.21.15.22:0/2429460975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592900614f0 con 0x559290066b40 2024-09-19T08:25:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.932+0000 7ff7c10ed640 1 --2- 172.21.15.22:0/2429460975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592900655d0 0x559290068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.932+0000 7ff7c08ec640 1 --2- 172.21.15.22:0/2429460975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290066b40 0x559290068660 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0983a0 tx=0x7ff7ac009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9404d7f74427729 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.932+0000 7ff7a6ffd640 1 -- 172.21.15.22:0/2429460975 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac020040 con 0x559290066b40 2024-09-19T08:25:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.932+0000 7ff7a6ffd640 1 -- 172.21.15.22:0/2429460975 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7ac09c0d0 con 0x559290066b40 2024-09-19T08:25:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.932+0000 7ff7a6ffd640 1 -- 172.21.15.22:0/2429460975 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac009cc0 con 0x559290066b40 2024-09-19T08:25:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.933+0000 7ff7c572a740 1 -- 172.21.15.22:0/2429460975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290066b40 msgr2=0x559290068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.933+0000 7ff7c572a740 1 --2- 172.21.15.22:0/2429460975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290066b40 0x559290068660 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0983a0 tx=0x7ff7ac009c30 comp rx=0 tx=0).stop 2024-09-19T08:25:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.933+0000 7ff7c572a740 1 -- 172.21.15.22:0/2429460975 shutdown_connections 2024-09-19T08:25:06.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.933+0000 7ff7c572a740 1 --2- 172.21.15.22:0/2429460975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559290070370 0x559290084e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.933+0000 7ff7c572a740 1 --2- 172.21.15.22:0/2429460975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290066b40 0x559290068660 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.933+0000 7ff7c572a740 1 --2- 172.21.15.22:0/2429460975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5592900655d0 0x559290068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.933+0000 7ff7c572a740 1 -- 172.21.15.22:0/2429460975 >> 172.21.15.22:0/2429460975 conn(0x55929002e310 msgr2=0x55929006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:06.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.933+0000 7ff7c572a740 1 -- 172.21.15.22:0/2429460975 shutdown_connections 2024-09-19T08:25:06.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.933+0000 7ff7c572a740 1 -- 172.21.15.22:0/2429460975 wait complete. 2024-09-19T08:25:06.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.933+0000 7ff7c572a740 1 -- start start 2024-09-19T08:25:06.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.933+0000 7ff7c572a740 1 Processor -- start 2024-09-19T08:25:06.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c572a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592900655d0 0x5592900832b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c572a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559290066b40 0x5592900837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c572a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290070370 0x5592900e9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c572a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc1abeb0 con 0x559290070370 2024-09-19T08:25:06.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c572a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc1ad320 con 0x559290066b40 2024-09-19T08:25:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c572a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc1ac580 con 0x5592900655d0 2024-09-19T08:25:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c572a740 10 client.0 _ll_register_callbacks cb 0x7ff7b40314c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c18ee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290070370 0x5592900e9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c18ee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290070370 0x5592900e9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56384/0 (socket says 172.21.15.22:56384) 2024-09-19T08:25:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c18ee640 1 -- 172.21.15.22:0/2714255961 learned_addr learned my addr 172.21.15.22:0/2714255961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c10ed640 1 --2- 172.21.15.22:0/2714255961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592900655d0 0x5592900832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c08ec640 1 --2- 172.21.15.22:0/2714255961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559290066b40 0x5592900837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c18ee640 1 -- 172.21.15.22:0/2714255961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592900655d0 msgr2=0x5592900832b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c18ee640 1 --2- 172.21.15.22:0/2714255961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592900655d0 0x5592900832b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c18ee640 1 -- 172.21.15.22:0/2714255961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559290066b40 msgr2=0x5592900837f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c18ee640 1 --2- 172.21.15.22:0/2714255961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559290066b40 0x5592900837f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c18ee640 1 -- 172.21.15.22:0/2714255961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592900e9cd0 con 0x559290070370 2024-09-19T08:25:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c10ed640 1 --2- 172.21.15.22:0/2714255961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592900655d0 0x5592900832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c18ee640 1 --2- 172.21.15.22:0/2714255961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290070370 0x5592900e9790 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4002230 tx=0x7ff7b4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45664314e3e42f35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7a4ff9640 1 -- 172.21.15.22:0/2714255961 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b405f030 con 0x559290070370 2024-09-19T08:25:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7a4ff9640 1 -- 172.21.15.22:0/2714255961 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ff7b4003b80 con 0x559290070370 2024-09-19T08:25:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.934+0000 7ff7c572a740 1 -- 172.21.15.22:0/2714255961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7ac09c0d0 con 0x559290070370 2024-09-19T08:25:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.935+0000 7ff7a4ff9640 1 -- 172.21.15.22:0/2714255961 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff7b4069050 con 0x559290070370 2024-09-19T08:25:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.935+0000 7ff7c572a740 10 client.17127 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.935+0000 7ff7c572a740 1 -- 172.21.15.22:0/2714255961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5592900eaf30 con 0x559290070370 2024-09-19T08:25:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.935+0000 7ff7a4ff9640 1 -- 172.21.15.22:0/2714255961 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7ac09c0d0 con 0x559290070370 2024-09-19T08:25:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.935+0000 7ff7a4ff9640 1 -- 172.21.15.22:0/2714255961 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7ff7b40305a0 con 0x559290070370 2024-09-19T08:25:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.935+0000 7ff7c572a740 10 client.17127 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.935+0000 7ff7c572a740 10 client.17127 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:25:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.935+0000 7ff7c572a740 20 client.17127 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.935+0000 7ff7c572a740 1 --2- 172.21.15.22:0/2714255961 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5592900eb610 0x5592900eda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.935+0000 7ff7c572a740 4 client.17127 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:25:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.935+0000 7ff7c572a740 1 -- 172.21.15.22:0/2714255961 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55928ffd2d70 con 0x5592900eb610 2024-09-19T08:25:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.935+0000 7ff7c10ed640 1 --2- 172.21.15.22:0/2714255961 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5592900eb610 0x5592900eda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7c10ed640 1 --2- 172.21.15.22:0/2714255961 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5592900eb610 0x5592900eda30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ede2c929d0826a4a server_cookie=ce3dd3ec41546a42 in_seq=0 out_seq=0 2024-09-19T08:25:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7a4ff9640 10 client.17127 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:25:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7a4ff9640 1 -- 172.21.15.22:0/2714255961 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55928ffd2d70 con 0x5592900eb610 2024-09-19T08:25:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7a4ff9640 10 client.17127 handle_command_reply: tid=0 2024-09-19T08:25:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7c572a740 1 client.17127 shutdown 2024-09-19T08:25:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7c572a740 1 -- 172.21.15.22:0/2714255961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290070370 msgr2=0x5592900e9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7c572a740 1 --2- 172.21.15.22:0/2714255961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290070370 0x5592900e9790 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4002230 tx=0x7ff7b4009360 comp rx=0 tx=0).stop 2024-09-19T08:25:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7c572a740 1 -- 172.21.15.22:0/2714255961 shutdown_connections 2024-09-19T08:25:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7c572a740 1 --2- 172.21.15.22:0/2714255961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559290070370 0x5592900e9790 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7c572a740 1 --2- 172.21.15.22:0/2714255961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559290066b40 0x5592900837f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7c572a740 1 --2- 172.21.15.22:0/2714255961 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x5592900eb610 0x5592900eda30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7c572a740 1 --2- 172.21.15.22:0/2714255961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592900655d0 0x5592900832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7c572a740 1 -- 172.21.15.22:0/2714255961 >> 172.21.15.22:0/2714255961 conn(0x55929002e310 msgr2=0x55929006e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7a4ff9640 0 client.17127 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:25:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7c572a740 1 -- 172.21.15.22:0/2714255961 shutdown_connections 2024-09-19T08:25:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.936+0000 7ff7c572a740 1 -- 172.21.15.22:0/2714255961 wait complete. 2024-09-19T08:25:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.937+0000 7ff7c572a740 20 client.17127 trim_cache size 0 max 16384 2024-09-19T08:25:06.947 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:25:06.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.937+0000 7ff7c3351640 1 -- 172.21.15.22:0/2758047684 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff7a80a3430 msgr2=0x7ff7a80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:06.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.937+0000 7ff7c3351640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff7a80a3430 0x7ff7a80a58f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x559290062ac0 tx=0x7ff7b800a2d0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.937+0000 7ff7c3351640 1 -- 172.21.15.22:0/2758047684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7bc101420 msgr2=0x7ff7bc1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:06.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.937+0000 7ff7c3351640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7bc101420 0x7ff7bc1989b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00d770 tx=0x7ff7ac00dc30 comp rx=0 tx=0).stop 2024-09-19T08:25:06.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.937+0000 7ff7c3351640 1 -- 172.21.15.22:0/2758047684 shutdown_connections 2024-09-19T08:25:06.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.937+0000 7ff7c3351640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7ff7a80a3430 0x7ff7a80a58f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.937+0000 7ff7c3351640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7bc106a80 0x7ff7bc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.937+0000 7ff7c3351640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7bc101420 0x7ff7bc1989b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.937+0000 7ff7c3351640 1 --2- 172.21.15.22:0/2758047684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7bc100a80 0x7ff7bc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:06.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.937+0000 7ff7c3351640 1 -- 172.21.15.22:0/2758047684 >> 172.21.15.22:0/2758047684 conn(0x7ff7bc0fc7a0 msgr2=0x7ff7bc06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:06.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.937+0000 7ff7c3351640 1 -- 172.21.15.22:0/2758047684 shutdown_connections 2024-09-19T08:25:06.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:06.937+0000 7ff7c3351640 1 -- 172.21.15.22:0/2758047684 wait complete. 2024-09-19T08:25:07.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:06 smithi064 ceph-mon[34900]: pgmap v311: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-19T08:25:07.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:06 smithi157 ceph-mon[35143]: pgmap v311: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-19T08:25:07.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:06 smithi022 ceph-mon[30963]: pgmap v311: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-19T08:25:07.785 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:25:08.275 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:25:08.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.846+0000 7fd561d10640 1 -- 172.21.15.22:0/1574820074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd55c106a70 msgr2=0x7fd55c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:08.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.846+0000 7fd561d10640 1 --2- 172.21.15.22:0/1574820074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd55c106a70 0x7fd55c106e50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd554009900 tx=0x7fd55402a420 comp rx=0 tx=0).stop 2024-09-19T08:25:08.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.846+0000 7fd561d10640 1 -- 172.21.15.22:0/1574820074 shutdown_connections 2024-09-19T08:25:08.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.846+0000 7fd561d10640 1 --2- 172.21.15.22:0/1574820074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55c101410 0x7fd55c06a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.846+0000 7fd561d10640 1 --2- 172.21.15.22:0/1574820074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd55c100a70 0x7fd55c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.846+0000 7fd561d10640 1 --2- 172.21.15.22:0/1574820074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd55c106a70 0x7fd55c106e50 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.846+0000 7fd561d10640 1 -- 172.21.15.22:0/1574820074 >> 172.21.15.22:0/1574820074 conn(0x7fd55c0fc790 msgr2=0x7fd55c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:08.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.849+0000 7fd561d10640 1 -- 172.21.15.22:0/1574820074 shutdown_connections 2024-09-19T08:25:08.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.851+0000 7fd561d10640 1 -- 172.21.15.22:0/1574820074 wait complete. 2024-09-19T08:25:08.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.852+0000 7fd561d10640 1 Processor -- start 2024-09-19T08:25:08.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.852+0000 7fd561d10640 1 -- start start 2024-09-19T08:25:08.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.852+0000 7fd561d10640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd55c100a70 0x7fd55c1162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.852+0000 7fd561d10640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55c101410 0x7fd55c116810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.853+0000 7fd561d10640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd55c106a70 0x7fd55c11bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.853+0000 7fd561d10640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c10fe80 con 0x7fd55c106a70 2024-09-19T08:25:08.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.853+0000 7fd561d10640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c10fd00 con 0x7fd55c101410 2024-09-19T08:25:08.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.853+0000 7fd561d10640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c110000 con 0x7fd55c100a70 2024-09-19T08:25:08.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.853+0000 7fd55b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd55c100a70 0x7fd55c1162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.853+0000 7fd55b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd55c100a70 0x7fd55c1162d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42348/0 (socket says 172.21.15.22:42348) 2024-09-19T08:25:08.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.853+0000 7fd55b7fe640 1 -- 172.21.15.22:0/3941709597 learned_addr learned my addr 172.21.15.22:0/3941709597 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:08.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.853+0000 7fd55affd640 1 --2- 172.21.15.22:0/3941709597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55c101410 0x7fd55c116810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.853+0000 7fd55b7fe640 1 -- 172.21.15.22:0/3941709597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55c101410 msgr2=0x7fd55c116810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:08.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.853+0000 7fd55b7fe640 1 --2- 172.21.15.22:0/3941709597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55c101410 0x7fd55c116810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.854+0000 7fd55b7fe640 1 -- 172.21.15.22:0/3941709597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd55c106a70 msgr2=0x7fd55c11bff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:25:08.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.854+0000 7fd55b7fe640 1 --2- 172.21.15.22:0/3941709597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd55c106a70 0x7fd55c11bff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.854+0000 7fd55b7fe640 1 -- 172.21.15.22:0/3941709597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c111550 con 0x7fd55c100a70 2024-09-19T08:25:08.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.858+0000 7fd55b7fe640 1 --2- 172.21.15.22:0/3941709597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd55c100a70 0x7fd55c1162d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd554009850 tx=0x7fd554005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:08.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.859+0000 7fd558ff9640 1 -- 172.21.15.22:0/3941709597 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd554040030 con 0x7fd55c100a70 2024-09-19T08:25:08.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.860+0000 7fd558ff9640 1 -- 172.21.15.22:0/3941709597 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd554003e30 con 0x7fd55c100a70 2024-09-19T08:25:08.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.860+0000 7fd561d10640 1 -- 172.21.15.22:0/3941709597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55c1117e0 con 0x7fd55c100a70 2024-09-19T08:25:08.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.861+0000 7fd561d10640 1 -- 172.21.15.22:0/3941709597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd55c111cc0 con 0x7fd55c100a70 2024-09-19T08:25:08.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.861+0000 7fd558ff9640 1 -- 172.21.15.22:0/3941709597 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55403cb40 con 0x7fd55c100a70 2024-09-19T08:25:08.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.866+0000 7fd5640e9740 1 Processor -- start 2024-09-19T08:25:08.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.870+0000 7fd5640e9740 1 -- start start 2024-09-19T08:25:08.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.870+0000 7fd5640e9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da5f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.870+0000 7fd5640e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 0x55a59da5fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.870+0000 7fd5640e9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 0x55a59da6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.870+0000 7fd5640e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a59d9c52f0 con 0x55a59da5f5d0 2024-09-19T08:25:08.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.870+0000 7fd5640e9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a59d96ccf0 con 0x55a59da6a370 2024-09-19T08:25:08.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.870+0000 7fd5640e9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a59d9c12b0 con 0x55a59da60b40 2024-09-19T08:25:08.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.871+0000 7fd55bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da5f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.871+0000 7fd55bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da5f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42354/0 (socket says 172.21.15.22:42354) 2024-09-19T08:25:08.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.871+0000 7fd55bfff640 1 -- 172.21.15.22:0/1799697242 learned_addr learned my addr 172.21.15.22:0/1799697242 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:08.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.871+0000 7fd55b7fe640 1 --2- 172.21.15.22:0/1799697242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 0x55a59da5fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.872+0000 7fd55bfff640 1 -- 172.21.15.22:0/1799697242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 msgr2=0x55a59da6c760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:25:08.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.872+0000 7fd55bfff640 1 --2- 172.21.15.22:0/1799697242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 0x55a59da6c760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.872+0000 7fd55bfff640 1 -- 172.21.15.22:0/1799697242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 msgr2=0x55a59da5fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:08.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.872+0000 7fd55bfff640 1 --2- 172.21.15.22:0/1799697242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 0x55a59da5fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.872+0000 7fd55bfff640 1 -- 172.21.15.22:0/1799697242 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a59d9513a0 con 0x55a59da60b40 2024-09-19T08:25:08.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.872+0000 7fd55b7fe640 1 --2- 172.21.15.22:0/1799697242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 0x55a59da5fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:25:08.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.872+0000 7fd55bfff640 1 --2- 172.21.15.22:0/1799697242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da5f090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd5440096c0 tx=0x7fd544009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd0c7f508ac6d478 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:08.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.873+0000 7fd551ffb640 1 -- 172.21.15.22:0/1799697242 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544015020 con 0x55a59da60b40 2024-09-19T08:25:08.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.873+0000 7fd551ffb640 1 -- 172.21.15.22:0/1799697242 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54400bda0 con 0x55a59da60b40 2024-09-19T08:25:08.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.873+0000 7fd551ffb640 1 -- 172.21.15.22:0/1799697242 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544010380 con 0x55a59da60b40 2024-09-19T08:25:08.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.875+0000 7fd558ff9640 1 -- 172.21.15.22:0/3941709597 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd554043030 con 0x7fd55c100a70 2024-09-19T08:25:08.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.875+0000 7fd5640e9740 1 -- 172.21.15.22:0/1799697242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 msgr2=0x55a59da5f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:08.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.875+0000 7fd5640e9740 1 --2- 172.21.15.22:0/1799697242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da5f090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd5440096c0 tx=0x7fd544009b80 comp rx=0 tx=0).stop 2024-09-19T08:25:08.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.876+0000 7fd5640e9740 1 -- 172.21.15.22:0/1799697242 shutdown_connections 2024-09-19T08:25:08.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.876+0000 7fd5640e9740 1 --2- 172.21.15.22:0/1799697242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 0x55a59da6c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.876+0000 7fd5640e9740 1 --2- 172.21.15.22:0/1799697242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 0x55a59da5fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.876+0000 7fd5640e9740 1 --2- 172.21.15.22:0/1799697242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da5f090 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.876+0000 7fd5640e9740 1 -- 172.21.15.22:0/1799697242 >> 172.21.15.22:0/1799697242 conn(0x55a59da28310 msgr2=0x55a59da5bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:08.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.878+0000 7fd558ff9640 1 --2- 172.21.15.22:0/3941709597 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd5480a3420 0x7fd5480a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.878+0000 7fd55affd640 1 --2- 172.21.15.22:0/3941709597 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd5480a3420 0x7fd5480a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.878+0000 7fd5640e9740 1 -- 172.21.15.22:0/1799697242 shutdown_connections 2024-09-19T08:25:08.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.878+0000 7fd5640e9740 1 -- 172.21.15.22:0/1799697242 wait complete. 2024-09-19T08:25:08.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.878+0000 7fd558ff9640 1 -- 172.21.15.22:0/3941709597 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fd5540f03a0 con 0x7fd55c100a70 2024-09-19T08:25:08.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.879+0000 7fd55affd640 1 --2- 172.21.15.22:0/3941709597 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd5480a3420 0x7fd5480a58e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55a59da60070 tx=0x7fd54c00d040 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:08.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.879+0000 7fd5640e9740 1 -- start start 2024-09-19T08:25:08.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.879+0000 7fd5640e9740 1 Processor -- start 2024-09-19T08:25:08.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.880+0000 7fd5640e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 0x55a59da69cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.880+0000 7fd5640e9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da66f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.880+0000 7fd5640e9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 0x55a59da674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.881+0000 7fd5640e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a59da6d070 con 0x55a59da5f5d0 2024-09-19T08:25:08.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.881+0000 7fd55affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da66f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.881+0000 7fd55affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da66f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42368/0 (socket says 172.21.15.22:42368) 2024-09-19T08:25:08.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.881+0000 7fd55affd640 1 -- 172.21.15.22:0/1849607811 learned_addr learned my addr 172.21.15.22:0/1849607811 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:08.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.881+0000 7fd5640e9740 1 -- 172.21.15.22:0/1849607811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a59d980060 con 0x55a59da6a370 2024-09-19T08:25:08.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.881+0000 7fd5640e9740 1 -- 172.21.15.22:0/1849607811 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a59da6d1f0 con 0x55a59da60b40 2024-09-19T08:25:08.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.881+0000 7fd5640e9740 10 client.0 _ll_register_callbacks cb 0x55a59d6d38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:08.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.881+0000 7fd55b7fe640 1 --2- 172.21.15.22:0/1849607811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 0x55a59da69cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.882+0000 7fd55bfff640 1 --2- 172.21.15.22:0/1849607811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 0x55a59da674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.882+0000 7fd55affd640 1 -- 172.21.15.22:0/1849607811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 msgr2=0x55a59da674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:08.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.882+0000 7fd55affd640 1 --2- 172.21.15.22:0/1849607811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 0x55a59da674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.882+0000 7fd55affd640 1 -- 172.21.15.22:0/1849607811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 msgr2=0x55a59da69cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:08.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.882+0000 7fd55affd640 1 --2- 172.21.15.22:0/1849607811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 0x55a59da69cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.882+0000 7fd55affd640 1 -- 172.21.15.22:0/1849607811 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a59d7363c0 con 0x55a59da60b40 2024-09-19T08:25:08.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.882+0000 7fd55bfff640 1 --2- 172.21.15.22:0/1849607811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 0x55a59da674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:08.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.882+0000 7fd55affd640 1 --2- 172.21.15.22:0/1849607811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da66f60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd54c016e00 tx=0x7fd54c004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12c4f8c7b83e96b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:08.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.883+0000 7fd53affd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:25:08.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.883+0000 7fd53affd640 1 -- 172.21.15.22:0/1849607811 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c025060 con 0x55a59da60b40 2024-09-19T08:25:08.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.883+0000 7fd53affd640 1 -- 172.21.15.22:0/1849607811 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fd54c0205f0 con 0x55a59da60b40 2024-09-19T08:25:08.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.884+0000 7fd5640e9740 1 -- 172.21.15.22:0/1849607811 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd54400bda0 con 0x55a59da60b40 2024-09-19T08:25:08.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.885+0000 7fd53affd640 1 -- 172.21.15.22:0/1849607811 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd54c015e50 con 0x55a59da60b40 2024-09-19T08:25:08.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.885+0000 7fd5640e9740 10 client.26599 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:08.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.885+0000 7fd5640e9740 1 -- 172.21.15.22:0/1849607811 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a59da78b40 con 0x55a59da60b40 2024-09-19T08:25:08.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.887+0000 7fd53affd640 1 -- 172.21.15.22:0/1849607811 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54400bda0 con 0x55a59da60b40 2024-09-19T08:25:08.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.887+0000 7fd53affd640 1 -- 172.21.15.22:0/1849607811 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fd54c0299b0 con 0x55a59da60b40 2024-09-19T08:25:08.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.887+0000 7fd5640e9740 10 client.26599 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:08.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.887+0000 7fd5640e9740 10 client.26599 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:25:08.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.887+0000 7fd5640e9740 20 client.26599 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:08.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.887+0000 7fd5640e9740 1 --2- 172.21.15.22:0/1849607811 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55a59da79f50 0x55a59da7c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.887+0000 7fd5640e9740 4 client.26599 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:25:08.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.887+0000 7fd5640e9740 1 -- 172.21.15.22:0/1849607811 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a59d9d2860 con 0x55a59da79f50 2024-09-19T08:25:08.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.889+0000 7fd55b7fe640 1 --2- 172.21.15.22:0/1849607811 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55a59da79f50 0x55a59da7c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.893+0000 7fd55b7fe640 1 --2- 172.21.15.22:0/1849607811 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55a59da79f50 0x55a59da7c390 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a083173b7ae0a7b server_cookie=8824ac7b1edf23bf in_seq=0 out_seq=0 2024-09-19T08:25:08.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.894+0000 7fd53affd640 10 client.26599 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:25:08.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.894+0000 7fd53affd640 1 -- 172.21.15.22:0/1849607811 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a59d9d2860 con 0x55a59da79f50 2024-09-19T08:25:08.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.894+0000 7fd53affd640 10 client.26599 handle_command_reply: tid=0 2024-09-19T08:25:08.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.894+0000 7fd5640e9740 1 client.26599 shutdown 2024-09-19T08:25:08.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.895+0000 7fd5640e9740 1 -- 172.21.15.22:0/1849607811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 msgr2=0x55a59da66f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:08.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.895+0000 7fd5640e9740 1 --2- 172.21.15.22:0/1849607811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da66f60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd54c016e00 tx=0x7fd54c004b50 comp rx=0 tx=0).stop 2024-09-19T08:25:08.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.896+0000 7fd5640e9740 1 -- 172.21.15.22:0/1849607811 shutdown_connections 2024-09-19T08:25:08.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.896+0000 7fd5640e9740 1 --2- 172.21.15.22:0/1849607811 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55a59da79f50 0x55a59da7c390 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.896+0000 7fd53affd640 0 client.26599 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:25:08.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.896+0000 7fd5640e9740 1 --2- 172.21.15.22:0/1849607811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 0x55a59da674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.896+0000 7fd5640e9740 1 --2- 172.21.15.22:0/1849607811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da66f60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.896+0000 7fd5640e9740 1 --2- 172.21.15.22:0/1849607811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 0x55a59da69cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.896+0000 7fd5640e9740 1 -- 172.21.15.22:0/1849607811 >> 172.21.15.22:0/1849607811 conn(0x55a59da28310 msgr2=0x55a59da61930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:08.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.897+0000 7fd5640e9740 1 -- 172.21.15.22:0/1849607811 shutdown_connections 2024-09-19T08:25:08.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.897+0000 7fd5640e9740 1 -- 172.21.15.22:0/1849607811 wait complete. 2024-09-19T08:25:08.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.898+0000 7fd5640e9740 20 client.26599 trim_cache size 0 max 16384 2024-09-19T08:25:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.925+0000 7fd5640e9740 1 Processor -- start 2024-09-19T08:25:08.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.926+0000 7fd5640e9740 1 -- start start 2024-09-19T08:25:08.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.926+0000 7fd5640e9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da5f5d0 0x55a59da687c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.926+0000 7fd5640e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da60b40 0x55a59da62120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.926+0000 7fd5640e9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da6a370 0x55a59da62660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.926+0000 7fd5640e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c1c7670 con 0x55a59da60b40 2024-09-19T08:25:08.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.926+0000 7fd5640e9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c073f20 con 0x55a59da5f5d0 2024-09-19T08:25:08.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.926+0000 7fd5640e9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c1021b0 con 0x55a59da6a370 2024-09-19T08:25:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.926+0000 7fd55bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da6a370 0x55a59da62660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.926+0000 7fd55bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da6a370 0x55a59da62660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42374/0 (socket says 172.21.15.22:42374) 2024-09-19T08:25:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.926+0000 7fd55bfff640 1 -- 172.21.15.22:0/3074065276 learned_addr learned my addr 172.21.15.22:0/3074065276 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.927+0000 7fd55affd640 1 --2- 172.21.15.22:0/3074065276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da60b40 0x55a59da62120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.927+0000 7fd55bfff640 1 -- 172.21.15.22:0/3074065276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da5f5d0 msgr2=0x55a59da687c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.927+0000 7fd55bfff640 1 --2- 172.21.15.22:0/3074065276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da5f5d0 0x55a59da687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.927+0000 7fd55bfff640 1 -- 172.21.15.22:0/3074065276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da60b40 msgr2=0x55a59da62120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.927+0000 7fd55bfff640 1 --2- 172.21.15.22:0/3074065276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da60b40 0x55a59da62120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.927+0000 7fd55bfff640 1 -- 172.21.15.22:0/3074065276 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a59da69670 con 0x55a59da6a370 2024-09-19T08:25:08.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.928+0000 7fd55bfff640 1 --2- 172.21.15.22:0/3074065276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da6a370 0x55a59da62660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd5440096c0 tx=0x7fd544007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f19f001eb5096e4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:08.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.928+0000 7fd5517fa640 1 -- 172.21.15.22:0/3074065276 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544015030 con 0x55a59da6a370 2024-09-19T08:25:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.928+0000 7fd5517fa640 1 -- 172.21.15.22:0/3074065276 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54400bba0 con 0x55a59da6a370 2024-09-19T08:25:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.928+0000 7fd5517fa640 1 -- 172.21.15.22:0/3074065276 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54401a820 con 0x55a59da6a370 2024-09-19T08:25:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.929+0000 7fd5640e9740 1 -- 172.21.15.22:0/3074065276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da6a370 msgr2=0x55a59da62660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.929+0000 7fd5640e9740 1 --2- 172.21.15.22:0/3074065276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da6a370 0x55a59da62660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd5440096c0 tx=0x7fd544007f90 comp rx=0 tx=0).stop 2024-09-19T08:25:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.929+0000 7fd5640e9740 1 -- 172.21.15.22:0/3074065276 shutdown_connections 2024-09-19T08:25:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.929+0000 7fd5640e9740 1 --2- 172.21.15.22:0/3074065276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da6a370 0x55a59da62660 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.929+0000 7fd5640e9740 1 --2- 172.21.15.22:0/3074065276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da60b40 0x55a59da62120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.929+0000 7fd5640e9740 1 --2- 172.21.15.22:0/3074065276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da5f5d0 0x55a59da687c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.929+0000 7fd5640e9740 1 -- 172.21.15.22:0/3074065276 >> 172.21.15.22:0/3074065276 conn(0x55a59da28310 msgr2=0x55a59da67040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.930+0000 7fd5640e9740 1 -- 172.21.15.22:0/3074065276 shutdown_connections 2024-09-19T08:25:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.930+0000 7fd5640e9740 1 -- 172.21.15.22:0/3074065276 wait complete. 2024-09-19T08:25:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.930+0000 7fd5640e9740 1 -- start start 2024-09-19T08:25:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.930+0000 7fd5640e9740 1 Processor -- start 2024-09-19T08:25:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.931+0000 7fd5640e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 0x55a59da6e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.931+0000 7fd5640e9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da6e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.931+0000 7fd5640e9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 0x55a59da6eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.931+0000 7fd5640e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c0723e0 con 0x55a59da5f5d0 2024-09-19T08:25:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.931+0000 7fd5640e9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c073850 con 0x55a59da6a370 2024-09-19T08:25:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.931+0000 7fd5640e9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c072ab0 con 0x55a59da60b40 2024-09-19T08:25:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.931+0000 7fd5640e9740 10 client.0 _ll_register_callbacks cb 0x7fd54c009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:08.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.932+0000 7fd55b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 0x55a59da6e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.932+0000 7fd55b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 0x55a59da6e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36854/0 (socket says 172.21.15.22:36854) 2024-09-19T08:25:08.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.932+0000 7fd55b7fe640 1 -- 172.21.15.22:0/19080825 learned_addr learned my addr 172.21.15.22:0/19080825 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:08.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.932+0000 7fd55bfff640 1 --2- 172.21.15.22:0/19080825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 0x55a59da6eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.932+0000 7fd55affd640 1 --2- 172.21.15.22:0/19080825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da6e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.933+0000 7fd55affd640 1 -- 172.21.15.22:0/19080825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 msgr2=0x55a59da6eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:08.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.933+0000 7fd55affd640 1 --2- 172.21.15.22:0/19080825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 0x55a59da6eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.933+0000 7fd55affd640 1 -- 172.21.15.22:0/19080825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 msgr2=0x55a59da6e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:08.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.933+0000 7fd55affd640 1 --2- 172.21.15.22:0/19080825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 0x55a59da6e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.933+0000 7fd55affd640 1 -- 172.21.15.22:0/19080825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a59dae3e80 con 0x55a59da60b40 2024-09-19T08:25:08.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.935+0000 7fd55affd640 1 --2- 172.21.15.22:0/19080825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da6e9b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd54c01f370 tx=0x7fd54c0092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1987a7ebcc4d703 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:08.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.935+0000 7fd53affd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-09-19T08:25:08.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.936+0000 7fd53affd640 1 -- 172.21.15.22:0/19080825 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c02f080 con 0x55a59da60b40 2024-09-19T08:25:08.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.936+0000 7fd53affd640 1 -- 172.21.15.22:0/19080825 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fd54c028ba0 con 0x55a59da60b40 2024-09-19T08:25:08.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.937+0000 7fd5640e9740 1 -- 172.21.15.22:0/19080825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd54400bba0 con 0x55a59da60b40 2024-09-19T08:25:08.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.938+0000 7fd53affd640 1 -- 172.21.15.22:0/19080825 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd54c028da0 con 0x55a59da60b40 2024-09-19T08:25:08.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.939+0000 7fd5640e9740 10 client.26605 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:08.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.939+0000 7fd5640e9740 1 -- 172.21.15.22:0/19080825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a59dae5180 con 0x55a59da60b40 2024-09-19T08:25:08.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.939+0000 7fd53affd640 1 -- 172.21.15.22:0/19080825 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54400bba0 con 0x55a59da60b40 2024-09-19T08:25:08.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.939+0000 7fd53affd640 1 -- 172.21.15.22:0/19080825 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fd54c033480 con 0x55a59da60b40 2024-09-19T08:25:08.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.939+0000 7fd5640e9740 10 client.26605 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:08.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.939+0000 7fd5640e9740 10 client.26605 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:25:08.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.939+0000 7fd5640e9740 20 client.26605 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:08.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.939+0000 7fd5640e9740 1 --2- 172.21.15.22:0/19080825 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55a59da79f50 0x55a59dae78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:08.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.939+0000 7fd5640e9740 4 client.26605 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:25:08.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.939+0000 7fd5640e9740 1 -- 172.21.15.22:0/19080825 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a59d9ccd70 con 0x55a59da79f50 2024-09-19T08:25:08.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.940+0000 7fd55b7fe640 1 --2- 172.21.15.22:0/19080825 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55a59da79f50 0x55a59dae78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:08.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.941+0000 7fd53affd640 10 client.26605 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:25:08.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.942+0000 7fd55b7fe640 1 --2- 172.21.15.22:0/19080825 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55a59da79f50 0x55a59dae78e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74ffbd432fc80045 server_cookie=b04792db94d987f5 in_seq=0 out_seq=0 2024-09-19T08:25:08.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.942+0000 7fd53affd640 1 -- 172.21.15.22:0/19080825 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a59d9ccd70 con 0x55a59da79f50 2024-09-19T08:25:08.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.942+0000 7fd53affd640 10 client.26605 handle_command_reply: tid=0 2024-09-19T08:25:08.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.942+0000 7fd5640e9740 1 client.26605 shutdown 2024-09-19T08:25:08.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.945+0000 7fd5640e9740 1 -- 172.21.15.22:0/19080825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 msgr2=0x55a59da6e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:08.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.945+0000 7fd5640e9740 1 --2- 172.21.15.22:0/19080825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da6e9b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd54c01f370 tx=0x7fd54c0092f0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.948+0000 7fd5640e9740 1 -- 172.21.15.22:0/19080825 shutdown_connections 2024-09-19T08:25:08.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.948+0000 7fd5640e9740 1 --2- 172.21.15.22:0/19080825 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55a59da79f50 0x55a59dae78e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.948+0000 7fd5640e9740 1 --2- 172.21.15.22:0/19080825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59da6a370 0x55a59da6eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.948+0000 7fd5640e9740 1 --2- 172.21.15.22:0/19080825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a59da60b40 0x55a59da6e9b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.948+0000 7fd5640e9740 1 --2- 172.21.15.22:0/19080825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a59da5f5d0 0x55a59da6e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.948+0000 7fd53affd640 0 client.26605 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:25:08.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.948+0000 7fd5640e9740 1 -- 172.21.15.22:0/19080825 >> 172.21.15.22:0/19080825 conn(0x55a59da28310 msgr2=0x55a59da5a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:08.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.949+0000 7fd5640e9740 1 -- 172.21.15.22:0/19080825 shutdown_connections 2024-09-19T08:25:08.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.949+0000 7fd5640e9740 1 -- 172.21.15.22:0/19080825 wait complete. 2024-09-19T08:25:08.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.950+0000 7fd5640e9740 20 client.26605 trim_cache size 0 max 16384 2024-09-19T08:25:08.951 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:25:08.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.951+0000 7fd561d10640 1 -- 172.21.15.22:0/3941709597 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd5480a3420 msgr2=0x7fd5480a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:08.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.951+0000 7fd561d10640 1 --2- 172.21.15.22:0/3941709597 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd5480a3420 0x7fd5480a58e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55a59da60070 tx=0x7fd54c00d040 comp rx=0 tx=0).stop 2024-09-19T08:25:08.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.951+0000 7fd561d10640 1 -- 172.21.15.22:0/3941709597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd55c100a70 msgr2=0x7fd55c1162d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:08.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.951+0000 7fd561d10640 1 --2- 172.21.15.22:0/3941709597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd55c100a70 0x7fd55c1162d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd554009850 tx=0x7fd554005230 comp rx=0 tx=0).stop 2024-09-19T08:25:08.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.951+0000 7fd561d10640 1 -- 172.21.15.22:0/3941709597 shutdown_connections 2024-09-19T08:25:08.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.951+0000 7fd561d10640 1 --2- 172.21.15.22:0/3941709597 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd5480a3420 0x7fd5480a58e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.951+0000 7fd561d10640 1 --2- 172.21.15.22:0/3941709597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd55c106a70 0x7fd55c11bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.951+0000 7fd561d10640 1 --2- 172.21.15.22:0/3941709597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55c101410 0x7fd55c116810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.951+0000 7fd561d10640 1 --2- 172.21.15.22:0/3941709597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd55c100a70 0x7fd55c1162d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:08.953 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:08 smithi022 ceph-mon[30963]: pgmap v312: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-19T08:25:08.953 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:08 smithi022 ceph-mon[30963]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T08:25:08.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.951+0000 7fd561d10640 1 -- 172.21.15.22:0/3941709597 >> 172.21.15.22:0/3941709597 conn(0x7fd55c0fc790 msgr2=0x7fd55c10db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:08.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.951+0000 7fd561d10640 1 -- 172.21.15.22:0/3941709597 shutdown_connections 2024-09-19T08:25:08.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:08.951+0000 7fd561d10640 1 -- 172.21.15.22:0/3941709597 wait complete. 2024-09-19T08:25:09.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:08 smithi064 ceph-mon[34900]: pgmap v312: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-19T08:25:09.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:08 smithi064 ceph-mon[34900]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T08:25:09.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:08 smithi157 ceph-mon[35143]: pgmap v312: 97 pgs: 97 active+clean; 1.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-19T08:25:09.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:08 smithi157 ceph-mon[35143]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T08:25:09.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:25:10.314 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:25:10.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.937+0000 7fae8a83c640 1 -- 172.21.15.22:0/703308133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae84114ab0 msgr2=0x7fae84116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.937+0000 7fae8a83c640 1 --2- 172.21.15.22:0/703308133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae84114ab0 0x7fae84116ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fae74009860 tx=0x7fae7402a3f0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.937+0000 7fae8a83c640 1 -- 172.21.15.22:0/703308133 shutdown_connections 2024-09-19T08:25:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.937+0000 7fae8a83c640 1 --2- 172.21.15.22:0/703308133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae84114ab0 0x7fae84116ea0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.937+0000 7fae8a83c640 1 --2- 172.21.15.22:0/703308133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae84109b10 0x7fae84114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.937+0000 7fae8a83c640 1 --2- 172.21.15.22:0/703308133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae841091f0 0x7fae841095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.937+0000 7fae8a83c640 1 -- 172.21.15.22:0/703308133 >> 172.21.15.22:0/703308133 conn(0x7fae8406e140 msgr2=0x7fae8406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.938+0000 7fae8a83c640 1 -- 172.21.15.22:0/703308133 shutdown_connections 2024-09-19T08:25:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.938+0000 7fae8a83c640 1 -- 172.21.15.22:0/703308133 wait complete. 2024-09-19T08:25:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.939+0000 7fae8a83c640 1 Processor -- start 2024-09-19T08:25:10.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.940+0000 7fae8a83c640 1 -- start start 2024-09-19T08:25:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.940+0000 7fae8a83c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae841091f0 0x7fae84118bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.940+0000 7fae8a83c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae84109b10 0x7fae84119100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.940+0000 7fae8a83c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae84114ab0 0x7fae84119640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.940+0000 7fae8a83c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fae8410c000 con 0x7fae841091f0 2024-09-19T08:25:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.940+0000 7fae8a83c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fae8410be80 con 0x7fae84109b10 2024-09-19T08:25:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.940+0000 7fae8a83c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fae8410c180 con 0x7fae84114ab0 2024-09-19T08:25:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.941+0000 7fae83fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae841091f0 0x7fae84118bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.941+0000 7fae83fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae841091f0 0x7fae84118bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36882/0 (socket says 172.21.15.22:36882) 2024-09-19T08:25:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.941+0000 7fae83fff640 1 -- 172.21.15.22:0/3992983361 learned_addr learned my addr 172.21.15.22:0/3992983361 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.941+0000 7fae83fff640 1 -- 172.21.15.22:0/3992983361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae84114ab0 msgr2=0x7fae84119640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.941+0000 7fae83fff640 1 --2- 172.21.15.22:0/3992983361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae84114ab0 0x7fae84119640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.941+0000 7fae83fff640 1 -- 172.21.15.22:0/3992983361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae84109b10 msgr2=0x7fae84119100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:25:10.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.941+0000 7fae83fff640 1 --2- 172.21.15.22:0/3992983361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae84109b10 0x7fae84119100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.941+0000 7fae83fff640 1 -- 172.21.15.22:0/3992983361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae841b3580 con 0x7fae841091f0 2024-09-19T08:25:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.942+0000 7fae83fff640 1 --2- 172.21.15.22:0/3992983361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae841091f0 0x7fae84118bc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00ed80 tx=0x7fae7c00c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.945+0000 7fae817fa640 1 -- 172.21.15.22:0/3992983361 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c01c070 con 0x7fae841091f0 2024-09-19T08:25:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.945+0000 7fae817fa640 1 -- 172.21.15.22:0/3992983361 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae7c017b20 con 0x7fae841091f0 2024-09-19T08:25:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.946+0000 7fae817fa640 1 -- 172.21.15.22:0/3992983361 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00fcc0 con 0x7fae841091f0 2024-09-19T08:25:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.946+0000 7fae8a83c640 1 -- 172.21.15.22:0/3992983361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae841b3870 con 0x7fae841091f0 2024-09-19T08:25:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.946+0000 7fae8a83c640 1 -- 172.21.15.22:0/3992983361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae841b3d30 con 0x7fae841091f0 2024-09-19T08:25:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.951+0000 7fae8cc15740 1 Processor -- start 2024-09-19T08:25:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.951+0000 7fae8cc15740 1 -- start start 2024-09-19T08:25:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.951+0000 7fae8cc15740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a48b40 0x555da9a47090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.951+0000 7fae8cc15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a475d0 0x555da9a47a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.951+0000 7fae8cc15740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9a54760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:10.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.951+0000 7fae8cc15740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555da99ad2f0 con 0x555da9a48b40 2024-09-19T08:25:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.951+0000 7fae8cc15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555da9954cf0 con 0x555da9a475d0 2024-09-19T08:25:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.951+0000 7fae8cc15740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555da99a92b0 con 0x555da9a52370 2024-09-19T08:25:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.952+0000 7fae837fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9a54760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.952+0000 7fae837fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9a54760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42402/0 (socket says 172.21.15.22:42402) 2024-09-19T08:25:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.952+0000 7fae837fe640 1 -- 172.21.15.22:0/318768168 learned_addr learned my addr 172.21.15.22:0/318768168 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.952+0000 7fae817fa640 1 -- 172.21.15.22:0/3992983361 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fae7c017cd0 con 0x7fae841091f0 2024-09-19T08:25:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.952+0000 7fae837fe640 1 -- 172.21.15.22:0/318768168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a475d0 msgr2=0x555da9a47a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.952+0000 7fae837fe640 1 --2- 172.21.15.22:0/318768168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a475d0 0x555da9a47a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.952+0000 7fae837fe640 1 -- 172.21.15.22:0/318768168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a48b40 msgr2=0x555da9a47090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.952+0000 7fae837fe640 1 --2- 172.21.15.22:0/318768168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a48b40 0x555da9a47090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.952+0000 7fae837fe640 1 -- 172.21.15.22:0/318768168 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555da99393a0 con 0x555da9a52370 2024-09-19T08:25:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.953+0000 7fae837fe640 1 --2- 172.21.15.22:0/318768168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9a54760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fae78009740 tx=0x7fae78009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d648b474a55f3377 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:10.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.953+0000 7fae817fa640 1 --2- 172.21.15.22:0/3992983361 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fae6c0a3420 0x7fae6c0a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.954+0000 7fae6a7fc640 1 -- 172.21.15.22:0/318768168 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78015020 con 0x555da9a52370 2024-09-19T08:25:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.954+0000 7fae6a7fc640 1 -- 172.21.15.22:0/318768168 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae78009e50 con 0x555da9a52370 2024-09-19T08:25:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.954+0000 7fae6a7fc640 1 -- 172.21.15.22:0/318768168 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78010380 con 0x555da9a52370 2024-09-19T08:25:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.954+0000 7fae817fa640 1 -- 172.21.15.22:0/3992983361 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fae7c0cfa10 con 0x7fae841091f0 2024-09-19T08:25:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.954+0000 7fae88dd9640 1 --2- 172.21.15.22:0/3992983361 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fae6c0a3420 0x7fae6c0a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.955+0000 7fae8cc15740 1 -- 172.21.15.22:0/318768168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 msgr2=0x555da9a54760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.955+0000 7fae8cc15740 1 --2- 172.21.15.22:0/318768168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9a54760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fae78009740 tx=0x7fae78009c00 comp rx=0 tx=0).stop 2024-09-19T08:25:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.955+0000 7fae88dd9640 1 --2- 172.21.15.22:0/3992983361 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fae6c0a3420 0x7fae6c0a58e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fae7402a9e0 tx=0x7fae74009540 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.957+0000 7fae8cc15740 1 -- 172.21.15.22:0/318768168 shutdown_connections 2024-09-19T08:25:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.957+0000 7fae8cc15740 1 --2- 172.21.15.22:0/318768168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9a54760 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.957+0000 7fae8cc15740 1 --2- 172.21.15.22:0/318768168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a475d0 0x555da9a47a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.957+0000 7fae8cc15740 1 --2- 172.21.15.22:0/318768168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a48b40 0x555da9a47090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.957+0000 7fae8cc15740 1 -- 172.21.15.22:0/318768168 >> 172.21.15.22:0/318768168 conn(0x555da9a10310 msgr2=0x555da9a43c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.958+0000 7fae8cc15740 1 -- 172.21.15.22:0/318768168 shutdown_connections 2024-09-19T08:25:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.958+0000 7fae8cc15740 1 -- 172.21.15.22:0/318768168 wait complete. 2024-09-19T08:25:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.958+0000 7fae8cc15740 1 -- start start 2024-09-19T08:25:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.959+0000 7fae8cc15740 1 Processor -- start 2024-09-19T08:25:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.959+0000 7fae8cc15740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a475d0 0x555da9a51da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.959+0000 7fae8cc15740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a48b40 0x555da9a4f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.959+0000 7fae8cc15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a52370 0x555da9a4f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.959+0000 7fae8cc15740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555da9a55070 con 0x555da9a48b40 2024-09-19T08:25:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.959+0000 7fae8cc15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555da9968060 con 0x555da9a52370 2024-09-19T08:25:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.959+0000 7fae8cc15740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555da9a551f0 con 0x555da9a475d0 2024-09-19T08:25:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.960+0000 7fae837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a48b40 0x555da9a4f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:10.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.960+0000 7fae837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a48b40 0x555da9a4f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36888/0 (socket says 172.21.15.22:36888) 2024-09-19T08:25:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.960+0000 7fae837fe640 1 -- 172.21.15.22:0/1618190244 learned_addr learned my addr 172.21.15.22:0/1618190244 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.960+0000 7fae8cc15740 10 client.0 _ll_register_callbacks cb 0x555da96bb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.960+0000 7fae837fe640 1 -- 172.21.15.22:0/1618190244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a475d0 msgr2=0x555da9a51da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:25:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.960+0000 7fae837fe640 1 --2- 172.21.15.22:0/1618190244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a475d0 0x555da9a51da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.960+0000 7fae837fe640 1 -- 172.21.15.22:0/1618190244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a52370 msgr2=0x555da9a4f7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:25:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.960+0000 7fae837fe640 1 --2- 172.21.15.22:0/1618190244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a52370 0x555da9a4f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.960+0000 7fae837fe640 1 -- 172.21.15.22:0/1618190244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555da971e3c0 con 0x555da9a48b40 2024-09-19T08:25:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.961+0000 7fae837fe640 1 --2- 172.21.15.22:0/1618190244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a48b40 0x555da9a4f050 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fae78012fd0 tx=0x7fae78011420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4640d70f7ccc67a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.961+0000 7fae637fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.961+0000 7fae637fe640 1 -- 172.21.15.22:0/1618190244 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7800bad0 con 0x555da9a48b40 2024-09-19T08:25:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.961+0000 7fae8cc15740 1 -- 172.21.15.22:0/1618190244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae78009e50 con 0x555da9a48b40 2024-09-19T08:25:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.962+0000 7fae637fe640 1 -- 172.21.15.22:0/1618190244 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fae7800bc70 con 0x555da9a48b40 2024-09-19T08:25:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.962+0000 7fae637fe640 1 -- 172.21.15.22:0/1618190244 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fae78004040 con 0x555da9a48b40 2024-09-19T08:25:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.964+0000 7fae8cc15740 10 client.17151 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.964+0000 7fae8cc15740 1 -- 172.21.15.22:0/1618190244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555da9a71b70 con 0x555da9a48b40 2024-09-19T08:25:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.965+0000 7fae637fe640 1 -- 172.21.15.22:0/1618190244 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae7801f070 con 0x555da9a48b40 2024-09-19T08:25:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.966+0000 7fae637fe640 1 -- 172.21.15.22:0/1618190244 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fae7801b430 con 0x555da9a48b40 2024-09-19T08:25:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.966+0000 7fae8cc15740 10 client.17151 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.966+0000 7fae8cc15740 10 client.17151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:25:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.966+0000 7fae8cc15740 20 client.17151 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.966+0000 7fae8cc15740 1 --2- 172.21.15.22:0/1618190244 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555da9a73340 0x555da9a757e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.966+0000 7fae8cc15740 4 client.17151 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:25:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.966+0000 7fae8cc15740 1 -- 172.21.15.22:0/1618190244 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fae78009e50 con 0x555da9a73340 2024-09-19T08:25:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.967+0000 7fae83fff640 1 --2- 172.21.15.22:0/1618190244 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555da9a73340 0x555da9a757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.968+0000 7fae83fff640 1 --2- 172.21.15.22:0/1618190244 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555da9a73340 0x555da9a757e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f78ca22e05a83b5b server_cookie=cd84014574d251e3 in_seq=0 out_seq=0 2024-09-19T08:25:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.968+0000 7fae637fe640 10 client.17151 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.969+0000 7fae637fe640 1 -- 172.21.15.22:0/1618190244 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fae78009e50 con 0x555da9a73340 2024-09-19T08:25:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.969+0000 7fae637fe640 10 client.17151 handle_command_reply: tid=0 2024-09-19T08:25:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.969+0000 7fae8cc15740 1 client.17151 shutdown 2024-09-19T08:25:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.969+0000 7fae8cc15740 1 -- 172.21.15.22:0/1618190244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a48b40 msgr2=0x555da9a4f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.969+0000 7fae8cc15740 1 --2- 172.21.15.22:0/1618190244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a48b40 0x555da9a4f050 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fae78012fd0 tx=0x7fae78011420 comp rx=0 tx=0).stop 2024-09-19T08:25:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.970+0000 7fae8cc15740 1 -- 172.21.15.22:0/1618190244 shutdown_connections 2024-09-19T08:25:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.970+0000 7fae8cc15740 1 --2- 172.21.15.22:0/1618190244 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555da9a73340 0x555da9a757e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.970+0000 7fae8cc15740 1 --2- 172.21.15.22:0/1618190244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a52370 0x555da9a4f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.970+0000 7fae8cc15740 1 --2- 172.21.15.22:0/1618190244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a48b40 0x555da9a4f050 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.970+0000 7fae8cc15740 1 --2- 172.21.15.22:0/1618190244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a475d0 0x555da9a51da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.970+0000 7fae8cc15740 1 -- 172.21.15.22:0/1618190244 >> 172.21.15.22:0/1618190244 conn(0x555da9a10310 msgr2=0x555da9a49930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.970+0000 7fae637fe640 0 client.17151 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:10.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.970+0000 7fae8cc15740 1 -- 172.21.15.22:0/1618190244 shutdown_connections 2024-09-19T08:25:10.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.971+0000 7fae8cc15740 1 -- 172.21.15.22:0/1618190244 wait complete. 2024-09-19T08:25:10.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.971+0000 7fae8cc15740 20 client.17151 trim_cache size 0 max 16384 2024-09-19T08:25:10.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.995+0000 7fae8cc15740 1 Processor -- start 2024-09-19T08:25:10.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.995+0000 7fae8cc15740 1 -- start start 2024-09-19T08:25:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.995+0000 7fae8cc15740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a475d0 0x555da9a4f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.995+0000 7fae8cc15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a48b40 0x555da9a4a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.995+0000 7fae8cc15740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9a4a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.995+0000 7fae8cc15740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fae841b6660 con 0x555da9a475d0 2024-09-19T08:25:10.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.995+0000 7fae8cc15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fae841b3ff0 con 0x555da9a48b40 2024-09-19T08:25:10.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.995+0000 7fae8cc15740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fae841a91d0 con 0x555da9a52370 2024-09-19T08:25:10.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.997+0000 7fae88dd9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9a4a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:10.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.997+0000 7fae837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a48b40 0x555da9a4a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:10.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.997+0000 7fae88dd9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9a4a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.22:42432/0 (socket says 172.21.15.22:42432) 2024-09-19T08:25:10.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.997+0000 7fae88dd9640 1 -- 172.21.15.22:0/91835923 learned_addr learned my addr 172.21.15.22:0/91835923 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:11.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.998+0000 7fae88dd9640 1 -- 172.21.15.22:0/91835923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a48b40 msgr2=0x555da9a4a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:11.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.998+0000 7fae88dd9640 1 --2- 172.21.15.22:0/91835923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a48b40 0x555da9a4a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.998+0000 7fae88dd9640 1 -- 172.21.15.22:0/91835923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a475d0 msgr2=0x555da9a4f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:11.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.998+0000 7fae88dd9640 1 --2- 172.21.15.22:0/91835923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a475d0 0x555da9a4f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:10.998+0000 7fae88dd9640 1 -- 172.21.15.22:0/91835923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555da9a54ad0 con 0x555da9a52370 2024-09-19T08:25:11.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.000+0000 7fae88dd9640 1 --2- 172.21.15.22:0/91835923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9a4a660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fae74034000 tx=0x7fae74030c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bdcb899a0459fc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:11.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.001+0000 7fae69ffb640 1 -- 172.21.15.22:0/91835923 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7404d020 con 0x555da9a52370 2024-09-19T08:25:11.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.001+0000 7fae69ffb640 1 -- 172.21.15.22:0/91835923 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae740046c0 con 0x555da9a52370 2024-09-19T08:25:11.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.001+0000 7fae69ffb640 1 -- 172.21.15.22:0/91835923 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae740496c0 con 0x555da9a52370 2024-09-19T08:25:11.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.002+0000 7fae8cc15740 1 -- 172.21.15.22:0/91835923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 msgr2=0x555da9a4a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:11.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.002+0000 7fae8cc15740 1 --2- 172.21.15.22:0/91835923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9a4a660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fae74034000 tx=0x7fae74030c20 comp rx=0 tx=0).stop 2024-09-19T08:25:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.005+0000 7fae8cc15740 1 -- 172.21.15.22:0/91835923 shutdown_connections 2024-09-19T08:25:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.005+0000 7fae8cc15740 1 --2- 172.21.15.22:0/91835923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9a4a660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.005+0000 7fae8cc15740 1 --2- 172.21.15.22:0/91835923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a48b40 0x555da9a4a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.005+0000 7fae8cc15740 1 --2- 172.21.15.22:0/91835923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a475d0 0x555da9a4f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.005+0000 7fae8cc15740 1 -- 172.21.15.22:0/91835923 >> 172.21.15.22:0/91835923 conn(0x555da9a10310 msgr2=0x555da9a4e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.006+0000 7fae8cc15740 1 -- 172.21.15.22:0/91835923 shutdown_connections 2024-09-19T08:25:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.006+0000 7fae8cc15740 1 -- 172.21.15.22:0/91835923 wait complete. 2024-09-19T08:25:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.006+0000 7fae8cc15740 1 -- start start 2024-09-19T08:25:11.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.006+0000 7fae8cc15740 1 Processor -- start 2024-09-19T08:25:11.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.007+0000 7fae8cc15740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a475d0 0x555da9a55a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:11.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.007+0000 7fae8cc15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a48b40 0x555da9a55f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:11.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.007+0000 7fae8cc15740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9adc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:11.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.007+0000 7fae8cc15740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fae841a9820 con 0x555da9a475d0 2024-09-19T08:25:11.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.007+0000 7fae8cc15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fae841aac90 con 0x555da9a48b40 2024-09-19T08:25:11.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.007+0000 7fae8cc15740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fae841a9ef0 con 0x555da9a52370 2024-09-19T08:25:11.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.007+0000 7fae8cc15740 10 client.0 _ll_register_callbacks cb 0x7fae78009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:11.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.008+0000 7fae83fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a475d0 0x555da9a55a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:11.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.008+0000 7fae83fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a475d0 0x555da9a55a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36914/0 (socket says 172.21.15.22:36914) 2024-09-19T08:25:11.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.008+0000 7fae83fff640 1 -- 172.21.15.22:0/783024238 learned_addr learned my addr 172.21.15.22:0/783024238 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:11.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.008+0000 7fae837fe640 1 --2- 172.21.15.22:0/783024238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a48b40 0x555da9a55f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:11.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.008+0000 7fae88dd9640 1 --2- 172.21.15.22:0/783024238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9adc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:11.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.008+0000 7fae83fff640 1 -- 172.21.15.22:0/783024238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 msgr2=0x555da9adc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:11.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.008+0000 7fae83fff640 1 --2- 172.21.15.22:0/783024238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9adc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.008+0000 7fae83fff640 1 -- 172.21.15.22:0/783024238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a48b40 msgr2=0x555da9a55f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:11.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.008+0000 7fae83fff640 1 --2- 172.21.15.22:0/783024238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a48b40 0x555da9a55f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.008+0000 7fae83fff640 1 -- 172.21.15.22:0/783024238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555da9adc5b0 con 0x555da9a475d0 2024-09-19T08:25:11.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.009+0000 7fae83fff640 1 --2- 172.21.15.22:0/783024238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a475d0 0x555da9a55a10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0cd290 tx=0x7fae7c00ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5bb0811dd3a8ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:11.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.009+0000 7fae637fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:11.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.009+0000 7fae637fe640 1 -- 172.21.15.22:0/783024238 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c0a0170 con 0x555da9a475d0 2024-09-19T08:25:11.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.009+0000 7fae8cc15740 1 -- 172.21.15.22:0/783024238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae740046c0 con 0x555da9a475d0 2024-09-19T08:25:11.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.009+0000 7fae637fe640 1 -- 172.21.15.22:0/783024238 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fae7c0cd3a0 con 0x555da9a475d0 2024-09-19T08:25:11.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.010+0000 7fae637fe640 1 -- 172.21.15.22:0/783024238 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fae7c0a4a30 con 0x555da9a475d0 2024-09-19T08:25:11.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.010+0000 7fae8cc15740 10 client.17154 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:11.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.010+0000 7fae8cc15740 1 -- 172.21.15.22:0/783024238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555da9aff920 con 0x555da9a475d0 2024-09-19T08:25:11.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.010+0000 7fae637fe640 1 -- 172.21.15.22:0/783024238 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae740046c0 con 0x555da9a475d0 2024-09-19T08:25:11.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.011+0000 7fae637fe640 1 -- 172.21.15.22:0/783024238 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fae7c00b030 con 0x555da9a475d0 2024-09-19T08:25:11.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.011+0000 7fae8cc15740 10 client.17154 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:11.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.011+0000 7fae8cc15740 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:25:11.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.011+0000 7fae8cc15740 20 client.17154 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:11.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.011+0000 7fae8cc15740 1 --2- 172.21.15.22:0/783024238 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555da9b00150 0x555da9b02570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:11.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.011+0000 7fae8cc15740 4 client.17154 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:25:11.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.011+0000 7fae8cc15740 1 -- 172.21.15.22:0/783024238 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555da99ba860 con 0x555da9b00150 2024-09-19T08:25:11.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.012+0000 7fae837fe640 1 --2- 172.21.15.22:0/783024238 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555da9b00150 0x555da9b02570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:11.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.015+0000 7fae637fe640 10 client.17154 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:11.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.015+0000 7fae837fe640 1 --2- 172.21.15.22:0/783024238 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555da9b00150 0x555da9b02570 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=103d43a58b5390a6 server_cookie=70b55d2de31d3527 in_seq=0 out_seq=0 2024-09-19T08:25:11.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.016+0000 7fae637fe640 1 -- 172.21.15.22:0/783024238 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555da99ba860 con 0x555da9b00150 2024-09-19T08:25:11.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.016+0000 7fae637fe640 10 client.17154 handle_command_reply: tid=0 2024-09-19T08:25:11.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.016+0000 7fae8cc15740 1 client.17154 shutdown 2024-09-19T08:25:11.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.017+0000 7fae8cc15740 1 -- 172.21.15.22:0/783024238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a475d0 msgr2=0x555da9a55a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:11.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.017+0000 7fae8cc15740 1 --2- 172.21.15.22:0/783024238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a475d0 0x555da9a55a10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0cd290 tx=0x7fae7c00ff90 comp rx=0 tx=0).stop 2024-09-19T08:25:11.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.017+0000 7fae8cc15740 1 -- 172.21.15.22:0/783024238 shutdown_connections 2024-09-19T08:25:11.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.017+0000 7fae8cc15740 1 --2- 172.21.15.22:0/783024238 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x555da9b00150 0x555da9b02570 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.017+0000 7fae8cc15740 1 --2- 172.21.15.22:0/783024238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555da9a52370 0x555da9adc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.017+0000 7fae8cc15740 1 --2- 172.21.15.22:0/783024238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555da9a48b40 0x555da9a55f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.017+0000 7fae8cc15740 1 --2- 172.21.15.22:0/783024238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555da9a475d0 0x555da9a55a10 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.017+0000 7fae637fe640 0 client.17154 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:11.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.017+0000 7fae8cc15740 1 -- 172.21.15.22:0/783024238 >> 172.21.15.22:0/783024238 conn(0x555da9a10310 msgr2=0x555da9a508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:11.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.018+0000 7fae8cc15740 1 -- 172.21.15.22:0/783024238 shutdown_connections 2024-09-19T08:25:11.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.018+0000 7fae8cc15740 1 -- 172.21.15.22:0/783024238 wait complete. 2024-09-19T08:25:11.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.019+0000 7fae8cc15740 20 client.17154 trim_cache size 0 max 16384 2024-09-19T08:25:11.019 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:25:11.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.020+0000 7fae8a83c640 1 -- 172.21.15.22:0/3992983361 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fae6c0a3420 msgr2=0x7fae6c0a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:11.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.020+0000 7fae8a83c640 1 --2- 172.21.15.22:0/3992983361 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fae6c0a3420 0x7fae6c0a58e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fae7402a9e0 tx=0x7fae74009540 comp rx=0 tx=0).stop 2024-09-19T08:25:11.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.020+0000 7fae8a83c640 1 -- 172.21.15.22:0/3992983361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae841091f0 msgr2=0x7fae84118bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:11.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.020+0000 7fae8a83c640 1 --2- 172.21.15.22:0/3992983361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae841091f0 0x7fae84118bc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00ed80 tx=0x7fae7c00c6e0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.020+0000 7fae8a83c640 1 -- 172.21.15.22:0/3992983361 shutdown_connections 2024-09-19T08:25:11.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.020+0000 7fae8a83c640 1 --2- 172.21.15.22:0/3992983361 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fae6c0a3420 0x7fae6c0a58e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.020+0000 7fae8a83c640 1 --2- 172.21.15.22:0/3992983361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae84114ab0 0x7fae84119640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.020+0000 7fae8a83c640 1 --2- 172.21.15.22:0/3992983361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae84109b10 0x7fae84119100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.020+0000 7fae8a83c640 1 --2- 172.21.15.22:0/3992983361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae841091f0 0x7fae84118bc0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:11.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.020+0000 7fae8a83c640 1 -- 172.21.15.22:0/3992983361 >> 172.21.15.22:0/3992983361 conn(0x7fae8406e140 msgr2=0x7fae84071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:11.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.021+0000 7fae8a83c640 1 -- 172.21.15.22:0/3992983361 shutdown_connections 2024-09-19T08:25:11.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:11.023+0000 7fae8a83c640 1 -- 172.21.15.22:0/3992983361 wait complete. 2024-09-19T08:25:11.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:10 smithi064 ceph-mon[34900]: pgmap v314: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:25:11.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:10 smithi157 ceph-mon[35143]: pgmap v314: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:25:11.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:10 smithi022 ceph-mon[30963]: pgmap v314: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:25:12.711 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T08:25:12.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.817+0000 7facaa32a640 1 -- 172.21.15.22:0/723845813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faca410c0b0 msgr2=0x7faca410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:12.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.817+0000 7facaa32a640 1 --2- 172.21.15.22:0/723845813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faca410c0b0 0x7faca410e550 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009ca0 tx=0x7fac9c02a950 comp rx=0 tx=0).stop 2024-09-19T08:25:12.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.817+0000 7facaa32a640 1 -- 172.21.15.22:0/723845813 shutdown_connections 2024-09-19T08:25:12.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.817+0000 7facaa32a640 1 --2- 172.21.15.22:0/723845813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faca410c0b0 0x7faca410e550 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:12.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.817+0000 7facaa32a640 1 --2- 172.21.15.22:0/723845813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca4103600 0x7faca410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.817+0000 7facaa32a640 1 --2- 172.21.15.22:0/723845813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca4102ce0 0x7faca41030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.817+0000 7facaa32a640 1 -- 172.21.15.22:0/723845813 >> 172.21.15.22:0/723845813 conn(0x7faca4077990 msgr2=0x7faca40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.817+0000 7facaa32a640 1 -- 172.21.15.22:0/723845813 shutdown_connections 2024-09-19T08:25:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.817+0000 7facaa32a640 1 -- 172.21.15.22:0/723845813 wait complete. 2024-09-19T08:25:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7facaa32a640 1 Processor -- start 2024-09-19T08:25:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7facaa32a640 1 -- start start 2024-09-19T08:25:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7facaa32a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faca4102ce0 0x7faca41982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7facaa32a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca4103600 0x7faca41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7facaa32a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca410c0b0 0x7faca419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7facaa32a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faca4111e70 con 0x7faca4102ce0 2024-09-19T08:25:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7facaa32a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faca4111cf0 con 0x7faca4103600 2024-09-19T08:25:12.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7facaa32a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faca4111ff0 con 0x7faca410c0b0 2024-09-19T08:25:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7faca3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faca4102ce0 0x7faca41982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7faca3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faca4102ce0 0x7faca41982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36934/0 (socket says 172.21.15.22:36934) 2024-09-19T08:25:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7faca3fff640 1 -- 172.21.15.22:0/2138432981 learned_addr learned my addr 172.21.15.22:0/2138432981 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7faca88c7640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca410c0b0 0x7faca419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7fac9bfff640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca4103600 0x7faca41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7faca3fff640 1 -- 172.21.15.22:0/2138432981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca410c0b0 msgr2=0x7faca419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7faca3fff640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca410c0b0 0x7faca419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7faca3fff640 1 -- 172.21.15.22:0/2138432981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca4103600 msgr2=0x7faca41987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7faca3fff640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca4103600 0x7faca41987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7faca3fff640 1 -- 172.21.15.22:0/2138432981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca419d290 con 0x7faca4102ce0 2024-09-19T08:25:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.818+0000 7fac9bfff640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca4103600 0x7faca41987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:25:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.819+0000 7faca3fff640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faca4102ce0 0x7faca41982a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fac9400d950 tx=0x7fac9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:12.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.819+0000 7faca1ffb640 1 -- 172.21.15.22:0/2138432981 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94017070 con 0x7faca4102ce0 2024-09-19T08:25:12.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.819+0000 7faca1ffb640 1 -- 172.21.15.22:0/2138432981 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac94009c30 con 0x7faca4102ce0 2024-09-19T08:25:12.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.819+0000 7faca1ffb640 1 -- 172.21.15.22:0/2138432981 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9400f6b0 con 0x7faca4102ce0 2024-09-19T08:25:12.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.819+0000 7facaa32a640 1 -- 172.21.15.22:0/2138432981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca419d580 con 0x7faca4102ce0 2024-09-19T08:25:12.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.819+0000 7facaa32a640 1 -- 172.21.15.22:0/2138432981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca41a4e10 con 0x7faca4102ce0 2024-09-19T08:25:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.820+0000 7facaa32a640 1 -- 172.21.15.22:0/2138432981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca41030c0 con 0x7faca4102ce0 2024-09-19T08:25:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.821+0000 7faca1ffb640 1 -- 172.21.15.22:0/2138432981 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fac940109d0 con 0x7faca4102ce0 2024-09-19T08:25:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.822+0000 7faca1ffb640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fac900a34e0 0x7fac900a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.822+0000 7fac9bfff640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fac900a34e0 0x7fac900a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.822+0000 7faca1ffb640 1 -- 172.21.15.22:0/2138432981 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fac940cf8f0 con 0x7faca4102ce0 2024-09-19T08:25:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.823+0000 7fac9bfff640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fac900a34e0 0x7fac900a59a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fac8c005fd0 tx=0x7fac8c0094d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:12.827+0000 7faca1ffb640 1 -- 172.21.15.22:0/2138432981 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fac94009de0 con 0x7faca4102ce0 2024-09-19T08:25:13.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:12 smithi022 ceph-mon[30963]: pgmap v315: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:25:13.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.026+0000 7facaa32a640 1 -- 172.21.15.22:0/2138432981 --> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7faca40fd690 con 0x7fac900a34e0 2024-09-19T08:25:13.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.034+0000 7faca1ffb640 1 -- 172.21.15.22:0/2138432981 <== mgr.15141 v2:172.21.15.22:6800/497251935 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7faca40fd690 con 0x7fac900a34e0 2024-09-19T08:25:13.034 INFO:teuthology.orchestra.run.smithi022.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T08:15:11", "first": "2024-09-19T08:16:00", "last": "2024-09-19T08:25:00", "last_pruned": "2024-09-19T08:25:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-09-19T08:25:13.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.036+0000 7facaa32a640 1 -- 172.21.15.22:0/2138432981 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fac900a34e0 msgr2=0x7fac900a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:13.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.036+0000 7facaa32a640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fac900a34e0 0x7fac900a59a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fac8c005fd0 tx=0x7fac8c0094d0 comp rx=0 tx=0).stop 2024-09-19T08:25:13.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.036+0000 7facaa32a640 1 -- 172.21.15.22:0/2138432981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faca4102ce0 msgr2=0x7faca41982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:13.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.036+0000 7facaa32a640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faca4102ce0 0x7faca41982a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fac9400d950 tx=0x7fac9400de10 comp rx=0 tx=0).stop 2024-09-19T08:25:13.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.036+0000 7facaa32a640 1 -- 172.21.15.22:0/2138432981 shutdown_connections 2024-09-19T08:25:13.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.036+0000 7facaa32a640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fac900a34e0 0x7fac900a59a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:13.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.036+0000 7facaa32a640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faca410c0b0 0x7faca419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:13.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.036+0000 7facaa32a640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca4103600 0x7faca41987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:13.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.036+0000 7facaa32a640 1 --2- 172.21.15.22:0/2138432981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faca4102ce0 0x7faca41982a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:13.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.036+0000 7facaa32a640 1 -- 172.21.15.22:0/2138432981 >> 172.21.15.22:0/2138432981 conn(0x7faca4077990 msgr2=0x7faca40fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:13.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.036+0000 7facaa32a640 1 -- 172.21.15.22:0/2138432981 shutdown_connections 2024-09-19T08:25:13.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.036+0000 7facaa32a640 1 -- 172.21.15.22:0/2138432981 wait complete. 2024-09-19T08:25:13.046 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T08:25:13.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:12 smithi064 ceph-mon[34900]: pgmap v315: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:25:13.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.152+0000 7f7150eaf640 1 -- 172.21.15.22:0/88657772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c102cc0 msgr2=0x7f714c1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:13.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.152+0000 7f7150eaf640 1 --2- 172.21.15.22:0/88657772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c102cc0 0x7f714c1030a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f713c009900 tx=0x7f713c02a480 comp rx=0 tx=0).stop 2024-09-19T08:25:13.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.153+0000 7f7150eaf640 1 -- 172.21.15.22:0/88657772 shutdown_connections 2024-09-19T08:25:13.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.153+0000 7f7150eaf640 1 --2- 172.21.15.22:0/88657772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f714c10c110 0x7f714c10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:13.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.153+0000 7f7150eaf640 1 --2- 172.21.15.22:0/88657772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714c1035e0 0x7f714c10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:13.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.153+0000 7f7150eaf640 1 --2- 172.21.15.22:0/88657772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c102cc0 0x7f714c1030a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:13.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.153+0000 7f7150eaf640 1 -- 172.21.15.22:0/88657772 >> 172.21.15.22:0/88657772 conn(0x7f714c0fc6b0 msgr2=0x7f714c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:13.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.153+0000 7f7150eaf640 1 -- 172.21.15.22:0/88657772 shutdown_connections 2024-09-19T08:25:13.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.153+0000 7f7150eaf640 1 -- 172.21.15.22:0/88657772 wait complete. 2024-09-19T08:25:13.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.153+0000 7f7150eaf640 1 Processor -- start 2024-09-19T08:25:13.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.153+0000 7f7150eaf640 1 -- start start 2024-09-19T08:25:13.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f7150eaf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f714c102cc0 0x7f714c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:13.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f7150eaf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714c1035e0 0x7f714c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:13.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f7150eaf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c10c110 0x7f714c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:13.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f7150eaf640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f714c111ef0 con 0x7f714c10c110 2024-09-19T08:25:13.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f7150eaf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f714c111d70 con 0x7f714c102cc0 2024-09-19T08:25:13.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f7150eaf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f714c112070 con 0x7f714c1035e0 2024-09-19T08:25:13.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f714ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c10c110 0x7f714c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:13.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f714ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c10c110 0x7f714c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36946/0 (socket says 172.21.15.22:36946) 2024-09-19T08:25:13.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f714ad9d640 1 -- 172.21.15.22:0/1006424308 learned_addr learned my addr 172.21.15.22:0/1006424308 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:13.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f7149d9b640 1 --2- 172.21.15.22:0/1006424308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714c1035e0 0x7f714c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:13.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f714a59c640 1 --2- 172.21.15.22:0/1006424308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f714c102cc0 0x7f714c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:13.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f714ad9d640 1 -- 172.21.15.22:0/1006424308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714c1035e0 msgr2=0x7f714c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:13.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f714ad9d640 1 --2- 172.21.15.22:0/1006424308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714c1035e0 0x7f714c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:13.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f714ad9d640 1 -- 172.21.15.22:0/1006424308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f714c102cc0 msgr2=0x7f714c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f714ad9d640 1 --2- 172.21.15.22:0/1006424308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f714c102cc0 0x7f714c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f714ad9d640 1 -- 172.21.15.22:0/1006424308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c19d350 con 0x7f714c10c110 2024-09-19T08:25:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f714ad9d640 1 --2- 172.21.15.22:0/1006424308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c10c110 0x7f714c19cc20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f714400eaa0 tx=0x7f714400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.154+0000 7f713b7fe640 1 -- 172.21.15.22:0/1006424308 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714400cdb0 con 0x7f714c10c110 2024-09-19T08:25:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.155+0000 7f713b7fe640 1 -- 172.21.15.22:0/1006424308 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7144005020 con 0x7f714c10c110 2024-09-19T08:25:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.155+0000 7f7150eaf640 1 -- 172.21.15.22:0/1006424308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714c19d640 con 0x7f714c10c110 2024-09-19T08:25:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.155+0000 7f713b7fe640 1 -- 172.21.15.22:0/1006424308 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71440116b0 con 0x7f714c10c110 2024-09-19T08:25:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.155+0000 7f7150eaf640 1 -- 172.21.15.22:0/1006424308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714c1a4ed0 con 0x7f714c10c110 2024-09-19T08:25:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.156+0000 7f7150eaf640 1 -- 172.21.15.22:0/1006424308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f714c1030a0 con 0x7f714c10c110 2024-09-19T08:25:13.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.157+0000 7f713b7fe640 1 -- 172.21.15.22:0/1006424308 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7144005560 con 0x7f714c10c110 2024-09-19T08:25:13.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.158+0000 7f713b7fe640 1 --2- 172.21.15.22:0/1006424308 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f71340a3460 0x7f71340a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:13.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.158+0000 7f714a59c640 1 --2- 172.21.15.22:0/1006424308 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f71340a3460 0x7f71340a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:13.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.158+0000 7f713b7fe640 1 -- 172.21.15.22:0/1006424308 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7f71440d02e0 con 0x7f714c10c110 2024-09-19T08:25:13.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.158+0000 7f714a59c640 1 --2- 172.21.15.22:0/1006424308 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f71340a3460 0x7f71340a5920 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f713c005ab0 tx=0x7f713c005a20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:13.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.161+0000 7f713b7fe640 1 -- 172.21.15.22:0/1006424308 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7144016030 con 0x7f714c10c110 2024-09-19T08:25:13.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:12 smithi157 ceph-mon[35143]: pgmap v315: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:25:13.348 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:25:13.348 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:25:13.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.362+0000 7f7150eaf640 1 -- 172.21.15.22:0/1006424308 --> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f714c0fe9d0 con 0x7f71340a3460 2024-09-19T08:25:13.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.370+0000 7f713b7fe640 1 -- 172.21.15.22:0/1006424308 <== mgr.15141 v2:172.21.15.22:6800/497251935 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f714c0fe9d0 con 0x7f71340a3460 2024-09-19T08:25:13.370 INFO:teuthology.orchestra.run.smithi022.stdout:/ 1m 6m3h 2024-09-19T08:25:13.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.373+0000 7f7150eaf640 1 -- 172.21.15.22:0/1006424308 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f71340a3460 msgr2=0x7f71340a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:13.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.373+0000 7f7150eaf640 1 --2- 172.21.15.22:0/1006424308 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f71340a3460 0x7f71340a5920 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f713c005ab0 tx=0x7f713c005a20 comp rx=0 tx=0).stop 2024-09-19T08:25:13.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.373+0000 7f7150eaf640 1 -- 172.21.15.22:0/1006424308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c10c110 msgr2=0x7f714c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:13.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.373+0000 7f7150eaf640 1 --2- 172.21.15.22:0/1006424308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c10c110 0x7f714c19cc20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f714400eaa0 tx=0x7f714400ef60 comp rx=0 tx=0).stop 2024-09-19T08:25:13.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.373+0000 7f7150eaf640 1 -- 172.21.15.22:0/1006424308 shutdown_connections 2024-09-19T08:25:13.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.373+0000 7f7150eaf640 1 --2- 172.21.15.22:0/1006424308 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f71340a3460 0x7f71340a5920 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:13.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.373+0000 7f7150eaf640 1 --2- 172.21.15.22:0/1006424308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c10c110 0x7f714c19cc20 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:13.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.373+0000 7f7150eaf640 1 --2- 172.21.15.22:0/1006424308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714c1035e0 0x7f714c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:13.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.373+0000 7f7150eaf640 1 --2- 172.21.15.22:0/1006424308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f714c102cc0 0x7f714c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:13.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.373+0000 7f7150eaf640 1 -- 172.21.15.22:0/1006424308 >> 172.21.15.22:0/1006424308 conn(0x7f714c0fc6b0 msgr2=0x7f714c0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:13.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.373+0000 7f7150eaf640 1 -- 172.21.15.22:0/1006424308 shutdown_connections 2024-09-19T08:25:13.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:13.373+0000 7f7150eaf640 1 -- 172.21.15.22:0/1006424308 wait complete. 2024-09-19T08:25:13.384 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-19T08:25:13.394 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-19T08:25:13.606 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:25:13.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:13 smithi022 ceph-mon[30963]: from='client.17160 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:25:14.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:13 smithi064 ceph-mon[34900]: from='client.17160 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:25:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.203+0000 7fc4720bb640 1 -- 172.21.15.22:0/766850049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46c100a80 msgr2=0x7fc46c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.203+0000 7fc4720bb640 1 --2- 172.21.15.22:0/766850049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46c100a80 0x7fc46c100ee0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc460009930 tx=0x7fc46002a420 comp rx=0 tx=0).stop 2024-09-19T08:25:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.203+0000 7fc4720bb640 1 -- 172.21.15.22:0/766850049 shutdown_connections 2024-09-19T08:25:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.203+0000 7fc4720bb640 1 --2- 172.21.15.22:0/766850049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc46c101420 0x7fc46c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.203+0000 7fc4720bb640 1 --2- 172.21.15.22:0/766850049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46c100a80 0x7fc46c100ee0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.203+0000 7fc4720bb640 1 --2- 172.21.15.22:0/766850049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c106a80 0x7fc46c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.203+0000 7fc4720bb640 1 -- 172.21.15.22:0/766850049 >> 172.21.15.22:0/766850049 conn(0x7fc46c0fc7a0 msgr2=0x7fc46c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.203+0000 7fc4720bb640 1 -- 172.21.15.22:0/766850049 shutdown_connections 2024-09-19T08:25:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.203+0000 7fc4720bb640 1 -- 172.21.15.22:0/766850049 wait complete. 2024-09-19T08:25:14.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.204+0000 7fc4720bb640 1 Processor -- start 2024-09-19T08:25:14.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.204+0000 7fc4720bb640 1 -- start start 2024-09-19T08:25:14.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.204+0000 7fc4720bb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46c100a80 0x7fc46c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.204+0000 7fc4720bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc46c101420 0x7fc46c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.204+0000 7fc4720bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c106a80 0x7fc46c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.204+0000 7fc4720bb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c111fa0 con 0x7fc46c100a80 2024-09-19T08:25:14.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.204+0000 7fc4720bb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c111e20 con 0x7fc46c101420 2024-09-19T08:25:14.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.204+0000 7fc4720bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c112120 con 0x7fc46c106a80 2024-09-19T08:25:14.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.204+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46c100a80 0x7fc46c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.204+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46c100a80 0x7fc46c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36972/0 (socket says 172.21.15.22:36972) 2024-09-19T08:25:14.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.205+0000 7fc46b7fe640 1 -- 172.21.15.22:0/1691023601 learned_addr learned my addr 172.21.15.22:0/1691023601 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:14.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.205+0000 7fc46affd640 1 --2- 172.21.15.22:0/1691023601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc46c101420 0x7fc46c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.205+0000 7fc46bfff640 1 --2- 172.21.15.22:0/1691023601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c106a80 0x7fc46c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.205+0000 7fc46b7fe640 1 -- 172.21.15.22:0/1691023601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c106a80 msgr2=0x7fc46c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:14.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.205+0000 7fc46b7fe640 1 --2- 172.21.15.22:0/1691023601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c106a80 0x7fc46c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.205+0000 7fc46b7fe640 1 -- 172.21.15.22:0/1691023601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc46c101420 msgr2=0x7fc46c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:14.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.205+0000 7fc46b7fe640 1 --2- 172.21.15.22:0/1691023601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc46c101420 0x7fc46c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.205+0000 7fc46b7fe640 1 -- 172.21.15.22:0/1691023601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46c19d450 con 0x7fc46c100a80 2024-09-19T08:25:14.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.205+0000 7fc46b7fe640 1 --2- 172.21.15.22:0/1691023601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46c100a80 0x7fc46c1984a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00b4a0 tx=0x7fc45c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:14.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.205+0000 7fc468ff9640 1 -- 172.21.15.22:0/1691023601 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c014070 con 0x7fc46c100a80 2024-09-19T08:25:14.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.205+0000 7fc468ff9640 1 -- 172.21.15.22:0/1691023601 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45c010af0 con 0x7fc46c100a80 2024-09-19T08:25:14.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.205+0000 7fc4720bb640 1 -- 172.21.15.22:0/1691023601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46c19d740 con 0x7fc46c100a80 2024-09-19T08:25:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.205+0000 7fc468ff9640 1 -- 172.21.15.22:0/1691023601 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c017d10 con 0x7fc46c100a80 2024-09-19T08:25:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.205+0000 7fc4720bb640 1 -- 172.21.15.22:0/1691023601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46c1a5020 con 0x7fc46c100a80 2024-09-19T08:25:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc468ff9640 1 -- 172.21.15.22:0/1691023601 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc45c017070 con 0x7fc46c100a80 2024-09-19T08:25:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc474494740 1 Processor -- start 2024-09-19T08:25:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc474494740 1 -- start start 2024-09-19T08:25:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc474494740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88beb6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc474494740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88beb7480 0x55e88bec1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc474494740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88bec1d40 0x55e88bec4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc474494740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e88be1c280 con 0x55e88bec1d40 2024-09-19T08:25:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc474494740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e88bdc3b10 con 0x55e88beb6b60 2024-09-19T08:25:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc474494740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e88be18240 con 0x55e88beb7480 2024-09-19T08:25:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc46affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88bec1d40 0x55e88bec4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc46affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88bec1d40 0x55e88bec4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36988/0 (socket says 172.21.15.22:36988) 2024-09-19T08:25:14.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc46affd640 1 -- 172.21.15.22:0/1700464525 learned_addr learned my addr 172.21.15.22:0/1700464525 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:14.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc468ff9640 1 --2- 172.21.15.22:0/1691023601 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc4540a3430 0x7fc4540a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc46bfff640 1 --2- 172.21.15.22:0/1700464525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88beb6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.208+0000 7fc46bfff640 1 --2- 172.21.15.22:0/1691023601 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc4540a3430 0x7fc4540a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.209+0000 7fc468ff9640 1 -- 172.21.15.22:0/1691023601 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fc45c0cef00 con 0x7fc46c100a80 2024-09-19T08:25:14.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.209+0000 7fc46b7fe640 1 --2- 172.21.15.22:0/1700464525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88beb7480 0x55e88bec1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.209+0000 7fc46affd640 1 -- 172.21.15.22:0/1700464525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88beb7480 msgr2=0x55e88bec1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:14.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.209+0000 7fc46affd640 1 --2- 172.21.15.22:0/1700464525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88beb7480 0x55e88bec1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.209+0000 7fc46affd640 1 -- 172.21.15.22:0/1700464525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 msgr2=0x55e88beb6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:14.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.209+0000 7fc46affd640 1 --2- 172.21.15.22:0/1700464525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88beb6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.209+0000 7fc46affd640 1 -- 172.21.15.22:0/1700464525 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88bdd01f0 con 0x55e88bec1d40 2024-09-19T08:25:14.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.209+0000 7fc46affd640 1 --2- 172.21.15.22:0/1700464525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88bec1d40 0x55e88bec4130 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc460009930 tx=0x7fc460031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ab8aa8ab8bfebd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:14.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.209+0000 7fc46bfff640 1 --2- 172.21.15.22:0/1691023601 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc4540a3430 0x7fc4540a58f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc464009990 tx=0x7fc4640096a0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:14.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.209+0000 7fc451ffb640 1 -- 172.21.15.22:0/1700464525 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460035040 con 0x55e88bec1d40 2024-09-19T08:25:14.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.209+0000 7fc451ffb640 1 -- 172.21.15.22:0/1700464525 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc460043be0 con 0x55e88bec1d40 2024-09-19T08:25:14.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.209+0000 7fc451ffb640 1 -- 172.21.15.22:0/1700464525 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4600047a0 con 0x55e88bec1d40 2024-09-19T08:25:14.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.210+0000 7fc474494740 1 -- 172.21.15.22:0/1700464525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88bec1d40 msgr2=0x55e88bec4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:14.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.210+0000 7fc474494740 1 --2- 172.21.15.22:0/1700464525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88bec1d40 0x55e88bec4130 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc460009930 tx=0x7fc460031630 comp rx=0 tx=0).stop 2024-09-19T08:25:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.210+0000 7fc474494740 1 -- 172.21.15.22:0/1700464525 shutdown_connections 2024-09-19T08:25:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.210+0000 7fc474494740 1 --2- 172.21.15.22:0/1700464525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88bec1d40 0x55e88bec4130 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.210+0000 7fc474494740 1 --2- 172.21.15.22:0/1700464525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88beb7480 0x55e88bec1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.210+0000 7fc474494740 1 --2- 172.21.15.22:0/1700464525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88beb6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.210+0000 7fc474494740 1 -- 172.21.15.22:0/1700464525 >> 172.21.15.22:0/1700464525 conn(0x55e88beb30f0 msgr2=0x55e88beb34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.210+0000 7fc474494740 1 -- 172.21.15.22:0/1700464525 shutdown_connections 2024-09-19T08:25:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.210+0000 7fc474494740 1 -- 172.21.15.22:0/1700464525 wait complete. 2024-09-19T08:25:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.210+0000 7fc474494740 1 -- start start 2024-09-19T08:25:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.210+0000 7fc474494740 1 Processor -- start 2024-09-19T08:25:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc474494740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88bec17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc474494740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88beb7480 0x55e88bebeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc474494740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88bec1d40 0x55e88bebf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc474494740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e88bec4910 con 0x55e88beb7480 2024-09-19T08:25:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc474494740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e88bdd6f10 con 0x55e88beb6b60 2024-09-19T08:25:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc474494740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e88bec4a90 con 0x55e88bec1d40 2024-09-19T08:25:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc474494740 10 client.0 _ll_register_callbacks cb 0x55e88bd9fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc46affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88beb7480 0x55e88bebeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc46affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88beb7480 0x55e88bebeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36990/0 (socket says 172.21.15.22:36990) 2024-09-19T08:25:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc46affd640 1 -- 172.21.15.22:0/3780037668 learned_addr learned my addr 172.21.15.22:0/3780037668 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc46b7fe640 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88bec17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc46bfff640 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88bec1d40 0x55e88bebf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc46affd640 1 -- 172.21.15.22:0/3780037668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88bec1d40 msgr2=0x55e88bebf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc46affd640 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88bec1d40 0x55e88bebf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc46affd640 1 -- 172.21.15.22:0/3780037668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 msgr2=0x55e88bec17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:14.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc46affd640 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88bec17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc46affd640 1 -- 172.21.15.22:0/3780037668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88bb8d6a0 con 0x55e88beb7480 2024-09-19T08:25:14.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc46b7fe640 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88bec17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:14.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc46bfff640 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88bec1d40 0x55e88bebf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:25:14.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc46affd640 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88beb7480 0x55e88bebeb00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc46002abd0 tx=0x7fc46002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0fc6a43b3919a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.211+0000 7fc44affd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.212+0000 7fc44affd640 1 -- 172.21.15.22:0/3780037668 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460035070 con 0x55e88beb7480 2024-09-19T08:25:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.212+0000 7fc474494740 1 -- 172.21.15.22:0/3780037668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc460043be0 con 0x55e88beb7480 2024-09-19T08:25:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.212+0000 7fc44affd640 1 -- 172.21.15.22:0/3780037668 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fc4600318e0 con 0x55e88beb7480 2024-09-19T08:25:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.212+0000 7fc44affd640 1 -- 172.21.15.22:0/3780037668 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc460050070 con 0x55e88beb7480 2024-09-19T08:25:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.212+0000 7fc474494740 10 client.17178 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.212+0000 7fc474494740 1 -- 172.21.15.22:0/3780037668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e88bed0400 con 0x55e88beb7480 2024-09-19T08:25:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.212+0000 7fc44affd640 1 -- 172.21.15.22:0/3780037668 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc460043be0 con 0x55e88beb7480 2024-09-19T08:25:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.212+0000 7fc44affd640 1 -- 172.21.15.22:0/3780037668 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fc460004bd0 con 0x55e88beb7480 2024-09-19T08:25:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.212+0000 7fc474494740 10 client.17178 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.212+0000 7fc474494740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:25:14.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.212+0000 7fc474494740 20 client.17178 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:14.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.212+0000 7fc474494740 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e88bed1750 0x55e88bed3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.212+0000 7fc474494740 4 client.17178 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:25:14.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.212+0000 7fc474494740 1 -- 172.21.15.22:0/3780037668 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e88be297f0 con 0x55e88bed1750 2024-09-19T08:25:14.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.213+0000 7fc46b7fe640 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e88bed1750 0x55e88bed3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.213+0000 7fc46b7fe640 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e88bed1750 0x55e88bed3b40 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ce496e45368aa1 server_cookie=c72dd579077ffcc6 in_seq=0 out_seq=0 2024-09-19T08:25:14.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.213+0000 7fc44affd640 10 client.17178 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:14.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc44affd640 1 -- 172.21.15.22:0/3780037668 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e88be297f0 con 0x55e88bed1750 2024-09-19T08:25:14.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc44affd640 10 client.17178 handle_command_reply: tid=0 2024-09-19T08:25:14.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc474494740 1 client.17178 shutdown 2024-09-19T08:25:14.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc474494740 1 -- 172.21.15.22:0/3780037668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88beb7480 msgr2=0x55e88bebeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:14.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc474494740 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88beb7480 0x55e88bebeb00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc46002abd0 tx=0x7fc46002a420 comp rx=0 tx=0).stop 2024-09-19T08:25:14.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc474494740 1 -- 172.21.15.22:0/3780037668 shutdown_connections 2024-09-19T08:25:14.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc474494740 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e88bed1750 0x55e88bed3b40 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc474494740 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88bec1d40 0x55e88bebf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc474494740 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88beb7480 0x55e88bebeb00 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc474494740 1 --2- 172.21.15.22:0/3780037668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88bec17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc44affd640 0 client.17178 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:14.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc474494740 1 -- 172.21.15.22:0/3780037668 >> 172.21.15.22:0/3780037668 conn(0x55e88beb30f0 msgr2=0x55e88beb2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:14.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc474494740 1 -- 172.21.15.22:0/3780037668 shutdown_connections 2024-09-19T08:25:14.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc474494740 1 -- 172.21.15.22:0/3780037668 wait complete. 2024-09-19T08:25:14.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.214+0000 7fc474494740 20 client.17178 trim_cache size 0 max 16384 2024-09-19T08:25:14.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc474494740 1 Processor -- start 2024-09-19T08:25:14.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc474494740 1 -- start start 2024-09-19T08:25:14.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc474494740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88becdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc474494740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88beb7480 0x55e88bec0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc474494740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88bec1d40 0x55e88bed6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc474494740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1ae070 con 0x55e88bec1d40 2024-09-19T08:25:14.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc474494740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1ab8e0 con 0x55e88beb6b60 2024-09-19T08:25:14.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc474494740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1a9750 con 0x55e88beb7480 2024-09-19T08:25:14.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc46bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88bec1d40 0x55e88bed6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc46bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88bec1d40 0x55e88bed6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36998/0 (socket says 172.21.15.22:36998) 2024-09-19T08:25:14.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88becdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc46bfff640 1 -- 172.21.15.22:0/3135385261 learned_addr learned my addr 172.21.15.22:0/3135385261 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:14.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc46affd640 1 --2- 172.21.15.22:0/3135385261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88beb7480 0x55e88bec0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc46bfff640 1 -- 172.21.15.22:0/3135385261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88beb7480 msgr2=0x55e88bec0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:14.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc46bfff640 1 --2- 172.21.15.22:0/3135385261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88beb7480 0x55e88bec0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc46bfff640 1 -- 172.21.15.22:0/3135385261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 msgr2=0x55e88becdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:14.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc46bfff640 1 --2- 172.21.15.22:0/3135385261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88becdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.232+0000 7fc46bfff640 1 -- 172.21.15.22:0/3135385261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88beb2f20 con 0x55e88bec1d40 2024-09-19T08:25:14.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.233+0000 7fc46affd640 1 --2- 172.21.15.22:0/3135385261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88beb7480 0x55e88bec0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:25:14.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.233+0000 7fc46b7fe640 1 --2- 172.21.15.22:0/3135385261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88becdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:25:14.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.233+0000 7fc46bfff640 1 --2- 172.21.15.22:0/3135385261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88bec1d40 0x55e88bed6b60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc464009d70 tx=0x7fc464009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d47430fae206a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:14.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.233+0000 7fc4517fa640 1 -- 172.21.15.22:0/3135385261 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464012d60 con 0x55e88bec1d40 2024-09-19T08:25:14.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.233+0000 7fc4517fa640 1 -- 172.21.15.22:0/3135385261 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4640109d0 con 0x55e88bec1d40 2024-09-19T08:25:14.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.233+0000 7fc4517fa640 1 -- 172.21.15.22:0/3135385261 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464005690 con 0x55e88bec1d40 2024-09-19T08:25:14.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.234+0000 7fc474494740 1 -- 172.21.15.22:0/3135385261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88bec1d40 msgr2=0x55e88bed6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:14.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.234+0000 7fc474494740 1 --2- 172.21.15.22:0/3135385261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88bec1d40 0x55e88bed6b60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc464009d70 tx=0x7fc464009920 comp rx=0 tx=0).stop 2024-09-19T08:25:14.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.234+0000 7fc474494740 1 -- 172.21.15.22:0/3135385261 shutdown_connections 2024-09-19T08:25:14.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.234+0000 7fc474494740 1 --2- 172.21.15.22:0/3135385261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88bec1d40 0x55e88bed6b60 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.234+0000 7fc474494740 1 --2- 172.21.15.22:0/3135385261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88beb7480 0x55e88bec0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.234+0000 7fc474494740 1 --2- 172.21.15.22:0/3135385261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88becdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.234+0000 7fc474494740 1 -- 172.21.15.22:0/3135385261 >> 172.21.15.22:0/3135385261 conn(0x55e88beb30f0 msgr2=0x55e88bec0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:14.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.234+0000 7fc474494740 1 -- 172.21.15.22:0/3135385261 shutdown_connections 2024-09-19T08:25:14.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.234+0000 7fc474494740 1 -- 172.21.15.22:0/3135385261 wait complete. 2024-09-19T08:25:14.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.234+0000 7fc474494740 1 -- start start 2024-09-19T08:25:14.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.234+0000 7fc474494740 1 Processor -- start 2024-09-19T08:25:14.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.234+0000 7fc474494740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88bed7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.234+0000 7fc474494740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88beb7480 0x55e88bed3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.234+0000 7fc474494740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88bec1d40 0x55e88bed40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc474494740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1a9da0 con 0x55e88beb7480 2024-09-19T08:25:14.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc474494740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1ab210 con 0x55e88beb6b60 2024-09-19T08:25:14.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc474494740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1aa470 con 0x55e88bec1d40 2024-09-19T08:25:14.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc474494740 10 client.0 _ll_register_callbacks cb 0x7fc46002aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:14.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc46affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88beb7480 0x55e88bed3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc46affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88beb7480 0x55e88bed3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37000/0 (socket says 172.21.15.22:37000) 2024-09-19T08:25:14.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc46affd640 1 -- 172.21.15.22:0/20150951 learned_addr learned my addr 172.21.15.22:0/20150951 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:14.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc46b7fe640 1 --2- 172.21.15.22:0/20150951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88bed7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc46bfff640 1 --2- 172.21.15.22:0/20150951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88bec1d40 0x55e88bed40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc46affd640 1 -- 172.21.15.22:0/20150951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88bec1d40 msgr2=0x55e88bed40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:14.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc46affd640 1 --2- 172.21.15.22:0/20150951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88bec1d40 0x55e88bed40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc46affd640 1 -- 172.21.15.22:0/20150951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 msgr2=0x55e88bed7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:14.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc46affd640 1 --2- 172.21.15.22:0/20150951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88bed7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc46affd640 1 -- 172.21.15.22:0/20150951 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88bed4910 con 0x55e88beb7480 2024-09-19T08:25:14.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc46bfff640 1 --2- 172.21.15.22:0/20150951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88bec1d40 0x55e88bed40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:25:14.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc46affd640 1 --2- 172.21.15.22:0/20150951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88beb7480 0x55e88bed3b40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc460003760 tx=0x7fc460005390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f27fda2585bd6dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:14.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc44affd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:14.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc44affd640 1 -- 172.21.15.22:0/20150951 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460035040 con 0x55e88beb7480 2024-09-19T08:25:14.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc44affd640 1 -- 172.21.15.22:0/20150951 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fc460031630 con 0x55e88beb7480 2024-09-19T08:25:14.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.235+0000 7fc474494740 1 -- 172.21.15.22:0/20150951 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4640109d0 con 0x55e88beb7480 2024-09-19T08:25:14.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.236+0000 7fc44affd640 1 -- 172.21.15.22:0/20150951 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc460031850 con 0x55e88beb7480 2024-09-19T08:25:14.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.236+0000 7fc474494740 10 client.17184 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:14.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.236+0000 7fc474494740 1 -- 172.21.15.22:0/20150951 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e88bf3d0b0 con 0x55e88beb7480 2024-09-19T08:25:14.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.236+0000 7fc44affd640 1 -- 172.21.15.22:0/20150951 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4640109d0 con 0x55e88beb7480 2024-09-19T08:25:14.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.236+0000 7fc44affd640 1 -- 172.21.15.22:0/20150951 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fc460003e60 con 0x55e88beb7480 2024-09-19T08:25:14.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.236+0000 7fc474494740 10 client.17184 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.236+0000 7fc474494740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:25:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.236+0000 7fc474494740 20 client.17184 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.236+0000 7fc474494740 1 --2- 172.21.15.22:0/20150951 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e88bf3d790 0x55e88bf3fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.236+0000 7fc474494740 4 client.17184 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:25:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.236+0000 7fc474494740 1 -- 172.21.15.22:0/20150951 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e88be23d00 con 0x55e88bf3d790 2024-09-19T08:25:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.236+0000 7fc46b7fe640 1 --2- 172.21.15.22:0/20150951 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e88bf3d790 0x55e88bf3fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.237+0000 7fc46b7fe640 1 --2- 172.21.15.22:0/20150951 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e88bf3d790 0x55e88bf3fbb0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f79fb80c4f7814b server_cookie=54dcd069867b98a4 in_seq=0 out_seq=0 2024-09-19T08:25:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.237+0000 7fc44affd640 10 client.17184 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc44affd640 1 -- 172.21.15.22:0/20150951 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e88be23d00 con 0x55e88bf3d790 2024-09-19T08:25:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc44affd640 10 client.17184 handle_command_reply: tid=0 2024-09-19T08:25:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc474494740 1 client.17184 shutdown 2024-09-19T08:25:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc474494740 1 -- 172.21.15.22:0/20150951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88beb7480 msgr2=0x55e88bed3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc474494740 1 --2- 172.21.15.22:0/20150951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88beb7480 0x55e88bed3b40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc460003760 tx=0x7fc460005390 comp rx=0 tx=0).stop 2024-09-19T08:25:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc474494740 1 -- 172.21.15.22:0/20150951 shutdown_connections 2024-09-19T08:25:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc474494740 1 --2- 172.21.15.22:0/20150951 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55e88bf3d790 0x55e88bf3fbb0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc474494740 1 --2- 172.21.15.22:0/20150951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e88bec1d40 0x55e88bed40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc474494740 1 --2- 172.21.15.22:0/20150951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e88beb7480 0x55e88bed3b40 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc474494740 1 --2- 172.21.15.22:0/20150951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e88beb6b60 0x55e88bed7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc44affd640 0 client.17184 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc474494740 1 -- 172.21.15.22:0/20150951 >> 172.21.15.22:0/20150951 conn(0x55e88beb30f0 msgr2=0x55e88beb2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc474494740 1 -- 172.21.15.22:0/20150951 shutdown_connections 2024-09-19T08:25:14.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc474494740 1 -- 172.21.15.22:0/20150951 wait complete. 2024-09-19T08:25:14.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.238+0000 7fc474494740 20 client.17184 trim_cache size 0 max 16384 2024-09-19T08:25:14.247 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:25:14.247 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-19T08:25:14.247 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "d98d6929-cc51-4014-a33e-c3d6b699c08e", 2024-09-19T08:25:14.247 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-19T08:25:14.248 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:25:14.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.239+0000 7fc4720bb640 1 -- 172.21.15.22:0/1691023601 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc4540a3430 msgr2=0x7fc4540a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:14.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.239+0000 7fc4720bb640 1 --2- 172.21.15.22:0/1691023601 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc4540a3430 0x7fc4540a58f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc464009990 tx=0x7fc4640096a0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.239+0000 7fc4720bb640 1 -- 172.21.15.22:0/1691023601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46c100a80 msgr2=0x7fc46c1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.239+0000 7fc4720bb640 1 --2- 172.21.15.22:0/1691023601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46c100a80 0x7fc46c1984a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00b4a0 tx=0x7fc45c00b960 comp rx=0 tx=0).stop 2024-09-19T08:25:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.239+0000 7fc4720bb640 1 -- 172.21.15.22:0/1691023601 shutdown_connections 2024-09-19T08:25:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.239+0000 7fc4720bb640 1 --2- 172.21.15.22:0/1691023601 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc4540a3430 0x7fc4540a58f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.239+0000 7fc4720bb640 1 --2- 172.21.15.22:0/1691023601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c106a80 0x7fc46c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.239+0000 7fc4720bb640 1 --2- 172.21.15.22:0/1691023601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc46c101420 0x7fc46c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.239+0000 7fc4720bb640 1 --2- 172.21.15.22:0/1691023601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46c100a80 0x7fc46c1984a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.239+0000 7fc4720bb640 1 -- 172.21.15.22:0/1691023601 >> 172.21.15.22:0/1691023601 conn(0x7fc46c0fc7a0 msgr2=0x7fc46c0fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.239+0000 7fc4720bb640 1 -- 172.21.15.22:0/1691023601 shutdown_connections 2024-09-19T08:25:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:14.239+0000 7fc4720bb640 1 -- 172.21.15.22:0/1691023601 wait complete. 2024-09-19T08:25:14.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:13 smithi157 ceph-mon[35143]: from='client.17160 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:25:15.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:14 smithi022 ceph-mon[30963]: pgmap v316: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-19T08:25:15.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:14 smithi022 ceph-mon[30963]: from='client.17166 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:25:15.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:14 smithi064 ceph-mon[34900]: pgmap v316: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-19T08:25:15.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:14 smithi064 ceph-mon[34900]: from='client.17166 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:25:15.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:25:15.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:14 smithi157 ceph-mon[35143]: pgmap v316: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-19T08:25:15.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:14 smithi157 ceph-mon[35143]: from='client.17166 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:25:15.409 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:25:15.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.926+0000 7f0d433c8640 1 -- 172.21.15.22:0/1944755257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d3c1008d0 msgr2=0x7f0d3c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:15.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.926+0000 7f0d433c8640 1 --2- 172.21.15.22:0/1944755257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d3c1008d0 0x7f0d3c100cb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009900 tx=0x7f0d3802a410 comp rx=0 tx=0).stop 2024-09-19T08:25:15.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.926+0000 7f0d433c8640 1 -- 172.21.15.22:0/1944755257 shutdown_connections 2024-09-19T08:25:15.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.926+0000 7f0d433c8640 1 --2- 172.21.15.22:0/1944755257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3c10e480 0x7f0d3c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.926+0000 7f0d433c8640 1 --2- 172.21.15.22:0/1944755257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c101280 0x7f0d3c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.926+0000 7f0d433c8640 1 --2- 172.21.15.22:0/1944755257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d3c1008d0 0x7f0d3c100cb0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.926+0000 7f0d433c8640 1 -- 172.21.15.22:0/1944755257 >> 172.21.15.22:0/1944755257 conn(0x7f0d3c0fc7a0 msgr2=0x7f0d3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.927+0000 7f0d433c8640 1 -- 172.21.15.22:0/1944755257 shutdown_connections 2024-09-19T08:25:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.927+0000 7f0d433c8640 1 -- 172.21.15.22:0/1944755257 wait complete. 2024-09-19T08:25:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.927+0000 7f0d433c8640 1 Processor -- start 2024-09-19T08:25:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.927+0000 7f0d433c8640 1 -- start start 2024-09-19T08:25:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.927+0000 7f0d433c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3c1008d0 0x7f0d3c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.927+0000 7f0d433c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d3c101280 0x7f0d3c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.927+0000 7f0d433c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c10e480 0x7f0d3c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.927+0000 7f0d433c8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1141b0 con 0x7f0d3c101280 2024-09-19T08:25:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.927+0000 7f0d433c8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c114030 con 0x7f0d3c1008d0 2024-09-19T08:25:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.927+0000 7f0d433c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c114330 con 0x7f0d3c10e480 2024-09-19T08:25:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d40963640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d3c101280 0x7f0d3c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d40963640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d3c101280 0x7f0d3c19ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37016/0 (socket says 172.21.15.22:37016) 2024-09-19T08:25:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d40963640 1 -- 172.21.15.22:0/2768436453 learned_addr learned my addr 172.21.15.22:0/2768436453 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d41164640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3c1008d0 0x7f0d3c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d41965640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c10e480 0x7f0d3c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d40963640 1 -- 172.21.15.22:0/2768436453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c10e480 msgr2=0x7f0d3c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d40963640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c10e480 0x7f0d3c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d40963640 1 -- 172.21.15.22:0/2768436453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3c1008d0 msgr2=0x7f0d3c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:15.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d40963640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3c1008d0 0x7f0d3c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d40963640 1 -- 172.21.15.22:0/2768436453 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c19f6a0 con 0x7f0d3c101280 2024-09-19T08:25:15.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d41965640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c10e480 0x7f0d3c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:15.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d40963640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d3c101280 0x7f0d3c19ac30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00d770 tx=0x7f0d2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:15.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d327fc640 1 -- 172.21.15.22:0/2768436453 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c017070 con 0x7f0d3c101280 2024-09-19T08:25:15.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d327fc640 1 -- 172.21.15.22:0/2768436453 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d2c005500 con 0x7f0d3c101280 2024-09-19T08:25:15.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d327fc640 1 -- 172.21.15.22:0/2768436453 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c004e00 con 0x7f0d3c101280 2024-09-19T08:25:15.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d433c8640 1 -- 172.21.15.22:0/2768436453 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c19f990 con 0x7f0d3c101280 2024-09-19T08:25:15.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.928+0000 7f0d433c8640 1 -- 172.21.15.22:0/2768436453 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c1a7270 con 0x7f0d3c101280 2024-09-19T08:25:15.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.931+0000 7f0d327fc640 1 -- 172.21.15.22:0/2768436453 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0d2c004fa0 con 0x7f0d3c101280 2024-09-19T08:25:15.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.931+0000 7f0d457a1740 1 Processor -- start 2024-09-19T08:25:15.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.931+0000 7f0d457a1740 1 -- start start 2024-09-19T08:25:15.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.931+0000 7f0d457a1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6a4a0 0x55742fb6a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.931+0000 7f0d457a1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6adc0 0x55742fb75140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.931+0000 7f0d457a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 0x55742fb77a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d457a1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55742fad02e0 con 0x55742fb6a4a0 2024-09-19T08:25:15.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d457a1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55742fa77ce0 con 0x55742fb6adc0 2024-09-19T08:25:15.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d457a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55742facc2a0 con 0x55742fb75680 2024-09-19T08:25:15.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d41965640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6a4a0 0x55742fb6a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d41965640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6a4a0 0x55742fb6a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37018/0 (socket says 172.21.15.22:37018) 2024-09-19T08:25:15.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d41965640 1 -- 172.21.15.22:0/2213954903 learned_addr learned my addr 172.21.15.22:0/2213954903 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:15.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d41164640 1 --2- 172.21.15.22:0/2213954903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6adc0 0x55742fb75140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d40963640 1 --2- 172.21.15.22:0/2213954903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 0x55742fb77a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d327fc640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0d280a3430 0x7f0d280a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d41965640 1 -- 172.21.15.22:0/2213954903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 msgr2=0x55742fb77a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:15.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d41965640 1 --2- 172.21.15.22:0/2213954903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 0x55742fb77a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d41965640 1 -- 172.21.15.22:0/2213954903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6adc0 msgr2=0x55742fb75140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:15.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d41965640 1 --2- 172.21.15.22:0/2213954903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6adc0 0x55742fb75140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d41965640 1 -- 172.21.15.22:0/2213954903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742fa5c440 con 0x55742fb6a4a0 2024-09-19T08:25:15.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d40963640 1 --2- 172.21.15.22:0/2213954903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 0x55742fb77a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:15.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d327fc640 1 -- 172.21.15.22:0/2768436453 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f0d2c0cf760 con 0x7f0d3c101280 2024-09-19T08:25:15.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d41965640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0d280a3430 0x7f0d280a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.932+0000 7f0d41965640 1 --2- 172.21.15.22:0/2213954903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6a4a0 0x55742fb6a880 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009f80 tx=0x7f0d34010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4326cb7643557d96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:15.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.933+0000 7f0d277fe640 1 -- 172.21.15.22:0/2213954903 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34022070 con 0x55742fb6a4a0 2024-09-19T08:25:15.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.933+0000 7f0d277fe640 1 -- 172.21.15.22:0/2213954903 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3401b380 con 0x55742fb6a4a0 2024-09-19T08:25:15.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.933+0000 7f0d277fe640 1 -- 172.21.15.22:0/2213954903 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3400e600 con 0x55742fb6a4a0 2024-09-19T08:25:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.933+0000 7f0d457a1740 1 -- 172.21.15.22:0/2213954903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6a4a0 msgr2=0x55742fb6a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.933+0000 7f0d41965640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0d280a3430 0x7f0d280a58f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d3401f000 tx=0x7f0d34020000 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.933+0000 7f0d457a1740 1 --2- 172.21.15.22:0/2213954903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6a4a0 0x55742fb6a880 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009f80 tx=0x7f0d34010860 comp rx=0 tx=0).stop 2024-09-19T08:25:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.933+0000 7f0d457a1740 1 -- 172.21.15.22:0/2213954903 shutdown_connections 2024-09-19T08:25:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.933+0000 7f0d457a1740 1 --2- 172.21.15.22:0/2213954903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 0x55742fb77a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.933+0000 7f0d457a1740 1 --2- 172.21.15.22:0/2213954903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6adc0 0x55742fb75140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.933+0000 7f0d457a1740 1 --2- 172.21.15.22:0/2213954903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6a4a0 0x55742fb6a880 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.933+0000 7f0d457a1740 1 -- 172.21.15.22:0/2213954903 >> 172.21.15.22:0/2213954903 conn(0x55742fb66a30 msgr2=0x55742fb66e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.933+0000 7f0d457a1740 1 -- 172.21.15.22:0/2213954903 shutdown_connections 2024-09-19T08:25:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.934+0000 7f0d457a1740 1 -- 172.21.15.22:0/2213954903 wait complete. 2024-09-19T08:25:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.934+0000 7f0d457a1740 1 -- start start 2024-09-19T08:25:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.934+0000 7f0d457a1740 1 Processor -- start 2024-09-19T08:25:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.934+0000 7f0d457a1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6a4a0 0x55742fb74fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.934+0000 7f0d457a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb6adc0 0x55742fb72250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.934+0000 7f0d457a1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb75680 0x55742fb72790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.934+0000 7f0d457a1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55742fb78250 con 0x55742fb75680 2024-09-19T08:25:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.934+0000 7f0d457a1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55742fa8b050 con 0x55742fb6a4a0 2024-09-19T08:25:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.934+0000 7f0d457a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55742fb783d0 con 0x55742fb6adc0 2024-09-19T08:25:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.934+0000 7f0d457a1740 10 client.0 _ll_register_callbacks cb 0x55742facb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d41965640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb75680 0x55742fb72790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d41164640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6a4a0 0x55742fb74fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d41965640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb75680 0x55742fb72790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37032/0 (socket says 172.21.15.22:37032) 2024-09-19T08:25:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d41164640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6a4a0 0x55742fb74fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.22:46762/0 (socket says 172.21.15.22:46762) 2024-09-19T08:25:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d41965640 1 -- 172.21.15.22:0/3644422379 learned_addr learned my addr 172.21.15.22:0/3644422379 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d40963640 1 --2- 172.21.15.22:0/3644422379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb6adc0 0x55742fb72250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d41965640 1 -- 172.21.15.22:0/3644422379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb6adc0 msgr2=0x55742fb72250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d41965640 1 --2- 172.21.15.22:0/3644422379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb6adc0 0x55742fb72250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d41965640 1 -- 172.21.15.22:0/3644422379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6a4a0 msgr2=0x55742fb74fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d41965640 1 --2- 172.21.15.22:0/3644422379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6a4a0 0x55742fb74fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d41965640 1 -- 172.21.15.22:0/3644422379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742f8413c0 con 0x55742fb75680 2024-09-19T08:25:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d41965640 1 --2- 172.21.15.22:0/3644422379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb75680 0x55742fb72790 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0d3401bdd0 tx=0x7f0d3401b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33173ad9a7d50713 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d24ff9640 1 -- 172.21.15.22:0/3644422379 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34022050 con 0x55742fb75680 2024-09-19T08:25:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d24ff9640 1 -- 172.21.15.22:0/3644422379 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f0d34004bd0 con 0x55742fb75680 2024-09-19T08:25:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d457a1740 1 -- 172.21.15.22:0/3644422379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3401b380 con 0x55742fb75680 2024-09-19T08:25:15.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d24ff9640 1 -- 172.21.15.22:0/3644422379 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0d34004dd0 con 0x55742fb75680 2024-09-19T08:25:15.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d457a1740 10 client.17196 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:15.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.935+0000 7f0d457a1740 1 -- 172.21.15.22:0/3644422379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55742fb83bd0 con 0x55742fb75680 2024-09-19T08:25:15.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.936+0000 7f0d24ff9640 1 -- 172.21.15.22:0/3644422379 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3401b380 con 0x55742fb75680 2024-09-19T08:25:15.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.936+0000 7f0d24ff9640 1 -- 172.21.15.22:0/3644422379 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f0d34005480 con 0x55742fb75680 2024-09-19T08:25:15.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.936+0000 7f0d457a1740 10 client.17196 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:15.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.936+0000 7f0d457a1740 10 client.17196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:25:15.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.936+0000 7f0d457a1740 20 client.17196 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:15.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.936+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3644422379 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55742fb85090 0x55742fb87480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.936+0000 7f0d457a1740 4 client.17196 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:25:15.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.936+0000 7f0d457a1740 1 -- 172.21.15.22:0/3644422379 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55742fadd850 con 0x55742fb85090 2024-09-19T08:25:15.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.936+0000 7f0d41164640 1 --2- 172.21.15.22:0/3644422379 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55742fb85090 0x55742fb87480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.937+0000 7f0d41164640 1 --2- 172.21.15.22:0/3644422379 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55742fb85090 0x55742fb87480 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87acffe4bef63c5 server_cookie=d82cc7b4672685e8 in_seq=0 out_seq=0 2024-09-19T08:25:15.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.937+0000 7f0d24ff9640 10 client.17196 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:15.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.937+0000 7f0d24ff9640 1 -- 172.21.15.22:0/3644422379 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55742fadd850 con 0x55742fb85090 2024-09-19T08:25:15.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.937+0000 7f0d24ff9640 10 client.17196 handle_command_reply: tid=0 2024-09-19T08:25:15.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.938+0000 7f0d457a1740 1 client.17196 shutdown 2024-09-19T08:25:15.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.938+0000 7f0d457a1740 1 -- 172.21.15.22:0/3644422379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb75680 msgr2=0x55742fb72790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:15.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.938+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3644422379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb75680 0x55742fb72790 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0d3401bdd0 tx=0x7f0d3401b7e0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.938+0000 7f0d457a1740 1 -- 172.21.15.22:0/3644422379 shutdown_connections 2024-09-19T08:25:15.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.938+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3644422379 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55742fb85090 0x55742fb87480 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.938+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3644422379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb75680 0x55742fb72790 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.938+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3644422379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb6adc0 0x55742fb72250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.938+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3644422379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6a4a0 0x55742fb74fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.938+0000 7f0d457a1740 1 -- 172.21.15.22:0/3644422379 >> 172.21.15.22:0/3644422379 conn(0x55742fb66a30 msgr2=0x55742fb6c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:15.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.938+0000 7f0d24ff9640 0 client.17196 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:15.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.938+0000 7f0d457a1740 1 -- 172.21.15.22:0/3644422379 shutdown_connections 2024-09-19T08:25:15.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.938+0000 7f0d457a1740 1 -- 172.21.15.22:0/3644422379 wait complete. 2024-09-19T08:25:15.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.938+0000 7f0d457a1740 20 client.17196 trim_cache size 0 max 16384 2024-09-19T08:25:15.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.955+0000 7f0d457a1740 1 Processor -- start 2024-09-19T08:25:15.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.955+0000 7f0d457a1740 1 -- start start 2024-09-19T08:25:15.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.955+0000 7f0d457a1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6a4a0 0x55742fb814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.955+0000 7f0d457a1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6adc0 0x55742fb8cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.955+0000 7f0d457a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 0x55742fb8d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.955+0000 7f0d457a1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1b2380 con 0x55742fb6a4a0 2024-09-19T08:25:15.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.955+0000 7f0d457a1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1afbf0 con 0x55742fb6adc0 2024-09-19T08:25:15.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.955+0000 7f0d457a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ada60 con 0x55742fb75680 2024-09-19T08:25:15.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d41164640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6a4a0 0x55742fb814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d41164640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6a4a0 0x55742fb814b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37038/0 (socket says 172.21.15.22:37038) 2024-09-19T08:25:15.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d41164640 1 -- 172.21.15.22:0/3505269985 learned_addr learned my addr 172.21.15.22:0/3505269985 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:15.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d41965640 1 --2- 172.21.15.22:0/3505269985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 0x55742fb8d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d40963640 1 --2- 172.21.15.22:0/3505269985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6adc0 0x55742fb8cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d41164640 1 -- 172.21.15.22:0/3505269985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 msgr2=0x55742fb8d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d41164640 1 --2- 172.21.15.22:0/3505269985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 0x55742fb8d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d41164640 1 -- 172.21.15.22:0/3505269985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6adc0 msgr2=0x55742fb8cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d41164640 1 --2- 172.21.15.22:0/3505269985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6adc0 0x55742fb8cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d41164640 1 -- 172.21.15.22:0/3505269985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742fb6abf0 con 0x55742fb6a4a0 2024-09-19T08:25:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d40963640 1 --2- 172.21.15.22:0/3505269985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6adc0 0x55742fb8cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:25:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d41164640 1 --2- 172.21.15.22:0/3505269985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6a4a0 0x55742fb814b0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009900 tx=0x7f0d380319e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce38992692c6d31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d26ffd640 1 -- 172.21.15.22:0/3505269985 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38035040 con 0x55742fb6a4a0 2024-09-19T08:25:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d26ffd640 1 -- 172.21.15.22:0/3505269985 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d38005500 con 0x55742fb6a4a0 2024-09-19T08:25:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.956+0000 7f0d26ffd640 1 -- 172.21.15.22:0/3505269985 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38004c60 con 0x55742fb6a4a0 2024-09-19T08:25:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.957+0000 7f0d457a1740 1 -- 172.21.15.22:0/3505269985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6a4a0 msgr2=0x55742fb814b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.957+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3505269985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6a4a0 0x55742fb814b0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009900 tx=0x7f0d380319e0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.957+0000 7f0d457a1740 1 -- 172.21.15.22:0/3505269985 shutdown_connections 2024-09-19T08:25:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.957+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3505269985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 0x55742fb8d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.957+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3505269985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6adc0 0x55742fb8cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.957+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3505269985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6a4a0 0x55742fb814b0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.957+0000 7f0d457a1740 1 -- 172.21.15.22:0/3505269985 >> 172.21.15.22:0/3505269985 conn(0x55742fb66a30 msgr2=0x55742fb742d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.957+0000 7f0d457a1740 1 -- 172.21.15.22:0/3505269985 shutdown_connections 2024-09-19T08:25:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.957+0000 7f0d457a1740 1 -- 172.21.15.22:0/3505269985 wait complete. 2024-09-19T08:25:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.957+0000 7f0d457a1740 1 -- start start 2024-09-19T08:25:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d457a1740 1 Processor -- start 2024-09-19T08:25:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d457a1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6a4a0 0x55742fb7ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d457a1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6adc0 0x55742fb7b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d457a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 0x55742fbedc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d457a1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ae0b0 con 0x55742fb6adc0 2024-09-19T08:25:15.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d457a1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1af520 con 0x55742fb6a4a0 2024-09-19T08:25:15.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d457a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ae780 con 0x55742fb75680 2024-09-19T08:25:15.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d457a1740 10 client.0 _ll_register_callbacks cb 0x7f0d34020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:15.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d40963640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6adc0 0x55742fb7b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d40963640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6adc0 0x55742fb7b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37042/0 (socket says 172.21.15.22:37042) 2024-09-19T08:25:15.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d40963640 1 -- 172.21.15.22:0/3074780469 learned_addr learned my addr 172.21.15.22:0/3074780469 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:15.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d41965640 1 --2- 172.21.15.22:0/3074780469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 0x55742fbedc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d41164640 1 --2- 172.21.15.22:0/3074780469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6a4a0 0x55742fb7ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d40963640 1 -- 172.21.15.22:0/3074780469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 msgr2=0x55742fbedc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:15.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.958+0000 7f0d40963640 1 --2- 172.21.15.22:0/3074780469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 0x55742fbedc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.959+0000 7f0d40963640 1 -- 172.21.15.22:0/3074780469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6a4a0 msgr2=0x55742fb7ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:15.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.959+0000 7f0d40963640 1 --2- 172.21.15.22:0/3074780469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6a4a0 0x55742fb7ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.959+0000 7f0d40963640 1 -- 172.21.15.22:0/3074780469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742fbee250 con 0x55742fb6adc0 2024-09-19T08:25:15.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.959+0000 7f0d41164640 1 --2- 172.21.15.22:0/3074780469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6a4a0 0x55742fb7ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:15.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.959+0000 7f0d40963640 1 --2- 172.21.15.22:0/3074780469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6adc0 0x55742fb7b140 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009610 tx=0x7f0d2c09b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a573adf80728c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:15.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.959+0000 7f0d24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:15.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.959+0000 7f0d24ff9640 1 -- 172.21.15.22:0/3074780469 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c020040 con 0x55742fb6adc0 2024-09-19T08:25:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.959+0000 7f0d24ff9640 1 -- 172.21.15.22:0/3074780469 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f0d2c009cc0 con 0x55742fb6adc0 2024-09-19T08:25:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.959+0000 7f0d457a1740 1 -- 172.21.15.22:0/3074780469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d38005500 con 0x55742fb6adc0 2024-09-19T08:25:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.959+0000 7f0d24ff9640 1 -- 172.21.15.22:0/3074780469 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0d2c09fd80 con 0x55742fb6adc0 2024-09-19T08:25:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.959+0000 7f0d457a1740 10 client.17202 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.959+0000 7f0d457a1740 1 -- 172.21.15.22:0/3074780469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55742fbef4b0 con 0x55742fb6adc0 2024-09-19T08:25:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.959+0000 7f0d24ff9640 1 -- 172.21.15.22:0/3074780469 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d38005500 con 0x55742fb6adc0 2024-09-19T08:25:15.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.959+0000 7f0d24ff9640 1 -- 172.21.15.22:0/3074780469 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f0d2c0cfb50 con 0x55742fb6adc0 2024-09-19T08:25:15.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.960+0000 7f0d457a1740 10 client.17202 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:15.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.960+0000 7f0d457a1740 10 client.17202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:25:15.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.960+0000 7f0d457a1740 20 client.17202 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:15.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.960+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3074780469 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55742fbefc90 0x55742fbf20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:15.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.960+0000 7f0d457a1740 4 client.17202 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:25:15.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.960+0000 7f0d457a1740 1 -- 172.21.15.22:0/3074780469 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55742fad7d60 con 0x55742fbefc90 2024-09-19T08:25:15.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.960+0000 7f0d41164640 1 --2- 172.21.15.22:0/3074780469 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55742fbefc90 0x55742fbf20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:15.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.960+0000 7f0d41164640 1 --2- 172.21.15.22:0/3074780469 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55742fbefc90 0x55742fbf20b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77aa491f3aa60499 server_cookie=4d1837f7ada529b8 in_seq=0 out_seq=0 2024-09-19T08:25:15.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.960+0000 7f0d24ff9640 10 client.17202 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:15.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.960+0000 7f0d24ff9640 1 -- 172.21.15.22:0/3074780469 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55742fad7d60 con 0x55742fbefc90 2024-09-19T08:25:15.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.960+0000 7f0d24ff9640 10 client.17202 handle_command_reply: tid=0 2024-09-19T08:25:15.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.960+0000 7f0d457a1740 1 client.17202 shutdown 2024-09-19T08:25:15.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.961+0000 7f0d457a1740 1 -- 172.21.15.22:0/3074780469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6adc0 msgr2=0x55742fb7b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:15.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.961+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3074780469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6adc0 0x55742fb7b140 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009610 tx=0x7f0d2c09b420 comp rx=0 tx=0).stop 2024-09-19T08:25:15.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.961+0000 7f0d457a1740 1 -- 172.21.15.22:0/3074780469 shutdown_connections 2024-09-19T08:25:15.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.961+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3074780469 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x55742fbefc90 0x55742fbf20b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.961+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3074780469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55742fb75680 0x55742fbedc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.961+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3074780469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55742fb6adc0 0x55742fb7b140 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.961+0000 7f0d457a1740 1 --2- 172.21.15.22:0/3074780469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55742fb6a4a0 0x55742fb7ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.961+0000 7f0d457a1740 1 -- 172.21.15.22:0/3074780469 >> 172.21.15.22:0/3074780469 conn(0x55742fb66a30 msgr2=0x55742fb65350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:15.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.961+0000 7f0d24ff9640 0 client.17202 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:15.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.961+0000 7f0d457a1740 1 -- 172.21.15.22:0/3074780469 shutdown_connections 2024-09-19T08:25:15.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.961+0000 7f0d457a1740 1 -- 172.21.15.22:0/3074780469 wait complete. 2024-09-19T08:25:15.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.961+0000 7f0d457a1740 20 client.17202 trim_cache size 0 max 16384 2024-09-19T08:25:15.971 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:25:15.971 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:25:15.971 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-19T08:25:15.971 INFO:teuthology.orchestra.run.smithi022.stdout: "d98d6929-cc51-4014-a33e-c3d6b699c08e": { 2024-09-19T08:25:15.971 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-19T08:25:15.971 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "d98d6929-cc51-4014-a33e-c3d6b699c08e", 2024-09-19T08:25:15.972 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-09-19T08:25:15.972 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:15.972 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:15.972 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:25:15.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.962+0000 7f0d433c8640 1 -- 172.21.15.22:0/2768436453 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0d280a3430 msgr2=0x7f0d280a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:15.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.962+0000 7f0d433c8640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0d280a3430 0x7f0d280a58f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d3401f000 tx=0x7f0d34020000 comp rx=0 tx=0).stop 2024-09-19T08:25:15.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.962+0000 7f0d433c8640 1 -- 172.21.15.22:0/2768436453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d3c101280 msgr2=0x7f0d3c19ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:15.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.962+0000 7f0d433c8640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d3c101280 0x7f0d3c19ac30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00d770 tx=0x7f0d2c00dc30 comp rx=0 tx=0).stop 2024-09-19T08:25:15.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.962+0000 7f0d433c8640 1 -- 172.21.15.22:0/2768436453 shutdown_connections 2024-09-19T08:25:15.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.962+0000 7f0d433c8640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f0d280a3430 0x7f0d280a58f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.962+0000 7f0d433c8640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3c10e480 0x7f0d3c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.962+0000 7f0d433c8640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d3c101280 0x7f0d3c19ac30 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.962+0000 7f0d433c8640 1 --2- 172.21.15.22:0/2768436453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3c1008d0 0x7f0d3c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:15.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.962+0000 7f0d433c8640 1 -- 172.21.15.22:0/2768436453 >> 172.21.15.22:0/2768436453 conn(0x7f0d3c0fc7a0 msgr2=0x7f0d3c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:15.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.962+0000 7f0d433c8640 1 -- 172.21.15.22:0/2768436453 shutdown_connections 2024-09-19T08:25:15.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:15.962+0000 7f0d433c8640 1 -- 172.21.15.22:0/2768436453 wait complete. 2024-09-19T08:25:16.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:15 smithi064 ceph-mon[34900]: scrub queued for path: / 2024-09-19T08:25:16.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:15 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:25:16.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:15 smithi064 ceph-mon[34900]: scrub summary: active paths [/] 2024-09-19T08:25:16.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:15 smithi064 ceph-mon[34900]: scrub summary: idle+waiting paths [/] 2024-09-19T08:25:16.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:15 smithi064 ceph-mon[34900]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T08:25:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:15 smithi157 ceph-mon[35143]: scrub queued for path: / 2024-09-19T08:25:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:15 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:25:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:15 smithi157 ceph-mon[35143]: scrub summary: active paths [/] 2024-09-19T08:25:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:15 smithi157 ceph-mon[35143]: scrub summary: idle+waiting paths [/] 2024-09-19T08:25:16.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:15 smithi157 ceph-mon[35143]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T08:25:16.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:15 smithi022 ceph-mon[30963]: scrub queued for path: / 2024-09-19T08:25:16.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:15 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:25:16.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:15 smithi022 ceph-mon[30963]: scrub summary: active paths [/] 2024-09-19T08:25:16.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:15 smithi022 ceph-mon[30963]: scrub summary: idle+waiting paths [/] 2024-09-19T08:25:16.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:15 smithi022 ceph-mon[30963]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T08:25:17.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:16 smithi022 ceph-mon[30963]: pgmap v317: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-19T08:25:17.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:16 smithi064 ceph-mon[34900]: pgmap v317: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-19T08:25:17.096 INFO:tasks.cephfs.filesystem:scrub status for tag:d98d6929-cc51-4014-a33e-c3d6b699c08e - {'path': '/', 'tag': 'd98d6929-cc51-4014-a33e-c3d6b699c08e', 'options': 'recursive,force'} 2024-09-19T08:25:17.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:16 smithi157 ceph-mon[35143]: pgmap v317: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-19T08:25:18.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:17 smithi064 ceph-mon[34900]: scrub summary: idle 2024-09-19T08:25:18.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:17 smithi157 ceph-mon[35143]: scrub summary: idle 2024-09-19T08:25:18.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:17 smithi022 ceph-mon[30963]: scrub summary: idle 2024-09-19T08:25:19.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:18 smithi064 ceph-mon[34900]: pgmap v319: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 107 B/s wr, 5 op/s 2024-09-19T08:25:19.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:18 smithi157 ceph-mon[35143]: pgmap v319: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 107 B/s wr, 5 op/s 2024-09-19T08:25:19.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:18 smithi022 ceph-mon[30963]: pgmap v319: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 107 B/s wr, 5 op/s 2024-09-19T08:25:21.091 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:20 smithi064 ceph-mon[34900]: pgmap v320: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-19T08:25:21.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:20 smithi157 ceph-mon[35143]: pgmap v320: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-19T08:25:21.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:20 smithi022 ceph-mon[30963]: pgmap v320: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-19T08:25:23.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:22 smithi064 ceph-mon[34900]: pgmap v321: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-19T08:25:23.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:22 smithi157 ceph-mon[35143]: pgmap v321: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-19T08:25:23.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:22 smithi022 ceph-mon[30963]: pgmap v321: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-19T08:25:25.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:24 smithi064 ceph-mon[34900]: pgmap v322: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T08:25:25.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:24 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:25:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:24 smithi157 ceph-mon[35143]: pgmap v322: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T08:25:25.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:24 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:25:25.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:24 smithi022 ceph-mon[30963]: pgmap v322: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T08:25:25.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:24 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:25:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: pgmap v323: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T08:25:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:26 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: pgmap v323: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T08:25:27.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:26 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: pgmap v323: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T08:25:27.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:27.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:27.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:26 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-19T08:25:28.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: Unable to set osd_memory_target on smithi064 to 184598220: error parsing value: Value '184598220' is below minimum 939524096 2024-09-19T08:25:28.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi022 to 177.0M 2024-09-19T08:25:28.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: Unable to set osd_memory_target on smithi022 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T08:25:28.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T08:25:28.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: Unable to set osd_memory_target on smithi157 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T08:25:28.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:25:28.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:25:28.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:27 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:28.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-19T08:25:28.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: Unable to set osd_memory_target on smithi064 to 184598220: error parsing value: Value '184598220' is below minimum 939524096 2024-09-19T08:25:28.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi022 to 177.0M 2024-09-19T08:25:28.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: Unable to set osd_memory_target on smithi022 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T08:25:28.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T08:25:28.254 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: Unable to set osd_memory_target on smithi157 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T08:25:28.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:25:28.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:25:28.255 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:27 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-19T08:25:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: Unable to set osd_memory_target on smithi064 to 184598220: error parsing value: Value '184598220' is below minimum 939524096 2024-09-19T08:25:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi022 to 177.0M 2024-09-19T08:25:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: Unable to set osd_memory_target on smithi022 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-19T08:25:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:25:28.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: Adjusting osd_memory_target on smithi157 to 176.0M 2024-09-19T08:25:28.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: Unable to set osd_memory_target on smithi157 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T08:25:28.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:25:28.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:25:28.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:27 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:25:29.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:29 smithi064 ceph-mon[34900]: pgmap v324: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T08:25:29.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:29 smithi157 ceph-mon[35143]: pgmap v324: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T08:25:29.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:29 smithi022 ceph-mon[30963]: pgmap v324: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T08:25:31.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:31 smithi064 ceph-mon[34900]: pgmap v325: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-19T08:25:31.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:31 smithi157 ceph-mon[35143]: pgmap v325: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-19T08:25:31.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:31 smithi022 ceph-mon[30963]: pgmap v325: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-19T08:25:33.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:33 smithi064 ceph-mon[34900]: pgmap v326: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:25:33.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:33 smithi157 ceph-mon[35143]: pgmap v326: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:25:33.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:33 smithi022 ceph-mon[30963]: pgmap v326: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:25:35.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:35 smithi064 ceph-mon[34900]: pgmap v327: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:25:35.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:35 smithi157 ceph-mon[35143]: pgmap v327: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:25:35.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:35 smithi022 ceph-mon[30963]: pgmap v327: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T08:25:37.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:37 smithi064 ceph-mon[34900]: pgmap v328: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:25:37.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:37 smithi157 ceph-mon[35143]: pgmap v328: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:25:37.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:37 smithi022 ceph-mon[30963]: pgmap v328: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:25:39.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:39 smithi064 ceph-mon[34900]: pgmap v329: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T08:25:39.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:39 smithi157 ceph-mon[35143]: pgmap v329: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T08:25:39.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:39 smithi022 ceph-mon[30963]: pgmap v329: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T08:25:41.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:41 smithi064 ceph-mon[34900]: pgmap v330: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T08:25:41.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:41 smithi157 ceph-mon[35143]: pgmap v330: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T08:25:41.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:41 smithi022 ceph-mon[30963]: pgmap v330: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T08:25:43.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:43 smithi064 ceph-mon[34900]: pgmap v331: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:25:43.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:43 smithi157 ceph-mon[35143]: pgmap v331: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:25:43.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:43 smithi022 ceph-mon[30963]: pgmap v331: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:25:45.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:45 smithi064 ceph-mon[34900]: pgmap v332: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T08:25:45.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:45 smithi157 ceph-mon[35143]: pgmap v332: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T08:25:45.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:45 smithi022 ceph-mon[30963]: pgmap v332: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T08:25:47.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:25:47.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:47 smithi064 ceph-mon[34900]: pgmap v333: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:25:47.374 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:25:47.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:47 smithi022 ceph-mon[30963]: pgmap v333: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:25:47.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:47 smithi157 ceph-mon[35143]: pgmap v333: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:25:47.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.878+0000 7fced014b640 1 -- 172.21.15.22:0/508712164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcec8100e20 msgr2=0x7fcec8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:47.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.878+0000 7fced014b640 1 --2- 172.21.15.22:0/508712164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcec8100e20 0x7fcec8101200 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009900 tx=0x7fcec402a480 comp rx=0 tx=0).stop 2024-09-19T08:25:47.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.878+0000 7fced014b640 1 -- 172.21.15.22:0/508712164 shutdown_connections 2024-09-19T08:25:47.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.878+0000 7fced014b640 1 --2- 172.21.15.22:0/508712164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcec8102ef0 0x7fcec810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.878+0000 7fced014b640 1 --2- 172.21.15.22:0/508712164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec8102570 0x7fcec81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.878+0000 7fced014b640 1 --2- 172.21.15.22:0/508712164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcec8100e20 0x7fcec8101200 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.878+0000 7fced014b640 1 -- 172.21.15.22:0/508712164 >> 172.21.15.22:0/508712164 conn(0x7fcec80fc7d0 msgr2=0x7fcec80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:47.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.878+0000 7fced014b640 1 -- 172.21.15.22:0/508712164 shutdown_connections 2024-09-19T08:25:47.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.878+0000 7fced014b640 1 -- 172.21.15.22:0/508712164 wait complete. 2024-09-19T08:25:47.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.878+0000 7fced014b640 1 Processor -- start 2024-09-19T08:25:47.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fced014b640 1 -- start start 2024-09-19T08:25:47.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fced014b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcec8100e20 0x7fcec81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fced014b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec8102570 0x7fcec8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fced014b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcec8102ef0 0x7fcec819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fced014b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81120a0 con 0x7fcec8102ef0 2024-09-19T08:25:47.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fced014b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcec8111f20 con 0x7fcec8100e20 2024-09-19T08:25:47.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fced014b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcec8112220 con 0x7fcec8102570 2024-09-19T08:25:47.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fcece6e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcec8102ef0 0x7fcec819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fcece6e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcec8102ef0 0x7fcec819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47456/0 (socket says 172.21.15.22:47456) 2024-09-19T08:25:47.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fcece6e8640 1 -- 172.21.15.22:0/155782632 learned_addr learned my addr 172.21.15.22:0/155782632 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fcecdee7640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcec8100e20 0x7fcec81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec8102570 0x7fcec8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fcece6e8640 1 -- 172.21.15.22:0/155782632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec8102570 msgr2=0x7fcec8198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fcece6e8640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec8102570 0x7fcec8198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fcece6e8640 1 -- 172.21.15.22:0/155782632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcec8100e20 msgr2=0x7fcec81984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fcece6e8640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcec8100e20 0x7fcec81984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fcece6e8640 1 -- 172.21.15.22:0/155782632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec819d470 con 0x7fcec8102ef0 2024-09-19T08:25:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.879+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec8102570 0x7fcec8198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.880+0000 7fcece6e8640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcec8102ef0 0x7fcec819cd90 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fcec000cce0 tx=0x7fcec000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.880+0000 7fcebeffd640 1 -- 172.21.15.22:0/155782632 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec000f450 con 0x7fcec8102ef0 2024-09-19T08:25:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.880+0000 7fcebeffd640 1 -- 172.21.15.22:0/155782632 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec000fad0 con 0x7fcec8102ef0 2024-09-19T08:25:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.880+0000 7fced014b640 1 -- 172.21.15.22:0/155782632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec819d760 con 0x7fcec8102ef0 2024-09-19T08:25:47.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.880+0000 7fcebeffd640 1 -- 172.21.15.22:0/155782632 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0018c10 con 0x7fcec8102ef0 2024-09-19T08:25:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.880+0000 7fced014b640 1 -- 172.21.15.22:0/155782632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec81a5040 con 0x7fcec8102ef0 2024-09-19T08:25:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.882+0000 7fcebeffd640 1 -- 172.21.15.22:0/155782632 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcec000f5f0 con 0x7fcec8102ef0 2024-09-19T08:25:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.882+0000 7fced2524740 1 Processor -- start 2024-09-19T08:25:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.883+0000 7fced2524740 1 -- start start 2024-09-19T08:25:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.883+0000 7fced2524740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 0x558de60de720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.883+0000 7fced2524740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60dec60 0x558de60ed550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.883+0000 7fced2524740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60eda90 0x558de60efe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.883+0000 7fced2524740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558de60442e0 con 0x558de60dec60 2024-09-19T08:25:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.883+0000 7fced2524740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558de5febce0 con 0x558de60de340 2024-09-19T08:25:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.883+0000 7fced2524740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558de60402a0 con 0x558de60eda90 2024-09-19T08:25:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.883+0000 7fcecdee7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60dec60 0x558de60ed550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.883+0000 7fcecdee7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60dec60 0x558de60ed550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47466/0 (socket says 172.21.15.22:47466) 2024-09-19T08:25:47.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.883+0000 7fcecdee7640 1 -- 172.21.15.22:0/2111545133 learned_addr learned my addr 172.21.15.22:0/2111545133 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:47.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.883+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/2111545133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 0x558de60de720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.883+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/2111545133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60eda90 0x558de60efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.884+0000 7fcebeffd640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fceb40a33b0 0x7fceb40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.884+0000 7fcecdee7640 1 -- 172.21.15.22:0/2111545133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60eda90 msgr2=0x558de60efe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:47.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.884+0000 7fcecdee7640 1 --2- 172.21.15.22:0/2111545133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60eda90 0x558de60efe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.884+0000 7fcecdee7640 1 -- 172.21.15.22:0/2111545133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 msgr2=0x558de60de720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:47.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.884+0000 7fcecdee7640 1 --2- 172.21.15.22:0/2111545133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 0x558de60de720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.884+0000 7fcece6e8640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fceb40a33b0 0x7fceb40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.884+0000 7fcecdee7640 1 -- 172.21.15.22:0/2111545133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558de5fd0440 con 0x558de60dec60 2024-09-19T08:25:47.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.884+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/2111545133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 0x558de60de720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:25:47.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.884+0000 7fcebeffd640 1 -- 172.21.15.22:0/155782632 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fcec00d0480 con 0x7fcec8102ef0 2024-09-19T08:25:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.884+0000 7fcecdee7640 1 --2- 172.21.15.22:0/2111545133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60dec60 0x558de60ed550 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009900 tx=0x7fcec4004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434a4194f3077fb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.884+0000 7fceb3fff640 1 -- 172.21.15.22:0/2111545133 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4035030 con 0x558de60dec60 2024-09-19T08:25:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.884+0000 7fceb3fff640 1 -- 172.21.15.22:0/2111545133 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec4040070 con 0x558de60dec60 2024-09-19T08:25:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.884+0000 7fceb3fff640 1 -- 172.21.15.22:0/2111545133 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4031a70 con 0x558de60dec60 2024-09-19T08:25:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.884+0000 7fcece6e8640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fceb40a33b0 0x7fceb40a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcec009c680 tx=0x7fcec006e470 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.885+0000 7fced2524740 1 -- 172.21.15.22:0/2111545133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60dec60 msgr2=0x558de60ed550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.885+0000 7fced2524740 1 --2- 172.21.15.22:0/2111545133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60dec60 0x558de60ed550 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009900 tx=0x7fcec4004170 comp rx=0 tx=0).stop 2024-09-19T08:25:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.885+0000 7fced2524740 1 -- 172.21.15.22:0/2111545133 shutdown_connections 2024-09-19T08:25:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.885+0000 7fced2524740 1 --2- 172.21.15.22:0/2111545133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60eda90 0x558de60efe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.885+0000 7fced2524740 1 --2- 172.21.15.22:0/2111545133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60dec60 0x558de60ed550 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.885+0000 7fced2524740 1 --2- 172.21.15.22:0/2111545133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 0x558de60de720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.885+0000 7fced2524740 1 -- 172.21.15.22:0/2111545133 >> 172.21.15.22:0/2111545133 conn(0x558de60daa30 msgr2=0x558de60dae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.885+0000 7fced2524740 1 -- 172.21.15.22:0/2111545133 shutdown_connections 2024-09-19T08:25:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.885+0000 7fced2524740 1 -- 172.21.15.22:0/2111545133 wait complete. 2024-09-19T08:25:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.886+0000 7fced2524740 1 -- start start 2024-09-19T08:25:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.886+0000 7fced2524740 1 Processor -- start 2024-09-19T08:25:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.886+0000 7fced2524740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 0x558de60ed420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.886+0000 7fced2524740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60dec60 0x558de60ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.886+0000 7fced2524740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60eda90 0x558de60eac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.886+0000 7fced2524740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558de60f0660 con 0x558de60eda90 2024-09-19T08:25:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.886+0000 7fced2524740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558de5fff050 con 0x558de60de340 2024-09-19T08:25:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.886+0000 7fced2524740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558de60f07e0 con 0x558de60dec60 2024-09-19T08:25:47.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.886+0000 7fced2524740 10 client.0 _ll_register_callbacks cb 0x558de603f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.886+0000 7fcecd6e6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60eda90 0x558de60eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.886+0000 7fcecd6e6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60eda90 0x558de60eac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47468/0 (socket says 172.21.15.22:47468) 2024-09-19T08:25:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.886+0000 7fcecd6e6640 1 -- 172.21.15.22:0/34762081 learned_addr learned my addr 172.21.15.22:0/34762081 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.886+0000 7fcecdee7640 1 --2- 172.21.15.22:0/34762081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60dec60 0x558de60ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.886+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/34762081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 0x558de60ed420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.887+0000 7fcecd6e6640 1 -- 172.21.15.22:0/34762081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60dec60 msgr2=0x558de60ea6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.887+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/34762081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60dec60 0x558de60ea6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.887+0000 7fcecd6e6640 1 -- 172.21.15.22:0/34762081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 msgr2=0x558de60ed420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.887+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/34762081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 0x558de60ed420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.887+0000 7fcecd6e6640 1 -- 172.21.15.22:0/34762081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558de5db53c0 con 0x558de60eda90 2024-09-19T08:25:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.887+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/34762081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60eda90 0x558de60eac50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fceb8014f80 tx=0x7fceb8016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30a7b7dfe3ce772 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.887+0000 7fceb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:47.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.887+0000 7fceb17fa640 1 -- 172.21.15.22:0/34762081 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb80140d0 con 0x558de60eda90 2024-09-19T08:25:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.887+0000 7fced2524740 1 -- 172.21.15.22:0/34762081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcec4040070 con 0x558de60eda90 2024-09-19T08:25:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.887+0000 7fceb17fa640 1 -- 172.21.15.22:0/34762081 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fceb8014270 con 0x558de60eda90 2024-09-19T08:25:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.887+0000 7fceb17fa640 1 -- 172.21.15.22:0/34762081 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fceb8005020 con 0x558de60eda90 2024-09-19T08:25:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.887+0000 7fced2524740 10 client.17214 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.887+0000 7fced2524740 1 -- 172.21.15.22:0/34762081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558de60fc0e0 con 0x558de60eda90 2024-09-19T08:25:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.887+0000 7fceb17fa640 1 -- 172.21.15.22:0/34762081 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec4040070 con 0x558de60eda90 2024-09-19T08:25:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.888+0000 7fceb17fa640 1 -- 172.21.15.22:0/34762081 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fceb8020020 con 0x558de60eda90 2024-09-19T08:25:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.888+0000 7fced2524740 10 client.17214 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.888+0000 7fced2524740 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:25:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.888+0000 7fced2524740 20 client.17214 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:47.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.888+0000 7fced2524740 1 --2- 172.21.15.22:0/34762081 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558de60fd4f0 0x558de60ff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.888+0000 7fced2524740 4 client.17214 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:25:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.888+0000 7fced2524740 1 -- 172.21.15.22:0/34762081 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558de6051850 con 0x558de60fd4f0 2024-09-19T08:25:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.888+0000 7fcecdee7640 1 --2- 172.21.15.22:0/34762081 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558de60fd4f0 0x558de60ff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.888+0000 7fceb17fa640 10 client.17214 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.888+0000 7fcecdee7640 1 --2- 172.21.15.22:0/34762081 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558de60fd4f0 0x558de60ff8e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dafd58c5f1ac6c5a server_cookie=1c2416bb1a10e338 in_seq=0 out_seq=0 2024-09-19T08:25:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.889+0000 7fceb17fa640 1 -- 172.21.15.22:0/34762081 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558de6051850 con 0x558de60fd4f0 2024-09-19T08:25:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.889+0000 7fceb17fa640 10 client.17214 handle_command_reply: tid=0 2024-09-19T08:25:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.889+0000 7fced2524740 1 client.17214 shutdown 2024-09-19T08:25:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.889+0000 7fced2524740 1 -- 172.21.15.22:0/34762081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60eda90 msgr2=0x558de60eac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.889+0000 7fced2524740 1 --2- 172.21.15.22:0/34762081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60eda90 0x558de60eac50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fceb8014f80 tx=0x7fceb8016040 comp rx=0 tx=0).stop 2024-09-19T08:25:47.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.889+0000 7fced2524740 1 -- 172.21.15.22:0/34762081 shutdown_connections 2024-09-19T08:25:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.889+0000 7fced2524740 1 --2- 172.21.15.22:0/34762081 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558de60fd4f0 0x558de60ff8e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.889+0000 7fced2524740 1 --2- 172.21.15.22:0/34762081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60eda90 0x558de60eac50 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.889+0000 7fced2524740 1 --2- 172.21.15.22:0/34762081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60dec60 0x558de60ea6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.889+0000 7fceb17fa640 0 client.17214 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.889+0000 7fced2524740 1 --2- 172.21.15.22:0/34762081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 0x558de60ed420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.889+0000 7fced2524740 1 -- 172.21.15.22:0/34762081 >> 172.21.15.22:0/34762081 conn(0x558de60daa30 msgr2=0x558de60e0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.889+0000 7fced2524740 1 -- 172.21.15.22:0/34762081 shutdown_connections 2024-09-19T08:25:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.889+0000 7fced2524740 1 -- 172.21.15.22:0/34762081 wait complete. 2024-09-19T08:25:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.890+0000 7fced2524740 20 client.17214 trim_cache size 0 max 16384 2024-09-19T08:25:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fced2524740 1 Processor -- start 2024-09-19T08:25:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fced2524740 1 -- start start 2024-09-19T08:25:47.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fced2524740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 0x558de60f9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fced2524740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60dec60 0x558de6105160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fced2524740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60eda90 0x558de61056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fced2524740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81adfb0 con 0x558de60eda90 2024-09-19T08:25:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fced2524740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81ab820 con 0x558de60de340 2024-09-19T08:25:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fced2524740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81a9690 con 0x558de60dec60 2024-09-19T08:25:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fcecd6e6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60eda90 0x558de61056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fcecd6e6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60eda90 0x558de61056a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47474/0 (socket says 172.21.15.22:47474) 2024-09-19T08:25:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fcecd6e6640 1 -- 172.21.15.22:0/4086584509 learned_addr learned my addr 172.21.15.22:0/4086584509 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fcecdee7640 1 --2- 172.21.15.22:0/4086584509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60dec60 0x558de6105160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/4086584509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 0x558de60f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fcecd6e6640 1 -- 172.21.15.22:0/4086584509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60dec60 msgr2=0x558de6105160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:47.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/4086584509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60dec60 0x558de6105160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fcecd6e6640 1 -- 172.21.15.22:0/4086584509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 msgr2=0x558de60f9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/4086584509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 0x558de60f9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.907+0000 7fcecd6e6640 1 -- 172.21.15.22:0/4086584509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558de60dea90 con 0x558de60eda90 2024-09-19T08:25:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.908+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/4086584509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60eda90 0x558de61056a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fceb80059b0 tx=0x7fceb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4629ed4d3c8e4dd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.908+0000 7fceb37fe640 1 -- 172.21.15.22:0/4086584509 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb800c870 con 0x558de60eda90 2024-09-19T08:25:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.908+0000 7fceb37fe640 1 -- 172.21.15.22:0/4086584509 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceb8014cd0 con 0x558de60eda90 2024-09-19T08:25:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.908+0000 7fceb37fe640 1 -- 172.21.15.22:0/4086584509 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8012c70 con 0x558de60eda90 2024-09-19T08:25:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.908+0000 7fced2524740 1 -- 172.21.15.22:0/4086584509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60eda90 msgr2=0x558de61056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.908+0000 7fced2524740 1 --2- 172.21.15.22:0/4086584509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60eda90 0x558de61056a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fceb80059b0 tx=0x7fceb80046c0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 -- 172.21.15.22:0/4086584509 shutdown_connections 2024-09-19T08:25:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 --2- 172.21.15.22:0/4086584509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60eda90 0x558de61056a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 --2- 172.21.15.22:0/4086584509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60dec60 0x558de6105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 --2- 172.21.15.22:0/4086584509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60de340 0x558de60f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 -- 172.21.15.22:0/4086584509 >> 172.21.15.22:0/4086584509 conn(0x558de60daa30 msgr2=0x558de60e5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 -- 172.21.15.22:0/4086584509 shutdown_connections 2024-09-19T08:25:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 -- 172.21.15.22:0/4086584509 wait complete. 2024-09-19T08:25:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 -- start start 2024-09-19T08:25:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 Processor -- start 2024-09-19T08:25:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60de340 0x558de60f1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60dec60 0x558de60f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60eda90 0x558de60f2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81a9ce0 con 0x558de60de340 2024-09-19T08:25:47.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81ab150 con 0x558de60dec60 2024-09-19T08:25:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81aa3b0 con 0x558de60eda90 2024-09-19T08:25:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.909+0000 7fced2524740 10 client.0 _ll_register_callbacks cb 0x7fceb800ce10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fcecd6e6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60de340 0x558de60f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fcecd6e6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60de340 0x558de60f1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47486/0 (socket says 172.21.15.22:47486) 2024-09-19T08:25:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fcecd6e6640 1 -- 172.21.15.22:0/1144421527 learned_addr learned my addr 172.21.15.22:0/1144421527 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fcecdee7640 1 --2- 172.21.15.22:0/1144421527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60dec60 0x558de60f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/1144421527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60eda90 0x558de60f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fcecd6e6640 1 -- 172.21.15.22:0/1144421527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60eda90 msgr2=0x558de60f2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/1144421527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60eda90 0x558de60f2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fcecd6e6640 1 -- 172.21.15.22:0/1144421527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60dec60 msgr2=0x558de60f1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/1144421527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60dec60 0x558de60f1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fcecd6e6640 1 -- 172.21.15.22:0/1144421527 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558de6167130 con 0x558de60de340 2024-09-19T08:25:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fcecd6e6640 1 --2- 172.21.15.22:0/1144421527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60de340 0x558de60f1710 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fceb8017040 tx=0x7fceb8014430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80d997f497db967 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fceb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fceb17fa640 1 -- 172.21.15.22:0/1144421527 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb803b060 con 0x558de60de340 2024-09-19T08:25:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fceb17fa640 1 -- 172.21.15.22:0/1144421527 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fceb8005690 con 0x558de60de340 2024-09-19T08:25:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.910+0000 7fced2524740 1 -- 172.21.15.22:0/1144421527 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceb8014cd0 con 0x558de60de340 2024-09-19T08:25:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.911+0000 7fceb17fa640 1 -- 172.21.15.22:0/1144421527 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fceb802f030 con 0x558de60de340 2024-09-19T08:25:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.911+0000 7fced2524740 10 client.17220 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.911+0000 7fced2524740 1 -- 172.21.15.22:0/1144421527 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558de6168380 con 0x558de60de340 2024-09-19T08:25:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.911+0000 7fceb17fa640 1 -- 172.21.15.22:0/1144421527 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceb8014cd0 con 0x558de60de340 2024-09-19T08:25:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.911+0000 7fceb17fa640 1 -- 172.21.15.22:0/1144421527 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fceb8020070 con 0x558de60de340 2024-09-19T08:25:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.911+0000 7fced2524740 10 client.17220 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.911+0000 7fced2524740 10 client.17220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:25:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.911+0000 7fced2524740 20 client.17220 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.911+0000 7fced2524740 1 --2- 172.21.15.22:0/1144421527 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558de6168930 0x558de616ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.911+0000 7fced2524740 4 client.17220 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:25:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.911+0000 7fced2524740 1 -- 172.21.15.22:0/1144421527 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558de604bd60 con 0x558de6168930 2024-09-19T08:25:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.911+0000 7fcecdee7640 1 --2- 172.21.15.22:0/1144421527 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558de6168930 0x558de616ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fcecdee7640 1 --2- 172.21.15.22:0/1144421527 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558de6168930 0x558de616ad20 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc3f622bf99246b7 server_cookie=ef8507fa4f8d45bc in_seq=0 out_seq=0 2024-09-19T08:25:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fceb17fa640 10 client.17220 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fceb17fa640 1 -- 172.21.15.22:0/1144421527 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558de604bd60 con 0x558de6168930 2024-09-19T08:25:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fceb17fa640 10 client.17220 handle_command_reply: tid=0 2024-09-19T08:25:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fced2524740 1 client.17220 shutdown 2024-09-19T08:25:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fced2524740 1 -- 172.21.15.22:0/1144421527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60de340 msgr2=0x558de60f1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fced2524740 1 --2- 172.21.15.22:0/1144421527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60de340 0x558de60f1710 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fceb8017040 tx=0x7fceb8014430 comp rx=0 tx=0).stop 2024-09-19T08:25:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fced2524740 1 -- 172.21.15.22:0/1144421527 shutdown_connections 2024-09-19T08:25:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fced2524740 1 --2- 172.21.15.22:0/1144421527 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x558de6168930 0x558de616ad20 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fced2524740 1 --2- 172.21.15.22:0/1144421527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558de60eda90 0x558de60f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fced2524740 1 --2- 172.21.15.22:0/1144421527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558de60dec60 0x558de60f1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fced2524740 1 --2- 172.21.15.22:0/1144421527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558de60de340 0x558de60f1710 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fceb17fa640 0 client.17220 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:47.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fced2524740 1 -- 172.21.15.22:0/1144421527 >> 172.21.15.22:0/1144421527 conn(0x558de60daa30 msgr2=0x558de60ebf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:47.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fced2524740 1 -- 172.21.15.22:0/1144421527 shutdown_connections 2024-09-19T08:25:47.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fced2524740 1 -- 172.21.15.22:0/1144421527 wait complete. 2024-09-19T08:25:47.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.912+0000 7fced2524740 20 client.17220 trim_cache size 0 max 16384 2024-09-19T08:25:47.933 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:25:47.933 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-19T08:25:47.933 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-09-19T08:25:47.933 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:25:47.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.913+0000 7fced014b640 1 -- 172.21.15.22:0/155782632 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fceb40a33b0 msgr2=0x7fceb40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:47.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.913+0000 7fced014b640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fceb40a33b0 0x7fceb40a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcec009c680 tx=0x7fcec006e470 comp rx=0 tx=0).stop 2024-09-19T08:25:47.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.913+0000 7fced014b640 1 -- 172.21.15.22:0/155782632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcec8102ef0 msgr2=0x7fcec819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.913+0000 7fced014b640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcec8102ef0 0x7fcec819cd90 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fcec000cce0 tx=0x7fcec000a620 comp rx=0 tx=0).stop 2024-09-19T08:25:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.913+0000 7fced014b640 1 -- 172.21.15.22:0/155782632 shutdown_connections 2024-09-19T08:25:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.913+0000 7fced014b640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fceb40a33b0 0x7fceb40a5870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.913+0000 7fced014b640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcec8102ef0 0x7fcec819cd90 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.913+0000 7fced014b640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec8102570 0x7fcec8198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.913+0000 7fced014b640 1 --2- 172.21.15.22:0/155782632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcec8100e20 0x7fcec81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:47.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.913+0000 7fced014b640 1 -- 172.21.15.22:0/155782632 >> 172.21.15.22:0/155782632 conn(0x7fcec80fc7d0 msgr2=0x7fcec80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:47.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.913+0000 7fced014b640 1 -- 172.21.15.22:0/155782632 shutdown_connections 2024-09-19T08:25:47.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:47.913+0000 7fced014b640 1 -- 172.21.15.22:0/155782632 wait complete. 2024-09-19T08:25:48.819 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:25:48.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:25:49.087 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:25:49.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:49 smithi064 ceph-mon[34900]: pgmap v334: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-19T08:25:49.364 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:49 smithi022 ceph-mon[30963]: pgmap v334: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-19T08:25:49.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:49 smithi157 ceph-mon[35143]: pgmap v334: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-19T08:25:49.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.588+0000 7fe226ed3640 1 -- 172.21.15.22:0/4153549095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe220102790 msgr2=0x7fe2201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.588+0000 7fe226ed3640 1 --2- 172.21.15.22:0/4153549095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe220102790 0x7fe2201067e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe210009900 tx=0x7fe21002a410 comp rx=0 tx=0).stop 2024-09-19T08:25:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.588+0000 7fe226ed3640 1 -- 172.21.15.22:0/4153549095 shutdown_connections 2024-09-19T08:25:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.588+0000 7fe226ed3640 1 --2- 172.21.15.22:0/4153549095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe220106f10 0x7fe22010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.588+0000 7fe226ed3640 1 --2- 172.21.15.22:0/4153549095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe220102790 0x7fe2201067e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.588+0000 7fe226ed3640 1 --2- 172.21.15.22:0/4153549095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe220101de0 0x7fe2201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.588+0000 7fe226ed3640 1 -- 172.21.15.22:0/4153549095 >> 172.21.15.22:0/4153549095 conn(0x7fe2200fd8c0 msgr2=0x7fe2200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.588+0000 7fe226ed3640 1 -- 172.21.15.22:0/4153549095 shutdown_connections 2024-09-19T08:25:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.588+0000 7fe226ed3640 1 -- 172.21.15.22:0/4153549095 wait complete. 2024-09-19T08:25:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe226ed3640 1 Processor -- start 2024-09-19T08:25:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe226ed3640 1 -- start start 2024-09-19T08:25:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe226ed3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe220101de0 0x7fe2200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:49.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe226ed3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe220102790 0x7fe220074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe226ed3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe220106f10 0x7fe220074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe226ed3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe220111f70 con 0x7fe220102790 2024-09-19T08:25:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe226ed3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe220111df0 con 0x7fe220101de0 2024-09-19T08:25:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe226ed3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201120f0 con 0x7fe220106f10 2024-09-19T08:25:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe217fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe220102790 0x7fe220074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe217fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe220102790 0x7fe220074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47512/0 (socket says 172.21.15.22:47512) 2024-09-19T08:25:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe217fff640 1 -- 172.21.15.22:0/242879997 learned_addr learned my addr 172.21.15.22:0/242879997 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe225470640 1 --2- 172.21.15.22:0/242879997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe220106f10 0x7fe220074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe224c6f640 1 --2- 172.21.15.22:0/242879997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe220101de0 0x7fe2200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe217fff640 1 -- 172.21.15.22:0/242879997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe220106f10 msgr2=0x7fe220074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe217fff640 1 --2- 172.21.15.22:0/242879997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe220106f10 0x7fe220074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe217fff640 1 -- 172.21.15.22:0/242879997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe220101de0 msgr2=0x7fe2200744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe217fff640 1 --2- 172.21.15.22:0/242879997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe220101de0 0x7fe2200744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.589+0000 7fe217fff640 1 -- 172.21.15.22:0/242879997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2201a6870 con 0x7fe220102790 2024-09-19T08:25:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.590+0000 7fe217fff640 1 --2- 172.21.15.22:0/242879997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe220102790 0x7fe220074a30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe210006010 tx=0x7fe2100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.590+0000 7fe215ffb640 1 -- 172.21.15.22:0/242879997 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21003f030 con 0x7fe220102790 2024-09-19T08:25:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.590+0000 7fe215ffb640 1 -- 172.21.15.22:0/242879997 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe210003ce0 con 0x7fe220102790 2024-09-19T08:25:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.590+0000 7fe226ed3640 1 -- 172.21.15.22:0/242879997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2201a6b00 con 0x7fe220102790 2024-09-19T08:25:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.590+0000 7fe215ffb640 1 -- 172.21.15.22:0/242879997 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210035070 con 0x7fe220102790 2024-09-19T08:25:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.590+0000 7fe226ed3640 1 -- 172.21.15.22:0/242879997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2201a6fe0 con 0x7fe220102790 2024-09-19T08:25:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.591+0000 7fe226ed3640 1 -- 172.21.15.22:0/242879997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2201021c0 con 0x7fe220102790 2024-09-19T08:25:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.592+0000 7fe215ffb640 1 -- 172.21.15.22:0/242879997 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe210042030 con 0x7fe220102790 2024-09-19T08:25:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.593+0000 7fe215ffb640 1 --2- 172.21.15.22:0/242879997 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe2080a33b0 0x7fe2080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.593+0000 7fe215ffb640 1 -- 172.21.15.22:0/242879997 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fe2100f0340 con 0x7fe220102790 2024-09-19T08:25:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.593+0000 7fe224c6f640 1 --2- 172.21.15.22:0/242879997 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe2080a33b0 0x7fe2080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.593+0000 7fe224c6f640 1 --2- 172.21.15.22:0/242879997 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe2080a33b0 0x7fe2080a5870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00bbe0 tx=0x7fe21c00b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.597+0000 7fe215ffb640 1 -- 172.21.15.22:0/242879997 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe2100bc780 con 0x7fe220102790 2024-09-19T08:25:49.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.808+0000 7fe226ed3640 1 -- 172.21.15.22:0/242879997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe22002d7c0 con 0x7fe220102790 2024-09-19T08:25:49.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.809+0000 7fe215ffb640 1 -- 172.21.15.22:0/242879997 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11909 (secure 0 0 0) 0x7fe2100c0a80 con 0x7fe220102790 2024-09-19T08:25:49.811 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:25:49.811 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":36,"btime":"2024-09-19T08:24:38:729727+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:24:38.451170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:25:49.811 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 36 2024-09-19T08:25:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.812+0000 7fe226ed3640 1 -- 172.21.15.22:0/242879997 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe2080a33b0 msgr2=0x7fe2080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.812+0000 7fe226ed3640 1 --2- 172.21.15.22:0/242879997 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe2080a33b0 0x7fe2080a5870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00bbe0 tx=0x7fe21c00b390 comp rx=0 tx=0).stop 2024-09-19T08:25:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.812+0000 7fe226ed3640 1 -- 172.21.15.22:0/242879997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe220102790 msgr2=0x7fe220074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.812+0000 7fe226ed3640 1 --2- 172.21.15.22:0/242879997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe220102790 0x7fe220074a30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe210006010 tx=0x7fe2100042f0 comp rx=0 tx=0).stop 2024-09-19T08:25:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.812+0000 7fe226ed3640 1 -- 172.21.15.22:0/242879997 shutdown_connections 2024-09-19T08:25:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.812+0000 7fe226ed3640 1 --2- 172.21.15.22:0/242879997 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe2080a33b0 0x7fe2080a5870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.812+0000 7fe226ed3640 1 --2- 172.21.15.22:0/242879997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe220106f10 0x7fe220074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.812+0000 7fe226ed3640 1 --2- 172.21.15.22:0/242879997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe220102790 0x7fe220074a30 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.812+0000 7fe226ed3640 1 --2- 172.21.15.22:0/242879997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe220101de0 0x7fe2200744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.812+0000 7fe226ed3640 1 -- 172.21.15.22:0/242879997 >> 172.21.15.22:0/242879997 conn(0x7fe2200fd8c0 msgr2=0x7fe2200fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.812+0000 7fe226ed3640 1 -- 172.21.15.22:0/242879997 shutdown_connections 2024-09-19T08:25:49.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:49.812+0000 7fe226ed3640 1 -- 172.21.15.22:0/242879997 wait complete. 2024-09-19T08:25:50.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:50 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/242879997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:25:50.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:50 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/242879997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:25:50.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:50 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/242879997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:25:50.649 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:25:50.915 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:25:51.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:51 smithi064 ceph-mon[34900]: pgmap v335: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:25:51.366 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:51 smithi022 ceph-mon[30963]: pgmap v335: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:25:51.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.449+0000 7fb515b34640 1 -- 172.21.15.22:0/2284573032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb51010c170 msgr2=0x7fb51010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:51.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.449+0000 7fb515b34640 1 --2- 172.21.15.22:0/2284573032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb51010c170 0x7fb51010e610 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb508009ca0 tx=0x7fb50802a950 comp rx=0 tx=0).stop 2024-09-19T08:25:51.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.449+0000 7fb515b34640 1 -- 172.21.15.22:0/2284573032 shutdown_connections 2024-09-19T08:25:51.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.449+0000 7fb515b34640 1 --2- 172.21.15.22:0/2284573032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb51010c170 0x7fb51010e610 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.449+0000 7fb515b34640 1 --2- 172.21.15.22:0/2284573032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb510103660 0x7fb51010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.449+0000 7fb515b34640 1 --2- 172.21.15.22:0/2284573032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb510102d40 0x7fb510103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.449+0000 7fb515b34640 1 -- 172.21.15.22:0/2284573032 >> 172.21.15.22:0/2284573032 conn(0x7fb5100fc7a0 msgr2=0x7fb5100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:51.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.449+0000 7fb515b34640 1 -- 172.21.15.22:0/2284573032 shutdown_connections 2024-09-19T08:25:51.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.449+0000 7fb515b34640 1 -- 172.21.15.22:0/2284573032 wait complete. 2024-09-19T08:25:51.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.449+0000 7fb515b34640 1 Processor -- start 2024-09-19T08:25:51.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.449+0000 7fb515b34640 1 -- start start 2024-09-19T08:25:51.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb515b34640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb510102d40 0x7fb510198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb515b34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb510103660 0x7fb5101989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb515b34640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb51010c170 0x7fb51019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb515b34640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb510111f50 con 0x7fb510103660 2024-09-19T08:25:51.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb515b34640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb510111dd0 con 0x7fb51010c170 2024-09-19T08:25:51.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb515b34640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101120d0 con 0x7fb510102d40 2024-09-19T08:25:51.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb506ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb510103660 0x7fb5101989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb506ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb510103660 0x7fb5101989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47536/0 (socket says 172.21.15.22:47536) 2024-09-19T08:25:51.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb506ffd640 1 -- 172.21.15.22:0/3555434046 learned_addr learned my addr 172.21.15.22:0/3555434046 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:51.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb50f7fe640 1 --2- 172.21.15.22:0/3555434046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb510102d40 0x7fb510198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb50ffff640 1 --2- 172.21.15.22:0/3555434046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb51010c170 0x7fb51019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb506ffd640 1 -- 172.21.15.22:0/3555434046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb510102d40 msgr2=0x7fb510198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:51.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb506ffd640 1 --2- 172.21.15.22:0/3555434046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb510102d40 0x7fb510198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb506ffd640 1 -- 172.21.15.22:0/3555434046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb51010c170 msgr2=0x7fb51019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:51.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb506ffd640 1 --2- 172.21.15.22:0/3555434046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb51010c170 0x7fb51019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.450+0000 7fb506ffd640 1 -- 172.21.15.22:0/3555434046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51019d410 con 0x7fb510103660 2024-09-19T08:25:51.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.451+0000 7fb506ffd640 1 --2- 172.21.15.22:0/3555434046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb510103660 0x7fb5101989a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800e9f0 tx=0x7fb4f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:51.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.451+0000 7fb50d7fa640 1 -- 172.21.15.22:0/3555434046 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f800ccc0 con 0x7fb510103660 2024-09-19T08:25:51.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.451+0000 7fb515b34640 1 -- 172.21.15.22:0/3555434046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51019d700 con 0x7fb510103660 2024-09-19T08:25:51.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.451+0000 7fb50d7fa640 1 -- 172.21.15.22:0/3555434046 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f800ce60 con 0x7fb510103660 2024-09-19T08:25:51.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.451+0000 7fb50d7fa640 1 -- 172.21.15.22:0/3555434046 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8010690 con 0x7fb510103660 2024-09-19T08:25:51.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.451+0000 7fb515b34640 1 -- 172.21.15.22:0/3555434046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5101a4fe0 con 0x7fb510103660 2024-09-19T08:25:51.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.454+0000 7fb517f0d740 1 Processor -- start 2024-09-19T08:25:51.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.454+0000 7fb50d7fa640 1 -- 172.21.15.22:0/3555434046 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb4f80046c0 con 0x7fb510103660 2024-09-19T08:25:51.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.454+0000 7fb517f0d740 1 -- start start 2024-09-19T08:25:51.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.454+0000 7fb517f0d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 0x55d7515df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.454+0000 7fb517f0d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 0x55d7515dfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.454+0000 7fb517f0d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515ea370 0x55d7515ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.454+0000 7fb517f0d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d7515452f0 con 0x55d7515ea370 2024-09-19T08:25:51.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.454+0000 7fb517f0d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d7514eccf0 con 0x55d7515df5d0 2024-09-19T08:25:51.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.454+0000 7fb517f0d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d7515412b0 con 0x55d7515e0b40 2024-09-19T08:25:51.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.454+0000 7fb506ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515ea370 0x55d7515ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.454+0000 7fb506ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515ea370 0x55d7515ec760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47546/0 (socket says 172.21.15.22:47546) 2024-09-19T08:25:51.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.454+0000 7fb50ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 0x55d7515df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.454+0000 7fb506ffd640 1 -- 172.21.15.22:0/3199726021 learned_addr learned my addr 172.21.15.22:0/3199726021 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:51.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.455+0000 7fb50f7fe640 1 --2- 172.21.15.22:0/3199726021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 0x55d7515dfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.455+0000 7fb506ffd640 1 -- 172.21.15.22:0/3199726021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 msgr2=0x55d7515df090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:51.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.455+0000 7fb506ffd640 1 --2- 172.21.15.22:0/3199726021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 0x55d7515df090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.455+0000 7fb506ffd640 1 -- 172.21.15.22:0/3199726021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 msgr2=0x55d7515dfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:51.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.455+0000 7fb506ffd640 1 --2- 172.21.15.22:0/3199726021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 0x55d7515dfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.455+0000 7fb506ffd640 1 -- 172.21.15.22:0/3199726021 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7514d13a0 con 0x55d7515ea370 2024-09-19T08:25:51.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.455+0000 7fb506ffd640 1 --2- 172.21.15.22:0/3199726021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515ea370 0x55d7515ec760 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80cd770 tx=0x7fb4f80d8270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ae6e833afcbaa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:51.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.455+0000 7fb50d7fa640 1 --2- 172.21.15.22:0/3555434046 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb4fc0a33b0 0x7fb4fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.455+0000 7fb505ffb640 1 -- 172.21.15.22:0/3199726021 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f80e3070 con 0x55d7515ea370 2024-09-19T08:25:51.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.455+0000 7fb50ffff640 1 --2- 172.21.15.22:0/3555434046 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb4fc0a33b0 0x7fb4fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.455+0000 7fb505ffb640 1 -- 172.21.15.22:0/3199726021 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f80856c0 con 0x55d7515ea370 2024-09-19T08:25:51.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.455+0000 7fb505ffb640 1 -- 172.21.15.22:0/3199726021 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f809c3c0 con 0x55d7515ea370 2024-09-19T08:25:51.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.455+0000 7fb50d7fa640 1 -- 172.21.15.22:0/3555434046 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fb4f80d0150 con 0x7fb510103660 2024-09-19T08:25:51.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.456+0000 7fb50ffff640 1 --2- 172.21.15.22:0/3555434046 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb4fc0a33b0 0x7fb4fc0a5870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb508004af0 tx=0x7fb508005d80 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:51.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.456+0000 7fb517f0d740 1 -- 172.21.15.22:0/3199726021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515ea370 msgr2=0x55d7515ec760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:51.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.456+0000 7fb517f0d740 1 --2- 172.21.15.22:0/3199726021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515ea370 0x55d7515ec760 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80cd770 tx=0x7fb4f80d8270 comp rx=0 tx=0).stop 2024-09-19T08:25:51.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.456+0000 7fb517f0d740 1 -- 172.21.15.22:0/3199726021 shutdown_connections 2024-09-19T08:25:51.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.456+0000 7fb517f0d740 1 --2- 172.21.15.22:0/3199726021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515ea370 0x55d7515ec760 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.456+0000 7fb517f0d740 1 --2- 172.21.15.22:0/3199726021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 0x55d7515dfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.456+0000 7fb517f0d740 1 --2- 172.21.15.22:0/3199726021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 0x55d7515df090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.456+0000 7fb517f0d740 1 -- 172.21.15.22:0/3199726021 >> 172.21.15.22:0/3199726021 conn(0x55d7515a8310 msgr2=0x55d7515dbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:51.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.456+0000 7fb517f0d740 1 -- 172.21.15.22:0/3199726021 shutdown_connections 2024-09-19T08:25:51.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.456+0000 7fb517f0d740 1 -- 172.21.15.22:0/3199726021 wait complete. 2024-09-19T08:25:51.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.457+0000 7fb517f0d740 1 -- start start 2024-09-19T08:25:51.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.457+0000 7fb517f0d740 1 Processor -- start 2024-09-19T08:25:51.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.457+0000 7fb517f0d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 0x55d7515e9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.457+0000 7fb517f0d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 0x55d7515e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.457+0000 7fb517f0d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515ea370 0x55d7515e75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.457+0000 7fb517f0d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d7515ed070 con 0x55d7515ea370 2024-09-19T08:25:51.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.457+0000 7fb517f0d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d751500060 con 0x55d7515df5d0 2024-09-19T08:25:51.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.457+0000 7fb517f0d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d7515ed1f0 con 0x55d7515e0b40 2024-09-19T08:25:51.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.457+0000 7fb517f0d740 10 client.0 _ll_register_callbacks cb 0x55d7512538c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:51.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.457+0000 7fb50ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515ea370 0x55d7515e75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.457+0000 7fb50ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515ea370 0x55d7515e75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47548/0 (socket says 172.21.15.22:47548) 2024-09-19T08:25:51.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.457+0000 7fb50ffff640 1 -- 172.21.15.22:0/2026905674 learned_addr learned my addr 172.21.15.22:0/2026905674 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:51.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.457+0000 7fb50f7fe640 1 --2- 172.21.15.22:0/2026905674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 0x55d7515e9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.457+0000 7fb506ffd640 1 --2- 172.21.15.22:0/2026905674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 0x55d7515e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb50ffff640 1 -- 172.21.15.22:0/2026905674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 msgr2=0x55d7515e6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:51.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb50ffff640 1 --2- 172.21.15.22:0/2026905674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 0x55d7515e6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb50ffff640 1 -- 172.21.15.22:0/2026905674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 msgr2=0x55d7515e9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:51.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb50ffff640 1 --2- 172.21.15.22:0/2026905674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 0x55d7515e9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb50ffff640 1 -- 172.21.15.22:0/2026905674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7512b63c0 con 0x55d7515ea370 2024-09-19T08:25:51.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb506ffd640 1 --2- 172.21.15.22:0/2026905674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 0x55d7515e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:25:51.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb50ffff640 1 --2- 172.21.15.22:0/2026905674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515ea370 0x55d7515e75e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb508009410 tx=0x7fb50802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ebcc21cda023e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:51.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb4eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:51.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb4eeffd640 1 -- 172.21.15.22:0/2026905674 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508034040 con 0x55d7515ea370 2024-09-19T08:25:51.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb4eeffd640 1 -- 172.21.15.22:0/2026905674 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fb508052470 con 0x55d7515ea370 2024-09-19T08:25:51.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb517f0d740 1 -- 172.21.15.22:0/2026905674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4f80856c0 con 0x55d7515ea370 2024-09-19T08:25:51.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb4eeffd640 1 -- 172.21.15.22:0/2026905674 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb508052670 con 0x55d7515ea370 2024-09-19T08:25:51.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb517f0d740 10 client.17238 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:51.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb517f0d740 1 -- 172.21.15.22:0/2026905674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d7515f8ae0 con 0x55d7515ea370 2024-09-19T08:25:51.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.458+0000 7fb4eeffd640 1 -- 172.21.15.22:0/2026905674 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f80856c0 con 0x55d7515ea370 2024-09-19T08:25:51.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.459+0000 7fb4eeffd640 1 -- 172.21.15.22:0/2026905674 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fb508049660 con 0x55d7515ea370 2024-09-19T08:25:51.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.459+0000 7fb517f0d740 10 client.17238 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:51.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.459+0000 7fb517f0d740 10 client.17238 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:25:51.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.459+0000 7fb517f0d740 20 client.17238 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:51.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.459+0000 7fb517f0d740 1 --2- 172.21.15.22:0/2026905674 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55d7515f9f40 0x55d7515fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.459+0000 7fb517f0d740 4 client.17238 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:25:51.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.459+0000 7fb517f0d740 1 -- 172.21.15.22:0/2026905674 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d751552860 con 0x55d7515f9f40 2024-09-19T08:25:51.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.459+0000 7fb50f7fe640 1 --2- 172.21.15.22:0/2026905674 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55d7515f9f40 0x55d7515fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.460+0000 7fb4eeffd640 10 client.17238 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:25:51.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.460+0000 7fb50f7fe640 1 --2- 172.21.15.22:0/2026905674 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55d7515f9f40 0x55d7515fc3a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35d4ae6ce4bced06 server_cookie=19ebe523b47204a6 in_seq=0 out_seq=0 2024-09-19T08:25:51.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.460+0000 7fb4eeffd640 1 -- 172.21.15.22:0/2026905674 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d751552860 con 0x55d7515f9f40 2024-09-19T08:25:51.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.460+0000 7fb4eeffd640 10 client.17238 handle_command_reply: tid=0 2024-09-19T08:25:51.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.460+0000 7fb517f0d740 1 client.17238 shutdown 2024-09-19T08:25:51.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.461+0000 7fb517f0d740 1 -- 172.21.15.22:0/2026905674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515ea370 msgr2=0x55d7515e75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:51.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.461+0000 7fb517f0d740 1 --2- 172.21.15.22:0/2026905674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515ea370 0x55d7515e75e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb508009410 tx=0x7fb50802a950 comp rx=0 tx=0).stop 2024-09-19T08:25:51.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.461+0000 7fb517f0d740 1 -- 172.21.15.22:0/2026905674 shutdown_connections 2024-09-19T08:25:51.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.461+0000 7fb517f0d740 1 --2- 172.21.15.22:0/2026905674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515ea370 0x55d7515e75e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.461+0000 7fb517f0d740 1 --2- 172.21.15.22:0/2026905674 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55d7515f9f40 0x55d7515fc3a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.461+0000 7fb517f0d740 1 --2- 172.21.15.22:0/2026905674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 0x55d7515e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.461+0000 7fb517f0d740 1 --2- 172.21.15.22:0/2026905674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 0x55d7515e9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.461+0000 7fb4eeffd640 0 client.17238 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:25:51.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.461+0000 7fb517f0d740 1 -- 172.21.15.22:0/2026905674 >> 172.21.15.22:0/2026905674 conn(0x55d7515a8310 msgr2=0x55d7515e1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:51.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.461+0000 7fb517f0d740 1 -- 172.21.15.22:0/2026905674 shutdown_connections 2024-09-19T08:25:51.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.461+0000 7fb517f0d740 1 -- 172.21.15.22:0/2026905674 wait complete. 2024-09-19T08:25:51.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.461+0000 7fb517f0d740 20 client.17238 trim_cache size 0 max 16384 2024-09-19T08:25:51.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.475+0000 7fb517f0d740 1 Processor -- start 2024-09-19T08:25:51.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.475+0000 7fb517f0d740 1 -- start start 2024-09-19T08:25:51.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.475+0000 7fb517f0d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 0x55d7515e8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.475+0000 7fb517f0d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515e0b40 0x55d7515e2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.475+0000 7fb517f0d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515ea370 0x55d7515e2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.475+0000 7fb517f0d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101b02a0 con 0x55d7515e0b40 2024-09-19T08:25:51.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.475+0000 7fb517f0d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101adb10 con 0x55d7515df5d0 2024-09-19T08:25:51.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.475+0000 7fb517f0d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101ab980 con 0x55d7515ea370 2024-09-19T08:25:51.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.475+0000 7fb506ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515e0b40 0x55d7515e2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.476+0000 7fb506ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515e0b40 0x55d7515e2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47556/0 (socket says 172.21.15.22:47556) 2024-09-19T08:25:51.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.476+0000 7fb506ffd640 1 -- 172.21.15.22:0/946014463 learned_addr learned my addr 172.21.15.22:0/946014463 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:51.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.476+0000 7fb50ffff640 1 --2- 172.21.15.22:0/946014463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515ea370 0x55d7515e2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.476+0000 7fb50f7fe640 1 --2- 172.21.15.22:0/946014463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 0x55d7515e8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.476+0000 7fb506ffd640 1 -- 172.21.15.22:0/946014463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515ea370 msgr2=0x55d7515e2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:51.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.476+0000 7fb506ffd640 1 --2- 172.21.15.22:0/946014463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515ea370 0x55d7515e2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.476+0000 7fb506ffd640 1 -- 172.21.15.22:0/946014463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 msgr2=0x55d7515e8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:51.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.476+0000 7fb506ffd640 1 --2- 172.21.15.22:0/946014463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 0x55d7515e8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.476+0000 7fb506ffd640 1 -- 172.21.15.22:0/946014463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7515ea0b0 con 0x55d7515e0b40 2024-09-19T08:25:51.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.476+0000 7fb506ffd640 1 --2- 172.21.15.22:0/946014463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515e0b40 0x55d7515e2120 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8098820 tx=0x7fb4f8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9956ba56b46ecafb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:51.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.476+0000 7fb5057fa640 1 -- 172.21.15.22:0/946014463 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8086af0 con 0x55d7515e0b40 2024-09-19T08:25:51.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.476+0000 7fb5057fa640 1 -- 172.21.15.22:0/946014463 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f809cac0 con 0x55d7515e0b40 2024-09-19T08:25:51.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.476+0000 7fb5057fa640 1 -- 172.21.15.22:0/946014463 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8086e40 con 0x55d7515e0b40 2024-09-19T08:25:51.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.477+0000 7fb517f0d740 1 -- 172.21.15.22:0/946014463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515e0b40 msgr2=0x55d7515e2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:51.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.477+0000 7fb517f0d740 1 --2- 172.21.15.22:0/946014463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515e0b40 0x55d7515e2120 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8098820 tx=0x7fb4f8009310 comp rx=0 tx=0).stop 2024-09-19T08:25:51.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.477+0000 7fb517f0d740 1 -- 172.21.15.22:0/946014463 shutdown_connections 2024-09-19T08:25:51.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.477+0000 7fb517f0d740 1 --2- 172.21.15.22:0/946014463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515ea370 0x55d7515e2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.477+0000 7fb517f0d740 1 --2- 172.21.15.22:0/946014463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515e0b40 0x55d7515e2120 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.477+0000 7fb517f0d740 1 --2- 172.21.15.22:0/946014463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515df5d0 0x55d7515e8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.477+0000 7fb517f0d740 1 -- 172.21.15.22:0/946014463 >> 172.21.15.22:0/946014463 conn(0x55d7515a8310 msgr2=0x55d7515e8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.477+0000 7fb517f0d740 1 -- 172.21.15.22:0/946014463 shutdown_connections 2024-09-19T08:25:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.477+0000 7fb517f0d740 1 -- 172.21.15.22:0/946014463 wait complete. 2024-09-19T08:25:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.477+0000 7fb517f0d740 1 -- start start 2024-09-19T08:25:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb517f0d740 1 Processor -- start 2024-09-19T08:25:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb517f0d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515df5d0 0x55d7515f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb517f0d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 0x55d7515e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb517f0d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515ea370 0x55d7515e4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb517f0d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101abfd0 con 0x55d7515df5d0 2024-09-19T08:25:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb517f0d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101ad440 con 0x55d7515ea370 2024-09-19T08:25:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb517f0d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101ac6a0 con 0x55d7515e0b40 2024-09-19T08:25:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb50f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515df5d0 0x55d7515f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb517f0d740 10 client.0 _ll_register_callbacks cb 0x7fb4f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb50f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515df5d0 0x55d7515f2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47570/0 (socket says 172.21.15.22:47570) 2024-09-19T08:25:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb50f7fe640 1 -- 172.21.15.22:0/2597513853 learned_addr learned my addr 172.21.15.22:0/2597513853 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb506ffd640 1 --2- 172.21.15.22:0/2597513853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 0x55d7515e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb50ffff640 1 --2- 172.21.15.22:0/2597513853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515ea370 0x55d7515e4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb50f7fe640 1 -- 172.21.15.22:0/2597513853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 msgr2=0x55d7515e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb50f7fe640 1 --2- 172.21.15.22:0/2597513853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 0x55d7515e3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb50f7fe640 1 -- 172.21.15.22:0/2597513853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515ea370 msgr2=0x55d7515e4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb50f7fe640 1 --2- 172.21.15.22:0/2597513853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515ea370 0x55d7515e4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb50f7fe640 1 -- 172.21.15.22:0/2597513853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7515e4830 con 0x55d7515df5d0 2024-09-19T08:25:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.478+0000 7fb50ffff640 1 --2- 172.21.15.22:0/2597513853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515ea370 0x55d7515e4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb50f7fe640 1 --2- 172.21.15.22:0/2597513853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515df5d0 0x55d7515f2aa0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb500004700 tx=0x7fb5000045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550f5a0ff1879679 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb4eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb4eeffd640 1 -- 172.21.15.22:0/2597513853 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500022030 con 0x55d7515df5d0 2024-09-19T08:25:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb4eeffd640 1 -- 172.21.15.22:0/2597513853 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fb500010430 con 0x55d7515df5d0 2024-09-19T08:25:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb517f0d740 1 -- 172.21.15.22:0/2597513853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4f809cac0 con 0x55d7515df5d0 2024-09-19T08:25:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb4eeffd640 1 -- 172.21.15.22:0/2597513853 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb500010630 con 0x55d7515df5d0 2024-09-19T08:25:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb517f0d740 10 client.17244 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb517f0d740 1 -- 172.21.15.22:0/2597513853 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d751664b60 con 0x55d7515df5d0 2024-09-19T08:25:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb4eeffd640 1 -- 172.21.15.22:0/2597513853 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f809cac0 con 0x55d7515df5d0 2024-09-19T08:25:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb4eeffd640 1 -- 172.21.15.22:0/2597513853 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fb500027970 con 0x55d7515df5d0 2024-09-19T08:25:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb517f0d740 10 client.17244 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb517f0d740 10 client.17244 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T08:25:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb517f0d740 20 client.17244 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb517f0d740 1 --2- 172.21.15.22:0/2597513853 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55d751665310 0x55d751667730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb517f0d740 4 client.17244 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:25:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.479+0000 7fb517f0d740 1 -- 172.21.15.22:0/2597513853 --> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d75154cd70 con 0x55d751665310 2024-09-19T08:25:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.480+0000 7fb506ffd640 1 --2- 172.21.15.22:0/2597513853 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55d751665310 0x55d751667730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.480+0000 7fb506ffd640 1 --2- 172.21.15.22:0/2597513853 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55d751665310 0x55d751667730 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a4759891b719adb server_cookie=c797600827e93788 in_seq=0 out_seq=0 2024-09-19T08:25:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.480+0000 7fb4eeffd640 10 client.17244 ms_handle_connect on v2:172.21.15.157:6836/892624399 2024-09-19T08:25:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.480+0000 7fb4eeffd640 1 -- 172.21.15.22:0/2597513853 <== mds.2 v2:172.21.15.157:6836/892624399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d75154cd70 con 0x55d751665310 2024-09-19T08:25:51.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.480+0000 7fb4eeffd640 10 client.17244 handle_command_reply: tid=0 2024-09-19T08:25:51.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.480+0000 7fb517f0d740 1 client.17244 shutdown 2024-09-19T08:25:51.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb517f0d740 1 -- 172.21.15.22:0/2597513853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515df5d0 msgr2=0x55d7515f2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:51.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb517f0d740 1 --2- 172.21.15.22:0/2597513853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515df5d0 0x55d7515f2aa0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb500004700 tx=0x7fb5000045a0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb517f0d740 1 -- 172.21.15.22:0/2597513853 shutdown_connections 2024-09-19T08:25:51.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb517f0d740 1 --2- 172.21.15.22:0/2597513853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7515ea370 0x55d7515e4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb517f0d740 1 --2- 172.21.15.22:0/2597513853 >> [v2:172.21.15.157:6836/892624399,v1:172.21.15.157:6837/892624399] conn(0x55d751665310 0x55d751667730 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb517f0d740 1 --2- 172.21.15.22:0/2597513853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d7515e0b40 0x55d7515e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb517f0d740 1 --2- 172.21.15.22:0/2597513853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d7515df5d0 0x55d7515f2aa0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb4eeffd640 0 client.17244 ms_handle_reset on v2:172.21.15.157:6836/892624399 2024-09-19T08:25:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb517f0d740 1 -- 172.21.15.22:0/2597513853 >> 172.21.15.22:0/2597513853 conn(0x55d7515a8310 msgr2=0x55d7515e9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb517f0d740 1 -- 172.21.15.22:0/2597513853 shutdown_connections 2024-09-19T08:25:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb517f0d740 1 -- 172.21.15.22:0/2597513853 wait complete. 2024-09-19T08:25:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb517f0d740 20 client.17244 trim_cache size 0 max 16384 2024-09-19T08:25:51.493 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:25:51.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb515b34640 1 -- 172.21.15.22:0/3555434046 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb4fc0a33b0 msgr2=0x7fb4fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:51.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb515b34640 1 --2- 172.21.15.22:0/3555434046 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb4fc0a33b0 0x7fb4fc0a5870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb508004af0 tx=0x7fb508005d80 comp rx=0 tx=0).stop 2024-09-19T08:25:51.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb515b34640 1 -- 172.21.15.22:0/3555434046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb510103660 msgr2=0x7fb5101989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:51.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.481+0000 7fb515b34640 1 --2- 172.21.15.22:0/3555434046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb510103660 0x7fb5101989a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800e9f0 tx=0x7fb4f800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.482+0000 7fb515b34640 1 -- 172.21.15.22:0/3555434046 shutdown_connections 2024-09-19T08:25:51.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.482+0000 7fb515b34640 1 --2- 172.21.15.22:0/3555434046 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb4fc0a33b0 0x7fb4fc0a5870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.482+0000 7fb515b34640 1 --2- 172.21.15.22:0/3555434046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb51010c170 0x7fb51019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.482+0000 7fb515b34640 1 --2- 172.21.15.22:0/3555434046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb510103660 0x7fb5101989a0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.482+0000 7fb515b34640 1 --2- 172.21.15.22:0/3555434046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb510102d40 0x7fb510198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:51.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.482+0000 7fb515b34640 1 -- 172.21.15.22:0/3555434046 >> 172.21.15.22:0/3555434046 conn(0x7fb5100fc7a0 msgr2=0x7fb5100fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:51.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.482+0000 7fb515b34640 1 -- 172.21.15.22:0/3555434046 shutdown_connections 2024-09-19T08:25:51.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:51.482+0000 7fb515b34640 1 -- 172.21.15.22:0/3555434046 wait complete. 2024-09-19T08:25:51.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:51 smithi157 ceph-mon[35143]: pgmap v335: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:25:52.336 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:25:52.595 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:25:53.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.144+0000 7f97a388e640 1 -- 172.21.15.22:0/3134850906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f979c10c170 msgr2=0x7f979c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:53.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.144+0000 7f97a388e640 1 --2- 172.21.15.22:0/3134850906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f979c10c170 0x7f979c10e610 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9794009ca0 tx=0x7f979402a950 comp rx=0 tx=0).stop 2024-09-19T08:25:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.144+0000 7f97a388e640 1 -- 172.21.15.22:0/3134850906 shutdown_connections 2024-09-19T08:25:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.144+0000 7f97a388e640 1 --2- 172.21.15.22:0/3134850906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f979c10c170 0x7f979c10e610 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.144+0000 7f97a388e640 1 --2- 172.21.15.22:0/3134850906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979c103660 0x7f979c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.144+0000 7f97a388e640 1 --2- 172.21.15.22:0/3134850906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f979c102d40 0x7f979c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.144+0000 7f97a388e640 1 -- 172.21.15.22:0/3134850906 >> 172.21.15.22:0/3134850906 conn(0x7f979c0fc7a0 msgr2=0x7f979c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.144+0000 7f97a388e640 1 -- 172.21.15.22:0/3134850906 shutdown_connections 2024-09-19T08:25:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.144+0000 7f97a388e640 1 -- 172.21.15.22:0/3134850906 wait complete. 2024-09-19T08:25:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.145+0000 7f97a388e640 1 Processor -- start 2024-09-19T08:25:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.145+0000 7f97a388e640 1 -- start start 2024-09-19T08:25:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.145+0000 7f97a388e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f979c102d40 0x7f979c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.145+0000 7f97a388e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f979c103660 0x7f979c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.145+0000 7f97a388e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979c10c170 0x7f979c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.145+0000 7f97a388e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f979c111f70 con 0x7f979c102d40 2024-09-19T08:25:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.145+0000 7f97a388e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f979c111df0 con 0x7f979c103660 2024-09-19T08:25:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.145+0000 7f97a388e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f979c1120f0 con 0x7f979c10c170 2024-09-19T08:25:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.145+0000 7f97a162a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f979c102d40 0x7f979c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.145+0000 7f97a162a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f979c102d40 0x7f979c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47592/0 (socket says 172.21.15.22:47592) 2024-09-19T08:25:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.145+0000 7f97a162a640 1 -- 172.21.15.22:0/1657506620 learned_addr learned my addr 172.21.15.22:0/1657506620 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.146+0000 7f97a0e29640 1 --2- 172.21.15.22:0/1657506620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f979c103660 0x7f979c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.146+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/1657506620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979c10c170 0x7f979c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.146+0000 7f97a162a640 1 -- 172.21.15.22:0/1657506620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979c10c170 msgr2=0x7f979c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.146+0000 7f97a162a640 1 --2- 172.21.15.22:0/1657506620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979c10c170 0x7f979c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.146+0000 7f97a162a640 1 -- 172.21.15.22:0/1657506620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f979c103660 msgr2=0x7f979c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.146+0000 7f97a162a640 1 --2- 172.21.15.22:0/1657506620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f979c103660 0x7f979c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.146+0000 7f97a162a640 1 -- 172.21.15.22:0/1657506620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979c19d2e0 con 0x7f979c102d40 2024-09-19T08:25:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.146+0000 7f97a162a640 1 --2- 172.21.15.22:0/1657506620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f979c102d40 0x7f979c198330 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f979800e9f0 tx=0x7f979800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.146+0000 7f97927fc640 1 -- 172.21.15.22:0/1657506620 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979800cd80 con 0x7f979c102d40 2024-09-19T08:25:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.146+0000 7f97927fc640 1 -- 172.21.15.22:0/1657506620 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9798010910 con 0x7f979c102d40 2024-09-19T08:25:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.146+0000 7f97a388e640 1 -- 172.21.15.22:0/1657506620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979c19d5d0 con 0x7f979c102d40 2024-09-19T08:25:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.146+0000 7f97927fc640 1 -- 172.21.15.22:0/1657506620 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798018990 con 0x7f979c102d40 2024-09-19T08:25:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.146+0000 7f97a388e640 1 -- 172.21.15.22:0/1657506620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f979c1a4eb0 con 0x7f979c102d40 2024-09-19T08:25:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.149+0000 7f97927fc640 1 -- 172.21.15.22:0/1657506620 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9798010ac0 con 0x7f979c102d40 2024-09-19T08:25:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.149+0000 7f97a5c67740 1 Processor -- start 2024-09-19T08:25:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.149+0000 7f97a5c67740 1 -- start start 2024-09-19T08:25:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.149+0000 7f97a5c67740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f4eb40 0x55d761f4d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.149+0000 7f97a5c67740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 0x55d761f4da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.149+0000 7f97a5c67740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f58370 0x55d761f5a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.149+0000 7f97a5c67740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d761eb32f0 con 0x55d761f4eb40 2024-09-19T08:25:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.149+0000 7f97a5c67740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d761e5acf0 con 0x55d761f58370 2024-09-19T08:25:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.149+0000 7f97a5c67740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d761eaf2b0 con 0x55d761f4d5d0 2024-09-19T08:25:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.149+0000 7f97a1e2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f4eb40 0x55d761f4d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.149+0000 7f97a1e2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f4eb40 0x55d761f4d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47608/0 (socket says 172.21.15.22:47608) 2024-09-19T08:25:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.149+0000 7f97a1e2b640 1 -- 172.21.15.22:0/1900819660 learned_addr learned my addr 172.21.15.22:0/1900819660 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97a162a640 1 --2- 172.21.15.22:0/1900819660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 0x55d761f4da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97a0e29640 1 --2- 172.21.15.22:0/1900819660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f58370 0x55d761f5a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97a1e2b640 1 -- 172.21.15.22:0/1900819660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 msgr2=0x55d761f4da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/1900819660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 0x55d761f4da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97a1e2b640 1 -- 172.21.15.22:0/1900819660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f58370 msgr2=0x55d761f5a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/1900819660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f58370 0x55d761f5a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97a1e2b640 1 -- 172.21.15.22:0/1900819660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d761e3f3a0 con 0x55d761f4eb40 2024-09-19T08:25:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97a0e29640 1 --2- 172.21.15.22:0/1900819660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f58370 0x55d761f5a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97927fc640 1 --2- 172.21.15.22:0/1657506620 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f97880a33b0 0x7f97880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/1900819660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f4eb40 0x55d761f4d090 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f979402ffa0 tx=0x7f9794009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c8f8fd998371d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:53.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97a162a640 1 --2- 172.21.15.22:0/1657506620 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f97880a33b0 0x7f97880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97927fc640 1 -- 172.21.15.22:0/1657506620 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f97980d0170 con 0x7f979c102d40 2024-09-19T08:25:53.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97877fe640 1 -- 172.21.15.22:0/1900819660 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794034040 con 0x55d761f4eb40 2024-09-19T08:25:53.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97877fe640 1 -- 172.21.15.22:0/1900819660 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9794003970 con 0x55d761f4eb40 2024-09-19T08:25:53.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.150+0000 7f97877fe640 1 -- 172.21.15.22:0/1900819660 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97940045a0 con 0x55d761f4eb40 2024-09-19T08:25:53.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.151+0000 7f97a5c67740 1 -- 172.21.15.22:0/1900819660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f4eb40 msgr2=0x55d761f4d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:53.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.151+0000 7f97a162a640 1 --2- 172.21.15.22:0/1657506620 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f97880a33b0 0x7f97880a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55d761f4aac0 tx=0x7f9798018420 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:53.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.151+0000 7f97a5c67740 1 --2- 172.21.15.22:0/1900819660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f4eb40 0x55d761f4d090 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f979402ffa0 tx=0x7f9794009a20 comp rx=0 tx=0).stop 2024-09-19T08:25:53.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.151+0000 7f97a5c67740 1 -- 172.21.15.22:0/1900819660 shutdown_connections 2024-09-19T08:25:53.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.151+0000 7f97a5c67740 1 --2- 172.21.15.22:0/1900819660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f58370 0x55d761f5a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.151+0000 7f97a5c67740 1 --2- 172.21.15.22:0/1900819660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 0x55d761f4da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.151+0000 7f97a5c67740 1 --2- 172.21.15.22:0/1900819660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f4eb40 0x55d761f4d090 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.151+0000 7f97a5c67740 1 -- 172.21.15.22:0/1900819660 >> 172.21.15.22:0/1900819660 conn(0x55d761f16310 msgr2=0x55d761f49c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:53.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.151+0000 7f97a5c67740 1 -- 172.21.15.22:0/1900819660 shutdown_connections 2024-09-19T08:25:53.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.151+0000 7f97a5c67740 1 -- 172.21.15.22:0/1900819660 wait complete. 2024-09-19T08:25:53.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.152+0000 7f97a5c67740 1 -- start start 2024-09-19T08:25:53.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.152+0000 7f97a5c67740 1 Processor -- start 2024-09-19T08:25:53.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.152+0000 7f97a5c67740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 0x55d761f57d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.152+0000 7f97a5c67740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4eb40 0x55d761f55000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.152+0000 7f97a5c67740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761f555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.152+0000 7f97a5c67740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d761f5b010 con 0x55d761f58370 2024-09-19T08:25:53.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.152+0000 7f97a5c67740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d761e6e060 con 0x55d761f4eb40 2024-09-19T08:25:53.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.152+0000 7f97a5c67740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d761f5b190 con 0x55d761f4d5d0 2024-09-19T08:25:53.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.152+0000 7f97a5c67740 10 client.0 _ll_register_callbacks cb 0x55d761bc18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:53.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.152+0000 7f97a1e2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761f555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.152+0000 7f97a1e2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761f555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47620/0 (socket says 172.21.15.22:47620) 2024-09-19T08:25:53.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.152+0000 7f97a1e2b640 1 -- 172.21.15.22:0/2877018766 learned_addr learned my addr 172.21.15.22:0/2877018766 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:53.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.152+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/2877018766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 0x55d761f57d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.152+0000 7f97a0e29640 1 --2- 172.21.15.22:0/2877018766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4eb40 0x55d761f55000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.153+0000 7f97a1e2b640 1 -- 172.21.15.22:0/2877018766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 msgr2=0x55d761f57d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:53.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.153+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/2877018766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 0x55d761f57d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.153+0000 7f97a1e2b640 1 -- 172.21.15.22:0/2877018766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4eb40 msgr2=0x55d761f55000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:53.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.153+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/2877018766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4eb40 0x55d761f55000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.153+0000 7f97a1e2b640 1 -- 172.21.15.22:0/2877018766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d761c243c0 con 0x55d761f58370 2024-09-19T08:25:53.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.153+0000 7f97a0e29640 1 --2- 172.21.15.22:0/2877018766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4eb40 0x55d761f55000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:53.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.153+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/2877018766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761f555d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9794009430 tx=0x7f97940040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf8724af84f3216 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:53.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.153+0000 7f9784ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:53.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.153+0000 7f9784ff9640 1 -- 172.21.15.22:0/2877018766 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794003d30 con 0x55d761f58370 2024-09-19T08:25:53.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.153+0000 7f97a5c67740 1 -- 172.21.15.22:0/2877018766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9794003970 con 0x55d761f58370 2024-09-19T08:25:53.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.153+0000 7f9784ff9640 1 -- 172.21.15.22:0/2877018766 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f979402f3a0 con 0x55d761f58370 2024-09-19T08:25:53.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.153+0000 7f9784ff9640 1 -- 172.21.15.22:0/2877018766 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f979402f5a0 con 0x55d761f58370 2024-09-19T08:25:53.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.153+0000 7f97a5c67740 10 client.17256 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:53.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.153+0000 7f97a5c67740 1 -- 172.21.15.22:0/2877018766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d761f5b010 con 0x55d761f58370 2024-09-19T08:25:53.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.154+0000 7f9784ff9640 1 -- 172.21.15.22:0/2877018766 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9794003970 con 0x55d761f58370 2024-09-19T08:25:53.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.154+0000 7f9784ff9640 1 -- 172.21.15.22:0/2877018766 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f979404f040 con 0x55d761f58370 2024-09-19T08:25:53.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.154+0000 7f97a5c67740 10 client.17256 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:53.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.154+0000 7f97a5c67740 10 client.17256 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:25:53.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.154+0000 7f97a5c67740 20 client.17256 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:53.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.154+0000 7f97a5c67740 1 --2- 172.21.15.22:0/2877018766 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d761f6c020 0x55d761f6e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.154+0000 7f97a5c67740 4 client.17256 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:25:53.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.154+0000 7f97a5c67740 1 -- 172.21.15.22:0/2877018766 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d761ec0860 con 0x55d761f6c020 2024-09-19T08:25:53.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.154+0000 7f97a0e29640 1 --2- 172.21.15.22:0/2877018766 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d761f6c020 0x55d761f6e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.154+0000 7f9784ff9640 10 client.17256 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:25:53.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.154+0000 7f97a0e29640 1 --2- 172.21.15.22:0/2877018766 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d761f6c020 0x55d761f6e410 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd226f126acdf5c1 server_cookie=1650cbfa4bbc7e74 in_seq=0 out_seq=0 2024-09-19T08:25:53.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.155+0000 7f9784ff9640 1 -- 172.21.15.22:0/2877018766 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d761ec0860 con 0x55d761f6c020 2024-09-19T08:25:53.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.155+0000 7f9784ff9640 10 client.17256 handle_command_reply: tid=0 2024-09-19T08:25:53.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.155+0000 7f97a5c67740 1 client.17256 shutdown 2024-09-19T08:25:53.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.155+0000 7f97a5c67740 1 -- 172.21.15.22:0/2877018766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 msgr2=0x55d761f555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:53.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.155+0000 7f97a5c67740 1 --2- 172.21.15.22:0/2877018766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761f555d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9794009430 tx=0x7f97940040e0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.155+0000 7f97a5c67740 1 -- 172.21.15.22:0/2877018766 shutdown_connections 2024-09-19T08:25:53.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.155+0000 7f97a5c67740 1 --2- 172.21.15.22:0/2877018766 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d761f6c020 0x55d761f6e410 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.155+0000 7f97a5c67740 1 --2- 172.21.15.22:0/2877018766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761f555d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.155+0000 7f97a5c67740 1 --2- 172.21.15.22:0/2877018766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4eb40 0x55d761f55000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.155+0000 7f9784ff9640 0 client.17256 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:25:53.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.155+0000 7f97a5c67740 1 --2- 172.21.15.22:0/2877018766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 0x55d761f57d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.155+0000 7f97a5c67740 1 -- 172.21.15.22:0/2877018766 >> 172.21.15.22:0/2877018766 conn(0x55d761f16310 msgr2=0x55d761f4f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:53.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.156+0000 7f97a5c67740 1 -- 172.21.15.22:0/2877018766 shutdown_connections 2024-09-19T08:25:53.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.156+0000 7f97a5c67740 1 -- 172.21.15.22:0/2877018766 wait complete. 2024-09-19T08:25:53.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.156+0000 7f97a5c67740 20 client.17256 trim_cache size 0 max 16384 2024-09-19T08:25:53.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a5c67740 1 Processor -- start 2024-09-19T08:25:53.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a5c67740 1 -- start start 2024-09-19T08:25:53.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a5c67740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4d5d0 0x55d761f68740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a5c67740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4eb40 0x55d761f50120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a5c67740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761f50660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a5c67740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f979c1ae040 con 0x55d761f58370 2024-09-19T08:25:53.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a5c67740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f979c1ab8b0 con 0x55d761f4d5d0 2024-09-19T08:25:53.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a5c67740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f979c1a9740 con 0x55d761f4eb40 2024-09-19T08:25:53.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a1e2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761f50660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a1e2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761f50660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47624/0 (socket says 172.21.15.22:47624) 2024-09-19T08:25:53.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a1e2b640 1 -- 172.21.15.22:0/1184511783 learned_addr learned my addr 172.21.15.22:0/1184511783 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:53.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/1184511783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4d5d0 0x55d761f68740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a1e2b640 1 -- 172.21.15.22:0/1184511783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4eb40 msgr2=0x55d761f50120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:53.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/1184511783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4eb40 0x55d761f50120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a1e2b640 1 -- 172.21.15.22:0/1184511783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4d5d0 msgr2=0x55d761f68740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:53.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/1184511783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4d5d0 0x55d761f68740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.169+0000 7f97a1e2b640 1 -- 172.21.15.22:0/1184511783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d761f50ba0 con 0x55d761f58370 2024-09-19T08:25:53.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.170+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/1184511783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761f50660 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9794005d50 tx=0x7f9794009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df896c0a70061051 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:53.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.170+0000 7f9786ffd640 1 -- 172.21.15.22:0/1184511783 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979403e040 con 0x55d761f58370 2024-09-19T08:25:53.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.170+0000 7f9786ffd640 1 -- 172.21.15.22:0/1184511783 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9794003970 con 0x55d761f58370 2024-09-19T08:25:53.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.170+0000 7f9786ffd640 1 -- 172.21.15.22:0/1184511783 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794030d00 con 0x55d761f58370 2024-09-19T08:25:53.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.170+0000 7f97a5c67740 1 -- 172.21.15.22:0/1184511783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 msgr2=0x55d761f50660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:53.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.170+0000 7f97a5c67740 1 --2- 172.21.15.22:0/1184511783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761f50660 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9794005d50 tx=0x7f9794009a20 comp rx=0 tx=0).stop 2024-09-19T08:25:53.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.170+0000 7f97a5c67740 1 -- 172.21.15.22:0/1184511783 shutdown_connections 2024-09-19T08:25:53.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.170+0000 7f97a5c67740 1 --2- 172.21.15.22:0/1184511783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761f50660 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.170+0000 7f97a5c67740 1 --2- 172.21.15.22:0/1184511783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4eb40 0x55d761f50120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.170+0000 7f97a5c67740 1 --2- 172.21.15.22:0/1184511783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4d5d0 0x55d761f68740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.170+0000 7f97a5c67740 1 -- 172.21.15.22:0/1184511783 >> 172.21.15.22:0/1184511783 conn(0x55d761f16310 msgr2=0x55d761f540f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:53.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.171+0000 7f97a5c67740 1 -- 172.21.15.22:0/1184511783 shutdown_connections 2024-09-19T08:25:53.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.171+0000 7f97a5c67740 1 -- 172.21.15.22:0/1184511783 wait complete. 2024-09-19T08:25:53.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.171+0000 7f97a5c67740 1 -- start start 2024-09-19T08:25:53.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.171+0000 7f97a5c67740 1 Processor -- start 2024-09-19T08:25:53.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.171+0000 7f97a5c67740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 0x55d761f60c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.171+0000 7f97a5c67740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4eb40 0x55d761f611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.171+0000 7f97a5c67740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761fd5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.171+0000 7f97a5c67740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f979c1a9d90 con 0x55d761f58370 2024-09-19T08:25:53.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.171+0000 7f97a5c67740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f979c1ab1e0 con 0x55d761f4eb40 2024-09-19T08:25:53.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.171+0000 7f97a5c67740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f979c1aa440 con 0x55d761f4d5d0 2024-09-19T08:25:53.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.171+0000 7f97a5c67740 10 client.0 _ll_register_callbacks cb 0x7f9794009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:53.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f97a1e2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761fd5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f97a1e2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761fd5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47632/0 (socket says 172.21.15.22:47632) 2024-09-19T08:25:53.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f97a1e2b640 1 -- 172.21.15.22:0/1830368067 learned_addr learned my addr 172.21.15.22:0/1830368067 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:53.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/1830368067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 0x55d761f60c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f97a0e29640 1 --2- 172.21.15.22:0/1830368067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4eb40 0x55d761f611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f97a1e2b640 1 -- 172.21.15.22:0/1830368067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 msgr2=0x55d761f60c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:53.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/1830368067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 0x55d761f60c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f97a1e2b640 1 -- 172.21.15.22:0/1830368067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4eb40 msgr2=0x55d761f611c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:53.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/1830368067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4eb40 0x55d761f611c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f97a1e2b640 1 -- 172.21.15.22:0/1830368067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d761fd6040 con 0x55d761f58370 2024-09-19T08:25:53.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f97a1e2b640 1 --2- 172.21.15.22:0/1830368067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761fd5b00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f979402b100 tx=0x7f9794000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8506ea8cfe2f00b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:53.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f9784ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:53.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f9784ff9640 1 -- 172.21.15.22:0/1830368067 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794030630 con 0x55d761f58370 2024-09-19T08:25:53.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f9784ff9640 1 -- 172.21.15.22:0/1830368067 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f979402fb50 con 0x55d761f58370 2024-09-19T08:25:53.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f97a5c67740 1 -- 172.21.15.22:0/1830368067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9794003970 con 0x55d761f58370 2024-09-19T08:25:53.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f9784ff9640 1 -- 172.21.15.22:0/1830368067 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f979402f740 con 0x55d761f58370 2024-09-19T08:25:53.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f97a5c67740 10 client.17262 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:53.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.172+0000 7f97a5c67740 1 -- 172.21.15.22:0/1830368067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d761fd7280 con 0x55d761f58370 2024-09-19T08:25:53.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.173+0000 7f9784ff9640 1 -- 172.21.15.22:0/1830368067 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9794003970 con 0x55d761f58370 2024-09-19T08:25:53.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.173+0000 7f9784ff9640 1 -- 172.21.15.22:0/1830368067 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f9794042070 con 0x55d761f58370 2024-09-19T08:25:53.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.173+0000 7f97a5c67740 10 client.17262 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:53.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.173+0000 7f97a5c67740 10 client.17262 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-19T08:25:53.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.173+0000 7f97a5c67740 20 client.17262 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:53.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.173+0000 7f97a5c67740 1 --2- 172.21.15.22:0/1830368067 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d761fd7830 0x55d761fd9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:53.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.173+0000 7f97a5c67740 4 client.17262 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:25:53.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.173+0000 7f97a5c67740 1 -- 172.21.15.22:0/1830368067 --> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d761ebad70 con 0x55d761fd7830 2024-09-19T08:25:53.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.173+0000 7f97a0e29640 1 --2- 172.21.15.22:0/1830368067 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d761fd7830 0x55d761fd9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:53.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.173+0000 7f9784ff9640 10 client.17262 ms_handle_connect on v2:172.21.15.64:6838/358484749 2024-09-19T08:25:53.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.173+0000 7f97a0e29640 1 --2- 172.21.15.22:0/1830368067 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d761fd7830 0x55d761fd9c20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99a0a4761c9a96e2 server_cookie=34d32a63d3bc2664 in_seq=0 out_seq=0 2024-09-19T08:25:53.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f9784ff9640 1 -- 172.21.15.22:0/1830368067 <== mds.1 v2:172.21.15.64:6838/358484749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d761ebad70 con 0x55d761fd7830 2024-09-19T08:25:53.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f9784ff9640 10 client.17262 handle_command_reply: tid=0 2024-09-19T08:25:53.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f97a5c67740 1 client.17262 shutdown 2024-09-19T08:25:53.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f97a5c67740 1 -- 172.21.15.22:0/1830368067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 msgr2=0x55d761fd5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:53.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f97a5c67740 1 --2- 172.21.15.22:0/1830368067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761fd5b00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f979402b100 tx=0x7f9794000f40 comp rx=0 tx=0).stop 2024-09-19T08:25:53.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f97a5c67740 1 -- 172.21.15.22:0/1830368067 shutdown_connections 2024-09-19T08:25:53.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f97a5c67740 1 --2- 172.21.15.22:0/1830368067 >> [v2:172.21.15.64:6838/358484749,v1:172.21.15.64:6839/358484749] conn(0x55d761fd7830 0x55d761fd9c20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f97a5c67740 1 --2- 172.21.15.22:0/1830368067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d761f58370 0x55d761fd5b00 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f97a5c67740 1 --2- 172.21.15.22:0/1830368067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d761f4eb40 0x55d761f611c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f97a5c67740 1 --2- 172.21.15.22:0/1830368067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d761f4d5d0 0x55d761f60c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f97a5c67740 1 -- 172.21.15.22:0/1830368067 >> 172.21.15.22:0/1830368067 conn(0x55d761f16310 msgr2=0x55d761f56860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:53.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f9784ff9640 0 client.17262 ms_handle_reset on v2:172.21.15.64:6838/358484749 2024-09-19T08:25:53.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f97a5c67740 1 -- 172.21.15.22:0/1830368067 shutdown_connections 2024-09-19T08:25:53.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f97a5c67740 1 -- 172.21.15.22:0/1830368067 wait complete. 2024-09-19T08:25:53.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.174+0000 7f97a5c67740 20 client.17262 trim_cache size 0 max 16384 2024-09-19T08:25:53.182 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:25:53.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.175+0000 7f97a388e640 1 -- 172.21.15.22:0/1657506620 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f97880a33b0 msgr2=0x7f97880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:53.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.175+0000 7f97a388e640 1 --2- 172.21.15.22:0/1657506620 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f97880a33b0 0x7f97880a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55d761f4aac0 tx=0x7f9798018420 comp rx=0 tx=0).stop 2024-09-19T08:25:53.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.175+0000 7f97a388e640 1 -- 172.21.15.22:0/1657506620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f979c102d40 msgr2=0x7f979c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:53.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.175+0000 7f97a388e640 1 --2- 172.21.15.22:0/1657506620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f979c102d40 0x7f979c198330 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f979800e9f0 tx=0x7f979800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.175+0000 7f97a388e640 1 -- 172.21.15.22:0/1657506620 shutdown_connections 2024-09-19T08:25:53.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.175+0000 7f97a388e640 1 --2- 172.21.15.22:0/1657506620 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f97880a33b0 0x7f97880a5870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.175+0000 7f97a388e640 1 --2- 172.21.15.22:0/1657506620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979c10c170 0x7f979c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.175+0000 7f97a388e640 1 --2- 172.21.15.22:0/1657506620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f979c103660 0x7f979c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.175+0000 7f97a388e640 1 --2- 172.21.15.22:0/1657506620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f979c102d40 0x7f979c198330 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:53.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.175+0000 7f97a388e640 1 -- 172.21.15.22:0/1657506620 >> 172.21.15.22:0/1657506620 conn(0x7f979c0fc7a0 msgr2=0x7f979c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:53.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.175+0000 7f97a388e640 1 -- 172.21.15.22:0/1657506620 shutdown_connections 2024-09-19T08:25:53.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:53.175+0000 7f97a388e640 1 -- 172.21.15.22:0/1657506620 wait complete. 2024-09-19T08:25:53.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:53 smithi064 ceph-mon[34900]: pgmap v336: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:25:53.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:53 smithi157 ceph-mon[35143]: pgmap v336: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:25:53.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:53 smithi022 ceph-mon[30963]: pgmap v336: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T08:25:53.938 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:25:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:25:54.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.831+0000 7fd16b4fb640 1 -- 172.21.15.22:0/1490824427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1641049a0 msgr2=0x7fd1641089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:54.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.831+0000 7fd16b4fb640 1 --2- 172.21.15.22:0/1490824427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1641049a0 0x7fd1641089f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd154009900 tx=0x7fd15402a410 comp rx=0 tx=0).stop 2024-09-19T08:25:54.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.831+0000 7fd16b4fb640 1 -- 172.21.15.22:0/1490824427 shutdown_connections 2024-09-19T08:25:54.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.831+0000 7fd16b4fb640 1 --2- 172.21.15.22:0/1490824427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd164109120 0x7fd164110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.831+0000 7fd16b4fb640 1 --2- 172.21.15.22:0/1490824427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1641049a0 0x7fd1641089f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.831+0000 7fd16b4fb640 1 --2- 172.21.15.22:0/1490824427 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd164103ff0 0x7fd1641043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.831+0000 7fd16b4fb640 1 -- 172.21.15.22:0/1490824427 >> 172.21.15.22:0/1490824427 conn(0x7fd1640ffab0 msgr2=0x7fd164101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:54.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.831+0000 7fd16b4fb640 1 -- 172.21.15.22:0/1490824427 shutdown_connections 2024-09-19T08:25:54.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.831+0000 7fd16b4fb640 1 -- 172.21.15.22:0/1490824427 wait complete. 2024-09-19T08:25:54.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.832+0000 7fd16b4fb640 1 Processor -- start 2024-09-19T08:25:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.832+0000 7fd16b4fb640 1 -- start start 2024-09-19T08:25:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.832+0000 7fd16b4fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd164103ff0 0x7fd16419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd16b4fb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1641049a0 0x7fd16419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd16b4fb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd164109120 0x7fd16419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd16b4fb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd164114140 con 0x7fd1641049a0 2024-09-19T08:25:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd16b4fb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd164113fc0 con 0x7fd164109120 2024-09-19T08:25:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd16b4fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641142c0 con 0x7fd164103ff0 2024-09-19T08:25:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd168a96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1641049a0 0x7fd16419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd168a96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1641049a0 0x7fd16419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47660/0 (socket says 172.21.15.22:47660) 2024-09-19T08:25:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd168a96640 1 -- 172.21.15.22:0/4125501380 learned_addr learned my addr 172.21.15.22:0/4125501380 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd169297640 1 --2- 172.21.15.22:0/4125501380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd164103ff0 0x7fd16419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd169a98640 1 --2- 172.21.15.22:0/4125501380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd164109120 0x7fd16419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd168a96640 1 -- 172.21.15.22:0/4125501380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd164103ff0 msgr2=0x7fd16419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd168a96640 1 --2- 172.21.15.22:0/4125501380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd164103ff0 0x7fd16419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd168a96640 1 -- 172.21.15.22:0/4125501380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd164109120 msgr2=0x7fd16419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd168a96640 1 --2- 172.21.15.22:0/4125501380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd164109120 0x7fd16419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd168a96640 1 -- 172.21.15.22:0/4125501380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16419f5f0 con 0x7fd1641049a0 2024-09-19T08:25:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd168a96640 1 --2- 172.21.15.22:0/4125501380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1641049a0 0x7fd16419ab80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd154006010 tx=0x7fd1540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd15a7fc640 1 -- 172.21.15.22:0/4125501380 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15403f030 con 0x7fd1641049a0 2024-09-19T08:25:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd15a7fc640 1 -- 172.21.15.22:0/4125501380 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd154003ce0 con 0x7fd1641049a0 2024-09-19T08:25:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd16b4fb640 1 -- 172.21.15.22:0/4125501380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16419f880 con 0x7fd1641049a0 2024-09-19T08:25:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.833+0000 7fd15a7fc640 1 -- 172.21.15.22:0/4125501380 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154035070 con 0x7fd1641049a0 2024-09-19T08:25:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.834+0000 7fd16b4fb640 1 -- 172.21.15.22:0/4125501380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1641a71c0 con 0x7fd1641049a0 2024-09-19T08:25:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.836+0000 7fd15a7fc640 1 -- 172.21.15.22:0/4125501380 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd154042030 con 0x7fd1641049a0 2024-09-19T08:25:54.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.836+0000 7fd16d8d4740 1 Processor -- start 2024-09-19T08:25:54.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.836+0000 7fd16d8d4740 1 -- start start 2024-09-19T08:25:54.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd16d8d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25d4b40 0x564bb25d3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd16d8d4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d35d0 0x564bb25d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd16d8d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25de370 0x564bb25e0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd16d8d4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564bb25392f0 con 0x564bb25d35d0 2024-09-19T08:25:54.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd16d8d4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564bb24e0cf0 con 0x564bb25d4b40 2024-09-19T08:25:54.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd16d8d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564bb25352b0 con 0x564bb25de370 2024-09-19T08:25:54.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd169297640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d35d0 0x564bb25d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd169297640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d35d0 0x564bb25d3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47672/0 (socket says 172.21.15.22:47672) 2024-09-19T08:25:54.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd169297640 1 -- 172.21.15.22:0/2635779104 learned_addr learned my addr 172.21.15.22:0/2635779104 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:54.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd15a7fc640 1 --2- 172.21.15.22:0/4125501380 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd1500a3430 0x7fd1500a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd168a96640 1 --2- 172.21.15.22:0/2635779104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25de370 0x564bb25e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd169a98640 1 --2- 172.21.15.22:0/2635779104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25d4b40 0x564bb25d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd169a98640 1 --2- 172.21.15.22:0/4125501380 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd1500a3430 0x7fd1500a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd169297640 1 -- 172.21.15.22:0/2635779104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25de370 msgr2=0x564bb25e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:54.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd169297640 1 --2- 172.21.15.22:0/2635779104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25de370 0x564bb25e0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd169297640 1 -- 172.21.15.22:0/2635779104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25d4b40 msgr2=0x564bb25d3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:54.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd169297640 1 --2- 172.21.15.22:0/2635779104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25d4b40 0x564bb25d3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd169297640 1 -- 172.21.15.22:0/2635779104 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bb24c53a0 con 0x564bb25d35d0 2024-09-19T08:25:54.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd169a98640 1 --2- 172.21.15.22:0/2635779104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25d4b40 0x564bb25d3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:54.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.837+0000 7fd15a7fc640 1 -- 172.21.15.22:0/4125501380 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fd1540f0210 con 0x7fd1641049a0 2024-09-19T08:25:54.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.838+0000 7fd169297640 1 --2- 172.21.15.22:0/2635779104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d35d0 0x564bb25d3a10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd16000b760 tx=0x7fd16000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67030804542e109 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:54.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.838+0000 7fd14f7fe640 1 -- 172.21.15.22:0/2635779104 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160003dd0 con 0x564bb25d35d0 2024-09-19T08:25:54.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.838+0000 7fd14f7fe640 1 -- 172.21.15.22:0/2635779104 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd160015070 con 0x564bb25d35d0 2024-09-19T08:25:54.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.838+0000 7fd14f7fe640 1 -- 172.21.15.22:0/2635779104 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160011440 con 0x564bb25d35d0 2024-09-19T08:25:54.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.839+0000 7fd16d8d4740 1 -- 172.21.15.22:0/2635779104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d35d0 msgr2=0x564bb25d3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:54.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.839+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/2635779104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d35d0 0x564bb25d3a10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd16000b760 tx=0x7fd16000bc20 comp rx=0 tx=0).stop 2024-09-19T08:25:54.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.839+0000 7fd169a98640 1 --2- 172.21.15.22:0/4125501380 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd1500a3430 0x7fd1500a58f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009990 tx=0x7fd15c0096a0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:54.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.839+0000 7fd16d8d4740 1 -- 172.21.15.22:0/2635779104 shutdown_connections 2024-09-19T08:25:54.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.839+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/2635779104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25de370 0x564bb25e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.839+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/2635779104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d35d0 0x564bb25d3a10 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.839+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/2635779104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25d4b40 0x564bb25d3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.839+0000 7fd16d8d4740 1 -- 172.21.15.22:0/2635779104 >> 172.21.15.22:0/2635779104 conn(0x564bb259c310 msgr2=0x564bb25cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:54.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.839+0000 7fd16d8d4740 1 -- 172.21.15.22:0/2635779104 shutdown_connections 2024-09-19T08:25:54.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.839+0000 7fd16d8d4740 1 -- 172.21.15.22:0/2635779104 wait complete. 2024-09-19T08:25:54.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.839+0000 7fd16d8d4740 1 -- start start 2024-09-19T08:25:54.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.839+0000 7fd16d8d4740 1 Processor -- start 2024-09-19T08:25:54.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd16d8d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25d35d0 0x564bb25ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd16d8d4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d4b40 0x564bb25dafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd16d8d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25de370 0x564bb25db5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd16d8d4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564bb25e1070 con 0x564bb25d4b40 2024-09-19T08:25:54.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd16d8d4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564bb24f4060 con 0x564bb25d35d0 2024-09-19T08:25:54.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd16d8d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564bb25e11f0 con 0x564bb25de370 2024-09-19T08:25:54.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd16d8d4740 10 client.0 _ll_register_callbacks cb 0x564bb22478c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:54.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd168a96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d4b40 0x564bb25dafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd168a96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d4b40 0x564bb25dafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47684/0 (socket says 172.21.15.22:47684) 2024-09-19T08:25:54.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd168a96640 1 -- 172.21.15.22:0/1672409070 learned_addr learned my addr 172.21.15.22:0/1672409070 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:54.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd169297640 1 --2- 172.21.15.22:0/1672409070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25d35d0 0x564bb25ddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd169a98640 1 --2- 172.21.15.22:0/1672409070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25de370 0x564bb25db5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd168a96640 1 -- 172.21.15.22:0/1672409070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25de370 msgr2=0x564bb25db5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:54.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd168a96640 1 --2- 172.21.15.22:0/1672409070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25de370 0x564bb25db5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd168a96640 1 -- 172.21.15.22:0/1672409070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25d35d0 msgr2=0x564bb25ddd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:54.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd168a96640 1 --2- 172.21.15.22:0/1672409070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25d35d0 0x564bb25ddd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd168a96640 1 -- 172.21.15.22:0/1672409070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bb22aa3c0 con 0x564bb25d4b40 2024-09-19T08:25:54.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd169a98640 1 --2- 172.21.15.22:0/1672409070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25de370 0x564bb25db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:25:54.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd168a96640 1 --2- 172.21.15.22:0/1672409070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d4b40 0x564bb25dafe0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd1540b8ab0 tx=0x7fd154009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f158e4ce40de0fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:54.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd14cff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:54.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd14cff9640 1 -- 172.21.15.22:0/1672409070 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154049040 con 0x564bb25d4b40 2024-09-19T08:25:54.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd14cff9640 1 -- 172.21.15.22:0/1672409070 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fd1540ba2f0 con 0x564bb25d4b40 2024-09-19T08:25:54.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.840+0000 7fd16d8d4740 1 -- 172.21.15.22:0/1672409070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd160015070 con 0x564bb25d4b40 2024-09-19T08:25:54.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.841+0000 7fd14cff9640 1 -- 172.21.15.22:0/1672409070 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd15403bcb0 con 0x564bb25d4b40 2024-09-19T08:25:54.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.841+0000 7fd16d8d4740 10 client.17274 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:54.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.841+0000 7fd16d8d4740 1 -- 172.21.15.22:0/1672409070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564bb25ecad0 con 0x564bb25d4b40 2024-09-19T08:25:54.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.841+0000 7fd14cff9640 1 -- 172.21.15.22:0/1672409070 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd160015070 con 0x564bb25d4b40 2024-09-19T08:25:54.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.841+0000 7fd14cff9640 1 -- 172.21.15.22:0/1672409070 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fd1540bc8b0 con 0x564bb25d4b40 2024-09-19T08:25:54.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.841+0000 7fd16d8d4740 10 client.17274 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:54.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.841+0000 7fd16d8d4740 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:25:54.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.841+0000 7fd16d8d4740 20 client.17274 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:54.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.841+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/1672409070 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564bb25edf60 0x564bb25f03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.841+0000 7fd16d8d4740 4 client.17274 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:25:54.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.841+0000 7fd16d8d4740 1 -- 172.21.15.22:0/1672409070 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bb2546860 con 0x564bb25edf60 2024-09-19T08:25:54.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.841+0000 7fd169297640 1 --2- 172.21.15.22:0/1672409070 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564bb25edf60 0x564bb25f03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.842+0000 7fd14cff9640 10 client.17274 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:54.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.842+0000 7fd169297640 1 --2- 172.21.15.22:0/1672409070 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564bb25edf60 0x564bb25f03a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b445c353cfea45 server_cookie=d61f419ce39fba82 in_seq=0 out_seq=0 2024-09-19T08:25:54.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.842+0000 7fd14cff9640 1 -- 172.21.15.22:0/1672409070 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bb2546860 con 0x564bb25edf60 2024-09-19T08:25:54.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.842+0000 7fd14cff9640 10 client.17274 handle_command_reply: tid=0 2024-09-19T08:25:54.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.842+0000 7fd16d8d4740 1 client.17274 shutdown 2024-09-19T08:25:54.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.843+0000 7fd16d8d4740 1 -- 172.21.15.22:0/1672409070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d4b40 msgr2=0x564bb25dafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:54.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.843+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/1672409070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d4b40 0x564bb25dafe0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd1540b8ab0 tx=0x7fd154009230 comp rx=0 tx=0).stop 2024-09-19T08:25:54.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.843+0000 7fd16d8d4740 1 -- 172.21.15.22:0/1672409070 shutdown_connections 2024-09-19T08:25:54.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.843+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/1672409070 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564bb25edf60 0x564bb25f03a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.843+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/1672409070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25de370 0x564bb25db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.843+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/1672409070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d4b40 0x564bb25dafe0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.843+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/1672409070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25d35d0 0x564bb25ddd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.843+0000 7fd14cff9640 0 client.17274 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:54.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.843+0000 7fd16d8d4740 1 -- 172.21.15.22:0/1672409070 >> 172.21.15.22:0/1672409070 conn(0x564bb259c310 msgr2=0x564bb25d5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:54.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.843+0000 7fd16d8d4740 1 -- 172.21.15.22:0/1672409070 shutdown_connections 2024-09-19T08:25:54.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.843+0000 7fd16d8d4740 1 -- 172.21.15.22:0/1672409070 wait complete. 2024-09-19T08:25:54.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.843+0000 7fd16d8d4740 20 client.17274 trim_cache size 0 max 16384 2024-09-19T08:25:54.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.856+0000 7fd16d8d4740 1 Processor -- start 2024-09-19T08:25:54.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.856+0000 7fd16d8d4740 1 -- start start 2024-09-19T08:25:54.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd16d8d4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d35d0 0x564bb25cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd16d8d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25d4b40 0x564bb25d4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd16d8d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25de370 0x564bb25ea310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd16d8d4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641b2390 con 0x564bb25d35d0 2024-09-19T08:25:54.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd16d8d4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641afc00 con 0x564bb25de370 2024-09-19T08:25:54.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd16d8d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641ada70 con 0x564bb25d4b40 2024-09-19T08:25:54.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd169297640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d35d0 0x564bb25cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd169297640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d35d0 0x564bb25cfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47692/0 (socket says 172.21.15.22:47692) 2024-09-19T08:25:54.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd169297640 1 -- 172.21.15.22:0/3582078389 learned_addr learned my addr 172.21.15.22:0/3582078389 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:54.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd168a96640 1 --2- 172.21.15.22:0/3582078389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25d4b40 0x564bb25d4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd169a98640 1 --2- 172.21.15.22:0/3582078389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25de370 0x564bb25ea310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd169297640 1 -- 172.21.15.22:0/3582078389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25d4b40 msgr2=0x564bb25d4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:54.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd169297640 1 --2- 172.21.15.22:0/3582078389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25d4b40 0x564bb25d4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd169297640 1 -- 172.21.15.22:0/3582078389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25de370 msgr2=0x564bb25ea310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:54.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd169297640 1 --2- 172.21.15.22:0/3582078389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25de370 0x564bb25ea310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd169297640 1 -- 172.21.15.22:0/3582078389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bb25d9530 con 0x564bb25d35d0 2024-09-19T08:25:54.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd168a96640 1 --2- 172.21.15.22:0/3582078389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25d4b40 0x564bb25d4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:54.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd169a98640 1 --2- 172.21.15.22:0/3582078389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25de370 0x564bb25ea310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:25:54.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd169297640 1 --2- 172.21.15.22:0/3582078389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d35d0 0x564bb25cfe10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd16000b760 tx=0x7fd16000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7627e6a612915f6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:54.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.857+0000 7fd14effd640 1 -- 172.21.15.22:0/3582078389 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160005e50 con 0x564bb25d35d0 2024-09-19T08:25:54.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.858+0000 7fd14effd640 1 -- 172.21.15.22:0/3582078389 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd160015040 con 0x564bb25d35d0 2024-09-19T08:25:54.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.858+0000 7fd14effd640 1 -- 172.21.15.22:0/3582078389 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160026430 con 0x564bb25d35d0 2024-09-19T08:25:54.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.858+0000 7fd16d8d4740 1 -- 172.21.15.22:0/3582078389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d35d0 msgr2=0x564bb25cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:54.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.858+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/3582078389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d35d0 0x564bb25cfe10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd16000b760 tx=0x7fd16000b5b0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.858+0000 7fd16d8d4740 1 -- 172.21.15.22:0/3582078389 shutdown_connections 2024-09-19T08:25:54.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.858+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/3582078389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25de370 0x564bb25ea310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.858+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/3582078389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25d4b40 0x564bb25d4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.858+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/3582078389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d35d0 0x564bb25cfe10 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.858+0000 7fd16d8d4740 1 -- 172.21.15.22:0/3582078389 >> 172.21.15.22:0/3582078389 conn(0x564bb259c310 msgr2=0x564bb25cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:54.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.858+0000 7fd16d8d4740 1 -- 172.21.15.22:0/3582078389 shutdown_connections 2024-09-19T08:25:54.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.858+0000 7fd16d8d4740 1 -- 172.21.15.22:0/3582078389 wait complete. 2024-09-19T08:25:54.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.859+0000 7fd16d8d4740 1 -- start start 2024-09-19T08:25:54.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.859+0000 7fd16d8d4740 1 Processor -- start 2024-09-19T08:25:54.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.859+0000 7fd16d8d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25d35d0 0x564bb25e2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.859+0000 7fd16d8d4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d4b40 0x564bb25e2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.859+0000 7fd16d8d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25de370 0x564bb25e2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.859+0000 7fd16d8d4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641ae0c0 con 0x564bb25d4b40 2024-09-19T08:25:54.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.859+0000 7fd16d8d4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641af530 con 0x564bb25de370 2024-09-19T08:25:54.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.859+0000 7fd16d8d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641ae790 con 0x564bb25d35d0 2024-09-19T08:25:54.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.859+0000 7fd16d8d4740 10 client.0 _ll_register_callbacks cb 0x7fd154005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:25:54.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.859+0000 7fd168a96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d4b40 0x564bb25e2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.859+0000 7fd168a96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d4b40 0x564bb25e2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47704/0 (socket says 172.21.15.22:47704) 2024-09-19T08:25:54.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.859+0000 7fd168a96640 1 -- 172.21.15.22:0/3730178930 learned_addr learned my addr 172.21.15.22:0/3730178930 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:54.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.859+0000 7fd169297640 1 --2- 172.21.15.22:0/3730178930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25d35d0 0x564bb25e2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.860+0000 7fd169a98640 1 --2- 172.21.15.22:0/3730178930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25de370 0x564bb25e2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.860+0000 7fd168a96640 1 -- 172.21.15.22:0/3730178930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25d35d0 msgr2=0x564bb25e2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:54.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.860+0000 7fd168a96640 1 --2- 172.21.15.22:0/3730178930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25d35d0 0x564bb25e2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.860+0000 7fd168a96640 1 -- 172.21.15.22:0/3730178930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25de370 msgr2=0x564bb25e2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:25:54.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.860+0000 7fd168a96640 1 --2- 172.21.15.22:0/3730178930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25de370 0x564bb25e2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.860+0000 7fd168a96640 1 -- 172.21.15.22:0/3730178930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bb2660490 con 0x564bb25d4b40 2024-09-19T08:25:54.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.860+0000 7fd168a96640 1 --2- 172.21.15.22:0/3730178930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d4b40 0x564bb25e2660 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd1540ba900 tx=0x7fd1540c4fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bef7ba61f917e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:54.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.860+0000 7fd14cff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-19T08:25:54.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.860+0000 7fd14cff9640 1 -- 172.21.15.22:0/3730178930 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1540a9060 con 0x564bb25d4b40 2024-09-19T08:25:54.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.860+0000 7fd14cff9640 1 -- 172.21.15.22:0/3730178930 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fd1540bbaa0 con 0x564bb25d4b40 2024-09-19T08:25:54.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.860+0000 7fd16d8d4740 1 -- 172.21.15.22:0/3730178930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd160015040 con 0x564bb25d4b40 2024-09-19T08:25:54.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.860+0000 7fd14cff9640 1 -- 172.21.15.22:0/3730178930 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd154049040 con 0x564bb25d4b40 2024-09-19T08:25:54.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.860+0000 7fd16d8d4740 10 client.17280 fetch_fsmap learned FSMap version 36 2024-09-19T08:25:54.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.860+0000 7fd16d8d4740 1 -- 172.21.15.22:0/3730178930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564bb2660490 con 0x564bb25d4b40 2024-09-19T08:25:54.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.861+0000 7fd14cff9640 1 -- 172.21.15.22:0/3730178930 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd15403b580 con 0x564bb25d4b40 2024-09-19T08:25:54.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.861+0000 7fd14cff9640 1 -- 172.21.15.22:0/3730178930 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fd1540b83a0 con 0x564bb25d4b40 2024-09-19T08:25:54.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.861+0000 7fd16d8d4740 10 client.17280 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T08:25:54.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.861+0000 7fd16d8d4740 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-19T08:25:54.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.861+0000 7fd16d8d4740 20 client.17280 populate_metadata read hostname 'smithi022' 2024-09-19T08:25:54.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.861+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/3730178930 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564bb2661a30 0x564bb2663e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:54.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.861+0000 7fd16d8d4740 4 client.17280 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:25:54.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.861+0000 7fd16d8d4740 1 -- 172.21.15.22:0/3730178930 --> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd160015040 con 0x564bb2661a30 2024-09-19T08:25:54.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.861+0000 7fd169297640 1 --2- 172.21.15.22:0/3730178930 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564bb2661a30 0x564bb2663e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:54.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.861+0000 7fd14cff9640 10 client.17280 ms_handle_connect on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:54.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.861+0000 7fd169297640 1 --2- 172.21.15.22:0/3730178930 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564bb2661a30 0x564bb2663e50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f969bee4b5442f2b server_cookie=d302d60f7261038 in_seq=0 out_seq=0 2024-09-19T08:25:54.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd14cff9640 1 -- 172.21.15.22:0/3730178930 <== mds.0 v2:172.21.15.157:6838/3802797238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd160015040 con 0x564bb2661a30 2024-09-19T08:25:54.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd14cff9640 10 client.17280 handle_command_reply: tid=0 2024-09-19T08:25:54.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd16d8d4740 1 client.17280 shutdown 2024-09-19T08:25:54.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd16d8d4740 1 -- 172.21.15.22:0/3730178930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d4b40 msgr2=0x564bb25e2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:54.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/3730178930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d4b40 0x564bb25e2660 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd1540ba900 tx=0x7fd1540c4fb0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd16d8d4740 1 -- 172.21.15.22:0/3730178930 shutdown_connections 2024-09-19T08:25:54.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/3730178930 >> [v2:172.21.15.157:6838/3802797238,v1:172.21.15.157:6839/3802797238] conn(0x564bb2661a30 0x564bb2663e50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/3730178930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bb25de370 0x564bb25e2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd14cff9640 0 client.17280 ms_handle_reset on v2:172.21.15.157:6838/3802797238 2024-09-19T08:25:54.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/3730178930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bb25d4b40 0x564bb25e2660 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd16d8d4740 1 --2- 172.21.15.22:0/3730178930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564bb25d35d0 0x564bb25e2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd16d8d4740 1 -- 172.21.15.22:0/3730178930 >> 172.21.15.22:0/3730178930 conn(0x564bb259c310 msgr2=0x564bb25cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:54.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd16d8d4740 1 -- 172.21.15.22:0/3730178930 shutdown_connections 2024-09-19T08:25:54.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd16d8d4740 1 -- 172.21.15.22:0/3730178930 wait complete. 2024-09-19T08:25:54.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.862+0000 7fd16d8d4740 20 client.17280 trim_cache size 0 max 16384 2024-09-19T08:25:54.889 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-19T08:25:54.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.863+0000 7fd16b4fb640 1 -- 172.21.15.22:0/4125501380 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd1500a3430 msgr2=0x7fd1500a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:54.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.863+0000 7fd16b4fb640 1 --2- 172.21.15.22:0/4125501380 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd1500a3430 0x7fd1500a58f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009990 tx=0x7fd15c0096a0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.863+0000 7fd16b4fb640 1 -- 172.21.15.22:0/4125501380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1641049a0 msgr2=0x7fd16419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:54.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.863+0000 7fd16b4fb640 1 --2- 172.21.15.22:0/4125501380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1641049a0 0x7fd16419ab80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd154006010 tx=0x7fd1540042f0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.863+0000 7fd16b4fb640 1 -- 172.21.15.22:0/4125501380 shutdown_connections 2024-09-19T08:25:54.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.863+0000 7fd16b4fb640 1 --2- 172.21.15.22:0/4125501380 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fd1500a3430 0x7fd1500a58f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.863+0000 7fd16b4fb640 1 --2- 172.21.15.22:0/4125501380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd164109120 0x7fd16419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.863+0000 7fd16b4fb640 1 --2- 172.21.15.22:0/4125501380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1641049a0 0x7fd16419ab80 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.863+0000 7fd16b4fb640 1 --2- 172.21.15.22:0/4125501380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd164103ff0 0x7fd16419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:54.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.863+0000 7fd16b4fb640 1 -- 172.21.15.22:0/4125501380 >> 172.21.15.22:0/4125501380 conn(0x7fd1640ffab0 msgr2=0x7fd164100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:54.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.863+0000 7fd16b4fb640 1 -- 172.21.15.22:0/4125501380 shutdown_connections 2024-09-19T08:25:54.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:54.863+0000 7fd16b4fb640 1 -- 172.21.15.22:0/4125501380 wait complete. 2024-09-19T08:25:55.227 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:55 smithi022 ceph-mon[30963]: pgmap v337: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T08:25:55.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:55 smithi064 ceph-mon[34900]: pgmap v337: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T08:25:55.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:55 smithi157 ceph-mon[35143]: pgmap v337: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T08:25:55.652 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-19T08:25:55.653 INFO:tasks.fwd_scrub:done joining 2024-09-19T08:25:55.653 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T08:25:55.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:25:55.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.775+0000 7fc58d2ac640 1 -- 172.21.15.22:0/3335881521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc588102c70 msgr2=0x7fc588103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:55.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.775+0000 7fc58d2ac640 1 --2- 172.21.15.22:0/3335881521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc588102c70 0x7fc588103050 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc578009ca0 tx=0x7fc57802a950 comp rx=0 tx=0).stop 2024-09-19T08:25:55.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.775+0000 7fc58d2ac640 1 -- 172.21.15.22:0/3335881521 shutdown_connections 2024-09-19T08:25:55.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.775+0000 7fc58d2ac640 1 --2- 172.21.15.22:0/3335881521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc58810c0c0 0x7fc58810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:55.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.775+0000 7fc58d2ac640 1 --2- 172.21.15.22:0/3335881521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc588103590 0x7fc58810ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:55.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.775+0000 7fc58d2ac640 1 --2- 172.21.15.22:0/3335881521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc588102c70 0x7fc588103050 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:55.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.775+0000 7fc58d2ac640 1 -- 172.21.15.22:0/3335881521 >> 172.21.15.22:0/3335881521 conn(0x7fc5880fc680 msgr2=0x7fc5880feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:55.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.775+0000 7fc58d2ac640 1 -- 172.21.15.22:0/3335881521 shutdown_connections 2024-09-19T08:25:55.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.775+0000 7fc58d2ac640 1 -- 172.21.15.22:0/3335881521 wait complete. 2024-09-19T08:25:55.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc58d2ac640 1 Processor -- start 2024-09-19T08:25:55.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc58d2ac640 1 -- start start 2024-09-19T08:25:55.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc58d2ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc588102c70 0x7fc5881983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:55.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc58d2ac640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc588103590 0x7fc588198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:55.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc58d2ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc58810c0c0 0x7fc58819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:55.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc58d2ac640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc588111ef0 con 0x7fc588103590 2024-09-19T08:25:55.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc58d2ac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc588111d70 con 0x7fc588102c70 2024-09-19T08:25:55.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc58d2ac640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc588112070 con 0x7fc58810c0c0 2024-09-19T08:25:55.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc5867fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc588103590 0x7fc588198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:55.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc5867fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc588103590 0x7fc588198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47734/0 (socket says 172.21.15.22:47734) 2024-09-19T08:25:55.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc5867fc640 1 -- 172.21.15.22:0/4179068836 learned_addr learned my addr 172.21.15.22:0/4179068836 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:55.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc586ffd640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc588102c70 0x7fc5881983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:55.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc5877fe640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc58810c0c0 0x7fc58819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:55.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc5867fc640 1 -- 172.21.15.22:0/4179068836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc58810c0c0 msgr2=0x7fc58819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:55.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc5867fc640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc58810c0c0 0x7fc58819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:55.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc5867fc640 1 -- 172.21.15.22:0/4179068836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc588102c70 msgr2=0x7fc5881983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:55.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc5867fc640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc588102c70 0x7fc5881983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:55.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc5867fc640 1 -- 172.21.15.22:0/4179068836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58819d3b0 con 0x7fc588103590 2024-09-19T08:25:55.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.776+0000 7fc5877fe640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc58810c0c0 0x7fc58819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:25:55.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.777+0000 7fc5867fc640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc588103590 0x7fc588198900 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00e9f0 tx=0x7fc57c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:55.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.777+0000 7fc56ffff640 1 -- 172.21.15.22:0/4179068836 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00ccc0 con 0x7fc588103590 2024-09-19T08:25:55.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.777+0000 7fc56ffff640 1 -- 172.21.15.22:0/4179068836 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc57c010910 con 0x7fc588103590 2024-09-19T08:25:55.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.777+0000 7fc58d2ac640 1 -- 172.21.15.22:0/4179068836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58819d6a0 con 0x7fc588103590 2024-09-19T08:25:55.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.777+0000 7fc56ffff640 1 -- 172.21.15.22:0/4179068836 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c018920 con 0x7fc588103590 2024-09-19T08:25:55.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.777+0000 7fc58d2ac640 1 -- 172.21.15.22:0/4179068836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5881a4f30 con 0x7fc588103590 2024-09-19T08:25:55.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.778+0000 7fc58d2ac640 1 -- 172.21.15.22:0/4179068836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc588103050 con 0x7fc588103590 2024-09-19T08:25:55.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.779+0000 7fc56ffff640 1 -- 172.21.15.22:0/4179068836 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc57c010430 con 0x7fc588103590 2024-09-19T08:25:55.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.780+0000 7fc56ffff640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc5700a3440 0x7fc5700a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:55.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.780+0000 7fc586ffd640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc5700a3440 0x7fc5700a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:55.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.780+0000 7fc56ffff640 1 -- 172.21.15.22:0/4179068836 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fc57c0cffe0 con 0x7fc588103590 2024-09-19T08:25:55.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.780+0000 7fc586ffd640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc5700a3440 0x7fc5700a5900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc57802af40 tx=0x7fc578005d80 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:55.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.784+0000 7fc56ffff640 1 -- 172.21.15.22:0/4179068836 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc57c09c950 con 0x7fc588103590 2024-09-19T08:25:55.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.960+0000 7fc58d2ac640 1 -- 172.21.15.22:0/4179068836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc588107170 con 0x7fc588103590 2024-09-19T08:25:55.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.961+0000 7fc56ffff640 1 -- 172.21.15.22:0/4179068836 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+23190 (secure 0 0 0) 0x7fc57c09c250 con 0x7fc588103590 2024-09-19T08:25:55.963 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:25:55.963 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":94,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","created":"2024-09-19T08:05:19.476952+0000","modified":"2024-09-19T08:25:15.491420+0000","last_up_change":"2024-09-19T08:12:10.339586+0000","last_in_change":"2024-09-19T08:11:47.653576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:08:25.175759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:12:57.608787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:12:59.824327+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":93,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ed67f2d-9b28-427a-b69f-676e7a2abcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6803","nonce":3442715307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6805","nonce":3442715307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6809","nonce":3442715307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6807","nonce":3442715307}]},"public_addr":"172.21.15.22:6803/3442715307","cluster_addr":"172.21.15.22:6805/3442715307","heartbeat_back_addr":"172.21.15.22:6809/3442715307","heartbeat_front_addr":"172.21.15.22:6807/3442715307","state":["exists","up"]},{"osd":1,"uuid":"bff3e283-4ab1-49d7-86d8-45d361995122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6801","nonce":2379700228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6803","nonce":2379700228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6807","nonce":2379700228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6805","nonce":2379700228}]},"public_addr":"172.21.15.64:6801/2379700228","cluster_addr":"172.21.15.64:6803/2379700228","heartbeat_back_addr":"172.21.15.64:6807/2379700228","heartbeat_front_addr":"172.21.15.64:6805/2379700228","state":["exists","up"]},{"osd":2,"uuid":"81421d08-7dc6-419b-a566-dee1e1e13b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6801","nonce":3363263867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6803","nonce":3363263867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6807","nonce":3363263867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6805","nonce":3363263867}]},"public_addr":"172.21.15.157:6801/3363263867","cluster_addr":"172.21.15.157:6803/3363263867","heartbeat_back_addr":"172.21.15.157:6807/3363263867","heartbeat_front_addr":"172.21.15.157:6805/3363263867","state":["exists","up"]},{"osd":3,"uuid":"34debfd0-3226-4d78-b9e6-e03dbc5190f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6811","nonce":4225229456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6813","nonce":4225229456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6817","nonce":4225229456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6815","nonce":4225229456}]},"public_addr":"172.21.15.22:6811/4225229456","cluster_addr":"172.21.15.22:6813/4225229456","heartbeat_back_addr":"172.21.15.22:6817/4225229456","heartbeat_front_addr":"172.21.15.22:6815/4225229456","state":["exists","up"]},{"osd":4,"uuid":"1293332c-d772-4ca9-bf76-79e0b29451ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6809","nonce":1858090423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6811","nonce":1858090423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6815","nonce":1858090423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6813","nonce":1858090423}]},"public_addr":"172.21.15.64:6809/1858090423","cluster_addr":"172.21.15.64:6811/1858090423","heartbeat_back_addr":"172.21.15.64:6815/1858090423","heartbeat_front_addr":"172.21.15.64:6813/1858090423","state":["exists","up"]},{"osd":5,"uuid":"240a6c42-89a2-41c1-948c-b09914b5722e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6809","nonce":2900701212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2900701212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2900701212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2900701212}]},"public_addr":"172.21.15.157:6809/2900701212","cluster_addr":"172.21.15.157:6811/2900701212","heartbeat_back_addr":"172.21.15.157:6815/2900701212","heartbeat_front_addr":"172.21.15.157:6813/2900701212","state":["exists","up"]},{"osd":6,"uuid":"80199030-109e-41c4-bc1c-731755464770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6819","nonce":4264426242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6821","nonce":4264426242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6825","nonce":4264426242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6823","nonce":4264426242}]},"public_addr":"172.21.15.22:6819/4264426242","cluster_addr":"172.21.15.22:6821/4264426242","heartbeat_back_addr":"172.21.15.22:6825/4264426242","heartbeat_front_addr":"172.21.15.22:6823/4264426242","state":["exists","up"]},{"osd":7,"uuid":"dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6817","nonce":3980224103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6819","nonce":3980224103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6823","nonce":3980224103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6821","nonce":3980224103}]},"public_addr":"172.21.15.64:6817/3980224103","cluster_addr":"172.21.15.64:6819/3980224103","heartbeat_back_addr":"172.21.15.64:6823/3980224103","heartbeat_front_addr":"172.21.15.64:6821/3980224103","state":["exists","up"]},{"osd":8,"uuid":"764b1e6e-48c9-49f9-8124-bc13bcba3a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6817","nonce":973678477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6819","nonce":973678477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6823","nonce":973678477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6821","nonce":973678477}]},"public_addr":"172.21.15.157:6817/973678477","cluster_addr":"172.21.15.157:6819/973678477","heartbeat_back_addr":"172.21.15.157:6823/973678477","heartbeat_front_addr":"172.21.15.157:6821/973678477","state":["exists","up"]},{"osd":9,"uuid":"82914818-df47-4374-9e3e-2f71fc76e78b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6827","nonce":710033990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6829","nonce":710033990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6833","nonce":710033990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6831","nonce":710033990}]},"public_addr":"172.21.15.22:6827/710033990","cluster_addr":"172.21.15.22:6829/710033990","heartbeat_back_addr":"172.21.15.22:6833/710033990","heartbeat_front_addr":"172.21.15.22:6831/710033990","state":["exists","up"]},{"osd":10,"uuid":"36ae177d-c454-4cba-ba21-cbeee0f10dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6825","nonce":373412752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6827","nonce":373412752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6831","nonce":373412752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6829","nonce":373412752}]},"public_addr":"172.21.15.64:6825/373412752","cluster_addr":"172.21.15.64:6827/373412752","heartbeat_back_addr":"172.21.15.64:6831/373412752","heartbeat_front_addr":"172.21.15.64:6829/373412752","state":["exists","up"]},{"osd":11,"uuid":"cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6825","nonce":1918889194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6827","nonce":1918889194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6831","nonce":1918889194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6829","nonce":1918889194}]},"public_addr":"172.21.15.157:6825/1918889194","cluster_addr":"172.21.15.157:6827/1918889194","heartbeat_back_addr":"172.21.15.157:6831/1918889194","heartbeat_front_addr":"172.21.15.157:6829/1918889194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:31.378218+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:56.794167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:20.790267+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:47.039106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:11.483842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:38.151466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:03.553308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:27.865321+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:53.086530+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:18.946661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:43.697243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:12:08.936588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/422860013":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/2837500116":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/1845420423":"2024-09-20T08:15:01.165466+0000","172.21.15.22:6800/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/3623845502":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2851750481":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2133059277":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/384027596":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/586163474":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/3712426751":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/1866276413":"2024-09-20T08:15:01.165466+0000","172.21.15.22:6801/1866276413":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/155709052":"2024-09-20T08:15:01.165466+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:25:55.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.963+0000 7fc58d2ac640 1 -- 172.21.15.22:0/4179068836 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc5700a3440 msgr2=0x7fc5700a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:55.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.963+0000 7fc58d2ac640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc5700a3440 0x7fc5700a5900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc57802af40 tx=0x7fc578005d80 comp rx=0 tx=0).stop 2024-09-19T08:25:55.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.963+0000 7fc58d2ac640 1 -- 172.21.15.22:0/4179068836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc588103590 msgr2=0x7fc588198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:55.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.963+0000 7fc58d2ac640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc588103590 0x7fc588198900 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00e9f0 tx=0x7fc57c00eeb0 comp rx=0 tx=0).stop 2024-09-19T08:25:55.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.963+0000 7fc58d2ac640 1 -- 172.21.15.22:0/4179068836 shutdown_connections 2024-09-19T08:25:55.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.963+0000 7fc58d2ac640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc5700a3440 0x7fc5700a5900 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:55.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.963+0000 7fc58d2ac640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc58810c0c0 0x7fc58819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:55.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.963+0000 7fc58d2ac640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc588103590 0x7fc588198900 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:55.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.963+0000 7fc58d2ac640 1 --2- 172.21.15.22:0/4179068836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc588102c70 0x7fc5881983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:55.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.963+0000 7fc58d2ac640 1 -- 172.21.15.22:0/4179068836 >> 172.21.15.22:0/4179068836 conn(0x7fc5880fc680 msgr2=0x7fc5880fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:55.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.964+0000 7fc58d2ac640 1 -- 172.21.15.22:0/4179068836 shutdown_connections 2024-09-19T08:25:55.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:55.964+0000 7fc58d2ac640 1 -- 172.21.15.22:0/4179068836 wait complete. 2024-09-19T08:25:55.976 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:08:25.175759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T08:12:57.608787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T08:12:59.824327+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 93, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:25:55.976 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:25:56.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.083+0000 7f0737726640 1 -- 172.21.15.22:0/240604606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f073010c0e0 msgr2=0x7f073010e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.083+0000 7f0737726640 1 --2- 172.21.15.22:0/240604606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f073010c0e0 0x7f073010e530 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0728009900 tx=0x7f072802a480 comp rx=0 tx=0).stop 2024-09-19T08:25:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.083+0000 7f0737726640 1 -- 172.21.15.22:0/240604606 shutdown_connections 2024-09-19T08:25:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.083+0000 7f0737726640 1 --2- 172.21.15.22:0/240604606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f073010c0e0 0x7f073010e530 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.083+0000 7f0737726640 1 --2- 172.21.15.22:0/240604606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730103630 0x7f073010bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.083+0000 7f0737726640 1 --2- 172.21.15.22:0/240604606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0730102d10 0x7f07301030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.083+0000 7f0737726640 1 -- 172.21.15.22:0/240604606 >> 172.21.15.22:0/240604606 conn(0x7f07300779d0 msgr2=0x7f07300fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.083+0000 7f0737726640 1 -- 172.21.15.22:0/240604606 shutdown_connections 2024-09-19T08:25:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.083+0000 7f0737726640 1 -- 172.21.15.22:0/240604606 wait complete. 2024-09-19T08:25:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f0737726640 1 Processor -- start 2024-09-19T08:25:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f0737726640 1 -- start start 2024-09-19T08:25:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f0737726640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0730102d10 0x7f0730100960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f0737726640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730103630 0x7f0730100ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f0737726640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073010c0e0 0x7f07301013e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f0737726640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0730111ed0 con 0x7f0730102d10 2024-09-19T08:25:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f0737726640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0730111d50 con 0x7f0730103630 2024-09-19T08:25:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f0737726640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0730112050 con 0x7f073010c0e0 2024-09-19T08:25:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f07354c2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0730102d10 0x7f0730100960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f07354c2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0730102d10 0x7f0730100960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47752/0 (socket says 172.21.15.22:47752) 2024-09-19T08:25:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f07354c2640 1 -- 172.21.15.22:0/2849771756 learned_addr learned my addr 172.21.15.22:0/2849771756 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f0735cc3640 1 --2- 172.21.15.22:0/2849771756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073010c0e0 0x7f07301013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f0734cc1640 1 --2- 172.21.15.22:0/2849771756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730103630 0x7f0730100ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f07354c2640 1 -- 172.21.15.22:0/2849771756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073010c0e0 msgr2=0x7f07301013e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f07354c2640 1 --2- 172.21.15.22:0/2849771756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073010c0e0 0x7f07301013e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f07354c2640 1 -- 172.21.15.22:0/2849771756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730103630 msgr2=0x7f0730100ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f07354c2640 1 --2- 172.21.15.22:0/2849771756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730103630 0x7f0730100ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f07354c2640 1 -- 172.21.15.22:0/2849771756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07301a6890 con 0x7f0730102d10 2024-09-19T08:25:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.084+0000 7f07354c2640 1 --2- 172.21.15.22:0/2849771756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0730102d10 0x7f0730100960 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f072c00eb90 tx=0x7f072c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.085+0000 7f07227fc640 1 -- 172.21.15.22:0/2849771756 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c004d20 con 0x7f0730102d10 2024-09-19T08:25:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.085+0000 7f07227fc640 1 -- 172.21.15.22:0/2849771756 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f072c00ee90 con 0x7f0730102d10 2024-09-19T08:25:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.085+0000 7f0737726640 1 -- 172.21.15.22:0/2849771756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07301a6b80 con 0x7f0730102d10 2024-09-19T08:25:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.085+0000 7f07227fc640 1 -- 172.21.15.22:0/2849771756 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c010830 con 0x7f0730102d10 2024-09-19T08:25:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.085+0000 7f0737726640 1 -- 172.21.15.22:0/2849771756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07301a70d0 con 0x7f0730102d10 2024-09-19T08:25:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.086+0000 7f0737726640 1 -- 172.21.15.22:0/2849771756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07301030f0 con 0x7f0730102d10 2024-09-19T08:25:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.087+0000 7f07227fc640 1 -- 172.21.15.22:0/2849771756 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f072c005020 con 0x7f0730102d10 2024-09-19T08:25:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.088+0000 7f07227fc640 1 --2- 172.21.15.22:0/2849771756 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f071c0a3440 0x7f071c0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.088+0000 7f0734cc1640 1 --2- 172.21.15.22:0/2849771756 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f071c0a3440 0x7f071c0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.088+0000 7f07227fc640 1 -- 172.21.15.22:0/2849771756 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f072c0d0310 con 0x7f0730102d10 2024-09-19T08:25:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.088+0000 7f0734cc1640 1 --2- 172.21.15.22:0/2849771756 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f071c0a3440 0x7f071c0a5900 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f07301023f0 tx=0x7f0724005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.092+0000 7f07227fc640 1 -- 172.21.15.22:0/2849771756 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f072c09cce0 con 0x7f0730102d10 2024-09-19T08:25:56.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:56 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/4179068836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:25:56.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.268+0000 7f0737726640 1 -- 172.21.15.22:0/2849771756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0730107190 con 0x7f0730102d10 2024-09-19T08:25:56.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.269+0000 7f07227fc640 1 -- 172.21.15.22:0/2849771756 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f072c09c5e0 con 0x7f0730102d10 2024-09-19T08:25:56.270 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-09-19T08:25:56.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.271+0000 7f0737726640 1 -- 172.21.15.22:0/2849771756 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f071c0a3440 msgr2=0x7f071c0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.271+0000 7f0737726640 1 --2- 172.21.15.22:0/2849771756 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f071c0a3440 0x7f071c0a5900 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f07301023f0 tx=0x7f0724005f90 comp rx=0 tx=0).stop 2024-09-19T08:25:56.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.271+0000 7f0737726640 1 -- 172.21.15.22:0/2849771756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0730102d10 msgr2=0x7f0730100960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.271+0000 7f0737726640 1 --2- 172.21.15.22:0/2849771756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0730102d10 0x7f0730100960 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f072c00eb90 tx=0x7f072c00c6a0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.271+0000 7f0737726640 1 -- 172.21.15.22:0/2849771756 shutdown_connections 2024-09-19T08:25:56.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.271+0000 7f0737726640 1 --2- 172.21.15.22:0/2849771756 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f071c0a3440 0x7f071c0a5900 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.271+0000 7f0737726640 1 --2- 172.21.15.22:0/2849771756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073010c0e0 0x7f07301013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.271+0000 7f0737726640 1 --2- 172.21.15.22:0/2849771756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730103630 0x7f0730100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.271+0000 7f0737726640 1 --2- 172.21.15.22:0/2849771756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0730102d10 0x7f0730100960 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.271+0000 7f0737726640 1 -- 172.21.15.22:0/2849771756 >> 172.21.15.22:0/2849771756 conn(0x7f07300779d0 msgr2=0x7f07300fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:56.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.271+0000 7f0737726640 1 -- 172.21.15.22:0/2849771756 shutdown_connections 2024-09-19T08:25:56.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.271+0000 7f0737726640 1 -- 172.21.15.22:0/2849771756 wait complete. 2024-09-19T08:25:56.283 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T08:25:56.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:56 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/4179068836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:25:56.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.389+0000 7f1ad2db9640 1 -- 172.21.15.22:0/2326674804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc100980 msgr2=0x7f1acc100de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.389+0000 7f1ad2db9640 1 --2- 172.21.15.22:0/2326674804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc100980 0x7f1acc100de0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009930 tx=0x7f1ac002a420 comp rx=0 tx=0).stop 2024-09-19T08:25:56.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.389+0000 7f1ad2db9640 1 -- 172.21.15.22:0/2326674804 shutdown_connections 2024-09-19T08:25:56.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.389+0000 7f1ad2db9640 1 --2- 172.21.15.22:0/2326674804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1acc101320 0x7f1acc10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.389+0000 7f1ad2db9640 1 --2- 172.21.15.22:0/2326674804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc100980 0x7f1acc100de0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.389+0000 7f1ad2db9640 1 --2- 172.21.15.22:0/2326674804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1acc106990 0x7f1acc106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.389+0000 7f1ad2db9640 1 -- 172.21.15.22:0/2326674804 >> 172.21.15.22:0/2326674804 conn(0x7f1acc0779d0 msgr2=0x7f1acc0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:56.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.389+0000 7f1ad2db9640 1 -- 172.21.15.22:0/2326674804 shutdown_connections 2024-09-19T08:25:56.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.389+0000 7f1ad2db9640 1 -- 172.21.15.22:0/2326674804 wait complete. 2024-09-19T08:25:56.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.389+0000 7f1ad2db9640 1 Processor -- start 2024-09-19T08:25:56.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.389+0000 7f1ad2db9640 1 -- start start 2024-09-19T08:25:56.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1ad2db9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1acc100980 0x7f1acc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:56.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1ad2db9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc101320 0x7f1acc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:56.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1ad2db9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1acc106990 0x7f1acc19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:56.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1ad2db9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc111e80 con 0x7f1acc101320 2024-09-19T08:25:56.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1ad2db9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc111d00 con 0x7f1acc100980 2024-09-19T08:25:56.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1ad2db9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc112000 con 0x7f1acc106990 2024-09-19T08:25:56.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1acbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc101320 0x7f1acc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:56.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1acbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc101320 0x7f1acc198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47768/0 (socket says 172.21.15.22:47768) 2024-09-19T08:25:56.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1acbfff640 1 -- 172.21.15.22:0/3230729136 learned_addr learned my addr 172.21.15.22:0/3230729136 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:56.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1ad0b55640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1acc100980 0x7f1acc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:56.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1ad1356640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1acc106990 0x7f1acc19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:56.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1acbfff640 1 -- 172.21.15.22:0/3230729136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1acc106990 msgr2=0x7f1acc19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1acbfff640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1acc106990 0x7f1acc19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1acbfff640 1 -- 172.21.15.22:0/3230729136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1acc100980 msgr2=0x7f1acc1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1acbfff640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1acc100980 0x7f1acc1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1acbfff640 1 -- 172.21.15.22:0/3230729136 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc19d4c0 con 0x7f1acc101320 2024-09-19T08:25:56.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1ad0b55640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1acc100980 0x7f1acc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:25:56.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1acbfff640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc101320 0x7f1acc198a10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009230 tx=0x7f1ac0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:56.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1ac9ffb640 1 -- 172.21.15.22:0/3230729136 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac003f030 con 0x7f1acc101320 2024-09-19T08:25:56.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1ac9ffb640 1 -- 172.21.15.22:0/3230729136 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac0004440 con 0x7f1acc101320 2024-09-19T08:25:56.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1ac9ffb640 1 -- 172.21.15.22:0/3230729136 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0035070 con 0x7f1acc101320 2024-09-19T08:25:56.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.390+0000 7f1ad2db9640 1 -- 172.21.15.22:0/3230729136 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1acc19d750 con 0x7f1acc101320 2024-09-19T08:25:56.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.391+0000 7f1ad2db9640 1 -- 172.21.15.22:0/3230729136 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1acc1a5040 con 0x7f1acc101320 2024-09-19T08:25:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.392+0000 7f1ad2db9640 1 -- 172.21.15.22:0/3230729136 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1acc100de0 con 0x7f1acc101320 2024-09-19T08:25:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.393+0000 7f1ac9ffb640 1 -- 172.21.15.22:0/3230729136 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1ac0042030 con 0x7f1acc101320 2024-09-19T08:25:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.394+0000 7f1ac9ffb640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f1ab40a3440 0x7f1ab40a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.394+0000 7f1ac9ffb640 1 -- 172.21.15.22:0/3230729136 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f1ac00f03c0 con 0x7f1acc101320 2024-09-19T08:25:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.394+0000 7f1ad0b55640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f1ab40a3440 0x7f1ab40a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.394+0000 7f1ad0b55640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f1ab40a3440 0x7f1ab40a5900 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009f40 tx=0x7f1abc005f90 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:56.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.398+0000 7f1ac9ffb640 1 -- 172.21.15.22:0/3230729136 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1ac00bc740 con 0x7f1acc101320 2024-09-19T08:25:56.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:56 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/4179068836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:25:56.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.574+0000 7f1ad2db9640 1 -- 172.21.15.22:0/3230729136 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1acc108f10 con 0x7f1acc101320 2024-09-19T08:25:56.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.574+0000 7f1ac9ffb640 1 -- 172.21.15.22:0/3230729136 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f1ac00c0a40 con 0x7f1acc101320 2024-09-19T08:25:56.575 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-09-19T08:25:56.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.576+0000 7f1ad2db9640 1 -- 172.21.15.22:0/3230729136 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f1ab40a3440 msgr2=0x7f1ab40a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.576+0000 7f1ad2db9640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f1ab40a3440 0x7f1ab40a5900 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009f40 tx=0x7f1abc005f90 comp rx=0 tx=0).stop 2024-09-19T08:25:56.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.576+0000 7f1ad2db9640 1 -- 172.21.15.22:0/3230729136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc101320 msgr2=0x7f1acc198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.576+0000 7f1ad2db9640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc101320 0x7f1acc198a10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009230 tx=0x7f1ac0004040 comp rx=0 tx=0).stop 2024-09-19T08:25:56.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.577+0000 7f1ad2db9640 1 -- 172.21.15.22:0/3230729136 shutdown_connections 2024-09-19T08:25:56.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.577+0000 7f1ad2db9640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f1ab40a3440 0x7f1ab40a5900 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.577+0000 7f1ad2db9640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1acc106990 0x7f1acc19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.577+0000 7f1ad2db9640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc101320 0x7f1acc198a10 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.577+0000 7f1ad2db9640 1 --2- 172.21.15.22:0/3230729136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1acc100980 0x7f1acc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.577+0000 7f1ad2db9640 1 -- 172.21.15.22:0/3230729136 >> 172.21.15.22:0/3230729136 conn(0x7f1acc0779d0 msgr2=0x7f1acc1039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:56.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.577+0000 7f1ad2db9640 1 -- 172.21.15.22:0/3230729136 shutdown_connections 2024-09-19T08:25:56.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.577+0000 7f1ad2db9640 1 -- 172.21.15.22:0/3230729136 wait complete. 2024-09-19T08:25:56.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T08:25:56.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.694+0000 7fdf4524c640 1 -- 172.21.15.22:0/549917671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf401008e0 msgr2=0x7fdf40100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.694+0000 7fdf4524c640 1 --2- 172.21.15.22:0/549917671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf401008e0 0x7fdf40100cc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009900 tx=0x7fdf3002a410 comp rx=0 tx=0).stop 2024-09-19T08:25:56.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.694+0000 7fdf4524c640 1 -- 172.21.15.22:0/549917671 shutdown_connections 2024-09-19T08:25:56.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.694+0000 7fdf4524c640 1 --2- 172.21.15.22:0/549917671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf40105580 0x7fdf4010d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.694+0000 7fdf4524c640 1 --2- 172.21.15.22:0/549917671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf40101290 0x7fdf40104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.694+0000 7fdf4524c640 1 --2- 172.21.15.22:0/549917671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf401008e0 0x7fdf40100cc0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.694+0000 7fdf4524c640 1 -- 172.21.15.22:0/549917671 >> 172.21.15.22:0/549917671 conn(0x7fdf40075ea0 msgr2=0x7fdf400762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:56.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.694+0000 7fdf4524c640 1 -- 172.21.15.22:0/549917671 shutdown_connections 2024-09-19T08:25:56.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.694+0000 7fdf4524c640 1 -- 172.21.15.22:0/549917671 wait complete. 2024-09-19T08:25:56.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.695+0000 7fdf4524c640 1 Processor -- start 2024-09-19T08:25:56.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.695+0000 7fdf4524c640 1 -- start start 2024-09-19T08:25:56.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.695+0000 7fdf4524c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf401008e0 0x7fdf40199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:56.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.695+0000 7fdf4524c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf40105580 0x7fdf401997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:56.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.695+0000 7fdf4524c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf40199d50 0x7fdf401a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:56.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.695+0000 7fdf4524c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdf40110e70 con 0x7fdf40105580 2024-09-19T08:25:56.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.695+0000 7fdf4524c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdf40110cf0 con 0x7fdf401008e0 2024-09-19T08:25:56.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.695+0000 7fdf4524c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdf40110ff0 con 0x7fdf40199d50 2024-09-19T08:25:56.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.695+0000 7fdf37fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf40105580 0x7fdf401997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:56.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.695+0000 7fdf37fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf40105580 0x7fdf401997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47788/0 (socket says 172.21.15.22:47788) 2024-09-19T08:25:56.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.695+0000 7fdf37fff640 1 -- 172.21.15.22:0/399863055 learned_addr learned my addr 172.21.15.22:0/399863055 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:56.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.695+0000 7fdf3ed9d640 1 --2- 172.21.15.22:0/399863055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf401008e0 0x7fdf40199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:56.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.695+0000 7fdf3f59e640 1 --2- 172.21.15.22:0/399863055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf40199d50 0x7fdf401a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:56.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.696+0000 7fdf37fff640 1 -- 172.21.15.22:0/399863055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf40199d50 msgr2=0x7fdf401a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.696+0000 7fdf37fff640 1 --2- 172.21.15.22:0/399863055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf40199d50 0x7fdf401a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.696+0000 7fdf37fff640 1 -- 172.21.15.22:0/399863055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf401008e0 msgr2=0x7fdf40199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.696+0000 7fdf37fff640 1 --2- 172.21.15.22:0/399863055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf401008e0 0x7fdf40199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.696+0000 7fdf37fff640 1 -- 172.21.15.22:0/399863055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf401a4ad0 con 0x7fdf40105580 2024-09-19T08:25:56.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.696+0000 7fdf37fff640 1 --2- 172.21.15.22:0/399863055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf40105580 0x7fdf401997c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800c970 tx=0x7fdf2800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:56.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.696+0000 7fdf3cd99640 1 -- 172.21.15.22:0/399863055 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28014070 con 0x7fdf40105580 2024-09-19T08:25:56.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.696+0000 7fdf4524c640 1 -- 172.21.15.22:0/399863055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf401a4dc0 con 0x7fdf40105580 2024-09-19T08:25:56.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.696+0000 7fdf3cd99640 1 -- 172.21.15.22:0/399863055 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf28004440 con 0x7fdf40105580 2024-09-19T08:25:56.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.696+0000 7fdf3cd99640 1 -- 172.21.15.22:0/399863055 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2800f450 con 0x7fdf40105580 2024-09-19T08:25:56.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.696+0000 7fdf4524c640 1 -- 172.21.15.22:0/399863055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf401a5290 con 0x7fdf40105580 2024-09-19T08:25:56.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.697+0000 7fdf4524c640 1 -- 172.21.15.22:0/399863055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf40100cc0 con 0x7fdf40105580 2024-09-19T08:25:56.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.698+0000 7fdf3cd99640 1 -- 172.21.15.22:0/399863055 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdf28017070 con 0x7fdf40105580 2024-09-19T08:25:56.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.699+0000 7fdf3cd99640 1 --2- 172.21.15.22:0/399863055 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdf2c0a33c0 0x7fdf2c0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:56.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.699+0000 7fdf3cd99640 1 -- 172.21.15.22:0/399863055 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fdf280cff70 con 0x7fdf40105580 2024-09-19T08:25:56.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.699+0000 7fdf3ed9d640 1 --2- 172.21.15.22:0/399863055 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdf2c0a33c0 0x7fdf2c0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:56.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.700+0000 7fdf3ed9d640 1 --2- 172.21.15.22:0/399863055 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdf2c0a33c0 0x7fdf2c0a5880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009230 tx=0x7fdf300058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:56.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.702+0000 7fdf3cd99640 1 -- 172.21.15.22:0/399863055 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdf2801e020 con 0x7fdf40105580 2024-09-19T08:25:56.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.879+0000 7fdf4524c640 1 -- 172.21.15.22:0/399863055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdf40101290 con 0x7fdf40105580 2024-09-19T08:25:56.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.879+0000 7fdf3cd99640 1 -- 172.21.15.22:0/399863055 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fdf2809c940 con 0x7fdf40105580 2024-09-19T08:25:56.881 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-09-19T08:25:56.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.881+0000 7fdf4524c640 1 -- 172.21.15.22:0/399863055 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdf2c0a33c0 msgr2=0x7fdf2c0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.881+0000 7fdf4524c640 1 --2- 172.21.15.22:0/399863055 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdf2c0a33c0 0x7fdf2c0a5880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009230 tx=0x7fdf300058d0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.881+0000 7fdf4524c640 1 -- 172.21.15.22:0/399863055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf40105580 msgr2=0x7fdf401997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:56.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.881+0000 7fdf4524c640 1 --2- 172.21.15.22:0/399863055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf40105580 0x7fdf401997c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800c970 tx=0x7fdf2800ce30 comp rx=0 tx=0).stop 2024-09-19T08:25:56.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.881+0000 7fdf4524c640 1 -- 172.21.15.22:0/399863055 shutdown_connections 2024-09-19T08:25:56.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.881+0000 7fdf4524c640 1 --2- 172.21.15.22:0/399863055 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fdf2c0a33c0 0x7fdf2c0a5880 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.881+0000 7fdf4524c640 1 --2- 172.21.15.22:0/399863055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf40199d50 0x7fdf401a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.881+0000 7fdf4524c640 1 --2- 172.21.15.22:0/399863055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf40105580 0x7fdf401997c0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.881+0000 7fdf4524c640 1 --2- 172.21.15.22:0/399863055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf401008e0 0x7fdf40199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:56.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.881+0000 7fdf4524c640 1 -- 172.21.15.22:0/399863055 >> 172.21.15.22:0/399863055 conn(0x7fdf40075ea0 msgr2=0x7fdf400fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:56.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.881+0000 7fdf4524c640 1 -- 172.21.15.22:0/399863055 shutdown_connections 2024-09-19T08:25:56.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.881+0000 7fdf4524c640 1 -- 172.21.15.22:0/399863055 wait complete. 2024-09-19T08:25:56.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T08:25:57.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.996+0000 7fe442726640 1 -- 172.21.15.22:0/1148409904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe43c101ab0 msgr2=0x7fe43c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:57.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.996+0000 7fe442726640 1 --2- 172.21.15.22:0/1148409904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe43c101ab0 0x7fe43c101f10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe430009930 tx=0x7fe43002a420 comp rx=0 tx=0).stop 2024-09-19T08:25:57.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.996+0000 7fe442726640 1 -- 172.21.15.22:0/1148409904 shutdown_connections 2024-09-19T08:25:57.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.997+0000 7fe442726640 1 --2- 172.21.15.22:0/1148409904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c102450 0x7fe43c10e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:57.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.997+0000 7fe442726640 1 --2- 172.21.15.22:0/1148409904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe43c101ab0 0x7fe43c101f10 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:57.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.997+0000 7fe442726640 1 --2- 172.21.15.22:0/1148409904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe43c107ac0 0x7fe43c107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:57.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.997+0000 7fe442726640 1 -- 172.21.15.22:0/1148409904 >> 172.21.15.22:0/1148409904 conn(0x7fe43c0fd7d0 msgr2=0x7fe43c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:57.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.997+0000 7fe442726640 1 -- 172.21.15.22:0/1148409904 shutdown_connections 2024-09-19T08:25:57.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.997+0000 7fe442726640 1 -- 172.21.15.22:0/1148409904 wait complete. 2024-09-19T08:25:57.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.997+0000 7fe442726640 1 Processor -- start 2024-09-19T08:25:57.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.997+0000 7fe442726640 1 -- start start 2024-09-19T08:25:57.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.997+0000 7fe442726640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c101ab0 0x7fe43c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:57.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.997+0000 7fe442726640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe43c102450 0x7fe43c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:57.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe442726640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe43c107ac0 0x7fe43c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:57.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe442726640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c111f00 con 0x7fe43c102450 2024-09-19T08:25:57.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe442726640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c111d80 con 0x7fe43c107ac0 2024-09-19T08:25:57.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe442726640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c112080 con 0x7fe43c101ab0 2024-09-19T08:25:57.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe43b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe43c102450 0x7fe43c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:57.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe43b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe43c102450 0x7fe43c198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47806/0 (socket says 172.21.15.22:47806) 2024-09-19T08:25:57.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe43b7fe640 1 -- 172.21.15.22:0/2126556953 learned_addr learned my addr 172.21.15.22:0/2126556953 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:25:57.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe440cc3640 1 --2- 172.21.15.22:0/2126556953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe43c107ac0 0x7fe43c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:57.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe43bfff640 1 --2- 172.21.15.22:0/2126556953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c101ab0 0x7fe43c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:57.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe43b7fe640 1 -- 172.21.15.22:0/2126556953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c101ab0 msgr2=0x7fe43c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:57.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe43b7fe640 1 --2- 172.21.15.22:0/2126556953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c101ab0 0x7fe43c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:57.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe43b7fe640 1 -- 172.21.15.22:0/2126556953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe43c107ac0 msgr2=0x7fe43c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe43b7fe640 1 --2- 172.21.15.22:0/2126556953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe43c107ac0 0x7fe43c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe43b7fe640 1 -- 172.21.15.22:0/2126556953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c19d3b0 con 0x7fe43c102450 2024-09-19T08:25:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe43b7fe640 1 --2- 172.21.15.22:0/2126556953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe43c102450 0x7fe43c198900 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe430009230 tx=0x7fe430004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe4397fa640 1 -- 172.21.15.22:0/2126556953 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43003f030 con 0x7fe43c102450 2024-09-19T08:25:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.998+0000 7fe442726640 1 -- 172.21.15.22:0/2126556953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43c19d640 con 0x7fe43c102450 2024-09-19T08:25:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.999+0000 7fe442726640 1 -- 172.21.15.22:0/2126556953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43c1a4f30 con 0x7fe43c102450 2024-09-19T08:25:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.999+0000 7fe4397fa640 1 -- 172.21.15.22:0/2126556953 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe430003ad0 con 0x7fe43c102450 2024-09-19T08:25:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.999+0000 7fe4397fa640 1 -- 172.21.15.22:0/2126556953 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430049060 con 0x7fe43c102450 2024-09-19T08:25:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:56.999+0000 7fe442726640 1 -- 172.21.15.22:0/2126556953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe428005290 con 0x7fe43c102450 2024-09-19T08:25:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.001+0000 7fe4397fa640 1 -- 172.21.15.22:0/2126556953 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe430042030 con 0x7fe43c102450 2024-09-19T08:25:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.001+0000 7fe4397fa640 1 --2- 172.21.15.22:0/2126556953 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe4240a33c0 0x7fe4240a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:25:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.002+0000 7fe4397fa640 1 -- 172.21.15.22:0/2126556953 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fe4300f00d0 con 0x7fe43c102450 2024-09-19T08:25:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.002+0000 7fe43bfff640 1 --2- 172.21.15.22:0/2126556953 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe4240a33c0 0x7fe4240a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:25:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.002+0000 7fe43bfff640 1 --2- 172.21.15.22:0/2126556953 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe4240a33c0 0x7fe4240a5880 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe42c005fd0 tx=0x7fe42c005850 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:25:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.005+0000 7fe4397fa640 1 -- 172.21.15.22:0/2126556953 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe4300bcb40 con 0x7fe43c102450 2024-09-19T08:25:57.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.229+0000 7fe442726640 1 -- 172.21.15.22:0/2126556953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fe428005850 con 0x7fe43c102450 2024-09-19T08:25:57.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.238+0000 7fe4397fa640 1 -- 172.21.15.22:0/2126556953 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+742870 (secure 0 0 0) 0x7fe4300bc440 con 0x7fe43c102450 2024-09-19T08:25:57.248 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:25:57.248 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:25:57.248 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 26, 2024-09-19T08:25:57.248 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.248 INFO:teuthology.orchestra.run.smithi022.stdout: "active_gid": 15141, 2024-09-19T08:25:57.249 INFO:teuthology.orchestra.run.smithi022.stdout: "active_name": "x", 2024-09-19T08:25:57.249 INFO:teuthology.orchestra.run.smithi022.stdout: "active_addrs": { 2024-09-19T08:25:57.249 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-19T08:25:57.249 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.249 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-09-19T08:25:57.249 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6800", 2024-09-19T08:25:57.249 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 497251935 2024-09-19T08:25:57.249 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.249 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.249 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-19T08:25:57.249 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6801", 2024-09-19T08:25:57.249 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 497251935 2024-09-19T08:25:57.249 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.249 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-19T08:25:57.250 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.250 INFO:teuthology.orchestra.run.smithi022.stdout: "active_addr": "172.21.15.22:6801/497251935", 2024-09-19T08:25:57.250 INFO:teuthology.orchestra.run.smithi022.stdout: "active_change": "2024-09-19T08:15:01.165788+0000", 2024-09-19T08:25:57.250 INFO:teuthology.orchestra.run.smithi022.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T08:25:57.250 INFO:teuthology.orchestra.run.smithi022.stdout: "available": true, 2024-09-19T08:25:57.250 INFO:teuthology.orchestra.run.smithi022.stdout: "standbys": [ 2024-09-19T08:25:57.250 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.250 INFO:teuthology.orchestra.run.smithi022.stdout: "gid": 24817, 2024-09-19T08:25:57.250 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "z", 2024-09-19T08:25:57.250 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_features": 4540701547738038271, 2024-09-19T08:25:57.250 INFO:teuthology.orchestra.run.smithi022.stdout: "available_modules": [ 2024-09-19T08:25:57.250 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.250 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "alerts", 2024-09-19T08:25:57.250 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.251 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.251 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.251 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:57.251 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:57.251 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.251 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.251 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.251 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:57.251 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.251 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.251 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.251 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T08:25:57.251 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.252 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.252 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.252 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.252 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.252 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.252 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.252 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.252 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.252 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.252 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.252 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.252 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.252 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.253 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.253 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.253 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.253 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.253 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.253 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.253 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.253 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.253 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.253 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.253 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.253 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.253 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.253 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.254 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.254 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.254 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.254 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.254 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.254 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.254 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.254 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.254 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.254 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.254 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.254 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.254 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.255 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.255 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.255 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.255 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.255 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.255 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.255 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.255 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.255 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.255 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.255 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.255 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.255 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.255 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.256 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.256 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.256 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.256 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.256 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.256 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.256 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.256 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.256 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.256 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.257 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.257 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.257 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.257 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.257 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.257 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.257 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.257 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.257 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_destination": { 2024-09-19T08:25:57.257 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_destination", 2024-09-19T08:25:57.257 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.257 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.257 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.257 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.258 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.258 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.258 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.258 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email address to send alerts to", 2024-09-19T08:25:57.258 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.258 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.258 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.258 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.258 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_from_name": { 2024-09-19T08:25:57.258 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_from_name", 2024-09-19T08:25:57.258 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.258 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.258 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.259 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "Ceph", 2024-09-19T08:25:57.259 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.259 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.259 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.259 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email From: name", 2024-09-19T08:25:57.259 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.259 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.259 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.259 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.259 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_host": { 2024-09-19T08:25:57.259 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_host", 2024-09-19T08:25:57.259 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.259 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.259 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.260 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.260 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.260 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.260 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.260 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP server", 2024-09-19T08:25:57.260 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.260 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.260 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.260 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.260 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_password": { 2024-09-19T08:25:57.260 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_password", 2024-09-19T08:25:57.260 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.260 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.261 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.261 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.261 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.261 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.261 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.261 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Password to authenticate with", 2024-09-19T08:25:57.261 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.261 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.261 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.261 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.261 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_port": { 2024-09-19T08:25:57.261 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_port", 2024-09-19T08:25:57.261 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.261 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.262 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.262 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "465", 2024-09-19T08:25:57.262 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.262 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.262 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.262 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP port", 2024-09-19T08:25:57.262 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.262 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.262 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.262 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.262 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_sender": { 2024-09-19T08:25:57.262 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_sender", 2024-09-19T08:25:57.262 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.262 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.263 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.263 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.263 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.263 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.263 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.263 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP envelope sender", 2024-09-19T08:25:57.263 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.263 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.263 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.263 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_ssl": { 2024-09-19T08:25:57.264 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_ssl", 2024-09-19T08:25:57.264 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.264 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.264 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.264 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.264 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.264 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.264 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.264 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T08:25:57.264 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.264 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.264 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.264 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.264 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_user": { 2024-09-19T08:25:57.265 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_user", 2024-09-19T08:25:57.265 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.265 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.265 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.265 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.265 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.265 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.265 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.265 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "User to authenticate as", 2024-09-19T08:25:57.265 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.265 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.265 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.265 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.266 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.266 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.266 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.266 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "balancer", 2024-09-19T08:25:57.266 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.266 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.266 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.266 INFO:teuthology.orchestra.run.smithi022.stdout: "active": { 2024-09-19T08:25:57.266 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "active", 2024-09-19T08:25:57.266 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.266 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.266 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.266 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.266 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.267 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.267 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.267 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T08:25:57.267 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.267 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.267 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.267 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.267 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_time": { 2024-09-19T08:25:57.267 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_time", 2024-09-19T08:25:57.267 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.267 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.267 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.267 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0000", 2024-09-19T08:25:57.267 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.268 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.268 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.268 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T08:25:57.268 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:25:57.268 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.268 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.268 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.268 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_weekday": { 2024-09-19T08:25:57.268 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_weekday", 2024-09-19T08:25:57.268 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:57.268 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.268 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.269 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:57.269 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-09-19T08:25:57.269 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-09-19T08:25:57.269 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.269 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T08:25:57.269 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:25:57.269 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.269 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.269 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.269 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_max_iterations": { 2024-09-19T08:25:57.269 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_max_iterations", 2024-09-19T08:25:57.269 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:57.269 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.269 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.270 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "25", 2024-09-19T08:25:57.270 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-09-19T08:25:57.270 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "250", 2024-09-19T08:25:57.270 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.270 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T08:25:57.270 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.270 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.270 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.270 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.270 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_metrics": { 2024-09-19T08:25:57.270 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_metrics", 2024-09-19T08:25:57.270 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.270 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.270 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.271 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T08:25:57.271 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.271 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.271 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.271 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T08:25:57.271 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T08:25:57.271 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.271 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.271 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.271 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_step": { 2024-09-19T08:25:57.271 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_step", 2024-09-19T08:25:57.271 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:57.271 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.271 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.272 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.5", 2024-09-19T08:25:57.272 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0.001", 2024-09-19T08:25:57.272 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "0.999", 2024-09-19T08:25:57.272 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.272 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "aggressiveness of optimization", 2024-09-19T08:25:57.272 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T08:25:57.272 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.272 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.272 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.272 INFO:teuthology.orchestra.run.smithi022.stdout: "end_time": { 2024-09-19T08:25:57.272 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_time", 2024-09-19T08:25:57.272 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.272 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.273 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.273 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2359", 2024-09-19T08:25:57.273 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.273 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.273 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.273 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T08:25:57.273 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:25:57.273 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.273 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.273 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.273 INFO:teuthology.orchestra.run.smithi022.stdout: "end_weekday": { 2024-09-19T08:25:57.273 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_weekday", 2024-09-19T08:25:57.273 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:57.273 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.274 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.274 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:57.274 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-09-19T08:25:57.274 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-09-19T08:25:57.274 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.274 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T08:25:57.274 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:25:57.274 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.274 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.274 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.274 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.274 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.274 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.274 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.275 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.275 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.275 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.275 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.275 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.275 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.275 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.275 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.275 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.275 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.275 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.275 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.275 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.275 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.276 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.276 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.276 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.276 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.276 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.276 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.276 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.276 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.276 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.276 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.276 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.277 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.277 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.277 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.277 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.277 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.277 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.277 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.277 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.277 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.277 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.277 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.277 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.277 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.278 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.278 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.278 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.278 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.278 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.278 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.278 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.278 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.278 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.278 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.278 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.278 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.278 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.278 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.279 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.279 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.279 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.279 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.279 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.279 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.279 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.279 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.279 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.279 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.279 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.279 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.279 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.280 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.280 INFO:teuthology.orchestra.run.smithi022.stdout: "min_score": { 2024-09-19T08:25:57.280 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_score", 2024-09-19T08:25:57.280 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:57.280 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.280 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.280 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:57.280 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.280 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.280 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.280 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T08:25:57.280 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.280 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.281 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.281 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.281 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-09-19T08:25:57.281 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-09-19T08:25:57.281 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.281 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.281 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.281 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "upmap", 2024-09-19T08:25:57.281 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.281 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.281 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.281 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-compat", 2024-09-19T08:25:57.281 INFO:teuthology.orchestra.run.smithi022.stdout: "none", 2024-09-19T08:25:57.281 INFO:teuthology.orchestra.run.smithi022.stdout: "read", 2024-09-19T08:25:57.282 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap", 2024-09-19T08:25:57.282 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap-read" 2024-09-19T08:25:57.282 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.282 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Balancer mode", 2024-09-19T08:25:57.282 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.282 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.282 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.282 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.282 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_ids": { 2024-09-19T08:25:57.282 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_ids", 2024-09-19T08:25:57.282 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.282 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.282 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.283 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.283 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.283 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.283 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.283 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T08:25:57.283 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.283 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.283 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.283 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.283 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:57.283 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:57.283 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.283 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.284 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.284 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:57.284 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.284 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.284 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.284 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T08:25:57.284 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.284 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.284 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.284 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.284 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_deviation": { 2024-09-19T08:25:57.284 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_deviation", 2024-09-19T08:25:57.284 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.285 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.285 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.285 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:57.285 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-09-19T08:25:57.285 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.285 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.285 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T08:25:57.285 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T08:25:57.285 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.285 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.285 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.285 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_optimizations": { 2024-09-19T08:25:57.285 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_optimizations", 2024-09-19T08:25:57.285 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:57.286 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.286 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.286 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.286 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.286 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.286 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.286 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T08:25:57.286 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.286 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.286 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.286 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.286 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.286 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.287 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.287 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm", 2024-09-19T08:25:57.287 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.287 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.287 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.287 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_down_multiplier": { 2024-09-19T08:25:57.287 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_down_multiplier", 2024-09-19T08:25:57.287 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:57.287 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.287 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.287 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-09-19T08:25:57.287 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.287 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.287 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.288 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T08:25:57.288 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.288 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.288 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.288 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.288 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_refresh_rate": { 2024-09-19T08:25:57.288 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_refresh_rate", 2024-09-19T08:25:57.288 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.288 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.288 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.288 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "20", 2024-09-19T08:25:57.288 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.288 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.289 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.289 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T08:25:57.289 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.289 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.289 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.289 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_starting_port": { 2024-09-19T08:25:57.289 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_starting_port", 2024-09-19T08:25:57.289 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.289 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.289 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.289 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4721", 2024-09-19T08:25:57.289 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.289 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.290 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.290 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T08:25:57.290 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.290 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.290 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.290 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.290 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_ptrace": { 2024-09-19T08:25:57.290 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_ptrace", 2024-09-19T08:25:57.290 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.290 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.290 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.290 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.291 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.291 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.291 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.291 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T08:25:57.291 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T08:25:57.291 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.291 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.291 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.291 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_interval": { 2024-09-19T08:25:57.291 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_interval", 2024-09-19T08:25:57.291 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.291 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.291 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.292 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:57.292 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.292 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.292 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.292 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T08:25:57.292 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.292 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.292 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.292 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.292 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_memory_target_ratio": { 2024-09-19T08:25:57.292 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T08:25:57.292 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:57.292 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.293 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.293 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.7", 2024-09-19T08:25:57.293 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.293 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.293 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.293 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T08:25:57.293 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.293 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.293 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.293 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.293 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm_log_destination": { 2024-09-19T08:25:57.293 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm_log_destination", 2024-09-19T08:25:57.293 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.293 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.294 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.294 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.294 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.294 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.294 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.294 INFO:teuthology.orchestra.run.smithi022.stdout: "file", 2024-09-19T08:25:57.294 INFO:teuthology.orchestra.run.smithi022.stdout: "file,syslog", 2024-09-19T08:25:57.294 INFO:teuthology.orchestra.run.smithi022.stdout: "syslog" 2024-09-19T08:25:57.294 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.294 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T08:25:57.294 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.294 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.294 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.294 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.295 INFO:teuthology.orchestra.run.smithi022.stdout: "cgroups_split": { 2024-09-19T08:25:57.295 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cgroups_split", 2024-09-19T08:25:57.295 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.295 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.295 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.295 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.295 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.295 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.295 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.295 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T08:25:57.295 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.295 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.295 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.296 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.296 INFO:teuthology.orchestra.run.smithi022.stdout: "config_checks_enabled": { 2024-09-19T08:25:57.296 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_checks_enabled", 2024-09-19T08:25:57.296 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.296 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.296 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.296 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.296 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.296 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.296 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.296 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T08:25:57.296 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.296 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.297 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.297 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.297 INFO:teuthology.orchestra.run.smithi022.stdout: "config_dashboard": { 2024-09-19T08:25:57.297 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_dashboard", 2024-09-19T08:25:57.297 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.297 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.297 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.297 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.297 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.297 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.297 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.297 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T08:25:57.297 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.297 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.298 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.298 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.298 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_alertmanager": { 2024-09-19T08:25:57.298 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_alertmanager", 2024-09-19T08:25:57.298 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.298 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.298 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.298 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T08:25:57.298 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.298 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.298 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.298 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-09-19T08:25:57.298 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.298 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.299 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.299 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.299 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_base": { 2024-09-19T08:25:57.299 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_base", 2024-09-19T08:25:57.299 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.299 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.299 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.299 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T08:25:57.299 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.299 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.299 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.299 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Container image name, without the tag", 2024-09-19T08:25:57.299 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.299 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.300 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.300 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.300 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_elasticsearch": { 2024-09-19T08:25:57.300 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_elasticsearch", 2024-09-19T08:25:57.300 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.300 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.300 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.300 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T08:25:57.300 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.300 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.300 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.300 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "elasticsearch container image", 2024-09-19T08:25:57.300 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.301 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.301 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.301 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.301 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_grafana": { 2024-09-19T08:25:57.301 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_grafana", 2024-09-19T08:25:57.301 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.301 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.301 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.301 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T08:25:57.301 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.301 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.301 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.301 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-09-19T08:25:57.302 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.302 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.302 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.302 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.302 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_haproxy": { 2024-09-19T08:25:57.302 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_haproxy", 2024-09-19T08:25:57.302 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.302 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.302 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.302 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T08:25:57.302 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.302 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.303 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.303 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "HAproxy container image", 2024-09-19T08:25:57.303 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.303 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.303 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.303 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.303 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_agent": { 2024-09-19T08:25:57.303 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_agent", 2024-09-19T08:25:57.303 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.303 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.303 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.303 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T08:25:57.303 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.303 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.304 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.304 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger agent container image", 2024-09-19T08:25:57.304 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.304 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.304 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.304 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.304 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_collector": { 2024-09-19T08:25:57.304 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_collector", 2024-09-19T08:25:57.304 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.304 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.304 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.304 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T08:25:57.304 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.305 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.305 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.305 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger collector container image", 2024-09-19T08:25:57.305 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.305 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.305 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.305 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.305 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_query": { 2024-09-19T08:25:57.305 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_query", 2024-09-19T08:25:57.305 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.305 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.305 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.305 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T08:25:57.306 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.306 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.306 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.306 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger query container image", 2024-09-19T08:25:57.306 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.306 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.306 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.306 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.306 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_keepalived": { 2024-09-19T08:25:57.306 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_keepalived", 2024-09-19T08:25:57.306 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.306 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.306 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.306 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T08:25:57.307 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.307 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.307 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.307 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Keepalived container image", 2024-09-19T08:25:57.307 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.307 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.307 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.307 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_loki": { 2024-09-19T08:25:57.307 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_loki", 2024-09-19T08:25:57.307 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.307 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.307 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.307 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T08:25:57.308 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.308 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.308 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.308 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Loki container image", 2024-09-19T08:25:57.308 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.308 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.308 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.308 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.308 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_node_exporter": { 2024-09-19T08:25:57.308 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_node_exporter", 2024-09-19T08:25:57.308 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.308 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.308 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.309 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T08:25:57.309 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.309 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.309 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.309 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-09-19T08:25:57.309 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.309 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.309 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.309 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.309 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_nvmeof": { 2024-09-19T08:25:57.309 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_nvmeof", 2024-09-19T08:25:57.309 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.309 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.310 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.310 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T08:25:57.310 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.310 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.310 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.310 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Nvme-of container image", 2024-09-19T08:25:57.310 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.310 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.310 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.310 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.310 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_prometheus": { 2024-09-19T08:25:57.310 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_prometheus", 2024-09-19T08:25:57.310 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.310 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.311 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.311 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T08:25:57.311 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.311 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.311 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.311 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-09-19T08:25:57.311 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.311 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.311 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.311 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.311 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_promtail": { 2024-09-19T08:25:57.311 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_promtail", 2024-09-19T08:25:57.311 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.311 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.312 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.312 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T08:25:57.312 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.312 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.312 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.312 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Promtail container image", 2024-09-19T08:25:57.312 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.312 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.312 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.312 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.312 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_samba": { 2024-09-19T08:25:57.312 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_samba", 2024-09-19T08:25:57.313 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.313 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.313 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.313 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T08:25:57.313 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.313 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.313 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.313 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Samba/SMB container image", 2024-09-19T08:25:57.313 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.313 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.313 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.313 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.313 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_snmp_gateway": { 2024-09-19T08:25:57.313 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_snmp_gateway", 2024-09-19T08:25:57.314 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.314 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.314 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.314 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T08:25:57.316 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.316 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.316 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.316 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SNMP Gateway container image", 2024-09-19T08:25:57.316 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.316 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.316 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.316 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.316 INFO:teuthology.orchestra.run.smithi022.stdout: "container_init": { 2024-09-19T08:25:57.316 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_init", 2024-09-19T08:25:57.316 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.316 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.317 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.317 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.317 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.317 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.317 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.317 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T08:25:57.317 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.317 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.317 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.317 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.317 INFO:teuthology.orchestra.run.smithi022.stdout: "daemon_cache_timeout": { 2024-09-19T08:25:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "daemon_cache_timeout", 2024-09-19T08:25:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T08:25:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.318 INFO:teuthology.orchestra.run.smithi022.stdout: "default_cephadm_command_timeout": { 2024-09-19T08:25:57.319 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T08:25:57.319 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.319 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.319 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.319 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "900", 2024-09-19T08:25:57.319 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.319 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.319 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.319 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T08:25:57.319 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.319 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.319 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.319 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.320 INFO:teuthology.orchestra.run.smithi022.stdout: "default_registry": { 2024-09-19T08:25:57.320 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_registry", 2024-09-19T08:25:57.320 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.320 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.320 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.320 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io", 2024-09-19T08:25:57.320 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.320 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.320 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.320 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T08:25:57.320 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.320 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.320 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.320 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.321 INFO:teuthology.orchestra.run.smithi022.stdout: "device_cache_timeout": { 2024-09-19T08:25:57.321 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_cache_timeout", 2024-09-19T08:25:57.321 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.321 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.321 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.321 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1800", 2024-09-19T08:25:57.321 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.321 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.321 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.321 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache device inventory", 2024-09-19T08:25:57.321 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.321 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.321 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.322 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.322 INFO:teuthology.orchestra.run.smithi022.stdout: "device_enhanced_scan": { 2024-09-19T08:25:57.322 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_enhanced_scan", 2024-09-19T08:25:57.322 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.322 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.322 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.322 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.322 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.322 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.322 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.322 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T08:25:57.322 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.322 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.322 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.323 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.323 INFO:teuthology.orchestra.run.smithi022.stdout: "facts_cache_timeout": { 2024-09-19T08:25:57.323 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "facts_cache_timeout", 2024-09-19T08:25:57.323 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.323 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.323 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.323 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:57.323 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.323 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.323 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.323 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache host facts data", 2024-09-19T08:25:57.323 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.323 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.323 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.324 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.324 INFO:teuthology.orchestra.run.smithi022.stdout: "grafana_dashboards_path": { 2024-09-19T08:25:57.324 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "grafana_dashboards_path", 2024-09-19T08:25:57.324 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.324 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.324 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.324 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T08:25:57.324 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.324 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.324 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.324 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T08:25:57.324 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.324 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.325 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.325 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.325 INFO:teuthology.orchestra.run.smithi022.stdout: "host_check_interval": { 2024-09-19T08:25:57.325 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "host_check_interval", 2024-09-19T08:25:57.325 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.325 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.325 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.325 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:57.325 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.325 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.325 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.325 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to perform a host check", 2024-09-19T08:25:57.325 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.325 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.326 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.326 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.326 INFO:teuthology.orchestra.run.smithi022.stdout: "hw_monitoring": { 2024-09-19T08:25:57.326 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hw_monitoring", 2024-09-19T08:25:57.326 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.326 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.326 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.326 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.326 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.326 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.326 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.326 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T08:25:57.326 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.326 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.327 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.327 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.327 INFO:teuthology.orchestra.run.smithi022.stdout: "inventory_list_all": { 2024-09-19T08:25:57.327 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "inventory_list_all", 2024-09-19T08:25:57.327 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.327 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.327 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.327 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.327 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.327 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.327 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.327 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T08:25:57.327 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.327 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.328 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.328 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.328 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.328 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.328 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.328 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.328 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.328 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.328 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.328 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.328 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.328 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.328 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.328 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.329 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.329 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.329 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.329 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.329 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.329 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.329 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.329 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.329 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.329 INFO:teuthology.orchestra.run.smithi022.stdout: "log_refresh_metadata": { 2024-09-19T08:25:57.329 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_refresh_metadata", 2024-09-19T08:25:57.329 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.329 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.330 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.330 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.330 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.330 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.330 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.330 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T08:25:57.330 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.330 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.330 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.330 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.330 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.330 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.331 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.331 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.331 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.331 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.331 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.331 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.331 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.331 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T08:25:57.331 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.331 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.331 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.331 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.331 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.332 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.332 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.332 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.332 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.332 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.332 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.332 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.332 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.332 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.332 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.332 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.332 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.332 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.333 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.333 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.333 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.333 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.333 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.333 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.333 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.333 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.333 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.333 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.333 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.333 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.333 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.334 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.334 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.334 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.334 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.334 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.334 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.334 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.334 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.334 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T08:25:57.335 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T08:25:57.335 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.335 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.335 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.335 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.335 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.335 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.335 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.335 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T08:25:57.335 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.335 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.335 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.335 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.336 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T08:25:57.336 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T08:25:57.336 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.336 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.336 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.336 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "*", 2024-09-19T08:25:57.336 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.336 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.336 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.336 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T08:25:57.336 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.336 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.336 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.336 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.337 INFO:teuthology.orchestra.run.smithi022.stdout: "max_count_per_host": { 2024-09-19T08:25:57.337 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_count_per_host", 2024-09-19T08:25:57.337 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.337 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.337 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.337 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.337 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.337 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.337 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.337 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of daemons per service per host", 2024-09-19T08:25:57.337 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.337 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.337 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.337 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.338 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd_draining_count": { 2024-09-19T08:25:57.338 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_osd_draining_count", 2024-09-19T08:25:57.338 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.338 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.338 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.338 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.338 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.338 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.338 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.338 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T08:25:57.338 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.338 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.338 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.338 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.339 INFO:teuthology.orchestra.run.smithi022.stdout: "migration_current": { 2024-09-19T08:25:57.339 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "migration_current", 2024-09-19T08:25:57.339 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.339 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.339 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.339 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.339 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.339 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.339 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.339 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "internal - do not modify", 2024-09-19T08:25:57.339 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.339 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.339 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.339 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.340 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-09-19T08:25:57.340 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-09-19T08:25:57.340 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.340 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.340 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.340 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "root", 2024-09-19T08:25:57.340 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.340 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.340 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.340 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm-package", 2024-09-19T08:25:57.340 INFO:teuthology.orchestra.run.smithi022.stdout: "root" 2024-09-19T08:25:57.340 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.340 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T08:25:57.341 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.341 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.341 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.341 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.341 INFO:teuthology.orchestra.run.smithi022.stdout: "oob_default_addr": { 2024-09-19T08:25:57.341 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "oob_default_addr", 2024-09-19T08:25:57.341 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.341 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.341 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.341 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "169.254.1.1", 2024-09-19T08:25:57.341 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.341 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.341 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.341 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T08:25:57.342 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.342 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.342 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.342 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.342 INFO:teuthology.orchestra.run.smithi022.stdout: "prometheus_alerts_path": { 2024-09-19T08:25:57.342 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus_alerts_path", 2024-09-19T08:25:57.342 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.342 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.342 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.342 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T08:25:57.342 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.342 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.342 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.343 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T08:25:57.343 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.343 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.343 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.343 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.343 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_insecure": { 2024-09-19T08:25:57.343 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_insecure", 2024-09-19T08:25:57.343 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.343 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.343 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.343 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.344 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.344 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.344 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.344 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T08:25:57.344 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.344 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.344 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.344 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.344 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_password": { 2024-09-19T08:25:57.344 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_password", 2024-09-19T08:25:57.344 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.344 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.344 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.345 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.345 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.345 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.345 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.345 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T08:25:57.345 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.345 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.345 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.345 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.345 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_url": { 2024-09-19T08:25:57.345 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_url", 2024-09-19T08:25:57.345 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.345 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.346 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.346 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.346 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.346 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.346 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.346 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T08:25:57.346 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.346 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.346 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.346 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.346 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_username": { 2024-09-19T08:25:57.346 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_username", 2024-09-19T08:25:57.346 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.347 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.347 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.347 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.347 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.347 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.347 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.347 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T08:25:57.347 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.347 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.347 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.347 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.347 INFO:teuthology.orchestra.run.smithi022.stdout: "secure_monitoring_stack": { 2024-09-19T08:25:57.347 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secure_monitoring_stack", 2024-09-19T08:25:57.347 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.348 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.348 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.348 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.348 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.348 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.348 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.348 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T08:25:57.348 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.348 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.348 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.348 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.348 INFO:teuthology.orchestra.run.smithi022.stdout: "service_discovery_port": { 2024-09-19T08:25:57.348 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "service_discovery_port", 2024-09-19T08:25:57.349 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.349 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.349 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.349 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8765", 2024-09-19T08:25:57.349 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.349 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.349 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.349 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "cephadm service discovery port", 2024-09-19T08:25:57.349 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.349 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.349 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.349 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.349 INFO:teuthology.orchestra.run.smithi022.stdout: "ssh_config_file": { 2024-09-19T08:25:57.349 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssh_config_file", 2024-09-19T08:25:57.350 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.350 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.350 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.350 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.350 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.350 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.350 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.350 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T08:25:57.350 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.350 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.350 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.350 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.350 INFO:teuthology.orchestra.run.smithi022.stdout: "use_agent": { 2024-09-19T08:25:57.350 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_agent", 2024-09-19T08:25:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T08:25:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: "use_repo_digest": { 2024-09-19T08:25:57.351 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_repo_digest", 2024-09-19T08:25:57.352 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.352 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.352 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.352 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.352 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.352 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.352 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.352 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T08:25:57.352 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.352 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.352 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.352 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.352 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_failed_host_check": { 2024-09-19T08:25:57.352 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_failed_host_check", 2024-09-19T08:25:57.353 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.353 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.353 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.353 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.353 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.353 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.353 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.353 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T08:25:57.353 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.353 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.353 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.353 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.353 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_daemons": { 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_daemons", 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_hosts": { 2024-09-19T08:25:57.354 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_hosts", 2024-09-19T08:25:57.355 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.355 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.355 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.355 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.355 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.355 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.355 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.355 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T08:25:57.355 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.355 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.355 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.355 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.355 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.356 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.356 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.356 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crash", 2024-09-19T08:25:57.356 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.356 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.356 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.356 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.356 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.356 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.356 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.356 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.356 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.357 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.357 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.357 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.357 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.357 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.357 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.357 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.357 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.357 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.357 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.357 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.357 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.357 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.358 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.358 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.358 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.358 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.358 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.358 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.358 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.358 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.358 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.358 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.358 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.358 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.358 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.358 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.359 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.359 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.359 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.359 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.359 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.359 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.359 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.359 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.359 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.359 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.359 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.359 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.359 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.359 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.360 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.360 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.360 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.360 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.360 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.360 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.360 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.360 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.360 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.360 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.360 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.360 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.360 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.361 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.361 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.361 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.361 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.361 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.361 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.361 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.361 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.361 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.361 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.361 INFO:teuthology.orchestra.run.smithi022.stdout: "retain_interval": { 2024-09-19T08:25:57.361 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retain_interval", 2024-09-19T08:25:57.361 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.362 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.362 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.362 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "31536000", 2024-09-19T08:25:57.362 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.362 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.362 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.362 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T08:25:57.362 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.362 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.362 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.362 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.362 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_recent_interval": { 2024-09-19T08:25:57.362 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_recent_interval", 2024-09-19T08:25:57.363 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.363 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.363 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.363 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1209600", 2024-09-19T08:25:57.363 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.363 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.363 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.363 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T08:25:57.363 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.363 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.363 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.363 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.363 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.363 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.364 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.364 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dashboard", 2024-09-19T08:25:57.364 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.364 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.364 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.364 INFO:teuthology.orchestra.run.smithi022.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T08:25:57.364 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T08:25:57.364 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.364 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.364 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.364 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.364 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.364 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.365 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.365 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.365 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.365 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.365 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.365 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.365 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T08:25:57.365 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T08:25:57.365 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.365 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.365 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.365 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.365 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.365 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T08:25:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T08:25:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.366 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_ENABLED": { 2024-09-19T08:25:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T08:25:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.367 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.368 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.368 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.368 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.368 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.368 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.368 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.368 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T08:25:57.368 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T08:25:57.368 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.368 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.368 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.368 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.368 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.369 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.369 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.369 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.369 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.369 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.369 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.369 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.369 INFO:teuthology.orchestra.run.smithi022.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T08:25:57.369 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T08:25:57.369 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.369 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.369 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.369 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.370 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.370 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.370 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.370 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.370 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.370 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.370 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.370 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.370 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T08:25:57.370 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T08:25:57.370 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.370 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.371 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.371 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.371 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.371 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.371 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.371 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.371 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.371 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.371 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.371 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.371 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T08:25:57.371 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T08:25:57.371 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.371 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.372 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.372 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.372 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.372 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.372 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.372 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.372 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.372 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.372 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.372 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.372 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T08:25:57.372 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T08:25:57.372 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.373 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.373 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.373 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.373 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.373 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.373 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.373 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.373 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.373 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.373 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.373 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.373 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T08:25:57.373 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T08:25:57.373 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.374 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.374 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.374 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.374 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.374 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.374 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.374 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.374 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.374 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.374 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.374 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.374 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T08:25:57.374 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T08:25:57.374 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.375 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.375 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.375 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.375 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.375 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.375 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.375 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.375 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.375 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.375 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.375 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.375 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T08:25:57.375 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T08:25:57.376 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.376 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.376 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.376 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.376 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.376 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.376 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.376 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.376 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.376 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.376 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.376 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.376 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T08:25:57.376 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T08:25:57.377 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.377 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.377 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.377 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.377 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.377 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.377 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.377 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.377 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.377 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.377 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.377 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.377 INFO:teuthology.orchestra.run.smithi022.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T08:25:57.377 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T08:25:57.378 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.378 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.378 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.378 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.378 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.378 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.378 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.378 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.378 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.378 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.378 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.378 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.378 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T08:25:57.378 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T08:25:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-09-19T08:25:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T08:25:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T08:25:57.380 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.380 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.380 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.380 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.380 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.380 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.380 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.380 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.380 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.380 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.380 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.380 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.380 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_URL": { 2024-09-19T08:25:57.381 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_URL", 2024-09-19T08:25:57.381 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.381 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.381 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.381 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.381 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.381 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.381 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.381 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.381 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.381 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.381 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.381 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.381 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T08:25:57.382 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T08:25:57.382 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.382 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.382 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.382 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-09-19T08:25:57.382 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.382 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.382 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.382 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.382 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.382 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.382 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.382 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.382 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T08:25:57.383 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T08:25:57.383 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.383 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.383 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.383 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.383 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.383 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.383 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.383 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.383 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.383 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.383 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.383 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.383 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T08:25:57.384 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T08:25:57.384 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.384 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.384 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.384 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.384 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.384 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.384 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.384 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.384 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.384 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.385 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.385 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.385 INFO:teuthology.orchestra.run.smithi022.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T08:25:57.385 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T08:25:57.385 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.385 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.385 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.385 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.385 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.385 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.385 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.385 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.385 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.385 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.386 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.386 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.386 INFO:teuthology.orchestra.run.smithi022.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T08:25:57.386 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T08:25:57.386 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.386 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.386 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.386 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.386 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.386 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.386 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.386 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.386 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.387 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.387 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.387 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.387 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T08:25:57.387 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T08:25:57.387 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.387 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.387 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.387 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.387 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.387 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.387 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.387 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.387 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.388 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.388 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.388 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.388 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T08:25:57.388 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T08:25:57.388 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.388 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.388 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.388 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.388 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.388 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.388 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.388 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.389 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.389 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.389 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.389 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.389 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T08:25:57.389 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T08:25:57.389 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.389 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.389 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.389 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.389 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.389 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.389 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.389 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.390 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.390 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.390 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.390 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.390 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T08:25:57.390 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T08:25:57.390 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.390 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.390 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.390 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.390 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.390 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.390 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.391 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.391 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.391 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.391 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.391 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.391 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T08:25:57.391 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T08:25:57.391 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.391 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.391 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.391 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.391 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.391 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.391 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.392 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.392 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.392 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.392 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.392 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.392 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T08:25:57.392 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T08:25:57.392 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.392 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.392 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.392 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.392 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.392 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.392 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.393 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.393 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.393 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.393 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.393 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.393 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T08:25:57.393 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T08:25:57.393 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.393 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.393 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.393 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.393 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.393 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.393 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.394 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.394 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.394 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.394 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.394 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.394 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T08:25:57.394 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T08:25:57.394 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.394 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.394 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.394 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.394 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.394 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.394 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.395 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.395 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.395 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.395 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.395 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.395 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T08:25:57.395 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T08:25:57.395 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.395 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.395 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.395 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.395 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.395 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.395 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.396 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.396 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.396 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.396 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.396 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.396 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T08:25:57.396 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T08:25:57.396 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.396 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.396 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.396 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.396 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.396 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.396 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.397 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.397 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.397 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.397 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.397 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.397 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T08:25:57.397 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T08:25:57.397 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.397 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.397 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.397 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T08:25:57.397 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.398 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.398 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.398 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.398 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.398 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.398 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.398 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.398 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T08:25:57.398 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T08:25:57.398 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.398 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.398 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.398 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.398 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.399 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.399 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.399 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.399 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.399 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.399 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.399 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.399 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T08:25:57.399 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T08:25:57.399 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.399 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.399 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.399 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8", 2024-09-19T08:25:57.400 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.400 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.400 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.400 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.400 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.400 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.400 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.400 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.400 INFO:teuthology.orchestra.run.smithi022.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T08:25:57.400 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T08:25:57.400 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.400 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.400 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.400 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "45", 2024-09-19T08:25:57.401 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.401 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.401 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.401 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.401 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.401 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.401 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.401 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.401 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T08:25:57.401 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T08:25:57.401 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.401 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.401 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.402 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.402 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.402 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.402 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.402 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.402 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.402 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.402 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.402 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.402 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T08:25:57.402 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T08:25:57.402 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.402 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.402 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.403 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-09-19T08:25:57.403 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.403 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.403 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.403 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.403 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.403 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.403 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.403 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.403 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T08:25:57.403 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T08:25:57.403 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.403 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.404 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.404 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.404 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.404 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.404 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.404 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.404 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.404 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.404 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.404 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.404 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T08:25:57.404 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T08:25:57.404 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.404 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T08:25:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T08:25:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.406 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.406 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.406 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:57.406 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.406 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.406 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.406 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.406 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.406 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.406 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.406 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.406 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T08:25:57.406 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T08:25:57.406 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.407 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.408 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.408 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:57.408 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.408 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.408 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.408 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.408 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.408 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.408 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.408 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.408 INFO:teuthology.orchestra.run.smithi022.stdout: "cross_origin_url": { 2024-09-19T08:25:57.408 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cross_origin_url", 2024-09-19T08:25:57.408 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: "crt_file": { 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crt_file", 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.409 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.410 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.410 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.410 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.410 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.410 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.410 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.410 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.410 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.410 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.410 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.410 INFO:teuthology.orchestra.run.smithi022.stdout: "debug": { 2024-09-19T08:25:57.410 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "debug", 2024-09-19T08:25:57.411 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.411 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.411 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.411 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.411 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.411 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.411 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.411 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable/disable debug options", 2024-09-19T08:25:57.411 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.411 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.411 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.411 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.411 INFO:teuthology.orchestra.run.smithi022.stdout: "jwt_token_ttl": { 2024-09-19T08:25:57.411 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "jwt_token_ttl", 2024-09-19T08:25:57.412 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.412 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.412 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.412 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "28800", 2024-09-19T08:25:57.412 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.412 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.412 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.412 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.412 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.412 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.412 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.412 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.413 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-09-19T08:25:57.413 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-09-19T08:25:57.413 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.413 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.413 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.413 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.413 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.413 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.413 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.413 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.413 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.413 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.414 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.414 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.414 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.414 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.414 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.414 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.414 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.414 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.414 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.414 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.414 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.414 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.414 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.414 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.415 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.415 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.415 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.415 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.415 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.415 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.415 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.415 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.415 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.415 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.415 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.415 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.415 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.416 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.416 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.416 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.416 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.416 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.416 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.416 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.416 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.416 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.416 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.416 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.416 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.416 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.417 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.417 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.417 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.417 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.417 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.417 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.417 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.417 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.417 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.417 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.417 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.417 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.417 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.418 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.418 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.418 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.418 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.418 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.418 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.418 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.418 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.418 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.418 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.418 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.418 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.418 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.418 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.419 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.419 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.419 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.419 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.419 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.419 INFO:teuthology.orchestra.run.smithi022.stdout: "motd": { 2024-09-19T08:25:57.419 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "motd", 2024-09-19T08:25:57.419 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.419 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.419 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.419 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.419 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.419 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.419 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.420 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "The message of the day", 2024-09-19T08:25:57.420 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.420 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.420 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.420 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.420 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect_resolve_ip_addr": { 2024-09-19T08:25:57.420 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T08:25:57.420 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.420 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.420 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.420 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.420 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.420 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.421 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.421 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.421 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.421 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.421 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.421 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.421 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-09-19T08:25:57.421 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-09-19T08:25:57.421 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.421 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.421 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.421 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-09-19T08:25:57.421 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.421 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.422 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.422 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.422 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.422 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.422 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.422 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.422 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-09-19T08:25:57.422 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-09-19T08:25:57.422 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.422 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.422 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.422 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8080", 2024-09-19T08:25:57.422 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.422 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.423 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.423 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.423 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.423 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.423 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.423 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.423 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-09-19T08:25:57.423 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-09-19T08:25:57.423 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.423 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.423 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.423 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.423 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.424 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.424 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.424 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.424 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.424 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.424 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.424 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.424 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl_server_port": { 2024-09-19T08:25:57.424 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl_server_port", 2024-09-19T08:25:57.424 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.424 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.424 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.424 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8443", 2024-09-19T08:25:57.424 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.425 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.425 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.425 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.425 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.425 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.425 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.425 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.425 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-09-19T08:25:57.425 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-09-19T08:25:57.425 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.425 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.426 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.426 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "redirect", 2024-09-19T08:25:57.426 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.426 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.426 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.426 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.426 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect" 2024-09-19T08:25:57.426 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.426 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.426 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.426 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.426 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.426 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.426 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-09-19T08:25:57.427 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-09-19T08:25:57.427 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.427 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.427 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.427 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-09-19T08:25:57.427 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-09-19T08:25:57.427 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-09-19T08:25:57.427 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.427 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.427 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.427 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.427 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.427 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.427 INFO:teuthology.orchestra.run.smithi022.stdout: "url_prefix": { 2024-09-19T08:25:57.428 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url_prefix", 2024-09-19T08:25:57.428 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.428 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.428 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.428 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.428 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.428 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.428 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.428 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.428 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.428 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.428 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.428 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.429 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.429 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.429 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.429 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2024-09-19T08:25:57.429 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.429 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.429 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.429 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_monitoring": { 2024-09-19T08:25:57.429 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_monitoring", 2024-09-19T08:25:57.429 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.429 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.429 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.429 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.430 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.430 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.430 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.430 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "monitor device health metrics", 2024-09-19T08:25:57.430 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.430 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.430 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.430 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.430 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.430 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.430 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.430 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.430 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.430 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.431 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.431 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.431 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.431 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.431 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.431 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.431 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.431 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.431 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.431 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.431 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.431 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.431 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.431 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.432 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.432 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.432 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.432 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.432 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.432 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.432 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.432 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.432 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.432 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.432 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.432 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.432 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.432 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.433 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.433 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.433 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.433 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.433 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.433 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.433 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.433 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.433 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.433 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.433 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.433 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.433 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.434 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.434 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.434 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.434 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.434 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.434 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.434 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.434 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.434 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.434 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.434 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.434 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.434 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.434 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.435 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.435 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.435 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.435 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.435 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.435 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.435 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.435 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.435 INFO:teuthology.orchestra.run.smithi022.stdout: "mark_out_threshold": { 2024-09-19T08:25:57.435 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mark_out_threshold", 2024-09-19T08:25:57.435 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.435 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.435 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.436 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2419200", 2024-09-19T08:25:57.436 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.436 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.436 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.436 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T08:25:57.436 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.436 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.436 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.436 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.436 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": { 2024-09-19T08:25:57.436 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_name", 2024-09-19T08:25:57.436 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.436 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.436 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.437 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "device_health_metrics", 2024-09-19T08:25:57.437 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.437 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.437 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.437 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T08:25:57.437 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.437 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.437 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.437 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.437 INFO:teuthology.orchestra.run.smithi022.stdout: "retention_period": { 2024-09-19T08:25:57.437 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retention_period", 2024-09-19T08:25:57.437 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.437 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.437 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.438 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15552000", 2024-09-19T08:25:57.438 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.438 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.438 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.438 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain device health metrics", 2024-09-19T08:25:57.438 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.438 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.438 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.438 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.438 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_frequency": { 2024-09-19T08:25:57.438 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_frequency", 2024-09-19T08:25:57.438 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.438 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.438 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.439 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-09-19T08:25:57.439 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.439 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.439 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.439 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T08:25:57.439 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.439 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.439 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.439 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.439 INFO:teuthology.orchestra.run.smithi022.stdout: "self_heal": { 2024-09-19T08:25:57.439 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "self_heal", 2024-09-19T08:25:57.439 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.440 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.440 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.440 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.440 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.440 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.440 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.440 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T08:25:57.440 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.440 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.440 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.440 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.440 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:57.440 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:57.441 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.441 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.441 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.441 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:57.441 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.441 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.441 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.441 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T08:25:57.441 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.441 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.441 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.441 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.441 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_threshold": { 2024-09-19T08:25:57.441 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_threshold", 2024-09-19T08:25:57.442 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.442 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.442 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.442 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7257600", 2024-09-19T08:25:57.442 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.442 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.442 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.442 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T08:25:57.442 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.442 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.442 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.442 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.442 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.443 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.443 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.443 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "diskprediction_local", 2024-09-19T08:25:57.443 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.443 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.443 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.443 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.443 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.443 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.443 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.443 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.443 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.443 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.443 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.444 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.444 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.444 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.444 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.444 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.444 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.444 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.444 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.444 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.444 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.444 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.444 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.444 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.445 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.445 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.445 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.445 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.445 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.445 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.445 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.445 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.445 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.445 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.445 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.445 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.445 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.446 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.446 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.446 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.446 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.446 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.446 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.446 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.446 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.446 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.446 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.446 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.446 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.446 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.446 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.447 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.448 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.448 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.448 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.448 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.448 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.448 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.448 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.448 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.448 INFO:teuthology.orchestra.run.smithi022.stdout: "predict_interval": { 2024-09-19T08:25:57.448 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predict_interval", 2024-09-19T08:25:57.448 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.448 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.448 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.448 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-09-19T08:25:57.449 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.449 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.449 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.449 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.449 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.449 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.449 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.449 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.449 INFO:teuthology.orchestra.run.smithi022.stdout: "predictor_model": { 2024-09-19T08:25:57.449 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predictor_model", 2024-09-19T08:25:57.449 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.449 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.449 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.449 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "prophetstor", 2024-09-19T08:25:57.450 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.450 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.450 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.450 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.450 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.450 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.450 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.450 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.450 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:57.450 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:57.450 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.450 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.450 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.450 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:57.451 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.451 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.451 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.451 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.451 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.451 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.451 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.451 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.451 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.451 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.451 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.451 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "influx", 2024-09-19T08:25:57.451 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": false, 2024-09-19T08:25:57.451 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "influxdb python module not found", 2024-09-19T08:25:57.452 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.452 INFO:teuthology.orchestra.run.smithi022.stdout: "batch_size": { 2024-09-19T08:25:57.452 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "batch_size", 2024-09-19T08:25:57.452 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.452 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.452 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.452 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5000", 2024-09-19T08:25:57.452 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.452 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.452 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.452 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T08:25:57.452 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.452 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.452 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.453 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.453 INFO:teuthology.orchestra.run.smithi022.stdout: "database": { 2024-09-19T08:25:57.453 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "database", 2024-09-19T08:25:57.453 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.453 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.453 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.453 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "ceph", 2024-09-19T08:25:57.453 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.453 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.453 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.453 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T08:25:57.453 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.454 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.454 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.454 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.454 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": { 2024-09-19T08:25:57.454 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hostname", 2024-09-19T08:25:57.454 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.454 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.454 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.454 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.454 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.454 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.454 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.454 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server hostname", 2024-09-19T08:25:57.454 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.455 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.455 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.455 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.455 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:57.455 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:57.455 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.455 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.455 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.455 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "30", 2024-09-19T08:25:57.455 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "5", 2024-09-19T08:25:57.455 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.455 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.455 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T08:25:57.455 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.456 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.456 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.456 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.456 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.456 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.456 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.456 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.456 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.456 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.456 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.456 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.456 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.456 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.457 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.457 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.457 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.457 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.457 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.457 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.457 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.457 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.457 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.457 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.457 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.457 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.457 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.457 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.458 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.458 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.458 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.458 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.458 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.458 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.458 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.458 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.458 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.458 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.458 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.458 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.458 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.458 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.459 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.459 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.459 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.459 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.459 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.459 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.459 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.459 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.459 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.459 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.459 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.459 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.459 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.459 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.461 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.461 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.461 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.461 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.461 INFO:teuthology.orchestra.run.smithi022.stdout: "password": { 2024-09-19T08:25:57.461 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "password", 2024-09-19T08:25:57.461 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.461 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.461 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.461 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.461 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.461 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.461 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.461 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "password of InfluxDB server user", 2024-09-19T08:25:57.462 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.462 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.462 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.462 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.462 INFO:teuthology.orchestra.run.smithi022.stdout: "port": { 2024-09-19T08:25:57.462 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "port", 2024-09-19T08:25:57.462 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.462 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.462 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.462 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8086", 2024-09-19T08:25:57.462 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.462 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.462 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.462 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server port", 2024-09-19T08:25:57.463 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.463 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.463 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.463 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.463 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-09-19T08:25:57.463 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-09-19T08:25:57.463 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.463 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.463 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.463 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "false", 2024-09-19T08:25:57.463 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.463 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.463 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.463 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:25:57.464 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.464 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.464 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.464 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.464 INFO:teuthology.orchestra.run.smithi022.stdout: "threads": { 2024-09-19T08:25:57.464 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threads", 2024-09-19T08:25:57.464 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.464 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.464 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.464 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:57.464 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-09-19T08:25:57.464 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "32", 2024-09-19T08:25:57.464 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.464 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T08:25:57.465 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.465 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.465 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.465 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.465 INFO:teuthology.orchestra.run.smithi022.stdout: "username": { 2024-09-19T08:25:57.465 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "username", 2024-09-19T08:25:57.465 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.465 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.465 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.465 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.465 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.465 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.465 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.465 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "username of InfluxDB server user", 2024-09-19T08:25:57.466 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.466 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.466 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.466 INFO:teuthology.orchestra.run.smithi022.stdout: "verify_ssl": { 2024-09-19T08:25:57.466 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "verify_ssl", 2024-09-19T08:25:57.466 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.466 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.466 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.466 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "true", 2024-09-19T08:25:57.466 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.466 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.467 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.467 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:25:57.467 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.467 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.467 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.467 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.467 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.467 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.467 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.467 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "insights", 2024-09-19T08:25:57.467 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.467 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.467 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.467 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.468 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.468 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.468 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.468 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.468 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.468 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.468 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.468 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.468 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.468 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.468 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.468 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.468 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.469 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.469 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.469 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.469 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.469 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.469 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.469 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.469 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.469 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.469 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.469 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.469 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.469 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.469 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.470 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.470 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.470 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.470 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.470 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.470 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.470 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.470 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.470 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.470 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.470 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.470 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.470 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.470 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.471 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.471 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.471 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.471 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.471 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.471 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.471 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.471 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.471 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.471 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.471 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.471 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.471 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.471 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.472 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.472 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.472 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.472 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.472 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.472 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.472 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.472 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.472 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.472 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.472 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.472 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.472 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.473 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.473 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.473 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.473 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.473 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "iostat", 2024-09-19T08:25:57.473 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.473 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.473 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.473 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.473 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.473 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.473 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.473 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.474 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.474 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.474 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.474 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.474 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.474 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.474 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.474 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.474 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.474 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.474 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.474 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.474 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.474 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.475 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.475 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.475 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.475 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.475 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.475 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.475 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.475 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.475 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.475 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.475 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.475 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.475 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.475 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.476 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.476 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.476 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.476 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.476 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.476 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.476 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.476 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.476 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.476 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.476 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.476 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.476 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.476 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.477 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.477 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.477 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.477 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.477 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.477 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.477 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.477 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.477 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.477 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.477 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.477 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.477 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.477 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.478 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.478 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.478 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.478 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.478 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.478 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.478 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.478 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.478 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.478 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.478 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.478 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.478 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "k8sevents", 2024-09-19T08:25:57.478 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.479 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.479 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.479 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_event_retention_days": { 2024-09-19T08:25:57.479 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ceph_event_retention_days", 2024-09-19T08:25:57.479 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.479 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.479 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.479 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7", 2024-09-19T08:25:57.479 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.479 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.479 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.479 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T08:25:57.479 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.479 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.480 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.480 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.480 INFO:teuthology.orchestra.run.smithi022.stdout: "config_check_secs": { 2024-09-19T08:25:57.480 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_check_secs", 2024-09-19T08:25:57.480 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.480 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.480 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.480 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.480 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "10", 2024-09-19T08:25:57.480 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.480 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.480 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T08:25:57.481 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.481 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.481 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.481 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.481 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.481 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.481 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.481 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.481 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.481 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.481 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.481 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.481 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.482 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.482 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.482 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.482 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.482 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.482 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.482 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.482 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.482 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.482 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.482 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.482 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.483 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.483 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.483 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.483 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.483 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.483 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.483 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.483 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.483 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.483 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.483 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.483 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.483 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.483 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.484 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.484 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.484 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.484 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.484 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.484 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.484 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.484 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.484 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.484 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.484 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.484 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.484 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.485 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.485 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.485 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.485 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.485 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.485 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.485 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.485 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.485 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.485 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.485 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.485 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.485 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.485 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.486 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.486 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.486 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.486 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.486 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.486 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.486 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.486 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.486 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.486 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.486 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.486 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "localpool", 2024-09-19T08:25:57.486 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.487 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.487 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.487 INFO:teuthology.orchestra.run.smithi022.stdout: "failure_domain": { 2024-09-19T08:25:57.487 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "failure_domain", 2024-09-19T08:25:57.487 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.487 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.487 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.487 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "host", 2024-09-19T08:25:57.487 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.487 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.487 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.487 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "failure domain for any created local pool", 2024-09-19T08:25:57.487 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T08:25:57.487 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.488 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.488 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.488 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.488 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.488 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.488 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.488 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.488 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.488 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.488 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.488 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.488 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.488 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.488 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.489 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.489 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.489 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.489 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.489 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.489 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.489 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.489 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.489 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.489 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.489 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.489 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.489 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.489 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.490 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.490 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.490 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.490 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.490 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.490 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.490 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.490 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.490 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.490 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.490 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.490 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.490 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.490 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.491 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.491 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.491 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.491 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.491 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.491 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.491 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.491 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.491 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.491 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.491 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.491 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.491 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.491 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.492 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.492 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.492 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.492 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.492 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.492 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.492 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.492 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.492 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.492 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.492 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.492 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.492 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": { 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_size", 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default min_size for any created local pool", 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T08:25:57.493 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.494 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.494 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.494 INFO:teuthology.orchestra.run.smithi022.stdout: "num_rep": { 2024-09-19T08:25:57.494 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "num_rep", 2024-09-19T08:25:57.494 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.494 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.494 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.494 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3", 2024-09-19T08:25:57.494 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.494 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.494 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.494 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default replica count for any created local pool", 2024-09-19T08:25:57.495 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.495 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.495 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.495 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.495 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": { 2024-09-19T08:25:57.495 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_num", 2024-09-19T08:25:57.495 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.495 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.495 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.495 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "128", 2024-09-19T08:25:57.495 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.495 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.495 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.495 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T08:25:57.496 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.496 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.496 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.496 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.496 INFO:teuthology.orchestra.run.smithi022.stdout: "prefix": { 2024-09-19T08:25:57.496 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prefix", 2024-09-19T08:25:57.496 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.496 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.496 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.496 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.496 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.496 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.496 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.497 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name prefix for any created local pool", 2024-09-19T08:25:57.497 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.497 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.497 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.497 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.497 INFO:teuthology.orchestra.run.smithi022.stdout: "subtree": { 2024-09-19T08:25:57.497 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "subtree", 2024-09-19T08:25:57.497 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.497 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.497 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.497 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "rack", 2024-09-19T08:25:57.497 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.497 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.497 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.498 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T08:25:57.498 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T08:25:57.498 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.498 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.498 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.498 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.498 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.498 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.498 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mds_autoscaler", 2024-09-19T08:25:57.498 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.498 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.498 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.498 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.499 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.500 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.500 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.500 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.500 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.500 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.500 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.500 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.500 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.500 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.500 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.500 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.500 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.500 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.501 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.501 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.501 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.501 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.501 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.501 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.501 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.501 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.501 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.501 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.501 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.501 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.501 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.501 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.502 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.502 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.502 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.502 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.502 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.502 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.502 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.502 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.502 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.502 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.502 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.502 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.502 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.502 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.503 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.503 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.503 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.503 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.503 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.503 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.503 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.503 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.503 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.503 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.503 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.503 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.503 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.503 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.504 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.504 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.504 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirroring", 2024-09-19T08:25:57.504 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.504 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.504 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.504 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.504 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.504 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.504 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.504 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.504 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.504 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.504 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.505 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.506 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.507 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.508 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.509 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.509 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.509 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.509 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.509 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.509 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.509 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.509 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.509 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.509 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.509 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "nfs", 2024-09-19T08:25:57.509 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.509 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.509 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.510 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.510 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.510 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.510 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.510 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.510 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.510 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.510 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.510 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.510 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.510 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.510 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.510 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.510 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.511 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.511 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.511 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.511 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.511 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.511 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.511 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.511 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.511 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.511 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.511 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.511 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.511 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.512 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.512 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.512 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.512 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.512 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.512 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.512 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.512 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.512 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.512 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.512 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.512 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.512 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.512 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.513 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.513 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.513 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.513 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.513 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.513 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.513 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.513 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.513 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.513 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.513 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.513 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.513 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.514 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.514 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.514 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.514 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.514 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.514 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.514 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.514 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.514 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.514 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.514 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.514 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.514 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.514 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.515 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.515 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.515 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.515 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.515 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.515 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-09-19T08:25:57.515 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.515 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.515 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.515 INFO:teuthology.orchestra.run.smithi022.stdout: "fail_fs": { 2024-09-19T08:25:57.515 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "fail_fs", 2024-09-19T08:25:57.515 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.515 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.515 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.516 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.516 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.516 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.516 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.516 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T08:25:57.516 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.516 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.516 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.516 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.516 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.516 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.516 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.516 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.516 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.518 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.518 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.518 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.518 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.518 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.518 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.518 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.518 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.518 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.518 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.518 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.518 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.518 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.518 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.519 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.519 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.519 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.519 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.519 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.519 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.519 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.519 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.519 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.519 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.519 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.519 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.519 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.519 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.520 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.520 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.520 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.520 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.520 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.520 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.520 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.520 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.520 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.520 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.520 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.520 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.520 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.520 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.521 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.521 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.521 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.521 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.521 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.521 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.521 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.521 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.521 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.521 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator": { 2024-09-19T08:25:57.521 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-09-19T08:25:57.521 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.521 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.522 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.522 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.522 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.522 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.522 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.522 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2024-09-19T08:25:57.522 INFO:teuthology.orchestra.run.smithi022.stdout: "rook", 2024-09-19T08:25:57.522 INFO:teuthology.orchestra.run.smithi022.stdout: "test_orchestrator" 2024-09-19T08:25:57.522 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.522 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Orchestrator backend", 2024-09-19T08:25:57.522 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.522 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.522 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.522 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.523 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.523 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.523 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.523 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_perf_query", 2024-09-19T08:25:57.523 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.523 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.523 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.523 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.523 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.523 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.523 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.523 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.524 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.524 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.524 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.524 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.524 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.524 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.524 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.524 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.524 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.524 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.524 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.524 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.524 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.524 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.525 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.526 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.526 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.526 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.526 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.526 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.526 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.526 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.526 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.526 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.526 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.526 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.526 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.526 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.526 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.527 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.527 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.527 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.527 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.527 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.527 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.527 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.527 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.527 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.527 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.527 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.527 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.527 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.527 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.528 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.528 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.528 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.528 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.528 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.528 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.528 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.528 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.528 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.528 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.528 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.528 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.528 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_support", 2024-09-19T08:25:57.529 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.529 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.529 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.529 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.529 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.529 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.529 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.529 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.529 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.529 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.529 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.529 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.529 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.529 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.530 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.530 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.530 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.530 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.530 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.530 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.530 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.530 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.530 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.530 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.530 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.530 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.531 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.531 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.531 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.531 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.531 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.531 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.531 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.531 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.531 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.531 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.531 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.531 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.531 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.531 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.532 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.532 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.532 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.532 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.532 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.532 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.532 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.532 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.532 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.532 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.532 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.532 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.532 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.532 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.533 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.533 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.533 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.533 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.533 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.533 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.533 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.533 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.533 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.533 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.533 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.533 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.533 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.533 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_autoscaler", 2024-09-19T08:25:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.535 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.535 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.535 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.535 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.535 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.535 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.535 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.535 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.535 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.535 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.535 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.535 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.535 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.535 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.536 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.536 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.536 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.536 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.536 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.536 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.536 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.536 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.536 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.536 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.536 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.536 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.536 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.536 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.537 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.537 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.537 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.537 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.537 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.537 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.537 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.537 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.537 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.537 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.537 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.537 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.538 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.538 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.538 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.538 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.538 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.538 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.538 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.538 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.538 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.538 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.538 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.538 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.538 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.539 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.539 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.539 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.539 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.539 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.539 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.539 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.539 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.539 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.539 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.539 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.539 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.539 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.539 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.540 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.540 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:57.540 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:57.540 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.540 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.540 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.540 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:57.540 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.540 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.540 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.540 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.540 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.540 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.540 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.541 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.541 INFO:teuthology.orchestra.run.smithi022.stdout: "threshold": { 2024-09-19T08:25:57.541 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threshold", 2024-09-19T08:25:57.541 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:57.541 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.541 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.541 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-09-19T08:25:57.541 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1.0", 2024-09-19T08:25:57.541 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.541 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.541 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "scaling threshold", 2024-09-19T08:25:57.541 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T08:25:57.541 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.542 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.542 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.542 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.542 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.542 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.542 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "progress", 2024-09-19T08:25:57.542 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.542 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.542 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.542 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_pg_recovery_event": { 2024-09-19T08:25:57.542 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_pg_recovery_event", 2024-09-19T08:25:57.542 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.542 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.542 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.543 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.543 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.543 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.543 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.543 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T08:25:57.543 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.543 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.543 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.543 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.543 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-09-19T08:25:57.543 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-09-19T08:25:57.543 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.543 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.544 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.544 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.544 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.544 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.544 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.544 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.544 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.544 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.544 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.544 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.544 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.544 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.544 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.544 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.545 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.545 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.545 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.545 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.545 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.545 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.545 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.545 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.545 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.545 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.545 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.545 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.545 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.546 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.546 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.546 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.546 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.546 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.546 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.546 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.546 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.546 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.546 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.546 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.546 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.546 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.546 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.547 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.547 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.547 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.547 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.547 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.547 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.547 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.547 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.547 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.547 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.547 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.547 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.547 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.547 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.548 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.548 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.548 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.548 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.548 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.548 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.548 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.548 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.548 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.548 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.548 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.548 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.548 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.548 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.549 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.549 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.549 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.549 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.549 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.549 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.549 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.549 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.549 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.549 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.549 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.549 INFO:teuthology.orchestra.run.smithi022.stdout: "max_completed_events": { 2024-09-19T08:25:57.549 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_completed_events", 2024-09-19T08:25:57.550 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.550 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.550 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.550 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "50", 2024-09-19T08:25:57.550 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.550 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.550 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.550 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "number of past completed events to remember", 2024-09-19T08:25:57.550 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.550 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.550 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.550 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.550 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:57.550 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long the module is going to sleep", 2024-09-19T08:25:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus", 2024-09-19T08:25:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "cache": { 2024-09-19T08:25:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cache", 2024-09-19T08:25:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.553 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.553 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.553 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.553 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.553 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.553 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.553 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.553 INFO:teuthology.orchestra.run.smithi022.stdout: "exclude_perf_counters": { 2024-09-19T08:25:57.553 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "exclude_perf_counters", 2024-09-19T08:25:57.553 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.553 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.553 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.553 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.554 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.554 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.554 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.554 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T08:25:57.554 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T08:25:57.554 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.554 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.554 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.554 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.554 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.554 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.554 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.554 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.554 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.555 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.555 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.555 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.555 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.555 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.555 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.555 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.555 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.555 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.555 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.555 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.555 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.555 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.555 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.556 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.556 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.556 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.556 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.556 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.556 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.556 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.556 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.556 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.556 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.556 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.556 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.557 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.557 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.557 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.557 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.557 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.557 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.557 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.557 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.557 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.557 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.557 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.557 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.557 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.557 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.558 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.558 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.558 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.558 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.558 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.558 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.558 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.558 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.558 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.558 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.558 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.558 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.558 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools": { 2024-09-19T08:25:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools", 2024-09-19T08:25:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.560 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.560 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.560 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.560 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.560 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.560 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.560 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.560 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.560 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.560 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.560 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.560 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T08:25:57.560 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T08:25:57.560 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.561 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.561 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.561 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300", 2024-09-19T08:25:57.561 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.561 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.561 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.561 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.561 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.561 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.561 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.561 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.561 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_interval": { 2024-09-19T08:25:57.561 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_interval", 2024-09-19T08:25:57.561 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:57.562 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.562 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.562 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15.0", 2024-09-19T08:25:57.562 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.562 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.562 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.562 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.562 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.562 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.562 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.562 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.562 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-09-19T08:25:57.562 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-09-19T08:25:57.562 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.563 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.563 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.563 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-09-19T08:25:57.563 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.563 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.563 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.563 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T08:25:57.563 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.563 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.563 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.563 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.563 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-09-19T08:25:57.563 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-09-19T08:25:57.563 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.564 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.564 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.564 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "9283", 2024-09-19T08:25:57.564 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.564 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.564 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.564 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T08:25:57.564 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.564 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.564 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.564 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.564 INFO:teuthology.orchestra.run.smithi022.stdout: "stale_cache_strategy": { 2024-09-19T08:25:57.564 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stale_cache_strategy", 2024-09-19T08:25:57.565 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.565 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.565 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.565 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "log", 2024-09-19T08:25:57.565 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.565 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.565 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.565 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.565 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.565 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.565 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.565 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.566 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-09-19T08:25:57.566 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-09-19T08:25:57.566 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.566 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.566 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.566 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "default", 2024-09-19T08:25:57.566 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.566 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.566 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.566 INFO:teuthology.orchestra.run.smithi022.stdout: "default", 2024-09-19T08:25:57.566 INFO:teuthology.orchestra.run.smithi022.stdout: "error" 2024-09-19T08:25:57.566 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.566 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.566 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.567 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.567 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.567 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.567 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-09-19T08:25:57.567 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-09-19T08:25:57.567 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.567 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.567 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.567 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-09-19T08:25:57.567 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-09-19T08:25:57.567 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-09-19T08:25:57.567 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.567 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.567 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.568 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.568 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.568 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.568 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.568 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.568 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.568 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2024-09-19T08:25:57.568 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.568 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.568 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.568 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.568 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.568 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.569 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.569 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.569 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.569 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.569 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.569 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.569 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.569 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.569 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.569 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.569 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.569 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.569 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.569 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.570 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.570 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.570 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.570 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.570 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.570 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.570 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.570 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.570 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.570 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.570 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.570 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.570 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.570 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.571 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.571 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.571 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.571 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.571 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.571 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.571 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.571 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.571 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.571 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.571 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.571 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.571 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.572 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.572 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.572 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.572 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.572 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.572 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.572 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.572 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.572 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.572 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.572 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.572 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.572 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.572 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.573 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.573 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.573 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.573 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.573 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.573 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.573 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.573 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.573 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.573 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.573 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.573 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_snap_create": { 2024-09-19T08:25:57.573 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_snap_create", 2024-09-19T08:25:57.573 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: "mirror_snapshot_schedule": { 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.574 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.575 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.575 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.575 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.575 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.575 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.575 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.575 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.575 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.575 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.575 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.575 INFO:teuthology.orchestra.run.smithi022.stdout: "trash_purge_schedule": { 2024-09-19T08:25:57.575 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "trash_purge_schedule", 2024-09-19T08:25:57.575 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.575 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.576 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.576 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.576 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.576 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.576 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.576 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.576 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.576 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.576 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.576 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.576 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.576 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "restful", 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_auth": { 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_auth", 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.577 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.578 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.578 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.578 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.578 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-09-19T08:25:57.578 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-09-19T08:25:57.578 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.578 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.578 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.578 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.578 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.578 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.578 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.578 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.578 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.579 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.579 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.579 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.579 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.579 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.579 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.579 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.579 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.580 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.580 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.580 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.580 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.580 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.580 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.580 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.580 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.580 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.580 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.580 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.580 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.580 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.581 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.581 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.581 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.581 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.581 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.581 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.581 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.581 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.581 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.581 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.581 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.581 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.581 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.581 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.582 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.582 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.582 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.582 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.582 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.582 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.582 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.582 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.582 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.582 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.582 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.582 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.582 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.583 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.583 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.583 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.583 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.583 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.583 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.583 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.583 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.583 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.583 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.583 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.583 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.583 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.583 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.584 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.584 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.584 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.584 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.584 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.584 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.584 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.584 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.584 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.584 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-09-19T08:25:57.584 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-09-19T08:25:57.584 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.584 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.585 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.585 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.585 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.585 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.585 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.585 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.585 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.585 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.585 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.585 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.585 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-09-19T08:25:57.585 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-09-19T08:25:57.585 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.585 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.586 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.586 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.586 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.586 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.586 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.586 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.586 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.586 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.586 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.586 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.586 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.586 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.586 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.586 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rgw", 2024-09-19T08:25:57.587 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.587 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.587 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.587 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.587 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.587 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.587 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.587 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.587 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.587 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.587 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.587 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.587 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.587 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.588 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.588 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.588 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.588 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.588 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.588 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.588 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.588 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.588 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.588 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.588 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.588 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.589 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.589 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.589 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.589 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.589 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.589 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.589 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.589 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.589 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.589 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.589 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.589 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.589 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.590 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.590 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.590 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.590 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.590 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.590 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.590 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.590 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.590 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.590 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.590 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.590 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.590 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.590 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.591 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.592 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.592 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.592 INFO:teuthology.orchestra.run.smithi022.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T08:25:57.592 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T08:25:57.592 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.592 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.592 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.592 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:57.592 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.592 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.592 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.592 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T08:25:57.592 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.593 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.593 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.593 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.593 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.593 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.593 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rook", 2024-09-19T08:25:57.593 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.593 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.593 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.593 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.593 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.594 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.594 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.594 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.594 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.594 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.594 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.594 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.594 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.594 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.594 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.594 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.594 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.594 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.595 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.595 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.595 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.595 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.595 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.595 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.595 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.595 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.595 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.595 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.595 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.595 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.595 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.596 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.596 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.596 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.596 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.596 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.596 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.596 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.596 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.596 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.596 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.596 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.596 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.596 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.597 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.597 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.597 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.597 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.597 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.597 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.597 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.597 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.597 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.597 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.597 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.597 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.597 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.598 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.598 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.598 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.598 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.598 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.598 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.598 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.598 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.598 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.598 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.598 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.598 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.598 INFO:teuthology.orchestra.run.smithi022.stdout: "storage_class": { 2024-09-19T08:25:57.599 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "storage_class", 2024-09-19T08:25:57.599 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.599 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.599 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.599 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "local", 2024-09-19T08:25:57.599 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.599 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.599 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.599 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T08:25:57.599 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.599 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.599 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.599 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.599 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.600 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "selftest", 2024-09-19T08:25:57.600 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.600 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.600 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.600 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.600 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.600 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.600 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.600 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.600 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.600 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.601 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.602 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.602 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.602 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.602 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.602 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.602 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.602 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.602 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.602 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.602 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.602 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.602 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.602 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.603 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.603 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.603 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.603 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.603 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.603 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.603 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.603 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.603 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.603 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.603 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.603 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.603 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.603 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.604 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.604 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.604 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.604 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.604 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.604 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.604 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.604 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.604 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.604 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.604 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.604 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.605 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.605 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.605 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.605 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.605 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.605 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.605 INFO:teuthology.orchestra.run.smithi022.stdout: "roption1": { 2024-09-19T08:25:57.605 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption1", 2024-09-19T08:25:57.605 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.605 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.605 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.605 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.605 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: "roption2": { 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption2", 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "xyz", 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.606 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.607 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.607 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.607 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.607 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.607 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.607 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption1": { 2024-09-19T08:25:57.607 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption1", 2024-09-19T08:25:57.607 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.607 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.607 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.607 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.608 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.608 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.608 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.608 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.608 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.608 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.608 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.608 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption2": { 2024-09-19T08:25:57.608 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption2", 2024-09-19T08:25:57.608 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.608 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.608 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.608 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.609 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.609 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.609 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.609 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.609 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.609 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.609 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.609 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption3": { 2024-09-19T08:25:57.609 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption3", 2024-09-19T08:25:57.609 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:57.609 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.609 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.610 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.610 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.610 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.610 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.610 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.610 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.610 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.610 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.610 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption4": { 2024-09-19T08:25:57.610 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption4", 2024-09-19T08:25:57.610 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.610 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.610 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption5": { 2024-09-19T08:25:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption5", 2024-09-19T08:25:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.612 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.612 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.612 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.612 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.612 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.612 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.612 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.612 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.612 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption6": { 2024-09-19T08:25:57.612 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption6", 2024-09-19T08:25:57.612 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.612 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.613 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.613 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.613 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.613 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.613 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.613 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.613 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.613 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.613 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.613 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption7": { 2024-09-19T08:25:57.613 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption7", 2024-09-19T08:25:57.613 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.613 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-09-19T08:25:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "42", 2024-09-19T08:25:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: "testkey": { 2024-09-19T08:25:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testkey", 2024-09-19T08:25:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.615 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.615 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.615 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.615 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.615 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.615 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.615 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.615 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.615 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.615 INFO:teuthology.orchestra.run.smithi022.stdout: "testlkey": { 2024-09-19T08:25:57.615 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testlkey", 2024-09-19T08:25:57.615 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.615 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.616 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.616 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.616 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.616 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.616 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.616 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.616 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.616 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.616 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.616 INFO:teuthology.orchestra.run.smithi022.stdout: "testnewline": { 2024-09-19T08:25:57.616 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testnewline", 2024-09-19T08:25:57.616 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.617 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.617 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.617 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.617 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.617 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.617 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.617 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.617 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.617 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.617 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.617 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.617 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.617 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_m_granularity": { 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_m_granularity", 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T08:25:57.618 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.619 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.619 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.619 INFO:teuthology.orchestra.run.smithi022.stdout: "dump_on_update": { 2024-09-19T08:25:57.619 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dump_on_update", 2024-09-19T08:25:57.619 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.619 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.619 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.619 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.619 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.619 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.619 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.619 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "dump database to debug log on update", 2024-09-19T08:25:57.619 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.620 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.620 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.620 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.620 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.620 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.620 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.620 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.620 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.620 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.620 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.620 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.620 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.621 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.621 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.621 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.621 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.621 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.621 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.621 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.621 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.621 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.621 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.621 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.621 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.622 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.622 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.622 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.622 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.622 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.622 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.622 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.622 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.622 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.622 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.622 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.622 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.622 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.623 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.623 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.623 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.623 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.623 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.623 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.623 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.623 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.623 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.623 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.623 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.623 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.623 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.624 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.624 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.624 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.624 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.624 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.624 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.624 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.624 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.624 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.624 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.624 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.624 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.624 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.625 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.625 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.625 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.625 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.625 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.625 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.625 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.625 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stats", 2024-09-19T08:25:57.625 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.625 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.625 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.625 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.626 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.626 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.626 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.626 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.626 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.626 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.626 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.626 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.626 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.626 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.626 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.626 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.626 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.626 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.627 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.627 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.627 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.627 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.627 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.627 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.627 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.627 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.627 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.627 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.627 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.627 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.627 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.628 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.628 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.628 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.628 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.628 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.628 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.628 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.628 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.628 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.628 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.628 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.628 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.629 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.629 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.629 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.629 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.629 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.629 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.629 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.629 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.629 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.629 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.629 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.629 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.629 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.630 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.630 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.630 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.630 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.630 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.630 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.630 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.630 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.630 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.630 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.630 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.630 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.630 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.630 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.631 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.631 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.631 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "status", 2024-09-19T08:25:57.631 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.631 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.631 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.631 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.631 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.631 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.631 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.631 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.631 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.631 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.632 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.632 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.632 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.632 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.632 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.632 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.632 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.632 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.632 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.632 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.632 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.632 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.632 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.634 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.634 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.634 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.634 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.634 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.634 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.634 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.634 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.634 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.634 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.634 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.634 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.634 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.634 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.636 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.636 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.636 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.636 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.636 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.636 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.636 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.636 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.636 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telegraf", 2024-09-19T08:25:57.636 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.636 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.637 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.637 INFO:teuthology.orchestra.run.smithi022.stdout: "address": { 2024-09-19T08:25:57.637 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "address", 2024-09-19T08:25:57.637 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.637 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.637 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.637 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T08:25:57.637 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.637 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.637 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.637 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.637 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.637 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.638 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.638 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:57.638 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:57.638 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.638 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.638 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.638 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15", 2024-09-19T08:25:57.638 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.638 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.638 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.638 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.638 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.638 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.639 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.639 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.639 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.639 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.639 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.639 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.639 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.639 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.639 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.639 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.639 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.639 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.639 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.640 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.640 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.640 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.640 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.640 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.640 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.640 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.640 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.640 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.640 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.640 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.640 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.641 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.641 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.641 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.641 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.641 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.641 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.641 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.641 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.641 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.641 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.641 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.641 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.641 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.642 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.642 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.642 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.642 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.642 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.642 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.642 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.642 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.642 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.642 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.642 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.642 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.642 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.642 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.643 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.643 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.643 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.643 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.643 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.643 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.643 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.643 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.643 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.643 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.643 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.643 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.643 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.644 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.644 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.644 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.644 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.644 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telemetry", 2024-09-19T08:25:57.644 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.644 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.644 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.644 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_basic": { 2024-09-19T08:25:57.644 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_basic", 2024-09-19T08:25:57.644 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.644 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.645 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.645 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.645 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.645 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.645 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.645 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T08:25:57.645 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.645 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.645 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.645 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_crash": { 2024-09-19T08:25:57.645 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_crash", 2024-09-19T08:25:57.645 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.645 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T08:25:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_device": { 2024-09-19T08:25:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_device", 2024-09-19T08:25:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.647 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.647 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.647 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.647 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.647 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.647 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T08:25:57.647 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.647 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.647 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.647 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_ident": { 2024-09-19T08:25:57.647 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_ident", 2024-09-19T08:25:57.647 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.647 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.648 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.648 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.648 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.648 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.648 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.648 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T08:25:57.648 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.648 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.648 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.648 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_perf": { 2024-09-19T08:25:57.648 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_perf", 2024-09-19T08:25:57.648 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.648 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.649 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.649 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.649 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.649 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.649 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.649 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T08:25:57.649 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.649 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.649 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.649 INFO:teuthology.orchestra.run.smithi022.stdout: "contact": { 2024-09-19T08:25:57.649 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "contact", 2024-09-19T08:25:57.650 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.650 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.650 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.650 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.650 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.650 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.650 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.650 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.650 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.650 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.650 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.650 INFO:teuthology.orchestra.run.smithi022.stdout: "description": { 2024-09-19T08:25:57.650 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "description", 2024-09-19T08:25:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: "device_url": { 2024-09-19T08:25:57.651 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_url", 2024-09-19T08:25:57.652 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.652 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.652 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.652 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T08:25:57.652 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.652 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.652 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.652 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.652 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.652 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.652 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.652 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-09-19T08:25:57.653 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-09-19T08:25:57.653 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.653 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.653 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.653 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.653 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.653 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.653 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.653 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.653 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.653 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.653 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.653 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:57.654 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:57.654 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.654 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.654 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.654 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "24", 2024-09-19T08:25:57.654 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "8", 2024-09-19T08:25:57.654 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.654 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.654 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.654 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.654 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.654 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.654 INFO:teuthology.orchestra.run.smithi022.stdout: "last_opt_revision": { 2024-09-19T08:25:57.655 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "last_opt_revision", 2024-09-19T08:25:57.655 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.655 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.655 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.655 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1", 2024-09-19T08:25:57.655 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.655 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.655 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.655 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.655 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.655 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.655 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.655 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard": { 2024-09-19T08:25:57.656 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard", 2024-09-19T08:25:57.656 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.656 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.656 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.656 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.656 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.656 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.656 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.656 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.656 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.656 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.656 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.656 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard_description": { 2024-09-19T08:25:57.657 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard_description", 2024-09-19T08:25:57.657 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.657 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.657 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.657 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.657 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.657 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.657 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.657 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.657 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.657 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.657 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.657 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.657 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.658 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.658 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.658 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.658 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.658 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.658 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.658 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.658 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.658 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.658 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.658 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.658 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.658 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.658 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.659 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.661 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.661 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.661 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.661 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.661 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.661 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.661 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.661 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.661 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.661 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.661 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.661 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.661 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: "organization": { 2024-09-19T08:25:57.662 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "organization", 2024-09-19T08:25:57.663 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.663 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.663 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.663 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.663 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.663 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.663 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.663 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.663 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.663 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.663 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: "proxy": { 2024-09-19T08:25:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "proxy", 2024-09-19T08:25:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.665 INFO:teuthology.orchestra.run.smithi022.stdout: "url": { 2024-09-19T08:25:57.665 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url", 2024-09-19T08:25:57.665 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.665 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.665 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.665 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T08:25:57.665 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.665 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.665 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.665 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.665 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.665 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.665 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.665 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.666 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.666 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.666 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "test_orchestrator", 2024-09-19T08:25:57.666 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.666 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.666 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.666 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.666 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.666 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.666 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.666 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.666 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.668 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.668 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.668 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.668 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.668 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.668 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.668 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.668 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.668 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.668 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.668 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.668 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.668 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.669 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.669 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.669 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.669 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.669 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.669 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.669 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.669 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.669 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.669 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.669 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.669 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.669 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.670 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.670 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.670 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.670 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.670 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.670 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.670 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.670 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.670 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.670 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.670 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.670 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.671 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.672 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.672 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.672 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.672 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.672 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.672 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.672 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.672 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.672 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.672 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.672 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.672 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.672 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.673 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.674 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.675 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.675 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.675 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.675 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.675 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.675 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.675 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.675 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.675 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.675 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.675 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.675 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.675 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.675 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.676 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.676 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.676 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.676 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.676 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.676 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.676 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.676 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.676 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.676 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.676 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.676 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.677 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.677 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_clones": { 2024-09-19T08:25:57.677 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_clones", 2024-09-19T08:25:57.677 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.677 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.677 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.677 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4", 2024-09-19T08:25:57.677 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.677 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.677 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.677 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T08:25:57.677 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: "periodic_async_work": { 2024-09-19T08:25:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "periodic_async_work", 2024-09-19T08:25:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Periodically check for async work", 2024-09-19T08:25:57.678 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.679 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.679 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.679 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_delay": { 2024-09-19T08:25:57.679 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_delay", 2024-09-19T08:25:57.679 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.679 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.679 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.679 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:57.679 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.679 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.679 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.679 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T08:25:57.680 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.680 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.680 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.680 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_no_wait": { 2024-09-19T08:25:57.680 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T08:25:57.680 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.680 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.680 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.680 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.680 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.680 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.680 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.680 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T08:25:57.681 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.681 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.681 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.681 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.681 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.681 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix", 2024-09-19T08:25:57.681 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.681 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.681 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.681 INFO:teuthology.orchestra.run.smithi022.stdout: "discovery_interval": { 2024-09-19T08:25:57.681 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "discovery_interval", 2024-09-19T08:25:57.682 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:57.682 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.682 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.682 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "100", 2024-09-19T08:25:57.682 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.682 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.682 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.682 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.682 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.682 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.682 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.682 INFO:teuthology.orchestra.run.smithi022.stdout: "identifier": { 2024-09-19T08:25:57.682 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "identifier", 2024-09-19T08:25:57.683 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.683 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.683 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.683 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.683 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.683 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.683 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.683 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.683 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.683 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.683 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.683 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:57.683 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:57.684 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.684 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.684 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.684 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:57.684 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.684 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.684 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.684 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.684 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.684 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.684 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.684 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.684 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.685 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.685 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.685 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.685 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.685 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.685 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.685 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.685 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.685 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.685 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.685 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.685 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.685 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.685 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.686 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.686 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.686 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.686 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.686 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.686 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.686 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.686 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.686 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.686 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.686 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.686 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.686 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.687 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.687 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.687 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.687 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.687 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.687 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.687 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.687 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.687 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.687 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.687 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.687 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.687 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.688 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.688 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.688 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.688 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.688 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.688 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.688 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.688 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.688 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.688 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.688 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.688 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.688 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.689 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.689 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.689 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.689 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.689 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.689 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.689 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.689 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.689 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.689 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.689 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.689 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_host": { 2024-09-19T08:25:57.689 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_host", 2024-09-19T08:25:57.690 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.690 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.690 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.690 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.690 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.690 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.690 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.690 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.690 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.690 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.690 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.690 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_port": { 2024-09-19T08:25:57.690 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_port", 2024-09-19T08:25:57.691 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.691 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.691 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.691 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10051", 2024-09-19T08:25:57.691 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.691 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.691 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.691 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.691 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.691 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.691 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.692 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_sender": { 2024-09-19T08:25:57.692 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_sender", 2024-09-19T08:25:57.692 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.692 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.692 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.692 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T08:25:57.692 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.692 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.692 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.692 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.692 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.692 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.692 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.693 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.693 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.693 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.693 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-19T08:25:57.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.693 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.693 INFO:teuthology.orchestra.run.smithi022.stdout: "gid": 24854, 2024-09-19T08:25:57.693 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "y", 2024-09-19T08:25:57.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_features": 4540701547738038271, 2024-09-19T08:25:57.693 INFO:teuthology.orchestra.run.smithi022.stdout: "available_modules": [ 2024-09-19T08:25:57.693 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.693 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "alerts", 2024-09-19T08:25:57.693 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.693 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.694 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.694 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:57.694 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:57.694 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.694 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.694 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.694 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:57.694 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.694 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.694 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.694 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T08:25:57.694 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.694 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.694 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.695 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.695 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.695 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.695 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.695 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.695 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.695 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.695 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.695 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.695 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.695 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.695 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.696 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.696 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.696 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.696 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.696 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.696 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.696 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.696 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.696 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.696 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.696 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.696 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.696 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.697 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.699 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.699 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.699 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.699 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.699 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.699 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.699 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.699 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.699 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.699 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.699 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.699 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.699 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.700 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.700 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_destination": { 2024-09-19T08:25:57.700 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_destination", 2024-09-19T08:25:57.700 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.700 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.700 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.700 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.700 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.700 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.700 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.700 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email address to send alerts to", 2024-09-19T08:25:57.700 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.701 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.701 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.701 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_from_name": { 2024-09-19T08:25:57.701 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_from_name", 2024-09-19T08:25:57.701 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.701 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.701 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.701 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "Ceph", 2024-09-19T08:25:57.701 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.701 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.701 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.701 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email From: name", 2024-09-19T08:25:57.701 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.702 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.702 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.702 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_host": { 2024-09-19T08:25:57.702 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_host", 2024-09-19T08:25:57.702 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.702 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.702 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.702 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.702 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.702 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.702 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.702 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP server", 2024-09-19T08:25:57.703 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.703 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.703 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.703 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_password": { 2024-09-19T08:25:57.703 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_password", 2024-09-19T08:25:57.703 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.703 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.703 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.703 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.703 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.703 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.703 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.703 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Password to authenticate with", 2024-09-19T08:25:57.704 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.704 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.704 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.704 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_port": { 2024-09-19T08:25:57.704 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_port", 2024-09-19T08:25:57.704 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.704 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.704 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.704 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "465", 2024-09-19T08:25:57.704 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.704 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.704 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.704 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP port", 2024-09-19T08:25:57.705 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.705 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.705 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.705 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_sender": { 2024-09-19T08:25:57.705 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_sender", 2024-09-19T08:25:57.705 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.705 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.705 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.705 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.705 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.705 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.705 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.706 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP envelope sender", 2024-09-19T08:25:57.706 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.706 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.706 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.706 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_ssl": { 2024-09-19T08:25:57.706 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_ssl", 2024-09-19T08:25:57.706 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.706 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.706 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.706 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.706 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.706 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.707 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.707 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T08:25:57.707 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.707 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.707 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.707 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_user": { 2024-09-19T08:25:57.707 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_user", 2024-09-19T08:25:57.707 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.707 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.707 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.707 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.707 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.707 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.708 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.708 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "User to authenticate as", 2024-09-19T08:25:57.708 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.708 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.708 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.708 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.708 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.708 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.708 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.708 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "balancer", 2024-09-19T08:25:57.708 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.708 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.708 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.709 INFO:teuthology.orchestra.run.smithi022.stdout: "active": { 2024-09-19T08:25:57.709 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "active", 2024-09-19T08:25:57.709 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.709 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.709 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.709 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.709 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.709 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.709 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.709 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T08:25:57.709 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.709 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.709 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.710 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_time": { 2024-09-19T08:25:57.710 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_time", 2024-09-19T08:25:57.710 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.710 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.710 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.710 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0000", 2024-09-19T08:25:57.710 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.710 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.710 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.710 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T08:25:57.710 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:25:57.710 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.710 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.711 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_weekday": { 2024-09-19T08:25:57.711 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_weekday", 2024-09-19T08:25:57.711 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:57.711 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.711 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.711 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:57.711 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-09-19T08:25:57.711 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-09-19T08:25:57.711 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.711 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T08:25:57.711 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:25:57.711 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.711 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.712 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_max_iterations": { 2024-09-19T08:25:57.712 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_max_iterations", 2024-09-19T08:25:57.712 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:57.712 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.712 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.712 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "25", 2024-09-19T08:25:57.712 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-09-19T08:25:57.712 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "250", 2024-09-19T08:25:57.712 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.712 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T08:25:57.712 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.712 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.712 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.713 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_metrics": { 2024-09-19T08:25:57.713 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_metrics", 2024-09-19T08:25:57.713 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.713 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.713 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.713 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T08:25:57.713 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.713 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.713 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.713 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T08:25:57.713 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T08:25:57.713 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.713 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.714 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.714 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_step": { 2024-09-19T08:25:57.714 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_step", 2024-09-19T08:25:57.714 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:57.714 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.714 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.714 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.5", 2024-09-19T08:25:57.714 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0.001", 2024-09-19T08:25:57.714 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "0.999", 2024-09-19T08:25:57.714 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.714 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "aggressiveness of optimization", 2024-09-19T08:25:57.714 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T08:25:57.714 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.714 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.715 INFO:teuthology.orchestra.run.smithi022.stdout: "end_time": { 2024-09-19T08:25:57.715 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_time", 2024-09-19T08:25:57.715 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.715 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.715 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.715 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2359", 2024-09-19T08:25:57.715 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.715 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.715 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.715 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T08:25:57.715 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:25:57.715 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.715 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: "end_weekday": { 2024-09-19T08:25:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_weekday", 2024-09-19T08:25:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-09-19T08:25:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-09-19T08:25:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T08:25:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:25:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.717 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.717 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.717 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.717 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.717 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.717 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.717 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.717 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.717 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.717 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.717 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.717 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.718 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.718 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.718 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.718 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.718 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.718 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.718 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.718 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.718 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.718 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.718 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.718 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.719 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.719 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.719 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.719 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.719 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.719 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.719 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.719 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.719 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.719 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.719 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.719 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.719 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.720 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.720 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.720 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.720 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.720 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.720 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.720 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.720 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.720 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.720 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.720 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.720 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.720 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.721 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.721 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.721 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.721 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.721 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.721 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.721 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.721 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.721 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.721 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.721 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.721 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.721 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.722 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.722 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.722 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.722 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.722 INFO:teuthology.orchestra.run.smithi022.stdout: "min_score": { 2024-09-19T08:25:57.722 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_score", 2024-09-19T08:25:57.722 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:57.722 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.722 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.722 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:57.722 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.722 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.722 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.723 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T08:25:57.723 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.723 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.723 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.723 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-09-19T08:25:57.723 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-09-19T08:25:57.723 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.723 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.723 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.723 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "upmap", 2024-09-19T08:25:57.723 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.723 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.723 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.724 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-compat", 2024-09-19T08:25:57.724 INFO:teuthology.orchestra.run.smithi022.stdout: "none", 2024-09-19T08:25:57.724 INFO:teuthology.orchestra.run.smithi022.stdout: "read", 2024-09-19T08:25:57.724 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap", 2024-09-19T08:25:57.724 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap-read" 2024-09-19T08:25:57.724 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.724 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Balancer mode", 2024-09-19T08:25:57.724 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.724 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.724 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.724 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_ids": { 2024-09-19T08:25:57.724 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_ids", 2024-09-19T08:25:57.725 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.725 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.725 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.725 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.725 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.725 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.725 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.725 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T08:25:57.725 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.725 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.725 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.725 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:57.725 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:57.726 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.726 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.726 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.726 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:57.726 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.726 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.726 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.726 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T08:25:57.726 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.726 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.726 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.726 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_deviation": { 2024-09-19T08:25:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_deviation", 2024-09-19T08:25:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-09-19T08:25:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T08:25:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T08:25:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_optimizations": { 2024-09-19T08:25:57.728 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_optimizations", 2024-09-19T08:25:57.728 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:57.728 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.728 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.728 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.728 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.728 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.728 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.728 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T08:25:57.728 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.728 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.728 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.728 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.728 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm", 2024-09-19T08:25:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_down_multiplier": { 2024-09-19T08:25:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_down_multiplier", 2024-09-19T08:25:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-09-19T08:25:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.729 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.730 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.730 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T08:25:57.730 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.730 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.730 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.730 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.730 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_refresh_rate": { 2024-09-19T08:25:57.730 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_refresh_rate", 2024-09-19T08:25:57.730 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.730 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.730 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.730 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "20", 2024-09-19T08:25:57.730 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.730 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.731 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.731 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T08:25:57.731 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.731 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.731 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.731 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_starting_port": { 2024-09-19T08:25:57.731 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_starting_port", 2024-09-19T08:25:57.731 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.731 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.731 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.731 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4721", 2024-09-19T08:25:57.731 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.731 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.732 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.732 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T08:25:57.732 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.732 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.732 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.732 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_ptrace": { 2024-09-19T08:25:57.732 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_ptrace", 2024-09-19T08:25:57.732 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.732 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.732 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.733 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.733 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.733 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.733 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.733 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T08:25:57.733 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T08:25:57.733 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.733 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.733 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_interval": { 2024-09-19T08:25:57.733 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_interval", 2024-09-19T08:25:57.733 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.733 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.734 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.734 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:57.734 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.734 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.734 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.734 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T08:25:57.734 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.734 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.734 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.734 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.734 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_memory_target_ratio": { 2024-09-19T08:25:57.734 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T08:25:57.734 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:57.734 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.735 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.735 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.7", 2024-09-19T08:25:57.735 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.735 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.735 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.735 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T08:25:57.735 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.735 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.735 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.735 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.735 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm_log_destination": { 2024-09-19T08:25:57.735 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm_log_destination", 2024-09-19T08:25:57.735 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.735 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.736 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.736 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.736 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.736 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.736 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.736 INFO:teuthology.orchestra.run.smithi022.stdout: "file", 2024-09-19T08:25:57.736 INFO:teuthology.orchestra.run.smithi022.stdout: "file,syslog", 2024-09-19T08:25:57.736 INFO:teuthology.orchestra.run.smithi022.stdout: "syslog" 2024-09-19T08:25:57.736 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.736 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T08:25:57.736 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.736 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.736 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.737 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.737 INFO:teuthology.orchestra.run.smithi022.stdout: "cgroups_split": { 2024-09-19T08:25:57.737 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cgroups_split", 2024-09-19T08:25:57.737 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.737 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.737 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.737 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.737 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.737 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.737 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.737 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T08:25:57.737 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.737 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.737 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.738 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.738 INFO:teuthology.orchestra.run.smithi022.stdout: "config_checks_enabled": { 2024-09-19T08:25:57.738 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_checks_enabled", 2024-09-19T08:25:57.738 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.738 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.738 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.738 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.738 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.738 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.738 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.738 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T08:25:57.738 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.738 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.738 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.739 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.739 INFO:teuthology.orchestra.run.smithi022.stdout: "config_dashboard": { 2024-09-19T08:25:57.739 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_dashboard", 2024-09-19T08:25:57.739 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.739 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.739 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.739 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.739 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.739 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.739 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.739 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T08:25:57.739 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.739 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.740 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.740 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.740 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_alertmanager": { 2024-09-19T08:25:57.740 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_alertmanager", 2024-09-19T08:25:57.740 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.740 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.740 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.740 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T08:25:57.740 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.740 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.740 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.740 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-09-19T08:25:57.740 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.740 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.741 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.741 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.741 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_base": { 2024-09-19T08:25:57.741 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_base", 2024-09-19T08:25:57.741 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.741 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.741 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.741 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T08:25:57.741 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.741 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.741 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.741 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Container image name, without the tag", 2024-09-19T08:25:57.741 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.741 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.742 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.742 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.742 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_elasticsearch": { 2024-09-19T08:25:57.742 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_elasticsearch", 2024-09-19T08:25:57.742 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.742 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.742 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.742 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T08:25:57.742 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.742 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.742 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.742 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "elasticsearch container image", 2024-09-19T08:25:57.742 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.742 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.743 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.743 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.743 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_grafana": { 2024-09-19T08:25:57.743 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_grafana", 2024-09-19T08:25:57.743 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.743 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.743 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.743 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T08:25:57.743 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.743 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.743 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.743 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-09-19T08:25:57.743 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.743 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.744 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.744 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.744 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_haproxy": { 2024-09-19T08:25:57.744 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_haproxy", 2024-09-19T08:25:57.744 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.744 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.744 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.744 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T08:25:57.744 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.744 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.744 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.744 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "HAproxy container image", 2024-09-19T08:25:57.745 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.745 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.745 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.745 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.745 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_agent": { 2024-09-19T08:25:57.745 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_agent", 2024-09-19T08:25:57.745 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.745 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.745 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.745 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T08:25:57.745 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.745 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.745 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.745 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger agent container image", 2024-09-19T08:25:57.746 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.746 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.746 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.746 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.746 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_collector": { 2024-09-19T08:25:57.746 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_collector", 2024-09-19T08:25:57.746 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.746 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.746 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.746 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T08:25:57.746 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.746 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.746 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.746 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger collector container image", 2024-09-19T08:25:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_query": { 2024-09-19T08:25:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_query", 2024-09-19T08:25:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T08:25:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger query container image", 2024-09-19T08:25:57.748 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.748 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.748 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.748 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.748 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_keepalived": { 2024-09-19T08:25:57.748 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_keepalived", 2024-09-19T08:25:57.748 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.748 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.748 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.748 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T08:25:57.748 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.748 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.748 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.749 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Keepalived container image", 2024-09-19T08:25:57.749 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.749 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.749 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.749 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.749 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_loki": { 2024-09-19T08:25:57.749 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_loki", 2024-09-19T08:25:57.749 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.749 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.749 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.749 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T08:25:57.749 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.749 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.749 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.750 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Loki container image", 2024-09-19T08:25:57.750 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.750 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.750 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.750 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.750 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_node_exporter": { 2024-09-19T08:25:57.750 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_node_exporter", 2024-09-19T08:25:57.750 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.750 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.750 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.750 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T08:25:57.750 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.750 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.750 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.751 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-09-19T08:25:57.751 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.751 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.751 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.751 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.751 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_nvmeof": { 2024-09-19T08:25:57.751 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_nvmeof", 2024-09-19T08:25:57.751 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.751 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.751 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.751 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T08:25:57.751 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.751 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Nvme-of container image", 2024-09-19T08:25:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_prometheus": { 2024-09-19T08:25:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_prometheus", 2024-09-19T08:25:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T08:25:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.753 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.753 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-09-19T08:25:57.753 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.753 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.753 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.753 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.753 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_promtail": { 2024-09-19T08:25:57.753 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_promtail", 2024-09-19T08:25:57.753 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.753 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.753 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.753 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T08:25:57.753 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.753 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.754 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.754 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Promtail container image", 2024-09-19T08:25:57.754 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.754 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.754 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.754 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.754 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_samba": { 2024-09-19T08:25:57.754 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_samba", 2024-09-19T08:25:57.754 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.754 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.754 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.754 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T08:25:57.754 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.755 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.755 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.755 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Samba/SMB container image", 2024-09-19T08:25:57.755 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.755 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.755 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.755 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.755 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_snmp_gateway": { 2024-09-19T08:25:57.755 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_snmp_gateway", 2024-09-19T08:25:57.755 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.755 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.755 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.755 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T08:25:57.755 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.756 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.756 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.756 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SNMP Gateway container image", 2024-09-19T08:25:57.756 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.756 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.756 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.756 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.756 INFO:teuthology.orchestra.run.smithi022.stdout: "container_init": { 2024-09-19T08:25:57.756 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_init", 2024-09-19T08:25:57.756 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.756 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.756 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T08:25:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "daemon_cache_timeout": { 2024-09-19T08:25:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "daemon_cache_timeout", 2024-09-19T08:25:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.757 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.758 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.758 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:57.758 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.758 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.758 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.758 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T08:25:57.758 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.758 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.758 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.758 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.758 INFO:teuthology.orchestra.run.smithi022.stdout: "default_cephadm_command_timeout": { 2024-09-19T08:25:57.758 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T08:25:57.758 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.758 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "900", 2024-09-19T08:25:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T08:25:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: "default_registry": { 2024-09-19T08:25:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_registry", 2024-09-19T08:25:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.759 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.760 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.760 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io", 2024-09-19T08:25:57.760 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.760 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.760 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.760 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T08:25:57.760 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.760 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.760 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.760 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.760 INFO:teuthology.orchestra.run.smithi022.stdout: "device_cache_timeout": { 2024-09-19T08:25:57.760 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_cache_timeout", 2024-09-19T08:25:57.761 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.761 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.761 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.761 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1800", 2024-09-19T08:25:57.761 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.761 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.761 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.761 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache device inventory", 2024-09-19T08:25:57.761 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.761 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.761 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.761 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.761 INFO:teuthology.orchestra.run.smithi022.stdout: "device_enhanced_scan": { 2024-09-19T08:25:57.761 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_enhanced_scan", 2024-09-19T08:25:57.762 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.762 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.762 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.762 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.762 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.762 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.762 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.762 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T08:25:57.762 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.762 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.762 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.762 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.762 INFO:teuthology.orchestra.run.smithi022.stdout: "facts_cache_timeout": { 2024-09-19T08:25:57.762 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "facts_cache_timeout", 2024-09-19T08:25:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache host facts data", 2024-09-19T08:25:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: "grafana_dashboards_path": { 2024-09-19T08:25:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "grafana_dashboards_path", 2024-09-19T08:25:57.764 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.764 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.764 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.764 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T08:25:57.764 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.764 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.764 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.764 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T08:25:57.764 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.764 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.764 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.764 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.764 INFO:teuthology.orchestra.run.smithi022.stdout: "host_check_interval": { 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "host_check_interval", 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to perform a host check", 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: "hw_monitoring": { 2024-09-19T08:25:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hw_monitoring", 2024-09-19T08:25:57.766 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.766 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.766 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.766 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.766 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.766 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.766 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.766 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T08:25:57.766 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.766 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.766 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.766 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.766 INFO:teuthology.orchestra.run.smithi022.stdout: "inventory_list_all": { 2024-09-19T08:25:57.766 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "inventory_list_all", 2024-09-19T08:25:57.767 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.767 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.767 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.767 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.767 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.767 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.767 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.767 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T08:25:57.767 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.767 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.767 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.767 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.767 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.768 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.768 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.768 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.768 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.768 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.768 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.768 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.768 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.768 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.768 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.768 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.768 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.768 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.769 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.769 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.769 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.769 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.769 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.769 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.769 INFO:teuthology.orchestra.run.smithi022.stdout: "log_refresh_metadata": { 2024-09-19T08:25:57.769 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_refresh_metadata", 2024-09-19T08:25:57.769 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.769 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.769 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.769 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.769 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.770 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.770 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.770 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T08:25:57.770 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.770 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.770 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.770 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.770 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.770 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.770 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.770 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.770 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.770 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.770 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.771 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.771 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.771 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T08:25:57.771 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.771 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.771 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.771 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.771 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.771 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.771 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.771 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.771 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.772 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.772 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.772 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.772 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.772 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.772 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.772 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.772 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.772 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.772 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.772 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.772 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.772 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.772 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.773 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.773 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.773 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.773 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.773 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.773 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.773 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.773 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.773 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.773 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.773 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.773 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.773 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.774 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.774 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T08:25:57.774 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T08:25:57.774 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.774 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.774 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.774 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.774 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.774 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.774 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.774 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T08:25:57.774 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.774 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.774 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.775 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.775 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T08:25:57.775 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T08:25:57.775 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.775 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.775 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.775 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "*", 2024-09-19T08:25:57.775 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.775 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.775 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.775 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T08:25:57.775 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.775 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.776 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.776 INFO:teuthology.orchestra.run.smithi022.stdout: "max_count_per_host": { 2024-09-19T08:25:57.776 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_count_per_host", 2024-09-19T08:25:57.776 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.776 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.776 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.776 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.776 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.776 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.776 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.776 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of daemons per service per host", 2024-09-19T08:25:57.776 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.776 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.777 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.777 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.777 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd_draining_count": { 2024-09-19T08:25:57.777 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_osd_draining_count", 2024-09-19T08:25:57.777 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.777 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.777 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.777 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.777 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.777 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.777 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.777 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T08:25:57.777 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.777 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.778 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.778 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.778 INFO:teuthology.orchestra.run.smithi022.stdout: "migration_current": { 2024-09-19T08:25:57.778 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "migration_current", 2024-09-19T08:25:57.778 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.778 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.778 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.778 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.778 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.778 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.778 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.778 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "internal - do not modify", 2024-09-19T08:25:57.778 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.778 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.779 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.779 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-09-19T08:25:57.779 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-09-19T08:25:57.779 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.779 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.779 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.779 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "root", 2024-09-19T08:25:57.779 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.779 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.779 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.779 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm-package", 2024-09-19T08:25:57.779 INFO:teuthology.orchestra.run.smithi022.stdout: "root" 2024-09-19T08:25:57.780 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.780 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T08:25:57.780 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.780 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.780 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.780 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.780 INFO:teuthology.orchestra.run.smithi022.stdout: "oob_default_addr": { 2024-09-19T08:25:57.780 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "oob_default_addr", 2024-09-19T08:25:57.780 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.780 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.780 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.780 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "169.254.1.1", 2024-09-19T08:25:57.780 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.780 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.781 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.781 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T08:25:57.781 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.781 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.781 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.781 INFO:teuthology.orchestra.run.smithi022.stdout: "prometheus_alerts_path": { 2024-09-19T08:25:57.781 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus_alerts_path", 2024-09-19T08:25:57.781 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.781 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.781 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.781 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T08:25:57.781 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.781 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.782 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.782 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T08:25:57.782 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.782 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.782 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.782 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.782 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_insecure": { 2024-09-19T08:25:57.782 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_insecure", 2024-09-19T08:25:57.782 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.782 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.782 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.782 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.783 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.783 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.783 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.783 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T08:25:57.783 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.783 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.783 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.783 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.783 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_password": { 2024-09-19T08:25:57.783 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_password", 2024-09-19T08:25:57.783 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.783 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.783 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.783 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.784 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.784 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.784 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.784 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T08:25:57.784 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.784 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.784 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.784 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.784 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_url": { 2024-09-19T08:25:57.784 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_url", 2024-09-19T08:25:57.784 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.784 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.784 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.785 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.785 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.785 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.785 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.785 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T08:25:57.785 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.785 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.785 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.785 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.785 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_username": { 2024-09-19T08:25:57.785 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_username", 2024-09-19T08:25:57.785 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.785 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.786 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.786 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.786 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.786 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.786 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.786 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T08:25:57.786 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.786 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.786 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.786 INFO:teuthology.orchestra.run.smithi022.stdout: "secure_monitoring_stack": { 2024-09-19T08:25:57.786 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secure_monitoring_stack", 2024-09-19T08:25:57.786 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.786 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.787 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.787 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.787 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.787 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.787 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.787 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T08:25:57.787 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.787 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.787 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.787 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.787 INFO:teuthology.orchestra.run.smithi022.stdout: "service_discovery_port": { 2024-09-19T08:25:57.787 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "service_discovery_port", 2024-09-19T08:25:57.787 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.788 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.788 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.788 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8765", 2024-09-19T08:25:57.788 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.788 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.788 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.788 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "cephadm service discovery port", 2024-09-19T08:25:57.788 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.788 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.788 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.788 INFO:teuthology.orchestra.run.smithi022.stdout: "ssh_config_file": { 2024-09-19T08:25:57.788 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssh_config_file", 2024-09-19T08:25:57.789 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.789 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.789 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.789 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.789 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.789 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.789 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.789 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T08:25:57.789 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.789 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.789 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.789 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.789 INFO:teuthology.orchestra.run.smithi022.stdout: "use_agent": { 2024-09-19T08:25:57.789 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_agent", 2024-09-19T08:25:57.790 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.790 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.790 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.790 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.790 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.790 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.790 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.790 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T08:25:57.790 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.790 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.790 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.790 INFO:teuthology.orchestra.run.smithi022.stdout: "use_repo_digest": { 2024-09-19T08:25:57.791 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_repo_digest", 2024-09-19T08:25:57.791 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.791 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.791 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.791 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.791 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.791 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.791 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.791 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T08:25:57.791 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.791 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.791 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.791 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.791 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_failed_host_check": { 2024-09-19T08:25:57.792 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_failed_host_check", 2024-09-19T08:25:57.792 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.792 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.792 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.792 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.792 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.792 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.792 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.792 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T08:25:57.792 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.792 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.792 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.792 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.793 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_daemons": { 2024-09-19T08:25:57.793 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_daemons", 2024-09-19T08:25:57.793 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.793 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.793 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.793 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.793 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.793 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.793 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.793 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T08:25:57.793 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.793 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.793 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.793 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.794 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_hosts": { 2024-09-19T08:25:57.794 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_hosts", 2024-09-19T08:25:57.794 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.794 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.794 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.794 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.794 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.794 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.794 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.794 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T08:25:57.794 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.794 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.794 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.795 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.795 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.795 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.795 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.795 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crash", 2024-09-19T08:25:57.795 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.795 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.795 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.795 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.795 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.795 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.795 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.795 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.795 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.796 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.796 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.796 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.796 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.796 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.796 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.796 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.796 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.796 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.796 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.796 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.796 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.796 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.797 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.797 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.797 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.797 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.797 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.797 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.797 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.797 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.797 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.797 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.797 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.797 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.798 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.798 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.798 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.798 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.798 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.798 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.798 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.798 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.798 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.798 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.798 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.798 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.798 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.798 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.799 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.799 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.799 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.799 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.799 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.799 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.799 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.799 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.799 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.799 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.799 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.799 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.799 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.800 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.800 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.800 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.800 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.800 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.800 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.800 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.800 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.800 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.800 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.800 INFO:teuthology.orchestra.run.smithi022.stdout: "retain_interval": { 2024-09-19T08:25:57.800 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retain_interval", 2024-09-19T08:25:57.800 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.801 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.801 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.801 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "31536000", 2024-09-19T08:25:57.801 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.801 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.801 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.801 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T08:25:57.801 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.801 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.801 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.801 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_recent_interval": { 2024-09-19T08:25:57.801 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_recent_interval", 2024-09-19T08:25:57.801 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.802 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.802 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.802 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1209600", 2024-09-19T08:25:57.802 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.802 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.802 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.802 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T08:25:57.802 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.802 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.802 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.802 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.802 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.802 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.803 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dashboard", 2024-09-19T08:25:57.803 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.803 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.803 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.803 INFO:teuthology.orchestra.run.smithi022.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T08:25:57.803 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T08:25:57.803 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.803 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.803 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.803 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.803 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.803 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.803 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.803 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.804 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.804 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.804 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.804 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T08:25:57.804 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T08:25:57.804 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.804 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.804 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.804 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.804 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.804 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.804 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.804 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.805 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.805 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.805 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.805 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T08:25:57.805 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T08:25:57.805 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.805 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.805 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.805 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.805 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.805 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.805 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.805 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.806 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.806 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.806 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.806 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.806 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_ENABLED": { 2024-09-19T08:25:57.806 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T08:25:57.806 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.806 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.806 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.806 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.806 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.806 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.806 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.806 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.807 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.807 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.807 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.807 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T08:25:57.807 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T08:25:57.807 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.807 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.807 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.807 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.807 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.807 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.807 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.807 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.808 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.808 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.808 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.808 INFO:teuthology.orchestra.run.smithi022.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T08:25:57.808 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T08:25:57.808 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.808 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.808 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.808 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.808 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.808 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.809 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.809 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.809 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.809 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.809 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.809 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T08:25:57.809 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T08:25:57.809 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.809 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.809 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.809 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.809 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.809 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.810 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.810 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.810 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.810 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.810 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.810 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T08:25:57.810 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T08:25:57.810 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.810 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.810 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.810 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.810 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.810 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.811 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.811 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.811 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.811 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.811 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.811 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T08:25:57.811 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T08:25:57.811 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.811 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.811 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.811 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.811 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.811 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.812 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.812 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.812 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.812 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.812 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.812 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T08:25:57.812 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T08:25:57.812 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.812 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.812 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.812 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.812 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.813 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.813 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.813 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.813 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.813 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.813 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.813 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T08:25:57.813 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T08:25:57.813 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.813 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.813 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.813 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.813 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.814 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.814 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.814 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.814 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.814 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.814 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.814 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T08:25:57.814 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T08:25:57.814 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.814 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.814 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.814 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.814 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.815 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.815 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.815 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.815 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.815 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.815 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.815 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T08:25:57.815 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T08:25:57.815 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.815 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.815 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.815 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.816 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.816 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.816 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.816 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.816 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.816 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.816 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.816 INFO:teuthology.orchestra.run.smithi022.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T08:25:57.816 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T08:25:57.816 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.816 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.816 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.816 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.817 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.817 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.817 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.817 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.817 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.817 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.817 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.817 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T08:25:57.817 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T08:25:57.817 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.817 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.817 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.817 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.818 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.819 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.819 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.819 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.819 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.819 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.819 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.819 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_URL": { 2024-09-19T08:25:57.819 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_URL", 2024-09-19T08:25:57.819 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.819 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.819 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.819 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.819 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.820 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.820 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.820 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.820 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.820 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.820 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.820 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T08:25:57.820 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T08:25:57.820 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.820 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.820 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.820 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-09-19T08:25:57.820 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.821 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.821 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.821 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.821 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.821 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.821 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.821 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T08:25:57.821 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T08:25:57.821 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.821 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.821 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.821 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.821 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.822 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.822 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.822 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.822 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.822 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.822 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.822 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T08:25:57.822 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T08:25:57.822 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.822 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.822 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.822 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.823 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.823 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.823 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.823 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.823 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.823 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.823 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.823 INFO:teuthology.orchestra.run.smithi022.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T08:25:57.823 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T08:25:57.823 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.823 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.823 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.824 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.824 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.824 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.824 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.824 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.824 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.824 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.824 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.824 INFO:teuthology.orchestra.run.smithi022.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T08:25:57.824 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T08:25:57.824 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.824 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.824 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.825 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.825 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.825 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.825 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.825 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.825 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.825 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.825 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.825 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T08:25:57.825 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T08:25:57.825 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.825 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.825 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.826 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.826 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.826 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.826 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.826 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.826 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.826 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.826 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.826 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T08:25:57.826 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T08:25:57.826 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.826 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.827 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.828 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.828 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.828 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.828 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.828 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.828 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.828 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.828 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.828 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T08:25:57.828 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T08:25:57.828 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.828 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.829 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.829 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.829 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.829 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.829 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.829 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.829 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.829 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.829 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.829 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T08:25:57.829 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T08:25:57.829 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.829 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.830 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.830 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.830 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.830 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.830 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.830 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.830 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.830 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.830 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.830 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T08:25:57.830 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T08:25:57.830 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.830 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.831 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.831 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.831 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.831 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.831 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.831 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.831 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.831 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.831 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.831 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.831 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T08:25:57.831 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T08:25:57.831 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.831 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.832 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.833 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.833 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.833 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.833 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.833 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.833 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.833 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.833 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.833 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T08:25:57.833 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T08:25:57.833 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.833 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.833 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.834 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.834 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.834 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.834 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.834 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.834 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.834 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.834 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.834 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T08:25:57.834 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T08:25:57.834 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.834 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.834 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.835 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.835 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.835 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.835 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.835 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.835 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.835 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.835 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.835 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T08:25:57.835 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T08:25:57.835 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.835 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.836 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.836 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T08:25:57.836 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.836 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.836 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.836 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.836 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.836 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.836 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.836 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T08:25:57.836 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T08:25:57.836 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.837 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.837 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.837 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.837 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.837 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.837 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.837 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.837 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.837 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.837 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.837 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T08:25:57.837 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T08:25:57.837 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8", 2024-09-19T08:25:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T08:25:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T08:25:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.839 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.839 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.839 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "45", 2024-09-19T08:25:57.839 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.839 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.839 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.839 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.839 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.839 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.839 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.839 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T08:25:57.839 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T08:25:57.839 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.840 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.840 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.840 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.840 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.840 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.840 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.840 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.840 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.840 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.840 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.840 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T08:25:57.840 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T08:25:57.841 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.841 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.841 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.841 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-09-19T08:25:57.841 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.841 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.841 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.841 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.841 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.841 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.841 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.841 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.841 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T08:25:57.841 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T08:25:57.842 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.842 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.842 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.842 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.842 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.842 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.842 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.842 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.842 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.842 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.842 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.843 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T08:25:57.843 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T08:25:57.843 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.843 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.843 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.843 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.843 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.843 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.843 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.843 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.843 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.843 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.843 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.843 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.844 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T08:25:57.844 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T08:25:57.844 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.844 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.844 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.844 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:57.844 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.844 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.844 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.844 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.844 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.844 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.844 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.845 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T08:25:57.845 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T08:25:57.845 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.845 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.845 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.845 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.845 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.845 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.845 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.845 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.845 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.845 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.845 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.846 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T08:25:57.846 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T08:25:57.846 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.846 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.846 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.846 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:57.846 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.846 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.846 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.846 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.846 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.846 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.846 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: "cross_origin_url": { 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cross_origin_url", 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.848 INFO:teuthology.orchestra.run.smithi022.stdout: "crt_file": { 2024-09-19T08:25:57.848 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crt_file", 2024-09-19T08:25:57.848 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.848 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.848 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.848 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.848 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.848 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.848 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.848 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.848 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.848 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.848 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.849 INFO:teuthology.orchestra.run.smithi022.stdout: "debug": { 2024-09-19T08:25:57.849 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "debug", 2024-09-19T08:25:57.849 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.849 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.849 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.849 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.849 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.849 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.849 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.849 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable/disable debug options", 2024-09-19T08:25:57.849 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.850 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.850 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.850 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.850 INFO:teuthology.orchestra.run.smithi022.stdout: "jwt_token_ttl": { 2024-09-19T08:25:57.850 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "jwt_token_ttl", 2024-09-19T08:25:57.850 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.850 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.850 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.850 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "28800", 2024-09-19T08:25:57.850 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.850 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.850 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.850 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.850 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-09-19T08:25:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-09-19T08:25:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.851 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.852 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.852 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.852 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.852 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.852 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.852 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.852 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.852 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.852 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.852 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.852 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.852 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.852 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.853 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.853 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.853 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.853 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.853 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.853 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.853 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.853 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.853 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.853 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.853 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.853 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.853 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.853 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.854 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.854 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.854 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.854 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.854 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.854 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.854 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.854 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.854 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.854 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.854 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.854 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.854 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.854 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.855 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.855 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.855 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.855 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.855 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.855 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.855 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.855 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.855 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.855 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.855 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.855 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.855 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.855 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.856 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.856 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.856 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.856 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.856 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.856 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.856 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.856 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.856 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.856 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.856 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.856 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.856 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.856 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.857 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.857 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.857 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.857 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.857 INFO:teuthology.orchestra.run.smithi022.stdout: "motd": { 2024-09-19T08:25:57.857 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "motd", 2024-09-19T08:25:57.857 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.857 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.857 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.857 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.857 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.857 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.857 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.857 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "The message of the day", 2024-09-19T08:25:57.858 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.858 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.858 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.858 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect_resolve_ip_addr": { 2024-09-19T08:25:57.858 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T08:25:57.858 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.858 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.858 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.858 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.858 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.858 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.858 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.858 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.859 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.859 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.859 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.859 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.859 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-09-19T08:25:57.859 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-09-19T08:25:57.859 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.859 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.859 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.859 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-09-19T08:25:57.859 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.859 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.859 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.859 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.860 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.860 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.860 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.860 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-09-19T08:25:57.860 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-09-19T08:25:57.860 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.860 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.860 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.860 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8080", 2024-09-19T08:25:57.860 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.860 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.860 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.860 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.861 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.862 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.862 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.862 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl_server_port": { 2024-09-19T08:25:57.862 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl_server_port", 2024-09-19T08:25:57.862 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.862 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.862 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.862 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8443", 2024-09-19T08:25:57.862 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.862 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.862 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.863 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.863 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.863 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.863 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.863 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-09-19T08:25:57.863 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-09-19T08:25:57.863 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.863 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.863 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.863 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "redirect", 2024-09-19T08:25:57.863 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.863 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.863 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.864 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.864 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect" 2024-09-19T08:25:57.864 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.864 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.864 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.864 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.864 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.864 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.864 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-09-19T08:25:57.864 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-09-19T08:25:57.864 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.864 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.864 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.865 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-09-19T08:25:57.865 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-09-19T08:25:57.865 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-09-19T08:25:57.865 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.865 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.865 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.865 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.865 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.865 INFO:teuthology.orchestra.run.smithi022.stdout: "url_prefix": { 2024-09-19T08:25:57.865 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url_prefix", 2024-09-19T08:25:57.865 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.865 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.865 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.866 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.866 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.866 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.866 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.866 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.866 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.866 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.866 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.866 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.866 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.866 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.866 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2024-09-19T08:25:57.866 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.867 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.867 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.867 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_monitoring": { 2024-09-19T08:25:57.867 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_monitoring", 2024-09-19T08:25:57.867 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.867 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.867 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.867 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.867 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.867 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.867 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.867 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "monitor device health metrics", 2024-09-19T08:25:57.867 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.867 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.868 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.868 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.868 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.868 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.868 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.868 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.868 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.868 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.868 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.868 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.868 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.868 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.869 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.869 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.869 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.869 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.869 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.869 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.869 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.869 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.869 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.869 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.869 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.869 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.869 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.870 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.870 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.870 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.870 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.870 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.870 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.870 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.870 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.870 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.870 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.870 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.870 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.870 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.871 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.872 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.872 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.872 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.872 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.872 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.872 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.872 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.872 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.872 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.872 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.872 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.872 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.872 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.873 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.873 INFO:teuthology.orchestra.run.smithi022.stdout: "mark_out_threshold": { 2024-09-19T08:25:57.873 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mark_out_threshold", 2024-09-19T08:25:57.873 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.873 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.873 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.873 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2419200", 2024-09-19T08:25:57.873 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.873 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.873 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.873 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T08:25:57.873 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": { 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_name", 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "device_health_metrics", 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.874 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.875 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.875 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.875 INFO:teuthology.orchestra.run.smithi022.stdout: "retention_period": { 2024-09-19T08:25:57.875 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retention_period", 2024-09-19T08:25:57.875 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.875 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.875 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.875 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15552000", 2024-09-19T08:25:57.875 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.875 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.875 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.875 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain device health metrics", 2024-09-19T08:25:57.875 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.875 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.876 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.876 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_frequency": { 2024-09-19T08:25:57.876 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_frequency", 2024-09-19T08:25:57.876 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.876 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.876 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.876 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-09-19T08:25:57.876 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.876 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.876 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.876 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T08:25:57.877 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.877 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.877 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.877 INFO:teuthology.orchestra.run.smithi022.stdout: "self_heal": { 2024-09-19T08:25:57.877 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "self_heal", 2024-09-19T08:25:57.877 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.877 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.877 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.877 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.877 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.877 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.877 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.877 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T08:25:57.878 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.878 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.878 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.878 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:57.878 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:57.878 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.878 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.878 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.878 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:57.878 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.878 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.878 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.878 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T08:25:57.879 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.879 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.879 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.879 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_threshold": { 2024-09-19T08:25:57.879 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_threshold", 2024-09-19T08:25:57.879 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.879 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.879 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.879 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7257600", 2024-09-19T08:25:57.879 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.879 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.879 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.879 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T08:25:57.880 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.880 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.880 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.880 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.880 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.880 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.880 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "diskprediction_local", 2024-09-19T08:25:57.880 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.880 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.880 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.880 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.880 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.881 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.881 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.881 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.881 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.881 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.881 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.881 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.881 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.881 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.881 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.881 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.881 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.881 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.881 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.882 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.882 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.882 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.882 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.882 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.882 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.882 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.882 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.882 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.882 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.882 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.882 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.882 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.883 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.884 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.884 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.884 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.884 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.884 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.884 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.884 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.884 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.884 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.884 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.884 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.884 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.884 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.885 INFO:teuthology.orchestra.run.smithi022.stdout: "predict_interval": { 2024-09-19T08:25:57.886 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predict_interval", 2024-09-19T08:25:57.886 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.886 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.886 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.886 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-09-19T08:25:57.886 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.886 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.886 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.886 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.886 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.886 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.886 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.886 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.887 INFO:teuthology.orchestra.run.smithi022.stdout: "predictor_model": { 2024-09-19T08:25:57.887 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predictor_model", 2024-09-19T08:25:57.887 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.887 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.887 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.887 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "prophetstor", 2024-09-19T08:25:57.887 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.887 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.887 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.887 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.887 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.887 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.887 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.887 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.888 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:57.888 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:57.888 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.888 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.888 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.888 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:57.888 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.888 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.888 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.888 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.888 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.888 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.888 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.888 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.889 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.889 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.889 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.889 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "influx", 2024-09-19T08:25:57.889 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": false, 2024-09-19T08:25:57.889 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "influxdb python module not found", 2024-09-19T08:25:57.889 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.889 INFO:teuthology.orchestra.run.smithi022.stdout: "batch_size": { 2024-09-19T08:25:57.889 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "batch_size", 2024-09-19T08:25:57.889 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.889 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.889 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.889 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5000", 2024-09-19T08:25:57.889 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.890 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.890 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.890 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T08:25:57.890 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.890 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.890 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.890 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.890 INFO:teuthology.orchestra.run.smithi022.stdout: "database": { 2024-09-19T08:25:57.890 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "database", 2024-09-19T08:25:57.890 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.890 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.890 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.890 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "ceph", 2024-09-19T08:25:57.891 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.891 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.891 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.891 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T08:25:57.891 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.891 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.891 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.891 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.891 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": { 2024-09-19T08:25:57.891 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hostname", 2024-09-19T08:25:57.891 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.891 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.891 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.891 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.892 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.892 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.892 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.892 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server hostname", 2024-09-19T08:25:57.892 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.892 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.892 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.892 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.892 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:57.892 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:57.892 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.892 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.892 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.893 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "30", 2024-09-19T08:25:57.893 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "5", 2024-09-19T08:25:57.893 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.893 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.893 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T08:25:57.893 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.893 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.893 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.893 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.893 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.893 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.893 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.893 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.894 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.894 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.894 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.894 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.894 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.894 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.894 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.894 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.894 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.894 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.894 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.894 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.894 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.894 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.895 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.895 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.895 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.895 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.895 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.895 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.895 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.895 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.895 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.895 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.895 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.895 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.895 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.895 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.896 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.896 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.896 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.896 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.896 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.896 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.896 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.896 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.896 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.896 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.896 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.896 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.897 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.897 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.897 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.897 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.897 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.897 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.897 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.897 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.897 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.897 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.897 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.897 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.897 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.898 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.898 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.898 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.898 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.898 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.898 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.898 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.898 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.898 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.898 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.898 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.898 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.898 INFO:teuthology.orchestra.run.smithi022.stdout: "password": { 2024-09-19T08:25:57.898 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "password", 2024-09-19T08:25:57.899 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.899 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.899 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.899 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.899 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.899 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.899 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.899 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "password of InfluxDB server user", 2024-09-19T08:25:57.899 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.899 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.899 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.899 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.899 INFO:teuthology.orchestra.run.smithi022.stdout: "port": { 2024-09-19T08:25:57.899 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "port", 2024-09-19T08:25:57.900 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.900 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.900 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.900 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8086", 2024-09-19T08:25:57.900 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.900 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.900 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.900 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server port", 2024-09-19T08:25:57.900 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.900 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.900 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.900 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.900 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-09-19T08:25:57.901 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-09-19T08:25:57.901 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.901 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.901 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.901 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "false", 2024-09-19T08:25:57.901 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.901 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.901 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.901 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:25:57.901 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.901 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.901 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.901 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.901 INFO:teuthology.orchestra.run.smithi022.stdout: "threads": { 2024-09-19T08:25:57.902 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threads", 2024-09-19T08:25:57.902 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.902 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.902 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.902 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:57.902 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-09-19T08:25:57.902 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "32", 2024-09-19T08:25:57.902 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.902 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T08:25:57.902 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.902 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.902 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.902 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.902 INFO:teuthology.orchestra.run.smithi022.stdout: "username": { 2024-09-19T08:25:57.903 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "username", 2024-09-19T08:25:57.903 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.903 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.903 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.903 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.903 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.903 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.903 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.903 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "username of InfluxDB server user", 2024-09-19T08:25:57.903 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.903 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.903 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.903 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.904 INFO:teuthology.orchestra.run.smithi022.stdout: "verify_ssl": { 2024-09-19T08:25:57.904 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "verify_ssl", 2024-09-19T08:25:57.904 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.904 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.904 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.904 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "true", 2024-09-19T08:25:57.904 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.904 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.904 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.904 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:25:57.904 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.904 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.904 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.905 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.905 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.905 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.905 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.905 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "insights", 2024-09-19T08:25:57.905 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.905 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.905 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.905 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.905 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.905 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.905 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.905 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.905 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.906 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.906 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.906 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.906 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.906 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.906 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.906 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.906 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.906 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.906 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.906 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.906 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.906 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.906 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.907 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.907 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.907 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.907 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.907 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.907 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.907 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.907 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.907 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.907 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.907 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.907 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.907 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.907 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.908 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.908 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.908 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.908 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.908 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.908 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.908 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.908 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.908 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.908 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.908 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.908 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.908 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.909 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.909 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.909 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.909 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.909 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.909 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.909 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.909 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.909 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.909 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.909 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.909 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.909 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.909 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.910 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.910 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.910 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.910 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.910 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.910 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.910 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.910 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.910 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.910 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.910 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.910 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.910 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "iostat", 2024-09-19T08:25:57.910 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.911 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.911 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.911 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.911 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.911 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.911 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.911 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.911 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.911 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.911 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.911 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.911 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.911 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.911 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.912 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.912 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.912 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.912 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.912 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.912 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.912 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.912 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.912 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.912 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.912 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.912 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.912 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.912 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.913 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.913 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.913 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.913 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.913 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.913 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.913 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.913 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.913 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.913 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.913 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.913 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.913 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.914 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.916 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.916 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.916 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.916 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.916 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.916 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "k8sevents", 2024-09-19T08:25:57.916 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.916 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.916 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.916 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_event_retention_days": { 2024-09-19T08:25:57.916 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ceph_event_retention_days", 2024-09-19T08:25:57.916 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.916 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.916 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.917 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7", 2024-09-19T08:25:57.917 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.917 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.917 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.917 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T08:25:57.917 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.917 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.917 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.917 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.917 INFO:teuthology.orchestra.run.smithi022.stdout: "config_check_secs": { 2024-09-19T08:25:57.917 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_check_secs", 2024-09-19T08:25:57.917 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.917 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "10", 2024-09-19T08:25:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T08:25:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "localpool", 2024-09-19T08:25:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "failure_domain": { 2024-09-19T08:25:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "failure_domain", 2024-09-19T08:25:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "host", 2024-09-19T08:25:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "failure domain for any created local pool", 2024-09-19T08:25:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T08:25:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": { 2024-09-19T08:25:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_size", 2024-09-19T08:25:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default min_size for any created local pool", 2024-09-19T08:25:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T08:25:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "num_rep": { 2024-09-19T08:25:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "num_rep", 2024-09-19T08:25:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3", 2024-09-19T08:25:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default replica count for any created local pool", 2024-09-19T08:25:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": { 2024-09-19T08:25:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_num", 2024-09-19T08:25:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "128", 2024-09-19T08:25:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T08:25:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "prefix": { 2024-09-19T08:25:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prefix", 2024-09-19T08:25:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name prefix for any created local pool", 2024-09-19T08:25:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: "subtree": { 2024-09-19T08:25:57.934 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "subtree", 2024-09-19T08:25:57.935 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.935 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.935 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.935 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "rack", 2024-09-19T08:25:57.935 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.935 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.935 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.935 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T08:25:57.935 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T08:25:57.935 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.935 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.935 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.935 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.936 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.936 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mds_autoscaler", 2024-09-19T08:25:57.936 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.936 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.936 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.936 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.936 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.936 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.936 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.936 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.936 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.936 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.936 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.937 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.937 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.937 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.937 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.937 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.937 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.937 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.937 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.937 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.937 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.937 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.937 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.937 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.938 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.938 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.938 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.938 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.938 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.938 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.938 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.938 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.938 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.938 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.938 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.938 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.938 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.939 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.939 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.939 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.939 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.939 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.939 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.939 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.939 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.939 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.939 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.939 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.939 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.939 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.939 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.940 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.940 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.940 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.940 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.940 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.940 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.940 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.940 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.940 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.940 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.940 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.940 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.941 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.941 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.941 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.941 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.941 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.941 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.941 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.941 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.941 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.941 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirroring", 2024-09-19T08:25:57.941 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.941 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.941 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.942 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.942 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.942 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.942 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.942 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.942 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.942 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.942 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.942 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.942 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.942 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.942 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.942 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.942 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.943 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.943 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.943 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.943 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.943 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.943 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.943 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.943 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.943 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.943 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.943 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.943 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.943 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.943 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.944 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.944 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.944 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.944 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.944 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.944 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.944 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.944 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.944 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.944 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.944 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.944 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.946 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.946 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.946 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.946 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.946 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.946 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.946 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.946 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.946 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.946 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.946 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.946 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.946 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.947 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.947 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.947 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.947 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "nfs", 2024-09-19T08:25:57.947 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.947 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.947 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.947 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.947 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.947 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.947 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.947 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.948 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.948 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.948 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.948 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.948 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.948 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.948 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.948 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.948 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.948 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.948 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.948 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.948 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.949 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.949 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.949 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.949 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.949 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.949 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.949 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.949 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.949 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.949 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.949 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.949 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.949 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.949 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.950 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.950 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.950 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.950 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.950 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.950 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.950 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.950 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.950 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.950 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.950 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.950 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.950 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.951 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.951 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.951 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.951 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.951 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.951 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.951 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.951 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.951 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.951 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.951 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.951 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.952 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.952 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.952 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.952 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.952 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.952 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.952 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.952 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.952 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.952 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.952 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.952 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.952 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.953 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.953 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-09-19T08:25:57.953 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.953 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.953 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.953 INFO:teuthology.orchestra.run.smithi022.stdout: "fail_fs": { 2024-09-19T08:25:57.953 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "fail_fs", 2024-09-19T08:25:57.953 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.953 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.953 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.953 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.953 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.953 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.953 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.954 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T08:25:57.954 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.954 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.954 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.954 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.954 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.954 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.954 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.954 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.954 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.954 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.954 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.954 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.955 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.956 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.956 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.956 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.956 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.956 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.956 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.956 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.956 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.956 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.956 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.956 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.956 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.957 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.957 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.957 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.957 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.957 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.957 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.957 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.957 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.957 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.957 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.957 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.957 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.957 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.957 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.958 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.958 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.958 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.958 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.958 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.958 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.958 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.958 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.958 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.958 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.958 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.958 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.958 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.958 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.959 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.959 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.959 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.959 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.959 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.959 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator": { 2024-09-19T08:25:57.959 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-09-19T08:25:57.959 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.959 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.959 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.959 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.959 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.959 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.960 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.960 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2024-09-19T08:25:57.960 INFO:teuthology.orchestra.run.smithi022.stdout: "rook", 2024-09-19T08:25:57.960 INFO:teuthology.orchestra.run.smithi022.stdout: "test_orchestrator" 2024-09-19T08:25:57.960 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.960 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Orchestrator backend", 2024-09-19T08:25:57.960 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.960 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.960 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.960 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.960 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.960 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.961 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_perf_query", 2024-09-19T08:25:57.961 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.961 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.961 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.961 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.961 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.961 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.961 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.961 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.961 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.961 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.961 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.961 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.962 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.962 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.962 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.962 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.962 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.962 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.962 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.962 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.962 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.962 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.962 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.962 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.962 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.962 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.963 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.963 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.963 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.963 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.963 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.963 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.963 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.963 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.963 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.963 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.963 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.963 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.963 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.964 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.964 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.964 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.964 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.964 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.964 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.964 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.964 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.964 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.964 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.964 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.964 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.964 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.965 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.965 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.965 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.965 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.965 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.965 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.965 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.965 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.965 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.965 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.965 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.965 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.965 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.965 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.966 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.966 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.966 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.966 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.966 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.966 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.966 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.966 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.966 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.966 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.966 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_support", 2024-09-19T08:25:57.966 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.966 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.967 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.967 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.967 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.967 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.967 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.967 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.967 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.967 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.967 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.967 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.967 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.967 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.967 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.967 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.968 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.968 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.968 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.968 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.968 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.968 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.968 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.968 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.968 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.968 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.968 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.968 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.968 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.968 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.969 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.969 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.969 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.969 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.969 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.969 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.969 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.969 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.969 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.969 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.969 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.969 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.969 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.969 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.970 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.970 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.970 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.970 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.970 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.970 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.970 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.970 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.970 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.970 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.970 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.970 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.970 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.971 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.971 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.971 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.971 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.971 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.971 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.971 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.971 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.971 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.971 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.971 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.971 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.971 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.971 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.972 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.972 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.972 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.972 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.972 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_autoscaler", 2024-09-19T08:25:57.972 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.972 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.972 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.972 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.972 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.972 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.972 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.973 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.973 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.973 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.973 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.973 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.973 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.973 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.973 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.973 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.973 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.973 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.973 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.973 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.974 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.974 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.974 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.974 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.974 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.974 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.974 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.974 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.974 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.974 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.974 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.974 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.974 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.974 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.975 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.975 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.975 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.975 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.975 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.975 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.975 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.975 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.975 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.975 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.975 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.975 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.976 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.976 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.976 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.976 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.976 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.976 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.976 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.976 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.976 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.976 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.976 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.976 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.976 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.976 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.977 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.977 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.977 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.977 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.977 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.977 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.977 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.977 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.977 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.977 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.977 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.977 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.977 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:57.978 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:57.978 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.978 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.978 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.978 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:57.978 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.978 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.978 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.978 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.978 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.978 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.978 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.978 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.979 INFO:teuthology.orchestra.run.smithi022.stdout: "threshold": { 2024-09-19T08:25:57.979 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threshold", 2024-09-19T08:25:57.979 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:57.979 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.979 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.979 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-09-19T08:25:57.979 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1.0", 2024-09-19T08:25:57.979 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.979 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.979 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "scaling threshold", 2024-09-19T08:25:57.979 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T08:25:57.979 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.979 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.979 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "progress", 2024-09-19T08:25:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_pg_recovery_event": { 2024-09-19T08:25:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_pg_recovery_event", 2024-09-19T08:25:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.980 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.981 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.981 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.981 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T08:25:57.981 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.981 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.981 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.981 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.981 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-09-19T08:25:57.981 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-09-19T08:25:57.981 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.981 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.981 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.981 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.982 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.982 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.982 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.982 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.982 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.982 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.982 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.982 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.982 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.982 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.982 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.982 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.983 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.983 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.983 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.983 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.983 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.983 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.983 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.983 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.983 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.983 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.983 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.983 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.983 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.983 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.984 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.984 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.984 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.984 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.984 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.984 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.984 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.984 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.984 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.984 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.984 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.984 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.984 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.985 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.985 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.985 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.985 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.985 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.985 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.985 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.985 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.985 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.985 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.985 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.985 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.985 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.986 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.986 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.986 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.986 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.986 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.986 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.986 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.986 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.986 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.986 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.986 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.986 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.986 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.986 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.987 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.987 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.987 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.987 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.987 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.987 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.987 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.987 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.987 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.987 INFO:teuthology.orchestra.run.smithi022.stdout: "max_completed_events": { 2024-09-19T08:25:57.987 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_completed_events", 2024-09-19T08:25:57.987 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.987 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.988 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.988 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "50", 2024-09-19T08:25:57.988 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.988 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.988 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.988 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "number of past completed events to remember", 2024-09-19T08:25:57.988 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.988 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.988 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.988 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:57.988 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:57.988 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:57.989 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.989 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.989 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:57.989 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.989 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.989 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.989 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long the module is going to sleep", 2024-09-19T08:25:57.989 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.989 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.989 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.989 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.989 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:57.990 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.990 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:57.990 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus", 2024-09-19T08:25:57.990 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:57.990 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:57.990 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:57.990 INFO:teuthology.orchestra.run.smithi022.stdout: "cache": { 2024-09-19T08:25:57.990 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cache", 2024-09-19T08:25:57.990 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.990 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.990 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.990 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.990 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.990 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.991 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.991 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.991 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.991 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.991 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.991 INFO:teuthology.orchestra.run.smithi022.stdout: "exclude_perf_counters": { 2024-09-19T08:25:57.991 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "exclude_perf_counters", 2024-09-19T08:25:57.991 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.991 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.991 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.991 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:57.991 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.991 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.992 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.992 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T08:25:57.992 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T08:25:57.992 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.992 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.992 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:57.992 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:57.992 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.992 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.992 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.992 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.992 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.993 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.993 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.993 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.993 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.993 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.993 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.993 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.993 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.993 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.993 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.993 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.993 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.994 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.994 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.994 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:57.994 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:57.994 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.994 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.994 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.994 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.994 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.994 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.994 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.994 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.994 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.995 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.995 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.995 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.995 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:57.995 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:57.995 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.995 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.995 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.995 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:57.995 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.995 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.995 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:57.995 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:57.996 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:57.996 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:57.996 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:57.996 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:57.996 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:57.996 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:57.996 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.996 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.996 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.996 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.996 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:57.996 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:57.997 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:57.997 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.997 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:57.997 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:57.997 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.997 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.997 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.997 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.997 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.997 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.997 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.997 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools": { 2024-09-19T08:25:57.997 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools", 2024-09-19T08:25:57.998 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:57.998 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.998 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.998 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:57.998 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.998 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.998 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.998 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.998 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.998 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.998 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.998 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T08:25:57.999 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T08:25:57.999 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:57.999 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:57.999 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:57.999 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300", 2024-09-19T08:25:57.999 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:57.999 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:57.999 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:57.999 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:57.999 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:57.999 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:57.999 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:57.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:57.999 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_interval": { 2024-09-19T08:25:58.000 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_interval", 2024-09-19T08:25:58.000 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:58.000 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.000 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.000 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15.0", 2024-09-19T08:25:58.000 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.000 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.000 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.000 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.000 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.000 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.000 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.001 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-09-19T08:25:58.001 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-09-19T08:25:58.001 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.001 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.001 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.001 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-09-19T08:25:58.001 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.001 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.001 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.001 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T08:25:58.001 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.001 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.001 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.001 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.002 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-09-19T08:25:58.002 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-09-19T08:25:58.002 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.002 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.002 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.002 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "9283", 2024-09-19T08:25:58.002 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.002 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.002 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.002 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T08:25:58.002 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.002 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.002 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.003 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.003 INFO:teuthology.orchestra.run.smithi022.stdout: "stale_cache_strategy": { 2024-09-19T08:25:58.003 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stale_cache_strategy", 2024-09-19T08:25:58.003 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.003 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.003 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.003 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "log", 2024-09-19T08:25:58.003 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.003 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.003 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.003 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.003 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.003 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.003 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.004 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-09-19T08:25:58.004 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-09-19T08:25:58.004 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.004 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.004 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.004 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "default", 2024-09-19T08:25:58.004 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.004 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.004 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.004 INFO:teuthology.orchestra.run.smithi022.stdout: "default", 2024-09-19T08:25:58.004 INFO:teuthology.orchestra.run.smithi022.stdout: "error" 2024-09-19T08:25:58.004 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.005 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.005 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.005 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.005 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.005 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-09-19T08:25:58.005 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-09-19T08:25:58.005 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.005 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.005 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.005 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-09-19T08:25:58.005 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-09-19T08:25:58.005 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-09-19T08:25:58.005 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.006 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.006 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.006 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.006 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.006 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.006 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.006 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.006 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2024-09-19T08:25:58.006 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.006 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.006 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.006 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.006 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.007 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.007 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.007 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.007 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.007 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.007 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.007 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.007 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.007 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.007 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.007 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.007 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.007 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.007 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.008 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.008 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.008 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.008 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.008 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.008 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.008 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.008 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.008 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.008 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.008 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.008 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.009 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.009 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.009 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.009 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.009 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.009 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.009 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.009 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.009 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.009 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.009 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.009 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.009 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.010 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.010 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.010 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.010 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.010 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.010 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.010 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.010 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.010 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.010 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.010 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.010 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.010 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_snap_create": { 2024-09-19T08:25:58.011 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_snap_create", 2024-09-19T08:25:58.012 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.012 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.012 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.012 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:58.012 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.012 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.012 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.012 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.012 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.012 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.012 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.012 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.012 INFO:teuthology.orchestra.run.smithi022.stdout: "mirror_snapshot_schedule": { 2024-09-19T08:25:58.013 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T08:25:58.013 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.013 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.013 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.013 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.013 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.013 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.013 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.013 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.013 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.013 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.013 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.013 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.013 INFO:teuthology.orchestra.run.smithi022.stdout: "trash_purge_schedule": { 2024-09-19T08:25:58.014 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "trash_purge_schedule", 2024-09-19T08:25:58.014 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.014 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.014 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.014 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.014 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.014 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.014 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.014 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.014 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.014 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.014 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.014 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.014 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.015 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.015 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.015 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "restful", 2024-09-19T08:25:58.015 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.015 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.015 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.015 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_auth": { 2024-09-19T08:25:58.015 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_auth", 2024-09-19T08:25:58.015 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.015 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.015 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.015 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.015 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.015 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.016 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.016 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.016 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.016 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.016 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.016 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.016 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-09-19T08:25:58.016 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-09-19T08:25:58.016 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.016 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.016 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.016 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.016 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.016 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.017 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.017 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.017 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.017 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.017 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.017 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.017 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.017 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.017 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.017 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.017 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.017 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.017 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.018 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.018 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.018 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.018 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.018 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.018 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.018 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.018 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.018 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.018 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.018 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.018 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.018 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.019 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.019 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.019 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.019 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.019 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.019 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.019 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.019 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.019 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.019 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.019 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.019 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.019 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.019 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.020 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.020 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.020 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.020 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.020 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.020 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.020 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.020 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.020 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.020 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.020 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.020 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.020 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.020 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.021 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.021 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.021 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.021 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.021 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.021 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.021 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.021 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.021 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.021 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.021 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.021 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.022 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.022 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.022 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.022 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.022 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.022 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.022 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.022 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.022 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-09-19T08:25:58.022 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-09-19T08:25:58.022 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.022 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.022 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.022 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-09-19T08:25:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-09-19T08:25:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rgw", 2024-09-19T08:25:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.025 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.025 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.025 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.025 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.025 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.025 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.025 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.025 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.025 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.025 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.025 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.025 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.025 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.025 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.026 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.027 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.027 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.027 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.027 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.027 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.027 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.027 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.027 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.027 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.027 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.027 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.027 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.027 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.028 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.028 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.028 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.028 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.028 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.028 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.028 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.028 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.028 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.028 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.028 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.028 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.028 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.029 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.029 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.029 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.029 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.029 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.029 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.029 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.029 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.029 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.029 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.029 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.029 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.029 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.030 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.030 INFO:teuthology.orchestra.run.smithi022.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T08:25:58.030 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T08:25:58.030 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.030 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.030 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.030 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:58.030 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.030 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.030 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.030 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T08:25:58.030 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.030 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.031 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.031 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.031 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.031 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rook", 2024-09-19T08:25:58.031 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.031 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.031 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.031 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.031 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.031 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.031 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.032 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.032 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.032 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.032 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.032 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.032 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.032 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.032 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.032 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.032 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.032 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.032 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.032 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.032 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.033 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.033 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.033 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.033 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.033 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.033 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.033 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.033 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.033 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.033 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.033 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.033 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.034 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.034 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.034 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.034 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.034 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.034 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.034 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.034 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.034 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.034 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.034 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.034 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.034 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.035 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.035 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.035 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.035 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.035 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.035 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.035 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.035 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.035 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.035 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.035 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.035 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.035 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.036 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.036 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.036 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.036 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.036 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.036 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.036 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.036 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.036 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.036 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.036 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.036 INFO:teuthology.orchestra.run.smithi022.stdout: "storage_class": { 2024-09-19T08:25:58.036 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "storage_class", 2024-09-19T08:25:58.037 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.037 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.037 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.037 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "local", 2024-09-19T08:25:58.037 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.037 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.037 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.037 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T08:25:58.037 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.037 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.037 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.037 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.037 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.037 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.038 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.038 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "selftest", 2024-09-19T08:25:58.038 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.038 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.038 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.038 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.038 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.038 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.038 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.038 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.038 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.038 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.038 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.038 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.039 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.039 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.039 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.039 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.039 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.039 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.039 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.039 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.039 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.039 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.039 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.039 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.039 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.040 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.040 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.040 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.040 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.040 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.040 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.040 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.040 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.040 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.040 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.040 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.040 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.041 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.041 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.041 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.041 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.041 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.041 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.041 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.041 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.041 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.041 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.041 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.041 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.041 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.041 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.042 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.042 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.042 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.042 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.042 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.042 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.042 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.042 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.042 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.042 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.042 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.042 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.042 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.043 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.043 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.043 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.043 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.043 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.043 INFO:teuthology.orchestra.run.smithi022.stdout: "roption1": { 2024-09-19T08:25:58.043 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption1", 2024-09-19T08:25:58.043 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.043 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.043 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.043 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.043 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.043 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.044 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.044 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.044 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.044 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.044 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.044 INFO:teuthology.orchestra.run.smithi022.stdout: "roption2": { 2024-09-19T08:25:58.044 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption2", 2024-09-19T08:25:58.044 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.044 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.044 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.044 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "xyz", 2024-09-19T08:25:58.044 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.044 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.045 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.045 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.045 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.045 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.045 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.045 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption1": { 2024-09-19T08:25:58.045 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption1", 2024-09-19T08:25:58.045 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.045 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.045 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.045 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.045 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.046 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.046 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.046 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.046 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.046 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.046 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.046 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption2": { 2024-09-19T08:25:58.046 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption2", 2024-09-19T08:25:58.046 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.046 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.046 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.046 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.046 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.047 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.047 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.047 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.047 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.047 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.047 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.047 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption3": { 2024-09-19T08:25:58.047 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption3", 2024-09-19T08:25:58.047 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:58.047 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.047 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.047 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.047 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.048 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.048 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.048 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.048 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.048 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.048 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.048 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption4": { 2024-09-19T08:25:58.048 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption4", 2024-09-19T08:25:58.048 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.048 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.048 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.048 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.049 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.049 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.049 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.049 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.049 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.049 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.049 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.049 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption5": { 2024-09-19T08:25:58.049 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption5", 2024-09-19T08:25:58.049 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.049 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.049 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.049 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.050 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.050 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.050 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.050 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.050 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.050 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.050 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.050 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption6": { 2024-09-19T08:25:58.050 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption6", 2024-09-19T08:25:58.050 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.050 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.050 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.050 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.051 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.051 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.051 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.051 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.051 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.051 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.051 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.051 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption7": { 2024-09-19T08:25:58.051 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption7", 2024-09-19T08:25:58.051 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.051 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.051 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.051 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.052 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-09-19T08:25:58.052 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "42", 2024-09-19T08:25:58.052 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.052 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.052 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.052 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.052 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.052 INFO:teuthology.orchestra.run.smithi022.stdout: "testkey": { 2024-09-19T08:25:58.052 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testkey", 2024-09-19T08:25:58.052 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.052 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.052 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: "testlkey": { 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testlkey", 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.053 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.054 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.054 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.054 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.054 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.054 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.054 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.054 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.054 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.054 INFO:teuthology.orchestra.run.smithi022.stdout: "testnewline": { 2024-09-19T08:25:58.054 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testnewline", 2024-09-19T08:25:58.054 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.054 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.054 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.054 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.055 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.055 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.055 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.055 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.055 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.055 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.055 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.055 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.055 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.055 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.055 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-09-19T08:25:58.055 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.055 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.056 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.056 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_m_granularity": { 2024-09-19T08:25:58.056 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_m_granularity", 2024-09-19T08:25:58.056 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.056 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.056 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.056 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.056 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.056 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.056 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.056 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T08:25:58.056 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.056 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.056 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.057 INFO:teuthology.orchestra.run.smithi022.stdout: "dump_on_update": { 2024-09-19T08:25:58.057 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dump_on_update", 2024-09-19T08:25:58.057 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.057 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.057 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.057 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.057 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.057 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.057 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.057 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "dump database to debug log on update", 2024-09-19T08:25:58.057 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.057 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.057 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.058 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.058 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.058 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.058 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.058 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.058 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.058 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.058 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.058 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.058 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.058 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.058 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.058 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.059 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.059 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.059 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.059 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.059 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.059 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.059 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.059 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.059 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.059 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.059 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.059 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.060 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.060 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.060 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.060 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.060 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.060 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.060 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.060 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.060 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.060 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.060 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.060 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.061 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.061 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.061 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.061 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.061 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.061 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.061 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.061 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.061 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.061 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.061 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.061 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.061 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.062 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.062 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.062 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.062 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.062 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.062 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.062 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.062 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.062 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.062 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.062 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.062 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.062 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.063 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.063 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.063 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.063 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.063 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.063 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.063 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.063 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.063 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stats", 2024-09-19T08:25:58.063 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.063 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.063 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.063 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.063 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.064 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.064 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.064 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.064 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.064 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.064 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.064 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.064 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.064 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.064 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.064 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.064 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.064 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.065 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.065 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.065 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.065 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.065 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.065 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.065 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.065 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.065 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.065 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.065 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.065 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.065 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.065 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.066 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.066 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.066 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.066 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.066 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.066 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.066 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.066 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.066 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.066 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.066 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.066 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.066 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.066 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.067 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.067 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.067 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.067 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.067 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.067 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.067 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.067 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.067 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.067 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.067 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.067 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.067 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.067 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.068 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.068 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.068 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.068 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.068 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.068 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.068 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.068 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.068 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.068 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.068 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.068 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.068 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.068 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "status", 2024-09-19T08:25:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.069 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.070 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.071 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.071 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.071 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.071 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.071 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.071 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.071 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.071 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.071 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.071 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.071 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.071 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.071 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.072 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.072 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.072 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.072 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.072 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.072 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.072 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.072 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.072 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.072 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.072 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.072 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.072 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.072 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.073 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.073 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.073 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.073 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.073 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.073 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.073 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.073 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.073 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.073 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.073 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.073 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.073 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.074 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.074 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.074 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.074 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.074 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.074 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.074 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.074 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.074 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.074 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telegraf", 2024-09-19T08:25:58.074 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.074 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.074 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.074 INFO:teuthology.orchestra.run.smithi022.stdout: "address": { 2024-09-19T08:25:58.075 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "address", 2024-09-19T08:25:58.075 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.075 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.075 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.075 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T08:25:58.075 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.075 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.075 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.075 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.075 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.075 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.075 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.075 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.075 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:58.076 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:58.076 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.076 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.076 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.076 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15", 2024-09-19T08:25:58.076 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.076 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.076 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.076 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.076 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.076 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.076 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.077 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.077 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.077 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.077 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.077 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.077 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.077 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.077 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.077 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.077 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.077 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.077 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.077 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.077 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.078 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.078 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.078 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.078 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.078 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.078 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.078 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.078 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.078 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.078 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.078 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.078 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.078 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.078 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.079 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.079 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.079 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.079 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.079 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.079 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.079 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.079 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.079 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.079 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.079 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.079 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.079 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.080 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.080 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.080 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.080 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.080 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.080 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.080 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.080 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.080 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.080 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.080 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.080 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.080 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.080 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telemetry", 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_basic": { 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_basic", 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.082 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.083 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.083 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.083 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.083 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T08:25:58.083 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.083 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.083 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.083 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.083 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_crash": { 2024-09-19T08:25:58.083 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_crash", 2024-09-19T08:25:58.083 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.083 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.083 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.083 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.084 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.084 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.084 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.084 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T08:25:58.084 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.084 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.084 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.084 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_device": { 2024-09-19T08:25:58.084 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_device", 2024-09-19T08:25:58.084 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.084 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.084 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.084 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.085 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.085 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.085 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.085 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T08:25:58.085 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.085 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.085 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.085 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.085 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_ident": { 2024-09-19T08:25:58.085 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_ident", 2024-09-19T08:25:58.085 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.085 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.085 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.086 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.086 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.086 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.086 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.086 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T08:25:58.086 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.086 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.086 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.086 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_perf": { 2024-09-19T08:25:58.086 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_perf", 2024-09-19T08:25:58.086 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.086 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.086 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.087 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.087 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.087 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.087 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.087 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T08:25:58.087 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.087 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.087 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.087 INFO:teuthology.orchestra.run.smithi022.stdout: "contact": { 2024-09-19T08:25:58.087 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "contact", 2024-09-19T08:25:58.087 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.088 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.088 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.088 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.088 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.088 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.088 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.088 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.088 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.088 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.088 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.088 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.088 INFO:teuthology.orchestra.run.smithi022.stdout: "description": { 2024-09-19T08:25:58.088 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "description", 2024-09-19T08:25:58.089 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.089 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.089 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.089 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.089 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.089 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.089 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.089 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.089 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.089 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.089 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.089 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.089 INFO:teuthology.orchestra.run.smithi022.stdout: "device_url": { 2024-09-19T08:25:58.089 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_url", 2024-09-19T08:25:58.090 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.090 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.090 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.090 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T08:25:58.090 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.090 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.090 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.090 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.090 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.090 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.090 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.090 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.090 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-09-19T08:25:58.091 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-09-19T08:25:58.091 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.091 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.091 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.091 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.091 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.091 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.091 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.091 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.091 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.091 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.091 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.091 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.091 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:58.092 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:58.092 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.092 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.092 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.092 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "24", 2024-09-19T08:25:58.092 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "8", 2024-09-19T08:25:58.092 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.092 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.092 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.092 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.092 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.092 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.092 INFO:teuthology.orchestra.run.smithi022.stdout: "last_opt_revision": { 2024-09-19T08:25:58.093 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "last_opt_revision", 2024-09-19T08:25:58.093 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.093 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.093 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.093 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1", 2024-09-19T08:25:58.093 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.093 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.093 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.093 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.093 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.093 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.093 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.093 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.093 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard": { 2024-09-19T08:25:58.094 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard", 2024-09-19T08:25:58.094 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.094 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.094 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.094 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.094 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.094 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.094 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.094 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.094 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.094 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.094 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.094 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.094 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard_description": { 2024-09-19T08:25:58.095 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard_description", 2024-09-19T08:25:58.095 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.095 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.095 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.095 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.095 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.095 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.095 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.095 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.095 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.095 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.095 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.095 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.095 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.096 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.096 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.096 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.096 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.096 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.096 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.096 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.096 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.096 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.096 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.096 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.096 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.096 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.097 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.098 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.098 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.098 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.098 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.098 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.098 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.098 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.098 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.098 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.098 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.098 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.098 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.098 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.099 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.099 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.099 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.099 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.099 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.099 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.099 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.099 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.099 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.099 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.099 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.099 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.099 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.099 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.100 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.100 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.100 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.100 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.100 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.100 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.100 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.100 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.100 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.100 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.100 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.100 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.100 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.100 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.101 INFO:teuthology.orchestra.run.smithi022.stdout: "organization": { 2024-09-19T08:25:58.101 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "organization", 2024-09-19T08:25:58.101 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.101 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.101 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.101 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.101 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.101 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.101 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.101 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.101 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.101 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.101 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.102 INFO:teuthology.orchestra.run.smithi022.stdout: "proxy": { 2024-09-19T08:25:58.102 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "proxy", 2024-09-19T08:25:58.102 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.102 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.102 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.102 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.102 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.102 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.102 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.102 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.102 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.102 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.102 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.103 INFO:teuthology.orchestra.run.smithi022.stdout: "url": { 2024-09-19T08:25:58.103 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url", 2024-09-19T08:25:58.103 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.103 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.103 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.103 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T08:25:58.103 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.103 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.103 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.103 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.103 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.103 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.103 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.104 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.104 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.104 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.104 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.104 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "test_orchestrator", 2024-09-19T08:25:58.104 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.104 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.104 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.104 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.104 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.104 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.104 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.104 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.104 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.105 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.105 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.105 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.105 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.105 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.105 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.105 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.105 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.105 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.105 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.105 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.105 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.105 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.106 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.106 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.106 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.106 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.106 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.106 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.106 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.106 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.106 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.106 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.106 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.106 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.106 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.107 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.107 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.107 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.107 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.107 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.107 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.107 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.107 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.107 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.107 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.107 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.107 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.107 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.108 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.108 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.108 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.108 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.108 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.108 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.108 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.108 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.108 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.108 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.108 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.108 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.108 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.109 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.109 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.109 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.109 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.109 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.109 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.109 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.109 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.109 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.109 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.109 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.109 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2024-09-19T08:25:58.110 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.110 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.110 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.110 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.110 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.110 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.110 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.110 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.110 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.110 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.110 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.110 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.110 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.110 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.111 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.111 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.111 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.111 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.111 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.111 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.111 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.111 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.111 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.111 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.111 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.111 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.111 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.111 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.112 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.112 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.112 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.112 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.112 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.112 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.112 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.112 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.112 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.112 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.112 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.112 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.112 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.113 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.113 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.113 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.113 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.113 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.113 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.113 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.113 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.113 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.113 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.113 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.113 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.113 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.113 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.114 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.114 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.114 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.114 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.114 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.114 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.114 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.114 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.114 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.114 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.114 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.114 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.114 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.115 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.115 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.115 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.115 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_clones": { 2024-09-19T08:25:58.115 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_clones", 2024-09-19T08:25:58.115 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.115 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.115 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.115 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4", 2024-09-19T08:25:58.115 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.115 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.115 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.115 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T08:25:58.116 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.116 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.116 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.116 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.116 INFO:teuthology.orchestra.run.smithi022.stdout: "periodic_async_work": { 2024-09-19T08:25:58.116 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "periodic_async_work", 2024-09-19T08:25:58.116 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.116 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.116 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.116 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.116 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.116 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.116 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.116 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Periodically check for async work", 2024-09-19T08:25:58.117 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.117 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.117 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.117 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_delay": { 2024-09-19T08:25:58.117 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_delay", 2024-09-19T08:25:58.117 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.117 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.117 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.117 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:58.117 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.117 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.117 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.117 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T08:25:58.118 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.118 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.118 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.118 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_no_wait": { 2024-09-19T08:25:58.118 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T08:25:58.118 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.118 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.118 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.118 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.118 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.118 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.118 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.119 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T08:25:58.119 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.119 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.119 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.119 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.119 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.119 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.119 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix", 2024-09-19T08:25:58.119 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.119 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.119 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.119 INFO:teuthology.orchestra.run.smithi022.stdout: "discovery_interval": { 2024-09-19T08:25:58.120 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "discovery_interval", 2024-09-19T08:25:58.120 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:58.120 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.120 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.120 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "100", 2024-09-19T08:25:58.120 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.120 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.120 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.120 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.120 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.120 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.120 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.121 INFO:teuthology.orchestra.run.smithi022.stdout: "identifier": { 2024-09-19T08:25:58.121 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "identifier", 2024-09-19T08:25:58.121 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.121 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.121 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.121 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.121 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.121 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.121 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.121 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.121 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.121 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.121 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.122 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:58.122 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:58.122 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.122 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.122 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.122 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:58.122 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.122 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.122 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.122 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.122 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.122 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.122 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.123 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.123 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.123 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.123 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.123 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.123 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.123 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.123 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.123 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.123 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.123 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.123 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.123 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.124 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.124 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.124 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.124 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.124 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.124 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.124 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.124 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.124 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.124 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.124 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.124 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.125 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.125 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.125 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.125 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.125 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.125 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.125 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.125 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.125 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.125 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.125 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.125 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.126 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.126 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.126 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.126 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.126 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.126 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.126 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.126 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.126 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.126 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.126 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.126 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.126 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.126 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.128 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.128 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.128 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_host": { 2024-09-19T08:25:58.128 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_host", 2024-09-19T08:25:58.128 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.128 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.128 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.128 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.128 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.128 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.128 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.128 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.128 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.129 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.129 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.129 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_port": { 2024-09-19T08:25:58.133 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_port", 2024-09-19T08:25:58.133 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.133 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.133 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.133 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10051", 2024-09-19T08:25:58.133 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.133 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.134 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.134 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.134 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.134 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.134 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.134 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_sender": { 2024-09-19T08:25:58.134 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_sender", 2024-09-19T08:25:58.134 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.134 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.134 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.134 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T08:25:58.134 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.134 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.135 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.135 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.135 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.135 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.135 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.135 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.135 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.135 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.135 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-19T08:25:58.135 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.135 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.135 INFO:teuthology.orchestra.run.smithi022.stdout: "modules": [ 2024-09-19T08:25:58.135 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2024-09-19T08:25:58.136 INFO:teuthology.orchestra.run.smithi022.stdout: "dashboard", 2024-09-19T08:25:58.136 INFO:teuthology.orchestra.run.smithi022.stdout: "iostat", 2024-09-19T08:25:58.136 INFO:teuthology.orchestra.run.smithi022.stdout: "nfs", 2024-09-19T08:25:58.136 INFO:teuthology.orchestra.run.smithi022.stdout: "restful", 2024-09-19T08:25:58.136 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_schedule" 2024-09-19T08:25:58.136 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.136 INFO:teuthology.orchestra.run.smithi022.stdout: "available_modules": [ 2024-09-19T08:25:58.136 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.136 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "alerts", 2024-09-19T08:25:58.136 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.136 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.136 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.136 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:58.136 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:58.137 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.137 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.137 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.137 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:58.137 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.137 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.137 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.137 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T08:25:58.137 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.137 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.137 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.137 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.137 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.138 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.138 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.138 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.138 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.138 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.138 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.138 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.138 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.138 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.138 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.138 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.138 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.138 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.138 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.139 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.139 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.139 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.139 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.139 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.139 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.139 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.139 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.139 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.139 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.139 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.139 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.139 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.140 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.140 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.140 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.140 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.140 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.140 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.140 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.140 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.140 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.140 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.140 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.140 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.140 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.142 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.142 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.142 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.142 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.142 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.142 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.142 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.142 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.142 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.142 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.142 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.142 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.142 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.143 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_destination": { 2024-09-19T08:25:58.143 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_destination", 2024-09-19T08:25:58.143 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.143 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.143 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.143 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.143 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.143 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.143 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.143 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email address to send alerts to", 2024-09-19T08:25:58.143 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.143 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.143 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.144 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_from_name": { 2024-09-19T08:25:58.144 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_from_name", 2024-09-19T08:25:58.144 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.144 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.144 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.144 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "Ceph", 2024-09-19T08:25:58.144 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.144 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.144 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.144 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email From: name", 2024-09-19T08:25:58.144 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.144 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.145 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.145 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.145 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_host": { 2024-09-19T08:25:58.145 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_host", 2024-09-19T08:25:58.145 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.145 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.145 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.145 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.145 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.145 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.145 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.145 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP server", 2024-09-19T08:25:58.145 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.146 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.146 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.146 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_password": { 2024-09-19T08:25:58.146 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_password", 2024-09-19T08:25:58.146 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.146 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.146 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.146 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.146 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.146 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.146 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.146 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Password to authenticate with", 2024-09-19T08:25:58.147 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.147 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.147 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.147 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_port": { 2024-09-19T08:25:58.147 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_port", 2024-09-19T08:25:58.147 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.147 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.147 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.147 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "465", 2024-09-19T08:25:58.147 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.147 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.147 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.147 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP port", 2024-09-19T08:25:58.148 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.148 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.148 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.148 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_sender": { 2024-09-19T08:25:58.148 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_sender", 2024-09-19T08:25:58.148 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.148 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.148 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.148 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.148 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.148 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.148 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.149 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP envelope sender", 2024-09-19T08:25:58.149 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.149 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.149 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.149 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_ssl": { 2024-09-19T08:25:58.149 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_ssl", 2024-09-19T08:25:58.149 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.149 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.149 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.149 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.149 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.149 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.149 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.150 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T08:25:58.150 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.150 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.150 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.150 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_user": { 2024-09-19T08:25:58.150 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_user", 2024-09-19T08:25:58.150 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.150 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.150 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.150 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.150 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.150 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.151 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.151 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "User to authenticate as", 2024-09-19T08:25:58.151 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.151 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.151 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.151 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.151 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.151 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.151 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "balancer", 2024-09-19T08:25:58.151 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.151 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.152 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.152 INFO:teuthology.orchestra.run.smithi022.stdout: "active": { 2024-09-19T08:25:58.152 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "active", 2024-09-19T08:25:58.152 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.152 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.152 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.152 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.152 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.152 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.152 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.152 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T08:25:58.152 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.152 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.152 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.153 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.153 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_time": { 2024-09-19T08:25:58.153 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_time", 2024-09-19T08:25:58.153 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.153 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.153 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.153 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0000", 2024-09-19T08:25:58.153 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.153 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.153 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.153 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T08:25:58.153 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:25:58.153 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.154 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.154 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.154 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_weekday": { 2024-09-19T08:25:58.154 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_weekday", 2024-09-19T08:25:58.154 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:58.154 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.154 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.154 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:58.154 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-09-19T08:25:58.154 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-09-19T08:25:58.154 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.154 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T08:25:58.154 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:25:58.154 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.155 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.155 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_max_iterations": { 2024-09-19T08:25:58.155 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_max_iterations", 2024-09-19T08:25:58.155 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:58.155 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.155 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.155 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "25", 2024-09-19T08:25:58.155 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-09-19T08:25:58.155 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "250", 2024-09-19T08:25:58.155 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.155 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T08:25:58.155 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.155 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.156 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.156 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_metrics": { 2024-09-19T08:25:58.156 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_metrics", 2024-09-19T08:25:58.156 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.156 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.156 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.156 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T08:25:58.156 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.156 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.156 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.156 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T08:25:58.156 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T08:25:58.157 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.157 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.157 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_step": { 2024-09-19T08:25:58.157 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_step", 2024-09-19T08:25:58.157 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:58.157 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.157 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.157 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.5", 2024-09-19T08:25:58.157 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0.001", 2024-09-19T08:25:58.157 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "0.999", 2024-09-19T08:25:58.157 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.157 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "aggressiveness of optimization", 2024-09-19T08:25:58.157 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T08:25:58.158 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.158 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.158 INFO:teuthology.orchestra.run.smithi022.stdout: "end_time": { 2024-09-19T08:25:58.158 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_time", 2024-09-19T08:25:58.158 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.158 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.158 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.158 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2359", 2024-09-19T08:25:58.158 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.158 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.158 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.158 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T08:25:58.159 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:25:58.159 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.159 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.159 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.159 INFO:teuthology.orchestra.run.smithi022.stdout: "end_weekday": { 2024-09-19T08:25:58.159 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_weekday", 2024-09-19T08:25:58.159 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:58.159 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.159 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.159 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:58.159 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-09-19T08:25:58.159 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-09-19T08:25:58.159 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.159 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T08:25:58.160 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:25:58.160 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.160 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.160 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.160 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.160 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.160 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.160 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.160 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.160 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.160 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.160 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.160 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.160 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.161 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.161 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.161 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.161 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.161 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.161 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.161 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.161 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.161 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.161 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.161 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.161 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.161 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.161 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.162 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.162 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.162 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.162 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.162 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.162 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.162 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.162 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.162 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.162 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.162 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.162 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.162 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.163 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.163 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.163 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.163 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.163 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.163 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.163 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.163 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.163 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.163 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.163 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.163 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.163 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.163 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.164 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.164 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.164 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.164 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.164 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.164 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.164 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.164 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.164 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.164 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.164 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.164 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.164 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.165 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.165 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.165 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.165 INFO:teuthology.orchestra.run.smithi022.stdout: "min_score": { 2024-09-19T08:25:58.165 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_score", 2024-09-19T08:25:58.165 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:58.165 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.165 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.165 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:58.165 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.165 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.165 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.165 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T08:25:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-09-19T08:25:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-09-19T08:25:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "upmap", 2024-09-19T08:25:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.166 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-compat", 2024-09-19T08:25:58.167 INFO:teuthology.orchestra.run.smithi022.stdout: "none", 2024-09-19T08:25:58.167 INFO:teuthology.orchestra.run.smithi022.stdout: "read", 2024-09-19T08:25:58.167 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap", 2024-09-19T08:25:58.167 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap-read" 2024-09-19T08:25:58.167 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.167 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Balancer mode", 2024-09-19T08:25:58.167 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.167 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.167 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.167 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_ids": { 2024-09-19T08:25:58.167 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_ids", 2024-09-19T08:25:58.167 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T08:25:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:58.168 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.169 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.169 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.169 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:58.169 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.169 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.169 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.169 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T08:25:58.169 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.169 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.169 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.169 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_deviation": { 2024-09-19T08:25:58.169 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_deviation", 2024-09-19T08:25:58.170 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.170 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.170 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.170 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:58.170 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-09-19T08:25:58.170 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.170 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.170 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T08:25:58.170 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T08:25:58.170 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.170 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.170 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_optimizations": { 2024-09-19T08:25:58.170 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_optimizations", 2024-09-19T08:25:58.171 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:58.171 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.171 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.171 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:58.171 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.171 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.171 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.171 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T08:25:58.171 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.171 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.171 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.171 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.171 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.172 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm", 2024-09-19T08:25:58.172 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.172 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.172 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.172 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_down_multiplier": { 2024-09-19T08:25:58.172 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_down_multiplier", 2024-09-19T08:25:58.172 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:58.172 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.172 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.172 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-09-19T08:25:58.172 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.172 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.173 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.173 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T08:25:58.173 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.173 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.173 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.173 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.173 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_refresh_rate": { 2024-09-19T08:25:58.173 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_refresh_rate", 2024-09-19T08:25:58.173 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.173 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.173 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.173 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "20", 2024-09-19T08:25:58.173 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.174 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.174 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.174 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T08:25:58.174 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.174 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.174 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.174 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_starting_port": { 2024-09-19T08:25:58.174 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_starting_port", 2024-09-19T08:25:58.174 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.174 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.174 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.174 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4721", 2024-09-19T08:25:58.174 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.175 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.175 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.175 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T08:25:58.175 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.175 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.175 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.175 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_ptrace": { 2024-09-19T08:25:58.175 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_ptrace", 2024-09-19T08:25:58.175 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.175 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.175 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.175 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.175 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.176 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.176 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.176 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T08:25:58.176 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T08:25:58.176 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.176 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.176 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.176 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_interval": { 2024-09-19T08:25:58.176 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_interval", 2024-09-19T08:25:58.176 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.176 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.176 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.176 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:58.177 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.177 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.177 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.177 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T08:25:58.177 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.177 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.177 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.177 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.177 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_memory_target_ratio": { 2024-09-19T08:25:58.177 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T08:25:58.177 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:58.177 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.177 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.177 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.7", 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm_log_destination": { 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm_log_destination", 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.178 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: "file", 2024-09-19T08:25:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: "file,syslog", 2024-09-19T08:25:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: "syslog" 2024-09-19T08:25:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T08:25:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: "cgroups_split": { 2024-09-19T08:25:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cgroups_split", 2024-09-19T08:25:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.180 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.180 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.180 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.180 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.180 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.180 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.180 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T08:25:58.180 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.180 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.180 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.180 INFO:teuthology.orchestra.run.smithi022.stdout: "config_checks_enabled": { 2024-09-19T08:25:58.180 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_checks_enabled", 2024-09-19T08:25:58.181 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.181 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.181 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.181 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.181 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.181 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.181 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.181 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T08:25:58.181 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.181 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.181 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.181 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.181 INFO:teuthology.orchestra.run.smithi022.stdout: "config_dashboard": { 2024-09-19T08:25:58.181 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_dashboard", 2024-09-19T08:25:58.182 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.182 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.182 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.182 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.182 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.182 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.182 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.182 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T08:25:58.182 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.182 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.182 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.182 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_alertmanager": { 2024-09-19T08:25:58.183 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_alertmanager", 2024-09-19T08:25:58.183 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.183 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.183 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.183 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T08:25:58.183 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.183 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.183 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.183 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-09-19T08:25:58.183 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.183 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.183 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.183 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_base": { 2024-09-19T08:25:58.184 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_base", 2024-09-19T08:25:58.184 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.184 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.184 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.184 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T08:25:58.184 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.184 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.184 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.184 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Container image name, without the tag", 2024-09-19T08:25:58.184 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.184 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.184 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.184 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_elasticsearch": { 2024-09-19T08:25:58.185 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_elasticsearch", 2024-09-19T08:25:58.185 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.185 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.185 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.185 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T08:25:58.185 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.185 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.185 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.185 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "elasticsearch container image", 2024-09-19T08:25:58.185 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.185 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.185 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.185 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.185 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_grafana": { 2024-09-19T08:25:58.186 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_grafana", 2024-09-19T08:25:58.186 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.186 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.186 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.186 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T08:25:58.186 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.186 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.186 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.186 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-09-19T08:25:58.186 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.186 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.186 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.186 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_haproxy": { 2024-09-19T08:25:58.187 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_haproxy", 2024-09-19T08:25:58.187 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.187 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.187 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.187 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T08:25:58.187 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.187 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.187 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.187 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "HAproxy container image", 2024-09-19T08:25:58.187 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.187 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.187 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.187 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.188 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_agent": { 2024-09-19T08:25:58.188 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_agent", 2024-09-19T08:25:58.188 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.188 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.188 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.188 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T08:25:58.188 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.188 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.188 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.188 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger agent container image", 2024-09-19T08:25:58.188 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.188 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.188 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.188 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.189 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_collector": { 2024-09-19T08:25:58.189 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_collector", 2024-09-19T08:25:58.189 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.189 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.189 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.189 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T08:25:58.189 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.189 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.189 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.189 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger collector container image", 2024-09-19T08:25:58.189 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.189 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.189 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.189 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.190 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_query": { 2024-09-19T08:25:58.190 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_query", 2024-09-19T08:25:58.190 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.190 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.190 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.190 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T08:25:58.190 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.190 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.190 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.190 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger query container image", 2024-09-19T08:25:58.190 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.190 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.190 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.191 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_keepalived": { 2024-09-19T08:25:58.191 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_keepalived", 2024-09-19T08:25:58.191 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.191 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.191 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.191 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T08:25:58.191 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.191 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.191 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.191 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Keepalived container image", 2024-09-19T08:25:58.191 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.191 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.191 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.191 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.192 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_loki": { 2024-09-19T08:25:58.192 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_loki", 2024-09-19T08:25:58.192 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.192 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.192 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.192 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T08:25:58.192 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.192 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.192 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.192 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Loki container image", 2024-09-19T08:25:58.192 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.192 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.192 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.193 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_node_exporter": { 2024-09-19T08:25:58.193 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_node_exporter", 2024-09-19T08:25:58.193 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.193 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.193 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.193 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T08:25:58.193 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.193 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.193 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.193 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-09-19T08:25:58.193 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.193 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.193 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_nvmeof": { 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_nvmeof", 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Nvme-of container image", 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.195 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_prometheus": { 2024-09-19T08:25:58.195 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_prometheus", 2024-09-19T08:25:58.195 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.195 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.195 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.195 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T08:25:58.195 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.195 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.195 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.195 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-09-19T08:25:58.195 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.195 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.195 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.196 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_promtail": { 2024-09-19T08:25:58.196 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_promtail", 2024-09-19T08:25:58.196 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.196 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.196 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.196 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T08:25:58.196 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.196 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.196 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.196 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Promtail container image", 2024-09-19T08:25:58.196 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.196 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.196 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.197 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.197 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_samba": { 2024-09-19T08:25:58.197 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_samba", 2024-09-19T08:25:58.197 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.197 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.197 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.197 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T08:25:58.197 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.197 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.197 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.197 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Samba/SMB container image", 2024-09-19T08:25:58.197 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.197 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.197 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.198 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.198 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_snmp_gateway": { 2024-09-19T08:25:58.198 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_snmp_gateway", 2024-09-19T08:25:58.198 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.198 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.198 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.198 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T08:25:58.198 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.198 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.198 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.198 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SNMP Gateway container image", 2024-09-19T08:25:58.198 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.198 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.199 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.199 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.199 INFO:teuthology.orchestra.run.smithi022.stdout: "container_init": { 2024-09-19T08:25:58.199 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_init", 2024-09-19T08:25:58.199 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.199 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.199 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.199 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.199 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.199 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.199 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.199 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T08:25:58.199 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.199 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.200 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.200 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.200 INFO:teuthology.orchestra.run.smithi022.stdout: "daemon_cache_timeout": { 2024-09-19T08:25:58.200 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "daemon_cache_timeout", 2024-09-19T08:25:58.200 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.200 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.200 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.200 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:58.200 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.200 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.200 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.200 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T08:25:58.200 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.200 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.201 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.201 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.201 INFO:teuthology.orchestra.run.smithi022.stdout: "default_cephadm_command_timeout": { 2024-09-19T08:25:58.201 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T08:25:58.201 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.201 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.201 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.201 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "900", 2024-09-19T08:25:58.201 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.201 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.201 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.201 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T08:25:58.201 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.201 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.202 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.202 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.202 INFO:teuthology.orchestra.run.smithi022.stdout: "default_registry": { 2024-09-19T08:25:58.202 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_registry", 2024-09-19T08:25:58.202 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.202 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.202 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.202 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io", 2024-09-19T08:25:58.202 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.202 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.202 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.202 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T08:25:58.202 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.202 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.203 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.203 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.203 INFO:teuthology.orchestra.run.smithi022.stdout: "device_cache_timeout": { 2024-09-19T08:25:58.203 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_cache_timeout", 2024-09-19T08:25:58.203 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.203 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.203 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.203 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1800", 2024-09-19T08:25:58.203 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.203 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.203 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.203 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache device inventory", 2024-09-19T08:25:58.203 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.203 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.204 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.204 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.204 INFO:teuthology.orchestra.run.smithi022.stdout: "device_enhanced_scan": { 2024-09-19T08:25:58.204 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_enhanced_scan", 2024-09-19T08:25:58.204 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.204 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.204 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.204 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.204 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.204 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.204 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.204 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T08:25:58.204 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.204 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.205 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.205 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.205 INFO:teuthology.orchestra.run.smithi022.stdout: "facts_cache_timeout": { 2024-09-19T08:25:58.205 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "facts_cache_timeout", 2024-09-19T08:25:58.205 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.205 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.205 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.205 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:58.205 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.205 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.205 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.205 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache host facts data", 2024-09-19T08:25:58.205 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.205 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.206 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.206 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.206 INFO:teuthology.orchestra.run.smithi022.stdout: "grafana_dashboards_path": { 2024-09-19T08:25:58.206 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "grafana_dashboards_path", 2024-09-19T08:25:58.206 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.206 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.206 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.206 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T08:25:58.206 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.206 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.206 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.206 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T08:25:58.206 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.206 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.207 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.207 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.207 INFO:teuthology.orchestra.run.smithi022.stdout: "host_check_interval": { 2024-09-19T08:25:58.207 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "host_check_interval", 2024-09-19T08:25:58.207 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.207 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.207 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.207 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:58.207 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.207 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.207 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.207 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to perform a host check", 2024-09-19T08:25:58.207 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.207 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.208 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.208 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.208 INFO:teuthology.orchestra.run.smithi022.stdout: "hw_monitoring": { 2024-09-19T08:25:58.208 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hw_monitoring", 2024-09-19T08:25:58.208 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.208 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.208 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.208 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.208 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.208 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.208 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.208 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T08:25:58.208 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.209 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.209 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.209 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.209 INFO:teuthology.orchestra.run.smithi022.stdout: "inventory_list_all": { 2024-09-19T08:25:58.209 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "inventory_list_all", 2024-09-19T08:25:58.209 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.209 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.209 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.209 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.209 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.209 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.209 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.209 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T08:25:58.209 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.210 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.210 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.210 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.210 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.210 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.210 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.210 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.210 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.210 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.210 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.210 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.210 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.210 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.211 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.211 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.211 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.211 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.211 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.211 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.211 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.211 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.211 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.211 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.211 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.211 INFO:teuthology.orchestra.run.smithi022.stdout: "log_refresh_metadata": { 2024-09-19T08:25:58.211 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_refresh_metadata", 2024-09-19T08:25:58.212 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.212 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.212 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.212 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.212 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.212 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.212 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.212 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T08:25:58.212 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.212 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.212 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.212 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.212 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.213 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.213 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.213 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.213 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.213 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.213 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.213 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.213 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.213 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T08:25:58.213 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.213 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.213 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.213 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.214 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.214 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.214 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.214 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.214 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.214 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.214 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.214 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.214 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.214 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.214 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.214 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.214 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.214 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.215 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.215 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.215 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.215 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.215 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.215 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.215 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.215 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.215 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.215 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.215 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.215 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.215 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.215 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.216 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.216 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.216 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.216 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.216 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.216 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.216 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.216 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T08:25:58.216 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T08:25:58.216 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.216 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.216 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.216 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.216 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.217 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.217 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.217 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T08:25:58.217 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.217 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.217 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.217 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.217 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T08:25:58.217 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T08:25:58.217 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.217 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.217 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.217 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "*", 2024-09-19T08:25:58.218 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.218 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.218 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.218 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T08:25:58.218 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.218 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.218 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.218 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.218 INFO:teuthology.orchestra.run.smithi022.stdout: "max_count_per_host": { 2024-09-19T08:25:58.218 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_count_per_host", 2024-09-19T08:25:58.218 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.218 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.218 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.218 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:58.219 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.219 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.219 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.219 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of daemons per service per host", 2024-09-19T08:25:58.219 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.219 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.219 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.219 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.219 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd_draining_count": { 2024-09-19T08:25:58.219 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_osd_draining_count", 2024-09-19T08:25:58.219 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.219 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.219 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.220 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:58.220 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.220 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.220 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.220 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T08:25:58.220 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.220 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.220 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.220 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.220 INFO:teuthology.orchestra.run.smithi022.stdout: "migration_current": { 2024-09-19T08:25:58.220 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "migration_current", 2024-09-19T08:25:58.220 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.220 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.220 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.221 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.221 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.221 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.221 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.221 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "internal - do not modify", 2024-09-19T08:25:58.221 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.221 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.221 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.221 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-09-19T08:25:58.221 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-09-19T08:25:58.221 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.221 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.221 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.222 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "root", 2024-09-19T08:25:58.222 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.222 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.222 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.222 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm-package", 2024-09-19T08:25:58.222 INFO:teuthology.orchestra.run.smithi022.stdout: "root" 2024-09-19T08:25:58.222 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.222 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T08:25:58.222 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.222 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.222 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.222 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.222 INFO:teuthology.orchestra.run.smithi022.stdout: "oob_default_addr": { 2024-09-19T08:25:58.222 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "oob_default_addr", 2024-09-19T08:25:58.223 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.223 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.223 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.223 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "169.254.1.1", 2024-09-19T08:25:58.223 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.223 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.223 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.223 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T08:25:58.223 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.223 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.223 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.223 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.223 INFO:teuthology.orchestra.run.smithi022.stdout: "prometheus_alerts_path": { 2024-09-19T08:25:58.223 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus_alerts_path", 2024-09-19T08:25:58.224 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.224 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.224 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.224 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T08:25:58.224 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.224 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.224 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.224 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T08:25:58.224 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.224 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.224 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.224 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.224 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_insecure": { 2024-09-19T08:25:58.224 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_insecure", 2024-09-19T08:25:58.225 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.225 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.225 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.225 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.225 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.225 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.225 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.225 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T08:25:58.225 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.225 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.225 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.225 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.225 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_password": { 2024-09-19T08:25:58.225 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_password", 2024-09-19T08:25:58.226 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.226 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.226 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.226 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.226 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.226 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.226 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.226 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T08:25:58.226 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.226 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.226 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.226 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.226 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_url": { 2024-09-19T08:25:58.227 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_url", 2024-09-19T08:25:58.227 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.227 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.227 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.227 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.227 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.227 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.227 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.227 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T08:25:58.227 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.227 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.227 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.227 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_username": { 2024-09-19T08:25:58.228 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_username", 2024-09-19T08:25:58.228 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.228 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.228 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.228 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.228 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.228 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.228 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.228 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T08:25:58.228 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.228 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.228 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.228 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.229 INFO:teuthology.orchestra.run.smithi022.stdout: "secure_monitoring_stack": { 2024-09-19T08:25:58.229 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secure_monitoring_stack", 2024-09-19T08:25:58.229 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.229 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.229 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.229 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.229 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.229 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.229 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.229 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T08:25:58.229 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.229 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.229 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.229 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.230 INFO:teuthology.orchestra.run.smithi022.stdout: "service_discovery_port": { 2024-09-19T08:25:58.230 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "service_discovery_port", 2024-09-19T08:25:58.230 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.230 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.230 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.230 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8765", 2024-09-19T08:25:58.230 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.230 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.230 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.230 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "cephadm service discovery port", 2024-09-19T08:25:58.230 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.230 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.230 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.230 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.231 INFO:teuthology.orchestra.run.smithi022.stdout: "ssh_config_file": { 2024-09-19T08:25:58.231 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssh_config_file", 2024-09-19T08:25:58.231 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.231 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.231 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.231 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.231 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.231 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.231 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.231 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T08:25:58.231 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.231 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.231 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: "use_agent": { 2024-09-19T08:25:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_agent", 2024-09-19T08:25:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T08:25:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.233 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.233 INFO:teuthology.orchestra.run.smithi022.stdout: "use_repo_digest": { 2024-09-19T08:25:58.233 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_repo_digest", 2024-09-19T08:25:58.233 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.233 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.233 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.233 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.233 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.233 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.233 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.233 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T08:25:58.233 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.233 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.233 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.234 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.234 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_failed_host_check": { 2024-09-19T08:25:58.234 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_failed_host_check", 2024-09-19T08:25:58.234 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.234 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.234 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.234 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.234 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.234 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.234 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.234 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T08:25:58.234 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.234 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.234 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.235 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.235 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_daemons": { 2024-09-19T08:25:58.235 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_daemons", 2024-09-19T08:25:58.235 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.235 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.235 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.235 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.235 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.235 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.235 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.235 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T08:25:58.235 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.235 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.235 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.236 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.236 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_hosts": { 2024-09-19T08:25:58.236 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_hosts", 2024-09-19T08:25:58.236 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.236 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.236 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.236 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.236 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.236 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.236 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.236 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T08:25:58.236 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.236 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.237 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.237 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.237 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.237 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.237 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.237 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crash", 2024-09-19T08:25:58.237 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.237 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.237 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.237 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.237 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.237 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.237 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.237 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.238 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.238 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.238 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.238 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.238 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.238 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.238 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.238 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.238 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.238 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.238 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.238 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.238 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.238 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.239 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.239 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.239 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.239 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.239 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.239 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.239 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.239 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.239 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.239 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.239 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.239 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.239 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.239 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.240 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.240 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.240 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.240 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.240 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.240 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.240 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.240 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.240 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.240 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.240 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.240 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.240 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.240 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.241 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.242 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.242 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.242 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.242 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.242 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.242 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.242 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.242 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.242 INFO:teuthology.orchestra.run.smithi022.stdout: "retain_interval": { 2024-09-19T08:25:58.242 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retain_interval", 2024-09-19T08:25:58.242 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.242 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.242 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.243 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "31536000", 2024-09-19T08:25:58.243 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.243 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.243 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.243 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T08:25:58.243 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.243 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.243 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.243 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.243 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_recent_interval": { 2024-09-19T08:25:58.243 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_recent_interval", 2024-09-19T08:25:58.243 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.243 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.244 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.244 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1209600", 2024-09-19T08:25:58.244 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.244 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.244 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.244 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T08:25:58.244 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.244 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.244 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.244 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.244 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.244 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.244 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.244 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dashboard", 2024-09-19T08:25:58.245 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.245 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.245 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.245 INFO:teuthology.orchestra.run.smithi022.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T08:25:58.245 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T08:25:58.245 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.245 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.245 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.245 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:58.245 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.245 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.245 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.245 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.246 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.246 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.246 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.246 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.246 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T08:25:58.246 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T08:25:58.246 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.246 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.246 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.246 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.246 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.246 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.246 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.246 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.247 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.247 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.247 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.247 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.247 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T08:25:58.247 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T08:25:58.247 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.247 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.247 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.247 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.247 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.247 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.247 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.248 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.248 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.248 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.248 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.248 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.248 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_ENABLED": { 2024-09-19T08:25:58.248 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T08:25:58.248 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.248 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.248 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.248 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.248 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.248 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.248 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.249 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.249 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.249 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.249 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.249 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.249 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T08:25:58.249 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T08:25:58.249 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.249 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.249 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.249 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.249 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.249 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.249 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T08:25:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T08:25:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.250 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.251 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.251 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.251 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.251 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.251 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.251 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T08:25:58.251 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T08:25:58.251 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.251 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.251 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.251 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.251 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.251 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.251 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.252 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.252 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.252 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.252 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.252 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T08:25:58.252 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T08:25:58.252 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.252 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.252 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.252 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.252 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.252 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.252 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.253 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.253 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.253 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.253 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.253 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.253 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T08:25:58.253 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T08:25:58.253 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.253 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.253 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.253 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.253 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.253 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.254 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.254 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.254 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.254 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.254 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.254 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.254 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T08:25:58.254 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T08:25:58.254 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.254 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.254 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.254 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.254 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.254 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.255 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.255 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.255 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.255 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.255 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.255 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.255 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T08:25:58.255 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T08:25:58.255 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.255 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.255 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.255 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.255 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.255 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.256 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.256 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.256 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.256 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.256 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.256 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.256 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T08:25:58.256 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T08:25:58.256 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.256 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.256 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.256 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.256 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.257 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.257 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.257 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.257 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.257 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.257 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.257 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.257 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T08:25:58.257 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T08:25:58.257 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.257 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.257 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.257 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.257 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.258 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.258 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.258 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.258 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.258 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.258 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.258 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.258 INFO:teuthology.orchestra.run.smithi022.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T08:25:58.258 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T08:25:58.258 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.258 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.258 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.258 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.258 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.259 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.259 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.259 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.259 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.259 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.259 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.259 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.259 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T08:25:58.259 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T08:25:58.259 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.259 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.259 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.260 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-09-19T08:25:58.260 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.260 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.260 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.260 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.260 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.260 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.260 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.260 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.260 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T08:25:58.260 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T08:25:58.260 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.260 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.260 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.261 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.261 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.261 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.261 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.261 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.261 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.261 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.261 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.261 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.261 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_URL": { 2024-09-19T08:25:58.261 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_URL", 2024-09-19T08:25:58.261 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.261 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.261 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.262 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.262 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.262 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.262 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.262 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.262 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.262 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.262 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.262 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.262 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T08:25:58.262 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T08:25:58.262 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.262 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.262 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.263 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-09-19T08:25:58.263 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.263 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.263 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.263 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.263 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.263 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.263 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.263 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T08:25:58.263 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T08:25:58.263 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.263 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.264 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.264 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.264 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.264 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.264 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.264 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.264 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.264 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.264 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.264 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.264 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T08:25:58.264 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T08:25:58.264 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.264 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.265 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.265 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.265 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.265 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.265 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.265 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.265 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.265 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.265 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.265 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.265 INFO:teuthology.orchestra.run.smithi022.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T08:25:58.265 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T08:25:58.265 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.265 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.266 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.266 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.266 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.266 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.266 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.266 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.266 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.266 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.266 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.266 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.266 INFO:teuthology.orchestra.run.smithi022.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T08:25:58.266 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T08:25:58.266 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.267 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.267 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.267 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.267 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.267 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.267 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.267 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.267 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.267 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.267 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.267 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.267 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T08:25:58.267 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T08:25:58.267 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T08:25:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T08:25:58.268 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T08:25:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T08:25:58.269 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T08:25:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T08:25:58.270 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.271 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.271 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.271 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.271 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.271 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.271 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.271 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.271 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.271 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.271 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.271 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.271 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T08:25:58.271 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T08:25:58.271 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.272 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.272 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.272 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.272 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.272 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.272 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.272 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.272 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.272 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.272 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.272 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.272 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T08:25:58.272 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T08:25:58.272 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.273 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.273 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.273 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.273 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.273 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.273 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.273 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.273 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.273 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.273 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.273 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.273 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T08:25:58.273 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T08:25:58.273 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.274 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.275 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.275 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.275 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.275 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.275 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.275 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.275 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.275 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.275 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.275 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.275 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T08:25:58.275 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T08:25:58.276 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.276 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.276 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.276 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.276 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.276 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.276 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.276 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.276 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.276 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.276 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.276 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.276 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T08:25:58.276 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T08:25:58.277 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.277 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.277 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.277 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.277 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.277 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.277 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.277 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.277 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.277 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.277 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.277 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.277 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T08:25:58.278 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T08:25:58.278 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.278 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.278 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.278 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T08:25:58.278 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.278 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.278 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.278 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.278 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.278 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.278 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.278 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.279 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T08:25:58.280 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T08:25:58.280 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.280 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.280 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.280 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8", 2024-09-19T08:25:58.280 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.280 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.280 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.280 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.280 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.280 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.280 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.280 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.281 INFO:teuthology.orchestra.run.smithi022.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T08:25:58.281 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T08:25:58.281 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.281 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.281 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.281 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "45", 2024-09-19T08:25:58.281 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.281 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.281 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.281 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.281 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.281 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.281 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.281 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.282 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T08:25:58.282 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T08:25:58.282 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.282 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.282 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.282 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.282 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.282 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.282 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.282 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.282 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.282 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.282 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.283 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.283 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T08:25:58.283 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T08:25:58.283 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.283 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.283 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.283 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-09-19T08:25:58.283 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.283 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.283 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.283 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.283 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.283 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.283 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.284 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.284 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T08:25:58.284 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T08:25:58.284 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.284 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.284 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.284 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.284 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.284 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.284 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.284 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.284 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.284 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.284 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.285 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.285 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T08:25:58.285 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T08:25:58.285 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.285 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.285 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.285 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.285 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.285 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.285 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.285 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.285 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.285 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.285 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.286 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.286 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T08:25:58.286 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T08:25:58.286 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.286 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.286 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.286 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:58.286 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.286 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.286 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.286 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.286 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.286 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.286 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.287 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.287 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T08:25:58.287 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T08:25:58.287 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.287 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.287 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.287 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:58.287 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.287 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.287 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.287 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.287 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.287 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.287 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.288 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.288 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T08:25:58.288 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T08:25:58.288 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.288 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.288 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.288 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:58.288 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.288 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.288 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.288 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.288 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.288 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: "cross_origin_url": { 2024-09-19T08:25:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cross_origin_url", 2024-09-19T08:25:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.290 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.290 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.290 INFO:teuthology.orchestra.run.smithi022.stdout: "crt_file": { 2024-09-19T08:25:58.290 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crt_file", 2024-09-19T08:25:58.290 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.290 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.290 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.290 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.290 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.290 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.290 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.290 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.290 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: "debug": { 2024-09-19T08:25:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "debug", 2024-09-19T08:25:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.291 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable/disable debug options", 2024-09-19T08:25:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "jwt_token_ttl": { 2024-09-19T08:25:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "jwt_token_ttl", 2024-09-19T08:25:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "28800", 2024-09-19T08:25:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.292 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-09-19T08:25:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-09-19T08:25:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.293 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.294 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.295 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.296 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.297 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.298 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "motd": { 2024-09-19T08:25:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "motd", 2024-09-19T08:25:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.299 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "The message of the day", 2024-09-19T08:25:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect_resolve_ip_addr": { 2024-09-19T08:25:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T08:25:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-09-19T08:25:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-09-19T08:25:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-09-19T08:25:58.301 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-09-19T08:25:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-09-19T08:25:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8080", 2024-09-19T08:25:58.302 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-09-19T08:25:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-09-19T08:25:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.303 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl_server_port": { 2024-09-19T08:25:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl_server_port", 2024-09-19T08:25:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.304 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8443", 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "redirect", 2024-09-19T08:25:58.305 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect" 2024-09-19T08:25:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-09-19T08:25:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-09-19T08:25:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.306 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-09-19T08:25:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-09-19T08:25:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-09-19T08:25:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "url_prefix": { 2024-09-19T08:25:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url_prefix", 2024-09-19T08:25:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.307 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.308 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2024-09-19T08:25:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_monitoring": { 2024-09-19T08:25:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_monitoring", 2024-09-19T08:25:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.309 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "monitor device health metrics", 2024-09-19T08:25:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.310 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.311 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.312 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.313 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.314 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "mark_out_threshold": { 2024-09-19T08:25:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mark_out_threshold", 2024-09-19T08:25:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2419200", 2024-09-19T08:25:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.315 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T08:25:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": { 2024-09-19T08:25:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_name", 2024-09-19T08:25:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "device_health_metrics", 2024-09-19T08:25:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.316 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T08:25:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "retention_period": { 2024-09-19T08:25:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retention_period", 2024-09-19T08:25:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15552000", 2024-09-19T08:25:58.317 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain device health metrics", 2024-09-19T08:25:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_frequency": { 2024-09-19T08:25:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_frequency", 2024-09-19T08:25:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-09-19T08:25:58.318 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "self_heal": { 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "self_heal", 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.319 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T08:25:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:58.320 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T08:25:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_threshold": { 2024-09-19T08:25:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_threshold", 2024-09-19T08:25:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7257600", 2024-09-19T08:25:58.321 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T08:25:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "diskprediction_local", 2024-09-19T08:25:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.322 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.323 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.324 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.325 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.326 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.327 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.328 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.328 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.328 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.328 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.328 INFO:teuthology.orchestra.run.smithi022.stdout: "predict_interval": { 2024-09-19T08:25:58.328 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predict_interval", 2024-09-19T08:25:58.328 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.328 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.328 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.328 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-09-19T08:25:58.328 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.328 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.328 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.329 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.329 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.329 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.329 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.329 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.329 INFO:teuthology.orchestra.run.smithi022.stdout: "predictor_model": { 2024-09-19T08:25:58.329 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predictor_model", 2024-09-19T08:25:58.329 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.329 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.329 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.329 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "prophetstor", 2024-09-19T08:25:58.329 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.329 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.329 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.330 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.330 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.330 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.330 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.330 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.330 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:58.330 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:58.330 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.330 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.330 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.330 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-09-19T08:25:58.330 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.330 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.330 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.331 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.331 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.331 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.331 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.331 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.331 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.331 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.331 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.331 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "influx", 2024-09-19T08:25:58.331 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": false, 2024-09-19T08:25:58.331 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "influxdb python module not found", 2024-09-19T08:25:58.331 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.331 INFO:teuthology.orchestra.run.smithi022.stdout: "batch_size": { 2024-09-19T08:25:58.332 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "batch_size", 2024-09-19T08:25:58.332 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.332 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.332 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.332 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5000", 2024-09-19T08:25:58.332 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.332 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.332 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.332 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T08:25:58.332 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.332 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.332 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.332 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.332 INFO:teuthology.orchestra.run.smithi022.stdout: "database": { 2024-09-19T08:25:58.333 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "database", 2024-09-19T08:25:58.333 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.333 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.333 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.333 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "ceph", 2024-09-19T08:25:58.333 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.333 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.333 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.333 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T08:25:58.333 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.333 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.333 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.333 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.334 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": { 2024-09-19T08:25:58.334 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hostname", 2024-09-19T08:25:58.334 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.334 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.334 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.334 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.334 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.334 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.334 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.334 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server hostname", 2024-09-19T08:25:58.334 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.334 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.334 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.334 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.335 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:58.335 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:58.335 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.335 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.335 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.335 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "30", 2024-09-19T08:25:58.335 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "5", 2024-09-19T08:25:58.335 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.335 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.335 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T08:25:58.335 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.335 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.335 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.335 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.336 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.336 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.336 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.336 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.336 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.336 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.336 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.336 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.336 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.336 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.336 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.336 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.336 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.336 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.337 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.337 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.337 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.337 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.337 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.337 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.337 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.337 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.337 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.337 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.337 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.337 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.337 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.337 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.338 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.338 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.338 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.338 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.338 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.338 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.338 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.338 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.338 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.338 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.338 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.338 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.338 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.338 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.339 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.339 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.339 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.339 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.339 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.339 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.339 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.339 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.339 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.339 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.339 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.339 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.339 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.339 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.340 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.340 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.340 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.340 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.340 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.340 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.340 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.340 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.340 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.340 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.340 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.340 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.340 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.340 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.341 INFO:teuthology.orchestra.run.smithi022.stdout: "password": { 2024-09-19T08:25:58.341 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "password", 2024-09-19T08:25:58.341 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.341 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.341 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.341 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.341 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.341 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.341 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.341 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "password of InfluxDB server user", 2024-09-19T08:25:58.341 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.341 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.341 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.341 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.342 INFO:teuthology.orchestra.run.smithi022.stdout: "port": { 2024-09-19T08:25:58.342 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "port", 2024-09-19T08:25:58.342 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.342 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.342 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.342 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8086", 2024-09-19T08:25:58.342 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.342 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.342 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.342 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server port", 2024-09-19T08:25:58.342 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.342 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.342 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.343 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.343 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-09-19T08:25:58.343 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-09-19T08:25:58.343 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.343 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.343 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.343 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "false", 2024-09-19T08:25:58.343 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.343 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.343 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.343 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:25:58.343 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.343 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.343 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.344 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.344 INFO:teuthology.orchestra.run.smithi022.stdout: "threads": { 2024-09-19T08:25:58.344 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threads", 2024-09-19T08:25:58.344 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.344 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.344 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.344 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:58.344 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-09-19T08:25:58.344 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "32", 2024-09-19T08:25:58.344 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.344 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T08:25:58.344 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.345 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.345 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.345 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.345 INFO:teuthology.orchestra.run.smithi022.stdout: "username": { 2024-09-19T08:25:58.345 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "username", 2024-09-19T08:25:58.345 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.345 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.345 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.345 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.345 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.345 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.345 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.345 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "username of InfluxDB server user", 2024-09-19T08:25:58.346 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.346 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.346 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.346 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.346 INFO:teuthology.orchestra.run.smithi022.stdout: "verify_ssl": { 2024-09-19T08:25:58.346 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "verify_ssl", 2024-09-19T08:25:58.346 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.346 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.346 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.346 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "true", 2024-09-19T08:25:58.346 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.346 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.346 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.346 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:25:58.347 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.347 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.347 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.347 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.347 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.347 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.347 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.347 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "insights", 2024-09-19T08:25:58.347 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.347 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.347 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.347 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.347 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.347 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.348 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.348 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.348 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.348 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.348 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.348 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.348 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.348 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.348 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.348 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.348 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.348 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.348 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.349 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.349 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.349 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.349 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.349 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.349 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.349 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.349 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.349 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.349 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.349 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.349 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.349 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.350 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.350 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.350 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.350 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.350 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.350 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.350 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.350 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.350 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.350 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.350 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.350 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.350 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.350 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.351 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.351 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.351 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.351 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.351 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.351 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.351 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.351 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.351 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.351 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.351 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.351 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.351 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.351 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.352 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.352 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.352 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.352 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.352 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.352 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.352 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.352 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.352 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.352 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.352 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.352 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.352 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.353 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.353 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.353 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.353 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "iostat", 2024-09-19T08:25:58.353 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.353 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.353 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.353 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.353 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.353 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.353 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.353 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.353 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.353 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.354 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.354 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.354 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.354 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.354 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.354 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.354 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.354 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.354 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.354 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.354 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.354 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.354 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.354 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.355 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.355 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.355 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.355 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.355 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.355 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.355 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.355 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.355 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.355 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.355 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.355 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.355 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.355 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.356 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.356 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.356 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.356 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.356 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.356 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.356 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.356 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.356 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.356 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.356 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.356 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.356 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.357 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.357 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.357 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.357 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.357 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.357 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.357 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.357 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.357 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.357 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.357 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.357 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.357 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.357 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.358 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.358 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.358 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.358 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.358 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.358 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.358 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.358 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.358 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.358 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.358 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.358 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "k8sevents", 2024-09-19T08:25:58.358 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.359 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.359 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.359 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_event_retention_days": { 2024-09-19T08:25:58.359 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ceph_event_retention_days", 2024-09-19T08:25:58.359 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.359 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.359 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.359 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7", 2024-09-19T08:25:58.359 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.359 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.359 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.359 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T08:25:58.359 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.359 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.360 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.360 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.360 INFO:teuthology.orchestra.run.smithi022.stdout: "config_check_secs": { 2024-09-19T08:25:58.360 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_check_secs", 2024-09-19T08:25:58.360 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.360 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.360 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.360 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:58.360 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "10", 2024-09-19T08:25:58.360 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.360 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.360 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T08:25:58.361 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.361 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.361 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.361 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.361 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.361 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.361 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.361 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.361 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.361 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.361 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.361 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.361 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.361 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.362 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.362 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.362 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.362 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.362 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.362 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.362 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.362 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.362 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.362 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.362 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.362 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.362 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.363 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.363 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.363 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.363 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.363 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.363 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.363 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.363 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.363 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.363 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.363 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.363 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.363 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.363 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.364 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.364 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.364 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.364 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.364 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.364 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.364 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.364 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.364 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.364 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.364 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.364 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.364 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.365 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.365 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.365 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.365 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.365 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.365 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.365 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.365 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.365 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.365 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.365 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.365 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.365 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.365 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.366 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.366 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.366 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.366 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.366 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.366 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.366 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.366 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.366 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.366 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "localpool", 2024-09-19T08:25:58.366 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.366 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.366 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.366 INFO:teuthology.orchestra.run.smithi022.stdout: "failure_domain": { 2024-09-19T08:25:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "failure_domain", 2024-09-19T08:25:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "host", 2024-09-19T08:25:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "failure domain for any created local pool", 2024-09-19T08:25:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T08:25:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.367 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.368 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.368 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.368 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.368 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.368 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.368 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.368 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.368 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.368 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.368 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.368 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.368 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.369 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.370 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.370 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.370 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.370 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.370 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.370 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.370 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.370 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.370 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.370 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.370 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.370 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.371 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.371 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.371 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.371 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.371 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.371 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.371 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.371 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.371 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.371 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.371 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.371 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.371 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.372 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.372 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.372 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.372 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.372 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.372 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.372 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.372 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.372 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.372 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.372 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.372 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.372 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.372 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.373 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.373 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.373 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.373 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.373 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": { 2024-09-19T08:25:58.373 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_size", 2024-09-19T08:25:58.373 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.373 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.373 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.373 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.373 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.373 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.373 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.373 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default min_size for any created local pool", 2024-09-19T08:25:58.374 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T08:25:58.374 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.374 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.374 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.374 INFO:teuthology.orchestra.run.smithi022.stdout: "num_rep": { 2024-09-19T08:25:58.374 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "num_rep", 2024-09-19T08:25:58.374 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.374 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.374 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.374 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3", 2024-09-19T08:25:58.374 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.374 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.374 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.374 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default replica count for any created local pool", 2024-09-19T08:25:58.375 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.375 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.375 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.375 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.375 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": { 2024-09-19T08:25:58.375 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_num", 2024-09-19T08:25:58.375 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.375 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.375 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.375 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "128", 2024-09-19T08:25:58.375 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.375 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.375 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.375 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T08:25:58.376 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.376 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.376 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.376 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.376 INFO:teuthology.orchestra.run.smithi022.stdout: "prefix": { 2024-09-19T08:25:58.376 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prefix", 2024-09-19T08:25:58.376 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.376 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.376 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.376 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.376 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.376 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.376 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.377 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name prefix for any created local pool", 2024-09-19T08:25:58.377 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.377 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.377 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.377 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.377 INFO:teuthology.orchestra.run.smithi022.stdout: "subtree": { 2024-09-19T08:25:58.377 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "subtree", 2024-09-19T08:25:58.377 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.377 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.377 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.377 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "rack", 2024-09-19T08:25:58.377 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.377 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.377 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.378 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T08:25:58.378 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T08:25:58.378 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.378 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.378 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.378 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.378 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.378 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.378 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mds_autoscaler", 2024-09-19T08:25:58.378 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.378 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.378 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.378 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.378 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.379 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.379 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.379 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.379 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.379 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.379 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.379 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.379 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.379 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.379 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.379 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.379 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.379 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.379 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.380 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.380 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.380 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.380 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.380 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.380 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.380 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.380 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.380 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.380 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.380 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.380 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.382 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.382 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.382 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.382 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.382 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.382 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.382 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.382 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.382 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.382 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.382 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.382 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.382 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.383 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.383 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.383 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.383 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.383 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.383 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.383 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.383 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.383 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.383 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.383 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.383 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.383 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.384 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.384 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.384 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.384 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.384 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.384 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirroring", 2024-09-19T08:25:58.384 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.384 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.384 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.384 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.384 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.384 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.384 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.384 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.385 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.385 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.385 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.385 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.385 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.385 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.385 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.385 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.385 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.385 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.385 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.385 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.385 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.386 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.386 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.386 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.386 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.386 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.386 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.386 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.386 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.386 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.386 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.386 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.386 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.386 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.386 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.387 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.387 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.387 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.387 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.387 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.387 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.387 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.387 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.387 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.387 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.387 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.387 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.387 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.387 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.388 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.388 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.388 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.388 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.388 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.388 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.388 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.388 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.388 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.388 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.388 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.388 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.388 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.389 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.389 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.389 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.389 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.389 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.389 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.389 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.389 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.389 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.389 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.389 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.389 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.389 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.389 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.390 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "nfs", 2024-09-19T08:25:58.390 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.390 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.390 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.390 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.390 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.390 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.390 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.390 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.390 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.390 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.390 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.390 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.391 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.391 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.391 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.391 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.391 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.391 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.391 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.391 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.391 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.391 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.391 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.391 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.391 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.391 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.392 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.392 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.392 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.392 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.392 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.392 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.392 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.392 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.392 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.392 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.392 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.392 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.392 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.393 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.393 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.393 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.393 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.393 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.393 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.393 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.393 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.393 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.393 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.393 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.393 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.393 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.393 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.394 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.394 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.394 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.394 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.394 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.394 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.394 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.394 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.394 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.394 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.394 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.394 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.394 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.395 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.395 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.395 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.395 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.395 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.395 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.395 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.395 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.395 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.395 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.395 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-09-19T08:25:58.395 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.395 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.395 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.396 INFO:teuthology.orchestra.run.smithi022.stdout: "fail_fs": { 2024-09-19T08:25:58.396 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "fail_fs", 2024-09-19T08:25:58.396 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.396 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.396 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.396 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.396 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.396 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.396 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.396 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T08:25:58.396 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.396 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.396 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.397 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.397 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.397 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.397 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.397 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.397 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.397 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.397 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.397 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.397 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.397 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.397 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.397 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.398 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.398 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.398 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.398 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.398 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.398 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.398 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.398 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.398 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.398 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.398 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.398 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.398 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.399 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.399 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.399 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.399 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.399 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.399 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.399 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.399 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.399 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.399 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.399 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.399 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.399 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.399 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.400 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.400 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.400 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.400 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.400 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.400 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.400 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.400 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.400 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.400 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.400 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.400 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.400 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.400 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.401 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.401 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.401 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.401 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.401 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.401 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.401 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.401 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.401 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.401 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.401 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.401 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.401 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.402 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.402 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.402 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.402 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.402 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator": { 2024-09-19T08:25:58.402 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-09-19T08:25:58.402 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.402 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.402 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.402 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.402 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.402 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.402 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.402 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2024-09-19T08:25:58.403 INFO:teuthology.orchestra.run.smithi022.stdout: "rook", 2024-09-19T08:25:58.403 INFO:teuthology.orchestra.run.smithi022.stdout: "test_orchestrator" 2024-09-19T08:25:58.403 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.403 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Orchestrator backend", 2024-09-19T08:25:58.403 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.403 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.403 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.403 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.403 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.403 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.403 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.403 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_perf_query", 2024-09-19T08:25:58.403 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.403 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.404 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.404 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.404 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.404 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.404 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.404 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.404 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.404 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.404 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.404 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.404 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.404 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.404 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.404 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.405 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.405 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.405 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.405 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.405 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.405 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.405 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.405 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.405 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.405 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.405 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.405 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.405 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.405 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.406 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.406 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.406 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.406 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.406 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.406 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.406 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.406 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.406 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.406 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.406 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.406 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.406 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.406 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.407 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.407 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.407 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.407 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.407 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.407 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.407 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.407 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.407 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.407 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.407 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.407 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.407 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.407 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.408 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.408 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.408 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.408 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.408 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.408 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.408 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.408 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.408 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.408 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.408 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.408 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.408 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.408 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.409 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.409 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.409 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.409 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.409 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_support", 2024-09-19T08:25:58.409 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.409 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.409 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.409 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.409 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.409 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.409 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.409 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.409 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.410 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.410 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.410 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.410 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.410 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.410 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.410 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.410 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.410 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.410 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.410 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.410 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.410 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.410 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.411 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.411 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.411 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.411 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.411 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.411 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.411 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.411 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.411 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.411 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.411 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.411 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.411 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.411 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.412 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.412 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.412 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.412 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.412 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.412 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.412 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.412 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.412 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.412 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.412 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.412 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.412 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.413 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.413 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.413 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.413 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.413 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.413 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.413 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.413 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.413 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.413 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.413 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.413 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.413 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.414 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.414 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.414 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.414 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.414 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.414 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.414 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.414 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.414 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.414 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.414 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.414 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.414 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.414 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_autoscaler", 2024-09-19T08:25:58.415 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.415 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.415 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.415 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.415 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.415 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.415 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.415 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.415 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.415 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.415 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.415 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.415 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.416 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.416 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.416 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.416 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.416 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.416 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.416 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.416 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.416 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.416 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.416 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.416 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.416 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.416 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.417 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.417 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.417 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.417 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.417 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.417 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.417 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.417 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.417 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.417 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.417 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.417 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.417 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.418 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.418 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.418 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.418 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.418 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.418 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.418 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.418 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.418 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.418 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.418 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.418 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.418 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.418 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.419 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.420 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.420 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.420 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.420 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.420 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.420 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:58.420 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:58.420 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.420 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.420 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.420 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:58.420 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.420 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: "threshold": { 2024-09-19T08:25:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threshold", 2024-09-19T08:25:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-09-19T08:25:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1.0", 2024-09-19T08:25:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "scaling threshold", 2024-09-19T08:25:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T08:25:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "progress", 2024-09-19T08:25:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_pg_recovery_event": { 2024-09-19T08:25:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_pg_recovery_event", 2024-09-19T08:25:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T08:25:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-09-19T08:25:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-09-19T08:25:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.426 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.426 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.426 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.426 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.426 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.426 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.426 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.426 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.426 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.426 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.426 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.426 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.426 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.426 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.427 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.427 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.427 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.427 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.427 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.427 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.427 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.427 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.427 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.427 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.427 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.427 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.427 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.427 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "max_completed_events": { 2024-09-19T08:25:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_completed_events", 2024-09-19T08:25:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "50", 2024-09-19T08:25:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "number of past completed events to remember", 2024-09-19T08:25:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-09-19T08:25:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-09-19T08:25:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long the module is going to sleep", 2024-09-19T08:25:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus", 2024-09-19T08:25:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "cache": { 2024-09-19T08:25:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cache", 2024-09-19T08:25:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "exclude_perf_counters": { 2024-09-19T08:25:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "exclude_perf_counters", 2024-09-19T08:25:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T08:25:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T08:25:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools": { 2024-09-19T08:25:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools", 2024-09-19T08:25:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T08:25:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T08:25:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.441 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300", 2024-09-19T08:25:58.441 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.441 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.441 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.441 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.441 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.441 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.441 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.441 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.441 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_interval": { 2024-09-19T08:25:58.441 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_interval", 2024-09-19T08:25:58.441 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:58.441 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.441 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.442 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15.0", 2024-09-19T08:25:58.442 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.442 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.442 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.442 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.442 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.442 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.442 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.442 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.442 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-09-19T08:25:58.442 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-09-19T08:25:58.442 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.442 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.442 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.443 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "9283", 2024-09-19T08:25:58.444 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.444 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.444 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.444 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T08:25:58.444 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.444 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.444 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.444 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.444 INFO:teuthology.orchestra.run.smithi022.stdout: "stale_cache_strategy": { 2024-09-19T08:25:58.444 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stale_cache_strategy", 2024-09-19T08:25:58.444 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.444 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.444 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.444 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "log", 2024-09-19T08:25:58.445 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.445 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.445 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.445 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.445 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.445 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.445 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.445 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.445 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-09-19T08:25:58.445 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-09-19T08:25:58.445 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.445 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.445 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "default", 2024-09-19T08:25:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: "default", 2024-09-19T08:25:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: "error" 2024-09-19T08:25:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-09-19T08:25:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-09-19T08:25:58.447 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.447 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.447 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.447 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-09-19T08:25:58.447 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-09-19T08:25:58.447 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-09-19T08:25:58.447 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.447 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.447 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.447 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.447 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.447 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.447 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.448 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.448 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.448 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2024-09-19T08:25:58.448 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.448 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.448 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.448 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.448 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.448 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.448 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.448 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.448 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.448 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.448 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.449 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.449 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.449 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.449 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.449 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.449 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.449 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.449 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.449 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.449 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.449 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.449 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.449 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.449 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.450 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.450 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.450 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.450 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.450 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.450 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.450 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.450 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.450 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.450 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.450 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.450 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.450 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.450 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.451 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.451 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.451 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.451 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.451 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.451 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.451 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.451 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.451 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.451 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.451 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.451 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.451 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.452 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.452 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.452 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.452 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.452 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.452 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.452 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.452 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.452 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.452 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.452 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.452 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.452 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.452 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.453 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.453 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.453 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.453 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.453 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.453 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.453 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.453 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_snap_create": { 2024-09-19T08:25:58.453 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_snap_create", 2024-09-19T08:25:58.453 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.453 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.453 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.453 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-09-19T08:25:58.454 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.454 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.454 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.454 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.454 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.454 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.454 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.454 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.454 INFO:teuthology.orchestra.run.smithi022.stdout: "mirror_snapshot_schedule": { 2024-09-19T08:25:58.454 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T08:25:58.454 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.454 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.454 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.454 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.455 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.455 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.455 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.455 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.455 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.455 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.455 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.455 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.455 INFO:teuthology.orchestra.run.smithi022.stdout: "trash_purge_schedule": { 2024-09-19T08:25:58.455 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "trash_purge_schedule", 2024-09-19T08:25:58.455 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.455 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.455 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.455 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.456 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.456 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.456 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.456 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.456 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.456 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.456 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.456 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.456 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.456 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.456 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.456 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "restful", 2024-09-19T08:25:58.456 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.456 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.457 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.457 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_auth": { 2024-09-19T08:25:58.457 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_auth", 2024-09-19T08:25:58.457 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.457 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.457 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.457 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.457 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.457 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.457 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.457 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.457 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.457 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.457 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.458 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.459 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.459 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.459 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.459 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.459 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.459 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.459 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.459 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.459 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.459 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.459 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.459 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.459 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.460 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.460 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.460 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.460 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.460 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.460 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.460 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.460 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.460 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.460 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.460 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.460 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.460 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.460 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.461 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.461 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.461 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.461 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.461 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.461 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.461 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.461 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.461 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.461 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.461 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.461 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.461 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.461 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-09-19T08:25:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-09-19T08:25:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.465 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.465 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.465 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-09-19T08:25:58.465 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-09-19T08:25:58.465 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.465 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.465 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.465 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.465 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.465 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.465 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.465 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.465 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.466 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.466 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.466 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.466 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.466 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.466 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rgw", 2024-09-19T08:25:58.466 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.466 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.466 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.466 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.466 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.466 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.466 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.467 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.467 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.467 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.467 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.467 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.467 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.467 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.467 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.467 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.467 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.467 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.467 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.467 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.467 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.469 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.469 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.469 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.469 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.469 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.469 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.469 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.469 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.469 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.469 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.469 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.469 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.469 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.470 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.470 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.470 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.470 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.470 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.470 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.470 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.470 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.470 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.470 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.470 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.470 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.470 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.470 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.471 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.471 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.471 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.471 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.471 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.471 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.471 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.471 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.471 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.471 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.471 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.471 INFO:teuthology.orchestra.run.smithi022.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T08:25:58.471 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T08:25:58.471 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.472 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.472 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.472 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-09-19T08:25:58.472 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.472 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.472 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.472 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T08:25:58.472 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.472 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.472 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.472 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.472 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.472 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.472 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.473 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rook", 2024-09-19T08:25:58.473 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.473 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.473 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.473 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.473 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.473 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.473 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.473 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.473 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.473 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.473 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.473 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.473 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.474 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.474 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.474 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.474 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.474 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.474 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.474 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.474 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.474 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.474 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.474 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.474 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.474 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.474 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.475 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.475 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.475 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.475 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.475 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.475 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.475 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.475 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.475 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.475 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.475 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.475 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.475 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.475 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.476 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.476 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.476 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.476 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.476 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.476 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.476 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.476 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.476 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.476 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.476 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.476 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.476 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.476 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.477 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.477 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.477 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.477 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.477 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.477 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.477 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.477 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.477 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.477 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.477 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.477 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.477 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.477 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.478 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.478 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.478 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.478 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.478 INFO:teuthology.orchestra.run.smithi022.stdout: "storage_class": { 2024-09-19T08:25:58.478 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "storage_class", 2024-09-19T08:25:58.478 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.478 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.478 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.478 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "local", 2024-09-19T08:25:58.478 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.478 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.478 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.478 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T08:25:58.479 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.479 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.479 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.479 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.479 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.479 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.479 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.479 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "selftest", 2024-09-19T08:25:58.479 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.479 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.479 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.479 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.479 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.479 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.480 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.480 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.480 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.480 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.480 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.480 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.480 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.480 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.480 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.480 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.480 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.480 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.480 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.481 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.481 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.481 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.481 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.481 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.481 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.481 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.481 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.481 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.481 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.481 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.481 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.481 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.482 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.482 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.482 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.482 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.482 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.482 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.482 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.482 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.482 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.482 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.482 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.482 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.482 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.482 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.483 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.483 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.483 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.483 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.483 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.483 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.483 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.483 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.483 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.483 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.483 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.483 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.483 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.483 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.484 INFO:teuthology.orchestra.run.smithi022.stdout: "roption1": { 2024-09-19T08:25:58.485 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption1", 2024-09-19T08:25:58.485 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.485 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.485 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.485 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.485 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.485 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.485 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.485 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.485 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.485 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.485 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.485 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.486 INFO:teuthology.orchestra.run.smithi022.stdout: "roption2": { 2024-09-19T08:25:58.486 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption2", 2024-09-19T08:25:58.486 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.486 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.486 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.486 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "xyz", 2024-09-19T08:25:58.486 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.486 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.486 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.486 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.486 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.486 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.486 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.486 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.487 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption1": { 2024-09-19T08:25:58.487 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption1", 2024-09-19T08:25:58.487 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.487 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.487 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.487 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.487 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.487 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.487 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.487 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.487 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.487 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.487 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.487 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.488 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption2": { 2024-09-19T08:25:58.488 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption2", 2024-09-19T08:25:58.488 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.488 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.488 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.488 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.488 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.488 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.488 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.488 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.488 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.488 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.488 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.489 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.489 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption3": { 2024-09-19T08:25:58.489 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption3", 2024-09-19T08:25:58.489 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-09-19T08:25:58.489 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.489 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.489 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.489 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.489 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.489 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.489 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.489 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.489 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.489 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.490 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.490 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption4": { 2024-09-19T08:25:58.490 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption4", 2024-09-19T08:25:58.490 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.490 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.490 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.490 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.490 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.490 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.490 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.490 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.490 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.490 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.490 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption5": { 2024-09-19T08:25:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption5", 2024-09-19T08:25:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.491 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption6": { 2024-09-19T08:25:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption6", 2024-09-19T08:25:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.492 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption7": { 2024-09-19T08:25:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption7", 2024-09-19T08:25:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-09-19T08:25:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "42", 2024-09-19T08:25:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.493 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: "testkey": { 2024-09-19T08:25:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testkey", 2024-09-19T08:25:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.494 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: "testlkey": { 2024-09-19T08:25:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testlkey", 2024-09-19T08:25:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.495 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: "testnewline": { 2024-09-19T08:25:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testnewline", 2024-09-19T08:25:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.496 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-09-19T08:25:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_m_granularity": { 2024-09-19T08:25:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_m_granularity", 2024-09-19T08:25:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.497 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T08:25:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: "dump_on_update": { 2024-09-19T08:25:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dump_on_update", 2024-09-19T08:25:58.498 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "dump database to debug log on update", 2024-09-19T08:25:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.499 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.500 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.501 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.502 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.503 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.503 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.503 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.503 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.503 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.503 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.503 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.503 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.503 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.503 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.503 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.503 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.503 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.503 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.504 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.504 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.504 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.504 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.504 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.504 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.504 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.504 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.504 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.504 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.504 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.504 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.504 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.505 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.505 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.505 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.505 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.505 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stats", 2024-09-19T08:25:58.505 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.505 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.505 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.505 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.505 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.505 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.505 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.505 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.505 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.506 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.506 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.506 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.506 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.506 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.506 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.506 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.506 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.506 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.506 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.506 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.506 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.506 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.506 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.507 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.507 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.507 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.507 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.507 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.507 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.507 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.507 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.507 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.507 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.507 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.507 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.507 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.508 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.508 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.508 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.508 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.508 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.508 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.508 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.508 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.508 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.508 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.508 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.508 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.508 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.508 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.509 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.509 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.509 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.509 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.509 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.509 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.509 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.509 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.509 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.509 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.509 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.509 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.509 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.510 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.510 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.510 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.510 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.510 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.510 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.510 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.510 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.510 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.510 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.510 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.510 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.510 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.510 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "status", 2024-09-19T08:25:58.511 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.511 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.511 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.511 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.511 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.511 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.511 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.511 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.511 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.511 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.511 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.511 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.511 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.511 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.512 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.512 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.512 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.512 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.512 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.512 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.512 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.512 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.512 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.512 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.512 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.512 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.512 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.512 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.513 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.513 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.513 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.513 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.513 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.513 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.513 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.513 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.513 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.513 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.513 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.513 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.513 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.513 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.514 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.514 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.514 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.514 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.514 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.514 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.514 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.514 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.514 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.514 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.514 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.514 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.514 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.514 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.515 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.515 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.515 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.515 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.515 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.515 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.515 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.515 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.515 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.515 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.515 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.515 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.515 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.515 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.516 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.516 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.516 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.516 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.516 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.516 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.516 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telegraf", 2024-09-19T08:25:58.516 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.516 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.516 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.516 INFO:teuthology.orchestra.run.smithi022.stdout: "address": { 2024-09-19T08:25:58.516 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "address", 2024-09-19T08:25:58.516 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T08:25:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15", 2024-09-19T08:25:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.519 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.520 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.520 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.520 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.520 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.520 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.520 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.520 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.520 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.520 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.520 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.520 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.520 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.520 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.521 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.521 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.521 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.521 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.521 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.521 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.521 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.521 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.521 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.521 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.521 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.521 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.521 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.521 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.522 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.522 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.522 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.522 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.522 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.522 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.522 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.522 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.522 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.522 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.522 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.522 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.522 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.522 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.523 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.523 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.523 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.523 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.523 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.523 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.523 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.523 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.523 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.523 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.523 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.523 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.523 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.523 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.524 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.524 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.524 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telemetry", 2024-09-19T08:25:58.524 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.524 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.524 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.524 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_basic": { 2024-09-19T08:25:58.524 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_basic", 2024-09-19T08:25:58.524 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.524 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.524 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.524 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.524 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.525 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.525 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.525 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T08:25:58.525 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.525 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.525 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.525 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.525 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_crash": { 2024-09-19T08:25:58.525 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_crash", 2024-09-19T08:25:58.525 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.525 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.525 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.525 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.525 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.526 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.526 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.526 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T08:25:58.526 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.526 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.526 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.526 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.526 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_device": { 2024-09-19T08:25:58.526 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_device", 2024-09-19T08:25:58.526 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.526 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.526 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.526 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.526 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.527 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.527 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.527 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T08:25:58.527 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.527 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.527 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.527 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.527 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_ident": { 2024-09-19T08:25:58.527 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_ident", 2024-09-19T08:25:58.527 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.527 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.527 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.527 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.527 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.528 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.528 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.528 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T08:25:58.528 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.528 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.528 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.528 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.528 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_perf": { 2024-09-19T08:25:58.528 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_perf", 2024-09-19T08:25:58.528 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.528 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.528 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.528 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.528 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T08:25:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: "contact": { 2024-09-19T08:25:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "contact", 2024-09-19T08:25:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.529 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.530 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.530 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.530 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.530 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.530 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.530 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.530 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.530 INFO:teuthology.orchestra.run.smithi022.stdout: "description": { 2024-09-19T08:25:58.530 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "description", 2024-09-19T08:25:58.530 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.530 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.530 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.530 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.530 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.531 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.531 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.531 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.531 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.531 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.531 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.531 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.531 INFO:teuthology.orchestra.run.smithi022.stdout: "device_url": { 2024-09-19T08:25:58.531 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_url", 2024-09-19T08:25:58.531 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.531 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.531 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.531 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T08:25:58.532 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.532 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.532 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.532 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.532 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.532 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.532 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.532 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.532 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-09-19T08:25:58.532 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-09-19T08:25:58.532 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.532 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.532 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.532 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.533 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.533 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.533 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.533 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.533 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.533 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.533 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.533 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.533 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:58.533 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:58.533 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.533 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.533 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.534 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "24", 2024-09-19T08:25:58.534 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "8", 2024-09-19T08:25:58.534 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.534 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.534 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.534 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.534 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.534 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.534 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.534 INFO:teuthology.orchestra.run.smithi022.stdout: "last_opt_revision": { 2024-09-19T08:25:58.534 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "last_opt_revision", 2024-09-19T08:25:58.534 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.534 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.534 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.535 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1", 2024-09-19T08:25:58.535 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.535 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.535 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.535 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.535 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.535 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.535 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.535 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.535 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard": { 2024-09-19T08:25:58.535 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard", 2024-09-19T08:25:58.535 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.535 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard_description": { 2024-09-19T08:25:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard_description", 2024-09-19T08:25:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.536 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.537 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.537 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.537 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.537 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.537 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.537 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.537 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.537 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.537 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.537 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.537 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.537 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.537 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.538 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.538 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.538 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.538 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.538 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.538 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.538 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.538 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.538 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.538 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.538 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.538 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.538 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.538 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.539 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.539 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.539 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.539 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.539 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.539 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.539 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.539 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.539 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.539 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.539 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.539 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.539 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.539 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.540 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.540 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.540 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.540 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.540 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.540 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.540 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.540 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.540 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.540 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.540 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.540 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.540 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.541 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.541 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.541 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.541 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.541 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.541 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.541 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.541 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.541 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.541 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.541 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.541 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.541 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.542 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.542 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.542 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.542 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.542 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.542 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.542 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.542 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.542 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.542 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.542 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.542 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.542 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.542 INFO:teuthology.orchestra.run.smithi022.stdout: "organization": { 2024-09-19T08:25:58.543 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "organization", 2024-09-19T08:25:58.543 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.543 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.543 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.543 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.543 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.543 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.543 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.543 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.543 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.543 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.543 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.543 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.543 INFO:teuthology.orchestra.run.smithi022.stdout: "proxy": { 2024-09-19T08:25:58.544 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "proxy", 2024-09-19T08:25:58.544 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.544 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.544 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.544 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.544 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.544 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.544 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.544 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.544 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.544 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.544 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.544 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.544 INFO:teuthology.orchestra.run.smithi022.stdout: "url": { 2024-09-19T08:25:58.545 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url", 2024-09-19T08:25:58.545 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.545 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.545 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.545 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T08:25:58.545 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.545 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.545 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.545 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.545 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.545 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.545 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.545 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.545 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.546 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.546 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.546 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "test_orchestrator", 2024-09-19T08:25:58.546 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.546 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.546 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.546 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.546 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.546 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.546 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.546 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.546 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.546 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.546 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.547 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.547 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.547 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.547 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.547 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.547 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.547 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.547 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.547 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.547 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.547 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.547 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.547 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.547 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.548 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.548 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.548 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.548 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.548 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.548 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.548 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.548 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.548 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.548 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.548 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.548 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.548 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.548 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.549 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.549 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.549 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.549 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.549 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.549 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.549 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.549 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.549 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.549 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.549 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.549 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.549 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.549 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.550 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.550 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.550 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.550 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.550 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.550 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.550 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.550 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.550 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.550 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.550 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.550 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.550 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.550 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.551 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.551 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.551 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.551 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.551 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.551 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.551 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.551 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.551 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.551 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2024-09-19T08:25:58.551 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.551 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.551 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.551 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.552 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.552 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.552 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.552 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.552 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.552 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.552 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.552 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.552 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.552 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.552 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.552 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.552 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.553 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.554 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.554 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.554 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.554 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.554 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.554 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.554 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.554 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.554 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.554 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.554 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.554 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.554 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.555 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.555 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.555 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.555 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.555 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.555 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.555 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.555 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.555 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.555 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.555 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.555 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.555 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.555 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.556 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.556 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.556 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.556 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.556 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.556 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.556 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.556 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.556 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.556 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.556 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.556 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.556 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.557 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.557 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.557 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_clones": { 2024-09-19T08:25:58.557 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_clones", 2024-09-19T08:25:58.557 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.557 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.557 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.557 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4", 2024-09-19T08:25:58.557 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.557 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.557 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.557 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T08:25:58.557 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.557 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.558 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.558 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.558 INFO:teuthology.orchestra.run.smithi022.stdout: "periodic_async_work": { 2024-09-19T08:25:58.558 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "periodic_async_work", 2024-09-19T08:25:58.558 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.558 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.558 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.558 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.558 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.558 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.558 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.558 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Periodically check for async work", 2024-09-19T08:25:58.558 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_delay": { 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_delay", 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.559 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.560 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.560 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.560 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_no_wait": { 2024-09-19T08:25:58.560 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T08:25:58.560 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.560 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.560 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.560 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-09-19T08:25:58.560 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.560 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.560 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.560 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T08:25:58.560 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.560 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.561 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.561 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.561 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.561 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.561 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.561 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix", 2024-09-19T08:25:58.561 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-09-19T08:25:58.561 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-09-19T08:25:58.561 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-09-19T08:25:58.561 INFO:teuthology.orchestra.run.smithi022.stdout: "discovery_interval": { 2024-09-19T08:25:58.561 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "discovery_interval", 2024-09-19T08:25:58.561 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-09-19T08:25:58.561 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.561 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.562 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "100", 2024-09-19T08:25:58.562 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.562 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.562 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.562 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.562 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.562 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.562 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.562 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.562 INFO:teuthology.orchestra.run.smithi022.stdout: "identifier": { 2024-09-19T08:25:58.562 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "identifier", 2024-09-19T08:25:58.562 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.562 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.562 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.563 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.563 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.563 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.563 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.563 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.563 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.563 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.563 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.563 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.563 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-09-19T08:25:58.563 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-09-19T08:25:58.563 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-09-19T08:25:58.563 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.563 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.564 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-09-19T08:25:58.564 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.564 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.564 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.564 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.564 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.564 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.564 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.564 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.564 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-09-19T08:25:58.564 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-09-19T08:25:58.564 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.564 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.565 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.565 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.565 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.565 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.565 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.565 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.565 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.565 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.565 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.565 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.565 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.565 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.565 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.565 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.566 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.566 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.566 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.566 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-09-19T08:25:58.566 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-09-19T08:25:58.566 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.566 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.566 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.566 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.566 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.566 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.566 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.566 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.566 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.567 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.567 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.567 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.567 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-09-19T08:25:58.567 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-09-19T08:25:58.567 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.567 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.567 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.567 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-09-19T08:25:58.567 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.567 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.567 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-09-19T08:25:58.567 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-09-19T08:25:58.568 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-09-19T08:25:58.568 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-09-19T08:25:58.568 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-09-19T08:25:58.568 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-09-19T08:25:58.568 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-09-19T08:25:58.568 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.568 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.568 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.568 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.568 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.568 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.568 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-09-19T08:25:58.568 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-09-19T08:25:58.568 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-09-19T08:25:58.569 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.569 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-19T08:25:58.569 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-09-19T08:25:58.569 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.569 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.569 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.569 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.569 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.569 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.569 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.569 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.569 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_host": { 2024-09-19T08:25:58.569 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_host", 2024-09-19T08:25:58.569 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.570 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.570 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.570 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-09-19T08:25:58.570 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.570 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.570 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.570 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.570 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.570 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.570 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.570 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.570 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_port": { 2024-09-19T08:25:58.570 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_port", 2024-09-19T08:25:58.570 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-09-19T08:25:58.571 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.571 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.571 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10051", 2024-09-19T08:25:58.571 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.571 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.571 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.571 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.571 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.571 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.571 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.571 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_sender": { 2024-09-19T08:25:58.571 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_sender", 2024-09-19T08:25:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-09-19T08:25:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-09-19T08:25:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-09-19T08:25:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T08:25:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-09-19T08:25:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-09-19T08:25:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-09-19T08:25:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-09-19T08:25:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-09-19T08:25:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-09-19T08:25:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-09-19T08:25:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.573 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.573 INFO:teuthology.orchestra.run.smithi022.stdout: "services": { 2024-09-19T08:25:58.573 INFO:teuthology.orchestra.run.smithi022.stdout: "dashboard": "https://172.21.15.22:8443/" 2024-09-19T08:25:58.573 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.573 INFO:teuthology.orchestra.run.smithi022.stdout: "always_on_modules": { 2024-09-19T08:25:58.573 INFO:teuthology.orchestra.run.smithi022.stdout: "octopus": [ 2024-09-19T08:25:58.573 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-09-19T08:25:58.573 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-09-19T08:25:58.573 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-09-19T08:25:58.573 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-09-19T08:25:58.573 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-09-19T08:25:58.573 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-09-19T08:25:58.573 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: "pacific": [ 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-09-19T08:25:58.574 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-09-19T08:25:58.575 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.575 INFO:teuthology.orchestra.run.smithi022.stdout: "quincy": [ 2024-09-19T08:25:58.575 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-09-19T08:25:58.575 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-09-19T08:25:58.575 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-09-19T08:25:58.575 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-09-19T08:25:58.575 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-09-19T08:25:58.575 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-09-19T08:25:58.575 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-09-19T08:25:58.575 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-09-19T08:25:58.575 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-09-19T08:25:58.575 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-09-19T08:25:58.575 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.575 INFO:teuthology.orchestra.run.smithi022.stdout: "reef": [ 2024-09-19T08:25:58.576 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-09-19T08:25:58.576 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-09-19T08:25:58.576 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-09-19T08:25:58.576 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-09-19T08:25:58.576 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-09-19T08:25:58.576 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-09-19T08:25:58.576 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-09-19T08:25:58.576 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-09-19T08:25:58.576 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-09-19T08:25:58.576 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-09-19T08:25:58.576 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-19T08:25:58.576 INFO:teuthology.orchestra.run.smithi022.stdout: "squid": [ 2024-09-19T08:25:58.576 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-09-19T08:25:58.576 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-09-19T08:25:58.577 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-09-19T08:25:58.577 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-09-19T08:25:58.577 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-09-19T08:25:58.577 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-09-19T08:25:58.577 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-09-19T08:25:58.577 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-09-19T08:25:58.577 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-09-19T08:25:58.577 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-09-19T08:25:58.577 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-19T08:25:58.577 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.577 INFO:teuthology.orchestra.run.smithi022.stdout: "last_failure_osd_epoch": 86, 2024-09-19T08:25:58.577 INFO:teuthology.orchestra.run.smithi022.stdout: "active_clients": [ 2024-09-19T08:25:58.577 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.577 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2024-09-19T08:25:58.578 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-19T08:25:58.578 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.578 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-09-19T08:25:58.578 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-09-19T08:25:58.578 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2252841285 2024-09-19T08:25:58.578 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.578 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-19T08:25:58.578 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.578 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.578 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "libcephsqlite", 2024-09-19T08:25:58.578 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-19T08:25:58.578 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.578 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-09-19T08:25:58.578 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-09-19T08:25:58.579 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2132177548 2024-09-19T08:25:58.579 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.579 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-19T08:25:58.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.579 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.579 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2024-09-19T08:25:58.579 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-19T08:25:58.579 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.579 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-09-19T08:25:58.579 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-09-19T08:25:58.579 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3187700041 2024-09-19T08:25:58.579 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.579 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-19T08:25:58.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.580 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.580 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-09-19T08:25:58.580 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-19T08:25:58.580 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.580 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-09-19T08:25:58.580 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-09-19T08:25:58.580 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4189724386 2024-09-19T08:25:58.580 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.580 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-19T08:25:58.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.580 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.580 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-09-19T08:25:58.580 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-19T08:25:58.580 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.581 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-09-19T08:25:58.581 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-09-19T08:25:58.581 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1567014700 2024-09-19T08:25:58.581 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.581 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-19T08:25:58.581 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:58.581 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.581 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2024-09-19T08:25:58.581 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-19T08:25:58.581 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-19T08:25:58.581 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-09-19T08:25:58.581 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-09-19T08:25:58.581 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3494522243 2024-09-19T08:25:58.581 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.582 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-19T08:25:58.582 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:58.582 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-19T08:25:58.582 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:25:58.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.242+0000 7fe442726640 1 -- 172.21.15.22:0/2126556953 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe4240a33c0 msgr2=0x7fe4240a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:58.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.242+0000 7fe442726640 1 --2- 172.21.15.22:0/2126556953 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe4240a33c0 0x7fe4240a5880 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe42c005fd0 tx=0x7fe42c005850 comp rx=0 tx=0).stop 2024-09-19T08:25:58.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.242+0000 7fe442726640 1 -- 172.21.15.22:0/2126556953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe43c102450 msgr2=0x7fe43c198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:25:58.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.242+0000 7fe442726640 1 --2- 172.21.15.22:0/2126556953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe43c102450 0x7fe43c198900 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe430009230 tx=0x7fe430004270 comp rx=0 tx=0).stop 2024-09-19T08:25:58.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.242+0000 7fe442726640 1 -- 172.21.15.22:0/2126556953 shutdown_connections 2024-09-19T08:25:58.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.242+0000 7fe442726640 1 --2- 172.21.15.22:0/2126556953 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fe4240a33c0 0x7fe4240a5880 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:58.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.242+0000 7fe442726640 1 --2- 172.21.15.22:0/2126556953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe43c107ac0 0x7fe43c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:58.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.242+0000 7fe442726640 1 --2- 172.21.15.22:0/2126556953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe43c102450 0x7fe43c198900 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:58.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.242+0000 7fe442726640 1 --2- 172.21.15.22:0/2126556953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c101ab0 0x7fe43c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:25:58.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.242+0000 7fe442726640 1 -- 172.21.15.22:0/2126556953 >> 172.21.15.22:0/2126556953 conn(0x7fe43c0fd7d0 msgr2=0x7fe43c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:25:58.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.242+0000 7fe442726640 1 -- 172.21.15.22:0/2126556953 shutdown_connections 2024-09-19T08:25:58.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:25:57.242+0000 7fe442726640 1 -- 172.21.15.22:0/2126556953 wait complete. 2024-09-19T08:25:58.584 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:57 smithi022 ceph-mon[30963]: pgmap v338: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:25:58.584 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:57 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2849771756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:25:58.585 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:57 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3230729136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T08:25:58.585 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:57 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/399863055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T08:25:58.585 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:57 smithi157 ceph-mon[35143]: pgmap v338: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:25:58.585 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:57 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2849771756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:25:58.585 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:57 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3230729136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T08:25:58.585 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:57 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/399863055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T08:25:58.585 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:58 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/2126556953' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T08:25:58.585 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:57 smithi064 ceph-mon[34900]: pgmap v338: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:25:58.585 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:57 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2849771756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:25:58.585 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:57 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3230729136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T08:25:58.585 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:57 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/399863055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T08:25:58.585 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:58 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/2126556953' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T08:25:58.592 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T08:25:58.592 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T08:25:58.593 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:58 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/2126556953' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T08:25:58.857 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:25:59.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:25:59 smithi022 ceph-mon[30963]: pgmap v339: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T08:25:59.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:25:59 smithi064 ceph-mon[34900]: pgmap v339: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T08:25:59.404 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:25:59.405 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:25:59.405 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 228, 2024-09-19T08:25:59.405 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 215, 2024-09-19T08:25:59.405 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1590336, 2024-09-19T08:25:59.405 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 131072, 2024-09-19T08:25:59.405 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2024-09-19T08:25:59.406 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2024-09-19T08:25:59.406 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.042921292, 2024-09-19T08:25:59.406 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.014124974, 2024-09-19T08:25:59.406 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.040750930, 2024-09-19T08:25:59.406 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:25:59.407 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:25:59.407 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 215, 2024-09-19T08:25:59.407 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.016493531, 2024-09-19T08:25:59.407 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000076714 2024-09-19T08:25:59.407 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:59.407 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:25:59.408 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:25:59.408 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:25:59.408 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:25:59.408 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:59.408 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1590336, 2024-09-19T08:25:59.408 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 131072 2024-09-19T08:25:59.409 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:59.409 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:25:59.409 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 2, 2024-09-19T08:25:59.409 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 163, 2024-09-19T08:25:59.409 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 320, 2024-09-19T08:25:59.409 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 185920, 2024-09-19T08:25:59.409 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 5, 2024-09-19T08:25:59.410 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 3, 2024-09-19T08:25:59.410 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.013744206, 2024-09-19T08:25:59.410 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.009128686, 2024-09-19T08:25:59.410 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.002375429, 2024-09-19T08:25:59.410 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:25:59.410 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:25:59.411 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 160, 2024-09-19T08:25:59.411 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.011995987, 2024-09-19T08:25:59.411 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000074974 2024-09-19T08:25:59.411 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:59.411 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:25:59.411 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:25:59.411 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:25:59.412 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:25:59.412 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:59.412 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T08:25:59.412 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 185920 2024-09-19T08:25:59.412 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:59.412 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:25:59.412 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 0, 2024-09-19T08:25:59.412 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 2, 2024-09-19T08:25:59.413 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 0, 2024-09-19T08:25:59.413 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 0, 2024-09-19T08:25:59.413 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2024-09-19T08:25:59.413 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2024-09-19T08:25:59.413 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.000212933, 2024-09-19T08:25:59.413 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T08:25:59.413 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.000135371, 2024-09-19T08:25:59.413 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:25:59.413 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:25:59.413 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:25:59.414 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:25:59.414 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:25:59.414 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:59.414 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:25:59.414 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:25:59.414 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:25:59.414 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:25:59.414 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:59.414 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T08:25:59.415 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T08:25:59.415 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:59.415 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-09-19T08:25:59.415 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2024-09-19T08:25:59.415 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-09-19T08:25:59.415 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:59.415 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-09-19T08:25:59.415 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:25:59.415 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-09-19T08:25:59.415 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:25:59.415 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:25:59.415 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:25:59.415 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:25:59.415 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:25:59.416 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:25:59.416 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:25:59.416 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:25:59.416 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:25:59.416 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:25:59.416 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:25:59.416 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:25:59.416 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:25:59.416 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-09-19T08:25:59.416 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:25:59.416 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-09-19T08:25:59.416 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:25:59.416 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:25:59.417 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:25:59.417 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:25:59.417 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:25:59.417 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:25:59.417 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:25:59.417 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-09-19T08:25:59.417 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:25:59.417 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:25:59.417 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:25:59.417 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-09-19T08:25:59.417 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-09-19T08:25:59.417 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-09-19T08:25:59.417 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:25:59.417 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:25:59.418 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:25:59.418 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:25:59.418 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 148995, 2024-09-19T08:25:59.418 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 31, 2024-09-19T08:25:59.418 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:25:59.418 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-09-19T08:25:59.418 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-09-19T08:25:59.418 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-09-19T08:25:59.418 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:25:59.418 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-09-19T08:25:59.418 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:25:59.418 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-09-19T08:25:59.418 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2024-09-19T08:25:59.419 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2024-09-19T08:25:59.419 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:25:59.419 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:25:59.419 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-09-19T08:25:59.419 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-09-19T08:25:59.419 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2024-09-19T08:25:59.419 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2024-09-19T08:25:59.419 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-09-19T08:25:59.419 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-09-19T08:25:59.419 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-09-19T08:25:59.419 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-09-19T08:25:59.419 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:25:59.419 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:25:59.419 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-09-19T08:25:59.420 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-09-19T08:25:59.420 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-09-19T08:25:59.420 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 226, 2024-09-19T08:25:59.420 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1555722, 2024-09-19T08:25:59.420 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-09-19T08:25:59.420 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 226, 2024-09-19T08:25:59.420 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-09-19T08:25:59.420 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-09-19T08:25:59.420 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 226, 2024-09-19T08:25:59.420 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1555722, 2024-09-19T08:25:59.420 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-09-19T08:25:59.420 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:25:59.420 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:25:59.420 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:25:59.421 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:59.421 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:25:59.421 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:25:59.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:25:59 smithi157 ceph-mon[35143]: pgmap v339: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T08:26:00.325 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-19T08:26:00.326 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-19T08:26:00.326 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-19T08:26:00.326 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-19T08:26:00.326 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-19T08:26:00.326 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-09-19T08:26:00.326 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T08:26:00.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T08:26:00.610 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:26:01.151 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:26:01.151 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:26:01.152 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 0, 2024-09-19T08:26:01.152 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 2, 2024-09-19T08:26:01.152 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 0, 2024-09-19T08:26:01.152 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 0, 2024-09-19T08:26:01.152 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2024-09-19T08:26:01.152 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2024-09-19T08:26:01.152 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.001246513, 2024-09-19T08:26:01.152 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T08:26:01.152 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.000223831, 2024-09-19T08:26:01.152 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:01.153 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:01.153 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:26:01.153 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:26:01.153 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:26:01.153 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:01.153 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:01.153 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:26:01.153 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:26:01.153 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:26:01.153 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:01.153 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T08:26:01.154 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T08:26:01.154 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:01.154 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:26:01.154 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 6, 2024-09-19T08:26:01.154 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 169, 2024-09-19T08:26:01.154 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 3344, 2024-09-19T08:26:01.154 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 187072, 2024-09-19T08:26:01.154 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 5, 2024-09-19T08:26:01.154 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 3, 2024-09-19T08:26:01.154 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.016360260, 2024-09-19T08:26:01.154 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.010042722, 2024-09-19T08:26:01.155 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.003987826, 2024-09-19T08:26:01.155 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:01.155 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:01.155 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 167, 2024-09-19T08:26:01.155 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.015455544, 2024-09-19T08:26:01.155 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000092548 2024-09-19T08:26:01.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:01.155 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:01.155 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:26:01.155 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:26:01.155 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:26:01.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:01.155 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-19T08:26:01.155 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 187072 2024-09-19T08:26:01.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:01.156 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:26:01.156 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 230, 2024-09-19T08:26:01.156 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 219, 2024-09-19T08:26:01.156 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1588944, 2024-09-19T08:26:01.156 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 134464, 2024-09-19T08:26:01.156 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2024-09-19T08:26:01.156 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2024-09-19T08:26:01.156 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.039050119, 2024-09-19T08:26:01.156 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.013882158, 2024-09-19T08:26:01.156 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.038905466, 2024-09-19T08:26:01.156 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:01.157 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:01.157 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 218, 2024-09-19T08:26:01.157 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.009434090, 2024-09-19T08:26:01.157 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000043275 2024-09-19T08:26:01.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:01.157 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:01.157 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:26:01.157 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:26:01.157 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:26:01.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:01.157 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1588944, 2024-09-19T08:26:01.158 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 134464 2024-09-19T08:26:01.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:01.158 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-09-19T08:26:01.158 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2024-09-19T08:26:01.158 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-09-19T08:26:01.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:01.158 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-09-19T08:26:01.158 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:26:01.158 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-09-19T08:26:01.158 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:26:01.158 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:26:01.158 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:26:01.158 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:26:01.159 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:26:01.159 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:26:01.159 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:26:01.159 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:26:01.159 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:26:01.159 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:26:01.159 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:26:01.159 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:26:01.159 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:26:01.159 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-09-19T08:26:01.159 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:26:01.159 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-09-19T08:26:01.159 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:26:01.160 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:26:01.160 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:26:01.160 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:26:01.160 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:26:01.160 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:26:01.160 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:26:01.160 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-09-19T08:26:01.160 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:26:01.160 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:26:01.160 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:26:01.160 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-09-19T08:26:01.160 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-09-19T08:26:01.160 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-09-19T08:26:01.161 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:26:01.161 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:26:01.162 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:26:01.162 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:26:01.162 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 149034, 2024-09-19T08:26:01.162 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 32, 2024-09-19T08:26:01.162 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:26:01.162 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-09-19T08:26:01.162 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-09-19T08:26:01.163 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-09-19T08:26:01.163 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:26:01.163 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-09-19T08:26:01.163 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:26:01.163 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-09-19T08:26:01.163 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2024-09-19T08:26:01.163 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2024-09-19T08:26:01.163 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:26:01.164 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:26:01.164 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-09-19T08:26:01.164 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-09-19T08:26:01.164 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2024-09-19T08:26:01.164 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2024-09-19T08:26:01.164 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-09-19T08:26:01.165 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-09-19T08:26:01.165 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-09-19T08:26:01.165 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-09-19T08:26:01.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:01.165 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:26:01.165 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-09-19T08:26:01.166 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-09-19T08:26:01.166 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-09-19T08:26:01.166 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 232, 2024-09-19T08:26:01.166 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1556502, 2024-09-19T08:26:01.166 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:01.166 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 232, 2024-09-19T08:26:01.167 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-09-19T08:26:01.167 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-09-19T08:26:01.167 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 232, 2024-09-19T08:26:01.167 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1556502, 2024-09-19T08:26:01.167 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-09-19T08:26:01.167 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:26:01.168 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:26:01.168 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:26:01.168 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:26:01.168 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:26:01.168 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:26:01.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:01 smithi157 ceph-mon[35143]: pgmap v340: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T08:26:01.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:01 smithi022 ceph-mon[30963]: pgmap v340: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T08:26:01.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:01 smithi064 ceph-mon[34900]: pgmap v340: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T08:26:02.006 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T08:26:02.006 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T08:26:02.006 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-19T08:26:02.006 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-19T08:26:02.006 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-19T08:26:02.006 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-19T08:26:02.007 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T08:26:02.007 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T08:26:02.265 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:26:02.810 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:26:02.811 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:26:02.811 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 4, 2024-09-19T08:26:02.811 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 4, 2024-09-19T08:26:02.811 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 3024, 2024-09-19T08:26:02.811 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 448, 2024-09-19T08:26:02.811 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2024-09-19T08:26:02.811 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2024-09-19T08:26:02.811 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.003301660, 2024-09-19T08:26:02.811 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000032602, 2024-09-19T08:26:02.811 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.002203678, 2024-09-19T08:26:02.811 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:02.811 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:02.812 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 3, 2024-09-19T08:26:02.812 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.004193248, 2024-09-19T08:26:02.812 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.001397749 2024-09-19T08:26:02.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:02.812 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:02.812 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:26:02.812 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:26:02.812 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:26:02.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:02.812 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T08:26:02.812 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T08:26:02.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:02.812 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:26:02.813 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 2, 2024-09-19T08:26:02.813 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 165, 2024-09-19T08:26:02.813 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 320, 2024-09-19T08:26:02.813 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 186272, 2024-09-19T08:26:02.813 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 5, 2024-09-19T08:26:02.813 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 3, 2024-09-19T08:26:02.813 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.013463734, 2024-09-19T08:26:02.813 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.009100973, 2024-09-19T08:26:02.813 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.002089560, 2024-09-19T08:26:02.813 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:02.813 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:02.813 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 162, 2024-09-19T08:26:02.813 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.011289716, 2024-09-19T08:26:02.814 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000069689 2024-09-19T08:26:02.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:02.814 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:02.814 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:26:02.814 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:26:02.814 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:26:02.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:02.814 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T08:26:02.814 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 186272 2024-09-19T08:26:02.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:02.814 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:26:02.814 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 226, 2024-09-19T08:26:02.814 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 215, 2024-09-19T08:26:02.815 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1587856, 2024-09-19T08:26:02.815 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 131904, 2024-09-19T08:26:02.815 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2024-09-19T08:26:02.815 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2024-09-19T08:26:02.815 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.042323559, 2024-09-19T08:26:02.815 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.014588319, 2024-09-19T08:26:02.815 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.040336590, 2024-09-19T08:26:02.815 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:02.815 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:02.815 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 214, 2024-09-19T08:26:02.815 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.013273330, 2024-09-19T08:26:02.815 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000062024 2024-09-19T08:26:02.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:02.816 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:02.816 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:26:02.816 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:26:02.816 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:26:02.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:02.816 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1587856, 2024-09-19T08:26:02.816 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 131904 2024-09-19T08:26:02.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:02.816 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-09-19T08:26:02.816 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2024-09-19T08:26:02.816 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-09-19T08:26:02.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:02.816 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-09-19T08:26:02.817 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:26:02.817 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-09-19T08:26:02.817 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:26:02.817 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:26:02.817 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:26:02.817 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:26:02.817 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:26:02.817 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:26:02.817 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:26:02.817 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:26:02.817 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:26:02.817 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:26:02.817 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:26:02.817 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:26:02.818 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:26:02.818 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-09-19T08:26:02.818 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:26:02.818 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-09-19T08:26:02.818 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:26:02.818 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:26:02.818 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:26:02.818 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:26:02.818 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:26:02.818 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:26:02.818 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:26:02.818 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-09-19T08:26:02.818 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:26:02.819 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:26:02.819 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:26:02.819 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-09-19T08:26:02.819 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-09-19T08:26:02.819 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-09-19T08:26:02.819 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:26:02.819 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:26:02.819 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:26:02.819 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:26:02.819 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 149017, 2024-09-19T08:26:02.819 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 31, 2024-09-19T08:26:02.819 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:26:02.819 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-09-19T08:26:02.819 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-09-19T08:26:02.820 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-09-19T08:26:02.820 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:26:02.820 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-09-19T08:26:02.820 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:26:02.820 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-09-19T08:26:02.820 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2024-09-19T08:26:02.820 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2024-09-19T08:26:02.820 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:26:02.820 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:26:02.820 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-09-19T08:26:02.820 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-09-19T08:26:02.820 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2024-09-19T08:26:02.820 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2024-09-19T08:26:02.821 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-09-19T08:26:02.821 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-09-19T08:26:02.821 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-09-19T08:26:02.821 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-09-19T08:26:02.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:02.821 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:26:02.821 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-09-19T08:26:02.821 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-09-19T08:26:02.821 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-09-19T08:26:02.821 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 228, 2024-09-19T08:26:02.821 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1555982, 2024-09-19T08:26:02.821 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:02.821 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 228, 2024-09-19T08:26:02.821 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-09-19T08:26:02.822 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-09-19T08:26:02.822 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 228, 2024-09-19T08:26:02.822 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1555982, 2024-09-19T08:26:02.822 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-09-19T08:26:02.822 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:26:02.822 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:26:02.822 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:26:02.822 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:26:02.822 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:26:02.822 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:26:03.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:03 smithi157 ceph-mon[35143]: pgmap v341: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:26:03.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:03 smithi022 ceph-mon[30963]: pgmap v341: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:26:03.568 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-19T08:26:03.568 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-19T08:26:03.568 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-19T08:26:03.568 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-19T08:26:03.568 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-19T08:26:03.568 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-09-19T08:26:03.568 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T08:26:03.569 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T08:26:03.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:03 smithi064 ceph-mon[34900]: pgmap v341: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T08:26:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:26:04.420 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T08:26:04.420 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:26:04.420 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 0, 2024-09-19T08:26:04.420 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 2, 2024-09-19T08:26:04.420 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 0, 2024-09-19T08:26:04.420 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 0, 2024-09-19T08:26:04.420 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2024-09-19T08:26:04.420 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2024-09-19T08:26:04.421 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.001344367, 2024-09-19T08:26:04.421 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T08:26:04.421 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.000312424, 2024-09-19T08:26:04.421 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:04.421 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:04.421 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:26:04.421 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:26:04.421 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:26:04.421 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:04.421 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:04.421 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:26:04.421 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:26:04.421 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:26:04.422 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:04.422 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T08:26:04.422 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T08:26:04.422 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:04.422 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:26:04.422 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 231, 2024-09-19T08:26:04.422 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 220, 2024-09-19T08:26:04.422 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1589216, 2024-09-19T08:26:04.422 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 135104, 2024-09-19T08:26:04.422 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2024-09-19T08:26:04.422 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2024-09-19T08:26:04.422 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.043444687, 2024-09-19T08:26:04.422 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.013691696, 2024-09-19T08:26:04.423 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.043109332, 2024-09-19T08:26:04.423 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:04.423 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:04.423 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 219, 2024-09-19T08:26:04.423 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.013651444, 2024-09-19T08:26:04.423 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000062335 2024-09-19T08:26:04.423 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:04.423 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:04.423 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:26:04.423 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:26:04.423 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:26:04.423 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:04.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1589216, 2024-09-19T08:26:04.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 135104 2024-09-19T08:26:04.424 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:04.424 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:26:04.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 6, 2024-09-19T08:26:04.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 171, 2024-09-19T08:26:04.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 3344, 2024-09-19T08:26:04.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 187424, 2024-09-19T08:26:04.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 5, 2024-09-19T08:26:04.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 3, 2024-09-19T08:26:04.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.014582128, 2024-09-19T08:26:04.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.008481824, 2024-09-19T08:26:04.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.004053586, 2024-09-19T08:26:04.425 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:04.425 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:04.425 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 169, 2024-09-19T08:26:04.425 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.020037317, 2024-09-19T08:26:04.425 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000118564 2024-09-19T08:26:04.425 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:04.425 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:04.425 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:26:04.425 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:26:04.425 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:26:04.425 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:04.425 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-19T08:26:04.425 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 187424 2024-09-19T08:26:04.425 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:04.426 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-09-19T08:26:04.426 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2024-09-19T08:26:04.426 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-09-19T08:26:04.426 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:04.426 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-09-19T08:26:04.426 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:26:04.426 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-09-19T08:26:04.426 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:26:04.426 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:26:04.426 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:26:04.426 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:26:04.426 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:26:04.426 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:26:04.427 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:26:04.427 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:26:04.427 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:26:04.427 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:26:04.427 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:26:04.427 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:26:04.427 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:26:04.427 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-09-19T08:26:04.427 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:26:04.427 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-09-19T08:26:04.427 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:26:04.427 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:26:04.427 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:26:04.428 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:26:04.428 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:26:04.428 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:26:04.428 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:26:04.428 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-09-19T08:26:04.428 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:26:04.428 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:26:04.428 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:26:04.428 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-09-19T08:26:04.428 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-09-19T08:26:04.428 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-09-19T08:26:04.428 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:26:04.428 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:26:04.429 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:26:04.429 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:26:04.429 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 148994, 2024-09-19T08:26:04.429 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 31, 2024-09-19T08:26:04.429 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:26:04.429 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-09-19T08:26:04.429 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-09-19T08:26:04.429 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-09-19T08:26:04.429 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:26:04.429 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-09-19T08:26:04.429 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:26:04.429 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-09-19T08:26:04.429 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2024-09-19T08:26:04.429 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2024-09-19T08:26:04.430 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:26:04.430 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:26:04.430 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-09-19T08:26:04.430 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-09-19T08:26:04.430 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2024-09-19T08:26:04.430 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2024-09-19T08:26:04.430 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-09-19T08:26:04.430 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-09-19T08:26:04.430 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-09-19T08:26:04.430 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-09-19T08:26:04.430 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-19T08:26:04.430 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:26:04.430 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-09-19T08:26:04.430 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-09-19T08:26:04.431 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-09-19T08:26:04.431 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 233, 2024-09-19T08:26:04.431 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1556632, 2024-09-19T08:26:04.431 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:04.431 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 233, 2024-09-19T08:26:04.431 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-09-19T08:26:04.431 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-09-19T08:26:04.431 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 233, 2024-09-19T08:26:04.431 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1556632, 2024-09-19T08:26:04.431 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-09-19T08:26:04.431 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-09-19T08:26:04.431 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-09-19T08:26:04.431 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-09-19T08:26:04.432 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:26:04.432 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-19T08:26:04.432 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T08:26:05.168 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T08:26:05.168 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T08:26:05.168 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-19T08:26:05.168 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-19T08:26:05.168 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-19T08:26:05.168 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-19T08:26:05.169 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T08:26:05.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T08:26:05.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:05 smithi022 ceph-mon[30963]: pgmap v342: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-19T08:26:05.435 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:26:05.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:05 smithi157 ceph-mon[35143]: pgmap v342: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-19T08:26:05.555 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:05 smithi064 ceph-mon[34900]: pgmap v342: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-19T08:26:05.995 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T08:26:05.996 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:26:05.996 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 48388, 2024-09-19T08:26:05.996 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 54448, 2024-09-19T08:26:05.996 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 17576411, 2024-09-19T08:26:05.996 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 48501747, 2024-09-19T08:26:05.996 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 10, 2024-09-19T08:26:05.996 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 5, 2024-09-19T08:26:05.996 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 2.270405429, 2024-09-19T08:26:05.996 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 1.025946929, 2024-09-19T08:26:05.996 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 1.955949821, 2024-09-19T08:26:05.996 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.060075175, 2024-09-19T08:26:05.997 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:05.997 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 54446, 2024-09-19T08:26:05.997 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1.803035461, 2024-09-19T08:26:05.997 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000033116 2024-09-19T08:26:05.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:05.997 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:05.997 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 86437, 2024-09-19T08:26:05.997 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.012918330, 2024-09-19T08:26:05.997 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000149 2024-09-19T08:26:05.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:05.997 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 16639152, 2024-09-19T08:26:05.997 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 36190912 2024-09-19T08:26:05.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:05.998 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:26:05.998 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 1647, 2024-09-19T08:26:05.998 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 1767, 2024-09-19T08:26:05.998 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2021233, 2024-09-19T08:26:05.998 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 1344213, 2024-09-19T08:26:05.998 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 34, 2024-09-19T08:26:05.998 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 9, 2024-09-19T08:26:05.998 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.142392489, 2024-09-19T08:26:05.998 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.050306873, 2024-09-19T08:26:05.998 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.073567382, 2024-09-19T08:26:05.998 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.025827509, 2024-09-19T08:26:05.998 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:05.998 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1767, 2024-09-19T08:26:05.999 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.071266479, 2024-09-19T08:26:05.999 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000040331 2024-09-19T08:26:05.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:05.999 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:05.999 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 67, 2024-09-19T08:26:05.999 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000011575, 2024-09-19T08:26:05.999 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000172 2024-09-19T08:26:05.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:05.999 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1686112, 2024-09-19T08:26:05.999 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 482272 2024-09-19T08:26:05.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:05.999 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:26:06.000 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 7501, 2024-09-19T08:26:06.000 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 11015, 2024-09-19T08:26:06.000 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2903855, 2024-09-19T08:26:06.000 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 5375334, 2024-09-19T08:26:06.000 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 18, 2024-09-19T08:26:06.000 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 6, 2024-09-19T08:26:06.000 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.545464236, 2024-09-19T08:26:06.000 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.205147862, 2024-09-19T08:26:06.000 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.343787707, 2024-09-19T08:26:06.000 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.042605945, 2024-09-19T08:26:06.000 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:06.000 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 11012, 2024-09-19T08:26:06.000 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.508433395, 2024-09-19T08:26:06.001 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000046170 2024-09-19T08:26:06.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.001 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:06.001 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 14115, 2024-09-19T08:26:06.001 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.002393604, 2024-09-19T08:26:06.001 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000169 2024-09-19T08:26:06.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.001 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 4304, 2024-09-19T08:26:06.001 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 132496 2024-09-19T08:26:06.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.001 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-19T08:26:06.001 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2024-09-19T08:26:06.001 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-19T08:26:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2024-09-19T08:26:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-09-19T08:26:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-09-19T08:26:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3635, 2024-09-19T08:26:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 2.887840067, 2024-09-19T08:26:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000794453 2024-09-19T08:26:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2024-09-19T08:26:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-09-19T08:26:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-09-19T08:26:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2465, 2024-09-19T08:26:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.188043436, 2024-09-19T08:26:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000076285 2024-09-19T08:26:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-09-19T08:26:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 29294, 2024-09-19T08:26:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 29045, 2024-09-19T08:26:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2024-09-19T08:26:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 29045, 2024-09-19T08:26:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 786.224961963, 2024-09-19T08:26:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.027069201 2024-09-19T08:26:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2024-09-19T08:26:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 249, 2024-09-19T08:26:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 0, 2024-09-19T08:26:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2024-09-19T08:26:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 29, 2024-09-19T08:26:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 2, 2024-09-19T08:26:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2024-09-19T08:26:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 17, 2024-09-19T08:26:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 2, 2024-09-19T08:26:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 2, 2024-09-19T08:26:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 13, 2024-09-19T08:26:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 13, 2024-09-19T08:26:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 2499, 2024-09-19T08:26:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 0, 2024-09-19T08:26:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 0, 2024-09-19T08:26:06.005 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 4, 2024-09-19T08:26:06.005 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 59483, 2024-09-19T08:26:06.005 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 52432, 2024-09-19T08:26:06.005 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 247, 2024-09-19T08:26:06.005 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 30, 2024-09-19T08:26:06.005 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:26:06.005 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2024-09-19T08:26:06.005 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 420, 2024-09-19T08:26:06.005 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 0, 2024-09-19T08:26:06.005 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2024-09-19T08:26:06.005 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 24, 2024-09-19T08:26:06.005 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 2307, 2024-09-19T08:26:06.005 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 10, 2024-09-19T08:26:06.006 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 976, 2024-09-19T08:26:06.006 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2024-09-19T08:26:06.006 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 3278, 2024-09-19T08:26:06.006 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 1, 2024-09-19T08:26:06.006 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 4, 2024-09-19T08:26:06.006 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 234, 2024-09-19T08:26:06.006 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 6, 2024-09-19T08:26:06.006 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 3278, 2024-09-19T08:26:06.006 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2024-09-19T08:26:06.006 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:26:06.006 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:26:06.006 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 0, 2024-09-19T08:26:06.006 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T08:26:06.006 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T08:26:06.007 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 3278, 2024-09-19T08:26:06.007 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 65, 2024-09-19T08:26:06.007 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 5053, 2024-09-19T08:26:06.007 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 4853, 2024-09-19T08:26:06.007 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T08:26:06.007 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:26:06.007 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:26:06.007 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 7456, 2024-09-19T08:26:06.007 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 1161, 2024-09-19T08:26:06.007 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 173, 2024-09-19T08:26:06.007 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 4434 2024-09-19T08:26:06.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.007 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2024-09-19T08:26:06.008 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 9, 2024-09-19T08:26:06.008 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 11, 2024-09-19T08:26:06.008 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 7, 2024-09-19T08:26:06.008 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 19, 2024-09-19T08:26:06.008 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 31, 2024-09-19T08:26:06.008 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 0, 2024-09-19T08:26:06.008 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2024-09-19T08:26:06.008 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:26:06.008 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 1276, 2024-09-19T08:26:06.008 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 1276, 2024-09-19T08:26:06.008 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2024-09-19T08:26:06.008 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2024-09-19T08:26:06.008 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2024-09-19T08:26:06.009 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:26:06.009 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:26:06.009 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2024-09-19T08:26:06.009 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2024-09-19T08:26:06.009 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_path": 8, 2024-09-19T08:26:06.009 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_inode": 2349, 2024-09-19T08:26:06.009 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T08:26:06.009 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 24, 2024-09-19T08:26:06.009 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2024-09-19T08:26:06.009 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 2, 2024-09-19T08:26:06.009 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2024-09-19T08:26:06.009 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2024-09-19T08:26:06.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.010 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2024-09-19T08:26:06.010 INFO:teuthology.orchestra.run.smithi064.stdout: "evlrg": 0, 2024-09-19T08:26:06.010 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 6484, 2024-09-19T08:26:06.010 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 0, 2024-09-19T08:26:06.010 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 0, 2024-09-19T08:26:06.010 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 6484, 2024-09-19T08:26:06.010 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2024-09-19T08:26:06.010 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 0, 2024-09-19T08:26:06.010 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 9, 2024-09-19T08:26:06.010 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 0, 2024-09-19T08:26:06.010 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 0, 2024-09-19T08:26:06.010 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 9, 2024-09-19T08:26:06.010 INFO:teuthology.orchestra.run.smithi064.stdout: "segmjr": 2, 2024-09-19T08:26:06.010 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2024-09-19T08:26:06.011 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 0, 2024-09-19T08:26:06.011 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 4194304, 2024-09-19T08:26:06.011 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 16367029, 2024-09-19T08:26:06.011 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 0, 2024-09-19T08:26:06.011 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2024-09-19T08:26:06.011 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 627, 2024-09-19T08:26:06.011 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 2.906078462, 2024-09-19T08:26:06.011 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.004634893 2024-09-19T08:26:06.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.011 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 0 2024-09-19T08:26:06.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.011 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2024-09-19T08:26:06.011 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 22, 2024-09-19T08:26:06.012 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 2686, 2024-09-19T08:26:06.012 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 2664, 2024-09-19T08:26:06.012 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 21, 2024-09-19T08:26:06.012 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 65, 2024-09-19T08:26:06.012 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 44, 2024-09-19T08:26:06.012 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 17, 2024-09-19T08:26:06.012 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 3981, 2024-09-19T08:26:06.012 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 3964, 2024-09-19T08:26:06.012 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 0, 2024-09-19T08:26:06.012 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 1756, 2024-09-19T08:26:06.012 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 1756, 2024-09-19T08:26:06.012 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 76612, 2024-09-19T08:26:06.012 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 223520 2024-09-19T08:26:06.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.013 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2024-09-19T08:26:06.013 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 33181, 2024-09-19T08:26:06.013 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 25, 2024-09-19T08:26:06.013 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 29294, 2024-09-19T08:26:06.013 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 29, 2024-09-19T08:26:06.013 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 2416, 2024-09-19T08:26:06.013 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2024-09-19T08:26:06.013 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 848, 2024-09-19T08:26:06.013 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 65.467434559, 2024-09-19T08:26:06.013 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.077202163 2024-09-19T08:26:06.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.013 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2024-09-19T08:26:06.014 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1674, 2024-09-19T08:26:06.014 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 10.315724155, 2024-09-19T08:26:06.014 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.006162320 2024-09-19T08:26:06.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.014 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2024-09-19T08:26:06.014 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.014 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.014 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.014 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2024-09-19T08:26:06.014 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.014 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.014 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.015 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2024-09-19T08:26:06.015 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 24437, 2024-09-19T08:26:06.015 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 697.101074653, 2024-09-19T08:26:06.015 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.028526458 2024-09-19T08:26:06.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.015 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2024-09-19T08:26:06.015 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.015 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.015 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.015 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2024-09-19T08:26:06.015 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.015 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.015 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.016 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2024-09-19T08:26:06.016 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.016 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.016 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.016 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2024-09-19T08:26:06.016 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.016 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.016 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.016 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2024-09-19T08:26:06.016 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.017 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.017 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.017 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2024-09-19T08:26:06.017 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.017 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.018 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.018 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2024-09-19T08:26:06.018 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.018 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.018 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.018 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2024-09-19T08:26:06.018 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.018 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.018 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.018 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2024-09-19T08:26:06.019 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.019 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.019 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.019 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2024-09-19T08:26:06.019 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 612, 2024-09-19T08:26:06.019 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.358588691, 2024-09-19T08:26:06.019 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000585929 2024-09-19T08:26:06.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.019 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2024-09-19T08:26:06.019 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 24, 2024-09-19T08:26:06.019 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.027549842, 2024-09-19T08:26:06.019 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001147910 2024-09-19T08:26:06.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.020 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2024-09-19T08:26:06.020 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 849, 2024-09-19T08:26:06.020 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 11.991066159, 2024-09-19T08:26:06.020 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.014123752 2024-09-19T08:26:06.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.020 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2024-09-19T08:26:06.020 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.020 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.020 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.020 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2024-09-19T08:26:06.020 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.020 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.021 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.021 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2024-09-19T08:26:06.021 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.021 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.021 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.021 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2024-09-19T08:26:06.021 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.021 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.021 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.021 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2024-09-19T08:26:06.022 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.022 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.022 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.022 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2024-09-19T08:26:06.022 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.022 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.022 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.022 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2024-09-19T08:26:06.022 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.022 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.022 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.023 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2024-09-19T08:26:06.023 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.023 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.023 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.023 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2024-09-19T08:26:06.023 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.023 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.023 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.023 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2024-09-19T08:26:06.023 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.023 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.023 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.024 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2024-09-19T08:26:06.024 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.024 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.024 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.024 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2024-09-19T08:26:06.024 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 601, 2024-09-19T08:26:06.024 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.963523904, 2024-09-19T08:26:06.024 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001603201 2024-09-19T08:26:06.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.024 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:26:06.025 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:26:06.025 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2024-09-19T08:26:06.025 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.025 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.025 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.025 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:06.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.025 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2024-09-19T08:26:06.025 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 2, 2024-09-19T08:26:06.025 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 5, 2024-09-19T08:26:06.025 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 3, 2024-09-19T08:26:06.025 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 2, 2024-09-19T08:26:06.025 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2024-09-19T08:26:06.026 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 1814, 2024-09-19T08:26:06.026 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 907, 2024-09-19T08:26:06.026 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 622, 2024-09-19T08:26:06.026 INFO:teuthology.orchestra.run.smithi064.stdout: "mdthresh_evicted": 0 2024-09-19T08:26:06.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.026 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-19T08:26:06.026 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:26:06.026 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-19T08:26:06.026 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:26:06.026 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:26:06.026 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:26:06.026 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:26:06.026 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:26:06.027 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:26:06.027 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:26:06.027 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:26:06.027 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:26:06.027 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:26:06.027 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:26:06.027 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:26:06.027 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:26:06.027 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-19T08:26:06.027 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:26:06.027 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-19T08:26:06.027 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:26:06.027 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:26:06.028 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:26:06.028 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:26:06.028 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:26:06.028 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:26:06.028 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:26:06.028 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-19T08:26:06.028 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:26:06.028 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:26:06.028 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:26:06.028 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-19T08:26:06.028 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-19T08:26:06.028 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-19T08:26:06.028 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:26:06.028 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:26:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:26:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:26:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 280263, 2024-09-19T08:26:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 106, 2024-09-19T08:26:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 88, 2024-09-19T08:26:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 1, 2024-09-19T08:26:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-19T08:26:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-19T08:26:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:26:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-19T08:26:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:26:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-19T08:26:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 8224, 2024-09-19T08:26:06.030 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 181, 2024-09-19T08:26:06.030 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:26:06.030 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:26:06.030 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-19T08:26:06.030 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-19T08:26:06.030 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 139589, 2024-09-19T08:26:06.030 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 529, 2024-09-19T08:26:06.030 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-19T08:26:06.030 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-19T08:26:06.030 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-19T08:26:06.030 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-19T08:26:06.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.030 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2024-09-19T08:26:06.030 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2024-09-19T08:26:06.031 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2024-09-19T08:26:06.031 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 5656, 2024-09-19T08:26:06.031 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 12892318, 2024-09-19T08:26:06.031 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 28, 2024-09-19T08:26:06.031 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 5656, 2024-09-19T08:26:06.031 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2024-09-19T08:26:06.031 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 5656, 2024-09-19T08:26:06.031 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 19.262615753, 2024-09-19T08:26:06.031 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.003405695 2024-09-19T08:26:06.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.031 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2024-09-19T08:26:06.031 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2024-09-19T08:26:06.031 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 5656, 2024-09-19T08:26:06.031 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 5878 2024-09-19T08:26:06.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.032 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 5656, 2024-09-19T08:26:06.032 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 3367, 2024-09-19T08:26:06.032 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 2289, 2024-09-19T08:26:06.032 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2024-09-19T08:26:06.032 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2024-09-19T08:26:06.032 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 0, 2024-09-19T08:26:06.032 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 10, 2024-09-19T08:26:06.032 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 89, 2024-09-19T08:26:06.032 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 716, 2024-09-19T08:26:06.032 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 85, 2024-09-19T08:26:06.032 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2024-09-19T08:26:06.032 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2024-09-19T08:26:06.033 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2024-09-19T08:26:06.033 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 2, 2024-09-19T08:26:06.033 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 1290, 2024-09-19T08:26:06.033 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2024-09-19T08:26:06.033 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2024-09-19T08:26:06.033 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2024-09-19T08:26:06.033 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 3278, 2024-09-19T08:26:06.033 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 20, 2024-09-19T08:26:06.033 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:26:06.033 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2024-09-19T08:26:06.033 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:26:06.033 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2024-09-19T08:26:06.033 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2024-09-19T08:26:06.034 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2024-09-19T08:26:06.034 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:26:06.034 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2024-09-19T08:26:06.034 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:26:06.034 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2024-09-19T08:26:06.034 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2024-09-19T08:26:06.034 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2024-09-19T08:26:06.034 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2024-09-19T08:26:06.034 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2024-09-19T08:26:06.034 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2024-09-19T08:26:06.034 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2024-09-19T08:26:06.034 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2024-09-19T08:26:06.034 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2024-09-19T08:26:06.034 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2024-09-19T08:26:06.035 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2024-09-19T08:26:06.035 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2024-09-19T08:26:06.035 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2024-09-19T08:26:06.035 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2024-09-19T08:26:06.035 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2024-09-19T08:26:06.035 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2024-09-19T08:26:06.035 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2024-09-19T08:26:06.035 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 94, 2024-09-19T08:26:06.035 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2024-09-19T08:26:06.035 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 11, 2024-09-19T08:26:06.035 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 12, 2024-09-19T08:26:06.035 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 12, 2024-09-19T08:26:06.035 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2024-09-19T08:26:06.035 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2024-09-19T08:26:06.036 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 317, 2024-09-19T08:26:06.036 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 0, 2024-09-19T08:26:06.036 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 71 2024-09-19T08:26:06.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.036 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2024-09-19T08:26:06.036 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 1, 2024-09-19T08:26:06.036 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:26:06.036 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2024-09-19T08:26:06.036 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2024-09-19T08:26:06.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.036 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2024-09-19T08:26:06.036 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 2359 2024-09-19T08:26:06.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.037 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2024-09-19T08:26:06.037 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2024-09-19T08:26:06.037 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 10, 2024-09-19T08:26:06.037 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2024-09-19T08:26:06.037 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 10, 2024-09-19T08:26:06.037 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed_ops": 1276, 2024-09-19T08:26:06.037 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 1276, 2024-09-19T08:26:06.037 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2024-09-19T08:26:06.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.037 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:26:06.037 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:06.037 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-19T08:26:06.037 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:06.038 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 57532, 2024-09-19T08:26:06.038 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 15019812, 2024-09-19T08:26:06.038 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:06.038 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 57532, 2024-09-19T08:26:06.038 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-19T08:26:06.038 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-19T08:26:06.038 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 57532, 2024-09-19T08:26:06.038 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 15019812, 2024-09-19T08:26:06.039 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:06.039 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.039 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.039 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.039 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:06.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.039 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2024-09-19T08:26:06.039 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:06.039 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-19T08:26:06.039 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:06.039 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-09-19T08:26:06.039 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-09-19T08:26:06.039 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:06.039 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:06.040 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 5656, 2024-09-19T08:26:06.040 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 13045290, 2024-09-19T08:26:06.040 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 5656, 2024-09-19T08:26:06.040 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 13045290, 2024-09-19T08:26:06.040 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:06.040 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.040 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.040 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.040 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:06.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.040 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2024-09-19T08:26:06.040 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:06.040 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2024-09-19T08:26:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-09-19T08:26:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-09-19T08:26:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 5656, 2024-09-19T08:26:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 5656, 2024-09-19T08:26:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 5656, 2024-09-19T08:26:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 5656, 2024-09-19T08:26:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:06.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.042 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:26:06.042 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:06.042 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-09-19T08:26:06.042 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:06.042 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 1276, 2024-09-19T08:26:06.042 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 152972, 2024-09-19T08:26:06.042 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:06.042 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 1276, 2024-09-19T08:26:06.042 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-19T08:26:06.042 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-19T08:26:06.042 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 89, 2024-09-19T08:26:06.042 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 152972, 2024-09-19T08:26:06.042 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:06.043 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.043 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.043 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.043 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:06.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:06.043 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x55fa4396a960": { 2024-09-19T08:26:06.043 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:06.043 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-09-19T08:26:06.043 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:06.043 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 6484, 2024-09-19T08:26:06.043 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 12172725, 2024-09-19T08:26:06.043 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:06.043 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 6484, 2024-09-19T08:26:06.044 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-19T08:26:06.044 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-19T08:26:06.044 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 627, 2024-09-19T08:26:06.044 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 12172725, 2024-09-19T08:26:06.044 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:06.044 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:06.044 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:06.044 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:06.044 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:06.044 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:06.044 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T08:26:07.026 INFO:tasks.check_counter:Daemon mds.b mds.exported=24 2024-09-19T08:26:07.026 INFO:tasks.check_counter:Daemon mds.b mds.imported=10 2024-09-19T08:26:07.026 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-19T08:26:07.026 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-09-19T08:26:07.026 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T08:26:07.026 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=2 2024-09-19T08:26:07.026 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T08:26:07.026 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T08:26:07.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:07 smithi064 ceph-mon[34900]: pgmap v343: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-19T08:26:07.452 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:26:07.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:07 smithi157 ceph-mon[35143]: pgmap v343: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-19T08:26:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:07 smithi022 ceph-mon[30963]: pgmap v343: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-19T08:26:08.183 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T08:26:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:26:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 1511, 2024-09-19T08:26:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 1511, 2024-09-19T08:26:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 367428915, 2024-09-19T08:26:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 479738, 2024-09-19T08:26:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 8, 2024-09-19T08:26:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 4, 2024-09-19T08:26:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.290407464, 2024-09-19T08:26:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.036699037, 2024-09-19T08:26:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.206915140, 2024-09-19T08:26:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.035658688, 2024-09-19T08:26:08.184 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1510, 2024-09-19T08:26:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.052963732, 2024-09-19T08:26:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000035075 2024-09-19T08:26:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T08:26:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T08:26:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:26:08.185 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 950, 2024-09-19T08:26:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 1116, 2024-09-19T08:26:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 302264019, 2024-09-19T08:26:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 757309, 2024-09-19T08:26:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 9, 2024-09-19T08:26:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 7, 2024-09-19T08:26:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.232575751, 2024-09-19T08:26:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.026792497, 2024-09-19T08:26:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.172857654, 2024-09-19T08:26:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.023618857, 2024-09-19T08:26:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1113, 2024-09-19T08:26:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.037503022, 2024-09-19T08:26:08.186 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000033695 2024-09-19T08:26:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T08:26:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 455840 2024-09-19T08:26:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:26:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 1807, 2024-09-19T08:26:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 1783, 2024-09-19T08:26:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 333517508, 2024-09-19T08:26:08.187 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 629676, 2024-09-19T08:26:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 5, 2024-09-19T08:26:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 4, 2024-09-19T08:26:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.348685469, 2024-09-19T08:26:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.056460418, 2024-09-19T08:26:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.251075741, 2024-09-19T08:26:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.043218482, 2024-09-19T08:26:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1782, 2024-09-19T08:26:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.067485845, 2024-09-19T08:26:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000037870 2024-09-19T08:26:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:08.188 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1666768, 2024-09-19T08:26:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 138048 2024-09-19T08:26:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-19T08:26:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2024-09-19T08:26:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-19T08:26:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2024-09-19T08:26:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-09-19T08:26:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-09-19T08:26:08.189 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 5781, 2024-09-19T08:26:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.433262696, 2024-09-19T08:26:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000074945 2024-09-19T08:26:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2024-09-19T08:26:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-09-19T08:26:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-09-19T08:26:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.190 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-09-19T08:26:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 0, 2024-09-19T08:26:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 0, 2024-09-19T08:26:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2024-09-19T08:26:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2024-09-19T08:26:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 0, 2024-09-19T08:26:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 0, 2024-09-19T08:26:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2024-09-19T08:26:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 0, 2024-09-19T08:26:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 0, 2024-09-19T08:26:08.191 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2024-09-19T08:26:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 152264, 2024-09-19T08:26:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 37923, 2024-09-19T08:26:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 114341, 2024-09-19T08:26:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 0, 2024-09-19T08:26:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 98088, 2024-09-19T08:26:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 0, 2024-09-19T08:26:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 0, 2024-09-19T08:26:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 0, 2024-09-19T08:26:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 1, 2024-09-19T08:26:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 0, 2024-09-19T08:26:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 0, 2024-09-19T08:26:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 0, 2024-09-19T08:26:08.192 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 0, 2024-09-19T08:26:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:26:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2024-09-19T08:26:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 0, 2024-09-19T08:26:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 0, 2024-09-19T08:26:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2024-09-19T08:26:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 0, 2024-09-19T08:26:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 0, 2024-09-19T08:26:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 0, 2024-09-19T08:26:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 0, 2024-09-19T08:26:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2024-09-19T08:26:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 0, 2024-09-19T08:26:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 0, 2024-09-19T08:26:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 0, 2024-09-19T08:26:08.193 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 0, 2024-09-19T08:26:08.194 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 0, 2024-09-19T08:26:08.194 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T08:26:08.194 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2024-09-19T08:26:08.194 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:26:08.194 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:26:08.194 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 0, 2024-09-19T08:26:08.194 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T08:26:08.194 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T08:26:08.194 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 0, 2024-09-19T08:26:08.194 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 0, 2024-09-19T08:26:08.194 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T08:26:08.194 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 0, 2024-09-19T08:26:08.194 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T08:26:08.195 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:26:08.195 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:26:08.195 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 0, 2024-09-19T08:26:08.195 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 0, 2024-09-19T08:26:08.195 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 0, 2024-09-19T08:26:08.195 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 0 2024-09-19T08:26:08.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.195 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2024-09-19T08:26:08.195 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 0, 2024-09-19T08:26:08.195 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 0, 2024-09-19T08:26:08.195 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 0, 2024-09-19T08:26:08.195 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 0, 2024-09-19T08:26:08.195 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 0, 2024-09-19T08:26:08.195 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 0, 2024-09-19T08:26:08.196 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2024-09-19T08:26:08.196 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:26:08.196 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 0, 2024-09-19T08:26:08.196 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 0, 2024-09-19T08:26:08.196 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2024-09-19T08:26:08.196 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2024-09-19T08:26:08.196 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2024-09-19T08:26:08.196 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:26:08.196 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:26:08.196 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2024-09-19T08:26:08.196 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2024-09-19T08:26:08.196 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_path": 0, 2024-09-19T08:26:08.196 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_inode": 0, 2024-09-19T08:26:08.196 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T08:26:08.197 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 0, 2024-09-19T08:26:08.197 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2024-09-19T08:26:08.197 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 0, 2024-09-19T08:26:08.197 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2024-09-19T08:26:08.197 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2024-09-19T08:26:08.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.197 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2024-09-19T08:26:08.197 INFO:teuthology.orchestra.run.smithi064.stdout: "evlrg": 0, 2024-09-19T08:26:08.197 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 0, 2024-09-19T08:26:08.197 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 0, 2024-09-19T08:26:08.197 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 0, 2024-09-19T08:26:08.198 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 274536, 2024-09-19T08:26:08.198 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2024-09-19T08:26:08.198 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 0, 2024-09-19T08:26:08.198 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 0, 2024-09-19T08:26:08.198 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 0, 2024-09-19T08:26:08.198 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 0, 2024-09-19T08:26:08.198 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 140, 2024-09-19T08:26:08.198 INFO:teuthology.orchestra.run.smithi064.stdout: "segmjr": 24, 2024-09-19T08:26:08.199 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2024-09-19T08:26:08.199 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 0, 2024-09-19T08:26:08.199 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 582036008, 2024-09-19T08:26:08.199 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 1004501773, 2024-09-19T08:26:08.199 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 1004491127, 2024-09-19T08:26:08.199 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2024-09-19T08:26:08.199 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.199 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.199 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.199 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 274536 2024-09-19T08:26:08.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.199 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2024-09-19T08:26:08.200 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 49402, 2024-09-19T08:26:08.200 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 95398, 2024-09-19T08:26:08.200 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 45996, 2024-09-19T08:26:08.200 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 1312, 2024-09-19T08:26:08.200 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 1622, 2024-09-19T08:26:08.200 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 310, 2024-09-19T08:26:08.200 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 152264, 2024-09-19T08:26:08.200 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 154098, 2024-09-19T08:26:08.200 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 1834, 2024-09-19T08:26:08.200 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 0, 2024-09-19T08:26:08.200 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 0, 2024-09-19T08:26:08.200 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 0, 2024-09-19T08:26:08.200 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 306508, 2024-09-19T08:26:08.201 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 215328 2024-09-19T08:26:08.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.201 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2024-09-19T08:26:08.201 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 0, 2024-09-19T08:26:08.201 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 0, 2024-09-19T08:26:08.201 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 0, 2024-09-19T08:26:08.201 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 0, 2024-09-19T08:26:08.201 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 0, 2024-09-19T08:26:08.201 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2024-09-19T08:26:08.201 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.201 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.201 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.201 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2024-09-19T08:26:08.202 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.202 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.202 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.202 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2024-09-19T08:26:08.202 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.202 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.202 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.202 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2024-09-19T08:26:08.202 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.202 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.202 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.203 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2024-09-19T08:26:08.203 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.203 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.203 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.203 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2024-09-19T08:26:08.203 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.203 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.203 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.203 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2024-09-19T08:26:08.203 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.203 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.203 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.204 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2024-09-19T08:26:08.204 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.204 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.204 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.204 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2024-09-19T08:26:08.204 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.204 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.204 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.204 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2024-09-19T08:26:08.204 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.205 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.205 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.205 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2024-09-19T08:26:08.205 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.205 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.205 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.206 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2024-09-19T08:26:08.206 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.206 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.206 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.206 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2024-09-19T08:26:08.206 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.206 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.206 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.206 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2024-09-19T08:26:08.207 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.207 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.207 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.207 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2024-09-19T08:26:08.207 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.207 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.207 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.207 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2024-09-19T08:26:08.207 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.207 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.207 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.208 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2024-09-19T08:26:08.208 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.208 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.208 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.208 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2024-09-19T08:26:08.208 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.208 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.208 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.208 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2024-09-19T08:26:08.208 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.208 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.209 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.209 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2024-09-19T08:26:08.209 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.209 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.209 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.209 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2024-09-19T08:26:08.209 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.209 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.209 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.209 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2024-09-19T08:26:08.209 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.210 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.210 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.210 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2024-09-19T08:26:08.210 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.210 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.210 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.210 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2024-09-19T08:26:08.210 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.210 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.210 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.210 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2024-09-19T08:26:08.211 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.211 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.211 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.211 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2024-09-19T08:26:08.211 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.211 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.211 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.211 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2024-09-19T08:26:08.211 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.211 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.211 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.212 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2024-09-19T08:26:08.212 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.212 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.212 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.212 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2024-09-19T08:26:08.212 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.212 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.212 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.212 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:26:08.212 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:26:08.212 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2024-09-19T08:26:08.213 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.213 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.213 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.213 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:08.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.213 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2024-09-19T08:26:08.213 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 2, 2024-09-19T08:26:08.213 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 5, 2024-09-19T08:26:08.213 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 3, 2024-09-19T08:26:08.213 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 4, 2024-09-19T08:26:08.213 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2024-09-19T08:26:08.213 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 0, 2024-09-19T08:26:08.213 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 0, 2024-09-19T08:26:08.213 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 3297, 2024-09-19T08:26:08.214 INFO:teuthology.orchestra.run.smithi064.stdout: "mdthresh_evicted": 0 2024-09-19T08:26:08.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.214 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-19T08:26:08.214 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:26:08.214 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-19T08:26:08.214 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:26:08.214 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:26:08.214 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:26:08.214 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:26:08.214 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:26:08.214 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:26:08.214 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:26:08.214 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:26:08.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:26:08.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:26:08.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:26:08.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:26:08.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:26:08.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-19T08:26:08.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:26:08.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-19T08:26:08.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:26:08.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:26:08.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:26:08.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:26:08.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:26:08.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:26:08.216 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:26:08.216 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-19T08:26:08.216 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:26:08.216 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:26:08.216 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:26:08.216 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-19T08:26:08.216 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-19T08:26:08.216 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-19T08:26:08.216 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:26:08.216 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:26:08.216 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:26:08.216 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:26:08.216 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 2093541, 2024-09-19T08:26:08.217 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 50189, 2024-09-19T08:26:08.217 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:26:08.217 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-19T08:26:08.217 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-19T08:26:08.217 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-19T08:26:08.217 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:26:08.217 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-19T08:26:08.217 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:26:08.217 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-19T08:26:08.217 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 8224, 2024-09-19T08:26:08.217 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 181, 2024-09-19T08:26:08.217 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:26:08.217 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:26:08.217 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-19T08:26:08.218 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-19T08:26:08.218 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 223321371, 2024-09-19T08:26:08.218 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2282572, 2024-09-19T08:26:08.218 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-19T08:26:08.218 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-19T08:26:08.218 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-19T08:26:08.218 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-19T08:26:08.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.218 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2024-09-19T08:26:08.218 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2024-09-19T08:26:08.218 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2024-09-19T08:26:08.218 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 4001, 2024-09-19T08:26:08.218 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 21, 2024-09-19T08:26:08.218 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 3, 2024-09-19T08:26:08.219 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 4001, 2024-09-19T08:26:08.219 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2024-09-19T08:26:08.219 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 4001, 2024-09-19T08:26:08.219 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 8.916821946, 2024-09-19T08:26:08.219 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.002228648 2024-09-19T08:26:08.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.219 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2024-09-19T08:26:08.219 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2024-09-19T08:26:08.219 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 4001, 2024-09-19T08:26:08.219 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 4002 2024-09-19T08:26:08.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.219 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 4001, 2024-09-19T08:26:08.219 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 4001, 2024-09-19T08:26:08.219 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 0, 2024-09-19T08:26:08.220 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2024-09-19T08:26:08.220 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2024-09-19T08:26:08.220 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 2645, 2024-09-19T08:26:08.220 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 0, 2024-09-19T08:26:08.220 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 1354, 2024-09-19T08:26:08.220 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 0, 2024-09-19T08:26:08.220 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 0, 2024-09-19T08:26:08.220 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2024-09-19T08:26:08.220 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2024-09-19T08:26:08.220 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2024-09-19T08:26:08.220 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2024-09-19T08:26:08.220 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 0, 2024-09-19T08:26:08.220 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2024-09-19T08:26:08.220 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2024-09-19T08:26:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2024-09-19T08:26:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 1, 2024-09-19T08:26:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 0, 2024-09-19T08:26:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:26:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2024-09-19T08:26:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:26:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2024-09-19T08:26:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2024-09-19T08:26:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2024-09-19T08:26:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:26:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2024-09-19T08:26:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:26:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2024-09-19T08:26:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2024-09-19T08:26:08.222 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2024-09-19T08:26:08.222 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2024-09-19T08:26:08.222 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2024-09-19T08:26:08.222 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2024-09-19T08:26:08.222 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2024-09-19T08:26:08.222 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2024-09-19T08:26:08.222 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2024-09-19T08:26:08.222 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2024-09-19T08:26:08.222 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2024-09-19T08:26:08.222 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2024-09-19T08:26:08.222 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2024-09-19T08:26:08.222 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2024-09-19T08:26:08.222 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2024-09-19T08:26:08.222 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2024-09-19T08:26:08.223 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2024-09-19T08:26:08.223 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 95, 2024-09-19T08:26:08.223 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2024-09-19T08:26:08.223 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 12, 2024-09-19T08:26:08.223 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 11, 2024-09-19T08:26:08.223 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 11, 2024-09-19T08:26:08.223 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2024-09-19T08:26:08.223 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2024-09-19T08:26:08.223 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 0, 2024-09-19T08:26:08.223 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 2, 2024-09-19T08:26:08.223 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 0 2024-09-19T08:26:08.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.223 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2024-09-19T08:26:08.223 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 0, 2024-09-19T08:26:08.224 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:26:08.224 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2024-09-19T08:26:08.224 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2024-09-19T08:26:08.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.224 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2024-09-19T08:26:08.224 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 0 2024-09-19T08:26:08.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.224 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2024-09-19T08:26:08.224 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2024-09-19T08:26:08.224 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T08:26:08.224 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2024-09-19T08:26:08.224 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 0, 2024-09-19T08:26:08.224 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed_ops": 0, 2024-09-19T08:26:08.225 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 0, 2024-09-19T08:26:08.225 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2024-09-19T08:26:08.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.225 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:26:08.225 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:08.225 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-19T08:26:08.225 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:08.225 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 4264, 2024-09-19T08:26:08.225 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1002810327, 2024-09-19T08:26:08.225 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:08.225 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 4264, 2024-09-19T08:26:08.225 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-19T08:26:08.225 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-19T08:26:08.225 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 4264, 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1002810327, 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:08.226 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:08.227 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 4001, 2024-09-19T08:26:08.227 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 1000451352, 2024-09-19T08:26:08.227 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 4001, 2024-09-19T08:26:08.227 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1000451352, 2024-09-19T08:26:08.227 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:08.227 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.227 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.227 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.227 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:08.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.227 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2024-09-19T08:26:08.227 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:08.227 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2024-09-19T08:26:08.228 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:08.228 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-09-19T08:26:08.228 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-09-19T08:26:08.228 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:08.228 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:08.228 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 4001, 2024-09-19T08:26:08.228 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 4001, 2024-09-19T08:26:08.228 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 4001, 2024-09-19T08:26:08.228 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 4001, 2024-09-19T08:26:08.228 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:08.228 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.228 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.228 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.228 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:08.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.229 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:26:08.229 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:08.229 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-09-19T08:26:08.229 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:08.229 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-09-19T08:26:08.229 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-09-19T08:26:08.229 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:08.229 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:08.229 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-19T08:26:08.229 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-19T08:26:08.229 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2024-09-19T08:26:08.229 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2024-09-19T08:26:08.229 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:08.230 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.230 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.230 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.230 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:08.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:08.230 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x5650a2072b40": { 2024-09-19T08:26:08.230 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:08.230 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-09-19T08:26:08.230 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:08.230 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-09-19T08:26:08.230 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-09-19T08:26:08.230 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:08.230 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:08.231 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-19T08:26:08.231 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-19T08:26:08.231 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2024-09-19T08:26:08.231 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2024-09-19T08:26:08.231 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:08.231 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:08.231 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:08.231 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:08.231 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:08.231 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:08.231 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T08:26:08.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:08 smithi064 ceph-mon[34900]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T08:26:08.504 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:08 smithi157 ceph-mon[35143]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T08:26:08.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:08 smithi022 ceph-mon[30963]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T08:26:09.036 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T08:26:09.036 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T08:26:09.036 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-19T08:26:09.036 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-19T08:26:09.036 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T08:26:09.036 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-19T08:26:09.036 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T08:26:09.036 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T08:26:09.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:09 smithi064 ceph-mon[34900]: pgmap v345: 97 pgs: 4 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 87 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:26:09.490 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:26:09.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:09 smithi157 ceph-mon[35143]: pgmap v345: 97 pgs: 4 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 87 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:26:09.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:09 smithi022 ceph-mon[30963]: pgmap v345: 97 pgs: 4 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 87 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:26:10.258 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T08:26:10.258 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:26:10.258 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 1392, 2024-09-19T08:26:10.258 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 1393, 2024-09-19T08:26:10.258 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 5128278, 2024-09-19T08:26:10.258 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 442348, 2024-09-19T08:26:10.258 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 6, 2024-09-19T08:26:10.258 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-09-19T08:26:10.259 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.123305612, 2024-09-19T08:26:10.259 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.046810022, 2024-09-19T08:26:10.259 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.030951390, 2024-09-19T08:26:10.259 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.033994494, 2024-09-19T08:26:10.259 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:10.259 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1391, 2024-09-19T08:26:10.259 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.050020415, 2024-09-19T08:26:10.259 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000035960 2024-09-19T08:26:10.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.259 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:10.259 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.259 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.260 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.260 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T08:26:10.260 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T08:26:10.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.260 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:26:10.260 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 519, 2024-09-19T08:26:10.260 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 684, 2024-09-19T08:26:10.260 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 8521282, 2024-09-19T08:26:10.260 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 620136, 2024-09-19T08:26:10.260 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 7, 2024-09-19T08:26:10.261 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 5, 2024-09-19T08:26:10.261 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.050454551, 2024-09-19T08:26:10.261 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.015287391, 2024-09-19T08:26:10.261 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.017957490, 2024-09-19T08:26:10.261 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.012763339, 2024-09-19T08:26:10.261 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:10.261 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 681, 2024-09-19T08:26:10.261 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.031773855, 2024-09-19T08:26:10.261 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000046657 2024-09-19T08:26:10.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.261 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:10.262 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.262 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.262 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.262 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T08:26:10.262 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 455664 2024-09-19T08:26:10.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.262 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:26:10.262 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 879, 2024-09-19T08:26:10.262 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 853, 2024-09-19T08:26:10.262 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 6017777, 2024-09-19T08:26:10.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 332947, 2024-09-19T08:26:10.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-09-19T08:26:10.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2024-09-19T08:26:10.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.093824848, 2024-09-19T08:26:10.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.027808368, 2024-09-19T08:26:10.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.059222470, 2024-09-19T08:26:10.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.017196871, 2024-09-19T08:26:10.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:10.263 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 853, 2024-09-19T08:26:10.263 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.034169565, 2024-09-19T08:26:10.263 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000040058 2024-09-19T08:26:10.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:10.264 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.264 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.264 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.264 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1669408, 2024-09-19T08:26:10.264 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 137376 2024-09-19T08:26:10.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.264 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-19T08:26:10.264 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2024-09-19T08:26:10.264 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-19T08:26:10.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.264 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2024-09-19T08:26:10.264 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-09-19T08:26:10.265 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-09-19T08:26:10.265 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 4225, 2024-09-19T08:26:10.265 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.096603536, 2024-09-19T08:26:10.265 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000022864 2024-09-19T08:26:10.265 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:10.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.265 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2024-09-19T08:26:10.265 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-09-19T08:26:10.265 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-09-19T08:26:10.265 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.265 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.265 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.265 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:10.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.266 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-09-19T08:26:10.266 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 0, 2024-09-19T08:26:10.266 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 0, 2024-09-19T08:26:10.266 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2024-09-19T08:26:10.266 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.266 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.266 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.266 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2024-09-19T08:26:10.266 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 0, 2024-09-19T08:26:10.267 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 0, 2024-09-19T08:26:10.267 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2024-09-19T08:26:10.267 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 0, 2024-09-19T08:26:10.267 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 0, 2024-09-19T08:26:10.267 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2024-09-19T08:26:10.267 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 1772, 2024-09-19T08:26:10.267 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 0, 2024-09-19T08:26:10.267 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 1772, 2024-09-19T08:26:10.267 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 0, 2024-09-19T08:26:10.267 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 1772, 2024-09-19T08:26:10.267 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 0, 2024-09-19T08:26:10.267 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 0, 2024-09-19T08:26:10.267 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 0, 2024-09-19T08:26:10.267 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 2, 2024-09-19T08:26:10.268 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 0, 2024-09-19T08:26:10.268 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 0, 2024-09-19T08:26:10.268 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 0, 2024-09-19T08:26:10.268 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 0, 2024-09-19T08:26:10.268 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:26:10.268 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2024-09-19T08:26:10.268 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 0, 2024-09-19T08:26:10.268 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 0, 2024-09-19T08:26:10.268 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2024-09-19T08:26:10.268 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 0, 2024-09-19T08:26:10.268 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 0, 2024-09-19T08:26:10.268 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 0, 2024-09-19T08:26:10.268 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 0, 2024-09-19T08:26:10.269 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2024-09-19T08:26:10.269 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 0, 2024-09-19T08:26:10.269 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 0, 2024-09-19T08:26:10.269 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 4, 2024-09-19T08:26:10.269 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 234, 2024-09-19T08:26:10.269 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 6, 2024-09-19T08:26:10.269 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T08:26:10.269 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2024-09-19T08:26:10.269 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:26:10.269 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:26:10.269 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 0, 2024-09-19T08:26:10.269 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T08:26:10.269 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T08:26:10.270 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 0, 2024-09-19T08:26:10.270 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 0, 2024-09-19T08:26:10.270 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T08:26:10.270 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 0, 2024-09-19T08:26:10.270 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T08:26:10.270 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:26:10.270 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:26:10.270 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 0, 2024-09-19T08:26:10.270 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 0, 2024-09-19T08:26:10.270 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 0, 2024-09-19T08:26:10.270 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 0 2024-09-19T08:26:10.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.270 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2024-09-19T08:26:10.270 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 0, 2024-09-19T08:26:10.271 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 0, 2024-09-19T08:26:10.271 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 0, 2024-09-19T08:26:10.271 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 0, 2024-09-19T08:26:10.271 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 0, 2024-09-19T08:26:10.271 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 0, 2024-09-19T08:26:10.271 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2024-09-19T08:26:10.271 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:26:10.271 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 0, 2024-09-19T08:26:10.271 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 0, 2024-09-19T08:26:10.271 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2024-09-19T08:26:10.271 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2024-09-19T08:26:10.271 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2024-09-19T08:26:10.271 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:26:10.272 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:26:10.272 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2024-09-19T08:26:10.272 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2024-09-19T08:26:10.272 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_path": 0, 2024-09-19T08:26:10.272 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_inode": 0, 2024-09-19T08:26:10.272 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T08:26:10.272 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 0, 2024-09-19T08:26:10.272 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2024-09-19T08:26:10.272 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 0, 2024-09-19T08:26:10.272 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2024-09-19T08:26:10.272 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2024-09-19T08:26:10.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.273 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2024-09-19T08:26:10.273 INFO:teuthology.orchestra.run.smithi064.stdout: "evlrg": 0, 2024-09-19T08:26:10.273 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 0, 2024-09-19T08:26:10.273 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 0, 2024-09-19T08:26:10.273 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 0, 2024-09-19T08:26:10.273 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 8205, 2024-09-19T08:26:10.273 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2024-09-19T08:26:10.273 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 0, 2024-09-19T08:26:10.273 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 0, 2024-09-19T08:26:10.273 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 0, 2024-09-19T08:26:10.274 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 0, 2024-09-19T08:26:10.274 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 12, 2024-09-19T08:26:10.274 INFO:teuthology.orchestra.run.smithi064.stdout: "segmjr": 2, 2024-09-19T08:26:10.274 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2024-09-19T08:26:10.274 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 0, 2024-09-19T08:26:10.274 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 4194304, 2024-09-19T08:26:10.274 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 21485727, 2024-09-19T08:26:10.274 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 21485667, 2024-09-19T08:26:10.274 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2024-09-19T08:26:10.274 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.274 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.274 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.275 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 8205 2024-09-19T08:26:10.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.275 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2024-09-19T08:26:10.275 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 94, 2024-09-19T08:26:10.275 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 2803, 2024-09-19T08:26:10.275 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 2709, 2024-09-19T08:26:10.275 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 24, 2024-09-19T08:26:10.275 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 80, 2024-09-19T08:26:10.275 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 56, 2024-09-19T08:26:10.275 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 1772, 2024-09-19T08:26:10.275 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 4386, 2024-09-19T08:26:10.275 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 2614, 2024-09-19T08:26:10.275 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 0, 2024-09-19T08:26:10.275 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 0, 2024-09-19T08:26:10.276 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 0, 2024-09-19T08:26:10.276 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 57760, 2024-09-19T08:26:10.276 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 215328 2024-09-19T08:26:10.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.276 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2024-09-19T08:26:10.276 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 0, 2024-09-19T08:26:10.276 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 0, 2024-09-19T08:26:10.276 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 0, 2024-09-19T08:26:10.276 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 0, 2024-09-19T08:26:10.276 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 0, 2024-09-19T08:26:10.276 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2024-09-19T08:26:10.276 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.276 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.277 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.277 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2024-09-19T08:26:10.277 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.277 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.277 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.277 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2024-09-19T08:26:10.277 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.277 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.277 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.277 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2024-09-19T08:26:10.277 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.278 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.278 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.278 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2024-09-19T08:26:10.278 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.278 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.278 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.278 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2024-09-19T08:26:10.278 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.278 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.278 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.279 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2024-09-19T08:26:10.279 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.279 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.279 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.279 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2024-09-19T08:26:10.279 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.279 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.279 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.279 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2024-09-19T08:26:10.279 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.279 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.279 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.280 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2024-09-19T08:26:10.280 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.280 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.280 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.280 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2024-09-19T08:26:10.280 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.280 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.280 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.280 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2024-09-19T08:26:10.281 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.281 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.281 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.281 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2024-09-19T08:26:10.281 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.282 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.282 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.282 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2024-09-19T08:26:10.282 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.282 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.282 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.282 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2024-09-19T08:26:10.282 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.282 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.283 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.283 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2024-09-19T08:26:10.283 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.283 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.283 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.283 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2024-09-19T08:26:10.283 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.283 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.283 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.283 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2024-09-19T08:26:10.283 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.284 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.284 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.284 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2024-09-19T08:26:10.284 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.284 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.284 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.284 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2024-09-19T08:26:10.284 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.284 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.284 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.285 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2024-09-19T08:26:10.285 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.285 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.285 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.285 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2024-09-19T08:26:10.285 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.285 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.285 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.285 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2024-09-19T08:26:10.285 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.285 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.286 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.286 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2024-09-19T08:26:10.286 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.286 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.286 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.286 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2024-09-19T08:26:10.286 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.286 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.286 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.286 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2024-09-19T08:26:10.287 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.287 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.287 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.287 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2024-09-19T08:26:10.287 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.287 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.287 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.287 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2024-09-19T08:26:10.287 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.287 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.287 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2024-09-19T08:26:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:26:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:26:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2024-09-19T08:26:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:10.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.289 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2024-09-19T08:26:10.289 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 2, 2024-09-19T08:26:10.289 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 6, 2024-09-19T08:26:10.289 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 4, 2024-09-19T08:26:10.289 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 5, 2024-09-19T08:26:10.289 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2024-09-19T08:26:10.289 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 0, 2024-09-19T08:26:10.289 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 0, 2024-09-19T08:26:10.289 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 3959, 2024-09-19T08:26:10.289 INFO:teuthology.orchestra.run.smithi064.stdout: "mdthresh_evicted": 0 2024-09-19T08:26:10.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.289 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-19T08:26:10.289 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:26:10.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-19T08:26:10.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:26:10.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:26:10.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:26:10.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:26:10.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:26:10.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:26:10.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:26:10.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:26:10.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:26:10.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:26:10.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:26:10.291 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:26:10.291 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:26:10.291 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-19T08:26:10.291 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:26:10.291 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-19T08:26:10.291 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:26:10.291 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:26:10.291 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:26:10.291 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:26:10.292 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:26:10.292 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:26:10.292 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:26:10.292 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-19T08:26:10.292 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:26:10.292 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:26:10.292 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:26:10.292 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-19T08:26:10.293 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-19T08:26:10.293 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-19T08:26:10.293 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:26:10.293 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:26:10.293 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:26:10.293 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:26:10.293 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 201166, 2024-09-19T08:26:10.293 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 154, 2024-09-19T08:26:10.293 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:26:10.293 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-19T08:26:10.293 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-19T08:26:10.293 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-19T08:26:10.293 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:26:10.294 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-19T08:26:10.294 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:26:10.294 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-19T08:26:10.294 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 8344, 2024-09-19T08:26:10.294 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 184, 2024-09-19T08:26:10.294 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:26:10.294 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:26:10.294 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-19T08:26:10.294 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-19T08:26:10.294 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 1568975, 2024-09-19T08:26:10.294 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 10203, 2024-09-19T08:26:10.294 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-19T08:26:10.294 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-19T08:26:10.294 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-19T08:26:10.295 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-19T08:26:10.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.295 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2024-09-19T08:26:10.295 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2024-09-19T08:26:10.295 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2024-09-19T08:26:10.295 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 2523, 2024-09-19T08:26:10.295 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 26, 2024-09-19T08:26:10.295 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 4, 2024-09-19T08:26:10.295 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 2523, 2024-09-19T08:26:10.295 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2024-09-19T08:26:10.295 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2523, 2024-09-19T08:26:10.295 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 3.447931084, 2024-09-19T08:26:10.295 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001366599 2024-09-19T08:26:10.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.296 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2024-09-19T08:26:10.296 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2024-09-19T08:26:10.296 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2523, 2024-09-19T08:26:10.296 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 2524 2024-09-19T08:26:10.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.296 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 2523, 2024-09-19T08:26:10.296 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 2523, 2024-09-19T08:26:10.296 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 0, 2024-09-19T08:26:10.296 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2024-09-19T08:26:10.296 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2024-09-19T08:26:10.296 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 1541, 2024-09-19T08:26:10.296 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 0, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 979, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 0, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 0, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 0, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 2, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 0, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:26:10.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2024-09-19T08:26:10.298 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:26:10.298 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2024-09-19T08:26:10.298 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2024-09-19T08:26:10.298 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2024-09-19T08:26:10.298 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:26:10.298 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2024-09-19T08:26:10.298 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:26:10.298 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2024-09-19T08:26:10.298 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2024-09-19T08:26:10.298 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2024-09-19T08:26:10.298 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2024-09-19T08:26:10.298 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2024-09-19T08:26:10.298 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2024-09-19T08:26:10.298 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2024-09-19T08:26:10.299 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2024-09-19T08:26:10.299 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2024-09-19T08:26:10.299 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2024-09-19T08:26:10.299 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2024-09-19T08:26:10.299 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2024-09-19T08:26:10.299 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2024-09-19T08:26:10.299 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2024-09-19T08:26:10.299 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2024-09-19T08:26:10.299 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2024-09-19T08:26:10.299 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2024-09-19T08:26:10.299 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 95, 2024-09-19T08:26:10.299 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2024-09-19T08:26:10.299 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 12, 2024-09-19T08:26:10.300 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 6, 2024-09-19T08:26:10.300 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 6, 2024-09-19T08:26:10.300 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2024-09-19T08:26:10.300 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2024-09-19T08:26:10.300 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 0, 2024-09-19T08:26:10.300 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 2, 2024-09-19T08:26:10.300 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 0 2024-09-19T08:26:10.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.300 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2024-09-19T08:26:10.300 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 0, 2024-09-19T08:26:10.300 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:26:10.300 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2024-09-19T08:26:10.301 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2024-09-19T08:26:10.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.301 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2024-09-19T08:26:10.301 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 0 2024-09-19T08:26:10.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.301 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2024-09-19T08:26:10.301 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2024-09-19T08:26:10.301 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T08:26:10.301 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2024-09-19T08:26:10.301 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 0, 2024-09-19T08:26:10.301 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed_ops": 0, 2024-09-19T08:26:10.301 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 0, 2024-09-19T08:26:10.301 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2024-09-19T08:26:10.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.302 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:26:10.302 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:10.302 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-19T08:26:10.302 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:10.302 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 2786, 2024-09-19T08:26:10.302 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 19400300, 2024-09-19T08:26:10.302 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:10.302 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 2786, 2024-09-19T08:26:10.302 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-19T08:26:10.302 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-19T08:26:10.302 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 2786, 2024-09-19T08:26:10.302 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 19400300, 2024-09-19T08:26:10.303 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:10.303 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.303 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.303 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.303 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:10.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.303 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2024-09-19T08:26:10.303 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:10.303 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-19T08:26:10.303 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:10.303 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-09-19T08:26:10.303 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-09-19T08:26:10.303 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:10.304 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:10.304 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 2523, 2024-09-19T08:26:10.304 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 17292317, 2024-09-19T08:26:10.304 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 2523, 2024-09-19T08:26:10.304 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 17292317, 2024-09-19T08:26:10.304 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:10.304 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.304 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.304 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:10.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.305 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2024-09-19T08:26:10.305 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:10.305 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2024-09-19T08:26:10.305 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:10.305 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-09-19T08:26:10.305 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-09-19T08:26:10.305 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:10.305 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:10.305 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 2523, 2024-09-19T08:26:10.305 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 2523, 2024-09-19T08:26:10.306 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 2523, 2024-09-19T08:26:10.306 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 2523, 2024-09-19T08:26:10.306 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:10.306 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.306 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.306 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.306 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:10.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.306 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:26:10.306 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:10.306 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-09-19T08:26:10.306 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:10.306 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-09-19T08:26:10.306 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-09-19T08:26:10.307 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:10.307 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:10.307 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-19T08:26:10.307 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-19T08:26:10.307 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2024-09-19T08:26:10.307 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2024-09-19T08:26:10.307 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:10.307 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.307 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.307 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.307 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:10.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:10.307 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x5567f1b5cb40": { 2024-09-19T08:26:10.308 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:10.308 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-09-19T08:26:10.308 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:10.308 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-09-19T08:26:10.308 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-09-19T08:26:10.308 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:10.308 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:10.308 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-19T08:26:10.308 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-19T08:26:10.308 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2024-09-19T08:26:10.308 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2024-09-19T08:26:10.308 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:10.308 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:10.309 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:10.309 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:10.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:10.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:10.309 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T08:26:11.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:11 smithi157 ceph-mon[35143]: pgmap v346: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:26:11.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:11 smithi022 ceph-mon[30963]: pgmap v346: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:26:11.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:11 smithi064 ceph-mon[34900]: pgmap v346: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:26:11.661 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-19T08:26:11.661 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-19T08:26:11.661 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-19T08:26:11.662 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-19T08:26:11.662 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T08:26:11.662 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-09-19T08:26:11.662 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T08:26:11.662 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T08:26:12.109 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:26:12.718 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T08:26:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:26:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 234, 2024-09-19T08:26:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 223, 2024-09-19T08:26:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1590032, 2024-09-19T08:26:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 137024, 2024-09-19T08:26:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-09-19T08:26:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-09-19T08:26:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.040568436, 2024-09-19T08:26:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.013800644, 2024-09-19T08:26:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.040403209, 2024-09-19T08:26:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 222, 2024-09-19T08:26:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.012017495, 2024-09-19T08:26:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000054132 2024-09-19T08:26:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1590032, 2024-09-19T08:26:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 137024 2024-09-19T08:26:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:26:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 6, 2024-09-19T08:26:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 173, 2024-09-19T08:26:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 3344, 2024-09-19T08:26:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 187776, 2024-09-19T08:26:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 5, 2024-09-19T08:26:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2024-09-19T08:26:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.014668510, 2024-09-19T08:26:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.007503122, 2024-09-19T08:26:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.003557047, 2024-09-19T08:26:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 171, 2024-09-19T08:26:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.022194628, 2024-09-19T08:26:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000129793 2024-09-19T08:26:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-19T08:26:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 187776 2024-09-19T08:26:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:26:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2024-09-19T08:26:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2024-09-19T08:26:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2024-09-19T08:26:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2024-09-19T08:26:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-09-19T08:26:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-09-19T08:26:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.000348571, 2024-09-19T08:26:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T08:26:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000281332, 2024-09-19T08:26:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T08:26:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T08:26:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-19T08:26:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-09-19T08:26:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-19T08:26:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-19T08:26:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:26:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-19T08:26:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:26:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:26:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:26:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:26:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:26:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:26:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:26:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:26:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:26:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:26:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:26:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:26:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:26:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-19T08:26:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:26:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-19T08:26:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:26:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:26:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:26:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:26:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:26:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:26:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:26:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-19T08:26:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:26:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:26:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:26:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-19T08:26:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-19T08:26:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-19T08:26:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:26:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:26:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:26:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:26:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 149998, 2024-09-19T08:26:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 30, 2024-09-19T08:26:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:26:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-19T08:26:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-19T08:26:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-19T08:26:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:26:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-19T08:26:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:26:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-19T08:26:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-09-19T08:26:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-09-19T08:26:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:26:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:26:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-19T08:26:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-19T08:26:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-09-19T08:26:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-09-19T08:26:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-19T08:26:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-19T08:26:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-19T08:26:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-19T08:26:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T08:26:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:26:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T08:26:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-19T08:26:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T08:26:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 236, 2024-09-19T08:26:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1557022, 2024-09-19T08:26:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 236, 2024-09-19T08:26:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-19T08:26:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-19T08:26:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 236, 2024-09-19T08:26:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1557022, 2024-09-19T08:26:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T08:26:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T08:26:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T08:26:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T08:26:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T08:26:12.731 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T08:26:13.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:13 smithi157 ceph-mon[35143]: pgmap v347: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:26:13.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:13 smithi022 ceph-mon[30963]: pgmap v347: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:26:13.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:13 smithi064 ceph-mon[34900]: pgmap v347: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T08:26:13.700 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-19T08:26:13.700 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-19T08:26:13.700 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-19T08:26:13.700 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-19T08:26:13.701 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-19T08:26:13.701 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-09-19T08:26:13.701 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T08:26:13.701 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T08:26:14.096 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:26:15.467 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:15 smithi157 ceph-mon[35143]: pgmap v348: 97 pgs: 26 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 59 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-19T08:26:15.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:15 smithi022 ceph-mon[30963]: pgmap v348: 97 pgs: 26 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 59 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-19T08:26:15.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:15 smithi064 ceph-mon[34900]: pgmap v348: 97 pgs: 26 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 59 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-19T08:26:15.688 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-09-19T08:26:15.688 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:26:15.688 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 489885, 2024-09-19T08:26:15.688 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 723681, 2024-09-19T08:26:15.689 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 186251817, 2024-09-19T08:26:15.689 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 700002406, 2024-09-19T08:26:15.689 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 14, 2024-09-19T08:26:15.689 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2024-09-19T08:26:15.689 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 26.394479412, 2024-09-19T08:26:15.689 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 12.733145301, 2024-09-19T08:26:15.689 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 78.842019204, 2024-09-19T08:26:15.689 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.994841491, 2024-09-19T08:26:15.689 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:15.689 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 723680, 2024-09-19T08:26:15.689 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 38.735451669, 2024-09-19T08:26:15.689 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000053525 2024-09-19T08:26:15.690 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.690 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:15.690 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 961275, 2024-09-19T08:26:15.690 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.130886602, 2024-09-19T08:26:15.690 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000136 2024-09-19T08:26:15.690 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.690 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 167375552, 2024-09-19T08:26:15.690 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 399577168 2024-09-19T08:26:15.690 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.690 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:26:15.690 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 94887, 2024-09-19T08:26:15.690 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 94934, 2024-09-19T08:26:15.690 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 34236390, 2024-09-19T08:26:15.691 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 461901923, 2024-09-19T08:26:15.691 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 37, 2024-09-19T08:26:15.691 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 10, 2024-09-19T08:26:15.691 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 7.821148779, 2024-09-19T08:26:15.691 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 3.822147786, 2024-09-19T08:26:15.691 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 1.337554112, 2024-09-19T08:26:15.691 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 2.213092608, 2024-09-19T08:26:15.691 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:15.691 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 94931, 2024-09-19T08:26:15.691 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 24.440647797, 2024-09-19T08:26:15.691 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000257456 2024-09-19T08:26:15.691 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:15.692 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 67, 2024-09-19T08:26:15.692 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000010226, 2024-09-19T08:26:15.692 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000152 2024-09-19T08:26:15.692 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 2816, 2024-09-19T08:26:15.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 134048 2024-09-19T08:26:15.692 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.692 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:26:15.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 89125, 2024-09-19T08:26:15.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 85280, 2024-09-19T08:26:15.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 38964747, 2024-09-19T08:26:15.693 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 354304113, 2024-09-19T08:26:15.693 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 12, 2024-09-19T08:26:15.693 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2024-09-19T08:26:15.693 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 6.405562316, 2024-09-19T08:26:15.693 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 2.798237371, 2024-09-19T08:26:15.693 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 1.503066647, 2024-09-19T08:26:15.693 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 1.841273883, 2024-09-19T08:26:15.693 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:15.693 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 85279, 2024-09-19T08:26:15.693 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 25.650599258, 2024-09-19T08:26:15.693 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000300784 2024-09-19T08:26:15.693 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.693 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:15.694 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 16905, 2024-09-19T08:26:15.694 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.002494987, 2024-09-19T08:26:15.694 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000147 2024-09-19T08:26:15.694 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.694 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1682896, 2024-09-19T08:26:15.694 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 156176 2024-09-19T08:26:15.694 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.694 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-09-19T08:26:15.694 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-09-19T08:26:15.694 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-09-19T08:26:15.694 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.694 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-09-19T08:26:15.694 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-09-19T08:26:15.695 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-09-19T08:26:15.695 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 67688, 2024-09-19T08:26:15.695 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 293.343380882, 2024-09-19T08:26:15.695 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.004333757 2024-09-19T08:26:15.695 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:15.695 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.695 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-09-19T08:26:15.695 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-09-19T08:26:15.695 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-09-19T08:26:15.695 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 71087, 2024-09-19T08:26:15.695 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 40.971031939, 2024-09-19T08:26:15.695 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000576350 2024-09-19T08:26:15.695 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:15.696 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.696 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-09-19T08:26:15.696 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 260359, 2024-09-19T08:26:15.696 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 257640, 2024-09-19T08:26:15.696 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-09-19T08:26:15.696 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 257640, 2024-09-19T08:26:15.696 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 8835.515474405, 2024-09-19T08:26:15.696 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.034294036 2024-09-19T08:26:15.696 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.696 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-09-19T08:26:15.696 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 2719, 2024-09-19T08:26:15.696 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 29, 2024-09-19T08:26:15.697 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-09-19T08:26:15.697 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 3150, 2024-09-19T08:26:15.697 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 170, 2024-09-19T08:26:15.697 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 17, 2024-09-19T08:26:15.697 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 83509, 2024-09-19T08:26:15.697 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 23908, 2024-09-19T08:26:15.697 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 59413, 2024-09-19T08:26:15.697 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 188, 2024-09-19T08:26:15.697 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 49310, 2024-09-19T08:26:15.697 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 2921, 2024-09-19T08:26:15.697 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 1, 2024-09-19T08:26:15.697 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 1, 2024-09-19T08:26:15.697 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 52, 2024-09-19T08:26:15.698 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 427638, 2024-09-19T08:26:15.698 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 358164, 2024-09-19T08:26:15.698 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 2707, 2024-09-19T08:26:15.698 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 43, 2024-09-19T08:26:15.698 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 2, 2024-09-19T08:26:15.698 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-09-19T08:26:15.698 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 984, 2024-09-19T08:26:15.698 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-09-19T08:26:15.698 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-09-19T08:26:15.698 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 15, 2024-09-19T08:26:15.698 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 1768, 2024-09-19T08:26:15.698 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 57, 2024-09-19T08:26:15.699 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 4469, 2024-09-19T08:26:15.699 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-09-19T08:26:15.699 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 117597, 2024-09-19T08:26:15.699 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-09-19T08:26:15.699 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 4, 2024-09-19T08:26:15.699 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 234, 2024-09-19T08:26:15.699 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 6, 2024-09-19T08:26:15.699 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 117597, 2024-09-19T08:26:15.699 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-09-19T08:26:15.699 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:26:15.699 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:26:15.699 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 775, 2024-09-19T08:26:15.699 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T08:26:15.700 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 775, 2024-09-19T08:26:15.700 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 116822, 2024-09-19T08:26:15.700 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 18, 2024-09-19T08:26:15.700 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 114792, 2024-09-19T08:26:15.700 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 127062, 2024-09-19T08:26:15.700 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 1, 2024-09-19T08:26:15.700 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:26:15.700 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:26:15.700 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 169742, 2024-09-19T08:26:15.700 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 53543, 2024-09-19T08:26:15.700 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 9941, 2024-09-19T08:26:15.700 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 96456 2024-09-19T08:26:15.700 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.701 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-09-19T08:26:15.701 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-09-19T08:26:15.701 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-09-19T08:26:15.701 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-09-19T08:26:15.701 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 44, 2024-09-19T08:26:15.701 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 26, 2024-09-19T08:26:15.701 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 174, 2024-09-19T08:26:15.701 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-09-19T08:26:15.701 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:26:15.701 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 45361, 2024-09-19T08:26:15.701 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 45187, 2024-09-19T08:26:15.701 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-09-19T08:26:15.701 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-09-19T08:26:15.702 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-09-19T08:26:15.702 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:26:15.702 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:26:15.702 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-09-19T08:26:15.702 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-09-19T08:26:15.702 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 8, 2024-09-19T08:26:15.702 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 96344, 2024-09-19T08:26:15.702 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T08:26:15.702 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 15, 2024-09-19T08:26:15.702 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-09-19T08:26:15.702 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 195, 2024-09-19T08:26:15.702 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-09-19T08:26:15.702 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-09-19T08:26:15.703 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.703 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-09-19T08:26:15.703 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-09-19T08:26:15.703 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 274536, 2024-09-19T08:26:15.703 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 192875, 2024-09-19T08:26:15.703 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 184351, 2024-09-19T08:26:15.703 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 90185, 2024-09-19T08:26:15.703 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-09-19T08:26:15.703 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 8524, 2024-09-19T08:26:15.703 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 400, 2024-09-19T08:26:15.703 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 272, 2024-09-19T08:26:15.703 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 260, 2024-09-19T08:26:15.703 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 140, 2024-09-19T08:26:15.703 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 8, 2024-09-19T08:26:15.704 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-09-19T08:26:15.704 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 12, 2024-09-19T08:26:15.704 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 582036008, 2024-09-19T08:26:15.704 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 1004501773, 2024-09-19T08:26:15.704 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2024-09-19T08:26:15.704 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-09-19T08:26:15.704 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 20524, 2024-09-19T08:26:15.704 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 861.211251525, 2024-09-19T08:26:15.704 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.041961179 2024-09-19T08:26:15.704 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.704 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2024-09-19T08:26:15.704 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.704 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-09-19T08:26:15.704 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 49421, 2024-09-19T08:26:15.705 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 95675, 2024-09-19T08:26:15.705 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 46254, 2024-09-19T08:26:15.705 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 1332, 2024-09-19T08:26:15.705 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 1640, 2024-09-19T08:26:15.705 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 308, 2024-09-19T08:26:15.705 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 83509, 2024-09-19T08:26:15.705 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 158147, 2024-09-19T08:26:15.705 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 74638, 2024-09-19T08:26:15.705 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 1, 2024-09-19T08:26:15.705 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 81041, 2024-09-19T08:26:15.705 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 81040, 2024-09-19T08:26:15.705 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 355124, 2024-09-19T08:26:15.705 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 223520 2024-09-19T08:26:15.706 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.706 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-09-19T08:26:15.706 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 266151, 2024-09-19T08:26:15.706 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 4911, 2024-09-19T08:26:15.706 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 260359, 2024-09-19T08:26:15.706 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 43, 2024-09-19T08:26:15.706 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 9815, 2024-09-19T08:26:15.706 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-09-19T08:26:15.706 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 45936, 2024-09-19T08:26:15.706 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 999.047865543, 2024-09-19T08:26:15.706 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.021748690 2024-09-19T08:26:15.706 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.706 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-09-19T08:26:15.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3226, 2024-09-19T08:26:15.707 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1037.845296884, 2024-09-19T08:26:15.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.321712739 2024-09-19T08:26:15.707 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.707 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-09-19T08:26:15.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.707 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.707 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.707 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-09-19T08:26:15.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.707 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.708 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.708 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-09-19T08:26:15.708 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 115904, 2024-09-19T08:26:15.708 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 5062.313213489, 2024-09-19T08:26:15.708 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.043676777 2024-09-19T08:26:15.708 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.708 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-09-19T08:26:15.708 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.708 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.708 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.708 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.708 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-09-19T08:26:15.708 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.709 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.709 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.709 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-09-19T08:26:15.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.709 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.709 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.709 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-09-19T08:26:15.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.709 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.709 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.710 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-09-19T08:26:15.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.710 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.710 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.710 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-09-19T08:26:15.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.710 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.710 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.710 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-09-19T08:26:15.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 175, 2024-09-19T08:26:15.710 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 8.090483196, 2024-09-19T08:26:15.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.046231332 2024-09-19T08:26:15.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.711 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-09-19T08:26:15.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.711 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.711 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-09-19T08:26:15.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.711 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.711 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-09-19T08:26:15.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 17227, 2024-09-19T08:26:15.712 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 504.756661736, 2024-09-19T08:26:15.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.029300322 2024-09-19T08:26:15.712 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.712 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-09-19T08:26:15.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1257, 2024-09-19T08:26:15.712 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.540371577, 2024-09-19T08:26:15.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001225434 2024-09-19T08:26:15.712 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.712 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-09-19T08:26:15.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 45914, 2024-09-19T08:26:15.712 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1104.784850756, 2024-09-19T08:26:15.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.024062047 2024-09-19T08:26:15.713 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.713 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-09-19T08:26:15.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.713 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.713 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.713 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-09-19T08:26:15.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.713 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.713 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.713 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-09-19T08:26:15.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 175, 2024-09-19T08:26:15.714 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 45.925185525, 2024-09-19T08:26:15.714 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.262429631 2024-09-19T08:26:15.714 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.714 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-09-19T08:26:15.714 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.714 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.714 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.714 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.714 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-09-19T08:26:15.714 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.714 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.714 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.714 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.715 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-09-19T08:26:15.715 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2024-09-19T08:26:15.715 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.190038480, 2024-09-19T08:26:15.715 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.190038480 2024-09-19T08:26:15.715 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.715 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-09-19T08:26:15.715 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.715 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.715 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.715 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.715 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-09-19T08:26:15.715 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.715 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.716 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.716 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.716 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-09-19T08:26:15.716 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.716 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.716 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.716 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.716 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-09-19T08:26:15.716 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.716 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.716 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.716 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.717 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-09-19T08:26:15.717 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.717 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.717 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.717 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.717 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-09-19T08:26:15.717 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 27825, 2024-09-19T08:26:15.717 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 71.021507219, 2024-09-19T08:26:15.717 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.002552435 2024-09-19T08:26:15.717 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.717 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:26:15.718 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:26:15.718 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-09-19T08:26:15.718 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.718 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.718 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.718 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:15.718 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.718 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-09-19T08:26:15.718 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2024-09-19T08:26:15.718 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 5, 2024-09-19T08:26:15.718 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 3, 2024-09-19T08:26:15.718 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 2, 2024-09-19T08:26:15.719 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-09-19T08:26:15.719 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 37730, 2024-09-19T08:26:15.719 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 18865, 2024-09-19T08:26:15.719 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 643, 2024-09-19T08:26:15.719 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-09-19T08:26:15.719 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.719 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-09-19T08:26:15.719 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:26:15.719 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-09-19T08:26:15.719 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:26:15.719 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:26:15.719 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:26:15.719 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:26:15.719 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:26:15.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:26:15.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:26:15.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:26:15.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:26:15.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:26:15.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:26:15.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:26:15.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:26:15.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-09-19T08:26:15.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:26:15.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-09-19T08:26:15.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:26:15.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:26:15.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:26:15.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:26:15.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:26:15.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:26:15.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:26:15.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-09-19T08:26:15.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:26:15.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:26:15.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:26:15.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-09-19T08:26:15.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-09-19T08:26:15.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-09-19T08:26:15.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:26:15.722 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:26:15.722 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:26:15.722 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:26:15.722 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 3197352, 2024-09-19T08:26:15.722 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 40319, 2024-09-19T08:26:15.722 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:26:15.722 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-09-19T08:26:15.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-09-19T08:26:15.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-09-19T08:26:15.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:26:15.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-09-19T08:26:15.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:26:15.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-09-19T08:26:15.722 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 8344, 2024-09-19T08:26:15.723 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 184, 2024-09-19T08:26:15.723 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:26:15.723 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:26:15.723 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-09-19T08:26:15.723 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-09-19T08:26:15.723 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 174600538, 2024-09-19T08:26:15.723 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 1611282, 2024-09-19T08:26:15.723 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-09-19T08:26:15.723 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-09-19T08:26:15.723 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-09-19T08:26:15.723 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-09-19T08:26:15.723 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.723 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-09-19T08:26:15.723 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-09-19T08:26:15.724 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-09-19T08:26:15.724 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 210824, 2024-09-19T08:26:15.724 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 1080763931, 2024-09-19T08:26:15.724 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 28, 2024-09-19T08:26:15.724 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 210824, 2024-09-19T08:26:15.724 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-09-19T08:26:15.724 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 210824, 2024-09-19T08:26:15.724 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1933.038383471, 2024-09-19T08:26:15.724 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.009168967 2024-09-19T08:26:15.724 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.724 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-09-19T08:26:15.724 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-09-19T08:26:15.724 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 210824, 2024-09-19T08:26:15.725 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 260914 2024-09-19T08:26:15.725 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.725 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 210824, 2024-09-19T08:26:15.725 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 118359, 2024-09-19T08:26:15.725 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 92465, 2024-09-19T08:26:15.725 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-09-19T08:26:15.725 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-09-19T08:26:15.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 1594, 2024-09-19T08:26:15.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 21431, 2024-09-19T08:26:15.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 733, 2024-09-19T08:26:15.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 21257, 2024-09-19T08:26:15.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 335, 2024-09-19T08:26:15.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-09-19T08:26:15.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-09-19T08:26:15.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-09-19T08:26:15.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 170, 2024-09-19T08:26:15.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 45699, 2024-09-19T08:26:15.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-09-19T08:26:15.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-09-19T08:26:15.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-09-19T08:26:15.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 117599, 2024-09-19T08:26:15.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 42862, 2024-09-19T08:26:15.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:26:15.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-09-19T08:26:15.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:26:15.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-09-19T08:26:15.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-09-19T08:26:15.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-09-19T08:26:15.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:26:15.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-09-19T08:26:15.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:26:15.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 1, 2024-09-19T08:26:15.727 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-09-19T08:26:15.727 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-09-19T08:26:15.727 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-09-19T08:26:15.727 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-09-19T08:26:15.727 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-09-19T08:26:15.727 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-09-19T08:26:15.727 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-09-19T08:26:15.727 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-09-19T08:26:15.727 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-09-19T08:26:15.728 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-09-19T08:26:15.728 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-09-19T08:26:15.728 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-09-19T08:26:15.728 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-09-19T08:26:15.728 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-09-19T08:26:15.728 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-09-19T08:26:15.728 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-09-19T08:26:15.728 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 95, 2024-09-19T08:26:15.728 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-09-19T08:26:15.728 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 12, 2024-09-19T08:26:15.728 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 12, 2024-09-19T08:26:15.728 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 12, 2024-09-19T08:26:15.728 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-09-19T08:26:15.729 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-09-19T08:26:15.730 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 7066, 2024-09-19T08:26:15.730 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 58, 2024-09-19T08:26:15.730 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 2109 2024-09-19T08:26:15.730 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.730 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-09-19T08:26:15.730 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2024-09-19T08:26:15.730 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:26:15.730 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-09-19T08:26:15.730 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-09-19T08:26:15.730 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-09-19T08:26:15.730 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 96379 2024-09-19T08:26:15.730 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.731 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-09-19T08:26:15.731 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-09-19T08:26:15.731 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 18, 2024-09-19T08:26:15.731 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-09-19T08:26:15.731 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 10, 2024-09-19T08:26:15.731 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 45195, 2024-09-19T08:26:15.731 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 45187, 2024-09-19T08:26:15.731 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-09-19T08:26:15.731 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.731 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:26:15.731 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:15.731 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T08:26:15.731 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:15.732 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 673893, 2024-09-19T08:26:15.732 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 176464120, 2024-09-19T08:26:15.732 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:15.732 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 673893, 2024-09-19T08:26:15.732 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T08:26:15.732 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T08:26:15.732 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 673893, 2024-09-19T08:26:15.732 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 176464120, 2024-09-19T08:26:15.732 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:15.732 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.732 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.732 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.732 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-09-19T08:26:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T08:26:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T08:26:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T08:26:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 210824, 2024-09-19T08:26:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 1086269810, 2024-09-19T08:26:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 210824, 2024-09-19T08:26:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1086269810, 2024-09-19T08:26:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:15.734 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.734 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.734 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.734 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:15.734 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.734 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-09-19T08:26:15.734 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:15.734 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-09-19T08:26:15.734 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:15.734 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T08:26:15.734 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T08:26:15.734 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:15.734 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:15.734 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 210824, 2024-09-19T08:26:15.735 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 210824, 2024-09-19T08:26:15.735 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 210824, 2024-09-19T08:26:15.735 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 210824, 2024-09-19T08:26:15.735 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:15.735 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.735 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.735 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.735 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:15.735 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.735 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:26:15.735 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:15.735 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-09-19T08:26:15.735 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:15.735 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 45187, 2024-09-19T08:26:15.736 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 5506343, 2024-09-19T08:26:15.736 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:15.736 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 45187, 2024-09-19T08:26:15.736 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T08:26:15.736 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T08:26:15.736 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 733, 2024-09-19T08:26:15.736 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 5506343, 2024-09-19T08:26:15.736 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:15.736 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.736 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.736 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.736 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:15.736 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:15.737 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55f7c481e960": { 2024-09-19T08:26:15.737 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:15.737 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-09-19T08:26:15.740 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:15.740 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 274536, 2024-09-19T08:26:15.740 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1000307469, 2024-09-19T08:26:15.740 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:15.740 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 274536, 2024-09-19T08:26:15.740 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T08:26:15.740 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T08:26:15.740 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 20524, 2024-09-19T08:26:15.740 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1000307469, 2024-09-19T08:26:15.740 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:15.741 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:15.741 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:15.741 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:15.741 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:15.741 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:15.741 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-09-19T08:26:16.940 INFO:tasks.check_counter:Daemon mds.c mds.exported=15 2024-09-19T08:26:16.940 INFO:tasks.check_counter:Daemon mds.c mds.imported=57 2024-09-19T08:26:16.940 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-19T08:26:16.941 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-19T08:26:16.941 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T08:26:16.941 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=170 2024-09-19T08:26:16.941 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T08:26:16.941 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T08:26:17.192 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:26:17.253 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:17 smithi157 ceph-mon[35143]: pgmap v349: 97 pgs: 26 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 59 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-19T08:26:17.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:17 smithi022 ceph-mon[30963]: pgmap v349: 97 pgs: 26 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 59 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-19T08:26:17.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:17 smithi064 ceph-mon[34900]: pgmap v349: 97 pgs: 26 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 59 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-19T08:26:17.690 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-09-19T08:26:17.690 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:26:17.690 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 2145, 2024-09-19T08:26:17.690 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 2147, 2024-09-19T08:26:17.690 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 12786350, 2024-09-19T08:26:17.690 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 682117, 2024-09-19T08:26:17.690 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2024-09-19T08:26:17.690 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 2, 2024-09-19T08:26:17.690 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.163152265, 2024-09-19T08:26:17.691 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.050965066, 2024-09-19T08:26:17.691 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.046783251, 2024-09-19T08:26:17.691 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.052692024, 2024-09-19T08:26:17.691 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:17.691 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2145, 2024-09-19T08:26:17.691 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.063312745, 2024-09-19T08:26:17.691 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000029516 2024-09-19T08:26:17.691 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.691 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:17.691 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.691 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.691 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.691 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T08:26:17.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T08:26:17.692 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.692 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:26:17.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 268, 2024-09-19T08:26:17.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 436, 2024-09-19T08:26:17.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 65820, 2024-09-19T08:26:17.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 544074, 2024-09-19T08:26:17.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 7, 2024-09-19T08:26:17.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-09-19T08:26:17.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.031182579, 2024-09-19T08:26:17.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.012117391, 2024-09-19T08:26:17.692 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.006838116, 2024-09-19T08:26:17.693 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.008109247, 2024-09-19T08:26:17.693 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:17.693 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 433, 2024-09-19T08:26:17.693 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.020694043, 2024-09-19T08:26:17.693 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000047792 2024-09-19T08:26:17.693 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.693 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:17.693 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.693 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.693 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.693 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.693 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T08:26:17.693 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 459424 2024-09-19T08:26:17.694 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.694 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:26:17.694 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 267, 2024-09-19T08:26:17.694 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 241, 2024-09-19T08:26:17.694 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1670332, 2024-09-19T08:26:17.694 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 139454, 2024-09-19T08:26:17.694 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 3, 2024-09-19T08:26:17.694 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 2, 2024-09-19T08:26:17.694 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.042110831, 2024-09-19T08:26:17.694 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.013518811, 2024-09-19T08:26:17.694 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.043491024, 2024-09-19T08:26:17.694 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000033221, 2024-09-19T08:26:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 241, 2024-09-19T08:26:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.017823535, 2024-09-19T08:26:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000073956 2024-09-19T08:26:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1670064, 2024-09-19T08:26:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 139136 2024-09-19T08:26:17.695 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.696 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-09-19T08:26:17.696 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-09-19T08:26:17.696 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-09-19T08:26:17.696 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.696 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-09-19T08:26:17.696 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-09-19T08:26:17.696 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-09-19T08:26:17.696 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4097, 2024-09-19T08:26:17.696 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.095362846, 2024-09-19T08:26:17.696 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000023276 2024-09-19T08:26:17.697 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:17.697 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.697 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-09-19T08:26:17.697 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-09-19T08:26:17.697 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-09-19T08:26:17.697 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.697 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.697 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.697 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:17.697 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.697 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-09-19T08:26:17.697 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-09-19T08:26:17.698 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-09-19T08:26:17.698 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-09-19T08:26:17.698 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.698 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.698 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.698 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.698 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-09-19T08:26:17.699 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-09-19T08:26:17.699 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-09-19T08:26:17.699 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-09-19T08:26:17.699 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2024-09-19T08:26:17.699 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-09-19T08:26:17.699 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-09-19T08:26:17.699 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 1284, 2024-09-19T08:26:17.699 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 0, 2024-09-19T08:26:17.699 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 1284, 2024-09-19T08:26:17.699 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-09-19T08:26:17.699 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 1284, 2024-09-19T08:26:17.699 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-09-19T08:26:17.700 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-09-19T08:26:17.700 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-09-19T08:26:17.700 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 1, 2024-09-19T08:26:17.700 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-09-19T08:26:17.700 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-09-19T08:26:17.700 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-09-19T08:26:17.700 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-09-19T08:26:17.700 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:26:17.700 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-09-19T08:26:17.700 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-09-19T08:26:17.700 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-09-19T08:26:17.700 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-09-19T08:26:17.700 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-09-19T08:26:17.700 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-09-19T08:26:17.701 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-09-19T08:26:17.701 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-09-19T08:26:17.701 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-09-19T08:26:17.701 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-09-19T08:26:17.701 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-09-19T08:26:17.701 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2024-09-19T08:26:17.701 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2024-09-19T08:26:17.701 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-09-19T08:26:17.701 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T08:26:17.701 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-09-19T08:26:17.701 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:26:17.701 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:26:17.701 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-09-19T08:26:17.702 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T08:26:17.702 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T08:26:17.702 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-09-19T08:26:17.702 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-09-19T08:26:17.702 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T08:26:17.702 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-09-19T08:26:17.702 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T08:26:17.702 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:26:17.702 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:26:17.702 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-09-19T08:26:17.702 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-09-19T08:26:17.702 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-09-19T08:26:17.702 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-09-19T08:26:17.702 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-09-19T08:26:17.703 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:26:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:26:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-09-19T08:26:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-09-19T08:26:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 0, 2024-09-19T08:26:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 0, 2024-09-19T08:26:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T08:26:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-09-19T08:26:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-09-19T08:26:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-09-19T08:26:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-09-19T08:26:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-09-19T08:26:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-09-19T08:26:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-09-19T08:26:17.705 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2024-09-19T08:26:17.705 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-09-19T08:26:17.705 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-09-19T08:26:17.705 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 6484, 2024-09-19T08:26:17.705 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-09-19T08:26:17.705 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-09-19T08:26:17.705 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2024-09-19T08:26:17.705 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-09-19T08:26:17.705 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-09-19T08:26:17.705 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 9, 2024-09-19T08:26:17.705 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 2, 2024-09-19T08:26:17.705 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-09-19T08:26:17.706 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-09-19T08:26:17.706 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-09-19T08:26:17.706 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 16367029, 2024-09-19T08:26:17.706 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 16362103, 2024-09-19T08:26:17.706 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-09-19T08:26:17.706 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.706 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.706 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.706 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.706 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 6484 2024-09-19T08:26:17.706 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.706 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-09-19T08:26:17.706 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 11, 2024-09-19T08:26:17.707 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 2682, 2024-09-19T08:26:17.707 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 2671, 2024-09-19T08:26:17.707 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 9, 2024-09-19T08:26:17.707 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 60, 2024-09-19T08:26:17.707 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 51, 2024-09-19T08:26:17.707 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 1284, 2024-09-19T08:26:17.707 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 3969, 2024-09-19T08:26:17.708 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 2685, 2024-09-19T08:26:17.708 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-09-19T08:26:17.708 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-09-19T08:26:17.708 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-09-19T08:26:17.708 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 55956, 2024-09-19T08:26:17.708 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 215328 2024-09-19T08:26:17.708 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.708 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-09-19T08:26:17.709 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-09-19T08:26:17.709 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-09-19T08:26:17.709 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-09-19T08:26:17.709 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-09-19T08:26:17.709 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-09-19T08:26:17.709 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-09-19T08:26:17.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.709 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.709 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.709 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-09-19T08:26:17.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.709 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.710 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.710 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-09-19T08:26:17.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.710 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.710 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.710 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-09-19T08:26:17.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.710 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.710 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.710 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-09-19T08:26:17.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.711 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.711 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-09-19T08:26:17.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.711 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.711 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-09-19T08:26:17.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.711 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.712 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-09-19T08:26:17.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.712 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.712 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.712 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-09-19T08:26:17.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.712 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.712 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.712 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-09-19T08:26:17.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.712 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.713 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.713 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-09-19T08:26:17.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.713 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.713 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.713 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-09-19T08:26:17.714 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.714 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.714 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.715 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.715 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-09-19T08:26:17.715 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.715 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.715 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.715 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.715 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-09-19T08:26:17.715 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.715 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.715 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.716 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.716 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-09-19T08:26:17.716 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.716 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.716 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.716 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.716 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-09-19T08:26:17.716 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.717 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.717 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.717 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.717 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-09-19T08:26:17.717 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.717 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.717 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.717 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.717 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-09-19T08:26:17.717 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.717 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.718 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.718 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.718 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-09-19T08:26:17.718 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.718 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.718 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.718 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.718 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-09-19T08:26:17.718 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.718 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.718 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.718 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.718 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-09-19T08:26:17.718 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.719 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.719 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.719 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.719 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-09-19T08:26:17.719 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.719 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.719 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.719 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.719 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-09-19T08:26:17.719 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.719 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.719 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.719 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.720 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-09-19T08:26:17.720 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.720 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.720 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.720 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.720 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-09-19T08:26:17.720 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.720 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.720 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.720 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.720 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-09-19T08:26:17.720 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.720 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.721 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.721 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.721 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-09-19T08:26:17.721 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.721 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.721 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.721 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.721 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-09-19T08:26:17.721 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.721 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.721 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.721 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.721 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-09-19T08:26:17.721 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.722 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.722 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.722 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.722 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:26:17.722 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:26:17.722 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-09-19T08:26:17.722 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.722 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.722 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.722 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:17.722 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.722 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-09-19T08:26:17.722 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2024-09-19T08:26:17.723 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 5, 2024-09-19T08:26:17.723 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 3, 2024-09-19T08:26:17.723 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 4, 2024-09-19T08:26:17.723 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-09-19T08:26:17.723 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-09-19T08:26:17.723 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-09-19T08:26:17.723 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 3229, 2024-09-19T08:26:17.723 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-09-19T08:26:17.723 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.723 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-09-19T08:26:17.723 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:26:17.723 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-09-19T08:26:17.723 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:26:17.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:26:17.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:26:17.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:26:17.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:26:17.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:26:17.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:26:17.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:26:17.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:26:17.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:26:17.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:26:17.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:26:17.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:26:17.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-09-19T08:26:17.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:26:17.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-09-19T08:26:17.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:26:17.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:26:17.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:26:17.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:26:17.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:26:17.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:26:17.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:26:17.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-09-19T08:26:17.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:26:17.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:26:17.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:26:17.726 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-09-19T08:26:17.726 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-09-19T08:26:17.726 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-09-19T08:26:17.726 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:26:17.726 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:26:17.726 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:26:17.726 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:26:17.726 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 190184, 2024-09-19T08:26:17.726 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 53, 2024-09-19T08:26:17.726 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:26:17.726 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-09-19T08:26:17.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-09-19T08:26:17.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-09-19T08:26:17.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 8344, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 184, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 958496, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 5274, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-09-19T08:26:17.727 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-09-19T08:26:17.728 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.728 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-09-19T08:26:17.728 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-09-19T08:26:17.728 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-09-19T08:26:17.728 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 2412, 2024-09-19T08:26:17.728 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 21, 2024-09-19T08:26:17.728 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 3, 2024-09-19T08:26:17.728 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 2412, 2024-09-19T08:26:17.728 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-09-19T08:26:17.728 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2412, 2024-09-19T08:26:17.728 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.754968577, 2024-09-19T08:26:17.728 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000727598 2024-09-19T08:26:17.728 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.729 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-09-19T08:26:17.729 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-09-19T08:26:17.729 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2412, 2024-09-19T08:26:17.729 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2413 2024-09-19T08:26:17.729 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.729 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 2412, 2024-09-19T08:26:17.729 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 2412, 2024-09-19T08:26:17.729 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2024-09-19T08:26:17.729 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-09-19T08:26:17.729 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-09-19T08:26:17.729 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 1483, 2024-09-19T08:26:17.729 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2024-09-19T08:26:17.729 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 927, 2024-09-19T08:26:17.729 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2024-09-19T08:26:17.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2024-09-19T08:26:17.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-09-19T08:26:17.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-09-19T08:26:17.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-09-19T08:26:17.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-09-19T08:26:17.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2024-09-19T08:26:17.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-09-19T08:26:17.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-09-19T08:26:17.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-09-19T08:26:17.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 1, 2024-09-19T08:26:17.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2024-09-19T08:26:17.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:26:17.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-09-19T08:26:17.730 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:26:17.731 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-09-19T08:26:17.731 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-09-19T08:26:17.731 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-09-19T08:26:17.731 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:26:17.731 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-09-19T08:26:17.731 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:26:17.731 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-09-19T08:26:17.731 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-09-19T08:26:17.731 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-09-19T08:26:17.731 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-09-19T08:26:17.731 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-09-19T08:26:17.731 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-09-19T08:26:17.731 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-09-19T08:26:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-09-19T08:26:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-09-19T08:26:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-09-19T08:26:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-09-19T08:26:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-09-19T08:26:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-09-19T08:26:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-09-19T08:26:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-09-19T08:26:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-09-19T08:26:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-09-19T08:26:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 95, 2024-09-19T08:26:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-09-19T08:26:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 12, 2024-09-19T08:26:17.732 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 5, 2024-09-19T08:26:17.733 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 5, 2024-09-19T08:26:17.733 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-09-19T08:26:17.733 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-09-19T08:26:17.733 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2024-09-19T08:26:17.733 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2024-09-19T08:26:17.733 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2024-09-19T08:26:17.733 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.733 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-09-19T08:26:17.733 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2024-09-19T08:26:17.733 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:26:17.733 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-09-19T08:26:17.733 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-09-19T08:26:17.733 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.734 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-09-19T08:26:17.734 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-09-19T08:26:17.734 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.734 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-09-19T08:26:17.734 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-09-19T08:26:17.734 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T08:26:17.734 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-09-19T08:26:17.734 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-09-19T08:26:17.734 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 0, 2024-09-19T08:26:17.734 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-09-19T08:26:17.734 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-09-19T08:26:17.734 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.734 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:26:17.734 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:17.735 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T08:26:17.735 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:17.735 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 2676, 2024-09-19T08:26:17.735 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 14265255, 2024-09-19T08:26:17.735 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:17.735 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 2676, 2024-09-19T08:26:17.735 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T08:26:17.735 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T08:26:17.735 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2676, 2024-09-19T08:26:17.735 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 14265255, 2024-09-19T08:26:17.735 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:17.735 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.735 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.735 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.736 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:17.736 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.736 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-09-19T08:26:17.736 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:17.736 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T08:26:17.736 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:17.736 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T08:26:17.736 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T08:26:17.736 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:17.736 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:17.736 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2412, 2024-09-19T08:26:17.736 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 12174542, 2024-09-19T08:26:17.736 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2412, 2024-09-19T08:26:17.736 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 12174542, 2024-09-19T08:26:17.737 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:17.737 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.737 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.737 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.737 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:17.737 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.737 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-09-19T08:26:17.737 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:17.737 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-09-19T08:26:17.737 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:17.737 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T08:26:17.737 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T08:26:17.737 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:17.737 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:17.738 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2412, 2024-09-19T08:26:17.738 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 2412, 2024-09-19T08:26:17.738 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2412, 2024-09-19T08:26:17.738 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2412, 2024-09-19T08:26:17.738 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:17.738 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.738 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.738 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.738 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:17.738 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.738 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:26:17.738 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:17.738 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-09-19T08:26:17.738 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T08:26:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T08:26:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T08:26:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T08:26:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-09-19T08:26:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-09-19T08:26:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55a48e314b40": { 2024-09-19T08:26:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-09-19T08:26:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T08:26:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T08:26:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T08:26:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T08:26:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-09-19T08:26:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-09-19T08:26:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:17.742 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-09-19T08:26:19.205 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-19T08:26:19.205 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-19T08:26:19.205 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-19T08:26:19.205 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-19T08:26:19.206 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T08:26:19.206 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-19T08:26:19.206 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T08:26:19.206 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T08:26:19.462 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:26:19.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:19 smithi157 ceph-mon[35143]: pgmap v350: 97 pgs: 26 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 59 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 199 B/s wr, 8 op/s 2024-09-19T08:26:19.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:19 smithi022 ceph-mon[30963]: pgmap v350: 97 pgs: 26 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 59 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 199 B/s wr, 8 op/s 2024-09-19T08:26:19.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:19 smithi064 ceph-mon[34900]: pgmap v350: 97 pgs: 26 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 59 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 199 B/s wr, 8 op/s 2024-09-19T08:26:19.947 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-09-19T08:26:19.947 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:26:19.947 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 2806, 2024-09-19T08:26:19.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 2565, 2024-09-19T08:26:19.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 803044, 2024-09-19T08:26:19.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 11816895, 2024-09-19T08:26:19.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 157, 2024-09-19T08:26:19.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 8, 2024-09-19T08:26:19.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.254863688, 2024-09-19T08:26:19.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.075564068, 2024-09-19T08:26:19.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.091595462, 2024-09-19T08:26:19.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.064977537, 2024-09-19T08:26:19.948 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:19.948 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2560, 2024-09-19T08:26:19.948 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.186990636, 2024-09-19T08:26:19.949 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000073043 2024-09-19T08:26:19.949 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.949 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:19.949 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 536, 2024-09-19T08:26:19.949 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000075217, 2024-09-19T08:26:19.949 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000140 2024-09-19T08:26:19.949 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.949 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 143488, 2024-09-19T08:26:19.949 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 34864 2024-09-19T08:26:19.949 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.949 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:26:19.949 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 39729, 2024-09-19T08:26:19.949 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 49821, 2024-09-19T08:26:19.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 15150184, 2024-09-19T08:26:19.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 33460492, 2024-09-19T08:26:19.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 11, 2024-09-19T08:26:19.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 7, 2024-09-19T08:26:19.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 2.350364674, 2024-09-19T08:26:19.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 1.070169004, 2024-09-19T08:26:19.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 1.851818163, 2024-09-19T08:26:19.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.111873592, 2024-09-19T08:26:19.950 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:19.950 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 49822, 2024-09-19T08:26:19.950 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2.995654426, 2024-09-19T08:26:19.950 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000060127 2024-09-19T08:26:19.950 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.951 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:19.951 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 75219, 2024-09-19T08:26:19.951 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.011967677, 2024-09-19T08:26:19.951 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000159 2024-09-19T08:26:19.951 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.951 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 11786032, 2024-09-19T08:26:19.951 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 23963488 2024-09-19T08:26:19.951 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.951 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:26:19.951 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 3227, 2024-09-19T08:26:19.951 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 3254, 2024-09-19T08:26:19.951 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 2687031, 2024-09-19T08:26:19.951 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 4900699, 2024-09-19T08:26:19.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 10, 2024-09-19T08:26:19.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2024-09-19T08:26:19.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.248273293, 2024-09-19T08:26:19.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.087890485, 2024-09-19T08:26:19.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.128831517, 2024-09-19T08:26:19.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.047798993, 2024-09-19T08:26:19.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:19.952 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3253, 2024-09-19T08:26:19.952 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.485683763, 2024-09-19T08:26:19.952 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000149303 2024-09-19T08:26:19.952 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.952 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:19.953 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1935, 2024-09-19T08:26:19.953 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000398372, 2024-09-19T08:26:19.953 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000205 2024-09-19T08:26:19.953 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.953 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1736096, 2024-09-19T08:26:19.953 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 168736 2024-09-19T08:26:19.953 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.953 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-09-19T08:26:19.953 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-09-19T08:26:19.953 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-09-19T08:26:19.953 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.953 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-09-19T08:26:19.954 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-09-19T08:26:19.954 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-09-19T08:26:19.954 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4013, 2024-09-19T08:26:19.954 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 4.034459192, 2024-09-19T08:26:19.954 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001005347 2024-09-19T08:26:19.954 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:19.954 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.954 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-09-19T08:26:19.954 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-09-19T08:26:19.954 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-09-19T08:26:19.954 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2762, 2024-09-19T08:26:19.954 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.323130706, 2024-09-19T08:26:19.954 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000116991 2024-09-19T08:26:19.955 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:19.955 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.955 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-09-19T08:26:19.955 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 17537, 2024-09-19T08:26:19.955 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 17348, 2024-09-19T08:26:19.955 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-09-19T08:26:19.955 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 17348, 2024-09-19T08:26:19.955 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 333.403047730, 2024-09-19T08:26:19.955 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.019218529 2024-09-19T08:26:19.955 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.955 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-09-19T08:26:19.955 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 189, 2024-09-19T08:26:19.955 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 6, 2024-09-19T08:26:19.956 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-09-19T08:26:19.956 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 48, 2024-09-19T08:26:19.956 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 4, 2024-09-19T08:26:19.956 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-09-19T08:26:19.956 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 200, 2024-09-19T08:26:19.956 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 1, 2024-09-19T08:26:19.956 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 132, 2024-09-19T08:26:19.956 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 67, 2024-09-19T08:26:19.956 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 198, 2024-09-19T08:26:19.956 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 2475, 2024-09-19T08:26:19.956 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 3, 2024-09-19T08:26:19.956 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 4, 2024-09-19T08:26:19.957 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 3, 2024-09-19T08:26:19.957 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 34500, 2024-09-19T08:26:19.957 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 29120, 2024-09-19T08:26:19.957 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 189, 2024-09-19T08:26:19.957 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 8, 2024-09-19T08:26:19.957 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:26:19.957 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-09-19T08:26:19.957 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 248, 2024-09-19T08:26:19.957 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-09-19T08:26:19.957 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-09-19T08:26:19.957 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 33, 2024-09-19T08:26:19.957 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 2162, 2024-09-19T08:26:19.957 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 5, 2024-09-19T08:26:19.958 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 792, 2024-09-19T08:26:19.958 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-09-19T08:26:19.958 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 2946, 2024-09-19T08:26:19.958 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-09-19T08:26:19.958 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 4, 2024-09-19T08:26:19.958 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 234, 2024-09-19T08:26:19.958 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 6, 2024-09-19T08:26:19.958 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 2946, 2024-09-19T08:26:19.958 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-09-19T08:26:19.958 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:26:19.958 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:26:19.958 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 144, 2024-09-19T08:26:19.958 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 13, 2024-09-19T08:26:19.959 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 131, 2024-09-19T08:26:19.959 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 2802, 2024-09-19T08:26:19.959 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 65, 2024-09-19T08:26:19.959 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 3981, 2024-09-19T08:26:19.959 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 4846, 2024-09-19T08:26:19.959 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 5, 2024-09-19T08:26:19.959 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:26:19.959 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:26:19.959 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 6580, 2024-09-19T08:26:19.959 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 1671, 2024-09-19T08:26:19.959 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 176, 2024-09-19T08:26:19.959 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 5084 2024-09-19T08:26:19.960 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.960 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-09-19T08:26:19.960 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 4, 2024-09-19T08:26:19.960 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 20, 2024-09-19T08:26:19.960 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 11, 2024-09-19T08:26:19.960 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 27, 2024-09-19T08:26:19.960 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 34, 2024-09-19T08:26:19.960 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 6, 2024-09-19T08:26:19.960 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-09-19T08:26:19.960 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:26:19.960 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 1580, 2024-09-19T08:26:19.960 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 1574, 2024-09-19T08:26:19.960 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-09-19T08:26:19.961 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-09-19T08:26:19.961 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-09-19T08:26:19.961 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:26:19.961 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:26:19.961 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-09-19T08:26:19.961 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-09-19T08:26:19.961 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 8, 2024-09-19T08:26:19.961 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 2076, 2024-09-19T08:26:19.961 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 13, 2024-09-19T08:26:19.961 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 33, 2024-09-19T08:26:19.961 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-09-19T08:26:19.961 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 4, 2024-09-19T08:26:19.961 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-09-19T08:26:19.962 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-09-19T08:26:19.962 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.962 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-09-19T08:26:19.962 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-09-19T08:26:19.962 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 8206, 2024-09-19T08:26:19.962 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-09-19T08:26:19.962 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-09-19T08:26:19.962 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 8206, 2024-09-19T08:26:19.962 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-09-19T08:26:19.962 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-09-19T08:26:19.962 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 12, 2024-09-19T08:26:19.962 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-09-19T08:26:19.963 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-09-19T08:26:19.963 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 12, 2024-09-19T08:26:19.963 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 2, 2024-09-19T08:26:19.963 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-09-19T08:26:19.963 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-09-19T08:26:19.963 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-09-19T08:26:19.963 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 21485839, 2024-09-19T08:26:19.963 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2024-09-19T08:26:19.963 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-09-19T08:26:19.963 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 888, 2024-09-19T08:26:19.963 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 4.249090287, 2024-09-19T08:26:19.963 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.004785011 2024-09-19T08:26:19.963 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.963 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2024-09-19T08:26:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-09-19T08:26:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 97, 2024-09-19T08:26:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 2832, 2024-09-19T08:26:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 2735, 2024-09-19T08:26:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 27, 2024-09-19T08:26:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 88, 2024-09-19T08:26:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 61, 2024-09-19T08:26:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 200, 2024-09-19T08:26:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 4467, 2024-09-19T08:26:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 4267, 2024-09-19T08:26:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 4, 2024-09-19T08:26:19.964 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 2185, 2024-09-19T08:26:19.965 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 2181, 2024-09-19T08:26:19.965 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 74680, 2024-09-19T08:26:19.965 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 231712 2024-09-19T08:26:19.965 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.965 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-09-19T08:26:19.965 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 21379, 2024-09-19T08:26:19.965 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 76, 2024-09-19T08:26:19.965 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 17537, 2024-09-19T08:26:19.965 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 38, 2024-09-19T08:26:19.965 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 2683, 2024-09-19T08:26:19.965 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-09-19T08:26:19.965 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1250, 2024-09-19T08:26:19.965 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 28.002180067, 2024-09-19T08:26:19.966 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.022401744 2024-09-19T08:26:19.966 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.966 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-09-19T08:26:19.966 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1981, 2024-09-19T08:26:19.966 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 19.413807047, 2024-09-19T08:26:19.966 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.009800003 2024-09-19T08:26:19.966 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.966 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-09-19T08:26:19.966 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.966 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.966 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.966 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.966 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-09-19T08:26:19.967 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.967 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.967 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.967 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.967 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-09-19T08:26:19.967 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 11187, 2024-09-19T08:26:19.967 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 238.890706410, 2024-09-19T08:26:19.967 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.021354313 2024-09-19T08:26:19.967 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.967 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-09-19T08:26:19.967 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.967 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.967 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.967 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.968 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-09-19T08:26:19.968 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2, 2024-09-19T08:26:19.968 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000293309, 2024-09-19T08:26:19.968 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000146654 2024-09-19T08:26:19.968 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.968 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-09-19T08:26:19.968 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.968 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.968 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.968 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.968 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-09-19T08:26:19.968 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.968 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.969 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.969 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.969 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-09-19T08:26:19.969 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.969 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.969 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.969 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.970 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-09-19T08:26:19.970 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 11, 2024-09-19T08:26:19.970 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.001538442, 2024-09-19T08:26:19.970 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000139858 2024-09-19T08:26:19.970 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.970 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-09-19T08:26:19.970 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 16, 2024-09-19T08:26:19.970 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 18.135275215, 2024-09-19T08:26:19.970 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 1.133454700 2024-09-19T08:26:19.970 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.970 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-09-19T08:26:19.971 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.971 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.971 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.971 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.971 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-09-19T08:26:19.971 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 11, 2024-09-19T08:26:19.971 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2.423062975, 2024-09-19T08:26:19.971 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.220278452 2024-09-19T08:26:19.971 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.971 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-09-19T08:26:19.971 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 799, 2024-09-19T08:26:19.971 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.063408706, 2024-09-19T08:26:19.971 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001330924 2024-09-19T08:26:19.972 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.972 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-09-19T08:26:19.972 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 100, 2024-09-19T08:26:19.972 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.502446851, 2024-09-19T08:26:19.972 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.005024468 2024-09-19T08:26:19.972 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.972 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-09-19T08:26:19.972 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 892, 2024-09-19T08:26:19.972 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 5.566828263, 2024-09-19T08:26:19.972 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.006240838 2024-09-19T08:26:19.972 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.972 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-09-19T08:26:19.972 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.973 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.973 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.973 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.973 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-09-19T08:26:19.973 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.973 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.973 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.973 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.973 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-09-19T08:26:19.973 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 10, 2024-09-19T08:26:19.973 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.113309383, 2024-09-19T08:26:19.973 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.011330938 2024-09-19T08:26:19.973 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.974 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-09-19T08:26:19.974 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 5, 2024-09-19T08:26:19.974 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.261949618, 2024-09-19T08:26:19.974 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.052389923 2024-09-19T08:26:19.974 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.974 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-09-19T08:26:19.974 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.974 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.974 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.974 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.974 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-09-19T08:26:19.974 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 7, 2024-09-19T08:26:19.974 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 17.459733079, 2024-09-19T08:26:19.975 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 2.494247582 2024-09-19T08:26:19.975 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.975 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-09-19T08:26:19.975 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.975 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.975 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.975 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.975 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-09-19T08:26:19.975 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.975 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.975 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.975 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.975 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-09-19T08:26:19.976 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.976 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.976 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.976 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.976 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-09-19T08:26:19.976 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 8, 2024-09-19T08:26:19.976 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.021643312, 2024-09-19T08:26:19.976 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.002705414 2024-09-19T08:26:19.976 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.976 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-09-19T08:26:19.976 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3, 2024-09-19T08:26:19.976 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.001706363, 2024-09-19T08:26:19.977 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000568787 2024-09-19T08:26:19.977 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.977 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-09-19T08:26:19.977 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1066, 2024-09-19T08:26:19.977 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.545158690, 2024-09-19T08:26:19.977 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001449492 2024-09-19T08:26:19.977 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.977 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:26:19.977 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:26:19.977 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-09-19T08:26:19.977 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.977 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.977 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.978 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:19.978 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.978 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-09-19T08:26:19.978 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2024-09-19T08:26:19.978 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 6, 2024-09-19T08:26:19.978 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 4, 2024-09-19T08:26:19.978 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 2, 2024-09-19T08:26:19.978 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-09-19T08:26:19.978 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 597, 2024-09-19T08:26:19.978 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 298, 2024-09-19T08:26:19.978 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 646, 2024-09-19T08:26:19.978 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-09-19T08:26:19.978 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.978 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-09-19T08:26:19.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:26:19.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-09-19T08:26:19.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:26:19.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:26:19.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:26:19.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:26:19.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:26:19.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:26:19.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:26:19.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:26:19.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:26:19.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:26:19.979 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:26:19.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:26:19.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:26:19.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-09-19T08:26:19.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:26:19.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-09-19T08:26:19.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:26:19.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:26:19.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:26:19.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:26:19.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:26:19.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:26:19.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:26:19.980 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-09-19T08:26:19.981 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:26:19.981 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:26:19.981 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:26:19.981 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-09-19T08:26:19.981 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-09-19T08:26:19.981 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-09-19T08:26:19.981 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:26:19.981 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:26:19.981 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:26:19.981 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:26:19.981 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 369845, 2024-09-19T08:26:19.981 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 194, 2024-09-19T08:26:19.981 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:26:19.982 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-09-19T08:26:19.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-09-19T08:26:19.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-09-19T08:26:19.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:26:19.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-09-19T08:26:19.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:26:19.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-09-19T08:26:19.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 8344, 2024-09-19T08:26:19.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 184, 2024-09-19T08:26:19.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:26:19.982 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:26:19.982 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-09-19T08:26:19.982 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-09-19T08:26:19.982 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 501925, 2024-09-19T08:26:19.983 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 4845, 2024-09-19T08:26:19.983 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-09-19T08:26:19.983 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-09-19T08:26:19.983 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-09-19T08:26:19.983 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-09-19T08:26:19.983 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.983 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-09-19T08:26:19.983 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-09-19T08:26:19.983 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-09-19T08:26:19.983 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 6426, 2024-09-19T08:26:19.983 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 18350465, 2024-09-19T08:26:19.983 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 36, 2024-09-19T08:26:19.983 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 6426, 2024-09-19T08:26:19.984 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-09-19T08:26:19.984 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6426, 2024-09-19T08:26:19.984 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 36.578344895, 2024-09-19T08:26:19.984 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.005692241 2024-09-19T08:26:19.984 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.984 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-09-19T08:26:19.984 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-09-19T08:26:19.984 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6426, 2024-09-19T08:26:19.984 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 6669 2024-09-19T08:26:19.984 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.984 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 6426, 2024-09-19T08:26:19.984 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 3058, 2024-09-19T08:26:19.985 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 3368, 2024-09-19T08:26:19.985 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-09-19T08:26:19.985 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-09-19T08:26:19.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 0, 2024-09-19T08:26:19.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 11, 2024-09-19T08:26:19.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 93, 2024-09-19T08:26:19.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 968, 2024-09-19T08:26:19.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 92, 2024-09-19T08:26:19.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-09-19T08:26:19.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-09-19T08:26:19.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-09-19T08:26:19.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 4, 2024-09-19T08:26:19.985 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 2090, 2024-09-19T08:26:19.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-09-19T08:26:19.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-09-19T08:26:19.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-09-19T08:26:19.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 2959, 2024-09-19T08:26:19.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 22, 2024-09-19T08:26:19.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:26:19.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-09-19T08:26:19.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:26:19.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-09-19T08:26:19.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-09-19T08:26:19.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-09-19T08:26:19.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:26:19.986 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-09-19T08:26:19.987 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:26:19.987 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 5, 2024-09-19T08:26:19.987 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-09-19T08:26:19.987 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-09-19T08:26:19.987 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-09-19T08:26:19.987 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-09-19T08:26:19.987 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-09-19T08:26:19.987 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-09-19T08:26:19.987 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-09-19T08:26:19.987 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-09-19T08:26:19.987 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-09-19T08:26:19.988 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-09-19T08:26:19.988 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-09-19T08:26:19.988 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-09-19T08:26:19.988 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-09-19T08:26:19.988 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-09-19T08:26:19.988 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-09-19T08:26:19.988 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-09-19T08:26:19.988 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 95, 2024-09-19T08:26:19.988 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-09-19T08:26:19.988 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 12, 2024-09-19T08:26:19.988 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 12, 2024-09-19T08:26:19.988 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 12, 2024-09-19T08:26:19.989 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-09-19T08:26:19.989 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-09-19T08:26:19.989 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 337, 2024-09-19T08:26:19.989 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 12, 2024-09-19T08:26:19.989 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 76 2024-09-19T08:26:19.989 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.989 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-09-19T08:26:19.989 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2024-09-19T08:26:19.989 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:26:19.989 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-09-19T08:26:19.989 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-09-19T08:26:19.989 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.989 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-09-19T08:26:19.990 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 2087 2024-09-19T08:26:19.990 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.990 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-09-19T08:26:19.990 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-09-19T08:26:19.990 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 10, 2024-09-19T08:26:19.990 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-09-19T08:26:19.990 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 10, 2024-09-19T08:26:19.990 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 1574, 2024-09-19T08:26:19.990 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 1574, 2024-09-19T08:26:19.990 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-09-19T08:26:19.990 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.990 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:26:19.990 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:19.991 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T08:26:19.991 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:19.991 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 45758, 2024-09-19T08:26:19.991 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 12934717, 2024-09-19T08:26:19.991 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:19.991 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 45758, 2024-09-19T08:26:19.991 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T08:26:19.991 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T08:26:19.991 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 45758, 2024-09-19T08:26:19.991 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 12934717, 2024-09-19T08:26:19.991 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:19.991 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.991 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.992 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.992 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:19.992 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.992 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-09-19T08:26:19.992 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:19.992 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T08:26:19.992 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:19.992 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T08:26:19.992 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T08:26:19.992 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:19.992 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:19.992 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 6426, 2024-09-19T08:26:19.993 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 18531719, 2024-09-19T08:26:19.993 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 6426, 2024-09-19T08:26:19.993 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 18531719, 2024-09-19T08:26:19.993 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:19.993 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.993 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.993 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.993 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:19.993 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.993 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-09-19T08:26:19.993 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:19.993 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-09-19T08:26:19.993 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:19.994 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-09-19T08:26:19.994 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-09-19T08:26:19.994 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:19.994 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-09-19T08:26:19.994 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 6426, 2024-09-19T08:26:19.994 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 6426, 2024-09-19T08:26:19.994 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 6426, 2024-09-19T08:26:19.994 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 6426, 2024-09-19T08:26:19.994 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:19.994 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.994 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.994 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.994 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:19.994 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.995 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:26:19.995 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:19.995 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-09-19T08:26:19.995 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:19.995 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 1574, 2024-09-19T08:26:19.995 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 181350, 2024-09-19T08:26:19.995 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:19.995 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 1574, 2024-09-19T08:26:19.995 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T08:26:19.995 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T08:26:19.995 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 80, 2024-09-19T08:26:19.995 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 181350, 2024-09-19T08:26:19.995 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:19.996 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.996 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.996 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.996 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:19.996 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:19.996 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x561b349b4960": { 2024-09-19T08:26:19.996 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:19.996 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-09-19T08:26:19.996 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:19.996 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 8206, 2024-09-19T08:26:19.996 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 17291535, 2024-09-19T08:26:19.996 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:19.996 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 8206, 2024-09-19T08:26:19.996 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T08:26:19.997 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T08:26:19.997 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 888, 2024-09-19T08:26:19.997 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 17291535, 2024-09-19T08:26:19.997 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:19.997 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:19.997 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:19.997 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:19.997 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:19.997 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:19.997 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-09-19T08:26:20.731 INFO:tasks.check_counter:Daemon mds.i mds.exported=33 2024-09-19T08:26:20.731 INFO:tasks.check_counter:Daemon mds.i mds.imported=5 2024-09-19T08:26:20.731 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-19T08:26:20.731 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=11 2024-09-19T08:26:20.731 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=5 2024-09-19T08:26:20.731 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=4 2024-09-19T08:26:20.732 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T08:26:20.732 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T08:26:20.991 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:26:21.380 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:21 smithi157 ceph-mon[35143]: pgmap v351: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T08:26:21.380 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:21 smithi157 ceph-mon[35143]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T08:26:21.481 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-09-19T08:26:21.482 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:26:21.482 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 237, 2024-09-19T08:26:21.482 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 226, 2024-09-19T08:26:21.482 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1590848, 2024-09-19T08:26:21.482 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 138944, 2024-09-19T08:26:21.482 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 4, 2024-09-19T08:26:21.482 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 0, 2024-09-19T08:26:21.482 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.038658111, 2024-09-19T08:26:21.483 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.012859730, 2024-09-19T08:26:21.483 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.039495808, 2024-09-19T08:26:21.483 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:21.483 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:21.483 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 225, 2024-09-19T08:26:21.483 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.010876408, 2024-09-19T08:26:21.483 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000048339 2024-09-19T08:26:21.483 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:21.483 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:21.483 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:21.483 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:21.484 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:21.484 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:21.484 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1590848, 2024-09-19T08:26:21.484 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 138944 2024-09-19T08:26:21.484 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:21.484 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:26:21.484 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 2, 2024-09-19T08:26:21.484 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 173, 2024-09-19T08:26:21.484 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 320, 2024-09-19T08:26:21.484 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 187680, 2024-09-19T08:26:21.484 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2024-09-19T08:26:21.485 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2024-09-19T08:26:21.485 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.009579009, 2024-09-19T08:26:21.485 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.006474275, 2024-09-19T08:26:21.485 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.001175237, 2024-09-19T08:26:21.485 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:21.485 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:21.485 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 170, 2024-09-19T08:26:21.485 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.015302421, 2024-09-19T08:26:21.485 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000090014 2024-09-19T08:26:21.485 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:21.485 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:21.485 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:21.485 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:21.486 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:21.486 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:21.486 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T08:26:21.486 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 187680 2024-09-19T08:26:21.486 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:21.486 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:26:21.486 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 4, 2024-09-19T08:26:21.486 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 4, 2024-09-19T08:26:21.486 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 3024, 2024-09-19T08:26:21.486 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 448, 2024-09-19T08:26:21.486 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 2, 2024-09-19T08:26:21.486 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 1, 2024-09-19T08:26:21.487 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.002244737, 2024-09-19T08:26:21.487 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.000040598, 2024-09-19T08:26:21.487 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.002164906, 2024-09-19T08:26:21.487 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:26:21.487 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:26:21.487 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3, 2024-09-19T08:26:21.487 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.004007002, 2024-09-19T08:26:21.487 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001335667 2024-09-19T08:26:21.487 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:21.487 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:26:21.487 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:21.487 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:21.487 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:21.488 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:21.488 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T08:26:21.488 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T08:26:21.488 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:21.488 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-09-19T08:26:21.488 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 0, 2024-09-19T08:26:21.488 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-09-19T08:26:21.488 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:21.488 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-09-19T08:26:21.488 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:26:21.488 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-09-19T08:26:21.488 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:26:21.488 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:26:21.489 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:26:21.489 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:26:21.489 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:26:21.489 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:26:21.489 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:26:21.489 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:26:21.489 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:26:21.489 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:26:21.489 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:26:21.489 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:26:21.489 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:26:21.489 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-09-19T08:26:21.490 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:26:21.490 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-09-19T08:26:21.490 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:26:21.490 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:26:21.490 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:26:21.490 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:26:21.490 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:26:21.490 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:26:21.490 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:26:21.490 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-09-19T08:26:21.490 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:26:21.490 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:26:21.490 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:26:21.491 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-09-19T08:26:21.491 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-09-19T08:26:21.491 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-09-19T08:26:21.491 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:26:21.491 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:26:21.491 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:26:21.491 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:26:21.491 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 148967, 2024-09-19T08:26:21.491 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 29, 2024-09-19T08:26:21.491 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:26:21.491 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-09-19T08:26:21.491 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-09-19T08:26:21.491 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-09-19T08:26:21.492 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:26:21.492 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-09-19T08:26:21.492 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:26:21.492 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-09-19T08:26:21.492 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 0, 2024-09-19T08:26:21.492 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 0, 2024-09-19T08:26:21.492 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:26:21.492 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:26:21.492 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-09-19T08:26:21.492 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-09-19T08:26:21.492 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 928, 2024-09-19T08:26:21.492 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 2, 2024-09-19T08:26:21.492 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-09-19T08:26:21.492 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-09-19T08:26:21.493 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-09-19T08:26:21.493 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-09-19T08:26:21.493 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-19T08:26:21.493 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:26:21.493 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-09-19T08:26:21.493 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-09-19T08:26:21.493 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-09-19T08:26:21.493 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 239, 2024-09-19T08:26:21.493 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1557412, 2024-09-19T08:26:21.493 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-09-19T08:26:21.493 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 239, 2024-09-19T08:26:21.493 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-09-19T08:26:21.493 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-09-19T08:26:21.494 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 239, 2024-09-19T08:26:21.494 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1557412, 2024-09-19T08:26:21.494 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-09-19T08:26:21.494 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-09-19T08:26:21.494 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-09-19T08:26:21.494 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-09-19T08:26:21.494 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:21.494 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-19T08:26:21.494 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-09-19T08:26:21.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:21 smithi022 ceph-mon[30963]: pgmap v351: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T08:26:21.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:21 smithi022 ceph-mon[30963]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T08:26:21.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:21 smithi064 ceph-mon[34900]: pgmap v351: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T08:26:21.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:21 smithi064 ceph-mon[34900]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T08:26:22.343 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-19T08:26:22.343 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-19T08:26:22.343 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-19T08:26:22.343 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-19T08:26:22.343 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-19T08:26:22.344 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-09-19T08:26:22.344 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T08:26:22.355 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T08:26:22.355 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:26:22.355 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:26:22.378 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T08:26:22.378 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:26:22.439 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T08:26:22.752 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:26:23.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.266+0000 7f189d2d7640 1 -- 172.21.15.22:0/3331757748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1898106f10 msgr2=0x7f189810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:23.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.266+0000 7f189d2d7640 1 --2- 172.21.15.22:0/3331757748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1898106f10 0x7f189810ea90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f1890009ca0 tx=0x7f189002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:26:23.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.266+0000 7f189d2d7640 1 -- 172.21.15.22:0/3331757748 shutdown_connections 2024-09-19T08:26:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.266+0000 7f189d2d7640 1 --2- 172.21.15.22:0/3331757748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1898106f10 0x7f189810ea90 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.266+0000 7f189d2d7640 1 --2- 172.21.15.22:0/3331757748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1898102790 0x7f18981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.266+0000 7f189d2d7640 1 --2- 172.21.15.22:0/3331757748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1898101de0 0x7f18981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.266+0000 7f189d2d7640 1 -- 172.21.15.22:0/3331757748 >> 172.21.15.22:0/3331757748 conn(0x7f18980fd8c0 msgr2=0x7f18980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.266+0000 7f189d2d7640 1 -- 172.21.15.22:0/3331757748 shutdown_connections 2024-09-19T08:26:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.266+0000 7f189d2d7640 1 -- 172.21.15.22:0/3331757748 wait complete. 2024-09-19T08:26:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.267+0000 7f189d2d7640 1 Processor -- start 2024-09-19T08:26:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.267+0000 7f189d2d7640 1 -- start start 2024-09-19T08:26:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.267+0000 7f189d2d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1898101de0 0x7f18981983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:23.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.267+0000 7f189d2d7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1898102790 0x7f18981988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:23.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.267+0000 7f189d2d7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1898106f10 0x7f189819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:23.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.267+0000 7f189d2d7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1898111fc0 con 0x7f1898101de0 2024-09-19T08:26:23.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.267+0000 7f189d2d7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1898111e40 con 0x7f1898106f10 2024-09-19T08:26:23.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.267+0000 7f189d2d7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1898112140 con 0x7f1898102790 2024-09-19T08:26:23.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.267+0000 7f1896ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1898101de0 0x7f18981983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:23.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.267+0000 7f1896ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1898101de0 0x7f18981983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38574/0 (socket says 172.21.15.22:38574) 2024-09-19T08:26:23.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.267+0000 7f1896ffd640 1 -- 172.21.15.22:0/781901097 learned_addr learned my addr 172.21.15.22:0/781901097 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:26:23.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.268+0000 7f188e7fc640 1 --2- 172.21.15.22:0/781901097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1898102790 0x7f18981988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:23.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.268+0000 7f18977fe640 1 --2- 172.21.15.22:0/781901097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1898106f10 0x7f189819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:23.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.268+0000 7f1896ffd640 1 -- 172.21.15.22:0/781901097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1898102790 msgr2=0x7f18981988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:23.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.268+0000 7f1896ffd640 1 --2- 172.21.15.22:0/781901097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1898102790 0x7f18981988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.268+0000 7f1896ffd640 1 -- 172.21.15.22:0/781901097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1898106f10 msgr2=0x7f189819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.268+0000 7f1896ffd640 1 --2- 172.21.15.22:0/781901097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1898106f10 0x7f189819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.268+0000 7f1896ffd640 1 -- 172.21.15.22:0/781901097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189819d350 con 0x7f1898101de0 2024-09-19T08:26:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.268+0000 7f1896ffd640 1 --2- 172.21.15.22:0/781901097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1898101de0 0x7f18981983a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f188800d950 tx=0x7f188800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.268+0000 7f1894ff9640 1 -- 172.21.15.22:0/781901097 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888017070 con 0x7f1898101de0 2024-09-19T08:26:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.268+0000 7f1894ff9640 1 -- 172.21.15.22:0/781901097 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1888009c30 con 0x7f1898101de0 2024-09-19T08:26:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.268+0000 7f189d2d7640 1 -- 172.21.15.22:0/781901097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189819d640 con 0x7f1898101de0 2024-09-19T08:26:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.268+0000 7f1894ff9640 1 -- 172.21.15.22:0/781901097 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188800f6b0 con 0x7f1898101de0 2024-09-19T08:26:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.268+0000 7f189d2d7640 1 -- 172.21.15.22:0/781901097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18981a4f20 con 0x7f1898101de0 2024-09-19T08:26:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.269+0000 7f189d2d7640 1 -- 172.21.15.22:0/781901097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18981021c0 con 0x7f1898101de0 2024-09-19T08:26:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.270+0000 7f1894ff9640 1 -- 172.21.15.22:0/781901097 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f18880109d0 con 0x7f1898101de0 2024-09-19T08:26:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.271+0000 7f1894ff9640 1 --2- 172.21.15.22:0/781901097 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f18840a3430 0x7f18840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.272+0000 7f188e7fc640 1 --2- 172.21.15.22:0/781901097 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f18840a3430 0x7f18840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:23.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.272+0000 7f1894ff9640 1 -- 172.21.15.22:0/781901097 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f18880cf860 con 0x7f1898101de0 2024-09-19T08:26:23.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.272+0000 7f188e7fc640 1 --2- 172.21.15.22:0/781901097 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f18840a3430 0x7f18840a58f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f18981088e0 tx=0x7f188000b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:23.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.276+0000 7f1894ff9640 1 -- 172.21.15.22:0/781901097 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1888009de0 con 0x7f1898101de0 2024-09-19T08:26:23.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.463+0000 7f189d2d7640 1 -- 172.21.15.22:0/781901097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1898199700 con 0x7f1898101de0 2024-09-19T08:26:23.464 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:23 smithi022 ceph-mon[30963]: pgmap v353: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T08:26:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.464+0000 7f1894ff9640 1 -- 172.21.15.22:0/781901097 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v96) ==== 81+0+942 (secure 0 0 0) 0x7f188809bbe0 con 0x7f1898101de0 2024-09-19T08:26:23.464 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/422860013 2024-09-20T08:15:01.165466+0000 2024-09-19T08:26:23.465 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2837500116 2024-09-20T08:15:01.165466+0000 2024-09-19T08:26:23.465 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1845420423 2024-09-20T08:15:01.165466+0000 2024-09-19T08:26:23.465 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/2990808963 2024-09-20T08:05:42.354701+0000 2024-09-19T08:26:23.465 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3623845502 2024-09-20T08:05:42.354701+0000 2024-09-19T08:26:23.465 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2851750481 2024-09-20T08:06:06.676475+0000 2024-09-19T08:26:23.465 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/4065414102 2024-09-20T08:06:06.676475+0000 2024-09-19T08:26:23.465 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/2990808963 2024-09-20T08:05:42.354701+0000 2024-09-19T08:26:23.465 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2133059277 2024-09-20T08:05:42.354701+0000 2024-09-19T08:26:23.465 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/384027596 2024-09-20T08:06:06.676475+0000 2024-09-19T08:26:23.465 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/4065414102 2024-09-20T08:06:06.676475+0000 2024-09-19T08:26:23.465 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/586163474 2024-09-20T08:05:42.354701+0000 2024-09-19T08:26:23.465 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3712426751 2024-09-20T08:06:06.676475+0000 2024-09-19T08:26:23.465 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/1866276413 2024-09-20T08:15:01.165466+0000 2024-09-19T08:26:23.466 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/1866276413 2024-09-20T08:15:01.165466+0000 2024-09-19T08:26:23.466 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/155709052 2024-09-20T08:15:01.165466+0000 2024-09-19T08:26:23.466 INFO:teuthology.orchestra.run.smithi022.stderr:listed 16 entries 2024-09-19T08:26:23.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.466+0000 7f189d2d7640 1 -- 172.21.15.22:0/781901097 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f18840a3430 msgr2=0x7f18840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.466+0000 7f189d2d7640 1 --2- 172.21.15.22:0/781901097 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f18840a3430 0x7f18840a58f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f18981088e0 tx=0x7f188000b390 comp rx=0 tx=0).stop 2024-09-19T08:26:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.466+0000 7f189d2d7640 1 -- 172.21.15.22:0/781901097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1898101de0 msgr2=0x7f18981983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.466+0000 7f189d2d7640 1 --2- 172.21.15.22:0/781901097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1898101de0 0x7f18981983a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f188800d950 tx=0x7f188800de10 comp rx=0 tx=0).stop 2024-09-19T08:26:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.466+0000 7f189d2d7640 1 -- 172.21.15.22:0/781901097 shutdown_connections 2024-09-19T08:26:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.466+0000 7f189d2d7640 1 --2- 172.21.15.22:0/781901097 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f18840a3430 0x7f18840a58f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.466+0000 7f189d2d7640 1 --2- 172.21.15.22:0/781901097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1898106f10 0x7f189819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.466+0000 7f189d2d7640 1 --2- 172.21.15.22:0/781901097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1898102790 0x7f18981988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.466+0000 7f189d2d7640 1 --2- 172.21.15.22:0/781901097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1898101de0 0x7f18981983a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.466+0000 7f189d2d7640 1 -- 172.21.15.22:0/781901097 >> 172.21.15.22:0/781901097 conn(0x7f18980fd8c0 msgr2=0x7f18980ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.467+0000 7f189d2d7640 1 -- 172.21.15.22:0/781901097 shutdown_connections 2024-09-19T08:26:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:23.467+0000 7f189d2d7640 1 -- 172.21.15.22:0/781901097 wait complete. 2024-09-19T08:26:23.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:23 smithi157 ceph-mon[35143]: pgmap v353: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T08:26:23.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:23 smithi064 ceph-mon[34900]: pgmap v353: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T08:26:24.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:24 smithi022 ceph-mon[30963]: pgmap v354: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T08:26:24.257 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:24 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/781901097' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T08:26:24.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:24 smithi157 ceph-mon[35143]: pgmap v354: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T08:26:24.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:24 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/781901097' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T08:26:24.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:24 smithi064 ceph-mon[34900]: pgmap v354: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T08:26:24.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:24 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/781901097' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T08:26:24.832 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T08:26:24.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:26:24.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-19T08:26:24.890 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-19T08:26:24.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:26:24.890 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:26:24.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:26:24.933 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-19T08:26:25.021 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:26:25.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:26:25.022 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:26:25.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:26:25.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-09-19T08:26:25.201 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T08:26:25.211 INFO:tasks.cephadm:Teardown begin 2024-09-19T08:26:25.212 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:26:25.213 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:26:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:26:26.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.077+0000 7f1627f15640 1 -- 172.21.15.22:0/1328625762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1620102790 msgr2=0x7f16201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:26.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.077+0000 7f1627f15640 1 --2- 172.21.15.22:0/1328625762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1620102790 0x7f16201067e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1614009900 tx=0x7f161402a410 comp rx=0 tx=0).stop 2024-09-19T08:26:26.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.077+0000 7f1627f15640 1 -- 172.21.15.22:0/1328625762 shutdown_connections 2024-09-19T08:26:26.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.077+0000 7f1627f15640 1 --2- 172.21.15.22:0/1328625762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1620106f10 0x7f162010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:26.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.077+0000 7f1627f15640 1 --2- 172.21.15.22:0/1328625762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1620102790 0x7f16201067e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:26.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.077+0000 7f1627f15640 1 --2- 172.21.15.22:0/1328625762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1620101de0 0x7f16201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:26.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.077+0000 7f1627f15640 1 -- 172.21.15.22:0/1328625762 >> 172.21.15.22:0/1328625762 conn(0x7f16200fd8c0 msgr2=0x7f16200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:26.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.077+0000 7f1627f15640 1 -- 172.21.15.22:0/1328625762 shutdown_connections 2024-09-19T08:26:26.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.077+0000 7f1627f15640 1 -- 172.21.15.22:0/1328625762 wait complete. 2024-09-19T08:26:26.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.078+0000 7f1627f15640 1 Processor -- start 2024-09-19T08:26:26.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.078+0000 7f1627f15640 1 -- start start 2024-09-19T08:26:26.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.078+0000 7f1627f15640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1620101de0 0x7f1620198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:26.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.078+0000 7f1627f15640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1620102790 0x7f16201989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:26.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.078+0000 7f1627f15640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1620106f10 0x7f162019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:26.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.078+0000 7f1627f15640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1620111f70 con 0x7f1620106f10 2024-09-19T08:26:26.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.078+0000 7f1627f15640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1620111df0 con 0x7f1620102790 2024-09-19T08:26:26.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.078+0000 7f1627f15640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16201120f0 con 0x7f1620101de0 2024-09-19T08:26:26.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.078+0000 7f16264b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1620106f10 0x7f162019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:26.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.078+0000 7f16264b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1620106f10 0x7f162019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38592/0 (socket says 172.21.15.22:38592) 2024-09-19T08:26:26.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.078+0000 7f16264b2640 1 -- 172.21.15.22:0/3971565181 learned_addr learned my addr 172.21.15.22:0/3971565181 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:26:26.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.078+0000 7f1625cb1640 1 --2- 172.21.15.22:0/3971565181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1620101de0 0x7f1620198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.079+0000 7f16254b0640 1 --2- 172.21.15.22:0/3971565181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1620102790 0x7f16201989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.079+0000 7f16264b2640 1 -- 172.21.15.22:0/3971565181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1620101de0 msgr2=0x7f1620198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.079+0000 7f16264b2640 1 --2- 172.21.15.22:0/3971565181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1620101de0 0x7f1620198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.079+0000 7f16264b2640 1 -- 172.21.15.22:0/3971565181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1620102790 msgr2=0x7f16201989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.079+0000 7f16264b2640 1 --2- 172.21.15.22:0/3971565181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1620102790 0x7f16201989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.079+0000 7f16264b2640 1 -- 172.21.15.22:0/3971565181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162019d440 con 0x7f1620106f10 2024-09-19T08:26:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.079+0000 7f16264b2640 1 --2- 172.21.15.22:0/3971565181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1620106f10 0x7f162019cd60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f161800d750 tx=0x7f161800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.079+0000 7f161effd640 1 -- 172.21.15.22:0/3971565181 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618017070 con 0x7f1620106f10 2024-09-19T08:26:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.079+0000 7f161effd640 1 -- 172.21.15.22:0/3971565181 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1618005500 con 0x7f1620106f10 2024-09-19T08:26:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.079+0000 7f161effd640 1 -- 172.21.15.22:0/3971565181 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618004d90 con 0x7f1620106f10 2024-09-19T08:26:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.079+0000 7f1627f15640 1 -- 172.21.15.22:0/3971565181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162019d730 con 0x7f1620106f10 2024-09-19T08:26:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.079+0000 7f1627f15640 1 -- 172.21.15.22:0/3971565181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16201a5010 con 0x7f1620106f10 2024-09-19T08:26:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.080+0000 7f1627f15640 1 -- 172.21.15.22:0/3971565181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16201021c0 con 0x7f1620106f10 2024-09-19T08:26:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.081+0000 7f161effd640 1 -- 172.21.15.22:0/3971565181 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1618004f30 con 0x7f1620106f10 2024-09-19T08:26:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.082+0000 7f161effd640 1 --2- 172.21.15.22:0/3971565181 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16080a33b0 0x7f16080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.082+0000 7f161effd640 1 -- 172.21.15.22:0/3971565181 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f16180cf9a0 con 0x7f1620106f10 2024-09-19T08:26:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.082+0000 7f1625cb1640 1 --2- 172.21.15.22:0/3971565181 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16080a33b0 0x7f16080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.083+0000 7f1625cb1640 1 --2- 172.21.15.22:0/3971565181 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16080a33b0 0x7f16080a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1610005fd0 tx=0x7f1610009680 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.086+0000 7f161effd640 1 -- 172.21.15.22:0/3971565181 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f161809bd20 con 0x7f1620106f10 2024-09-19T08:26:26.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.297+0000 7f1627f15640 1 -- 172.21.15.22:0/3971565181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f162010a0f0 con 0x7f1620106f10 2024-09-19T08:26:26.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.298+0000 7f161effd640 1 -- 172.21.15.22:0/3971565181 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11909 (secure 0 0 0) 0x7f16180a0020 con 0x7f1620106f10 2024-09-19T08:26:26.298 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:26:26.298 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":36,"btime":"2024-09-19T08:24:38:729727+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:24:38.451170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:26:26.299 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 36 2024-09-19T08:26:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.300+0000 7f1627f15640 1 -- 172.21.15.22:0/3971565181 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16080a33b0 msgr2=0x7f16080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.300+0000 7f1627f15640 1 --2- 172.21.15.22:0/3971565181 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16080a33b0 0x7f16080a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1610005fd0 tx=0x7f1610009680 comp rx=0 tx=0).stop 2024-09-19T08:26:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.300+0000 7f1627f15640 1 -- 172.21.15.22:0/3971565181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1620106f10 msgr2=0x7f162019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.300+0000 7f1627f15640 1 --2- 172.21.15.22:0/3971565181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1620106f10 0x7f162019cd60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f161800d750 tx=0x7f161800dc10 comp rx=0 tx=0).stop 2024-09-19T08:26:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.300+0000 7f1627f15640 1 -- 172.21.15.22:0/3971565181 shutdown_connections 2024-09-19T08:26:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.300+0000 7f1627f15640 1 --2- 172.21.15.22:0/3971565181 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f16080a33b0 0x7f16080a5870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.300+0000 7f1627f15640 1 --2- 172.21.15.22:0/3971565181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1620106f10 0x7f162019cd60 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.300+0000 7f1627f15640 1 --2- 172.21.15.22:0/3971565181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1620102790 0x7f16201989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.300+0000 7f1627f15640 1 --2- 172.21.15.22:0/3971565181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1620101de0 0x7f1620198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.300+0000 7f1627f15640 1 -- 172.21.15.22:0/3971565181 >> 172.21.15.22:0/3971565181 conn(0x7f16200fd8c0 msgr2=0x7f16200fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:26.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.301+0000 7f1627f15640 1 -- 172.21.15.22:0/3971565181 shutdown_connections 2024-09-19T08:26:26.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:26.301+0000 7f1627f15640 1 -- 172.21.15.22:0/3971565181 wait complete. 2024-09-19T08:26:26.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:26 smithi022 ceph-mon[30963]: pgmap v355: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T08:26:26.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:26 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/3971565181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:26:26.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:26 smithi157 ceph-mon[35143]: pgmap v355: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T08:26:26.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:26 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/3971565181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:26:26.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:26 smithi064 ceph-mon[34900]: pgmap v355: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T08:26:26.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:26 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/3971565181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:26:27.005 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:26:27.274 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:26:27.719 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:27 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:26:27.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:27 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:26:27.836 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:27 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:26:27.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.803+0000 7f6456199640 1 -- 172.21.15.22:0/634938212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6450102570 msgr2=0x7f64501029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:27.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.803+0000 7f6456199640 1 --2- 172.21.15.22:0/634938212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6450102570 0x7f64501029b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f6444009900 tx=0x7f644402a480 comp rx=0 tx=0).stop 2024-09-19T08:26:27.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.803+0000 7f6456199640 1 -- 172.21.15.22:0/634938212 shutdown_connections 2024-09-19T08:26:27.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.803+0000 7f6456199640 1 --2- 172.21.15.22:0/634938212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6450102ef0 0x7f645010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:27.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.803+0000 7f6456199640 1 --2- 172.21.15.22:0/634938212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6450102570 0x7f64501029b0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:27.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.803+0000 7f6456199640 1 --2- 172.21.15.22:0/634938212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6450100e20 0x7f6450101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:27.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.803+0000 7f6456199640 1 -- 172.21.15.22:0/634938212 >> 172.21.15.22:0/634938212 conn(0x7f64500fc7d0 msgr2=0x7f64500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:27.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.803+0000 7f6456199640 1 -- 172.21.15.22:0/634938212 shutdown_connections 2024-09-19T08:26:27.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.803+0000 7f6456199640 1 -- 172.21.15.22:0/634938212 wait complete. 2024-09-19T08:26:27.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.803+0000 7f6456199640 1 Processor -- start 2024-09-19T08:26:27.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f6456199640 1 -- start start 2024-09-19T08:26:27.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f6456199640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6450100e20 0x7f64500fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:27.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f6456199640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6450102570 0x7f64500fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:27.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f6456199640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6450102ef0 0x7f64500fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:27.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f6456199640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6450110f80 con 0x7f6450102570 2024-09-19T08:26:27.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f6456199640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6450110e00 con 0x7f6450100e20 2024-09-19T08:26:27.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f6456199640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6450111100 con 0x7f6450102ef0 2024-09-19T08:26:27.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f644effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6450102570 0x7f64500fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:27.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f644effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6450102570 0x7f64500fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57704/0 (socket says 172.21.15.22:57704) 2024-09-19T08:26:27.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f644f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6450100e20 0x7f64500fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:27.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f644effd640 1 -- 172.21.15.22:0/970126630 learned_addr learned my addr 172.21.15.22:0/970126630 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:26:27.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f644ffff640 1 --2- 172.21.15.22:0/970126630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6450102ef0 0x7f64500fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:27.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f644f7fe640 1 -- 172.21.15.22:0/970126630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6450102ef0 msgr2=0x7f64500fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:27.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f644f7fe640 1 --2- 172.21.15.22:0/970126630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6450102ef0 0x7f64500fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:27.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f644f7fe640 1 -- 172.21.15.22:0/970126630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6450102570 msgr2=0x7f64500fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:27.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f644f7fe640 1 --2- 172.21.15.22:0/970126630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6450102570 0x7f64500fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:27.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.804+0000 7f644f7fe640 1 -- 172.21.15.22:0/970126630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64501a6a20 con 0x7f6450100e20 2024-09-19T08:26:27.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.805+0000 7f644f7fe640 1 --2- 172.21.15.22:0/970126630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6450100e20 0x7f64500fcc80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f644000d720 tx=0x7f644000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:27.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.805+0000 7f644cff9640 1 -- 172.21.15.22:0/970126630 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440017070 con 0x7f6450100e20 2024-09-19T08:26:27.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.805+0000 7f644cff9640 1 -- 172.21.15.22:0/970126630 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644000de30 con 0x7f6450100e20 2024-09-19T08:26:27.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.805+0000 7f6456199640 1 -- 172.21.15.22:0/970126630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64501a6d10 con 0x7f6450100e20 2024-09-19T08:26:27.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.805+0000 7f644cff9640 1 -- 172.21.15.22:0/970126630 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64400047a0 con 0x7f6450100e20 2024-09-19T08:26:27.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.805+0000 7f6456199640 1 -- 172.21.15.22:0/970126630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64501a71a0 con 0x7f6450100e20 2024-09-19T08:26:27.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.806+0000 7f6456199640 1 -- 172.21.15.22:0/970126630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6450101200 con 0x7f6450100e20 2024-09-19T08:26:27.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.807+0000 7f644cff9640 1 -- 172.21.15.22:0/970126630 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f64400099d0 con 0x7f6450100e20 2024-09-19T08:26:27.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.808+0000 7f644cff9640 1 --2- 172.21.15.22:0/970126630 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f64380a33b0 0x7f64380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:27.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.808+0000 7f644cff9640 1 -- 172.21.15.22:0/970126630 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f64400cf9f0 con 0x7f6450100e20 2024-09-19T08:26:27.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.808+0000 7f644effd640 1 --2- 172.21.15.22:0/970126630 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f64380a33b0 0x7f64380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:27.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.808+0000 7f644effd640 1 --2- 172.21.15.22:0/970126630 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f64380a33b0 0x7f64380a5870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6444005770 tx=0x7f6444005a20 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:27.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:27.812+0000 7f644cff9640 1 -- 172.21.15.22:0/970126630 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6440020020 con 0x7f6450100e20 2024-09-19T08:26:28.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:28.030+0000 7f6456199640 1 -- 172.21.15.22:0/970126630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f645010a0a0 con 0x7f6450100e20 2024-09-19T08:26:28.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:28.033+0000 7f644cff9640 1 -- 172.21.15.22:0/970126630 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11909 (secure 0 0 0) 0x7f644009c360 con 0x7f6450100e20 2024-09-19T08:26:28.033 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:26:28.033 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":36,"btime":"2024-09-19T08:24:38:729727+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3714062414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3714062414},{"type":"v1","addr":"172.21.15.22:6835","nonce":3714062414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3668291316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3668291316},{"type":"v1","addr":"172.21.15.22:6837","nonce":3668291316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3762277518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3762277518},{"type":"v1","addr":"172.21.15.22:6839","nonce":3762277518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1594373985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1594373985},{"type":"v1","addr":"172.21.15.22:6841","nonce":1594373985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/138036781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":138036781},{"type":"v1","addr":"172.21.15.64:6833","nonce":138036781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2040635685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2040635685},{"type":"v1","addr":"172.21.15.157:6833","nonce":2040635685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:13:01.503449+0000","modified":"2024-09-19T08:24:38.451170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24479,"mds_2":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/3333087560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":3333087560},{"type":"v1","addr":"172.21.15.64:6835","nonce":3333087560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/4158350679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4158350679},{"type":"v1","addr":"172.21.15.157:6835","nonce":4158350679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6837/2142121431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2142121431},{"type":"v1","addr":"172.21.15.64:6837","nonce":2142121431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6837/892624399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":892624399},{"type":"v1","addr":"172.21.15.157:6837","nonce":892624399}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6839/358484749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":358484749},{"type":"v1","addr":"172.21.15.64:6839","nonce":358484749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.157:6839/3802797238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":3802797238},{"type":"v1","addr":"172.21.15.157:6839","nonce":3802797238}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24460,24479,24481]},"id":1}]} 2024-09-19T08:26:28.034 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 36 2024-09-19T08:26:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:28.035+0000 7f6456199640 1 -- 172.21.15.22:0/970126630 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f64380a33b0 msgr2=0x7f64380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:28.035+0000 7f6456199640 1 --2- 172.21.15.22:0/970126630 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f64380a33b0 0x7f64380a5870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6444005770 tx=0x7f6444005a20 comp rx=0 tx=0).stop 2024-09-19T08:26:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:28.035+0000 7f6456199640 1 -- 172.21.15.22:0/970126630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6450100e20 msgr2=0x7f64500fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:28.035+0000 7f6456199640 1 --2- 172.21.15.22:0/970126630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6450100e20 0x7f64500fcc80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f644000d720 tx=0x7f644000dbe0 comp rx=0 tx=0).stop 2024-09-19T08:26:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:28.036+0000 7f6456199640 1 -- 172.21.15.22:0/970126630 shutdown_connections 2024-09-19T08:26:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:28.036+0000 7f6456199640 1 --2- 172.21.15.22:0/970126630 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f64380a33b0 0x7f64380a5870 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:28.036+0000 7f6456199640 1 --2- 172.21.15.22:0/970126630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6450102ef0 0x7f64500fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:28.036+0000 7f6456199640 1 --2- 172.21.15.22:0/970126630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6450102570 0x7f64500fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:28.036+0000 7f6456199640 1 --2- 172.21.15.22:0/970126630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6450100e20 0x7f64500fcc80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:28.036+0000 7f6456199640 1 -- 172.21.15.22:0/970126630 >> 172.21.15.22:0/970126630 conn(0x7f64500fc7d0 msgr2=0x7f645006a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:28.036+0000 7f6456199640 1 -- 172.21.15.22:0/970126630 shutdown_connections 2024-09-19T08:26:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:28.036+0000 7f6456199640 1 -- 172.21.15.22:0/970126630 wait complete. 2024-09-19T08:26:28.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:26:28.938 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:26:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:28 smithi157 ceph-mon[35143]: pgmap v356: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 614 B/s wr, 6 op/s 2024-09-19T08:26:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:28 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:26:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:28 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:26:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:28 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:26:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:28 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:26:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:28 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/970126630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:26:29.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:28 smithi157 ceph-mon[35143]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:26:29.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:28 smithi022 ceph-mon[30963]: pgmap v356: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 614 B/s wr, 6 op/s 2024-09-19T08:26:29.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:28 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:26:29.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:28 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:26:29.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:28 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:26:29.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:28 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:26:29.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:28 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/970126630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:26:29.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:28 smithi022 ceph-mon[30963]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:26:29.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:28 smithi064 ceph-mon[34900]: pgmap v356: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 614 B/s wr, 6 op/s 2024-09-19T08:26:29.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:28 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:26:29.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:28 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:26:29.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:28 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:26:29.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:28 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:26:29.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:28 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/970126630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:26:29.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:28 smithi064 ceph-mon[34900]: from='mgr.15141 ' entity='mgr.x' 2024-09-19T08:26:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.461+0000 7fc3b2c13640 1 -- 172.21.15.22:0/1073888825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ac101420 msgr2=0x7fc3ac10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.461+0000 7fc3b2c13640 1 --2- 172.21.15.22:0/1073888825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ac101420 0x7fc3ac10d950 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009cd0 tx=0x7fc3a402a8e0 comp rx=0 tx=0).stop 2024-09-19T08:26:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.461+0000 7fc3b2c13640 1 -- 172.21.15.22:0/1073888825 shutdown_connections 2024-09-19T08:26:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.461+0000 7fc3b2c13640 1 --2- 172.21.15.22:0/1073888825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ac101420 0x7fc3ac10d950 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.461+0000 7fc3b2c13640 1 --2- 172.21.15.22:0/1073888825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3ac100a80 0x7fc3ac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.461+0000 7fc3b2c13640 1 --2- 172.21.15.22:0/1073888825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac106a80 0x7fc3ac106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.461+0000 7fc3b2c13640 1 -- 172.21.15.22:0/1073888825 >> 172.21.15.22:0/1073888825 conn(0x7fc3ac0fc7a0 msgr2=0x7fc3ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.461+0000 7fc3b2c13640 1 -- 172.21.15.22:0/1073888825 shutdown_connections 2024-09-19T08:26:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.461+0000 7fc3b2c13640 1 -- 172.21.15.22:0/1073888825 wait complete. 2024-09-19T08:26:29.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3b2c13640 1 Processor -- start 2024-09-19T08:26:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3b2c13640 1 -- start start 2024-09-19T08:26:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3b2c13640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3ac100a80 0x7fc3ac0fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3b2c13640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ac101420 0x7fc3ac0fd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3b2c13640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac106a80 0x7fc3ac0fd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3b2c13640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac110e80 con 0x7fc3ac101420 2024-09-19T08:26:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3b2c13640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac110d00 con 0x7fc3ac100a80 2024-09-19T08:26:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3b2c13640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac111000 con 0x7fc3ac106a80 2024-09-19T08:26:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3abfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ac101420 0x7fc3ac0fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3abfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ac101420 0x7fc3ac0fd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57730/0 (socket says 172.21.15.22:57730) 2024-09-19T08:26:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3abfff640 1 -- 172.21.15.22:0/749374485 learned_addr learned my addr 172.21.15.22:0/749374485 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:26:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3b11b0640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac106a80 0x7fc3ac0fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3b09af640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3ac100a80 0x7fc3ac0fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3abfff640 1 -- 172.21.15.22:0/749374485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac106a80 msgr2=0x7fc3ac0fd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3abfff640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac106a80 0x7fc3ac0fd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3abfff640 1 -- 172.21.15.22:0/749374485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3ac100a80 msgr2=0x7fc3ac0fcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3abfff640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3ac100a80 0x7fc3ac0fcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3abfff640 1 -- 172.21.15.22:0/749374485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac1a68f0 con 0x7fc3ac101420 2024-09-19T08:26:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3b09af640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3ac100a80 0x7fc3ac0fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:26:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.462+0000 7fc3abfff640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ac101420 0x7fc3ac0fd250 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc39400c920 tx=0x7fc39400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.463+0000 7fc3a9ffb640 1 -- 172.21.15.22:0/749374485 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394007c70 con 0x7fc3ac101420 2024-09-19T08:26:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.463+0000 7fc3a9ffb640 1 -- 172.21.15.22:0/749374485 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc394007e10 con 0x7fc3ac101420 2024-09-19T08:26:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.463+0000 7fc3b2c13640 1 -- 172.21.15.22:0/749374485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ac1a6be0 con 0x7fc3ac101420 2024-09-19T08:26:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.463+0000 7fc3a9ffb640 1 -- 172.21.15.22:0/749374485 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394018890 con 0x7fc3ac101420 2024-09-19T08:26:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.463+0000 7fc3b2c13640 1 -- 172.21.15.22:0/749374485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ac1a7120 con 0x7fc3ac101420 2024-09-19T08:26:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.464+0000 7fc3b2c13640 1 -- 172.21.15.22:0/749374485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ac100ee0 con 0x7fc3ac101420 2024-09-19T08:26:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.465+0000 7fc3a9ffb640 1 -- 172.21.15.22:0/749374485 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc394018a70 con 0x7fc3ac101420 2024-09-19T08:26:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.466+0000 7fc3a9ffb640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc3980a33b0 0x7fc3980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:29.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.466+0000 7fc3b09af640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc3980a33b0 0x7fc3980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:29.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.466+0000 7fc3a9ffb640 1 -- 172.21.15.22:0/749374485 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fc3940cff90 con 0x7fc3ac101420 2024-09-19T08:26:29.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.466+0000 7fc3b09af640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc3980a33b0 0x7fc3980a5870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc39c005fd0 tx=0x7fc39c0094d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:29.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.470+0000 7fc3a9ffb640 1 -- 172.21.15.22:0/749374485 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc394014030 con 0x7fc3ac101420 2024-09-19T08:26:29.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.648+0000 7fc3b2c13640 1 -- 172.21.15.22:0/749374485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3ac10a000 con 0x7fc3ac101420 2024-09-19T08:26:29.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.649+0000 7fc3a9ffb640 1 -- 172.21.15.22:0/749374485 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+23190 (secure 0 0 0) 0x7fc39400de20 con 0x7fc3ac101420 2024-09-19T08:26:29.649 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T08:26:29.650 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":96,"fsid":"c0ad60c6-765d-11ef-bceb-c7b262605968","created":"2024-09-19T08:05:19.476952+0000","modified":"2024-09-19T08:26:20.507659+0000","last_up_change":"2024-09-19T08:12:10.339586+0000","last_in_change":"2024-09-19T08:11:47.653576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:08:25.175759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:12:57.608787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:12:59.824327+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ed67f2d-9b28-427a-b69f-676e7a2abcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6803","nonce":3442715307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6805","nonce":3442715307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6809","nonce":3442715307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3442715307},{"type":"v1","addr":"172.21.15.22:6807","nonce":3442715307}]},"public_addr":"172.21.15.22:6803/3442715307","cluster_addr":"172.21.15.22:6805/3442715307","heartbeat_back_addr":"172.21.15.22:6809/3442715307","heartbeat_front_addr":"172.21.15.22:6807/3442715307","state":["exists","up"]},{"osd":1,"uuid":"bff3e283-4ab1-49d7-86d8-45d361995122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6801","nonce":2379700228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6803","nonce":2379700228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6807","nonce":2379700228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2379700228},{"type":"v1","addr":"172.21.15.64:6805","nonce":2379700228}]},"public_addr":"172.21.15.64:6801/2379700228","cluster_addr":"172.21.15.64:6803/2379700228","heartbeat_back_addr":"172.21.15.64:6807/2379700228","heartbeat_front_addr":"172.21.15.64:6805/2379700228","state":["exists","up"]},{"osd":2,"uuid":"81421d08-7dc6-419b-a566-dee1e1e13b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6801","nonce":3363263867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6803","nonce":3363263867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6807","nonce":3363263867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3363263867},{"type":"v1","addr":"172.21.15.157:6805","nonce":3363263867}]},"public_addr":"172.21.15.157:6801/3363263867","cluster_addr":"172.21.15.157:6803/3363263867","heartbeat_back_addr":"172.21.15.157:6807/3363263867","heartbeat_front_addr":"172.21.15.157:6805/3363263867","state":["exists","up"]},{"osd":3,"uuid":"34debfd0-3226-4d78-b9e6-e03dbc5190f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6811","nonce":4225229456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6813","nonce":4225229456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6817","nonce":4225229456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":4225229456},{"type":"v1","addr":"172.21.15.22:6815","nonce":4225229456}]},"public_addr":"172.21.15.22:6811/4225229456","cluster_addr":"172.21.15.22:6813/4225229456","heartbeat_back_addr":"172.21.15.22:6817/4225229456","heartbeat_front_addr":"172.21.15.22:6815/4225229456","state":["exists","up"]},{"osd":4,"uuid":"1293332c-d772-4ca9-bf76-79e0b29451ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6809","nonce":1858090423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6811","nonce":1858090423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6815","nonce":1858090423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1858090423},{"type":"v1","addr":"172.21.15.64:6813","nonce":1858090423}]},"public_addr":"172.21.15.64:6809/1858090423","cluster_addr":"172.21.15.64:6811/1858090423","heartbeat_back_addr":"172.21.15.64:6815/1858090423","heartbeat_front_addr":"172.21.15.64:6813/1858090423","state":["exists","up"]},{"osd":5,"uuid":"240a6c42-89a2-41c1-948c-b09914b5722e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6809","nonce":2900701212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2900701212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2900701212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2900701212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2900701212}]},"public_addr":"172.21.15.157:6809/2900701212","cluster_addr":"172.21.15.157:6811/2900701212","heartbeat_back_addr":"172.21.15.157:6815/2900701212","heartbeat_front_addr":"172.21.15.157:6813/2900701212","state":["exists","up"]},{"osd":6,"uuid":"80199030-109e-41c4-bc1c-731755464770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6819","nonce":4264426242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6821","nonce":4264426242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6825","nonce":4264426242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":4264426242},{"type":"v1","addr":"172.21.15.22:6823","nonce":4264426242}]},"public_addr":"172.21.15.22:6819/4264426242","cluster_addr":"172.21.15.22:6821/4264426242","heartbeat_back_addr":"172.21.15.22:6825/4264426242","heartbeat_front_addr":"172.21.15.22:6823/4264426242","state":["exists","up"]},{"osd":7,"uuid":"dc0bb62d-1bea-40c8-b3a5-eee9f7d026bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6817","nonce":3980224103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6819","nonce":3980224103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6823","nonce":3980224103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3980224103},{"type":"v1","addr":"172.21.15.64:6821","nonce":3980224103}]},"public_addr":"172.21.15.64:6817/3980224103","cluster_addr":"172.21.15.64:6819/3980224103","heartbeat_back_addr":"172.21.15.64:6823/3980224103","heartbeat_front_addr":"172.21.15.64:6821/3980224103","state":["exists","up"]},{"osd":8,"uuid":"764b1e6e-48c9-49f9-8124-bc13bcba3a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6817","nonce":973678477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6819","nonce":973678477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6823","nonce":973678477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":973678477},{"type":"v1","addr":"172.21.15.157:6821","nonce":973678477}]},"public_addr":"172.21.15.157:6817/973678477","cluster_addr":"172.21.15.157:6819/973678477","heartbeat_back_addr":"172.21.15.157:6823/973678477","heartbeat_front_addr":"172.21.15.157:6821/973678477","state":["exists","up"]},{"osd":9,"uuid":"82914818-df47-4374-9e3e-2f71fc76e78b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6827","nonce":710033990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6829","nonce":710033990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6833","nonce":710033990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":710033990},{"type":"v1","addr":"172.21.15.22:6831","nonce":710033990}]},"public_addr":"172.21.15.22:6827/710033990","cluster_addr":"172.21.15.22:6829/710033990","heartbeat_back_addr":"172.21.15.22:6833/710033990","heartbeat_front_addr":"172.21.15.22:6831/710033990","state":["exists","up"]},{"osd":10,"uuid":"36ae177d-c454-4cba-ba21-cbeee0f10dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6825","nonce":373412752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6827","nonce":373412752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6831","nonce":373412752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":373412752},{"type":"v1","addr":"172.21.15.64:6829","nonce":373412752}]},"public_addr":"172.21.15.64:6825/373412752","cluster_addr":"172.21.15.64:6827/373412752","heartbeat_back_addr":"172.21.15.64:6831/373412752","heartbeat_front_addr":"172.21.15.64:6829/373412752","state":["exists","up"]},{"osd":11,"uuid":"cb50a41a-39ac-4d85-9a81-14ebd4d3b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6825","nonce":1918889194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6827","nonce":1918889194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6831","nonce":1918889194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1918889194},{"type":"v1","addr":"172.21.15.157:6829","nonce":1918889194}]},"public_addr":"172.21.15.157:6825/1918889194","cluster_addr":"172.21.15.157:6827/1918889194","heartbeat_back_addr":"172.21.15.157:6831/1918889194","heartbeat_front_addr":"172.21.15.157:6829/1918889194","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:31.378218+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:07:56.794167+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:20.790267+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:08:47.039106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:11.483842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:09:38.151466+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:03.553308+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:27.865321+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:10:53.086530+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:18.946661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:11:43.697243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:12:08.936588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/422860013":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/2837500116":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/1845420423":"2024-09-20T08:15:01.165466+0000","172.21.15.22:6800/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/3623845502":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2851750481":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6801/2990808963":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/2133059277":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/384027596":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/4065414102":"2024-09-20T08:06:06.676475+0000","172.21.15.22:0/586163474":"2024-09-20T08:05:42.354701+0000","172.21.15.22:0/3712426751":"2024-09-20T08:06:06.676475+0000","172.21.15.22:6800/1866276413":"2024-09-20T08:15:01.165466+0000","172.21.15.22:6801/1866276413":"2024-09-20T08:15:01.165466+0000","172.21.15.22:0/155709052":"2024-09-20T08:15:01.165466+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:26:29.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.652+0000 7fc3b2c13640 1 -- 172.21.15.22:0/749374485 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc3980a33b0 msgr2=0x7fc3980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:29.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.652+0000 7fc3b2c13640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc3980a33b0 0x7fc3980a5870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc39c005fd0 tx=0x7fc39c0094d0 comp rx=0 tx=0).stop 2024-09-19T08:26:29.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.652+0000 7fc3b2c13640 1 -- 172.21.15.22:0/749374485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ac101420 msgr2=0x7fc3ac0fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:29.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.652+0000 7fc3b2c13640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ac101420 0x7fc3ac0fd250 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc39400c920 tx=0x7fc39400cde0 comp rx=0 tx=0).stop 2024-09-19T08:26:29.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.652+0000 7fc3b2c13640 1 -- 172.21.15.22:0/749374485 shutdown_connections 2024-09-19T08:26:29.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.652+0000 7fc3b2c13640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fc3980a33b0 0x7fc3980a5870 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:29.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.652+0000 7fc3b2c13640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ac106a80 0x7fc3ac0fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:29.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.652+0000 7fc3b2c13640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ac101420 0x7fc3ac0fd250 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:29.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.652+0000 7fc3b2c13640 1 --2- 172.21.15.22:0/749374485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3ac100a80 0x7fc3ac0fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:29.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.652+0000 7fc3b2c13640 1 -- 172.21.15.22:0/749374485 >> 172.21.15.22:0/749374485 conn(0x7fc3ac0fc7a0 msgr2=0x7fc3ac06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:29.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.652+0000 7fc3b2c13640 1 -- 172.21.15.22:0/749374485 shutdown_connections 2024-09-19T08:26:29.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:29.652+0000 7fc3b2c13640 1 -- 172.21.15.22:0/749374485 wait complete. 2024-09-19T08:26:30.379 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T08:26:30.642 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:26:30.905 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:30 smithi022 ceph-mon[30963]: pgmap v357: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-19T08:26:30.905 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:30 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/749374485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:26:31.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:30 smithi157 ceph-mon[35143]: pgmap v357: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-19T08:26:31.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:30 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/749374485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:26:31.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:30 smithi064 ceph-mon[34900]: pgmap v357: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-19T08:26:31.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:30 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/749374485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:26:31.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.129+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/676786983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aac102df0 msgr2=0x7f5aac1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:31.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.129+0000 7f5ab2c7d640 1 --2- 172.21.15.22:0/676786983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aac102df0 0x7f5aac1031d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009900 tx=0x7f5aa802a410 comp rx=0 tx=0).stop 2024-09-19T08:26:31.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.129+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/676786983 shutdown_connections 2024-09-19T08:26:31.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.129+0000 7f5ab2c7d640 1 --2- 172.21.15.22:0/676786983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aac10c2f0 0x7f5aac10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:31.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.129+0000 7f5ab2c7d640 1 --2- 172.21.15.22:0/676786983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aac1037a0 0x7f5aac10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:31.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.129+0000 7f5ab2c7d640 1 --2- 172.21.15.22:0/676786983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aac102df0 0x7f5aac1031d0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:31.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.129+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/676786983 >> 172.21.15.22:0/676786983 conn(0x7f5aac0fc7d0 msgr2=0x7f5aac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:31.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.129+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/676786983 shutdown_connections 2024-09-19T08:26:31.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.129+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/676786983 wait complete. 2024-09-19T08:26:31.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.129+0000 7f5ab2c7d640 1 Processor -- start 2024-09-19T08:26:31.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.129+0000 7f5ab2c7d640 1 -- start start 2024-09-19T08:26:31.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab2c7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aac102df0 0x7f5aac198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:31.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab2c7d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aac1037a0 0x7f5aac198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:31.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab2c7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aac10c2f0 0x7f5aac19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:31.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab2c7d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac112170 con 0x7f5aac10c2f0 2024-09-19T08:26:31.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab2c7d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac111ff0 con 0x7f5aac1037a0 2024-09-19T08:26:31.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab2c7d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1122f0 con 0x7f5aac102df0 2024-09-19T08:26:31.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab121a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aac10c2f0 0x7f5aac19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:31.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab121a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aac10c2f0 0x7f5aac19cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57752/0 (socket says 172.21.15.22:57752) 2024-09-19T08:26:31.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab121a640 1 -- 172.21.15.22:0/1698801848 learned_addr learned my addr 172.21.15.22:0/1698801848 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:26:31.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab0a19640 1 --2- 172.21.15.22:0/1698801848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aac102df0 0x7f5aac198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:31.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5aa3fff640 1 --2- 172.21.15.22:0/1698801848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aac1037a0 0x7f5aac198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:31.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab121a640 1 -- 172.21.15.22:0/1698801848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aac102df0 msgr2=0x7f5aac198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:31.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab121a640 1 --2- 172.21.15.22:0/1698801848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aac102df0 0x7f5aac198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:31.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab121a640 1 -- 172.21.15.22:0/1698801848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aac1037a0 msgr2=0x7f5aac198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:31.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab121a640 1 --2- 172.21.15.22:0/1698801848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aac1037a0 0x7f5aac198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:31.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab121a640 1 -- 172.21.15.22:0/1698801848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac19d5f0 con 0x7f5aac10c2f0 2024-09-19T08:26:31.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab121a640 1 --2- 172.21.15.22:0/1698801848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aac10c2f0 0x7f5aac19cf10 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400b970 tx=0x7f5aa400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:31.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5aa1ffb640 1 -- 172.21.15.22:0/1698801848 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4019070 con 0x7f5aac10c2f0 2024-09-19T08:26:31.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5aa1ffb640 1 -- 172.21.15.22:0/1698801848 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa400cb40 con 0x7f5aac10c2f0 2024-09-19T08:26:31.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5aa1ffb640 1 -- 172.21.15.22:0/1698801848 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4007650 con 0x7f5aac10c2f0 2024-09-19T08:26:31.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.130+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/1698801848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aac19d8e0 con 0x7f5aac10c2f0 2024-09-19T08:26:31.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.131+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/1698801848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aac1a51c0 con 0x7f5aac10c2f0 2024-09-19T08:26:31.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.132+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/1698801848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aac06a790 con 0x7f5aac10c2f0 2024-09-19T08:26:31.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.133+0000 7f5aa1ffb640 1 -- 172.21.15.22:0/1698801848 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5aa401c070 con 0x7f5aac10c2f0 2024-09-19T08:26:31.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.134+0000 7f5aa1ffb640 1 --2- 172.21.15.22:0/1698801848 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5a940a3430 0x7f5a940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:31.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.134+0000 7f5aa1ffb640 1 -- 172.21.15.22:0/1698801848 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f5aa40cf670 con 0x7f5aac10c2f0 2024-09-19T08:26:31.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.134+0000 7f5ab0a19640 1 --2- 172.21.15.22:0/1698801848 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5a940a3430 0x7f5a940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:31.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.134+0000 7f5ab0a19640 1 --2- 172.21.15.22:0/1698801848 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5a940a3430 0x7f5a940a58f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8000c00 tx=0x7f5aa80058d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:31.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.137+0000 7f5aa1ffb640 1 -- 172.21.15.22:0/1698801848 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5aa409baa0 con 0x7f5aac10c2f0 2024-09-19T08:26:31.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.348+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/1698801848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5aac1031d0 con 0x7f5aac10c2f0 2024-09-19T08:26:31.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.350+0000 7f5aa1ffb640 1 -- 172.21.15.22:0/1698801848 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7f5aac1031d0 con 0x7f5aac10c2f0 2024-09-19T08:26:31.350 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T08:26:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.353+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/1698801848 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5a940a3430 msgr2=0x7f5a940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.353+0000 7f5ab2c7d640 1 --2- 172.21.15.22:0/1698801848 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5a940a3430 0x7f5a940a58f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8000c00 tx=0x7f5aa80058d0 comp rx=0 tx=0).stop 2024-09-19T08:26:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.353+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/1698801848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aac10c2f0 msgr2=0x7f5aac19cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.353+0000 7f5ab2c7d640 1 --2- 172.21.15.22:0/1698801848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aac10c2f0 0x7f5aac19cf10 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400b970 tx=0x7f5aa400be30 comp rx=0 tx=0).stop 2024-09-19T08:26:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.353+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/1698801848 shutdown_connections 2024-09-19T08:26:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.353+0000 7f5ab2c7d640 1 --2- 172.21.15.22:0/1698801848 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f5a940a3430 0x7f5a940a58f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.353+0000 7f5ab2c7d640 1 --2- 172.21.15.22:0/1698801848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aac10c2f0 0x7f5aac19cf10 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.353+0000 7f5ab2c7d640 1 --2- 172.21.15.22:0/1698801848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aac1037a0 0x7f5aac198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.353+0000 7f5ab2c7d640 1 --2- 172.21.15.22:0/1698801848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aac102df0 0x7f5aac198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.353+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/1698801848 >> 172.21.15.22:0/1698801848 conn(0x7f5aac0fc7d0 msgr2=0x7f5aac0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.353+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/1698801848 shutdown_connections 2024-09-19T08:26:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:31.353+0000 7f5ab2c7d640 1 -- 172.21.15.22:0/1698801848 wait complete. 2024-09-19T08:26:31.640 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:31 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a[30959]: 2024-09-19T08:26:31.348+0000 7f0bc3d62640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:26:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1698801848' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T08:26:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-mon[35143]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T08:26:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-mon[35143]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:26:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1698801848' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T08:26:31.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-mon[35143]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T08:26:31.754 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-mon[35143]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T08:26:31.754 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-f[65067]: ignoring --setuser ceph since I am not root 2024-09-19T08:26:31.754 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-f[65067]: ignoring --setgroup ceph since I am not root 2024-09-19T08:26:31.754 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-mds[65071]: -- 172.21.15.157:0/3101681833 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ed88d21860 con 0x55ed88cff400 2024-09-19T08:26:31.754 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-f[65067]: starting mds.f at 2024-09-19T08:26:31.754 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-c[65370]: ignoring --setuser ceph since I am not root 2024-09-19T08:26:31.755 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-c[65370]: ignoring --setgroup ceph since I am not root 2024-09-19T08:26:31.755 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-mds[65374]: -- 172.21.15.157:0/56399608 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558c1c66f860 con 0x558c1c64c800 2024-09-19T08:26:31.755 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-c[65370]: starting mds.c at 2024-09-19T08:26:31.755 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i[65670]: ignoring --setuser ceph since I am not root 2024-09-19T08:26:31.755 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i[65670]: ignoring --setgroup ceph since I am not root 2024-09-19T08:26:31.756 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-mds[65674]: -- 172.21.15.157:0/1675546748 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f620c29860 con 0x55f620c06800 2024-09-19T08:26:31.756 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:26:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i[65670]: starting mds.i at 2024-09-19T08:26:31.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1698801848' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T08:26:31.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-mon[34900]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T08:26:31.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-mon[34900]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:26:31.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1698801848' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T08:26:31.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-mon[34900]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T08:26:31.837 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-mon[34900]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T08:26:31.838 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-b[64527]: ignoring --setuser ceph since I am not root 2024-09-19T08:26:31.838 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-b[64527]: ignoring --setgroup ceph since I am not root 2024-09-19T08:26:31.838 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-mds[64531]: -- 172.21.15.64:0/3999956764 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561d7fe4bc20 con 0x561d7fe29400 2024-09-19T08:26:31.838 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-b[64527]: starting mds.b at 2024-09-19T08:26:31.838 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-h[64214]: ignoring --setuser ceph since I am not root 2024-09-19T08:26:31.838 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-h[64214]: ignoring --setgroup ceph since I am not root 2024-09-19T08:26:31.838 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-mds[64218]: -- 172.21.15.64:0/1597799662 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ab4aa95c20 con 0x55ab4aa73000 2024-09-19T08:26:31.838 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-h[64214]: starting mds.h at 2024-09-19T08:26:31.839 INFO:journalctl@ceph.mds.e.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-e[63911]: ignoring --setuser ceph since I am not root 2024-09-19T08:26:31.839 INFO:journalctl@ceph.mds.e.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-e[63911]: ignoring --setgroup ceph since I am not root 2024-09-19T08:26:31.839 INFO:journalctl@ceph.mds.e.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-mds[63915]: -- 172.21.15.64:0/1792118046 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d19c9d860 con 0x562d19c7a800 2024-09-19T08:26:31.839 INFO:journalctl@ceph.mds.e.smithi064.stdout:Sep 19 08:26:31 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-e[63911]: starting mds.e at 2024-09-19T08:26:32.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:31 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1698801848' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T08:26:32.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:31 smithi022 ceph-mon[30963]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T08:26:32.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:31 smithi022 ceph-mon[30963]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:26:32.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:31 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1698801848' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T08:26:32.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:31 smithi022 ceph-mon[30963]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T08:26:32.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:31 smithi022 ceph-mon[30963]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T08:26:32.048 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T08:26:32.305 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:26:32.706 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:32 smithi022 ceph-mon[30963]: pgmap v359: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-19T08:26:32.706 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:32 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.64:6834/256383644,v1:172.21.15.64:6836/256383644] up:boot 2024-09-19T08:26:32.706 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:32 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.64:6835/2971215235,v1:172.21.15.64:6837/2971215235] up:boot 2024-09-19T08:26:32.706 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:32 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.64:6838/628240791,v1:172.21.15.64:6839/628240791] up:boot 2024-09-19T08:26:32.706 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:32 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.157:6834/2617585551,v1:172.21.15.157:6837/2617585551] up:boot 2024-09-19T08:26:32.706 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:32 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.157:6835/1807640526,v1:172.21.15.157:6838/1807640526] up:boot 2024-09-19T08:26:32.706 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:32 smithi022 ceph-mon[30963]: mds.? [v2:172.21.15.157:6836/2559739818,v1:172.21.15.157:6839/2559739818] up:boot 2024-09-19T08:26:32.707 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:32 smithi022 ceph-mon[30963]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T08:26:32.707 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:32 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:26:32.707 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:32 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:26:32.707 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:32 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:26:32.707 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:32 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:26:32.707 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:32 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:26:32.707 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:32 smithi022 ceph-mon[30963]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:26:32.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.790+0000 7f77ea9c0640 1 -- 172.21.15.22:0/31078177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e4100a80 msgr2=0x7f77e4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:32.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.790+0000 7f77ea9c0640 1 --2- 172.21.15.22:0/31078177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e4100a80 0x7f77e4100ee0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009930 tx=0x7f77d802a420 comp rx=0 tx=0).stop 2024-09-19T08:26:32.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.790+0000 7f77ea9c0640 1 -- 172.21.15.22:0/31078177 shutdown_connections 2024-09-19T08:26:32.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.790+0000 7f77ea9c0640 1 --2- 172.21.15.22:0/31078177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e4101420 0x7f77e410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:32.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.790+0000 7f77ea9c0640 1 --2- 172.21.15.22:0/31078177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e4100a80 0x7f77e4100ee0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:32.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.790+0000 7f77ea9c0640 1 --2- 172.21.15.22:0/31078177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77e4106a80 0x7f77e4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:32.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.790+0000 7f77ea9c0640 1 -- 172.21.15.22:0/31078177 >> 172.21.15.22:0/31078177 conn(0x7f77e40fc7a0 msgr2=0x7f77e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:32.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.790+0000 7f77ea9c0640 1 -- 172.21.15.22:0/31078177 shutdown_connections 2024-09-19T08:26:32.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.790+0000 7f77ea9c0640 1 -- 172.21.15.22:0/31078177 wait complete. 2024-09-19T08:26:32.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.791+0000 7f77ea9c0640 1 Processor -- start 2024-09-19T08:26:32.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.791+0000 7f77ea9c0640 1 -- start start 2024-09-19T08:26:32.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.791+0000 7f77ea9c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e4100a80 0x7f77e4197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:32.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.791+0000 7f77ea9c0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77e4101420 0x7f77e4197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:32.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77ea9c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e4106a80 0x7f77e419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:32.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77ea9c0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f77e4110e40 con 0x7f77e4106a80 2024-09-19T08:26:32.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77ea9c0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77e4110cc0 con 0x7f77e4100a80 2024-09-19T08:26:32.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77ea9c0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77e4110fc0 con 0x7f77e4101420 2024-09-19T08:26:32.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77e8f5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e4106a80 0x7f77e419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:32.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77e8f5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e4106a80 0x7f77e419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57778/0 (socket says 172.21.15.22:57778) 2024-09-19T08:26:32.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77e8f5d640 1 -- 172.21.15.22:0/787793137 learned_addr learned my addr 172.21.15.22:0/787793137 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:26:32.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77e3fff640 1 --2- 172.21.15.22:0/787793137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e4100a80 0x7f77e4197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:32.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77e37fe640 1 --2- 172.21.15.22:0/787793137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77e4101420 0x7f77e4197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:32.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77e8f5d640 1 -- 172.21.15.22:0/787793137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77e4101420 msgr2=0x7f77e4197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:32.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77e8f5d640 1 --2- 172.21.15.22:0/787793137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77e4101420 0x7f77e4197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:32.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77e8f5d640 1 -- 172.21.15.22:0/787793137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e4100a80 msgr2=0x7f77e4197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:32.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77e8f5d640 1 --2- 172.21.15.22:0/787793137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e4100a80 0x7f77e4197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:32.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77e8f5d640 1 -- 172.21.15.22:0/787793137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e419c2b0 con 0x7f77e4106a80 2024-09-19T08:26:32.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77e8f5d640 1 --2- 172.21.15.22:0/787793137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e4106a80 0x7f77e419bbd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00eaa0 tx=0x7f77dc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:32.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77e17fa640 1 -- 172.21.15.22:0/787793137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc004d20 con 0x7f77e4106a80 2024-09-19T08:26:32.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77e17fa640 1 -- 172.21.15.22:0/787793137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77dc005020 con 0x7f77e4106a80 2024-09-19T08:26:32.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77ea9c0640 1 -- 172.21.15.22:0/787793137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e419c5a0 con 0x7f77e4106a80 2024-09-19T08:26:32.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77e17fa640 1 -- 172.21.15.22:0/787793137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0116d0 con 0x7f77e4106a80 2024-09-19T08:26:32.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.792+0000 7f77ea9c0640 1 -- 172.21.15.22:0/787793137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e41a3e80 con 0x7f77e4106a80 2024-09-19T08:26:32.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.793+0000 7f77ea9c0640 1 -- 172.21.15.22:0/787793137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e4100ee0 con 0x7f77e4106a80 2024-09-19T08:26:32.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.795+0000 7f77e17fa640 1 -- 172.21.15.22:0/787793137 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f77dc0051d0 con 0x7f77e4106a80 2024-09-19T08:26:32.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.795+0000 7f77e17fa640 1 --2- 172.21.15.22:0/787793137 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f77cc0a33b0 0x7f77cc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:32.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.796+0000 7f77e3fff640 1 --2- 172.21.15.22:0/787793137 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f77cc0a33b0 0x7f77cc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:32.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.796+0000 7f77e17fa640 1 -- 172.21.15.22:0/787793137 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8756+0+0 (secure 0 0 0) 0x7f77dc0d0410 con 0x7f77e4106a80 2024-09-19T08:26:32.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.796+0000 7f77e3fff640 1 --2- 172.21.15.22:0/787793137 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f77cc0a33b0 0x7f77cc0a5870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f77d400ab60 tx=0x7f77d400a2d0 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:32.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:32.799+0000 7f77e17fa640 1 -- 172.21.15.22:0/787793137 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f77dc016030 con 0x7f77e4106a80 2024-09-19T08:26:33.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:32 smithi157 ceph-mon[35143]: pgmap v359: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-19T08:26:33.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:32 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.64:6834/256383644,v1:172.21.15.64:6836/256383644] up:boot 2024-09-19T08:26:33.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:32 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.64:6835/2971215235,v1:172.21.15.64:6837/2971215235] up:boot 2024-09-19T08:26:33.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:32 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.64:6838/628240791,v1:172.21.15.64:6839/628240791] up:boot 2024-09-19T08:26:33.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:32 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.157:6834/2617585551,v1:172.21.15.157:6837/2617585551] up:boot 2024-09-19T08:26:33.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:32 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.157:6835/1807640526,v1:172.21.15.157:6838/1807640526] up:boot 2024-09-19T08:26:33.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:32 smithi157 ceph-mon[35143]: mds.? [v2:172.21.15.157:6836/2559739818,v1:172.21.15.157:6839/2559739818] up:boot 2024-09-19T08:26:33.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:32 smithi157 ceph-mon[35143]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T08:26:33.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:32 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:26:33.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:32 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:26:33.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:32 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:26:33.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:32 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:26:33.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:32 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:26:33.004 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:32 smithi157 ceph-mon[35143]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:26:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:33.010+0000 7f77ea9c0640 1 -- 172.21.15.22:0/787793137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f77e4108a80 con 0x7f77e4106a80 2024-09-19T08:26:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:32 smithi064 ceph-mon[34900]: pgmap v359: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-19T08:26:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:32 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.64:6834/256383644,v1:172.21.15.64:6836/256383644] up:boot 2024-09-19T08:26:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:32 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.64:6835/2971215235,v1:172.21.15.64:6837/2971215235] up:boot 2024-09-19T08:26:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:32 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.64:6838/628240791,v1:172.21.15.64:6839/628240791] up:boot 2024-09-19T08:26:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:32 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.157:6834/2617585551,v1:172.21.15.157:6837/2617585551] up:boot 2024-09-19T08:26:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:32 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.157:6835/1807640526,v1:172.21.15.157:6838/1807640526] up:boot 2024-09-19T08:26:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:32 smithi064 ceph-mon[34900]: mds.? [v2:172.21.15.157:6836/2559739818,v1:172.21.15.157:6839/2559739818] up:boot 2024-09-19T08:26:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:32 smithi064 ceph-mon[34900]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T08:26:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:32 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:26:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:32 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:26:33.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:32 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:26:33.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:32 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:26:33.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:32 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:26:33.088 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:32 smithi064 ceph-mon[34900]: from='mgr.15141 172.21.15.22:0/4163854768' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:26:33.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:33.646+0000 7f77e17fa640 1 -- 172.21.15.22:0/787793137 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f77dc09cd00 con 0x7f77e4106a80 2024-09-19T08:26:33.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:33.648+0000 7f77ea9c0640 1 -- 172.21.15.22:0/787793137 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f77cc0a33b0 msgr2=0x7f77cc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:33.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:33.648+0000 7f77ea9c0640 1 --2- 172.21.15.22:0/787793137 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f77cc0a33b0 0x7f77cc0a5870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f77d400ab60 tx=0x7f77d400a2d0 comp rx=0 tx=0).stop 2024-09-19T08:26:33.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:33.648+0000 7f77ea9c0640 1 -- 172.21.15.22:0/787793137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e4106a80 msgr2=0x7f77e419bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:33.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:33.648+0000 7f77ea9c0640 1 --2- 172.21.15.22:0/787793137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e4106a80 0x7f77e419bbd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00eaa0 tx=0x7f77dc00eea0 comp rx=0 tx=0).stop 2024-09-19T08:26:33.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:33.648+0000 7f77ea9c0640 1 -- 172.21.15.22:0/787793137 shutdown_connections 2024-09-19T08:26:33.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:33.648+0000 7f77ea9c0640 1 --2- 172.21.15.22:0/787793137 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7f77cc0a33b0 0x7f77cc0a5870 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:33.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:33.648+0000 7f77ea9c0640 1 --2- 172.21.15.22:0/787793137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e4106a80 0x7f77e419bbd0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:33.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:33.648+0000 7f77ea9c0640 1 --2- 172.21.15.22:0/787793137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77e4101420 0x7f77e4197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:33.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:33.648+0000 7f77ea9c0640 1 --2- 172.21.15.22:0/787793137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77e4100a80 0x7f77e4197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:33.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:33.648+0000 7f77ea9c0640 1 -- 172.21.15.22:0/787793137 >> 172.21.15.22:0/787793137 conn(0x7f77e40fc7a0 msgr2=0x7f77e40fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:33.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:33.649+0000 7f77ea9c0640 1 -- 172.21.15.22:0/787793137 shutdown_connections 2024-09-19T08:26:33.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:33.649+0000 7f77ea9c0640 1 -- 172.21.15.22:0/787793137 wait complete. 2024-09-19T08:26:34.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:33 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/787793137' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:26:34.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:33 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/787793137' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:26:34.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:33 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/787793137' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:26:34.389 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T08:26:34.655 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:26:34.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:34 smithi022 ceph-mon[30963]: pgmap v360: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-19T08:26:34.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:34 smithi022 ceph-mon[30963]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T08:26:34.899 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:34 smithi022 ceph-mon[30963]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T08:26:34.899 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:34 smithi022 ceph-mon[30963]: Cluster is now healthy 2024-09-19T08:26:34.899 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:34 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/787793137' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:26:34.899 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:34 smithi022 ceph-mon[30963]: fsmap 12 up:standby 2024-09-19T08:26:35.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:34 smithi157 ceph-mon[35143]: pgmap v360: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-19T08:26:35.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:34 smithi157 ceph-mon[35143]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T08:26:35.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:34 smithi157 ceph-mon[35143]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T08:26:35.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:34 smithi157 ceph-mon[35143]: Cluster is now healthy 2024-09-19T08:26:35.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:34 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/787793137' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:26:35.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:34 smithi157 ceph-mon[35143]: fsmap 12 up:standby 2024-09-19T08:26:35.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:34 smithi064 ceph-mon[34900]: pgmap v360: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-19T08:26:35.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:34 smithi064 ceph-mon[34900]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T08:26:35.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:34 smithi064 ceph-mon[34900]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T08:26:35.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:34 smithi064 ceph-mon[34900]: Cluster is now healthy 2024-09-19T08:26:35.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:34 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/787793137' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:26:35.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:34 smithi064 ceph-mon[34900]: fsmap 12 up:standby 2024-09-19T08:26:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.117+0000 7fb60c02f640 1 -- 172.21.15.22:0/3526961904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb604112b90 msgr2=0x7fb604114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.117+0000 7fb60c02f640 1 --2- 172.21.15.22:0/3526961904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb604112b90 0x7fb604114f80 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009ca0 tx=0x7fb5fc02a950 comp rx=0 tx=0).stop 2024-09-19T08:26:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.117+0000 7fb60c02f640 1 -- 172.21.15.22:0/3526961904 shutdown_connections 2024-09-19T08:26:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.117+0000 7fb60c02f640 1 --2- 172.21.15.22:0/3526961904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb604112b90 0x7fb604114f80 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.117+0000 7fb60c02f640 1 --2- 172.21.15.22:0/3526961904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb604074eb0 0x7fb604112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.117+0000 7fb60c02f640 1 --2- 172.21.15.22:0/3526961904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb604074590 0x7fb604074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.117+0000 7fb60c02f640 1 -- 172.21.15.22:0/3526961904 >> 172.21.15.22:0/3526961904 conn(0x7fb6040fd8c0 msgr2=0x7fb6040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.118+0000 7fb60c02f640 1 -- 172.21.15.22:0/3526961904 shutdown_connections 2024-09-19T08:26:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.118+0000 7fb60c02f640 1 -- 172.21.15.22:0/3526961904 wait complete. 2024-09-19T08:26:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.118+0000 7fb60c02f640 1 Processor -- start 2024-09-19T08:26:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.118+0000 7fb60c02f640 1 -- start start 2024-09-19T08:26:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.118+0000 7fb60c02f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb604074590 0x7fb6041a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.118+0000 7fb60c02f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb604074eb0 0x7fb6041a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.118+0000 7fb60c02f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb604112b90 0x7fb60419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.118+0000 7fb60c02f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb6041185b0 con 0x7fb604074eb0 2024-09-19T08:26:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.118+0000 7fb60c02f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb604118430 con 0x7fb604112b90 2024-09-19T08:26:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.118+0000 7fb60c02f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb604118730 con 0x7fb604074590 2024-09-19T08:26:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb6095ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb604074eb0 0x7fb6041a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb6095ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb604074eb0 0x7fb6041a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57802/0 (socket says 172.21.15.22:57802) 2024-09-19T08:26:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb60a5cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb604112b90 0x7fb60419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb6095ca640 1 -- 172.21.15.22:0/1696311279 learned_addr learned my addr 172.21.15.22:0/1696311279 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:26:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb609dcb640 1 --2- 172.21.15.22:0/1696311279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb604074590 0x7fb6041a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb6095ca640 1 -- 172.21.15.22:0/1696311279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb604074590 msgr2=0x7fb6041a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb6095ca640 1 --2- 172.21.15.22:0/1696311279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb604074590 0x7fb6041a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb6095ca640 1 -- 172.21.15.22:0/1696311279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb604112b90 msgr2=0x7fb60419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb6095ca640 1 --2- 172.21.15.22:0/1696311279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb604112b90 0x7fb60419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb6095ca640 1 -- 172.21.15.22:0/1696311279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60419f000 con 0x7fb604074eb0 2024-09-19T08:26:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb6095ca640 1 --2- 172.21.15.22:0/1696311279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb604074eb0 0x7fb6041a4c50 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400b4a0 tx=0x7fb5f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb5faffd640 1 -- 172.21.15.22:0/1696311279 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4014070 con 0x7fb604074eb0 2024-09-19T08:26:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb5faffd640 1 -- 172.21.15.22:0/1696311279 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5f4010af0 con 0x7fb604074eb0 2024-09-19T08:26:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb60c02f640 1 -- 172.21.15.22:0/1696311279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60419f2f0 con 0x7fb604074eb0 2024-09-19T08:26:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb5faffd640 1 -- 172.21.15.22:0/1696311279 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4017d10 con 0x7fb604074eb0 2024-09-19T08:26:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.119+0000 7fb60c02f640 1 -- 172.21.15.22:0/1696311279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6041ab5e0 con 0x7fb604074eb0 2024-09-19T08:26:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.120+0000 7fb60c02f640 1 -- 172.21.15.22:0/1696311279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb604074970 con 0x7fb604074eb0 2024-09-19T08:26:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.121+0000 7fb5faffd640 1 -- 172.21.15.22:0/1696311279 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb5f4017070 con 0x7fb604074eb0 2024-09-19T08:26:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.122+0000 7fb5faffd640 1 --2- 172.21.15.22:0/1696311279 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb5f00a3430 0x7fb5f00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.122+0000 7fb609dcb640 1 --2- 172.21.15.22:0/1696311279 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb5f00a3430 0x7fb5f00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.122+0000 7fb5faffd640 1 -- 172.21.15.22:0/1696311279 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8756+0+0 (secure 0 0 0) 0x7fb5f40cefb0 con 0x7fb604074eb0 2024-09-19T08:26:35.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.123+0000 7fb609dcb640 1 --2- 172.21.15.22:0/1696311279 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb5f00a3430 0x7fb5f00a58f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb60419fa10 tx=0x7fb60000b390 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:35.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.127+0000 7fb5faffd640 1 -- 172.21.15.22:0/1696311279 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb5f409b360 con 0x7fb604074eb0 2024-09-19T08:26:35.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.302+0000 7fb60c02f640 1 -- 172.21.15.22:0/1696311279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb6041ab930 con 0x7fb604074eb0 2024-09-19T08:26:35.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.653+0000 7fb5faffd640 1 -- 172.21.15.22:0/1696311279 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) ==== 181+0+0 (secure 0 0 0) 0x7fb5f409f660 con 0x7fb604074eb0 2024-09-19T08:26:35.654 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-09-19T08:26:35.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.657+0000 7fb60c02f640 1 -- 172.21.15.22:0/1696311279 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb5f00a3430 msgr2=0x7fb5f00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:35.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.657+0000 7fb60c02f640 1 --2- 172.21.15.22:0/1696311279 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb5f00a3430 0x7fb5f00a58f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb60419fa10 tx=0x7fb60000b390 comp rx=0 tx=0).stop 2024-09-19T08:26:35.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.657+0000 7fb60c02f640 1 -- 172.21.15.22:0/1696311279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb604074eb0 msgr2=0x7fb6041a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:35.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.657+0000 7fb60c02f640 1 --2- 172.21.15.22:0/1696311279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb604074eb0 0x7fb6041a4c50 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400b4a0 tx=0x7fb5f400b960 comp rx=0 tx=0).stop 2024-09-19T08:26:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.658+0000 7fb60c02f640 1 -- 172.21.15.22:0/1696311279 shutdown_connections 2024-09-19T08:26:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.658+0000 7fb60c02f640 1 --2- 172.21.15.22:0/1696311279 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fb5f00a3430 0x7fb5f00a58f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.658+0000 7fb60c02f640 1 --2- 172.21.15.22:0/1696311279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb604112b90 0x7fb60419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.658+0000 7fb60c02f640 1 --2- 172.21.15.22:0/1696311279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb604074eb0 0x7fb6041a4c50 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.658+0000 7fb60c02f640 1 --2- 172.21.15.22:0/1696311279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb604074590 0x7fb6041a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.658+0000 7fb60c02f640 1 -- 172.21.15.22:0/1696311279 >> 172.21.15.22:0/1696311279 conn(0x7fb6040fd8c0 msgr2=0x7fb6040ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.658+0000 7fb60c02f640 1 -- 172.21.15.22:0/1696311279 shutdown_connections 2024-09-19T08:26:35.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:35.659+0000 7fb60c02f640 1 -- 172.21.15.22:0/1696311279 wait complete. 2024-09-19T08:26:35.994 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:35 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1696311279' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T08:26:36.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:35 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1696311279' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T08:26:36.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:35 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1696311279' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T08:26:36.335 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T08:26:36.600 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:26:36.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:36 smithi022 ceph-mon[30963]: pgmap v361: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-19T08:26:36.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:36 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/1696311279' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T08:26:36.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:36 smithi022 ceph-mon[30963]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T08:26:37.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:36 smithi157 ceph-mon[35143]: pgmap v361: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-19T08:26:37.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:36 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/1696311279' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T08:26:37.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:36 smithi157 ceph-mon[35143]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T08:26:37.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:36 smithi064 ceph-mon[34900]: pgmap v361: 97 pgs: 97 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-19T08:26:37.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:36 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/1696311279' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T08:26:37.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:36 smithi064 ceph-mon[34900]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T08:26:37.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.083+0000 7fa8d3986640 1 -- 172.21.15.22:0/3327119358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8cc1036e0 msgr2=0x7fa8cc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:37.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.083+0000 7fa8d3986640 1 --2- 172.21.15.22:0/3327119358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8cc1036e0 0x7fa8cc107b90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009930 tx=0x7fa8c002a420 comp rx=0 tx=0).stop 2024-09-19T08:26:37.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.084+0000 7fa8d3986640 1 -- 172.21.15.22:0/3327119358 shutdown_connections 2024-09-19T08:26:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.084+0000 7fa8d3986640 1 --2- 172.21.15.22:0/3327119358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8cc1082a0 0x7fa8cc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.084+0000 7fa8d3986640 1 --2- 172.21.15.22:0/3327119358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8cc1036e0 0x7fa8cc107b90 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.084+0000 7fa8d3986640 1 --2- 172.21.15.22:0/3327119358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc102d30 0x7fa8cc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.084+0000 7fa8d3986640 1 -- 172.21.15.22:0/3327119358 >> 172.21.15.22:0/3327119358 conn(0x7fa8cc0fc7d0 msgr2=0x7fa8cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.084+0000 7fa8d3986640 1 -- 172.21.15.22:0/3327119358 shutdown_connections 2024-09-19T08:26:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.084+0000 7fa8d3986640 1 -- 172.21.15.22:0/3327119358 wait complete. 2024-09-19T08:26:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.084+0000 7fa8d3986640 1 Processor -- start 2024-09-19T08:26:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.084+0000 7fa8d3986640 1 -- start start 2024-09-19T08:26:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d3986640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8cc102d30 0x7fa8cc198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d3986640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc1036e0 0x7fa8cc198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d3986640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8cc1082a0 0x7fa8cc19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d3986640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1120a0 con 0x7fa8cc1082a0 2024-09-19T08:26:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d3986640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc111f20 con 0x7fa8cc1036e0 2024-09-19T08:26:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d3986640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc112220 con 0x7fa8cc102d30 2024-09-19T08:26:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d1f23640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8cc1082a0 0x7fa8cc19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d1f23640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8cc1082a0 0x7fa8cc19cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57818/0 (socket says 172.21.15.22:57818) 2024-09-19T08:26:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d1f23640 1 -- 172.21.15.22:0/371169749 learned_addr learned my addr 172.21.15.22:0/371169749 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T08:26:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d0f21640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc1036e0 0x7fa8cc198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d1722640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8cc102d30 0x7fa8cc198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d1f23640 1 -- 172.21.15.22:0/371169749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8cc102d30 msgr2=0x7fa8cc198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d1f23640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8cc102d30 0x7fa8cc198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d1f23640 1 -- 172.21.15.22:0/371169749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc1036e0 msgr2=0x7fa8cc198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d1f23640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc1036e0 0x7fa8cc198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d1f23640 1 -- 172.21.15.22:0/371169749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc19d5f0 con 0x7fa8cc1082a0 2024-09-19T08:26:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d0f21640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc1036e0 0x7fa8cc198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:26:37.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d1722640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8cc102d30 0x7fa8cc198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:26:37.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8d1f23640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8cc1082a0 0x7fa8cc19cf10 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00eaa0 tx=0x7fa8bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:37.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.085+0000 7fa8ca7fc640 1 -- 172.21.15.22:0/371169749 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00cdb0 con 0x7fa8cc1082a0 2024-09-19T08:26:37.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.086+0000 7fa8ca7fc640 1 -- 172.21.15.22:0/371169749 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8bc005020 con 0x7fa8cc1082a0 2024-09-19T08:26:37.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.086+0000 7fa8d3986640 1 -- 172.21.15.22:0/371169749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc19d8e0 con 0x7fa8cc1082a0 2024-09-19T08:26:37.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.086+0000 7fa8ca7fc640 1 -- 172.21.15.22:0/371169749 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0116b0 con 0x7fa8cc1082a0 2024-09-19T08:26:37.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.086+0000 7fa8d3986640 1 -- 172.21.15.22:0/371169749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc1a51c0 con 0x7fa8cc1082a0 2024-09-19T08:26:37.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.087+0000 7fa8d3986640 1 -- 172.21.15.22:0/371169749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8cc103110 con 0x7fa8cc1082a0 2024-09-19T08:26:37.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.088+0000 7fa8ca7fc640 1 -- 172.21.15.22:0/371169749 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa8bc005560 con 0x7fa8cc1082a0 2024-09-19T08:26:37.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.089+0000 7fa8ca7fc640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa8b40a3430 0x7fa8b40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:26:37.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.089+0000 7fa8d1722640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa8b40a3430 0x7fa8b40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:26:37.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.089+0000 7fa8ca7fc640 1 -- 172.21.15.22:0/371169749 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8202+0+0 (secure 0 0 0) 0x7fa8bc0cfe10 con 0x7fa8cc1082a0 2024-09-19T08:26:37.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.089+0000 7fa8d1722640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa8b40a3430 0x7fa8b40a58f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009b40 tx=0x7fa8c4005e50 comp rx=0 tx=0).ready entity=mgr.15141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:26:37.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.093+0000 7fa8ca7fc640 1 -- 172.21.15.22:0/371169749 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa8bc016030 con 0x7fa8cc1082a0 2024-09-19T08:26:37.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.277+0000 7fa8d3986640 1 -- 172.21.15.22:0/371169749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa8cc10a2b0 con 0x7fa8cc1082a0 2024-09-19T08:26:37.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.660+0000 7fa8ca7fc640 1 -- 172.21.15.22:0/371169749 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) ==== 169+0+0 (secure 0 0 0) 0x7fa8bc09c930 con 0x7fa8cc1082a0 2024-09-19T08:26:37.661 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-09-19T08:26:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.664+0000 7fa8d3986640 1 -- 172.21.15.22:0/371169749 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa8b40a3430 msgr2=0x7fa8b40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.664+0000 7fa8d3986640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa8b40a3430 0x7fa8b40a58f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009b40 tx=0x7fa8c4005e50 comp rx=0 tx=0).stop 2024-09-19T08:26:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.664+0000 7fa8d3986640 1 -- 172.21.15.22:0/371169749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8cc1082a0 msgr2=0x7fa8cc19cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:26:37.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.664+0000 7fa8d3986640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8cc1082a0 0x7fa8cc19cf10 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00eaa0 tx=0x7fa8bc00ef60 comp rx=0 tx=0).stop 2024-09-19T08:26:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.665+0000 7fa8d3986640 1 -- 172.21.15.22:0/371169749 shutdown_connections 2024-09-19T08:26:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.665+0000 7fa8d3986640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.22:6800/497251935,v1:172.21.15.22:6801/497251935] conn(0x7fa8b40a3430 0x7fa8b40a58f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.665+0000 7fa8d3986640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8cc1082a0 0x7fa8cc19cf10 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.665+0000 7fa8d3986640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8cc1036e0 0x7fa8cc198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.665+0000 7fa8d3986640 1 --2- 172.21.15.22:0/371169749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8cc102d30 0x7fa8cc198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:26:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.665+0000 7fa8d3986640 1 -- 172.21.15.22:0/371169749 >> 172.21.15.22:0/371169749 conn(0x7fa8cc0fc7d0 msgr2=0x7fa8cc105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:26:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.666+0000 7fa8d3986640 1 -- 172.21.15.22:0/371169749 shutdown_connections 2024-09-19T08:26:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T08:26:37.666+0000 7fa8d3986640 1 -- 172.21.15.22:0/371169749 wait complete. 2024-09-19T08:26:38.003 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:37 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/371169749' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T08:26:38.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:37 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/371169749' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T08:26:38.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:37 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/371169749' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T08:26:38.350 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:26:38.379 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:26:38.410 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:26:38.448 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T08:26:38.448 DEBUG:teuthology.orchestra.run.smithi022:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T08:26:38.465 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T08:26:38.482 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T08:26:38.509 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T08:26:38.509 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T08:26:38.509 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.a 2024-09-19T08:26:38.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:38 smithi022 systemd[1]: Stopping Ceph mon.a for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:26:38.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:38 smithi157 ceph-mon[35143]: pgmap v363: 65 pgs: 65 active+clean; 583 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:26:38.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:38 smithi157 ceph-mon[35143]: from='client.? 172.21.15.22:0/371169749' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T08:26:38.753 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:38 smithi157 ceph-mon[35143]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T08:26:39.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:38 smithi022 ceph-mon[30963]: pgmap v363: 65 pgs: 65 active+clean; 583 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:26:39.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:38 smithi022 ceph-mon[30963]: from='client.? 172.21.15.22:0/371169749' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T08:26:39.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:38 smithi022 ceph-mon[30963]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T08:26:39.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:38 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a[30959]: 2024-09-19T08:26:38.818+0000 7f0bc956d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:26:39.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:38 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a[30959]: 2024-09-19T08:26:38.818+0000 7f0bc956d640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T08:26:39.086 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:38 smithi064 ceph-mon[34900]: pgmap v363: 65 pgs: 65 active+clean; 583 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:26:39.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:38 smithi064 ceph-mon[34900]: from='client.? 172.21.15.22:0/371169749' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T08:26:39.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:38 smithi064 ceph-mon[34900]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T08:26:39.378 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:39 smithi022 podman[109499]: 2024-09-19 08:26:39.075291897 +0000 UTC m=+0.369195356 container died 3ded52544efcb5ce71aa7934b2bcd324cdb40cddc4de64baffba0593d6b24d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T08:26:39.744 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:39 smithi022 podman[109499]: 2024-09-19 08:26:39.476425093 +0000 UTC m=+0.770328545 container cleanup 3ded52544efcb5ce71aa7934b2bcd324cdb40cddc4de64baffba0593d6b24d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T08:26:39.744 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 19 08:26:39 smithi022 bash[109499]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-a 2024-09-19T08:26:39.904 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T08:26:39.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:26:39.947 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T08:26:39.947 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T08:26:39.948 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.b 2024-09-19T08:26:40.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:39 smithi064 systemd[1]: Stopping Ceph mon.b for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:26:40.586 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:40 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-b[34896]: 2024-09-19T08:26:40.251+0000 7f90ba793640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:26:40.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:40 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-b[34896]: 2024-09-19T08:26:40.251+0000 7f90ba793640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T08:26:40.587 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:40 smithi064 podman[70338]: 2024-09-19 08:26:40.384152752 +0000 UTC m=+0.245564393 container died a7fa335b0bdd1fffcf20f9bdbd3902b2481410b5b1d060d2b7a4172705d129ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T08:26:40.994 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:40 smithi064 podman[70338]: 2024-09-19 08:26:40.809978709 +0000 UTC m=+0.671390341 container cleanup a7fa335b0bdd1fffcf20f9bdbd3902b2481410b5b1d060d2b7a4172705d129ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T08:26:40.994 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:40 smithi064 bash[70338]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-b 2024-09-19T08:26:41.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T08:26:41.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:40 smithi064 podman[70350]: 2024-09-19 08:26:40.994733097 +0000 UTC m=+0.610383692 container remove a7fa335b0bdd1fffcf20f9bdbd3902b2481410b5b1d060d2b7a4172705d129ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T08:26:41.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:41 smithi064 systemd[1]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-19T08:26:41.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:41 smithi064 systemd[1]: Stopped Ceph mon.b for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:26:41.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Sep 19 08:26:41 smithi064 systemd[1]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.b.service: Consumed 12.317s CPU time. 2024-09-19T08:26:41.281 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:26:41.281 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T08:26:41.281 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T08:26:41.282 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.c 2024-09-19T08:26:41.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:41 smithi157 systemd[1]: Stopping Ceph mon.c for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:26:41.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:41 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-c[35137]: 2024-09-19T08:26:41.663+0000 7fb960f23640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:26:41.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:41 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-c[35137]: 2024-09-19T08:26:41.663+0000 7fb960f23640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T08:26:42.223 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:41 smithi157 podman[71214]: 2024-09-19 08:26:41.939267598 +0000 UTC m=+0.400967552 container died 9ec3c64fd242e22f1eb94f8c969df320ea45cfc78c822e2bbf419247659fea77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T08:26:42.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:42 smithi157 podman[71214]: 2024-09-19 08:26:42.338949441 +0000 UTC m=+0.800649395 container cleanup 9ec3c64fd242e22f1eb94f8c969df320ea45cfc78c822e2bbf419247659fea77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T08:26:42.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:Sep 19 08:26:42 smithi157 bash[71214]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mon-c 2024-09-19T08:26:42.766 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T08:26:42.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:26:42.816 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T08:26:42.817 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T08:26:42.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mgr.x 2024-09-19T08:26:43.131 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:26:42 smithi022 systemd[1]: Stopping Ceph mgr.x for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:26:43.448 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:26:43 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x[31189]: 2024-09-19T08:26:43.130+0000 7f98e3180640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:26:43.448 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:26:43 smithi022 podman[109618]: 2024-09-19 08:26:43.162320015 +0000 UTC m=+0.146386347 container died e278914c599895cc093a3e397c64fe28ec77a9c06767e2c794178116786fa934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T08:26:43.757 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:26:43 smithi022 podman[109618]: 2024-09-19 08:26:43.657404821 +0000 UTC m=+0.641471178 container cleanup e278914c599895cc093a3e397c64fe28ec77a9c06767e2c794178116786fa934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T08:26:43.757 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:26:43 smithi022 bash[109618]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x 2024-09-19T08:26:44.136 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 19 08:26:43 smithi022 podman[109632]: 2024-09-19 08:26:43.884770168 +0000 UTC m=+0.718477942 container remove e278914c599895cc093a3e397c64fe28ec77a9c06767e2c794178116786fa934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T08:26:44.163 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T08:26:44.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:26:44.203 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T08:26:44.203 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T08:26:44.204 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mgr.y 2024-09-19T08:26:44.542 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:26:44 smithi064 systemd[1]: Stopping Ceph mgr.y for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:26:44.837 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:26:44 smithi064 podman[70460]: 2024-09-19 08:26:44.543412116 +0000 UTC m=+0.145972790 container died faa40f033561bc887f55731d1bffdeb39648aa7731317b172482669823256c85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T08:26:45.483 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:26:45 smithi064 podman[70460]: 2024-09-19 08:26:45.231930219 +0000 UTC m=+0.834490887 container cleanup faa40f033561bc887f55731d1bffdeb39648aa7731317b172482669823256c85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T08:26:45.483 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Sep 19 08:26:45 smithi064 bash[70460]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-y 2024-09-19T08:26:45.660 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T08:26:45.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:26:45.695 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T08:26:45.695 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T08:26:45.695 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mgr.z 2024-09-19T08:26:46.003 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:26:45 smithi157 systemd[1]: Stopping Ceph mgr.z for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:26:46.753 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:26:46 smithi157 podman[71337]: 2024-09-19 08:26:46.451029661 +0000 UTC m=+0.559274303 container died f6f15a448e8ba70a72f5f96b4f3beea7e7f18eb78c8e8db726aa811f3893735d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T08:26:47.253 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:26:46 smithi157 podman[71337]: 2024-09-19 08:26:46.878022273 +0000 UTC m=+0.986266913 container cleanup f6f15a448e8ba70a72f5f96b4f3beea7e7f18eb78c8e8db726aa811f3893735d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T08:26:47.253 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:26:46 smithi157 bash[71337]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z 2024-09-19T08:26:47.859 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:26:47 smithi157 podman[71349]: 2024-09-19 08:26:47.596599874 +0000 UTC m=+1.142027665 container remove f6f15a448e8ba70a72f5f96b4f3beea7e7f18eb78c8e8db726aa811f3893735d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mgr-z, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T08:26:47.859 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Sep 19 08:26:47 smithi157 systemd[1]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-19T08:26:47.864 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T08:26:47.899 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:26:47.899 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T08:26:47.899 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T08:26:47.900 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.0 2024-09-19T08:26:48.256 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:26:47 smithi022 systemd[1]: Stopping Ceph osd.0 for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:26:49.007 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:26:48 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0[44787]: 2024-09-19T08:26:48.571+0000 7f543ae48640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:26:49.007 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:26:48 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0[44787]: 2024-09-19T08:26:48.571+0000 7f543ae48640 -1 osd.0 99 *** Got signal Terminated *** 2024-09-19T08:26:49.007 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:26:48 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0[44787]: 2024-09-19T08:26:48.571+0000 7f543ae48640 -1 osd.0 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:26:54.414 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:26:54 smithi022 podman[109738]: 2024-09-19 08:26:54.118669424 +0000 UTC m=+6.017223062 container died a36e989e6a8bbec5bad68f1457ddd4f999873f5cb1b39c8c94bf2eb05c5177fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T08:26:54.757 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:26:54 smithi022 podman[109738]: 2024-09-19 08:26:54.512864463 +0000 UTC m=+6.411418098 container cleanup a36e989e6a8bbec5bad68f1457ddd4f999873f5cb1b39c8c94bf2eb05c5177fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T08:26:54.757 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:26:54 smithi022 bash[109738]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0 2024-09-19T08:26:55.147 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:26:54 smithi022 podman[109755]: 2024-09-19 08:26:54.781175099 +0000 UTC m=+0.660715744 container remove a36e989e6a8bbec5bad68f1457ddd4f999873f5cb1b39c8c94bf2eb05c5177fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T08:26:55.459 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:26:55 smithi022 podman[109842]: 2024-09-19 08:26:55.050981342 +0000 UTC m=+0.016558243 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:26:55.459 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:26:55 smithi022 podman[109842]: 2024-09-19 08:26:55.1654722 +0000 UTC m=+0.131049160 container create 9960483e80c98727eabaeb410faf3ca5fabcced38e1afd1d1b167efb7d32520a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-09-19T08:26:55.757 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:26:55 smithi022 podman[109842]: 2024-09-19 08:26:55.459997969 +0000 UTC m=+0.425574873 container init 9960483e80c98727eabaeb410faf3ca5fabcced38e1afd1d1b167efb7d32520a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T08:26:55.757 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:26:55 smithi022 podman[109842]: 2024-09-19 08:26:55.463607673 +0000 UTC m=+0.429184573 container start 9960483e80c98727eabaeb410faf3ca5fabcced38e1afd1d1b167efb7d32520a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T08:26:55.757 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:26:55 smithi022 podman[109842]: 2024-09-19 08:26:55.560116579 +0000 UTC m=+0.525693486 container attach 9960483e80c98727eabaeb410faf3ca5fabcced38e1afd1d1b167efb7d32520a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T08:26:55.757 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 19 08:26:55 smithi022 podman[109842]: 2024-09-19 08:26:55.654366131 +0000 UTC m=+0.619943023 container died 9960483e80c98727eabaeb410faf3ca5fabcced38e1afd1d1b167efb7d32520a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T08:26:56.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T08:26:56.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:26:56.404 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T08:26:56.405 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T08:26:56.405 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.1 2024-09-19T08:26:56.543 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:26:56 smithi064 systemd[1]: Stopping Ceph osd.1 for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:26:57.050 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:26:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1[39381]: 2024-09-19T08:26:56.735+0000 7f24c54ae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:26:57.050 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:26:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1[39381]: 2024-09-19T08:26:56.735+0000 7f24c54ae640 -1 osd.1 99 *** Got signal Terminated *** 2024-09-19T08:26:57.050 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:26:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1[39381]: 2024-09-19T08:26:56.735+0000 7f24c54ae640 -1 osd.1 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:27:02.387 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:27:02 smithi064 podman[70587]: 2024-09-19 08:27:02.098334154 +0000 UTC m=+5.484849408 container died e95db63a18ad3509710dac4e2d2bf04b0b0b3a6ac874e2d4f64be0210ee3593c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T08:27:02.837 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:27:02 smithi064 podman[70587]: 2024-09-19 08:27:02.58758889 +0000 UTC m=+5.974104142 container cleanup e95db63a18ad3509710dac4e2d2bf04b0b0b3a6ac874e2d4f64be0210ee3593c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T08:27:02.837 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:27:02 smithi064 bash[70587]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1 2024-09-19T08:27:03.156 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:27:02 smithi064 podman[70613]: 2024-09-19 08:27:02.839154439 +0000 UTC m=+0.738999103 container remove e95db63a18ad3509710dac4e2d2bf04b0b0b3a6ac874e2d4f64be0210ee3593c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T08:27:03.458 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:27:03 smithi064 podman[70703]: 2024-09-19 08:27:03.059051504 +0000 UTC m=+0.015786640 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:27:03.459 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:27:03 smithi064 podman[70703]: 2024-09-19 08:27:03.215382834 +0000 UTC m=+0.172118019 container create 69ede3e2f1a9b5b56250c1036c5553e5cfda8dfbaf2d8d1eaadc33f0d2c7dcd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-09-19T08:27:03.836 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:27:03 smithi064 podman[70703]: 2024-09-19 08:27:03.459514987 +0000 UTC m=+0.416250114 container init 69ede3e2f1a9b5b56250c1036c5553e5cfda8dfbaf2d8d1eaadc33f0d2c7dcd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-19T08:27:03.837 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:27:03 smithi064 podman[70703]: 2024-09-19 08:27:03.463020855 +0000 UTC m=+0.419755982 container start 69ede3e2f1a9b5b56250c1036c5553e5cfda8dfbaf2d8d1eaadc33f0d2c7dcd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T08:27:03.837 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:27:03 smithi064 podman[70703]: 2024-09-19 08:27:03.559694754 +0000 UTC m=+0.516429881 container attach 69ede3e2f1a9b5b56250c1036c5553e5cfda8dfbaf2d8d1eaadc33f0d2c7dcd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T08:27:03.837 INFO:journalctl@ceph.osd.1.smithi064.stdout:Sep 19 08:27:03 smithi064 podman[70703]: 2024-09-19 08:27:03.657885466 +0000 UTC m=+0.614620614 container died 69ede3e2f1a9b5b56250c1036c5553e5cfda8dfbaf2d8d1eaadc33f0d2c7dcd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T08:27:04.248 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T08:27:04.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:27:04.287 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T08:27:04.287 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T08:27:04.288 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.2 2024-09-19T08:27:04.624 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:27:04 smithi157 systemd[1]: Stopping Ceph osd.2 for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:27:05.003 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:27:04 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2[40099]: 2024-09-19T08:27:04.622+0000 7f446f568640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:27:05.003 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:27:04 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2[40099]: 2024-09-19T08:27:04.622+0000 7f446f568640 -1 osd.2 99 *** Got signal Terminated *** 2024-09-19T08:27:05.003 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:27:04 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2[40099]: 2024-09-19T08:27:04.622+0000 7f446f568640 -1 osd.2 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:27:10.383 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:27:10 smithi157 podman[71474]: 2024-09-19 08:27:10.075030982 +0000 UTC m=+5.578628279 container died 8022bd2f4d650ba6c2d279699e4ccf6b148fb53791eca66980264ba9bb708521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T08:27:10.753 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:27:10 smithi157 podman[71474]: 2024-09-19 08:27:10.482155169 +0000 UTC m=+5.985752455 container cleanup 8022bd2f4d650ba6c2d279699e4ccf6b148fb53791eca66980264ba9bb708521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T08:27:10.753 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:27:10 smithi157 bash[71474]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2 2024-09-19T08:27:11.159 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:27:10 smithi157 podman[71491]: 2024-09-19 08:27:10.758418989 +0000 UTC m=+0.681563238 container remove 8022bd2f4d650ba6c2d279699e4ccf6b148fb53791eca66980264ba9bb708521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T08:27:11.411 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:27:11 smithi157 podman[71581]: 2024-09-19 08:27:11.062446461 +0000 UTC m=+0.016522722 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:27:11.411 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:27:11 smithi157 podman[71581]: 2024-09-19 08:27:11.184427492 +0000 UTC m=+0.138503749 container create 7db134ddb24464619e2595140a2b05afdb4302dc9cd528cb684181c22e77cba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T08:27:11.753 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:27:11 smithi157 podman[71581]: 2024-09-19 08:27:11.412218932 +0000 UTC m=+0.366295190 container init 7db134ddb24464619e2595140a2b05afdb4302dc9cd528cb684181c22e77cba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-19T08:27:11.754 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:27:11 smithi157 podman[71581]: 2024-09-19 08:27:11.415618437 +0000 UTC m=+0.369694708 container start 7db134ddb24464619e2595140a2b05afdb4302dc9cd528cb684181c22e77cba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T08:27:11.754 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:27:11 smithi157 podman[71581]: 2024-09-19 08:27:11.495649577 +0000 UTC m=+0.449725848 container attach 7db134ddb24464619e2595140a2b05afdb4302dc9cd528cb684181c22e77cba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-19T08:27:12.132 INFO:journalctl@ceph.osd.2.smithi157.stdout:Sep 19 08:27:11 smithi157 podman[71581]: 2024-09-19 08:27:11.770104672 +0000 UTC m=+0.724180933 container died 7db134ddb24464619e2595140a2b05afdb4302dc9cd528cb684181c22e77cba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2) 2024-09-19T08:27:12.445 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T08:27:12.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:27:12.484 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T08:27:12.484 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T08:27:12.484 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.3 2024-09-19T08:27:12.924 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:12 smithi022 systemd[1]: Stopping Ceph osd.3 for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:27:13.257 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:12 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3[51394]: 2024-09-19T08:27:12.923+0000 7fa16530f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:27:13.257 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:12 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3[51394]: 2024-09-19T08:27:12.923+0000 7fa16530f640 -1 osd.3 99 *** Got signal Terminated *** 2024-09-19T08:27:13.257 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:12 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3[51394]: 2024-09-19T08:27:12.923+0000 7fa16530f640 -1 osd.3 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:27:15.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:14 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:14.589+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:16.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:15 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:15.603+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:16.007 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:15 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3[51394]: 2024-09-19T08:27:15.596+0000 7fa15d91d640 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:48.768625+0000 front 2024-09-19T08:26:48.768648+0000 (oldest deadline 2024-09-19T08:27:14.668759+0000) 2024-09-19T08:27:16.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:15 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:15.516+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:16.757 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:16 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3[51394]: 2024-09-19T08:27:16.576+0000 7fa15d91d640 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:48.768625+0000 front 2024-09-19T08:26:48.768648+0000 (oldest deadline 2024-09-19T08:27:14.668759+0000) 2024-09-19T08:27:16.757 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:16 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:16.495+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:17.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:16 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:16.565+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:17.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:16 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:16.612+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:17.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:16 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:16.910+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:17.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:17 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:17.012+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:17.757 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:17 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:17.479+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:17.757 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:17 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3[51394]: 2024-09-19T08:27:17.559+0000 7fa15d91d640 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:48.768625+0000 front 2024-09-19T08:26:48.768648+0000 (oldest deadline 2024-09-19T08:27:14.668759+0000) 2024-09-19T08:27:17.972 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:17 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:17.530+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:18.086 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:17 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:17.599+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:18.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:17 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:17.972+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:18.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:17 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:17.872+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:18.699 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:18 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:18.444+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:18.699 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:18 smithi022 podman[109942]: 2024-09-19 08:27:18.405824693 +0000 UTC m=+5.711158286 container died e962e50610c1d4227c0d4b961e58a071bb25cd5af752868d9c220ff97e08ff51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T08:27:18.949 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:18 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:18.520+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:19.006 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:18 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:18.862+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:19.007 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:18 smithi022 podman[109942]: 2024-09-19 08:27:18.899236895 +0000 UTC m=+6.204570523 container cleanup e962e50610c1d4227c0d4b961e58a071bb25cd5af752868d9c220ff97e08ff51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T08:27:19.007 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:18 smithi022 bash[109942]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3 2024-09-19T08:27:19.086 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:18 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:18.593+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:19.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:18 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:18.948+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:19.439 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:19 smithi022 podman[109958]: 2024-09-19 08:27:19.175966962 +0000 UTC m=+0.770115964 container remove e962e50610c1d4227c0d4b961e58a071bb25cd5af752868d9c220ff97e08ff51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T08:27:19.696 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:19 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:19.439+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:19.697 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:19 smithi022 podman[110049]: 2024-09-19 08:27:19.434724105 +0000 UTC m=+0.016266982 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:27:19.697 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:19 smithi022 podman[110049]: 2024-09-19 08:27:19.551654937 +0000 UTC m=+0.133197813 container create 62928856f96855c711438a1fcc31d1ccf6c8dae2981fa896181deb5dbb5a3ffd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T08:27:19.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:19 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:19.484+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:19.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:19 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:19.283+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:20.086 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:19 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:19.643+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:20.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:19 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:19.973+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:20.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:19 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:19.890+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:20.257 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:19 smithi022 podman[110049]: 2024-09-19 08:27:19.929677224 +0000 UTC m=+0.511220117 container init 62928856f96855c711438a1fcc31d1ccf6c8dae2981fa896181deb5dbb5a3ffd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T08:27:20.257 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:19 smithi022 podman[110049]: 2024-09-19 08:27:19.933235839 +0000 UTC m=+0.514778716 container start 62928856f96855c711438a1fcc31d1ccf6c8dae2981fa896181deb5dbb5a3ffd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:27:20.257 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:20 smithi022 podman[110049]: 2024-09-19 08:27:20.004419186 +0000 UTC m=+0.585962070 container attach 62928856f96855c711438a1fcc31d1ccf6c8dae2981fa896181deb5dbb5a3ffd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:27:20.257 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 19 08:27:20 smithi022 podman[110049]: 2024-09-19 08:27:20.128357165 +0000 UTC m=+0.709900056 container died 62928856f96855c711438a1fcc31d1ccf6c8dae2981fa896181deb5dbb5a3ffd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T08:27:20.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:20 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:20.498+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:20.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:20 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:20.265+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:20.756 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:20 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:20.465+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:20.798 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T08:27:20.843 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:27:20.843 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T08:27:20.844 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T08:27:20.844 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.4 2024-09-19T08:27:21.087 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:20 smithi064 systemd[1]: Stopping Ceph osd.4 for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:27:21.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:20 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:20.671+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:21.253 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:21 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:21.236+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:21.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:21 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:21.000+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:21.256 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:20 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:20.846+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:21.587 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:21 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4[45813]: 2024-09-19T08:27:21.166+0000 7f5f78be6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:27:21.587 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:21 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4[45813]: 2024-09-19T08:27:21.166+0000 7f5f78be6640 -1 osd.4 99 *** Got signal Terminated *** 2024-09-19T08:27:21.587 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:21 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4[45813]: 2024-09-19T08:27:21.166+0000 7f5f78be6640 -1 osd.4 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:27:21.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:21 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:21.466+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:21.756 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:21 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:21.460+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:22.086 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:21 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:21.642+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:22.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:22 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:22.000+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:22.253 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:22 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:22.250+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:22.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:21 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:21.833+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:22.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:22 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:22.428+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:22.756 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:22 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:22.411+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:22.756 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:22 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:22.411+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:23.086 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:22 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:22.683+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:23.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:22 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:22.683+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:23.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:22 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:22.866+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:23.445 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:23 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:23.277+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:23.445 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:23 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:23.030+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:23.445 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:23 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:23.030+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:23.748 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:23 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:23.445+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:23.756 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:23 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:23.378+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:23.757 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:23 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:23.378+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:24.086 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:23 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:23.681+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:24.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:23 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:23.681+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:24.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:23 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:23.872+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:24.324 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:24 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:24.056+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:24.325 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:24 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:24.056+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:24.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:24 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:24.324+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:24.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:24 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:24.324+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.559029+0000 front 2024-09-19T08:27:00.558992+0000 (oldest deadline 2024-09-19T08:27:24.059095+0000) 2024-09-19T08:27:24.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:24 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:24.482+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:24.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:24 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:24.482+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:24.756 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:24 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:24.410+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:24.756 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:24 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:24.410+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:24.837 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:24 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4[45813]: 2024-09-19T08:27:24.711+0000 7f5f751fc640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.919917+0000 front 2024-09-19T08:26:58.919966+0000 (oldest deadline 2024-09-19T08:27:24.220130+0000) 2024-09-19T08:27:24.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:24 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:24.676+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:24.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:24 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:24.676+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:24.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:24 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:24.542+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:25.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:24 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:24.870+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:25.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:24 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:24.870+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:25.367 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:25 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:25.013+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:25.367 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:25 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:25.013+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:25.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:25 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:25.367+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:25.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:25 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:25.367+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.559029+0000 front 2024-09-19T08:27:00.558992+0000 (oldest deadline 2024-09-19T08:27:24.059095+0000) 2024-09-19T08:27:25.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:25 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:25.440+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:25.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:25 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:25.440+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:25.757 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:25 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:25.455+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:25.757 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:25 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:25.455+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:25.836 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:25 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4[45813]: 2024-09-19T08:27:25.758+0000 7f5f751fc640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.919917+0000 front 2024-09-19T08:26:58.919966+0000 (oldest deadline 2024-09-19T08:27:24.220130+0000) 2024-09-19T08:27:25.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:25 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:25.690+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:25.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:25 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:25.690+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:25.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:25 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:25.582+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:26.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:25 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:25.866+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:26.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:25 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:25.866+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:26.346 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:26 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:26.058+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:26.346 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:26 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:26.058+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:26.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:26 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:26.346+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:26.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:26 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:26.346+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.559029+0000 front 2024-09-19T08:27:00.558992+0000 (oldest deadline 2024-09-19T08:27:24.059095+0000) 2024-09-19T08:27:26.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:26 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:26.417+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:26.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:26 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:26.417+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:26.756 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:26 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:26.448+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:26.756 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:26 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:26.448+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:26.836 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:26 smithi064 podman[70805]: 2024-09-19 08:27:26.581894424 +0000 UTC m=+5.542655902 container died 5850ed38c0eefdd6b1ee961ed575f92a0329ebbd1833401b774bc8e05c09a542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T08:27:26.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:26 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:26.557+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:26.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:26 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:26.709+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:26.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:26 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:26.709+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:27.167 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:26 smithi064 podman[70805]: 2024-09-19 08:27:26.983560127 +0000 UTC m=+5.944321598 container cleanup 5850ed38c0eefdd6b1ee961ed575f92a0329ebbd1833401b774bc8e05c09a542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T08:27:27.167 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:26 smithi064 bash[70805]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4 2024-09-19T08:27:27.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:26 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:26.885+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:27.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:26 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:26.885+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:27.367 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:27 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:27.088+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:27.367 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:27 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:27.088+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:27.583 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:27 smithi064 podman[70822]: 2024-09-19 08:27:27.168394178 +0000 UTC m=+0.586794287 container remove 5850ed38c0eefdd6b1ee961ed575f92a0329ebbd1833401b774bc8e05c09a542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T08:27:27.754 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:27 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:27.367+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:27.754 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:27 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:27.367+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.559029+0000 front 2024-09-19T08:27:00.558992+0000 (oldest deadline 2024-09-19T08:27:24.059095+0000) 2024-09-19T08:27:27.754 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:27 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:27.398+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:27.754 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:27 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:27.398+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:27.756 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:27 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:27.492+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:27.756 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:27 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:27.492+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:27.836 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:27 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:27.583+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:27.837 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:27 smithi064 podman[70919]: 2024-09-19 08:27:27.536511145 +0000 UTC m=+0.016584214 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:27:27.837 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:27 smithi064 podman[70919]: 2024-09-19 08:27:27.660882379 +0000 UTC m=+0.140955442 container create b223570c6f30bb299d127712d09ee03c4463e1d517a058e444f4e2c8a0393185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T08:27:27.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:27 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:27.695+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:27.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:27 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:27.695+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:28.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:27 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:27.935+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:28.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:27 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:27.935+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:28.337 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:27 smithi064 podman[70919]: 2024-09-19 08:27:27.888381177 +0000 UTC m=+0.368454244 container init b223570c6f30bb299d127712d09ee03c4463e1d517a058e444f4e2c8a0393185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T08:27:28.337 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:27 smithi064 podman[70919]: 2024-09-19 08:27:27.891861421 +0000 UTC m=+0.371934477 container start b223570c6f30bb299d127712d09ee03c4463e1d517a058e444f4e2c8a0393185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T08:27:28.337 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:27 smithi064 podman[70919]: 2024-09-19 08:27:27.980122388 +0000 UTC m=+0.460195450 container attach b223570c6f30bb299d127712d09ee03c4463e1d517a058e444f4e2c8a0393185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T08:27:28.338 INFO:journalctl@ceph.osd.4.smithi064.stdout:Sep 19 08:27:28 smithi064 podman[70919]: 2024-09-19 08:27:28.10763114 +0000 UTC m=+0.587704206 container died b223570c6f30bb299d127712d09ee03c4463e1d517a058e444f4e2c8a0393185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T08:27:28.375 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:28 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:28.044+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:28.375 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:28 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:28.044+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:28.722 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:28 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:28.557+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:28.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:28 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:28.375+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:28.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:28 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:28.375+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:28.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:28 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:28.376+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:28.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:28 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:28.376+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.559029+0000 front 2024-09-19T08:27:00.558992+0000 (oldest deadline 2024-09-19T08:27:24.059095+0000) 2024-09-19T08:27:28.778 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T08:27:28.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:27:28.820 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T08:27:28.820 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T08:27:28.820 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.5 2024-09-19T08:27:28.920 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:28 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:28.533+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:28.920 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:28 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:28.533+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:29.086 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:28 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:28.722+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:29.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:28 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:28.722+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:29.134 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:29 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:29.065+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:29.134 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:29 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:29.065+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:29.135 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:28 smithi157 systemd[1]: Stopping Ceph osd.5 for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:27:29.135 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:29 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:29.134+0000 7f32d83e4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:27:29.135 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:29 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:29.134+0000 7f32d83e4640 -1 osd.5 99 *** Got signal Terminated *** 2024-09-19T08:27:29.135 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:29 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:29.134+0000 7f32d83e4640 -1 osd.5 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:27:29.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:28 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:28.920+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:29.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:28 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:28.920+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:29.392 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:29 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:29.343+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:29.393 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:29 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:29.343+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.559029+0000 front 2024-09-19T08:27:00.558992+0000 (oldest deadline 2024-09-19T08:27:24.059095+0000) 2024-09-19T08:27:29.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:29 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:29.391+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:29.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:29 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:29.391+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:29.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:29 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:29.571+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:29.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:29 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:29.712+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:29.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:29 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:29.712+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:29.930 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:29 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:29.531+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:29.930 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:29 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:29.531+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:30.256 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:29 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:29.930+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:30.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:29 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:29.930+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:30.388 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:30 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:30.115+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:30.388 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:30 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:30.115+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:30.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:30 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:30.388+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:30.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:30 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:30.388+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.559029+0000 front 2024-09-19T08:27:00.558992+0000 (oldest deadline 2024-09-19T08:27:24.059095+0000) 2024-09-19T08:27:30.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:30 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:30.416+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:30.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:30 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:30.416+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:30.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:30 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:30.416+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:30.836 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:30 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:30.551+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:30.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:30 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:30.674+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:30.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:30 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:30.674+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:30.975 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:30 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:30.536+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:30.976 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:30 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:30.536+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:31.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:30 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:30.975+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:31.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:30 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:30.975+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:31.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:30 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:30.975+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.800061+0000 front 2024-09-19T08:27:05.800133+0000 (oldest deadline 2024-09-19T08:27:30.400531+0000) 2024-09-19T08:27:31.342 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:31.068+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:31.342 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:31.068+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:31.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:31.343+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:31.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:31.343+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.559029+0000 front 2024-09-19T08:27:00.558992+0000 (oldest deadline 2024-09-19T08:27:24.059095+0000) 2024-09-19T08:27:31.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:31.443+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:31.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:31.443+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:31.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:31 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:31.443+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:31.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:31 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:31.554+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:31.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:31 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:31.654+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:31.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:31 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:31.654+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:31.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:31 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:31.654+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:31.951 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:31 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:31.581+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:31.952 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:31 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:31.581+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:32.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:31 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:31.951+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:32.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:31 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:31.951+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:32.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:31 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:31.951+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.800061+0000 front 2024-09-19T08:27:05.800133+0000 (oldest deadline 2024-09-19T08:27:30.400531+0000) 2024-09-19T08:27:32.343 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:32 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:32.052+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:32.343 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:32 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:32.052+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:32.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:32 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:32.343+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:32.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:32 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:32.343+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.559029+0000 front 2024-09-19T08:27:00.558992+0000 (oldest deadline 2024-09-19T08:27:24.059095+0000) 2024-09-19T08:27:32.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:32 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:32.343+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.260510+0000 front 2024-09-19T08:27:06.260543+0000 (oldest deadline 2024-09-19T08:27:31.560609+0000) 2024-09-19T08:27:32.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:32 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:32.478+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:32.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:32 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:32.478+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:32.754 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:32 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:32.478+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:32.836 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:32 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:32.526+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:32.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:32 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:32.650+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:32.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:32 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:32.650+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:32.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:32 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:32.650+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:32.946 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:32 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:32.591+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:32.946 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:32 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:32.591+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:32.946 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:32 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:32.591+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:33.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:32 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:32.946+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:33.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:32 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:32.946+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:33.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:32 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:32.946+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.800061+0000 front 2024-09-19T08:27:05.800133+0000 (oldest deadline 2024-09-19T08:27:30.400531+0000) 2024-09-19T08:27:33.335 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:33 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:33.034+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:33.335 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:33 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:33.034+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:33.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:33 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:33.335+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.556838+0000 front 2024-09-19T08:26:52.556708+0000 (oldest deadline 2024-09-19T08:27:18.957428+0000) 2024-09-19T08:27:33.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:33 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:33.335+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.559029+0000 front 2024-09-19T08:27:00.558992+0000 (oldest deadline 2024-09-19T08:27:24.059095+0000) 2024-09-19T08:27:33.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:33 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5[46999]: 2024-09-19T08:27:33.335+0000 7f32d09f2640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.260510+0000 front 2024-09-19T08:27:06.260543+0000 (oldest deadline 2024-09-19T08:27:31.560609+0000) 2024-09-19T08:27:33.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:33 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:33.496+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:33.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:33 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:33.496+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:33.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:33 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:33.496+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:33.836 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:33 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:33.550+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:33.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:33 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:33.664+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:33.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:33 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:33.664+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:33.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:33 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:33.664+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:33.972 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:33 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:33.614+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:33.972 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:33 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:33.614+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:33.972 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:33 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:33.614+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:34.256 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:33 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:33.972+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:34.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:33 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:33.972+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:34.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:33 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:33.972+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.800061+0000 front 2024-09-19T08:27:05.800133+0000 (oldest deadline 2024-09-19T08:27:30.400531+0000) 2024-09-19T08:27:34.476 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:34 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:34.047+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:34.476 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:34 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:34.047+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:34.753 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:34 smithi157 podman[71696]: 2024-09-19 08:27:34.589213952 +0000 UTC m=+5.570433911 container died 44655cee5d6f6d6f42e13e1d9f1ec67d754f7f25a15e40681dc0a896db12a703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T08:27:34.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:34 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:34.475+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:34.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:34 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:34.475+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:34.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:34 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:34.475+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:34.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:34 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:34.576+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:34.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:34 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:34.676+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:34.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:34 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:34.676+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:34.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:34 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:34.676+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:35.003 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:34 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:34.635+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:35.004 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:34 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:34.635+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:35.004 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:34 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:34.635+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:35.244 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:35 smithi157 podman[71696]: 2024-09-19 08:27:35.00183313 +0000 UTC m=+5.983053118 container cleanup 44655cee5d6f6d6f42e13e1d9f1ec67d754f7f25a15e40681dc0a896db12a703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T08:27:35.245 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:35 smithi157 bash[71696]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5 2024-09-19T08:27:35.245 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:35 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:35.063+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:35.245 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:35 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:35.063+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:35.245 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:35 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:35.063+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:35.256 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:35 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:35.003+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:35.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:35 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:35.003+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:35.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:35 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:35.003+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.800061+0000 front 2024-09-19T08:27:05.800133+0000 (oldest deadline 2024-09-19T08:27:30.400531+0000) 2024-09-19T08:27:35.503 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:35 smithi157 podman[71716]: 2024-09-19 08:27:35.24487716 +0000 UTC m=+0.637567090 container remove 44655cee5d6f6d6f42e13e1d9f1ec67d754f7f25a15e40681dc0a896db12a703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T08:27:35.503 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:35 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:35.438+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:35.503 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:35 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:35.438+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:35.503 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:35 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:35.438+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:35.922 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:35 smithi157 podman[71800]: 2024-09-19 08:27:35.570880629 +0000 UTC m=+0.015988474 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:27:35.923 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:35 smithi157 podman[71800]: 2024-09-19 08:27:35.695726767 +0000 UTC m=+0.140834598 container create e18a69a0644c804f0385c027a9f8d14ca808182af984812a276fff1d9ea37c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T08:27:36.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:35 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:35.679+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:36.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:35 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:35.679+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:36.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:35 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:35.679+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:36.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:35 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:35.624+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:36.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:35 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:35.624+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:36.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:35 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:35.709+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:36.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:35 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:35.709+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:36.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:35 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:35.709+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:36.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:36 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:36.053+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:36.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:36 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:36.053+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:36.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:36 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:36.053+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:36.254 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:35 smithi157 podman[71800]: 2024-09-19 08:27:35.923347628 +0000 UTC m=+0.368455459 container init e18a69a0644c804f0385c027a9f8d14ca808182af984812a276fff1d9ea37c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T08:27:36.254 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:35 smithi157 podman[71800]: 2024-09-19 08:27:35.926697573 +0000 UTC m=+0.371805407 container start e18a69a0644c804f0385c027a9f8d14ca808182af984812a276fff1d9ea37c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T08:27:36.254 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:36 smithi157 podman[71800]: 2024-09-19 08:27:36.031700655 +0000 UTC m=+0.476808490 container attach e18a69a0644c804f0385c027a9f8d14ca808182af984812a276fff1d9ea37c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T08:27:36.254 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 19 08:27:36 smithi157 podman[71800]: 2024-09-19 08:27:36.116557488 +0000 UTC m=+0.561665330 container died e18a69a0644c804f0385c027a9f8d14ca808182af984812a276fff1d9ea37c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T08:27:36.506 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:36 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:36.049+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:36.507 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:36 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:36.049+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:36.507 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:36 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:36.049+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.800061+0000 front 2024-09-19T08:27:05.800133+0000 (oldest deadline 2024-09-19T08:27:30.400531+0000) 2024-09-19T08:27:36.693 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:36 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:36.450+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:36.694 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:36 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:36.450+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:36.694 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:36 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:36.450+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:36.723 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T08:27:36.765 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:27:36.765 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T08:27:36.765 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T08:27:36.765 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.6 2024-09-19T08:27:37.007 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:36 smithi022 systemd[1]: Stopping Ceph osd.6 for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:27:37.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:36 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:36.711+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:37.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:36 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:36.711+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:37.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:36 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:36.711+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:37.086 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:36 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:36.603+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:37.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:36 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:36.603+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:37.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:36 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:36.719+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:37.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:36 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:36.719+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:37.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:36 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:36.719+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:37.489 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:37 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:37.004+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:37.489 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:37 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:37.004+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:37.489 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:37 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:37.004+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:37.506 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:37.010+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:37.507 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:37.010+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:37.507 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:37.010+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.800061+0000 front 2024-09-19T08:27:05.800133+0000 (oldest deadline 2024-09-19T08:27:30.400531+0000) 2024-09-19T08:27:37.507 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:37.085+0000 7f79d37c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:27:37.507 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:37.085+0000 7f79d37c1640 -1 osd.6 99 *** Got signal Terminated *** 2024-09-19T08:27:37.507 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:37.085+0000 7f79d37c1640 -1 osd.6 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:27:37.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:37 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:37.489+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:37.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:37 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:37.489+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:37.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:37 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:37.489+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:37.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:37 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:37.619+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:37.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:37 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:37.619+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:37.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:37 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:37.673+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:37.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:37 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:37.673+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:37.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:37 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:37.673+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:37.979 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:37.666+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:37.979 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:37.666+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:37.979 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:37.666+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:38.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:38 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:37.999+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:38.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:38 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:37.999+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:38.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:38 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:37.999+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:38.256 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:37.979+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:38.256 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:37.979+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:38.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:37.979+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.800061+0000 front 2024-09-19T08:27:05.800133+0000 (oldest deadline 2024-09-19T08:27:30.400531+0000) 2024-09-19T08:27:38.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:38 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:38.462+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:38.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:38 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:38.462+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:38.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:38 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:38.462+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:39.007 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:38 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:38.929+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:39.007 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:38 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:38.929+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:39.007 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:38 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:38.929+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.800061+0000 front 2024-09-19T08:27:05.800133+0000 (oldest deadline 2024-09-19T08:27:30.400531+0000) 2024-09-19T08:27:39.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:38 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:38.700+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:39.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:38 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:38.700+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:39.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:38 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:38.700+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:39.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:38 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:38.617+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:39.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:38 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:38.617+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:39.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:38 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:38.696+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:39.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:38 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:38.696+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:39.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:38 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:38.696+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:39.497 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:39 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:39.036+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:39.497 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:39 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:39.036+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:39.497 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:39 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:39.036+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:39.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:39 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:39.497+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:39.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:39 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:39.497+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:39.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:39 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:39.497+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:39.922 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:39 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:39.658+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:39.922 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:39 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:39.658+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:39.923 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:39 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:39.658+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:40.086 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:39 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:39.614+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:40.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:39 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:39.614+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:40.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:39 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:39.723+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:40.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:39 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:39.723+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:40.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:39 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:39.723+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:40.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:39 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:39.995+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:40.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:39 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:39.995+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:40.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:39 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:39.995+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:40.256 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:39 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:39.922+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:40.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:39 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:39.922+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:40.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:39 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:39.922+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.800061+0000 front 2024-09-19T08:27:05.800133+0000 (oldest deadline 2024-09-19T08:27:30.400531+0000) 2024-09-19T08:27:40.935 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:40 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:40.612+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:40.935 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:40 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:40.612+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:40.935 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:40 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:40.612+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:41.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:40 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:40.521+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:41.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:40 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:40.521+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:41.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:40 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:40.521+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:41.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:40 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:40.637+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:41.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:40 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:40.637+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:41.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:40 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:40.637+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.738188+0000 front 2024-09-19T08:27:17.738173+0000 (oldest deadline 2024-09-19T08:27:40.038011+0000) 2024-09-19T08:27:41.088 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:40 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:40.718+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:41.088 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:40 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:40.718+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:41.088 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:40 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:40.718+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:41.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:40 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:40.934+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:41.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:40 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:40.934+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:41.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:40 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:40.934+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.800061+0000 front 2024-09-19T08:27:05.800133+0000 (oldest deadline 2024-09-19T08:27:30.400531+0000) 2024-09-19T08:27:41.478 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:41 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:41.035+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:41.478 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:41 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:41.035+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:41.479 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:41 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:41.035+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:41.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:41 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:41.479+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:41.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:41 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:41.479+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:41.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:41 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:41.479+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:41.908 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:41.578+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:41.908 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:41.578+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:41.909 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:41.578+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:41.909 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:41.578+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:42.086 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:41 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:41.602+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:42.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:41 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:41.602+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:42.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:41 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:41.602+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.738188+0000 front 2024-09-19T08:27:17.738173+0000 (oldest deadline 2024-09-19T08:27:40.038011+0000) 2024-09-19T08:27:42.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:41 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:41.756+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:42.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:41 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:41.756+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:42.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:41 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:41.756+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:42.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:41.908+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:49.496415+0000 front 2024-09-19T08:26:49.496445+0000 (oldest deadline 2024-09-19T08:27:16.497264+0000) 2024-09-19T08:27:42.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:41.908+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:01.198461+0000 front 2024-09-19T08:27:01.198526+0000 (oldest deadline 2024-09-19T08:27:24.098563+0000) 2024-09-19T08:27:42.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:41.908+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.800061+0000 front 2024-09-19T08:27:05.800133+0000 (oldest deadline 2024-09-19T08:27:30.400531+0000) 2024-09-19T08:27:42.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:41 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6[57889]: 2024-09-19T08:27:41.908+0000 7f79cbdcf640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.402176+0000 front 2024-09-19T08:27:17.402201+0000 (oldest deadline 2024-09-19T08:27:41.502406+0000) 2024-09-19T08:27:42.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:42 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:42.004+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:42.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:42 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:42.004+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:42.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:42 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:42.004+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:42.836 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:42 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:42.573+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:42.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:42 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:42.573+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:42.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:42 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:42.573+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.738188+0000 front 2024-09-19T08:27:17.738173+0000 (oldest deadline 2024-09-19T08:27:40.038011+0000) 2024-09-19T08:27:42.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:42 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:42.738+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:42.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:42 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:42.738+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:42.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:42 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:42.738+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:42.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:42 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:42.738+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:42.994 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:42 smithi022 podman[110151]: 2024-09-19 08:27:42.57928079 +0000 UTC m=+5.607794412 container died 4ca125d64fd09f22794e3e130fa560fd694a88ef5deb0f6dcc21411c29f01c36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T08:27:42.994 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:42 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:42.550+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:42.994 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:42 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:42.550+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:42.994 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:42 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:42.550+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:42.994 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:42 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:42.550+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:43.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:42 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:42.522+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:43.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:42 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:42.522+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:43.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:42 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:42.522+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:43.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:42 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:42.522+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:43.256 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:43 smithi022 podman[110151]: 2024-09-19 08:27:43.18527251 +0000 UTC m=+6.213786125 container cleanup 4ca125d64fd09f22794e3e130fa560fd694a88ef5deb0f6dcc21411c29f01c36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-19T08:27:43.257 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:43 smithi022 bash[110151]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6 2024-09-19T08:27:43.504 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:43 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:43.049+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:43.504 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:43 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:43.049+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:43.504 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:43 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:43.049+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:43.757 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:43 smithi022 podman[110166]: 2024-09-19 08:27:43.436744577 +0000 UTC m=+0.850956385 container remove 4ca125d64fd09f22794e3e130fa560fd694a88ef5deb0f6dcc21411c29f01c36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T08:27:43.757 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:43 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:43.536+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:43.757 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:43 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:43.536+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:43.757 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:43 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:43.536+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:43.757 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:43 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:43.536+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:43.836 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:43 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:43.541+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:43.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:43 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:43.541+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:43.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:43 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:43.541+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.738188+0000 front 2024-09-19T08:27:17.738173+0000 (oldest deadline 2024-09-19T08:27:40.038011+0000) 2024-09-19T08:27:43.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:43 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:43.764+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:43.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:43 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:43.764+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:43.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:43 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:43.764+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:43.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:43 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:43.764+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:44.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:43 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:43.532+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:44.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:43 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:43.532+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:44.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:43 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:43.532+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:44.004 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:43 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:43.532+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:44.156 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:43 smithi022 podman[110254]: 2024-09-19 08:27:43.669312119 +0000 UTC m=+0.016738010 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:27:44.156 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:43 smithi022 podman[110254]: 2024-09-19 08:27:43.779348963 +0000 UTC m=+0.126774881 container create e311898d629afdce52c97b78a9ea1713b45af4e7624c44d134fa383132e1e29a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T08:27:44.156 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:44 smithi022 podman[110254]: 2024-09-19 08:27:44.156741647 +0000 UTC m=+0.504167524 container init e311898d629afdce52c97b78a9ea1713b45af4e7624c44d134fa383132e1e29a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T08:27:44.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:44 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:44.036+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:44.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:44 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:44.036+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:44.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:44 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:44.036+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:44.507 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:44 smithi022 podman[110254]: 2024-09-19 08:27:44.160419547 +0000 UTC m=+0.507845424 container start e311898d629afdce52c97b78a9ea1713b45af4e7624c44d134fa383132e1e29a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-19T08:27:44.507 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:44 smithi022 podman[110254]: 2024-09-19 08:27:44.248320873 +0000 UTC m=+0.595746766 container attach e311898d629afdce52c97b78a9ea1713b45af4e7624c44d134fa383132e1e29a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T08:27:44.507 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 19 08:27:44 smithi022 podman[110254]: 2024-09-19 08:27:44.352871923 +0000 UTC m=+0.700297816 container died e311898d629afdce52c97b78a9ea1713b45af4e7624c44d134fa383132e1e29a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-19T08:27:44.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:44 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:44.545+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:44.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:44 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:44.545+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:44.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:44 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:44.545+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.738188+0000 front 2024-09-19T08:27:17.738173+0000 (oldest deadline 2024-09-19T08:27:40.038011+0000) 2024-09-19T08:27:44.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:44 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:44.753+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:44.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:44 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:44.753+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:44.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:44 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:44.753+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:44.838 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:44 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:44.753+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:44.926 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:44.518+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:44.927 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:44.519+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:44.927 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:44.519+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:44.927 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:44 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:44.519+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:44.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T08:27:44.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:27:44.982 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T08:27:44.982 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T08:27:44.982 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.7 2024-09-19T08:27:45.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:44 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:44.516+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:45.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:44 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:44.516+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:45.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:44 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:44.516+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:45.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:44 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:44.516+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:45.294 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:45 smithi064 systemd[1]: Stopping Ceph osd.7 for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:27:45.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:45 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:45.026+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:45.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:45 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:45.026+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:45.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:45 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:45.026+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:45.566 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:45 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:45.293+0000 7f759b0ac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:27:45.566 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:45 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:45.293+0000 7f759b0ac640 -1 osd.7 99 *** Got signal Terminated *** 2024-09-19T08:27:45.566 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:45 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:45.293+0000 7f759b0ac640 -1 osd.7 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:27:45.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:45 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:45.564+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:45.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:45 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:45.564+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:45.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:45 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:45.564+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.738188+0000 front 2024-09-19T08:27:17.738173+0000 (oldest deadline 2024-09-19T08:27:40.038011+0000) 2024-09-19T08:27:45.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:45 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:45.752+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:45.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:45 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:45.752+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:45.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:45 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:45.752+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:45.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:45 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:45.752+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:46.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:45 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:45.515+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:46.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:45 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:45.515+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:46.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:45 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:45.516+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:46.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:45 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:45.516+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:46.006 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:45 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:45.556+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:46.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:45 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:45.556+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:46.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:45 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:45.556+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:46.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:45 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:45.556+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:46.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:46 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:46.058+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:46.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:46 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:46.058+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:46.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:46 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:46.058+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:46.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:46 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:46.584+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:46.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:46 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:46.584+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:46.837 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:46 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:46.584+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.738188+0000 front 2024-09-19T08:27:17.738173+0000 (oldest deadline 2024-09-19T08:27:40.038011+0000) 2024-09-19T08:27:46.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:46 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:46.733+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:46.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:46 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:46.733+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:46.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:46 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:46.733+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:46.838 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:46 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:46.733+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:47.004 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:46 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:46.546+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:47.004 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:46 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:46.546+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:47.004 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:46 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:46.546+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:47.005 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:46 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:46.546+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:47.006 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:46 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:46.587+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:47.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:46 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:46.587+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:47.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:46 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:46.587+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:47.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:46 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:46.587+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:47.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:47 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:47.059+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:47.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:47 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:47.059+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:47.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:47 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:47.059+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:48.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:47 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:47.594+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:48.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:47 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:47.594+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:48.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:47 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:47.594+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:48.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:47 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:47.594+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:48.006 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:47.624+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:48.006 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:47.624+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:48.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:47.624+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:48.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:47 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:47.624+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:48.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:47 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:47.592+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:48.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:47 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:47.592+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:48.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:47 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:47.592+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.738188+0000 front 2024-09-19T08:27:17.738173+0000 (oldest deadline 2024-09-19T08:27:40.038011+0000) 2024-09-19T08:27:48.088 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:47 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:47.709+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:48.088 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:47 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:47.709+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:48.088 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:47 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:47.709+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:48.088 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:47 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:47.709+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:48.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:48 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:48.049+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:48.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:48 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:48.049+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:48.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:48 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:48.049+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:49.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:48 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:48.601+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:49.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:48 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:48.601+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:49.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:48 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:48.601+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:49.003 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:48 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:48.601+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:49.006 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:48 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:48.601+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:49.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:48 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:48.601+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:49.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:48 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:48.601+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:49.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:48 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:48.601+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:49.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:48 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:48.632+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:49.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:48 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:48.632+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:49.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:48 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:48.632+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.738188+0000 front 2024-09-19T08:27:17.738173+0000 (oldest deadline 2024-09-19T08:27:40.038011+0000) 2024-09-19T08:27:49.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:48 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:48.755+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:49.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:48 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:48.755+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:49.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:48 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:48.755+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:49.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:48 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:48.755+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:49.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:48 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:48.755+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:27:49.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:49 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:49.019+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:49.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:49 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:49.019+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:49.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:49 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:49.019+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:49.970 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:49 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:49.647+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:49.971 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:49 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:49.647+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:49.971 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:49 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:49.647+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:49.971 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:49 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:49.647+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:50.006 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:49 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:49.634+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:50.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:49 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:49.634+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:50.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:49 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:49.634+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:50.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:49 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:49.634+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:50.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:49 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:49.624+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.135605+0000 front 2024-09-19T08:27:00.135577+0000 (oldest deadline 2024-09-19T08:27:23.635792+0000) 2024-09-19T08:27:50.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:49 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:49.624+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.536887+0000 front 2024-09-19T08:27:09.536962+0000 (oldest deadline 2024-09-19T08:27:35.436896+0000) 2024-09-19T08:27:50.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:49 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7[52316]: 2024-09-19T08:27:49.624+0000 7f75976c2640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.738188+0000 front 2024-09-19T08:27:17.738173+0000 (oldest deadline 2024-09-19T08:27:40.038011+0000) 2024-09-19T08:27:50.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:49 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:49.711+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:50.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:49 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:49.711+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:50.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:49 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:49.711+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:50.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:49 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:49.711+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:50.088 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:49 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:49.711+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:27:50.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:49 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:49.969+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:50.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:49 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:49.969+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:50.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:49 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:49.969+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:50.988 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:50 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:50.634+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:50.988 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:50 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:50.634+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:50.989 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:50 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:50.634+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:50.989 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:50 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:50.634+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:51.006 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:50 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:50.588+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:51.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:50 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:50.588+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:51.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:50 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:50.588+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:51.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:50 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:50.588+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:51.089 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:50 smithi064 podman[71024]: 2024-09-19 08:27:50.814539371 +0000 UTC m=+5.640980244 container died 949b1a1772859c496e8cef61a171bb5db3ab1954913ea504eff114ce780317fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-09-19T08:27:51.089 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:50 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:50.681+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:51.089 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:50 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:50.681+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:51.089 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:50 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:50.681+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:51.089 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:50 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:50.681+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:51.089 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:50 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:50.681+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:27:51.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:50 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:50.988+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:51.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:50 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:50.988+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:51.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:50 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:50.988+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:51.488 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:51 smithi064 podman[71024]: 2024-09-19 08:27:51.203209897 +0000 UTC m=+6.029650769 container cleanup 949b1a1772859c496e8cef61a171bb5db3ab1954913ea504eff114ce780317fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T08:27:51.488 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:51 smithi064 bash[71024]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7 2024-09-19T08:27:51.620 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:51 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:51.619+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:51.620 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:51 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:51.619+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:51.620 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:51 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:51.619+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:51.620 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:51 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:51.619+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:51.827 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:51 smithi064 podman[71039]: 2024-09-19 08:27:51.488118641 +0000 UTC m=+0.672500304 container remove 949b1a1772859c496e8cef61a171bb5db3ab1954913ea504eff114ce780317fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T08:27:51.828 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:51 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:51.666+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:51.828 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:51 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:51.666+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:51.828 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:51 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:51.666+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:51.828 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:51 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:51.666+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:51.828 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:51 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:51.666+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:27:51.940 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:51 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:51.616+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:51.941 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:51 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:51.616+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:51.941 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:51 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:51.616+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:51.941 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:51 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:51.616+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:52.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:51 smithi064 podman[71129]: 2024-09-19 08:27:51.730675027 +0000 UTC m=+0.016397723 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:27:52.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:51 smithi064 podman[71129]: 2024-09-19 08:27:51.838942598 +0000 UTC m=+0.124665330 container create 623a512039d4d00891f20468f34fdd1f1dd5920dc894b77082e40a7be48d32e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T08:27:52.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:52 smithi064 podman[71129]: 2024-09-19 08:27:52.057901225 +0000 UTC m=+0.343623914 container init 623a512039d4d00891f20468f34fdd1f1dd5920dc894b77082e40a7be48d32e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T08:27:52.087 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:52 smithi064 podman[71129]: 2024-09-19 08:27:52.061259325 +0000 UTC m=+0.346982011 container start 623a512039d4d00891f20468f34fdd1f1dd5920dc894b77082e40a7be48d32e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-19T08:27:52.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:51 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:51.940+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:52.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:51 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:51.940+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:52.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:51 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:51.940+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:52.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:51 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:51.940+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:27:52.560 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:52 smithi064 podman[71129]: 2024-09-19 08:27:52.158022898 +0000 UTC m=+0.443745595 container attach 623a512039d4d00891f20468f34fdd1f1dd5920dc894b77082e40a7be48d32e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True) 2024-09-19T08:27:52.560 INFO:journalctl@ceph.osd.7.smithi064.stdout:Sep 19 08:27:52 smithi064 podman[71129]: 2024-09-19 08:27:52.251620984 +0000 UTC m=+0.537343670 container died 623a512039d4d00891f20468f34fdd1f1dd5920dc894b77082e40a7be48d32e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:27:52.836 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:52 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:52.659+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:52.836 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:52 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:52.659+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:52.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:52 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:52.659+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:52.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:52 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:52.659+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:52.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:52 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:52.659+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:27:52.898 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T08:27:52.902 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:52 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:52.624+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:52.902 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:52 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:52.624+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:52.902 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:52 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:52.624+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:52.903 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:52 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:52.624+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:52.936 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:27:52.937 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T08:27:52.937 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T08:27:52.937 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.8 2024-09-19T08:27:53.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:52 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:52.626+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:53.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:52 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:52.626+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:53.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:52 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:52.626+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:53.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:52 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:52.626+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:53.253 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:52 smithi157 systemd[1]: Stopping Ceph osd.8 for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:27:53.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:52 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:52.902+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:53.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:52 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:52.902+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:53.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:52 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:52.902+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:53.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:52 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:52.902+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:27:53.618 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:53 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:53.253+0000 7fee821e7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:27:53.619 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:53 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:53.253+0000 7fee821e7640 -1 osd.8 99 *** Got signal Terminated *** 2024-09-19T08:27:53.619 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:53 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:53.253+0000 7fee821e7640 -1 osd.8 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:27:53.619 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:53 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:53.618+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:53.619 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:53 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:53.618+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:53.619 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:53 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:53.618+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:53.619 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:53 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:53.618+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:53.998 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:53 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:53.709+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:53.998 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:53 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:53.709+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:53.998 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:53 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:53.709+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:53.998 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:53 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:53.709+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:53.998 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:53 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:53.709+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:27:54.006 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:53 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:53.656+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:54.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:53 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:53.656+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:54.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:53 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:53.656+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:54.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:53 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:53.656+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:54.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:53 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:53.914+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:54.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:53 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:53.914+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:54.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:53 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:53.914+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:54.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:53 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:53.914+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:27:54.919 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:54 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:54.622+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:54.919 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:54 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:54.622+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:54.919 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:54 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:54.622+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:54.919 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:54 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:54.622+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:55.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:54 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:54.651+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:55.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:54 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:54.651+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:55.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:54 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:54.651+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:55.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:54 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:54.651+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:55.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:54 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:54.736+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:55.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:54 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:54.736+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:55.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:54 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:54.736+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:55.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:54 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:54.736+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:55.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:54 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:54.736+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:27:55.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:54 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:54.918+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:55.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:54 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:54.918+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:55.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:54 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:54.918+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:55.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:54 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:54.918+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:27:55.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:55 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:55.575+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:55.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:55 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:55.575+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:55.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:55 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:55.575+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:55.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:55 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:55.575+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:56.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:55 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:55.680+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:56.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:55 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:55.680+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:56.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:55 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:55.680+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:56.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:55 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:55.680+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:56.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:55 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:55.743+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:56.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:55 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:55.743+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:56.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:55 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:55.743+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:56.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:55 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:55.743+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:56.088 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:55 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:55.743+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:27:56.088 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:55 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:55.743+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:27:56.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:55 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:55.950+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:56.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:55 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:55.950+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:56.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:55 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:55.950+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:56.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:55 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:55.950+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:27:56.925 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:56.595+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:56.925 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:56.595+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:56.925 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:56.595+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:56.925 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:56.595+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:57.006 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:56 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:56.663+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:57.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:56 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:56.663+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:57.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:56 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:56.663+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:57.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:56 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:56.663+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:57.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:56.722+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:57.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:56.722+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:57.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:56.722+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:57.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:56.722+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:57.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:56.722+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:27:57.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:56 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:56.722+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:27:57.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:56.924+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:57.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:56.924+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:57.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:56.924+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:57.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:56 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:56.924+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:27:57.880 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:57 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:57.615+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:53.273487+0000 front 2024-09-19T08:26:53.273727+0000 (oldest deadline 2024-09-19T08:27:14.373620+0000) 2024-09-19T08:27:57.880 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:57 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:57.615+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:27:00.775158+0000 front 2024-09-19T08:27:00.775031+0000 (oldest deadline 2024-09-19T08:27:23.675048+0000) 2024-09-19T08:27:57.880 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:57 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:57.615+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:05.976055+0000 front 2024-09-19T08:27:05.976011+0000 (oldest deadline 2024-09-19T08:27:30.076128+0000) 2024-09-19T08:27:57.880 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:57 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:57.615+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.077613+0000 front 2024-09-19T08:27:17.077674+0000 (oldest deadline 2024-09-19T08:27:42.377474+0000) 2024-09-19T08:27:57.881 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:57 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8[53934]: 2024-09-19T08:27:57.615+0000 7fee7a7f5640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.279730+0000 front 2024-09-19T08:27:32.279749+0000 (oldest deadline 2024-09-19T08:27:56.880364+0000) 2024-09-19T08:27:58.006 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:57 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:57.677+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:58.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:57 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:57.677+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:58.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:57 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:57.677+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:58.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:57 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:57.677+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:58.086 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:57 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:57.680+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:58.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:57 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:57.680+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:58.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:57 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:57.680+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:58.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:57 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:57.680+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:58.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:57 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:57.680+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:27:58.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:57 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:57.680+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:27:58.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:57 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:57.879+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:58.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:57 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:57.879+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:58.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:57 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:57.879+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:58.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:57 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:57.879+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:27:58.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:57 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:57.879+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:27:58.996 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:58 smithi157 podman[71902]: 2024-09-19 08:27:58.688106363 +0000 UTC m=+5.551389752 container died 1150a2ee506067d40895c1339d4d45eec3f98cdf3b7b10bece8da98bd4cca1ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T08:27:58.997 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:58 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:58.867+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:27:58.997 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:58 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:58.867+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:27:58.997 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:58 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:58.867+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:27:58.997 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:58 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:58.867+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:27:58.997 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:58 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:58.867+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:27:59.006 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:58 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:58.695+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:27:59.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:58 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:58.695+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:27:59.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:58 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:58.695+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:27:59.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:58 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:58.695+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:27:59.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:58 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:58.695+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.381689+0000 front 2024-09-19T08:27:32.381722+0000 (oldest deadline 2024-09-19T08:27:58.281724+0000) 2024-09-19T08:27:59.086 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:58 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:58.676+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:27:59.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:58 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:58.676+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:27:59.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:58 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:58.676+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:27:59.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:58 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:58.676+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:27:59.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:58 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:58.676+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:27:59.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:58 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:58.676+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:27:59.253 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:59 smithi157 podman[71902]: 2024-09-19 08:27:59.113060433 +0000 UTC m=+5.976343817 container cleanup 1150a2ee506067d40895c1339d4d45eec3f98cdf3b7b10bece8da98bd4cca1ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T08:27:59.253 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:59 smithi157 bash[71902]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8 2024-09-19T08:27:59.753 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:59 smithi157 podman[71929]: 2024-09-19 08:27:59.372834542 +0000 UTC m=+0.678471110 container remove 1150a2ee506067d40895c1339d4d45eec3f98cdf3b7b10bece8da98bd4cca1ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T08:28:00.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:59 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:59.700+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:28:00.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:59 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:59.700+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:28:00.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:59 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:59.700+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:28:00.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:59 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:59.700+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:28:00.008 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:27:59 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:27:59.700+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.381689+0000 front 2024-09-19T08:27:32.381722+0000 (oldest deadline 2024-09-19T08:27:58.281724+0000) 2024-09-19T08:28:00.034 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:59 smithi157 podman[72017]: 2024-09-19 08:27:59.68369541 +0000 UTC m=+0.016216005 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:28:00.034 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:27:59 smithi157 podman[72017]: 2024-09-19 08:27:59.807043283 +0000 UTC m=+0.139563865 container create 8f5fdf9d30136826e4ba981bfa30919454d9f6611eb61f297b172389e87d3345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:28:00.034 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:28:00 smithi157 podman[72017]: 2024-09-19 08:28:00.03446603 +0000 UTC m=+0.366986611 container init 8f5fdf9d30136826e4ba981bfa30919454d9f6611eb61f297b172389e87d3345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T08:28:00.034 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:59.888+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:00.034 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:59.888+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:00.034 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:59.888+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:00.034 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:59.888+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:00.035 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:27:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:27:59.888+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:00.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:59.640+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:00.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:59.640+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:00.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:59.640+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:00.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:59.640+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:00.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:59.640+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:00.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:27:59 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:27:59.640+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:00.503 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:28:00 smithi157 podman[72017]: 2024-09-19 08:28:00.038891383 +0000 UTC m=+0.371411967 container start 8f5fdf9d30136826e4ba981bfa30919454d9f6611eb61f297b172389e87d3345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:28:00.503 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:28:00 smithi157 podman[72017]: 2024-09-19 08:28:00.142824678 +0000 UTC m=+0.475345278 container attach 8f5fdf9d30136826e4ba981bfa30919454d9f6611eb61f297b172389e87d3345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T08:28:00.504 INFO:journalctl@ceph.osd.8.smithi157.stdout:Sep 19 08:28:00 smithi157 podman[72017]: 2024-09-19 08:28:00.229897144 +0000 UTC m=+0.562417729 container died 8f5fdf9d30136826e4ba981bfa30919454d9f6611eb61f297b172389e87d3345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T08:28:00.951 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T08:28:00.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:28:00.995 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T08:28:00.995 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T08:28:00.995 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.9 2024-09-19T08:28:01.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:00.741+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:28:01.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:00.741+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:28:01.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:00.741+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:28:01.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:00.741+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:28:01.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:00 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:00.741+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.381689+0000 front 2024-09-19T08:27:32.381722+0000 (oldest deadline 2024-09-19T08:27:58.281724+0000) 2024-09-19T08:28:01.086 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:00 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:00.593+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:01.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:00 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:00.593+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:01.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:00 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:00.593+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:01.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:00 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:00.593+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:01.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:00 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:00.593+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:01.087 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:00 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:00.593+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:01.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:00 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:00.893+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:01.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:00 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:00.893+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:01.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:00 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:00.893+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:01.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:00 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:00.893+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:01.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:00 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:00.893+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:01.329 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:01 smithi022 systemd[1]: Stopping Ceph osd.9 for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:28:01.723 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:01.328+0000 7fcfdbab7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:28:01.724 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:01.328+0000 7fcfdbab7640 -1 osd.9 99 *** Got signal Terminated *** 2024-09-19T08:28:01.724 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:01.328+0000 7fcfdbab7640 -1 osd.9 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:28:01.724 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:01.724+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:28:01.724 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:01.724+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:28:01.724 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:01.724+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:28:01.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:01 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:01.550+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:01.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:01 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:01.550+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:01.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:01 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:01.550+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:01.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:01 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:01.550+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:01.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:01 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:01.550+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:01.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:01 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:01.550+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:02.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:01.724+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:28:02.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:01 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:01.724+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.381689+0000 front 2024-09-19T08:27:32.381722+0000 (oldest deadline 2024-09-19T08:27:58.281724+0000) 2024-09-19T08:28:02.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:01 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:01.901+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:02.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:01 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:01.901+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:02.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:01 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:01.901+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:02.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:01 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:01.901+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:02.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:01 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:01.901+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:02.836 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:02 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:02.556+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:02.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:02 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:02.556+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:02.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:02 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:02.556+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:02.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:02 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:02.556+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:02.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:02 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:02.556+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:02.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:02 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:02.556+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:03.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:02.682+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:28:03.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:02.682+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:28:03.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:02.682+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:28:03.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:02.682+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:28:03.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:02 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:02.682+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.381689+0000 front 2024-09-19T08:27:32.381722+0000 (oldest deadline 2024-09-19T08:27:58.281724+0000) 2024-09-19T08:28:03.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:02 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:02.873+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:03.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:02 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:02.873+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:03.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:02 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:02.873+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:03.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:02 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:02.873+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:03.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:02 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:02.873+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:03.836 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:03 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:03.551+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:03.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:03 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:03.551+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:03.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:03 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:03.551+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:03.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:03 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:03.551+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:03.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:03 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:03.551+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:03.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:03 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:03.551+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:04.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:03.650+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:28:04.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:03.650+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:28:04.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:03.650+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:28:04.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:03.650+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:28:04.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:03 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:03.650+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.381689+0000 front 2024-09-19T08:27:32.381722+0000 (oldest deadline 2024-09-19T08:27:58.281724+0000) 2024-09-19T08:28:04.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:03 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:03.903+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:04.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:03 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:03.903+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:04.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:03 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:03.903+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:04.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:03 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:03.903+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:04.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:03 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:03.903+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:04.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:04 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:04.580+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:04.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:04 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:04.580+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:04.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:04 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:04.580+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:04.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:04 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:04.580+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:04.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:04 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:04.580+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:04.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:04 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:04.580+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:05.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:04 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:04.682+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:28:05.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:04 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:04.682+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:28:05.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:04 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:04.682+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:28:05.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:04 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:04.682+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:28:05.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:04 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:04.682+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.381689+0000 front 2024-09-19T08:27:32.381722+0000 (oldest deadline 2024-09-19T08:27:58.281724+0000) 2024-09-19T08:28:05.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:04 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:04.902+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:05.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:04 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:04.902+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:05.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:04 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:04.903+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:05.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:04 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:04.903+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:05.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:04 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:04.903+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:05.587 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:05 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:05.565+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:05.606 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:05 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:05.565+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:05.606 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:05 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:05.565+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:05.606 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:05 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:05.565+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:05.606 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:05 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:05.565+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:05.606 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:05 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:05.565+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:05.606 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:05 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:05.565+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.929086+0000 front 2024-09-19T08:27:41.929104+0000 (oldest deadline 2024-09-19T08:28:05.429092+0000) 2024-09-19T08:28:06.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:05.726+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:51.774608+0000 front 2024-09-19T08:26:51.774580+0000 (oldest deadline 2024-09-19T08:27:15.274726+0000) 2024-09-19T08:28:06.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:05.726+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:57.575929+0000 front 2024-09-19T08:26:57.575883+0000 (oldest deadline 2024-09-19T08:27:22.275896+0000) 2024-09-19T08:28:06.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:05.726+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:07.576734+0000 front 2024-09-19T08:27:07.576757+0000 (oldest deadline 2024-09-19T08:27:31.676828+0000) 2024-09-19T08:28:06.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:05.726+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.279080+0000 front 2024-09-19T08:27:17.279163+0000 (oldest deadline 2024-09-19T08:27:41.379294+0000) 2024-09-19T08:28:06.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:05.726+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.381689+0000 front 2024-09-19T08:27:32.381722+0000 (oldest deadline 2024-09-19T08:27:58.281724+0000) 2024-09-19T08:28:06.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:05 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9[64450]: 2024-09-19T08:28:05.726+0000 7fcfd80cd640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.782718+0000 front 2024-09-19T08:27:41.782775+0000 (oldest deadline 2024-09-19T08:28:04.682817+0000) 2024-09-19T08:28:06.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:05 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:05.926+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:06.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:05 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:05.926+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:06.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:05 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:05.926+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:06.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:05 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:05.926+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:06.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:05 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:05.926+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:06.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:06 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:06.536+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:06.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:06 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:06.536+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:06.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:06 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:06.536+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:06.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:06 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:06.536+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:06.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:06 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:06.536+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:06.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:06 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:06.536+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:06.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:06 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:06.536+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.929086+0000 front 2024-09-19T08:27:41.929104+0000 (oldest deadline 2024-09-19T08:28:05.429092+0000) 2024-09-19T08:28:07.136 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:06 smithi022 podman[110361]: 2024-09-19 08:28:06.834558201 +0000 UTC m=+5.627020768 container died 62df57551882076a20f4e69019001229ed98f5388788e1276e94c494c4e64229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T08:28:07.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:06 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:06.930+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:07.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:06 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:06.930+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:07.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:06 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:06.930+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:07.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:06 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:06.930+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:07.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:06 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:06.930+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:07.662 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:07 smithi022 podman[110361]: 2024-09-19 08:28:07.394987643 +0000 UTC m=+6.187450194 container cleanup 62df57551882076a20f4e69019001229ed98f5388788e1276e94c494c4e64229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T08:28:07.662 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:07 smithi022 bash[110361]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9 2024-09-19T08:28:07.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:07 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:07.563+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:07.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:07 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:07.563+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:07.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:07 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:07.563+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:07.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:07 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:07.563+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:07.838 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:07 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:07.563+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:07.838 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:07 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:07.563+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:07.838 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:07 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:07.563+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.929086+0000 front 2024-09-19T08:27:41.929104+0000 (oldest deadline 2024-09-19T08:28:05.429092+0000) 2024-09-19T08:28:08.007 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:07 smithi022 podman[110377]: 2024-09-19 08:28:07.662853327 +0000 UTC m=+0.829711223 container remove 62df57551882076a20f4e69019001229ed98f5388788e1276e94c494c4e64229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T08:28:08.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:07 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:07.939+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:08.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:07 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:07.939+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:08.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:07 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:07.939+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:08.255 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:07 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:07.939+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:08.255 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:07 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:07.939+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:08.255 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:07 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:07.939+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:08.407 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:08 smithi022 podman[110466]: 2024-09-19 08:28:07.928164489 +0000 UTC m=+0.016445107 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:28:08.407 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:08 smithi022 podman[110466]: 2024-09-19 08:28:08.038984801 +0000 UTC m=+0.127265469 container create b4625544c915b20f881335ac68a8c563cbe56197ac7b7f4dbd210a729ab52b2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T08:28:08.407 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:08 smithi022 podman[110466]: 2024-09-19 08:28:08.407704792 +0000 UTC m=+0.495985409 container init b4625544c915b20f881335ac68a8c563cbe56197ac7b7f4dbd210a729ab52b2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T08:28:08.757 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:08 smithi022 podman[110466]: 2024-09-19 08:28:08.411222599 +0000 UTC m=+0.499503204 container start b4625544c915b20f881335ac68a8c563cbe56197ac7b7f4dbd210a729ab52b2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:28:08.757 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:08 smithi022 podman[110466]: 2024-09-19 08:28:08.482823457 +0000 UTC m=+0.571104063 container attach b4625544c915b20f881335ac68a8c563cbe56197ac7b7f4dbd210a729ab52b2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:28:08.757 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 19 08:28:08 smithi022 podman[110466]: 2024-09-19 08:28:08.603355462 +0000 UTC m=+0.691636074 container died b4625544c915b20f881335ac68a8c563cbe56197ac7b7f4dbd210a729ab52b2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T08:28:08.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:08 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:08.522+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:08.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:08 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:08.522+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:08.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:08 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:08.522+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:08.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:08 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:08.522+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:08.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:08 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:08.522+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:08.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:08 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:08.522+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:08.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:08 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:08.522+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.929086+0000 front 2024-09-19T08:27:41.929104+0000 (oldest deadline 2024-09-19T08:28:05.429092+0000) 2024-09-19T08:28:09.202 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T08:28:09.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:08.959+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:09.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:08.959+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:09.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:08.959+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:09.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:08.959+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:09.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:08.959+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:09.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:08.959+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:09.290 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:28:09.290 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T08:28:09.291 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T08:28:09.291 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.10 2024-09-19T08:28:09.587 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:09 smithi064 systemd[1]: Stopping Ceph osd.10 for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:28:09.587 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:09 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:09.475+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:09.587 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:09 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:09.475+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:09.587 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:09 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:09.475+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:09.587 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:09 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:09.475+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:09.588 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:09 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:09.475+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:09.588 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:09 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:09.475+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:09.588 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:09 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:09.475+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.929086+0000 front 2024-09-19T08:27:41.929104+0000 (oldest deadline 2024-09-19T08:28:05.429092+0000) 2024-09-19T08:28:10.089 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:09 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:09.598+0000 7ff5381c2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:28:10.089 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:09 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:09.598+0000 7ff5381c2640 -1 osd.10 99 *** Got signal Terminated *** 2024-09-19T08:28:10.089 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:09 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:09.598+0000 7ff5381c2640 -1 osd.10 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:28:10.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:09 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:09.923+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:10.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:09 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:09.923+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:10.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:09 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:09.923+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:10.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:09 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:09.923+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:10.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:09 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:09.923+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:10.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:09 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:09.923+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:10.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:10.427+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:10.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:10.427+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:10.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:10.427+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:10.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:10.427+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:10.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:10.427+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:10.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:10.427+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:10.838 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:10 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:10.427+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.929086+0000 front 2024-09-19T08:27:41.929104+0000 (oldest deadline 2024-09-19T08:28:05.429092+0000) 2024-09-19T08:28:11.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:10 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:10.907+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:11.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:10 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:10.907+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:11.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:10 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:10.907+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:11.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:10 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:10.907+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:11.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:10 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:10.907+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:11.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:10 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:10.907+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:11.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:11 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:11.420+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:11.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:11 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:11.420+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:11.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:11 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:11.420+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:11.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:11 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:11.420+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:11.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:11 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:11.420+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:11.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:11 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:11.420+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:11.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:11 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:11.420+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.929086+0000 front 2024-09-19T08:27:41.929104+0000 (oldest deadline 2024-09-19T08:28:05.429092+0000) 2024-09-19T08:28:12.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:11 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:11.875+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:12.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:11 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:11.876+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:12.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:11 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:11.876+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:12.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:11 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:11.876+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:12.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:11 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:11.876+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:12.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:11 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:11.876+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:12.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:12 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:12.442+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:12.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:12 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:12.442+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:12.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:12 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:12.442+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:12.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:12 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:12.442+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:12.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:12 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:12.442+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:12.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:12 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:12.442+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:12.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:12 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:12.442+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.929086+0000 front 2024-09-19T08:27:41.929104+0000 (oldest deadline 2024-09-19T08:28:05.429092+0000) 2024-09-19T08:28:13.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:12 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:12.851+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:13.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:12 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:12.852+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:13.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:12 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:12.852+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:13.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:12 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:12.852+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:13.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:12 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:12.852+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:13.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:12 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:12.852+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:13.836 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:13 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:13.450+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:13.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:13 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:13.450+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:13.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:13 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:13.450+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:13.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:13 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:13.450+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:13.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:13 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:13.450+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:13.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:13 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:13.450+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:13.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:13 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:13.450+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.929086+0000 front 2024-09-19T08:27:41.929104+0000 (oldest deadline 2024-09-19T08:28:05.429092+0000) 2024-09-19T08:28:14.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:13 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:13.837+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:14.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:13 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:13.837+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:14.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:13 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:13.837+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:14.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:13 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:13.837+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:14.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:13 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:13.837+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:14.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:13 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:13.837+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:14.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:14 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:14.473+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.421358+0000 front 2024-09-19T08:26:52.421336+0000 (oldest deadline 2024-09-19T08:27:16.521144+0000) 2024-09-19T08:28:14.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:14 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:14.473+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:58.821862+0000 front 2024-09-19T08:26:58.821940+0000 (oldest deadline 2024-09-19T08:27:22.322036+0000) 2024-09-19T08:28:14.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:14 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:14.473+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:06.923798+0000 front 2024-09-19T08:27:06.923810+0000 (oldest deadline 2024-09-19T08:27:31.524446+0000) 2024-09-19T08:28:14.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:14 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:14.473+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-09-19T08:27:17.425719+0000 front 2024-09-19T08:27:17.425478+0000 (oldest deadline 2024-09-19T08:27:42.725325+0000) 2024-09-19T08:28:14.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:14 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:14.473+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:22.725675+0000 front 2024-09-19T08:27:22.725615+0000 (oldest deadline 2024-09-19T08:27:47.926388+0000) 2024-09-19T08:28:14.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:14 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:14.473+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:32.627490+0000 front 2024-09-19T08:27:32.627550+0000 (oldest deadline 2024-09-19T08:27:54.927572+0000) 2024-09-19T08:28:14.837 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:14 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10[58867]: 2024-09-19T08:28:14.473+0000 7ff5307d0640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.929086+0000 front 2024-09-19T08:27:41.929104+0000 (oldest deadline 2024-09-19T08:28:05.429092+0000) 2024-09-19T08:28:15.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:14 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:14.855+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:15.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:14 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:14.855+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:15.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:14 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:14.855+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:15.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:14 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:14.855+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:15.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:14 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:14.855+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:15.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:14 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:14.855+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:15.388 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:15 smithi064 podman[71247]: 2024-09-19 08:28:15.125767657 +0000 UTC m=+5.633062914 container died 7bd01999cc7cf22e5d96d9cb6d6212fa3d6dc2ea6a55980db43ab994f73df7f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T08:28:15.388 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:15 smithi064 podman[71247]: 2024-09-19 08:28:15.388823693 +0000 UTC m=+5.896118946 container cleanup 7bd01999cc7cf22e5d96d9cb6d6212fa3d6dc2ea6a55980db43ab994f73df7f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T08:28:15.656 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:15 smithi064 bash[71247]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10 2024-09-19T08:28:16.006 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:15 smithi064 podman[71262]: 2024-09-19 08:28:15.656813571 +0000 UTC m=+0.528691393 container remove 7bd01999cc7cf22e5d96d9cb6d6212fa3d6dc2ea6a55980db43ab994f73df7f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T08:28:16.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:15 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:15.808+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:16.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:15 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:15.808+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:16.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:15 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:15.808+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:16.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:15 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:15.808+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:16.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:15 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:15.808+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:16.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:15 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:15.808+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:16.292 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:16 smithi064 podman[71350]: 2024-09-19 08:28:16.007373333 +0000 UTC m=+0.110568352 container create 65044de1a8cdc57be071037245a921620a68357d796ccec26bb8488772ae29bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T08:28:16.292 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:16 smithi064 podman[71350]: 2024-09-19 08:28:15.913083352 +0000 UTC m=+0.016278387 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:28:16.293 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:16 smithi064 podman[71350]: 2024-09-19 08:28:16.209682426 +0000 UTC m=+0.312877446 container init 65044de1a8cdc57be071037245a921620a68357d796ccec26bb8488772ae29bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-19T08:28:16.293 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:16 smithi064 podman[71350]: 2024-09-19 08:28:16.21312489 +0000 UTC m=+0.316319916 container start 65044de1a8cdc57be071037245a921620a68357d796ccec26bb8488772ae29bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:28:16.293 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:16 smithi064 podman[71350]: 2024-09-19 08:28:16.292758485 +0000 UTC m=+0.395953515 container attach 65044de1a8cdc57be071037245a921620a68357d796ccec26bb8488772ae29bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T08:28:16.587 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:16 smithi064 podman[71350]: 2024-09-19 08:28:16.404738091 +0000 UTC m=+0.507933120 container died 65044de1a8cdc57be071037245a921620a68357d796ccec26bb8488772ae29bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:28:16.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T08:28:16.999 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:16 smithi064 podman[71373]: 2024-09-19 08:28:16.946049315 +0000 UTC m=+0.535619646 container remove 65044de1a8cdc57be071037245a921620a68357d796ccec26bb8488772ae29bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:28:16.999 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:16 smithi064 systemd[1]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.10.service: Deactivated successfully. 2024-09-19T08:28:16.999 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:16 smithi064 systemd[1]: Stopped Ceph osd.10 for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:28:16.999 INFO:journalctl@ceph.osd.10.smithi064.stdout:Sep 19 08:28:16 smithi064 systemd[1]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.10.service: Consumed 1min 38.551s CPU time. 2024-09-19T08:28:17.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:28:17.015 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T08:28:17.015 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T08:28:17.015 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.11 2024-09-19T08:28:17.067 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:16 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:16.800+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:17.067 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:16 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:16.800+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:17.067 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:16 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:16.800+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:17.067 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:16 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:16.800+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:17.067 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:16 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:16.800+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:17.067 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:16 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:16.800+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:17.067 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:16 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:16.800+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-09-19T08:27:47.278183+0000 front 2024-09-19T08:27:47.278217+0000 (oldest deadline 2024-09-19T08:28:16.678483+0000) 2024-09-19T08:28:17.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:17 smithi157 systemd[1]: Stopping Ceph osd.11 for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:28:17.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:17 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:17.306+0000 7fcfc8c2c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:28:17.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:17 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:17.306+0000 7fcfc8c2c640 -1 osd.11 99 *** Got signal Terminated *** 2024-09-19T08:28:17.503 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:17 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:17.306+0000 7fcfc8c2c640 -1 osd.11 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:28:18.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:17 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:17.784+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:18.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:17 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:17.784+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:18.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:17 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:17.784+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:18.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:17 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:17.784+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:18.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:17 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:17.784+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:18.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:17 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:17.784+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:18.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:17 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:17.784+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-09-19T08:27:47.278183+0000 front 2024-09-19T08:27:47.278217+0000 (oldest deadline 2024-09-19T08:28:16.678483+0000) 2024-09-19T08:28:19.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:18 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:18.817+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:19.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:18 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:18.817+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:19.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:18 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:18.817+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:19.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:18 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:18.817+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:19.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:18 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:18.817+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:19.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:18 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:18.817+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:19.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:18 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:18.817+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-09-19T08:27:47.278183+0000 front 2024-09-19T08:27:47.278217+0000 (oldest deadline 2024-09-19T08:28:16.678483+0000) 2024-09-19T08:28:20.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:19 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:19.768+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:20.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:19 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:19.768+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:20.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:19 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:19.768+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:20.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:19 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:19.768+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:20.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:19 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:19.768+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:20.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:19 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:19.768+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:20.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:19 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:19.768+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-09-19T08:27:47.278183+0000 front 2024-09-19T08:27:47.278217+0000 (oldest deadline 2024-09-19T08:28:16.678483+0000) 2024-09-19T08:28:21.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:20 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:20.813+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:21.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:20 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:20.813+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:21.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:20 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:20.813+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:21.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:20 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:20.813+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:21.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:20 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:20.813+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:21.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:20 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:20.813+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:21.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:20 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:20.813+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-09-19T08:27:47.278183+0000 front 2024-09-19T08:27:47.278217+0000 (oldest deadline 2024-09-19T08:28:16.678483+0000) 2024-09-19T08:28:22.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:21 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:21.823+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-09-19T08:26:52.270799+0000 front 2024-09-19T08:26:52.270891+0000 (oldest deadline 2024-09-19T08:27:16.970812+0000) 2024-09-19T08:28:22.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:21 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:21.823+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-09-19T08:26:56.971206+0000 front 2024-09-19T08:26:56.971258+0000 (oldest deadline 2024-09-19T08:27:22.271355+0000) 2024-09-19T08:28:22.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:21 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:21.823+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-09-19T08:27:09.272921+0000 front 2024-09-19T08:27:09.273059+0000 (oldest deadline 2024-09-19T08:27:34.573042+0000) 2024-09-19T08:28:22.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:21 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:21.823+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-09-19T08:27:25.575704+0000 front 2024-09-19T08:27:25.575772+0000 (oldest deadline 2024-09-19T08:27:51.475670+0000) 2024-09-19T08:28:22.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:21 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:21.823+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-09-19T08:27:31.476145+0000 front 2024-09-19T08:27:31.476178+0000 (oldest deadline 2024-09-19T08:27:57.376251+0000) 2024-09-19T08:28:22.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:21 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:21.823+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-09-19T08:27:41.977652+0000 front 2024-09-19T08:27:41.977599+0000 (oldest deadline 2024-09-19T08:28:07.277604+0000) 2024-09-19T08:28:22.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:21 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:21.823+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-09-19T08:27:47.278183+0000 front 2024-09-19T08:27:47.278217+0000 (oldest deadline 2024-09-19T08:28:16.678483+0000) 2024-09-19T08:28:22.254 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:21 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11[60509]: 2024-09-19T08:28:21.823+0000 7fcfc5242640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.157:6820 osd.8 since back 2024-09-19T08:27:56.679004+0000 front 2024-09-19T08:27:56.678987+0000 (oldest deadline 2024-09-19T08:28:21.379057+0000) 2024-09-19T08:28:23.003 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:22 smithi157 podman[72119]: 2024-09-19 08:28:22.717851088 +0000 UTC m=+5.503836428 container died 706a931c51e73011798705fb7668a979d580607004b024b6106d29d5b064d690 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T08:28:23.323 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:23 smithi157 podman[72119]: 2024-09-19 08:28:23.123075898 +0000 UTC m=+5.909061247 container cleanup 706a931c51e73011798705fb7668a979d580607004b024b6106d29d5b064d690 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:28:23.324 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:23 smithi157 bash[72119]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11 2024-09-19T08:28:23.633 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:23 smithi157 podman[72135]: 2024-09-19 08:28:23.324140469 +0000 UTC m=+0.603967659 container remove 706a931c51e73011798705fb7668a979d580607004b024b6106d29d5b064d690 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T08:28:23.935 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:23 smithi157 podman[72225]: 2024-09-19 08:28:23.633635204 +0000 UTC m=+0.107531168 container create be68f4bf5d86c1d9180e37cc26c0b191aaf78a3fba71424db6015107c0b5ee00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:28:23.935 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:23 smithi157 podman[72225]: 2024-09-19 08:28:23.54205067 +0000 UTC m=+0.015946640 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:28:23.935 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:23 smithi157 podman[72225]: 2024-09-19 08:28:23.861083793 +0000 UTC m=+0.334979753 container init be68f4bf5d86c1d9180e37cc26c0b191aaf78a3fba71424db6015107c0b5ee00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11-deactivate, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T08:28:23.935 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:23 smithi157 podman[72225]: 2024-09-19 08:28:23.864478025 +0000 UTC m=+0.338373988 container start be68f4bf5d86c1d9180e37cc26c0b191aaf78a3fba71424db6015107c0b5ee00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T08:28:23.935 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:23 smithi157 podman[72225]: 2024-09-19 08:28:23.935839769 +0000 UTC m=+0.409735743 container attach be68f4bf5d86c1d9180e37cc26c0b191aaf78a3fba71424db6015107c0b5ee00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:28:24.253 INFO:journalctl@ceph.osd.11.smithi157.stdout:Sep 19 08:28:24 smithi157 podman[72225]: 2024-09-19 08:28:24.051200733 +0000 UTC m=+0.525096694 container died be68f4bf5d86c1d9180e37cc26c0b191aaf78a3fba71424db6015107c0b5ee00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T08:28:24.538 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T08:28:24.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:28:24.576 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T08:28:24.576 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T08:28:24.576 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.a 2024-09-19T08:28:24.929 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:28:24 smithi022 systemd[1]: Stopping Ceph mds.a for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:28:25.256 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:28:24 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-a[71234]: 2024-09-19T08:28:24.928+0000 7fcd821aa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:28:25.256 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:28:24 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-a[71234]: 2024-09-19T08:28:24.928+0000 7fcd821aa640 -1 mds.a *** got signal Terminated *** 2024-09-19T08:28:28.346 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:28:28 smithi022 podman[110570]: 2024-09-19 08:28:28.039970126 +0000 UTC m=+3.259046292 container stop df7590e175c906ac6c31adbc9d3793b9a20f79eb739553950a36b7869e0d7271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:28:28.346 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:28:28 smithi022 podman[110570]: 2024-09-19 08:28:28.065948683 +0000 UTC m=+3.285024848 container died df7590e175c906ac6c31adbc9d3793b9a20f79eb739553950a36b7869e0d7271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-a, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T08:28:28.679 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:28:28 smithi022 podman[110570]: 2024-09-19 08:28:28.503951708 +0000 UTC m=+3.723027916 container cleanup df7590e175c906ac6c31adbc9d3793b9a20f79eb739553950a36b7869e0d7271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-a, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T08:28:28.679 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:28:28 smithi022 bash[110570]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-a 2024-09-19T08:28:28.939 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 19 08:28:28 smithi022 podman[110594]: 2024-09-19 08:28:28.680147129 +0000 UTC m=+0.629944376 container remove df7590e175c906ac6c31adbc9d3793b9a20f79eb739553950a36b7869e0d7271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T08:28:28.943 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T08:28:28.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:28:28.981 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T08:28:28.981 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T08:28:28.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.d 2024-09-19T08:28:29.257 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:28:29 smithi022 systemd[1]: Stopping Ceph mds.d for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:28:29.756 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:28:29 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-d[70057]: 2024-09-19T08:28:29.307+0000 7f9f5db5e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:28:29.757 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:28:29 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-d[70057]: 2024-09-19T08:28:29.307+0000 7f9f5db5e640 -1 mds.d *** got signal Terminated *** 2024-09-19T08:28:33.165 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:28:32 smithi022 podman[110704]: 2024-09-19 08:28:32.905699089 +0000 UTC m=+3.698588881 container died 9aa1f6ec73dc7e57b932f893b6badcad897e254d74ea99a59eb0495755be1c86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:28:33.507 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:28:33 smithi022 podman[110704]: 2024-09-19 08:28:33.257577875 +0000 UTC m=+4.050467660 container cleanup 9aa1f6ec73dc7e57b932f893b6badcad897e254d74ea99a59eb0495755be1c86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T08:28:33.507 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 19 08:28:33 smithi022 bash[110704]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-d 2024-09-19T08:28:33.699 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T08:28:33.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:28:33.736 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T08:28:33.736 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T08:28:33.736 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.g 2024-09-19T08:28:34.052 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:28:33 smithi022 systemd[1]: Stopping Ceph mds.g for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:28:34.506 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:28:34 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-g[70838]: 2024-09-19T08:28:34.052+0000 7f6d4522c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:28:34.507 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:28:34 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-g[70838]: 2024-09-19T08:28:34.052+0000 7f6d4522c640 -1 mds.g *** got signal Terminated *** 2024-09-19T08:28:36.844 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:28:36 smithi022 podman[110823]: 2024-09-19 08:28:36.545630933 +0000 UTC m=+2.587380620 container died 5a88e318c867672fffd68803441a182964e50470cba4dc2fe873485c7e0065a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-g, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T08:28:37.256 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:28:37 smithi022 podman[110823]: 2024-09-19 08:28:37.020057292 +0000 UTC m=+3.061806979 container cleanup 5a88e318c867672fffd68803441a182964e50470cba4dc2fe873485c7e0065a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-g, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T08:28:37.257 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 19 08:28:37 smithi022 bash[110823]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-g 2024-09-19T08:28:37.506 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T08:28:37.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:28:37.541 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T08:28:37.541 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T08:28:37.541 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.j 2024-09-19T08:28:37.874 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:28:37 smithi022 systemd[1]: Stopping Ceph mds.j for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:28:38.256 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:28:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-j[69651]: 2024-09-19T08:28:37.874+0000 7f5ed6be8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:28:38.256 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:28:37 smithi022 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-j[69651]: 2024-09-19T08:28:37.874+0000 7f5ed6be8640 -1 mds.j *** got signal Terminated *** 2024-09-19T08:28:41.199 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:28:40 smithi022 podman[110943]: 2024-09-19 08:28:40.911775903 +0000 UTC m=+3.138340349 container died d180067f1c3f4250d1ec3d0f14e74a419c5400bd4be3cd5d3b32a9f3ed5c656f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T08:28:41.506 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:28:41 smithi022 podman[110943]: 2024-09-19 08:28:41.399075808 +0000 UTC m=+3.625640252 container cleanup d180067f1c3f4250d1ec3d0f14e74a419c5400bd4be3cd5d3b32a9f3ed5c656f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T08:28:41.507 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 19 08:28:41 smithi022 bash[110943]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-j 2024-09-19T08:28:41.783 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T08:28:41.821 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:28:41.821 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T08:28:41.822 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T08:28:41.822 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.b 2024-09-19T08:28:42.113 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:28:41 smithi064 systemd[1]: Stopping Ceph mds.b for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:28:42.379 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:28:42 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-b[64527]: 2024-09-19T08:28:42.112+0000 7faf3e5d3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:28:42.379 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:28:42 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-b[64527]: 2024-09-19T08:28:42.112+0000 7faf3e5d3640 -1 mds.b *** got signal Terminated *** 2024-09-19T08:28:43.705 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:28:43 smithi064 podman[71452]: 2024-09-19 08:28:43.41334221 +0000 UTC m=+1.397412275 container died 702394fa37745be1348fd73692c211f02c5effa17dd0471cd98288e2132c06fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-b, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-09-19T08:28:44.030 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:28:43 smithi064 podman[71452]: 2024-09-19 08:28:43.804868961 +0000 UTC m=+1.788939013 container cleanup 702394fa37745be1348fd73692c211f02c5effa17dd0471cd98288e2132c06fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T08:28:44.030 INFO:journalctl@ceph.mds.b.smithi064.stdout:Sep 19 08:28:43 smithi064 bash[71452]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-b 2024-09-19T08:28:44.177 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T08:28:44.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:28:44.213 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T08:28:44.213 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T08:28:44.214 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.e 2024-09-19T08:28:44.586 INFO:journalctl@ceph.mds.e.smithi064.stdout:Sep 19 08:28:44 smithi064 systemd[1]: Stopping Ceph mds.e for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:28:44.587 INFO:journalctl@ceph.mds.e.smithi064.stdout:Sep 19 08:28:44 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-e[63911]: 2024-09-19T08:28:44.524+0000 7f4b097c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:28:44.587 INFO:journalctl@ceph.mds.e.smithi064.stdout:Sep 19 08:28:44 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-e[63911]: 2024-09-19T08:28:44.524+0000 7f4b097c1640 -1 mds.e *** got signal Terminated *** 2024-09-19T08:28:47.726 INFO:journalctl@ceph.mds.e.smithi064.stdout:Sep 19 08:28:47 smithi064 podman[71582]: 2024-09-19 08:28:47.423765356 +0000 UTC m=+2.977949474 container died 7906b0b24f3806dc3f4b2c059a98daa34c33e79e1c3f24f8382ff967ca9f626c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-e, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T08:28:48.042 INFO:journalctl@ceph.mds.e.smithi064.stdout:Sep 19 08:28:47 smithi064 podman[71582]: 2024-09-19 08:28:47.833627928 +0000 UTC m=+3.387812051 container cleanup 7906b0b24f3806dc3f4b2c059a98daa34c33e79e1c3f24f8382ff967ca9f626c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T08:28:48.042 INFO:journalctl@ceph.mds.e.smithi064.stdout:Sep 19 08:28:47 smithi064 bash[71582]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-e 2024-09-19T08:28:48.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T08:28:48.230 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:28:48.230 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T08:28:48.230 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T08:28:48.230 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.h 2024-09-19T08:28:48.586 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:28:48 smithi064 systemd[1]: Stopping Ceph mds.h for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:28:48.587 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:28:48 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-h[64214]: 2024-09-19T08:28:48.537+0000 7f36deac4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:28:48.587 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:28:48 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-h[64214]: 2024-09-19T08:28:48.537+0000 7f36deac4640 -1 mds.h *** got signal Terminated *** 2024-09-19T08:28:51.686 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:28:51 smithi064 podman[71702]: 2024-09-19 08:28:51.419261471 +0000 UTC m=+2.974407647 container died af25c49e9d7496012ac5681bb0d01f1290d3309c823e2829cf7760803cfc5917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T08:28:52.046 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:28:51 smithi064 podman[71702]: 2024-09-19 08:28:51.686938018 +0000 UTC m=+3.242084194 container cleanup af25c49e9d7496012ac5681bb0d01f1290d3309c823e2829cf7760803cfc5917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-h, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:28:52.046 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:28:51 smithi064 bash[71702]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-h 2024-09-19T08:28:52.046 INFO:journalctl@ceph.mds.h.smithi064.stdout:Sep 19 08:28:51 smithi064 podman[71716]: 2024-09-19 08:28:51.92975824 +0000 UTC m=+0.506652774 container remove af25c49e9d7496012ac5681bb0d01f1290d3309c823e2829cf7760803cfc5917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-h, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:28:52.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T08:28:52.093 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:28:52.093 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T08:28:52.093 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T08:28:52.094 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.k 2024-09-19T08:28:52.336 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:28:52 smithi064 systemd[1]: Stopping Ceph mds.k for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:28:52.836 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:28:52 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-k[63597]: 2024-09-19T08:28:52.448+0000 7f4cb449f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:28:52.836 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:28:52 smithi064 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-k[63597]: 2024-09-19T08:28:52.449+0000 7f4cb449f640 -1 mds.k *** got signal Terminated *** 2024-09-19T08:28:53.836 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:28:53 smithi064 podman[71822]: 2024-09-19 08:28:53.633542784 +0000 UTC m=+1.319811282 container died 32cd86275c8fa951bf9bfaad6f9952683376c44ee7e64c1887d92a247d40c2a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T08:28:54.309 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:28:53 smithi064 podman[71822]: 2024-09-19 08:28:53.941092421 +0000 UTC m=+1.627360918 container cleanup 32cd86275c8fa951bf9bfaad6f9952683376c44ee7e64c1887d92a247d40c2a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-k, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T08:28:54.310 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:28:53 smithi064 bash[71822]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-k 2024-09-19T08:28:54.310 INFO:journalctl@ceph.mds.k.smithi064.stdout:Sep 19 08:28:54 smithi064 podman[71833]: 2024-09-19 08:28:54.108693349 +0000 UTC m=+0.468126756 container remove 32cd86275c8fa951bf9bfaad6f9952683376c44ee7e64c1887d92a247d40c2a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-k, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T08:28:54.322 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T08:28:54.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:28:54.353 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T08:28:54.353 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T08:28:54.353 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.c 2024-09-19T08:28:54.647 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:28:54 smithi157 systemd[1]: Stopping Ceph mds.c for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:28:54.910 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:28:54 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-c[65370]: 2024-09-19T08:28:54.646+0000 7fb72e580640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:28:54.910 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:28:54 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-c[65370]: 2024-09-19T08:28:54.646+0000 7fb72e580640 -1 mds.c *** got signal Terminated *** 2024-09-19T08:28:59.003 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:28:58 smithi157 podman[72339]: 2024-09-19 08:28:58.654136292 +0000 UTC m=+4.109183123 container died 876264be95c99539ae64db5df7424ca3ae45d5f3efe6c94f207a088c6a6148db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:28:59.306 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:28:59 smithi157 podman[72339]: 2024-09-19 08:28:59.096982868 +0000 UTC m=+4.552029696 container cleanup 876264be95c99539ae64db5df7424ca3ae45d5f3efe6c94f207a088c6a6148db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T08:28:59.306 INFO:journalctl@ceph.mds.c.smithi157.stdout:Sep 19 08:28:59 smithi157 bash[72339]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-c 2024-09-19T08:28:59.531 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T08:28:59.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:28:59.567 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T08:28:59.567 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T08:28:59.567 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.f 2024-09-19T08:28:59.892 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:28:59 smithi157 systemd[1]: Stopping Ceph mds.f for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:29:00.253 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:28:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-f[65067]: 2024-09-19T08:28:59.891+0000 7f7e396dd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:29:00.253 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:28:59 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-f[65067]: 2024-09-19T08:28:59.891+0000 7f7e396dd640 -1 mds.f *** got signal Terminated *** 2024-09-19T08:29:03.729 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:29:03 smithi157 podman[72472]: 2024-09-19 08:29:03.454663512 +0000 UTC m=+3.655188685 container died b84a29081160548ce7faebf21acda2df756cfaf50fe3245341e6dd93ddacd371 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T08:29:04.003 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:29:03 smithi157 podman[72472]: 2024-09-19 08:29:03.800303275 +0000 UTC m=+4.000828444 container cleanup b84a29081160548ce7faebf21acda2df756cfaf50fe3245341e6dd93ddacd371 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-f, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-09-19T08:29:04.003 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:29:03 smithi157 bash[72472]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-f 2024-09-19T08:29:04.003 INFO:journalctl@ceph.mds.f.smithi157.stdout:Sep 19 08:29:03 smithi157 podman[72484]: 2024-09-19 08:29:03.976475225 +0000 UTC m=+0.514901079 container remove b84a29081160548ce7faebf21acda2df756cfaf50fe3245341e6dd93ddacd371 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T08:29:04.210 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T08:29:04.244 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:29:04.244 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T08:29:04.245 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T08:29:04.245 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.i 2024-09-19T08:29:04.503 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:29:04 smithi157 systemd[1]: Stopping Ceph mds.i for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:29:05.003 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:29:04 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i[65670]: 2024-09-19T08:29:04.563+0000 7f09794d8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:29:05.003 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:29:04 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i[65670]: 2024-09-19T08:29:04.563+0000 7f09794d8640 -1 mds.i *** got signal Terminated *** 2024-09-19T08:29:07.714 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:29:07 smithi157 podman[72594]: 2024-09-19 08:29:07.455722608 +0000 UTC m=+2.993700197 container died b42f9d647701cf597db6eae403ecee54bc895ab1181855fc6237a00c2c232242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T08:29:07.964 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:29:07 smithi157 podman[72594]: 2024-09-19 08:29:07.797856617 +0000 UTC m=+3.335834206 container cleanup b42f9d647701cf597db6eae403ecee54bc895ab1181855fc6237a00c2c232242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:29:07.965 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:29:07 smithi157 bash[72594]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i 2024-09-19T08:29:08.204 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T08:29:08.226 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:29:07 smithi157 podman[72608]: 2024-09-19 08:29:07.965491894 +0000 UTC m=+0.504167919 container remove b42f9d647701cf597db6eae403ecee54bc895ab1181855fc6237a00c2c232242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T08:29:08.226 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:29:08 smithi157 systemd[1]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-19T08:29:08.226 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:29:08 smithi157 systemd[1]: Stopped Ceph mds.i for c0ad60c6-765d-11ef-bceb-c7b262605968. 2024-09-19T08:29:08.227 INFO:journalctl@ceph.mds.i.smithi157.stdout:Sep 19 08:29:08 smithi157 systemd[1]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.i.service: Consumed 35.801s CPU time. 2024-09-19T08:29:08.237 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:29:08.237 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T08:29:08.238 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T08:29:08.238 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.l 2024-09-19T08:29:08.503 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:29:08 smithi157 systemd[1]: Stopping Ceph mds.l for c0ad60c6-765d-11ef-bceb-c7b262605968... 2024-09-19T08:29:09.003 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:29:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-l[64763]: 2024-09-19T08:29:08.584+0000 7f891fab1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:29:09.003 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:29:08 smithi157 ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-l[64763]: 2024-09-19T08:29:08.584+0000 7f891fab1640 -1 mds.l *** got signal Terminated *** 2024-09-19T08:29:11.085 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:29:10 smithi157 podman[72715]: 2024-09-19 08:29:10.828234295 +0000 UTC m=+2.370259114 container died d09a90274cdad782e9eace4cf0dd852674d73624655044a5241db379df48eacb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:29:11.460 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:29:11 smithi157 podman[72715]: 2024-09-19 08:29:11.18485784 +0000 UTC m=+2.726882660 container cleanup d09a90274cdad782e9eace4cf0dd852674d73624655044a5241db379df48eacb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-l, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T08:29:11.460 INFO:journalctl@ceph.mds.l.smithi157.stdout:Sep 19 08:29:11 smithi157 bash[72715]: ceph-c0ad60c6-765d-11ef-bceb-c7b262605968-mds-l 2024-09-19T08:29:11.603 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0ad60c6-765d-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T08:29:11.635 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:29:11.635 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T08:29:11.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T08:29:11.849 INFO:teuthology.orchestra.run.smithi022.stdout:Deleting cluster with fsid: c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:29:14.443 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T08:29:14.651 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:29:17.157 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T08:29:17.358 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:29:19.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:29:19.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:29:20.008 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:29:20.035 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T08:29:20.036 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911842/remote/smithi022/crash 2024-09-19T08:29:20.037 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/crash -- . 2024-09-19T08:29:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:tar: /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T08:29:20.067 INFO:teuthology.orchestra.run.smithi022.stderr:tar: Error is not recoverable: exiting now 2024-09-19T08:29:20.069 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911842/remote/smithi064/crash 2024-09-19T08:29:20.070 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/crash -- . 2024-09-19T08:29:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T08:29:20.107 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2024-09-19T08:29:20.109 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911842/remote/smithi157/crash 2024-09-19T08:29:20.109 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/crash -- . 2024-09-19T08:29:20.150 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /var/lib/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T08:29:20.150 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now 2024-09-19T08:29:20.151 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T08:29:20.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T08:29:20.191 INFO:tasks.cephadm:Compressing logs... 2024-09-19T08:29:20.192 DEBUG:teuthology.orchestra.run.smithi022:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T08:29:20.235 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T08:29:20.237 DEBUG:teuthology.orchestra.run.smithi157:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T08:29:20.261 INFO:teuthology.orchestra.run.smithi022.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T08:29:20.261 INFO:teuthology.orchestra.run.smithi022.stderr:: No such file or directory 2024-09-19T08:29:20.261 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T08:29:20.262 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T08:29:20.262 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T08:29:20.264 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.9.log: /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T08:29:20.265 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T08:29:20.265 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T08:29:20.265 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T08:29:20.265 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mon.a.log: 84.5% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T08:29:20.265 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T08:29:20.265 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.log 2024-09-19T08:29:20.266 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.6.log: /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.g.log: 90.3% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T08:29:20.267 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T08:29:20.267 INFO:teuthology.orchestra.run.smithi064.stderr:: No such file or directory 2024-09-19T08:29:20.267 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T08:29:20.267 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T08:29:20.267 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.10.log: /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T08:29:20.268 INFO:teuthology.orchestra.run.smithi022.stderr: 94.6% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T08:29:20.271 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T08:29:20.271 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T08:29:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T08:29:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T08:29:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T08:29:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T08:29:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T08:29:20.272 INFO:teuthology.orchestra.run.smithi064.stderr: 94.5% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T08:29:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.log 2024-09-19T08:29:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mgr.y.log: 81.7% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T08:29:20.274 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T08:29:20.275 INFO:teuthology.orchestra.run.smithi022.stderr: 94.6% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T08:29:20.275 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T08:29:20.275 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T08:29:20.275 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T08:29:20.275 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mgr.x.log: 86.7% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T08:29:20.278 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.log: 86.7% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T08:29:20.278 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T08:29:20.278 INFO:teuthology.orchestra.run.smithi022.stderr: 94.6% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T08:29:20.278 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T08:29:20.278 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T08:29:20.278 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T08:29:20.279 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.h.log: 91.0% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T08:29:20.280 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-19T08:29:20.280 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T08:29:20.280 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T08:29:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T08:29:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T08:29:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T08:29:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T08:29:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T08:29:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T08:29:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.log 2024-09-19T08:29:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T08:29:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T08:29:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T08:29:20.285 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T08:29:20.285 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T08:29:20.285 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T08:29:20.286 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T08:29:20.286 INFO:teuthology.orchestra.run.smithi064.stderr: 91.9% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T08:29:20.286 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T08:29:20.286 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T08:29:20.287 INFO:teuthology.orchestra.run.smithi022.stderr: 94.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T08:29:20.287 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 94.7% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T08:29:20.297 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T08:29:20.340 INFO:teuthology.orchestra.run.smithi157.stderr: 81.7% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T08:29:20.383 INFO:teuthology.orchestra.run.smithi157.stderr: 92.0% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T08:29:20.408 INFO:teuthology.orchestra.run.smithi157.stderr: 90.5% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T08:29:20.557 INFO:teuthology.orchestra.run.smithi157.stderr: 90.8% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T08:29:20.562 INFO:teuthology.orchestra.run.smithi022.stderr: 90.2% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T08:29:20.679 INFO:teuthology.orchestra.run.smithi157.stderr: 86.7% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T08:29:20.756 INFO:teuthology.orchestra.run.smithi064.stderr: 91.7% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T08:29:20.792 INFO:teuthology.orchestra.run.smithi157.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T08:29:21.022 INFO:teuthology.orchestra.run.smithi064.stderr: 94.9% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T08:29:21.160 INFO:teuthology.orchestra.run.smithi022.stderr: 90.4% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T08:29:25.792 INFO:teuthology.orchestra.run.smithi157.stderr: 91.6% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T08:29:28.265 INFO:teuthology.orchestra.run.smithi157.stderr: 94.9% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T08:29:30.397 INFO:teuthology.orchestra.run.smithi157.stderr: 94.6% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T08:29:32.498 INFO:teuthology.orchestra.run.smithi022.stderr: 91.6% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T08:29:34.516 INFO:teuthology.orchestra.run.smithi064.stderr: 94.9% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T08:29:36.754 INFO:teuthology.orchestra.run.smithi064.stderr: 91.7% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T08:29:36.991 INFO:teuthology.orchestra.run.smithi022.stderr: 91.6% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T08:29:37.581 INFO:teuthology.orchestra.run.smithi022.stderr: 91.7% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T08:29:40.207 INFO:teuthology.orchestra.run.smithi064.stderr: 91.6% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T08:29:40.541 INFO:teuthology.orchestra.run.smithi022.stderr: 91.8% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T08:29:40.543 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-19T08:29:40.543 INFO:teuthology.orchestra.run.smithi022.stderr:real 0m20.294s 2024-09-19T08:29:40.543 INFO:teuthology.orchestra.run.smithi022.stderr:user 1m5.706s 2024-09-19T08:29:40.543 INFO:teuthology.orchestra.run.smithi022.stderr:sys 0m1.982s 2024-09-19T08:29:41.066 INFO:teuthology.orchestra.run.smithi064.stderr: 91.6% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T08:29:41.932 INFO:teuthology.orchestra.run.smithi064.stderr: 94.8% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T08:29:42.039 INFO:teuthology.orchestra.run.smithi064.stderr: 91.6% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T08:29:42.041 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-09-19T08:29:42.042 INFO:teuthology.orchestra.run.smithi064.stderr:real 0m21.789s 2024-09-19T08:29:42.042 INFO:teuthology.orchestra.run.smithi064.stderr:user 1m52.920s 2024-09-19T08:29:42.042 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m3.030s 2024-09-19T08:30:09.528 INFO:teuthology.orchestra.run.smithi157.stderr: 91.7% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T08:30:16.626 INFO:teuthology.orchestra.run.smithi157.stderr: 91.8% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T08:31:06.174 INFO:teuthology.orchestra.run.smithi157.stderr: 91.7% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T08:31:08.045 INFO:teuthology.orchestra.run.smithi157.stderr: 91.4% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T08:35:49.608 INFO:teuthology.orchestra.run.smithi157.stderr: 93.5% -- replaced with /var/log/ceph/c0ad60c6-765d-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T08:35:49.609 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-09-19T08:35:49.609 INFO:teuthology.orchestra.run.smithi157.stderr:real 6m29.357s 2024-09-19T08:35:49.610 INFO:teuthology.orchestra.run.smithi157.stderr:user 5m43.788s 2024-09-19T08:35:49.610 INFO:teuthology.orchestra.run.smithi157.stderr:sys 0m13.469s 2024-09-19T08:35:49.610 INFO:tasks.cephadm:Archiving logs... 2024-09-19T08:35:49.610 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911842/remote/smithi022/log 2024-09-19T08:35:49.612 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T08:36:00.730 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911842/remote/smithi064/log 2024-09-19T08:36:00.731 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T08:36:15.171 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911842/remote/smithi157/log 2024-09-19T08:36:15.172 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T08:37:11.615 INFO:tasks.cephadm:Removing cluster... 2024-09-19T08:37:11.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 --force 2024-09-19T08:37:11.829 INFO:teuthology.orchestra.run.smithi022.stdout:Deleting cluster with fsid: c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:37:12.201 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 --force 2024-09-19T08:37:12.411 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:37:12.804 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0ad60c6-765d-11ef-bceb-c7b262605968 --force 2024-09-19T08:37:13.321 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: c0ad60c6-765d-11ef-bceb-c7b262605968 2024-09-19T08:37:13.891 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T08:37:13.892 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T08:37:13.912 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T08:37:13.931 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T08:37:13.949 INFO:tasks.cephadm:Teardown complete 2024-09-19T08:37:13.949 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T08:37:13.961 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T08:37:13.961 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T08:37:13.964 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T08:37:13.974 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T08:37:14.102 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T08:37:14.113 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T08:37:14.113 DEBUG:teuthology.orchestra.run.smithi022:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:37:14.116 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:37:14.118 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:37:14.131 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:37:14.133 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:37:14.133 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:37:14.136 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:37:14.136 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-09-19T08:37:14.136 INFO:teuthology.orchestra.run.smithi022.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 +1346us[+1355us] +/- 51ms 2024-09-19T08:37:14.136 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 -3685us[-3675us] +/- 73ms 2024-09-19T08:37:14.136 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +1738us[+1747us] +/- 75ms 2024-09-19T08:37:14.136 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:37:14.139 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:37:14.139 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-09-19T08:37:14.139 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +1358us[+1334us] +/- 51ms 2024-09-19T08:37:14.140 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -3663us[-3688us] +/- 73ms 2024-09-19T08:37:14.140 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +1757us[+1733us] +/- 75ms 2024-09-19T08:37:14.140 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:37:14.164 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:37:14.164 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-09-19T08:37:14.164 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +1359us[+1355us] +/- 51ms 2024-09-19T08:37:14.164 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 -3677us[-3677us] +/- 73ms 2024-09-19T08:37:14.164 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 +1762us[+1758us] +/- 75ms 2024-09-19T08:37:14.164 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:37:14.165 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T08:37:14.176 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T08:37:14.176 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T08:37:14.189 DEBUG:teuthology.orchestra.run.smithi022:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T08:37:14.279 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T08:37:14.355 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T08:37:14.467 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T08:37:14.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:37:14.500 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T08:37:14.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:37:14.532 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T08:37:14.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:37:14.560 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T08:37:14.571 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T08:37:14.581 INFO:teuthology.task.internal:Duration was 2342.030665 seconds 2024-09-19T08:37:14.582 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T08:37:14.596 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T08:37:14.597 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T08:37:14.599 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T08:37:14.602 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T08:37:14.642 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T08:37:14.643 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T08:37:14.667 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T08:37:15.126 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T08:37:15.126 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi022.front.sepia.ceph.com 2024-09-19T08:37:15.127 DEBUG:teuthology.orchestra.run.smithi022:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T08:37:15.151 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-09-19T08:37:15.151 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T08:37:15.197 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-09-19T08:37:15.197 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T08:37:15.221 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T08:37:15.221 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T08:37:15.224 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T08:37:15.240 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T08:37:15.295 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T08:37:15.295 DEBUG:teuthology.orchestra.run.smithi022:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T08:37:15.297 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T08:37:15.310 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T08:37:16.407 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T08:37:16.419 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T08:37:16.419 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T08:37:16.445 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T08:37:16.474 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T08:37:16.677 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T08:37:16.688 DEBUG:teuthology.orchestra.run.smithi022:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:37:16.690 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:37:16.693 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:37:16.712 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = core 2024-09-19T08:37:16.718 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-09-19T08:37:16.738 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-09-19T08:37:16.760 DEBUG:teuthology.orchestra.run.smithi022:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:37:16.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:37:16.783 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:37:16.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:37:16.801 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:37:16.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:37:16.818 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T08:37:16.830 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T08:37:16.831 DEBUG:teuthology.misc:Transferring archived files from smithi022:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911842/remote/smithi022 2024-09-19T08:37:16.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T08:37:16.898 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911842/remote/smithi064 2024-09-19T08:37:16.898 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T08:37:16.937 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911842/remote/smithi157 2024-09-19T08:37:16.937 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T08:37:16.977 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T08:37:16.978 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T08:37:16.980 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T08:37:16.983 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T08:37:17.029 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T08:37:17.043 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T08:37:17.043 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T08:37:17.056 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T08:37:17.056 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T08:37:17.059 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T08:37:17.061 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T08:37:17.073 INFO:teuthology.orchestra.run.smithi022.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 08:37 /home/ubuntu/cephtest 2024-09-19T08:37:17.076 INFO:teuthology.orchestra.run.smithi064.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 08:37 /home/ubuntu/cephtest 2024-09-19T08:37:17.085 INFO:teuthology.orchestra.run.smithi157.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 08:37 /home/ubuntu/cephtest 2024-09-19T08:37:17.086 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T08:37:17.098 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T08:37:17.124 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 2342.0306651592255 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T08:37:17.124 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T08:37:17.216 INFO:teuthology.run:pass